Sometimes, astronomy starts with philosophy. Beyond the pretty pictures and complex math, the motivating questions that drive  astronomers are things like, “Are we alone?”, “Why are we here?”, or “Will the universe last forever?”

When Saul Perlmutter (now at University of California, Berkeley) was a graduate student in the 1980s, it was the fate of the universe that was keeping him up at night.

Since the early 20th century, astronomers had known the universe was expanding — but would it expand forever? Or would gravity eventually pull it all back together in a Big Crunch? There simply weren’t enough data to decide.

The answer came down to the universe’s density, which would be reflected in the “shape” of space itself. In a closed universe, density is high, and the mutual gravity of galaxies will eventually recollapse the cosmos, pulling everything back together again. In such a universe, parallel lines ultimately converge. In an open universe, on the other hand, there is not enough mass to put expansion in reverse, so the space between galaxies just keeps growing. In this universe, parallel lines diverge.

A third option lies in between: In a flat universe, in which parallel lines stay parallel, there’s just enough mass to slow and eventually halt its expansion — but only after infinite time.

Yet, even though theory suggested the universe was indeed flat, astronomers didn’t see enough matter to make it so. Without enough matter, and hence gravity, expansion would continue forever. Astronomers thus thought that determining how much matter there really was could help match observations with theory and thus predict our universe’s future.

To address these big questions, scientists turned to exploding white dwarfs, which create Type Ia supernovae. The researchers could calculate how intrinsically luminous such an event becomes, compare that against the observed brightness, and deduce its distance. Pairing the distance with a measure of how rapidly the supernova’s host galaxy is receding from us (that is, its redshift) reveals the expansion rate at that point. By gathering enough such measurements, astronomers could see how the expansion rate has changed over cosmological history, shedding light on the matter density and the fate of our universe.

 

Tricky Observations

Planning observations of supernovae is difficult. The explosions are rare and their locations unpredictable. To evade these problems, two groups of scientists — the Supernova Cosmology Project (SCP), cofounded by Perlmutter, and the High-Z Supernova Search Team, led by Brian Schmidt (now at the Australian National University) and Nicholas Suntzeff (now at Texas A&M) —masterfully coordinated telescope time on several telescopes around the world. The two teams imaged pieces of sky, collectively measuring tens of thousands of galaxies. A few weeks later, they’d snap another picture of the same sky patches. Comparing the before and after, the astronomers looked for points of light that weren’t there before.

The observers would then follow up on newly discovered light sources with other ground-based telescopes, and then finally with Hubble. Feverishly, the two groups worked to find as many viable supernovae as possible, in a heated race with each other.

One of the astronomers on the High-Z team was Adam Riess (now at Johns Hopkins University), who was helping analyze the data coming in. If the universe’s expansion were slowing down due to the mutual gravitational attraction of many galaxies, then the distant supernovae should be relatively bright. But that’s not what Riess found. Instead, distant supernovae were much dimmer than expected — even more so than anticipated for an open universe.

“The answer I got from my computer was [that the universe had] negative mass,” he says. “Now, that doesn’t make sense.” Of course the universe has mass, he thought. There had to be some sort of bug in the code.

Or maybe he had stumbled onto something bigger.

Riess sent an email to Schmidt. It contained a plot of supernova data and a simple subject line: “What do you think of this?”

“I could only think of what he might have done wrong,” Schmidt recalled in a 2006 essay. As far as he knew, Perlmutter’s SCP team was finding that the universe was decelerating. “It is one thing to get a different answer than the competition, it is quite another to get a different answer, and have your answer be crazy.”

Meanwhile, unbeknownst to Schmidt, the SCP team was actually finding the same odd result.

Perlmutter recalls, “We had the main job of a scientist . . . 97% of your time is trying to figure out how you’re going wrong.” They checked if the way supernovae evolve changed over cosmological time. They checked if there was some weird “gray dust” that scatters all wavelengths of light, dimming distant supernovae without being detectable itself. They even checked if the gravity of intervening galaxies had bent the light of some supernovae in such a way as to make it fainter. Or perhaps there really was just a bug in the code.

But if the data held up, then there seemed to be a choice between two possibilities: Either the universe contained a negative amount of mass (which was obviously untrue), or something was working against gravity, pushing the cosmos apart at an ever-faster clip.

The idea of a mysterious energy working against gravity wasn’t new. Albert Einstein had suggested that a cosmological constant (denoted lambda, Λ) could hold the universe motionless and keep it from collapsing under its own gravity. However, once it was discovered that distant galaxies were moving away from one another, Einstein discarded the idea. James Peebles (Princeton) invoked the constant again in the 1980s to up the energy density of the universe and allow space to be flat, even though the matter density seemed to be low. The teams’ data suggested Peebles was right.

Yet, invoking a mysterious and unknown energy is a big deal. Both groups wanted to be sure. In his book The Extravagant Universe, High-Z team member Robert Kirshner (Harvard University) recalls an email he wrote: “I am worried . . . you might need some lambda. In your heart, you know this is wrong, though your head tells you [that] you don’t care and you’re just reporting the observations.”

 

 

Debates, Arguments, and Results

Today, a quarter century later, there is still no clear consensus about what happened next. But events unfolded something like this.

In late 1997, Perlmutter and colleagues presented their work to physicists at various departments. Wanting to be cautious, the researchers stressed that their results were preliminary. But at the end of one of Perlmutter’s talks, physicist Joel Primack (University of California, Santa Cruz) stood up. Barely able to contain himself, he explained to everyone in the room that these results were amazing, because they implied that there was a cosmological constant.

The next month was the January 1998 meeting of the American Astronomical Society, one of the biggest gatherings in the astronomy community. Both supernova groups presented there, telling a similar story: Our universe has low matter density and will continue to expand forever. Ariel Goobar (Stockholm University, Sweden), a member of the SCP team, stated in a press release that “astrophysicists may have to invoke Einstein’s cosmological constant” if the results from the supernova data held up.

At the same press conference, another team, composed of Ruth Daly and Eddie Guerra (both then at Princeton), looked at galaxies with extensive jets shooting from their cores. By comparing the jets’ observed lengths with that predicted by their evolution, the researchers calculated each galaxy’s distance. In their January 1998 press release, they indicated that their data showed that the universe would not only expand forever but that its expansion was accelerating.

Other corroborating evidence came out around the same time, such as work from Neta Bahcall and Xiaohui Fan (also both then at Princeton) on the evolution of galaxy clusters, which also indicated a low matter density in the universe.

Both supernova teams met again in February at a conference. In front of a hushed audience, Alexei Filippenko (University of California, Berkeley) of the High-Z Supernova Search Team proclaimed in plainer language that they had the evidence: There was “antigravity” in the cosmos — the thing we today call dark energy.

Commotion followed. The media jumped in. By the time of a workshop in May 1998, a straw poll indicated two-thirds of the scientists thought the supernova evidence made a strong case for dark energy. The discovery was named Science magazine’s Breakthrough of the Year, and members of both supernova teams were awarded the 2011 Nobel Prize in Physics.

 

 

Changing Cosmic Understanding

In retrospect, it may seem surprising that the scientific community was so eager to accept the existence of a force dubbed dark energy when even now, 26 years later, we still don’t know what it is. However, the agreement between two highly competitive and extremely thorough research groups helped the idea gain acceptance. “All the i’s have been dotted, all the t’s have been crossed,” says Riess.

Also, not long after the announcements, an entirely independent method used observations of the Big Bang’s afterglow, known as the cosmic microwave background, to confirm both the the universe’s low matter density and the existence of dark energy. Additional data from extensive galaxy and supernova surveys, as well as studies of how galaxy clusters evolve over time, have all confirmed that dark energy makes up more than two-thirds of the universe. Energy, not matter, dominates our cosmos and its fate.

The geometry of our universe is currently flat, but that doesn’t mean it will expand forever. Dark energy is a big unknown, including how it will evolve with time. The universe may continue to accelerate or, if dark energy proves changeable, it may yet recollapse.

Today, dark energy continues to be one of the greatest discoveries — and mysteries — in astrophysics. The irony isn’t lost on Perlmutter. “The odd thing about this result was that all the questions I thought we’d get to answer, we didn’t get to answer,” he says. “So we still don’t know whether the universe is going to last forever, and we still don’t know whether it’s infinite or not.”

These questions await the next generation of scientists.

 

ELIZABETH FERNANDEZ is a science writer with a Ph.D. in
astronomy. She writes about science and society and has read
S&T since she was young.