The summer of 2017 was a smashing success for Albert Einstein. On Thursday 25 May, in the waning weeks of spring that year, the journal Physical Review Letters posted a paper on its website. It described a nineteeen-year campaign to watch two stars revolving around the gargantuan black hole at the precise center of our own Milky Way. These two stars are special because their orbits are very close to the black hole, so they whirl around it at speeds as high as a few percent of the speed of light, or 20 to 30 million kilometers per hour. These orbits are ideal to test Einstein’s theory and to search for any possible deviations from its predictions. No deviations were found by the team, headed by Andrea Ghez of the University of California in Los Angeles. Einstein passed yet another test, the first one to involve orbits around a black hole.
On Tuesday 18 July, at 8 p.m., standing in front of a forest of computer monitors in the German town of Darmstadt, a scientist at the mission operation center gave the kill signal. Five seconds later, 1.5 million kilometers from Earth, the LISA Pathfinder satellite shut down. A sigh of relief mixed with sadness was heard through the room. For sixteen months, two identical cubes of a gold and platinum alloy, 1.8 inches on a side, floated freely inside evacuated chambers in the satellite, maintaining almost exactly the same separation. The satellite had to periodically adjust slightly to account for shifts in its position caused by the bombardment of protons and radiation from the Sun. If either cube made contact with the walls of its chamber, it would be a disaster. Specially made spacecraft thrusters and delicate sensors were essential if this mission was to succeed. For sixteen months, the inside of this satellite was the quietest place in the universe. The success of the mission moved scientists one step closer to fulfilling a dream: the observation of gravitational waves with a space detector, known as LISA.
The month of August that summer was even better. On Monday 14 August the LIGO gravitational wave detectors in the USA and the Virgo detector in Italy picked up the signal from two black holes that merged 1.4 billion years ago. This wasn’t the first gravitational wave signal detected—that momentous discovery had happened almost two years earlier—but it was the first to be detected simultaneously by the LIGO instruments in Washington state, near the Hanford nuclear reservation, and in Louisiana, near Baton Rouge, and by the newly operational Virgo detector near Pisa, Italy. The triple detection enabled the scientists to do a much better job of pinpointing the location of the source on the sky.
Three days later, another burst of gravitational waves jiggled the sensitive mirrors of the LIGO and Virgo detectors. A few seconds after that, the Fermi Gamma-Ray Space Telescope, orbiting 534 kilometers above the Earth, sensed a burst of gamma rays coming from the same part of the sky. Some rapid detective work located the galaxy where both signals originated, and during the subsequent hours and days astronomers around the world observed light in all its forms, from X-rays to radio waves, arriving at Earth from that same location. This time, the source was two neutron stars about 140 million light years from Earth producing waves of gravity as the stars spiraled toward each other and merged. This was followed by a nuclear fireball of unimaginable power.
This single observation revealed wonders about the universe that not even Einstein would have imagined. If you are wearing a gold necklace or a platinum ring, then there is a good chance that those precious (and expensive) elements were produced in a nuclear cataclysm just like the one observed on that day. In fact, most of the gold and platinum in the universe is now thought to have been produced in the explosions that result when neutron stars collide.
If that wasn’t enough, the mere fact that the gravitational waves, emitted right before the neutron stars merged, and the first gamma rays, emitted right after the merger, arrived within 2 seconds of each other after traveling 140 million light years revealed that the speed of gravitational waves and the speed of light are the same to fifteen decimal places. Amazingly, this is precisely what Einstein had predicted in 1916.
The following week, on Monday 21 August, a lone amateur astro- nomer named Don Bruns settled into a folding patio chair near the top of Casper Mountain in Wyoming. With the press of a key, his laptop began instructing a TeleVue Optics NP101is telescope to take a series of photographs of the Sun during the total solar eclipse that crossed the United States that day. His goal was to replicate a famous 1919 experiment carried out by a British team of professional astronomers headed by Arthur Stanley Eddington. Eddington’s measurements showed that gravity bends light exactly as Einstein had predicted, thereby overturning Newton’s theory and making Einstein an international celebrity. Bruns wanted to see what could be done by a non-professional astronomer armed only with a modern commercial telescope, a CCD (charge-coupled device) camera, and computer-controlled instruments. After analyzing his data Bruns also verified that light is bent by the Sun just as Einstein predicted, and the accuracy of his measurements beat Eddington’s by a factor of three.
Many of these events were covered by the press, using headlines like “Einstein was right, again.” They reinforced an almost fairy-tale version of the story of general relativity that goes something like this: in 1905, Einstein, working as a lowly clerk at the Patent Office in Bern, Switzerland, created the Special Theory of Relativity. He then turned his attention to gravity, and after ten years of hard work created the General Theory of Relativity. In 1919 Eddington verified the theory by measuring the bending of starlight. Einstein became famous, his theory was triumphant, and everybody lived happily ever after.
The actual story of general relativity is more complicated. Back in the 1920s there was considerable skepticism about Eddington’s results, particularly among American astronomers. Attempts in 1917 to measure the shift in the wavelength of sunlight toward the red end of the spectrum, an effect that Einstein considered another crucial test of his theory, failed to detect the effect. This apparently hurt Einstein’s chances for the Nobel Prize until 1921, when it was finally awarded for his work on the photoelectric effect, not for general relativity.
The theory was considered to be extremely complicated, with exotic new concepts like curved spacetime that baffled most physicists and astronomers at the time, to say nothing of the general public. The headline of an article on relativity in the 9 November 1919 issue of the New York Times stated, “A book for 12 wise men / No more in all the world could comprehend it, said Einstein when his daring publishers accepted it.” Einstein himself may have used some such phrase as early as 1916 in reference to a popular book on relativity that he had written. Another story has this idea originating with Eddington. Soon after the publication of the final form of general relativity in 1916, Eddington was one of the first to appreciate its importance, and set out to master the theory and then to organize a team to measure the bending of light. At the close of the November 1919 joint meeting of the Royal Astronomical Society and the Royal Society of London at which Eddington reported the successful measurements, a colleague purportedly said, “Professor Eddington, you must be one of three people in the world who understand general relativity!” to which Eddington demurred. The colleague persisted, saying, “Don’t be modest, Eddington.” Eddington replied, “On the contrary, I am trying to think who the third person could be.”
Perhaps only a handful of people understood it, but millions were fascinated by it and wanted to read about it and about Einstein. In the popular press, the scientific revolution engendered by general relativity was placed on a par with the insights of Copernicus, Kepler and Newton. Editorial after editorial marveled at what was called one of the greatest achievements in the history of human thought, but at the same time complained about the difficulty of understanding it. Einstein himself wrote a long article for The Times of London in late 1919, attempting to explain the theory to a general audience. His picture graced the cover of the 14 December 1919 issue of the German news magazine Berliner Illustrirte Zeitung, with the caption “A new great figure in world history.”
But there was a sense among scientists that the only thing this complex theorizing was good for was to predict minute deviations from Newton’s grand theory. Experimentalists held sway in the physics of that time, and they felt that general relativity would never play a role in mainstream science.
As a result of this skepticism, the science of general relativity gradually became stagnant and sterile. By the mid 1920s Einstein had turned most of his attention to what would become a futile quest for a unified field theory that would combine gravitation and electromagnetism, and many other relativity researchers followed suit. With only a few exceptions, most work in general relativity during the next thirty-five years was devoted to abstract mathematical questions and issues of principle, and was carried out by a small band of practitioners. Science historian Jean Eisenstaedt has called this period the “low water mark” for Einstein’s theory. A classic illustration of the attitude toward Einstein’s theory at the end of this period is the advice given in 1962 to a newly minted graduate of the California Institute of Technology, about to head for Princeton to do graduate work. A famous Caltech astronomer advised him that he should absolutely not work on general relativity when he got to Princeton, because it would never have anything useful to contribute to physics or astronomy. Luckily for many of us, the student, Kip Thorne, ignored the advice.
As Kip headed east for the ivy covered walls of Princeton, Caltech astronomers were on the verge of announcing the discovery of strange objects that they called “quasistellar radio sources,” or quasars. These were very distant, very energetic sources of radio waves that defied explanation in terms of conventional physics. A few people started to wonder if general relativity might help to provide an explanation, and convened a special conference on quasars, bringing together astrophysicists and general relativists. Held in Dallas in December 1963, this historic conference would come to be known as the First Texas Symposium on Relativistic Astrophysics. Within a few years, other discoveries pointed to a definite role for general relativity in astrophysics. In 1965 came the detection of the cosmic background radiation left over from the big bang. In 1967 radio astronomers discovered the first of many pulsars, now understood to be rapidly spinning neutron stars. And 1971 saw the discovery of a compact, powerful X-ray source orbiting a normal star, the first black hole candidate. You needed general relativity if you wanted to begin to understand these phenomena.
These discoveries helped bring about a renaissance for general relativity, in which it would begin to rejoin the mainstream of physics and astronomy. This was aided by advances in technology, such as atomic clocks, lasers and superconductors, and by the development of the space program, which would provide the tools to perform new high-precision tests of Einstein’s theory, putting it on a solid experimental foundation. After all, if you needed to use a relativistic theory of gravity to understand quasars, pulsars and the cosmic background radiation, it would be very good to know if Einstein’s theory was the correct one to use. For despite Einstein’s fame, by this time there were alternative theories of gravity competing with general relativity for primacy. One of these, known as the Brans–Dicke theory, named after Princeton University’s Robert Dicke and his student Carl Brans, made a credible claim that it was just as viable as Einstein’s theory. This competition sparked a major effort to carry out new and better experimental tests to determine if Einstein was right or not.
Other factors helped set the stage for this rebirth of general relativity research. Ironically, one of these may have been the death of Einstein himself in April 1955. No other topic in physics was so closely tied to a single, towering individual. It was not uncommon in those days for the few practitioners of general relativity to make a pilgrimage to Princeton to describe their work to the great one, and, hopefully, to receive his approval. On the occasion of the centenary of general relativity in 2015, the French mathematical physicist Yvonne Choquet-Bruhat wrote a charming reminiscence of her visit to the Institute for Advanced Study in 1951 as a 27-year-old postdoctoral researcher. She visited Einstein several times during that year, explaining her mathematical work on the existence of solutions of Einstein’s equations, and listening to Einstein describe his work on unified field theory. He pronounced himself very pleased with her work, which actually turned out to be one of the major milestones of the field. His work on the unified theory ultimately went nowhere. But after Einstein passed from the scene, the field was somehow free to make its own way forward.
Another factor may have been an emerging sense of community among the small group of specialists in relativity. Beginning with a conference in July 1955 in Bern, Switzerland to commemorate the fiftieth anniversary of special relativity, regular international meetings on the subject were organized. By 1959, at the third such meeting, held in Royaumont, France, leaders of the field formed the “International Committee” on general relativity, to aid the organization of future meetings, to disseminate lists of published papers in the field and to provide information on research groups around the world. This would ultimately evolve into the International Society on General Relativity and Gravitation, with elected officers, annual dues and its own scientific journal.
The fact is that science is more than a body of knowledge. It is also a community of researchers who advance the field by sharing knowledge, collaborating and competing with one another, and even correcting each other, so that scientific facts can be established and progress can be made. Some historians of science believe that the emergence of this community of relativists in the late 1950s made it possible for them to respond nimbly and effectively to the new astronomical discoveries of the 1960s.
By the time of the 1979 centenary of the birth of Einstein, the relativistic renaissance that had begun in the 1960s was in full swing. The outpouring of books commemorating that birthday attested to the vigor and excitement of research in the field. A “toolbox” of techniques for solving Einstein’s complicated equations in a wide range of situations had been developed, and researchers were turning to computers to help solve the more complex problems. Many experimental tests of general relativity had been performed. Some exploited new technologies to do improved versions of “classic” experiments, such as measurements of the bending of light using radio waves from quasars rather than using visible light from stars. Others were new tests never envisioned by Einstein, such as the “Shapiro time delay,” an excess delay in the propagation of radar signals as they passed near the Sun on their way to track the orbits of planets or spacecraft.
Black holes were an accepted and understood part of the theory, and observational evidence that they actually exist was mounting. A model for the basic structure and evolution of the universe was in good shape, and cosmologists were beginning to explore what might have happened in the first trillionth of a second after the big bang. The nature of quasars ironically remained a mystery after twenty years of study, while a pretty good theory of pulsars was in hand.
The centenary year itself was inaugurated by a stunning announcement. At the ninth Texas Symposium on Relativistic Astrophysics, held in Munich in December 1978, Joseph H. Taylor of the University of Massachusetts described the latest result from a remarkable new testing ground for general relativity that he and his student Russell Hulse had discovered in 1974. This new arena for experimental relativity was a pulsar in orbit about a companion star, colloquially called the “binary pulsar.” Taylor reported how observations of the orbit of the pulsar since 1974 had led to the first confirmation of one of the most important predictions of Einstein’s theory: the existence of gravitational waves. Nineteen years later, Taylor and Hulse would win the Nobel Prize in Physics, the first ever bestowed for work related to general relativity. If general relativity seemed to be triumphant in 1919, it was surely triumphant now.
But there were some clouds on the horizon, and there would be more to come. None of them posed a direct threat to the supremacy of Einstein’s theory—as of 1979 it had not failed a single experimental test, and that perfect record continues to today—but they suggested the possibility that Einstein might not have had the final word on gravity.
The first cloud was theoretical. In addition to developing special and general relativity, Einstein was one of the pioneers of quantum mechanics. And although Einstein ultimately found himself at odds with some of the interpretations of quantum physics, he would have been the first to admit that it was spectacularly successful in its ability to account for measurable effects in the subatomic world. And even some of the “spooky” probabilistic effects that he frequently railed against (“God does not play dice!”) have been shown by recent experiments to be true and unavoidable. Quantum mechanics rules everything from the practical (chemistry, semiconductors, MRI, nuclear energy, the workings of your smartphone, the list is almost endless) to the exotic (quarks, the Higgs boson, quantum computers, …). The basic forces of the inner world—the strong force, which governs atomic nuclei, the electromagnetic force, which governs charged particles and light, and the weak force, which governs some forms of radioactive decay and is intimately linked to the elusive particles called neutrinos—are all understood today using quantum mechanics. This notion that “the quantum” rules everything is so pervasive in physics that it is almost an act of faith that the weakest of the forces, gravity, must also somehow be “quantized.”
It is more than just faith, however. When treated as a pure, exact theory, general relativity actually seems to sow the seeds of its own destruction. The most common example of this is the black hole, which, as we will see later in this book, contains a “singularity” at its center. A singularity is a point in space where the warpage of spacetime, the density, the pressure and the energy all become infinite. Similarly singular behavior occurred at the instant of the universal “big bang,” according to standard general relativity. But the appearance of infinities in the predictions of a theory has frequently been interpreted as a signal that the theory itself is breaking down. For example, when experiments carried out in 1911 indicated that the atom consists of a tiny positively charged nucleus surrounded by orbiting negatively charged electrons, scientists realized immediately that there was a problem. The revolving electrons should emit electromagnetic radiation, lose energy and spiral inward toward the nucleus, emitting along the way a potentially infinite amount of energy. This was theoretically untenable, and of course it also contradicted experiment. The quantum mechanical model of the atom devised by Niels Bohr came to the rescue by requiring that electrons stay on fixed orbits. They would only radiate light when they made a “transition” or a jump from one orbit to an adjacent orbit of lower energy. And every atom had a “ground state,” an orbit of lowest energy, from which no further jumps were permitted. The more fully developed quantum mechanics of Erwin Schrödinger and Werner Heisenberg refined this picture in terms of probabilities and the uncertainty principle.
In the case of the black hole or big bang singularities, the hope was that quantum mechanics might come to the rescue in a similar way, for example by somehow preventing things from falling all the way to the black hole singularity, the way the atomic ground state of an atom keeps the electron from falling all the way into the nucleus.
But therein lies the rub. After almost a hundred years we still do not have an acceptable quantum theory of gravity. This is not for a lack of trying, and there is a sizable, worldwide research effort attacking this problem from a dizzying array of directions, with arcane names like canonical quantum gravity, superstring theory, causal set theory, loop quantum gravity and the AdS/CFT correspondence, to name just a few. We won’t burden you with a list of reasons why this problem is so difficult, but instead we will skip to the bottom line: general relativity as Einstein formulated it in 1915 cannot be the quantum theory of gravity. Whatever that theory turns out to be, it will be different from general relativity. The questions are how different, and can we ever detect those differences? One viewpoint asserts that quantum gravity becomes relevant only between the big bang and about a tenth of an atto-yottosecond (or 10−43 seconds) later, or at energies a million billion times higher than the levels achieved by the Large Hadron Collider in Geneva. Inside a black hole, quantum effects would kick in only at similarly incredibly short distances from the singularity, and in any case, since the event horizon that surrounds the singularity prevents the escape of any information about what is going on, why would we care? Consequently, according to this viewpoint, we will never be able to perform an experiment capable of detecting a quantum gravity effect. This raises a scientific conundrum. If you have two theories and there is no conceivable, practical way ever to perform an experiment to test them, how do you decide which one is correct? Elegance? Simplicity? A democratic vote? Faith? And is that science?
Our viewpoint on quantum gravity is agnostic. As you will discover in this book, we will be mostly concerned with experiments that can be performed in a finite amount of time, with a finite (if sometimes rather large) budget. As a result, the book will not contain a detailed description of the different attempts to develop a theory of quantum gravity, a topic with a long and complicated history of its own. We must admit, however, that the scientific community knows so little about how to make general relativity compatible with the quantum that it is entirely possible that some future experiment might stumble on an effect in conflict with the predictions of general relativity. If so, such an experiment or observation could point the way forward toward a theory of quantum gravity. For now, we can only continue to test Einstein’s theory to higher precision and in new regimes.
Another cloud on Einstein’s horizon is called dark matter. Most physicists and astronomers are now convinced that only about 4 percent of the mass of the observable universe is made up of the normal matter that we know and love because we are made of it: protons, neutrons, electrons and other elementary particles that comprise what physicists call the Standard Model. About 23 percent is called “dark matter.” (We’ll get to the remaining 73 percent in a moment.) The evidence supporting dark matter is compelling. The velocities of stars and gas around spiral galaxies, the velocities of galaxies within clusters of galaxies, and the bending of starlight around galaxies and clusters are all too large to be explained by the mass of ordinary matter making up the visible galaxies themselves. There is additional mass around these objects that does not emit light, but that attracts gravitationally, hence the name “dark” matter. Observations of the pattern of tiny fluctuations in the intensity of the cosmic background radiation confirm the presence of dark matter. Even the formation of galaxies themselves, beginning about a million years after the big bang, cannot be properly understood unless there is dark matter, using its gravitational attraction to tug the ordinary matter into the blobs that ultimately collapsed and coalesced to form the stars and galaxies that we observe.
A leading candidate for dark matter is an elementary particle that is not part of the Standard Model; by suitably tweaking that model, particle theorists have come up with numerous plausible candidates. If so, millions of these particles should be passing through your body every second as you read this book, and thus a sensitive enough detector ought to be able to sense some of them. But despite almost forty years of experimental effort carried out by physicists around the world, no detection has been made. To some, this has become a bit of an embarrassment, and so they have turned to an alternative approach: why not modify gravity itself? Although, as you will discover in this book, general relativity has been confirmed in many different arenas, it has not been well tested on the very large distance scales of galaxies, clusters of galaxies, or the observable universe as a whole. So perhaps a suitable modification of general relativity would solve the dark matter problem. So far, while most of these “modified gravity” theories have not been very successful, a future theory could be, with its predictions confirmed by future observations, especially ones that focus on these large distance scales.
What is the remaining 73 percent? Enter the final cloud on the horizon: “dark energy.” In 1998, astronomers studying very distant supernova explosions were forced to conclude, from the pattern of their data, that the expansion of the universe is speeding up, not slowing down. Given that we have known since 1929 that the universe is expanding, your intuition is perhaps telling you that the expansion should be slowing down. After all, mass makes gravity and gravity attracts, so, just as the Earth’s gravity causes a ball thrown in the air to slow down and come back down, the universe should make its own expansion slow down (whether it continues to expand but ever more slowly or halts and starts to contract is a separate question). And indeed, Einstein’s general relativity predicts unambiguously that the expansion of the universe should slow down or decelerate, not speed up. So when this accelerated expansion was detected, it was a major shock.
But then theorists got to work coming up with ideas to account for this new phenomenon. One set of ideas was dubbed “dark energy” by University of Chicago cosmologist Michael Turner, in analogy with dark matter—the adjective is rather prophetic, because we are still pretty much in the dark about both of them. If you ascribed to this substance the right gravitationally repulsive or “anti-gravity” properties, and if you let it contribute about 73 percent of the total mass and energy in the observable universe, then it fits all the observational data beautifully. In fact the cosmological model based on this, called the “Lambda-CDM” model, has proven to be remarkably successful in explaining a diverse array of data on the universe at the largest scales. Here, the Greek capital letter lambda (Λ) refers to dark energy, while CDM refers to cold dark matter. But trying to understand what dark energy is from deeper principles of quantum mechanics and particle physics has led to a plethora of competing models, which are very hard to distinguish among through experiment or observation.
Another idea was to resurrect what Einstein called his “greatest blunder.” Already in 1916 he was thinking about applying his new theory to the universe as a whole. But to his horror, he realized that the equations demanded that the universe must either expand or contract. It could not be static. At the time, conventional wisdom held that the universe was actually static, perfectly unchanging. In fact, it was not even known that there were galaxies outside the Milky Way. To get around this, he added what he called a “cosmological term” to his original equations. This term would have the effect of introducing a repulsion that would counteract the natural tendency of the universe to contract under its own gravitational attraction, leading to a nicely balanced, static universe. The size of the term was governed by a “cosmological constant,” which came to be denoted by the Greek letter Λ (in homage to Einstein, dark energy aficionados have adopted the same symbol). So by picking the “right” value for his cosmological constant, Einstein could make everything right with the consensus view of the universe.
But along came data to mess with Einstein’s world view. First was the discovery of galaxies external to our own, along with evidence that many of them were moving away from us. And then, Edwin Hubble, an astronomer at the Mount Wilson Observatory in California, announced in 1929 that the data on the motions of galaxies implied that the universe was not static but was expanding. Einstein was now forced to drop his cosmological term, since its sole purpose was to make the universe static. With the new knowledge that the universal expansion is accelerating, it is a simple matter to bring back Einstein’s cosmological term, since it naturally provides the repulsive effect needed to counteract the gravitational deceleration. The value of the cosmological constant needed to explain the acceleration of the universe is much smaller than what Einstein needed for a static universe, and so the effect of adding this term to his equations is utterly negligible for everything but cosmology itself. You might call this the “minimal” modification to Einstein’s theory.
A third idea is to modify general relativity more drastically, but still with enough fine tuning so as not to violate the agreement with the many experiments that we will be describing in this book. This turns out to be not so easy. In developing general relativity, Einstein was driven by a desire for elegance and simplicity in the structure of the theory, and he was remarkably successful. For all the talk about how complicated his theory appeared at the time of Eddington’s announcement, from a more modern perspective, general relativity is the simplest theory of gravity you could possibly imagine. And it turns out that efforts to modify it for cosmological purposes generally lead to very ugly, complicated theories. Of course, it is not clear that the true theory of nature needs to be elegant or beautiful, since after all, these are human concepts that may have nothing to do with the physical world. In other words, the universe is a messy and dirty place, so it may be that the correct theory to describe it is equally messy and dirty.
None of these “clouds,” quantum gravity, dark matter or dark energy (and others we don’t have space to discuss), directly invalidates general relativity. Still, they leave us with the disquieting feeling that it might be necessary to take gravity “beyond Einstein”; to develop a theory that agrees with general relativity in all the realms where it has been tested precisely, but that might deviate from it, either in the realm of ultra-short scales and ultra-high energies, as in quantum gravity, or in the realm of ultra-large scales as in cosmology.
This book will focus on the many precision experiments and observations that have been carried out to test Einstein’s theory in different realms (laboratory, solar system and astrophysical). But after reading about how general relativity has passed test after test, you might be tempted to say “Einstein is still right,” so let’s be done. However, in science in general, and in physics specifically, the acceptance of any theory is always provisional, because no theory can be fully tested in all possible realms of its applicability to perfect or infinite precision. The best we can do is to extend our experimentation into wider and wider realms and to higher and higher precision, in the hope of either building further confidence in the theory, or finding a deviation that might lead us to a new, more fundamental, and more complete theory. The history of science is full of examples of both outcomes.
For general relativity, the arena for experimental tests began with the solar system, notably with the famous measurement of light bending in 1919. By the 1970s, the arena had been extended to astrophysical scales, with the discovery of the binary pulsar. But since the 1970s, the precision to which tests can be performed in both of these realms has greatly improved, and completely new arenas have opened up, to include gravitational waves and black holes, for example. The events of Einstein’s wonderful summer of 2017 illustrate all these arenas for putting general relativity to the test.