1    The Great Predictor

It was many years ago that I first encountered the Great Predictor.

I was thrilled to meet him. I’d been looking forward to the encounter for years. The Great Predictor was famous—world famous. He was legendary for the number of his predictions, and for their amazing accuracy. Many people had relied on those predictions, and always with profit. And he could tell you the most amazing stuff—this, that, and the other thing.

What intrigued me the most, however, was how bizarre were some of the predictions. “Tomorrow you will be in two places at once” was one. “On Wednesday an event will occur for which there is no cause” was another.

How could such things be? I was captivated by the strangeness of these prognostications. Could such weird things really come to pass? That’s why I had been so anxious to meet the Predictor. For years I had looked forward to finally getting to know him.

At long last I was getting my wish. I was twenty years old, and I was thrilled. I was sitting in a classroom, in college, on the first day of a course called Introduction to Quantum Mechanics.

Quantum mechanics is one of the glories of our age. The theory lies at the very heart of modern society. I once saw an estimate that a full third of our economy involves products based on it. Much of science involves it too, not to mention a good fraction of all the Nobel prizes awarded over the past century. Quantum mechanics is one of our most valuable forecasters, and its forecasts always turn out right. It has immeasurably altered our conception of the natural world. It is a triumph.

But it is not an unalloyed triumph. As the years have passed my initial admiration for quantum mechanics has become mixed with a certain confusion. I have never felt comfortable with the theory.

Let me introduce myself. In college and graduate school I studied physics—but when it came time to start doing research, I turned to astrophysics. That is the field in which I have worked for my entire career. But astrophysics is really just a branch of physics, so it was not so great a shift as all that.

And throughout my career I have maintained my early fascination with quantum mechanics. Somehow, I never felt that I really understood the theory. It always sat lodged in the back of my mind—enigmatic, mysterious, enticing. Over and over again, I found myself thinking that someday I really ought to go back and figure it all out, and finally put all those early juvenile confusions to rest.

For truth to tell, in college I never felt fully comfortable in most of the courses I was taking. Always I felt that yes, I was doing the homework, and yes I was getting by decently on the tests—but no, I did not fully comprehend those words, those formulas and equations and vast principles that my teachers were so confidently espousing, and that I was so dutifully memorizing. Throughout it all I kept telling myself that in the long run I would understand things. For the time being, however, memorization and practice solving problems would carry me through.

And I was right. Memorization and practice solving problems did carry me through. And more than that: in the long run, I did figure things out, and I came to some degree of understanding of all the various subjects I was studying.

All but one. All but quantum mechanics. That subject I never figured out.

Many years later, when I was in the middle of my career, I encountered a colleague who was as fascinated—and as confused—by the theory as I. As the years rolled by we kept discussing the issue, first casually, then more seriously. We formed a discussion group of like-minded colleagues. We organized a conference. And ultimately, we wrote a book on the theory’s mysteries.

From the outset we knew that one of the book’s chapters was going to be on something called Bell’s Theorem. To be honest I found myself dreading getting to work on that particular topic. While I had never felt comfortable with quantum mechanics in general, Bell’s Theorem was a topic that I felt positively unnerved by. Over and over again I had tried to master it, and over and over again I had failed. I actually recall wishing at one point that we could skip the whole damn thing.

In the long run we bit the bullet and sat down and worked out some sort of understanding of John Bell’s celebrated discovery. We wrote that chapter, and we wrote the rest of the book, and it was published.

Nobody objected to what we had written in that chapter. No colleagues ever pointed out any errors within it. So, I told myself, we must have gotten it right. We must have actually figured out Bell’s Theorem.

Skip forward many years. Time passed: my attention turned to other things. But as the years rolled by, I noticed an old, familiar sensation—the sensation, nibbling quietly at the back of my thoughts, that something was wrong, that something was still eluding me. And one day, I looked at my face in the mirror—this is literally true—and I spoke aloud. “Greenstein,” I said to my reflection, “you were just kidding yourself, weren’t you? You never really understood Bell’s Theorem at all, did you?”

It was time to confess, and I did confess: in writing that chapter I had simply repeated the strategy that had proved so successful in college. I had said the right words and written down the right formulas—but I had not understood them.

“Time to get going,” I told my reflection.

And I did. This book is the result.

John Bell’s famous theorem had been meant to answer a specific question. It will take me several chapters even to describe the question he set out to address, and to set it in proper context. Suffice it to say here that Bell’s question involves some of the deepest issues that human thought may address—issues involving the ultimate nature of being. All in all, it is an unusual situation. Physics is good at telling us how to fly to the moon, how to control magnetic fields or build a better clock. But such weighty matters as metaphysics? That’s another matter.

People usually think that metaphysics—the study of existence and the ultimate nature of reality—is a purely philosophical subject. But Bell’s Theorem showed that experiments could be performed that would tell us something about it. Thus a new field of study has come into being: not just physics, not even experimental physics, but experimental metaphysics. In the third section of this book I will describe it to you.

The experiments Bell suggested have been performed. The results are astonishing. I know of no easy way to briefly summarize the significance of those experiments, and the impact they have. But one thing I do know: they are revolutionary.

The Great Predictor can do so many different things. The range of his forecasts is astonishing. Quantum theory predicts the rate of radioactive decay. It decrees that two hydrogen atoms will combine with an oxygen atom to form a water molecule, and it tells you the structure of that molecule and the energy released when it forms. It says that copper should be an electrical conductor but rubber an insulator. It predicts the structure of atoms—it predicts the very existence of atoms. It tells us this and that and the other thing. We have been immeasurably enriched by paying attention to these predictions.

If the Great Predictor were not so useful I wouldn’t be so interested in him. And I wouldn’t be so interested were not his forecasts so invariably correct. But they are correct. Not once in history has he ever been wrong.

To appreciate how remarkable this is, compare our Great Predictor with some other, lesser predictors. We have many in our society. An investor in the stock market forecasts how the market will behave. The weather bureau forecasts the weather. News media predict elections. Are they always right? Do they succeed in predicting the future in each and every situation? Of course not! In fact they do only somewhat better than the rest of us.

But what can we say of an investor who is correct more often than we? We say that she knows some things that the rest of us do not know. We say that she knows something about the innermost secret plans of corporations, regulatory agencies, and other investors. And since the weather bureau does not do all that badly we say that it knows a bit about the vagaries of wind, intrusions of high pressure, and shifts in humidity. The media know something about the opinions of voters. We say that the investor and the weather bureau and the media have to some degree succeeded in piercing the veil of appearances, and they have perceived something of an underlying truth that is hidden to the rest of us.

And the Great Predictor: what is the reality that he perceives? What are the truths that only he can see?

We physicists have a term for those truths: we call them “hidden variables.” They are “variables” because they could have one value or another—an electron could be here or there, an atom could have this energy or that. And they are “hidden” because we do not see them: they are hidden from our gaze. “Hidden variables” is physicist-speak for what is actually going on: the real physical situation that we do not perceive, but that the Great Predictor apparently does—the reality about which he seems to know so much.

There’s that word again: “reality.” Metaphysics. I am starting to describe the background to John Bell’s wonderful discovery. And, not to put too fine a point on it, the very question of whether hidden variables exist is the whole point of this book.

You might think it is all very obvious … but if there is anything quantum theory has taught us, it is that nothing about the microworld is obvious.

It took me a long time to write this book. The reason is that I wrote it to put my thoughts in order—but those thoughts refused to settle down. They skittered around madly. I kept trying to understand the situation and failing. I would go through the proof of John Bell’s wonderful theorem, not just once but over and over again—but I would end up as mystified as before. My problem wasn’t the mathematics: it was what the mathematics meant. And when I asked myself what it meant … why, my mind would just go blank.

That was a signal. I know myself well enough to realize that if I find it hard to even think about something, it is a message that there is some enormous gap in my understanding. Somewhere, something was missing from my thoughts. But what?

By now I know the answer to that question. By now I know that all along I had been operating in two ways at once. On the one hand, I was thinking in the normal way: the automobile is right there and it is going that way at such-and-such a speed. And on the other hand, I was thinking in terms of quantum mechanics. And what I now realize was that all along I had been operating in both modes at the same time. I was moving seamlessly and smoothly from one sphere of thought to the other. And most important of all: this moving from one to the other was unconscious.

If something is unconscious it just might cause you trouble. That, I ultimately came to realize, was what had been giving me so much grief for so very long.

In the pages that follow, I will invite you, the reader, to think along with me in the first of these two ways of approaching the microworld—the normal way, the non-quantum-mechanical way. I’m not doing this to be nasty. I’m doing it because this is how our minds naturally work. Not only that: it is how scientists approach their work—every scientist: biologists and geologists and chemists and, indeed, even physicists before quantum mechanics came along. It seems to be the right way to think. It is certainly the way I myself used to think. I’ll go further: it seems to be the only way we can think.

There’s only one problem: the new science of experimental metaphysics has shown that this way of thinking is wrong.

Figure 1.1

A typical tabletop quantum experiment. These experiments are not particularly spectacular to look at—but their results can be earthshaking. (This one is an experiment by David Hall of Amherst College, on a quantum phenomenon known as a Bose–Einstein condensate.) Photo by George Greenstein.

In the years that saw the creation of quantum mechanics, the theory’s founders battled over the philosophical issues it raised. Albert Einstein, in particular, never accepted the theory—a theory he helped to create, and for which he won the Nobel Prize. For decades the arguments Einstein raised remained unresolved, until John Bell’s famous discovery put them in a new and unexpected light. A series of experiments was performed—and I want to recount these experiments—that shed an extraordinary new light on the nature of the microworld.

The equipment involved in these experiments is not so very spectacular—nothing like the orbiting Hubble Space Telescope or the mighty Large Hadron Collider. Nowadays you can do a metaphysical experiment with gadgets that would fit on a tabletop. Anyone visiting such an experiment would come away pretty much unimpressed.

But while the experiments don’t look so impressive, their results are. They are earthshaking. For in truth we have never before encountered anything like the revolution in thought that the new science of experimental metaphysics is forcing on us. It has led us into a realm unlike anything that has ever come before. We have known for decades that the world of the quantum was strange. But not until John Bell came along did we realize just how strange it is.

This book is about that strangeness. And it is about an argument that, although it began as a matter of abstract philosophy, by now has blossomed into what promises to be a multimillion-dollar industry, an industry based on quantum strangeness.

So there’s a lot to talk about. Maybe it’s time to get going.