3
The Quantum Evangelist
How God plays dice
‘If only I knew more mathematics!’
Erwin Schrödinger, 1925
After his doctoral examination, von Neumann quickly secured a grant from the Rockefeller Foundation and headed to Hilbert’s Göttingen, the centre of the mathematical world. Also in Göttingen at that time there was another boy wonder, the twenty-three-year-old Werner Heisenberg, who was laying the groundwork for a successful – but bewildering – new science of the atom and its constituents, which was soon christened ‘quantum mechanics’. The theory would explain many odd experimental results from preceding decades but threatened to overturn cherished ideas scientists had held about the nature of reality for hundreds of years. Quantum mechanics opened a rift between cause and effect, banishing the Newtonian clockwork universe, where tick had reliably followed tock.
Beginning in 1900, when German physicist Max Planck reluctantly introduced the radical idea that energy might be absorbed or emitted in lumps, or ‘quanta’, discoveries that had seemed at first trivial would challenge, then revolutionize, physics just as Russell’s paradoxes were shaking the foundations of mathematics. Building upon Planck’s idea, Einstein would in 1905 theorize that light itself might be composed of a stream of particles, the first hint that quantum entities had both wave-like and particle-like properties.
While von Neumann was still a schoolchild, Danish physicist Niels Bohr was busily cobbling together a new model of the atom that awkwardly melded Newtonian physics with the ‘quanta’ of Planck and Einstein. In Bohr’s quantum atom of 1913, electrons could occupy only certain special orbits and jumped from one orbit to another by absorbing a chunk of energy exactly equal to the difference in energy between them.
Brilliant though it was, Bohr’s model was a jury-rigged affair that raised as many questions as it answered. What held electrons in their ‘special’ orbits? How did they hop from one to another in an instant? ‘The more successes the quantum theory enjoys, the more stupid it looks,’ said Einstein, who realized early on that the shotgun wedding of classical and quantum concepts could not last.1 Physicists soon wanted an amicable divorce.
In 1925, Heisenberg formulated the first rigorous approach to quantum theory, now known as ‘matrix mechanics’. His jauntily titled Quantum Theoretical Re-interpretation of Kinematic and Mechanical Relations landed like a bomb at the end of that summer.2 But as von Neumann arrived in Göttingen in 1926, Erwin Schrödinger, a professor at the University of Zurich, proposed a completely different approach to quantum mechanics based on waves. Despite bearing no resemblance to Heisenberg’s matrix mechanics, Schrödinger’s ‘wave mechanics’ worked just as well. Could two such wildly different looking theories be describing the same quantum reality? What followed was five of the most remarkable years in the history of science, during which a mechanics to describe the quantum world would be forged, much of it at Göttingen.
Having already made a name for himself by tackling some of the most vexing questions in mathematics, von Neumann now turned to this question, one of the biggest puzzles in contemporary physics. He would eventually show decisively that at the deepest level Heisenberg’s and Schrödinger’s theories were one and the same. Upon that insight, he would build the first rigorous framework for the new science, influencing generations and bringing clarity to the hunt for its meaning.
The core ideas in Heisenberg’s revolutionary paper were assembled during a two-week stay in June 1925 at Heligoland, a sparsely inhabited rock shaped like a wizard’s hat that lies some 30 miles north of the German coast. Puffed up by a severe attack of hay fever, the outdoorsman had gone to take the pollen-free North Sea air, hiking and swimming and hoping for a breakthrough that would make sense of the puzzles thrown up by Bohr’s work. Heisenberg wanted a mathematical framework that would account for the things that scientists could actually see in the laboratory: chiefly, the frequencies and relative intensities of ‘spectral lines’. Excite atoms, by vaporizing a sliver of material in a flame or passing a current through a gas, and they will emit radiation. The bright colours of neon lights and the sickly yellow hue of a sodium vapour lamp are a result of the excited atoms within producing intense light at characteristic wavelengths. By the early twentieth century, that each element produced a unique set of spectral lines was well known.
Bohr had proposed these sharp spikes in the spectrum of radiated light were caused by excited electrons tumbling back to the ground state of an atom, in the process emitting light waves with energy equal to the difference between the higher and lower orbits. Heisenberg accepted this but he rejected the physical implications of Bohr’s model – electrons had never been seen spinning in orbits around an atom’s nucleus (nor would they be). Heisenberg instead stuck to the observed facts. He showed the frequencies of atomic emission lines could be represented conveniently in an array, with rows and columns representing, respectively, the initial and final energy levels of electrons producing them. When written like this, the frequency of radiation emitted by an electron falling from energy level 4 to 2, for example, would be found at row 4, column 2 of his array. But since electron transitions between different energy levels appeared to take place more or less instantly,3 there was no way to know whether an electron had jumped to its final state directly or passed through an intermediate state on the way.4 According to the laws of probability, the chance of two transitions occurring one after the other is equal to their individual probabilities multiplied together.5 To find the overall probability of all possible transitions easily, Heisenberg arranged the individual transition probabilities in an array too, and multiplied rows and columns together.6 When he did so, he discovered a strange property of his arrays: multiplying one array, A, by another, B often gave a different answer from multiplying B by A.7 This troubled him because he knew ordinary numbers do not behave in this way. As every schoolchild learns, multiplying 3 by 7 gives the same answer as multiplying 7 by 3. Mathematicians say multiplication is commutative because the two numbers can be multiplied in either order: A×B = B×A. But this was not the case with Heisenberg’s arrays. They did not commute.
Heisenberg returned to Göttingen, where he was an assistant to the theorist Max Born. He showed what he described as his ‘crazy paper’8 to Born, who encouraged him to publish, writing to Einstein that it was ‘rather mystifying’ but also ‘true and profound’.9 Only after the paper was in print did Born remember that he had been taught about similar arrays years earlier. Named ‘matrices’ by the English mathematician James Sylvester in 1850, their properties were elucidated by Sylvester’s friend and collaborator Arthur Cayley, though Chinese mathematicians made use of them some 2,000 years earlier. Born had used them in a paper on relativity in 1909 and, crucially, as he now recalled, matrix multiplication is not commutative. Heisenberg had rediscovered a type of mathematics with ancient roots. (The basics of matrix algebra, so unfamiliar to him, are now taught to high-school children.)
Inspired by Heisenberg’s work on transition probabilities, Born intuited a formula10 connecting the position of a particle with its momentum, showing that the two did not commute either. Multiplying position by momentum or, conversely, momentum by position gives slightly different results. The difference (less than a trillionth of a trillionth of a billionth of 1 joule-second) is far too small to be noticed in everyday life but is large enough to be significant at the atomic scale. Pondering the physical meaning of noncommutativity led Heisenberg in 1927 to an extraordinary new law of nature, which stated that the position and momentum of a particle cannot both ever have exact values at the same time. And if it is impossible to know at any moment both the location and velocity of a particle exactly then one cannot, as physicists had long assumed, predict where it will be next. Heisenberg’s insight was to become known as his uncertainty principle.11
Schrödinger’s formulation of quantum mechanics, which appeared shortly after Heisenberg’s, looked as different as could be. A late bloomer by the exacting standards of maths and physics at the time, Schrödinger had secured a professorship at the University of Zürich in 1925, aged thirty-seven. In October that year, he had latched on to the work of the French duke Louis de Broglie, who had proposed that particles such as electrons had both wave-like and particle-like properties.12 The baffling ‘wave–particle’ duality that de Broglie espoused found few immediate converts – how could matter be both particle and wave? But experiments in 1927 would prove him right: a stream of electrons could be diffracted and made to interfere much like light, the principle behind the electron microscope.13 Schrödinger, however, realized what was missing from de Broglie’s work was an equation describing how matter waves snaked through space and time, similar to those derived for light (and other electromagnetic waves) by Scottish physicist James Clerk Maxwell in the nineteenth century.
Anxious to make his name with a big discovery, Schrödinger worked through a two-week tryst with an ex-lover in an Alpine resort that Christmas, returning to Zurich in January to apply his new wave equation to some of the key problems that were being thrown up by atomic physics. ‘A late erotic outburst’ was how Hermann Weyl, a close friend of Schrödinger (and his wife’s lover), would describe the deluge of academic papers that were to follow. Among them was a complete description of the hydrogen atom spectrum based on his theory and a version of his equation that showed how the waves evolved over time.
What Schrödinger’s mysterious waves really were, however, no one knew. Water waves and sound waves, for instance, are transmitted by the movement of water or air molecules. But what medium were ‘matter waves’ travelling through? Physicists rushed to embrace wave mechanics nevertheless, thankful that, unlike Heisenberg’s matrix methods, Schrödinger’s maths was reassuringly familiar and his equations often easier to solve.
In the case of the hydrogen atom, Schrödinger substituted into his equation values for the masses and charges of the electron and nucleus and a formula, from classical physics, for the electrical energy of the two particles. He then found functions that satisfied his equation – a process that could be accomplished by an undergraduate mathematician.14 These ‘wave functions’, to which Schrödinger assigned the Greek letter psi (ψ), describe how the height of the wave (amplitude) varies in space and time. For the hydrogen atom, there are an infinite number of solutions to the Schrödinger equation, each representing one of Bohr’s special orbits. The overall wave function for the atom, ψn, is an infinite sum or ‘superposition’ of them all.15
In the spring of 1925, there had been no theory that adequately described the physics of the atom. Less than twelve months later, there were two. Both theories seemed to do the job but were so different that many physicists wondered if they could both really be correct. Schrödinger confessed he was ‘repelled’16 by the instantaneous quantum jumps of Heisenberg’s theory. When electron transitions took place in Schrödinger’s theory, the wave function describing the atom changed smoothly from one form to another. Heisenberg was even blunter about the failings of Schrödinger’s wave mechanics. ‘It’s crap,’ he wrote to Pauli.17 He was particularly disturbed by the physical picture that Schrödinger was trying to paint of the atom’s inner workings. Heisenberg, who had avoided anything that could not be observed directly in his matrix mechanics, balked at the central role given to exotic, invisible ‘waves’ in Schrödinger’s work. Born later showed that there was no easy physical interpretation of the wave function at all – it was a wave of probability, an ethereal entity that was not carried by or moving through anything.
Some physicists, satisfied there were now two formulations of quantum mechanics that appeared to give the right answers, shrugged off the unresolved questions swirling around quantum theory. Just choose the theory best suited to the problem at hand, they suggested, and hang the existential consequences. Even Heisenberg turned to Schrödinger’s wave mechanics to calculate the spectrum of the helium atom. But Bohr, Einstein and other scientists were disturbed the two theories seemed to be saying very different things about the nature of reality. The more mathematically minded were equally uneasy that there was no straightforward way to reconcile the two theories. There had to be some deeper connection, they reasoned, between Heisenberg’s infinitely large arrays of numbers and Schrödinger’s strange formless waves of undulating probabilities. But what could it be?
At Göttingen, von Neumann heard about matrix mechanics first-hand. He was keen to help Hilbert extend his programme of axiomatization to physics. By sheer coincidence, both he and his hero were experts in the underlying mathematics of quantum theory.
In mathematical terms, Schrödinger was applying a mathematical ‘operator’ (the energy operator, known as the ‘Hamiltonian’) to his wave function to extract from it information about the energy of the system. Crudely speaking, operators are mathematical instructions. The solutions (i.e. the wave functions) to equations like Schrödinger’s are ‘eigenfunctions’. The answers (i.e. the energy levels of an atom) that pop out of the equation after eigenfunctions are substituted in are ‘eigenvalues’. Hilbert himself had come up with these terms in 1904, based on the German word eigen, meaning ‘characteristic’ or ‘inherent’. He had also pioneered spectral theory, which broadened the mathematics of operators and eigenvalues. A ‘spectrum’ in Hilbert’s theory was the complete set of eigenvalues (i.e. solutions) associated with a particular operator. For example, the ‘spectrum’ of the Hamiltonian is, in the case of the hydrogen atom, the complete set of all allowed energy levels.
When Hilbert realized his two-decade-old spectral theory was proving useful in the dazzling new world of the quantum atom, he was delighted. ‘I developed my theory’, he said, ‘from purely mathematical interests, and even called it “spectral analysis” without any presentiment that it would later find application to the actual spectrum of physics.’ But the grand old man of mathematics, now in his sixties, was still confused by what he heard about quantum mechanics in Heisenberg’s presentations.
Hilbert asked his assistant, Lothar Nordheim, to explain things to him but found the paper Nordheim produced to be unintelligible. When von Neumann saw it, he immediately realized that the deep mathematical structure of quantum theory could be recast into terms familiar to Hilbert. Nordheim’s paper was his first clue to the essence of quantum theory, the common thread running through wave and matrix mechanics.
When wave and matrix mechanics appeared, many physicists suspected the missing link between the two theories would be discovered only by reconciling the two kinds of infinity that lay at their hearts. An atom has an infinite number of orbits, for example, so Heisenberg’s matrices must also be of infinite size to represent all possible transitions between them. The members of such a matrix can, with sufficient patience, be lined up with a list of the counting numbers – they are ‘countably’ infinite.18 Schrödinger’s formulation, on the other hand, yielded wave functions describing in many instances an uncountably infinite number of possibilities. An electron that is not bound to an atom, for example, could according to quantum theory be literally anywhere.19 Until a measurement is made to determine its actual whereabouts, the electron is in a superposition of states, each corresponding to the electron being at some position (specified by coordinates x, y, z).20 Heisenberg’s matrices (with their countable elements) and Schrödinger’s continuous wave functions are said to occupy different types of ‘space’. ‘Every attempt to relate the two,’ von Neumann warns, ‘must run into great difficulties’ and ‘cannot be achieved without some violence to the formalism and to mathematics’.21
One person who nonetheless tried to do exactly that was the taciturn British theoretical physicist Paul Dirac, whom novelist Ian McEwan describes as ‘a man entirely claimed by science, bereft of small talk and other human skills’.22 His Cambridge colleagues even named a unit of speech after him: a ‘dirac’ amounted to a single, solitary word per hour. Dirac would later fall in love with and marry Margit Wigner, the sister of Johnny’s school friend Eugene. He would even learn to tell a joke or two. But in the 1920s, the young Dirac was a man who cared little for anything other than advanced physics and who, in the words of Freeman Dyson, ‘seemed to be able to conjure laws of nature from pure thought’.23
Dirac began to expound his version of quantum theory in 1925.24 In his 1930 book The Principles of Quantum Mechanics,25 he set out an ingenious trick to merge the ‘discrete’ space of Heisenberg’s matrices and the other ‘continuous’ space of Schrödinger’s waves. The key to Dirac’s approach was a special mathematical device that is now named after him: the Dirac delta function. This is a very peculiar entity indeed: everywhere except the origin, the function is equal to 0; but at the origin, it is infinitely high. The area under this vanishingly thin spike was defined by Dirac to be equal to one.
Dirac’s function was forbidden by the rules of mathematics. He did not care. When Hilbert chided that the delta function could lead to mathematical contradictions, Dirac airily replied, ‘Did I get into mathematical contradictions?’26 Armed with his delta function, Dirac was able to show wave and matrix mechanics might after all be two sides of the same coin. The delta function acts as a sort of salami slicer, cutting up the wave function into manageable, ultra-thin slivers in space. If one accepts the use of Dirac’s delta function, then the mathematical complications of reconciling wave and matrix mechanics appear to be magicked away. The wave function is chopped into bite-size chunks at every point in space. It seems that just as in matrix mechanics, there are then an infinite number of elements to contend with rather than a smoothly varying wave.
Like many mathematicians, von Neumann was dissatisfied with this imperfect union. He dismissed the delta function as ‘improper’, ‘impossible’ and a ‘mathematical fiction’. He wanted a less sloppy take on the new science. The vital clue to von Neumann’s rigorous reformulation of quantum mechanics lay in the early work of Hilbert.
Soon after Schrödinger unveiled his wave formulation of quantum physics, von Neumann, Dirac, Born and others realized that the mathematics of operators, eigenvalues and eigenfunctions could be useful in matrix mechanics too. Operators could be written as matrices.27 But operators have to act on something, and whereas in Schrödinger’s theory they acted on wave functions, Heisenberg had made no reference to quantum ‘states’ in his early work because they could not be observed directly (he was working only with the intensities and frequencies of spectral lines). The concept was consequently introduced to matrix mechanics, with an infinitely long column or row matrix (i.e. a single vertical or horizontal lane of numbers) representing a state in Heisenberg’s theory in much the same way that the wave function did in Schrödinger’s.
A row or column matrix can be thought of as a vector, pointing to coordinates given by the numbers in the matrix. Since a state matrix is comprised of an infinitely long series of numbers, an infinite number of axes is required to represent this vector. This sort of infinite-dimensional space is impossible for anyone to picture. Nonetheless, the maths dealing with these daunting spaces had been set out by Hilbert in the first decade of the twentieth century, and von Neumann, who now quickly made himself the world’s leading expert on the subject, named them ‘Hilbert spaces’ in honour of his mentor.28
By definition, to be a proper Hilbert space, the squares of each number comprising a vector added together has to be finite.29 Hilbert was exploring these spaces because they are mathematically interesting, and all sorts of results from school geometry (such as Pythagoras’ theorem) apply. Crucially, Hilbert spaces can be also formed by certain sets of functions as well as numbers. One class of function that had been shown to form a Hilbert space were those that are square integrable – squared and summed over all space, such functions are finite.
The quantum wave function is just such a function. Born had shown that the square of the amplitude of the wave function at any point indicated the chance that a particle will be found at that particular position. Since it is certain that a particle must be somewhere in space, the wave function squared and summed over all space must be 1. That means quantum wave functions are square-integrable and form a Hilbert space.30
Von Neumann may have lacked Dirac’s intuitive, almost mystical, physical insight but he was a far better mathematician. In 1907, mathematicians Frigyes Riesz and Ernst Fischer had, within months of each other, independently published proofs of an important result relating to square integrable functions, and von Neumann realized that their work could connect wave and matrix mechanics. Square integrable functions such as the wave function can be represented by an infinite series of orthogonal functions,31 sets of mathematically independent functions that can be added together to make any other.32 Imagine having to fill a 124-litre trough exactly to the brim with 20-, 10- and 7-litre buckets. One way to do it would be with five bucketfuls of the first, one of the second and two of the third. A wave function can be similarly ‘topped up’ by adding up bits of other functions. How much of each function is required is indicated by their coefficients.33 What Riesz and Fischer showed was that if the square of the wave function is 1, then the sum of each of these coefficients squared is 1 too.34
Armed with this theorem, von Neumann quickly spotted the link between the Heisenberg and Schrödinger theories: the coefficients of the expanded wave function were exactly the elements that appear in the state matrix. According to Riesz-Fischer, the two seemingly disparate spaces are in fact the same; and the two spaces were, as von Neumann put it, ‘the real analytical substrata of the wave and matrix theories’.35 Giants of quantum theory like Dirac and Schrödinger had tried to prove the equivalence of the two. Von Neumann was the first to crack it, showing decisively that wave and matrix mechanics were fundamentally the same theory. But never before had two descriptions of the same phenomena implied such different pictures of reality. Newton’s gravitational law described how planets wheeled through the heavens, the kinetic theory of gases assumed the motion of huge numbers of particles accounted for their properties, but what, if anything, did the mathematics of the quantum theory represent? Von Neumann had built a rock in the midst of a sea of possibilities.
Von Neumann’s stay in Göttingen was brief, though he was to visit many more times over the next few years. When his Rockefeller fellowship ran out in 1927, he was offered a job at the University of Berlin. He was the youngest Privatdocent the university had ever appointed. The position gave him no salary, only the right to lecture and receive fees directly from students. But there were distractions. The German Empire had collapsed after the end of the First World War. Berlin was now the wild, decadent capital of the Weimar Republic. A popular ditty among Berliners at the time, ran:
Du bist verrückt, mein Kind,
Du mußt nach Berlin,
Wo die Verrückten sind,
Da jehörst de hin.
You are crazy, my child,
You must go to Berlin,
That’s where the crazy are,
That’s where you belong.36
The twenty-three-year old lapped it up. The bookish Wigner was also in Berlin but apart from socializing with his fellow Hungarians (Teller, Szilard and von Neumann) and attending the lively physics colloquia, he led a rather monastic existence. Von Neumann, Wigner recollected, lived a very different sort of a life. ‘He was sort of a bon vivant, and went to cabarets and all that.’37
As well as the vibrant nightlife of Berlin, the scientific culture was second to none. German, not English, was the language of science in the 1920s. Practically all the founding papers of quantum mechanics were written in it. There was a flood of congresses and conferences for young researchers to attend. Academic talks would often spill over into coffee houses and bars. ‘The United States in those years was a bit like Russia: a large country without first-rate scientific training or research,’ Wigner told an interviewer in 1988. ‘Germany was then the greatest scientific nation on earth.’
Von Neumann’s usual approach to giving seminars was not to spoil them by over-preparing. He would often think through what he might say on the train journey to the conference, turn up at the seminar with no notes and then race through the maths. If he filled up the blackboard, he would rub out a swathe of earlier equations and plough on. Those not as quick on the uptake as he was – i.e. nearly everyone – referred to his inimitable seminar style as ‘proof by erasure’. Any tensions that arose, however, he could, and often did, defuse by telling risqué jokes in any of three different languages. When someone else’s presentation bored him, he would look engrossed while mentally retreating from the room to think about other, more interesting mathematical problems.
Von Neumann enjoyed his Berlin years immensely but he realized the chances of securing a paid professorship would be better elsewhere. He took up the offer of a job at Hamburg in 1929, hoping to be promoted quickly to a full professorship. He would not be there long.
Meanwhile he was busy reducing the whole of quantum mechanics to its mathematical essentials, just as he had set theory. First working with Nordheim and (nominally) Hilbert, then later on his own, von Neumann developed the thinking that would culminate in his 1932 masterpiece of mathematical physics, Mathematical Foundations of Quantum Mechanics, which showed how quantum theory emerged naturally by considering the mathematical properties of Hilbert space.38 Satisfied that he had produced the most rigorous formulation of quantum theory, he turned his attention to the most contentious question of the day in physics: what on earth was going on beneath all that elegant maths?
Physicists have wrestled with what quantum mechanics is really telling us about the nature of the physical world since its early days. The failure to come up with an acceptable interpretation of the theory even led students at Schrödinger’s university to make up a ditty gently ribbing their great professor:
Erwin with his psi can do
Calculations quite a few,
But one thing has not been seen:
Just what does psi really mean?39
The existence of GPS, computer chips, lasers and electron microscopes attest that quantum theory works beautifully. But nearly a hundred years after Heisenberg published his paper on matrix mechanics there is still no agreement on its meaning. In the interim, a plethora of exotic ideas have been put forward to make sense of what quantum physics is saying about reality. All have their passionate advocates, but none have yet been proven. Physicists ruefully joke that though new interpretations of quantum physics arrive with astonishing regularity, none ever go away. For many, that joke is turning sour. ‘It is a bad sign’, theoretical physicist Steven Weinberg noted recently, ‘that those physicists today who are most comfortable with quantum mechanics do not agree with one another about what it all means.’40
At the heart of the problem is the borderland between quantum and classical physics, where the interactions of atoms and photons are revealed to us via instruments like microscopes and spectrometers or our own eyes. According to quantum theory, a particle can be in a superposition of infinitely many states. In the case of a free electron, for example, the particle is anywhere and everywhere and its wave function is a superposition of states representing all these possibilities.
Now imagine that we ‘catch’ an electron on a phosphor screen. When an electron collides with the screen, the phosphor coating emits a shower of photons, and we see a flash, telling us the approximate position of the electron. The electron is now in one state with a single corresponding (eigen)value for its position. At no point does an observer ‘see’ the electron’s wave function splayed improperly across all space. The electron is either in a superposition of states that are inaccessible to an observer – or, after observation, localized at a point. As if embarrassed by its own naked quantumness, the particle appears to have donned classical clothes the moment it is observed.
The two situations, before and after an observation, are completely different, and this duality, von Neumann says, is ‘fundamental to the theory’. The particle is at first described by a wave function (comprised of all possible states). This wave function is a solution of the Schrödinger equation, perfectly describing the particle anywhere in space and at any point in time. Just as a satellite’s orbit around the Earth can be calculated from moment to moment thanks to Newton and Einstein’s equations, Schrödinger’s equation allows the evolution of the wave function to be known precisely anywhere in space and time. This behaviour is as deterministic as Newton’s laws of motion. But when we try to find out something about a particle, like its position or momentum, the wave function pops like a bubble and the particle adopts, at random, a state out of all the available possibilities. This process is discontinuous and cannot be reversed: once the particle has ‘selected’ a particular state, Schrödinger’s equation no longer holds, and the other states are lost as possibilities. Now known as ‘wave function collapse’, this process is unlike anything in classical physics.
In Mathematical Foundations of Quantum Mechanics, von Neumann credits Bohr with first identifying these two essentially incompatible processes in 1929. But Bohr’s rambling essay of that year rather obscures the issue.41 He suggests that the measuring instrument (the microscope, for example), a large ‘classical’ object, intruding into the quantum world, is somehow necessary for the irreversible change that occurs during an observation. As von Neumann notes, however, there is no clear-cut divide between the classical and quantum realms. Measuring devices are, after all, made of atoms that obey the laws of quantum physics. There is nothing in the maths to say if or when a collection of atoms gets ‘big’ enough to pop the wave function.
The question of how, when or even if, the wave function collapses is at the root of the so-called ‘measurement problem’, and the differences between the plethora of interpretations of quantum mechanics today usually hinge on their respective answers to this question, first dissected and analysed thoroughly in von Neumann’s book of 1932.
Von Neumann’s discussion of measurement begins simply: he imagines measuring the temperature of something (a cup of coffee, say) using a mercury thermometer. This requires at least one person, an observer, to see where the column of mercury has risen to on the thermometer’s scale. Between thermometer and observer, von Neumann argues, one can insert any number of processes.
The light entering the observer’s eye, for example, is a stream of photons that have been reflected by the mercury column, and refracted by the observer’s eye before hitting the retina. Next, the photons are transformed by retinal cells into electrical signals that travel up the optic nerve to the brain. Furthermore, the signals from the optic nerve might be expected to induce chemical reactions in the brain. But no matter how many such steps we add, von Neumann argues, the sequence of events must end with someone perceiving these events. ‘That is,’ he says, ‘we are obliged always to divide the world into two parts, the one being the observed system, the other the observer.’
But what about the steps in between? The most straightforward interpretation of quantum mechanics would seem to require that any number of such steps would lead to the same result – at least as far as the observer is concerned. If this were not so, then the way that someone chose to slice up the problem would give different predictions about what the observer sees. No one wants a theory that gives different answers for what is, in essence, the same problem. So von Neumann worked out if quantum mechanics did indeed give the same answer for all scenarios that start and end in the same way.
To do so, von Neumann divided the world into three parts. In the first case, part I is the system being measured (the cup of coffee); part II, the measuring device (thermometer); and part III, the light from the device and the observer. In the second case, part I is the cup and the thermometer; part II includes the path of light from the thermometer to the observer’s retina; and part III, the observer from the retina onwards. For his third example, von Neumann considered the situation where part I encompasses everything up to the observer’s eye; II, the retina, optic nerve and brain; and III, what he called the observer’s ‘abstract “ego”’. He then calculated for his three examples the consequences of putting the boundary (the point at which the wave function collapses) between part I and the rest of the experiment. Next he shifted the boundary so that wave function collapse occurred after parts I and II but before III and recalculated the outcomes from the observer’s point of view.
To do the maths, von Neumann needed to work out what happened in quantum theory when a pair of objects interact. In this situation, he found the quantum states of the coffee cup and thermometer, for instance, can no longer be described independently of each other or even as a superposition of their individual states. According to his formalism, their wave functions become so inextricably intertwined that both must be represented together by a single wave function. Schrödinger would in 1935 coin the term ‘quantum entanglement’ to describe this phenomenon. This means that measuring some property of one of the pair instantly collapses the wave function of the whole system, even if the objects are separated by some vast distance after their initial interaction. Einstein, who was probably the first to fully appreciate this consequence of entanglement and did not like it one bit, called it ‘spooky action at a distance’.42
Von Neumann was always rather more relaxed about the weirder aspects of quantum physics than Einstein. He wanted to know if the duality inherent in the new quantum physics meant the theory would contradict itself. Reassuringly, he found it did not. Wherever he put the dividing line between quantum and classical, the answer, as far as an observer was concerned, was the same. ‘This boundary,’ he concluded, ‘can be pushed arbitrarily far into the interior of the body of the actual observer.’ And that was true, said von Neumann, right up until the act of perception (whatever that was). The ‘boundary’ that he describes is now known as the ‘Heisenberg cut’. More rarely (but perhaps more fairly) it is called the Heisenberg-von Neumann cut.
Von Neumann’s results implied that in principle anything could be treated as a quantum object, whatever its size or complexity – as long as wave function collapse occurred (instantly) at some point in the chain between the system being observed and the consciousness of the person doing the observing. In this picture, it makes no sense to talk about the properties of an object (whether it be a photon, coffee cup or thermometer) until a measurement is made. None of these objects could be said to be somewhere in particular, for example, unless their wave functions had collapsed. This would become a fundamental tenet of the ‘Copenhagen interpretation’, for many years the prevailing view of what quantum mechanics means.43 According to Copenhagen, the theory does not tell us what quantum reality ‘is’, only what can be known. Many physicists were attracted to it as it allowed them to get on with the business of physics without getting bogged down in speculations about things they could not see (no coincidence that Heisenberg, who rejected unobservable phenomena in his original formulation of matrix mechanics, promoted this view). Others felt the approach side-stepped the bigger questions posed by the theory. ‘Shut up and calculate’ was how the physicist David Mermin summarized the Copenhagenists’ approach in 1989.44
Some of the most eminent founders of quantum mechanics were not altogether happy with the emerging consensus that von Neumann was helping to build. What, for instance, caused the wave function to collapse? Von Neumann did not tackle this problem head-on in his book. Others, including his friend Wigner, who often discussed such things with him, would later suggest that the consciousness of the (human) observer was responsible – a conclusion implied but not stated overtly in von Neumann’s work.45 Einstein strongly objected to this idea – the Dutch physicist and historian Abraham Pais recalled ‘that during one walk Einstein suddenly stopped, turned to me and asked whether I really believed that the moon exists only when I look at it’.46 Einstein (and he was hardly alone) felt that things should have properties regardless of whether there was someone there to see them.
Mathematical Foundations of Quantum Mechanics is the work of an exceptional mathematician. One of the work’s earliest fans would be a teenager, who ordered the book in the original German as his prize after winning a school competition.47 ‘Very interesting, and not at all difficult reading’ was how Alan Turing described von Neumann’s classic in a letter to his mother the following year.48 But von Neumann’s book was also the work of a rather cocksure young man. To some, it seemed like the twenty-eight-year-old upstart was suggesting his book was the last word on quantum mechanics.
Erwin Schrödinger disagreed. Three years after the publication of von Neumann’s book, Schrödinger discussed with Einstein the weaknesses in what would become known as the Copenhagen interpretation of quantum mechanics. Inspired by their frenzied exchange of letters, Schrödinger posed the most famous thought experiment of all time to highlight the absurdity of applying quantum mechanics willy-nilly to everyday objects.49 If the rules of quantum mechanics could, as von Neumann argued, be applied just as well to large things, then why, thought Schrödinger, should they not apply to insects? Or mice? Or cats?
‘One can even set up quite ridiculous cases,’ wrote Schrödinger in his 1935 paper.
A cat is penned up in a steel chamber, along with the following diabolical device (which must be secured against direct interference by the cat): in a Geiger counter there is a tiny bit of radioactive substance, so small, that perhaps in the course of one hour one of the atoms decays, but also, with equal probability, perhaps none; if it happens, the counter tube discharges and through a relay releases a hammer which shatters a small flask of hydrocyanic acid. If one has left this entire system to itself for an hour, one would say that the cat still lives if meanwhile no atom has decayed. The first atomic decay would have poisoned it. The psi-function of the entire system would express this by having in it the living and the dead cat (pardon the expression) mixed or smeared out in equal parts.
Schrödinger’s cat was a gotcha of the highest order, a takedown of efforts to paper over the cracks in quantum theory. A cat, most people would agree, can be either dead or alive. But if we follow von Neumann’s logic, until someone opens the chamber, the cat’s wave function is entangled (the term is used for the first time in this paper) with that of the radioactive substance, and the unfortunate feline is both alive and dead. If quantum mechanics can result in such patently obvious nonsense at the macroscopic scale, how can we know the theory ‘truly’ describes the atomic realm? Schrödinger was intimating that quantum theory was not the end of the road. ‘The theory yields much,’ Einstein famously wrote to Born, ‘but it hardly brings us closer to the Old One’s secrets. I, in any case, am convinced that He does not play dice.’50 Like Einstein, Schrödinger felt there must be another, deeper theory underlying quantum mechanics that would provide a more sensible physical picture of what was going on. The moon exists, even if there is no one to see it. An electron must have properties – be somewhere, for example – before it is caught on a phosphor-coated screen. In what has become the most controversial part of his book in recent years, von Neumann discusses this idea – and seemingly dismisses it.
Quantum mechanics, as we have seen, is strikingly different from the physical theories that preceded it. If the Copenhagen interpretation is correct, then the collapse of the wave function results in an unpredictable outcome. A particle that is observed adopts a state at random out of all those available. This means that quantum theory is neither causal (we can’t trace back exactly the events that result in a particle ending up where we observe it to be) nor deterministic (because the outcome of a particular observation is determined in part by chance). One way to restore causality and determinism to the quantum world, and realism to boot (so particles have properties even if no one measures them), is to assert the existence of ‘hidden variables’ or ‘hidden parameters’: properties that are associated with all particles but inaccessible to observers.51 In such schemes, these unobservable parameters completely determine the state of the system. The element of chance is eliminated: no dice-playing God is required.52 Von Neumann was sceptical that a theory based on hidden variables could reproduce all the predictions of quantum physics. In the Mathematical Foundations he sets about demonstrating the very great difficulties that a hidden variable theory would encounter in doing so.
Imagine making some sort of measurement on an ensemble of many quantum particles (hydrogen atoms, say). Now make the same measurement on another, identical ensemble. In keeping with quantum theory and countless experiments, the measurements give different results. If the same measurement is made on a huge number of ensembles, then you find the results are distributed across a range of values. Any collection of particles displaying this sort of statistical variation is called a dispersive ensemble and so, according to quantum physics all ensembles are dispersive.
There are two possible reasons why an ensemble might be dispersive, von Neumann says. One explanation might be that while the ensembles seem identical, the values of the hidden variables associated with the particles of each ensemble are different and, in sum, these unobservable parameters (which differ from ensemble to ensemble) account for the range of results obtained from measurements. This means that the ensembles cannot be composed of particles that all have the same hidden variable values (otherwise a measurement on an ensemble would always give the same result); in physics terms, they cannot be homogeneous. A second explanation is that contemporary quantum theory is correct, and the results of measurement are randomly distributed (so hidden variables are not necessary).
What von Neumann then proceeds to prove is that dispersive ensembles in quantum mechanics are homogeneous. All of the particles in the ensemble are in the same quantum superposition of states until there’s a measurement. As he has already shown that hidden variables would mean that ensembles cannot, in general, be homogeneous, von Neumann can rule them out.
Von Neumann’s proof electrified those who were leaning towards the Copenhagen interpretation. As word spread that the young genius had decisively rejected hidden variable theories, ‘Von Neumann was hailed by his followers and credited even by his opponents,’ says the historian Max Jammer.53 By this time, von Neumann was enjoying the comforts of a new life in the United States.
Towards the end of October 1930, Wigner had received, out of the blue, an offer of a one-term lectureship from Princeton University. If that were not enough, the telegram quoted a salary so high – more than seven times what Wigner was earning in Berlin – he thought there must have been a mistake during the message’s transmission. He quickly learned that von Neumann had received a letter from Princeton a couple of weeks earlier with the offer of even more money. ‘It was clearly Jancsi that Princeton really wanted,’ said Wigner.54 Unbeknownst to him, von Neumann’s letter asked if Wigner should be invited too. Luckily for Wigner, von Neumann agreed this would be a good idea. He added that there would, however, be a short delay before he could take up the post because he wanted to ‘fix a family matter’. Von Neumann was going to Budapest to get married.
The scheme to entice the two Hungarians had been cooked up by Oswald Veblen, a distinguished Princeton professor of mathematics. America was an intellectual backwater and Veblen wanted to change that by poaching some of Europe’s most brilliant mathematicians with the offer of huge American salaries. He had secured millions of dollars from the Rockefeller foundation and wealthy private donors to erect a grand new building, named Fine Hall, for the mathematics department. Now he just needed the mathematicians to fill it. Veblen came under pressure from the wider faculty to hire a physicist. Von Neumann and Wigner had recently co-authored papers on the puzzling spectra of atoms more complex than hydrogen. So Veblen came up with the perfect compromise: invite both Hungarians for half a year.
Wigner’s boat arrived in New York harbour in early January 1930. Von Neumann arrived with his new wife, Mariette Kövesi, about a day later.55 ‘I met him, and we spoke Hungarian,’ said Wigner. ‘We agreed that we should try to become somewhat American.’ On that day, Jancsi became Johnny, and Jeno˝ became Eugene. ‘Jancsi felt at home in America from the first day,’ Wigner continued. ‘He was a cheerful man, an optimist who loved money and believed firmly in human progress. Such men were far more common in the United States than in the Jewish circles of central Europe.’
Within a year or two of the publication of his book, von Neumann’s ‘impossibility proof’ became gospel in the world of quantum theory. For decades, any young physicist keen to advance their careers would think twice before venturing to work on an alternative to the prevailing theory. ‘Many generations of graduate students who might have been tempted to try to construct hidden-variables theories,’ said physicist David Mermin in 1993, ‘were beaten into submission by the claim that von Neumann, 1932, had proved that it could not be done.’56
But what, exactly, had von Neumann proved? The problem was that by dint of his reputation – and the fact that Foundations was not translated into English for a further two decades – few would closely scrutinize the proof itself. One person who did, two years after von Neumann’s book was published, was the German mathematician and philosopher Grete Hermann.
Hermann had studied mathematics at Göttingen, and that she got there at all was already something of an achievement: girls were not generally admitted to the gymnasium she attended, and she had needed special dispensation to start her schooling. After graduating from the university, she became the only female doctoral student of the only female professor of mathematics there, the brilliant Emmy Noether. Just a few years earlier, historians and linguists at the university had tried to block Noether’s own appointment, forcing Hilbert to intervene on her behalf. ‘I do not see that the sex of the candidate is an argument against her admission,’ he retorted. ‘We are a university, not a bath house.’ The hostility helped forge a bond between the two women, and Hermann would remember Noether fondly in her memoirs. When, after passing her doctoral examination in February 1925, Hermann announced her intention to pursue philosophy, Noether, who was in the midst of finding a job for her at the University of Freiburg, was not pleased: ‘She studies mathematics for four years, and suddenly discovers her philosophical heart!’57
Hermann was as passionately committed to socialism as she was to the philosophy of Immanuel Kant. She joined the International Socialist Militant League, a part of the German resistance movement. She eventually fled to London and by becoming a British citizen through a marriage of convenience avoided being interned by the authorities. She returned to Germany after the war to help with reconstruction and was a harsh critic of intellectuals who had chosen to live and work under the Third Reich.
Sometime in 1934, she travelled to the University of Leipzig, where Heisenberg was a professor, to defend Kant’s conception of causality from the onslaught of quantum theory. ‘Grete Hermann believed she could prove the causal law – in the form Kant had given it – was unshakable,’ Heisenberg wrote later. ‘Now the new quantum mechanics seemed to be challenging the Kantian conception, and she had accordingly decided to fight the matter out with us.’58 Heisenberg was impressed enough to devote a full chapter of his autobiography to Hermann’s arguments.
Shortly after her time in Leipzig, Hermann published her critique of von Neumann’s impossibility proof as part of a longer paper on quantum mechanics. She had identified a weakness in one of his assumptions, ‘the additivity postulate’, which she argued meant the proof was circular.59 Essentially, Hermann said, von Neumann had shown his Hilbert space perfectly explained quantum physics and had then assumed any theory must have the same mathematical structure. But, she continued, if in future a hidden variable theory was discovered that could account perfectly for everything in quantum mechanics, there was no reason at all to assume it would resemble von Neumann’s.
Hermann had in 1933 sent an earlier essay containing her discussion of the impossibility proof to Dirac, Heisenberg and others to prepare the ground.60 That did not prevent her 1935 paper sinking into obscurity.61 Hermann herself did not appear to attach much importance to it: she did not include the argument against von Neumann in the abridged version of the paper that was published by the prestigious journal Naturwissenschaften.62 Quite possibly she felt that rigorous philosophy, not more mathematics, was required to save determinism.63
Not until 1966, thirty years after Hermann had published her critique, were the limitations of the impossibility proof to become more widely known. ‘The von Neumann proof, when you actually come to grips with it, falls apart in your hands!’ John Stewart Bell was to declare many years later. ‘There is nothing to it. It’s not just flawed, it’s silly!’64 Born in Belfast into a poor family, Bell was the only one of four siblings to stay at school after the age of fourteen. He worked as a technician in the physics department at Queen’s University, where after a year he secured a small scholarship and went on to get a degree in experimental physics and one in mathematical physics a year later in 1949. Feeling guilty for being dependent on his parents for so long, Bell immediately found a job at the Atomic Energy Research Establishment at Harwell. He would only be able to pursue a PhD years later, getting his doctorate in 1956. Four years later he moved to CERN in Geneva with his wife, fellow physicist Mary Bell, where he worked on particle physics and the design of particle accelerators. ‘I am a quantum engineer, but on Sundays I have principles,’ he proudly told a gathering of PhD students during an impromptu seminar in 1983.65 The principles that Bell was interested in were, of course, those of quantum mechanics.
Ever since Bell had first studied physics, he had felt that something was ‘rotten’ in quantum physics. What bothered him most was the duality von Neumann had identified, the movable ‘cut’ between the quantum and classical worlds. He was attracted to the idea of hidden variable theories because they could in principle make this boundary disappear. The quantum lottery of wave function collapse would be unnecessary, and there would be a smooth transition from quantum to classical or, in Bell’s words, ‘a homogeneous account of the world’. Still Bell accepted what was by then the consensus: von Neumann’s proof precluded hidden variable theories. Bell could not read German and an English translation of von Neumann’s book would not be available for another three years. He accepted the second-hand accounts he read.
That changed in 1952. ‘I saw the impossible done,’ Bell said.66 In two papers the American physicist David Bohm had described a hidden variable theory that could reproduce the results of quantum mechanics in their entirety. The Copenhagen interpretation was by now orthodoxy. Bohm’s scheme was heresy, but he was an outsider with little to lose. He had been hauled up in front of the House Committee on Un-American Activities in 1949 for his communist affiliations, then arrested for refusing to answer questions. Though he was acquitted, Princeton refused to reinstate him, and he was advised to leave the United States by his former doctoral adviser, Robert Oppenheimer. Bohm took the hint, accepting the offer of a professorship at the University of São Paulo. His theory was the work of exile.
Bohm had cleverly modified Schrödinger’s equation so that the wave function is transformed into a ‘pilot wave’. The trajectories of particles are guided by this wave such that their behaviour was in keeping with the rules of quantum mechanics. Any physical changes that might affect a particle, no matter how far away they occurred, are instantly transmitted by the pilot wave, which pervades the whole universe. If all the factors affecting a particle are known, then in principle its path can be calculated exactly from beginning to end. Bohm’s theory allows determinists to have their quantum cake and eat it.
Bell could not believe it. ‘Why is the pilot wave picture ignored in text books?’ he asked in 1983. ‘Should it not be taught, not as the only way, but as an antidote to the prevailing complacency?’67 But Bell was busy with his day job. Only in 1964, during a year-long sabbatical at the Stanford Linear Accelerator Center in California, did he return to the impossibility proof, now with an English translation of von Neumann’s book. At Stanford, he independently discovered the same flaw that Hermann had highlighted so many years earlier. The resulting paper eventually appeared in 1966.68 (The cause of the delay was that a letter sent to Stanford from the journal’s editor was not forwarded to Bell, who had returned to CERN.) But Bell’s work was published by Reviews of Modern Physics, one of the most prestigious journals in the field. He was also, quite soon, to become famous. His refutation of von Neumann’s work did not suffer the same fate as that of Hermann. An interest in the meaning of quantum mechanics was soon no longer necessarily career suicide. Liberated, some physicists began to examine the foundations of quantum theory, as they had in the 1920s, when it was born. The end of the Copenhagen hegemony was nigh, and new interpretations began to spring up like weeds.
Debate still rages among quantum physicists over von Neumann’s ‘impossibility proof’. Mermin is among those who believe that von Neumann erred and Bell and Hermann correctly identified his error.69 Jeffrey Bub and, separately, Dennis Dieks have argued that von Neumann never meant to rule out all possible hidden variables – only a subset of them.70 In essence, they say that all von Neumann was aiming to prove is that no hidden variable theory can have the same mathematical structure as his own; they cannot be Hilbert space theories. And that is certainly the case: Bohm’s theory, for example, is quite different from von Neumann’s.
While Heisenberg and Pauli branded Bohm’s theory as ‘metaphysical’ or ‘ideological’, von Neumann was not dismissive, as Bohm himself notes with some pride and more than a little relief. ‘It appears that von Neumann has agreed that my interpretation is logically consistent and leads to all results of the usual interpretation. (This I am told by some people.)’ Bohm wrote to Pauli shortly before his theory was published. ‘Also, he came to a talk of mine and did not raise any objections.’71
Bohm might have hoped for Einstein to embrace his ideas, which restored both realism (particles exist at all times in Bohmian mechanics) and determinism. Einstein was, however, less kind than von Neumann. Disappointed that Bohm had not rid quantum mechanics of ‘spooky action at a distance’ (which he could not abide) he privately called Bohm’s theory ‘too cheap’.72 He had been similarly unimpressed a quarter of a century earlier, when de Broglie had presented a nascent version of the pilot wave picture. Einstein, who plumbed the quantum’s depths, never devised a satisfactory alternative of his own.
Even Bell, who championed Bohm’s approach, had reservations. ‘Terrible things happened in the Bohm theory,’ he conceded. ‘For example, the [paths of] particles were instantaneously changed when anyone moved a magnet anywhere in the universe.’ Bell wanted to explore this aspect of Bohm’s theory further. The same year that he wrote his paper criticizing von Neumann’s impossibility proof, he was also working on another, exploring whether any theory could account for the results of quantum mechanics without appearing to need, like Bohm’s, some sort of instantaneous signalling between particles separated by vast distances. This ‘nonlocality’ is built into both standard quantum mechanics (via entanglement) and Bohm’s theory (via the all-seeing pilot wave). And that looks awkward at first glance because according to Einstein’s special theory of relativity (which no experiment has ever contradicted) nothing can travel faster than light. Einstein, with two collaborators, Nathan Rosen and Boris Podolsky, had pointed this out himself with a famous thought experiment that became known as the EPR paradox.73 The upshot of their paper was that quantum theory must be incomplete because, according to the theory, a measurement of a particular property on one of a pair of entangled particles immediately determines the corresponding state of the other, no matter how far away it is. Since no signal between the two can travel faster than light, the authors reasoned, these values must somehow be fixed before measurement and not determined by the act of measurement, as quantum theory dictates.
In fact, as is now widely appreciated, quantum theory does not violate special relativity. A measurement on one of a pair of entangled particles does not directly affect the state of the other; there is only a correlation between the two and no causal link. No message can ever be sent faster than light by means of an entangled pair because to understand the message, its recipient would have to know the result of the measurement on the sender’s particle. There is no ‘spooky action at a distance’, as Einstein feared, because there is no ‘action’. But what Bell wondered was: could any ‘local’ hidden variables theory account for the correlations between the two particles that quantum theory attributed to entanglement? What if, as Bell would put it later, there was no weird quantum entanglement but just a case of Bertlmann’s socks? Reinhold Bertlmann, Bell’s friend and collaborator, always wore socks of different colours. ‘When you see that the first sock is pink you can be already sure that the second sock will not be pink,’ wrote Bell. ‘And is not the EPR business just the same?’
If that were the case and hidden variables had determined the relevant properties prior to any measurements, there would be no need for the strange ideas of the Copenhagenists. Was it possible to differentiate between the two possibilities? Bell’s genius was to realize that it was.
He pictured a simpler, more practical version of the EPR thought experiment that had been devised by Bohm. In this, two entangled particles are created and fly apart until the distance between them means they cannot communicate at slower-than-light speeds in the time it takes to perform a measurement on them. Bohm’s suggestion was to measure spin, a quantum property of subatomic particles like electrons and photons. A particle can either be ‘spin-up’ or ‘spin-down’. He proposed splitting a hydrogen molecule, with no spin, into a pair of hydrogen atoms. Since the total spin of the two atoms must still be zero, one must be spin-up and the other spin-down. If the orientations of the two spin detectors are aligned, then this is exactly the result one gets, 100 per cent of the time.
Bell’s idea was to vary the relative orientations of the two detectors so that there was an angle between the two measurements. Now if one of the two particles is measured as having spin-up, the other one is not always spin-down. According to quantum theory, however, the fates of the two particles are still linked and, as a result, the outcome of one spin-up and the other spin-down is still strongly preferred. What Bell showed mathematically is that, for certain orientations of the two detectors, the correlation between the spins of the two particles would have to be lower, on average, for a local hidden variable theory than quantum theory. Bell’s theorem takes the form of an inequality that places a limit on how high such correlations can be for any local hidden variables theory. Any correlation higher than this limit is said to ‘violate’ Bell’s inequality and would mean either quantum theory or some other non-local theory like Bohm’s must be at work. The laser technology required to probe a particle’s spin soon improved enough to allow Bell’s theorem to be experimentally tested.74 Physicists John Clauser and Stuart Freedman at the University of California at Berkeley carried out the first Bell test in 1972. Their experiment and the dozens since have all found a violation of Bell’s inequality – a result that only quantum theory or a non-local theory like Bohm’s can explain.
While Bohm’s theory struggled to win widespread acceptance (though Bell continued to champion it), another was to be ignored altogether, only to spawn countless science fiction stories and half-baked mystical philosophies (and more than a few research papers) when it re-emerged over a decade later.
The progenitor of the ‘Many Worlds’ interpretation was a young American theorist named Hugh Everett III, who began his graduate studies at Princeton University in the mathematics department. By coincidence, he spent his first year working on game theory, a field which von Neumann had helped found with his 1944 book, Theory of Games and Economic Behavior. Soon, however, he took courses in quantum mechanics and, in 1954, mathematical physics with von Neumann’s old friend Wigner. Von Neumann’s book on quantum theory didn’t come out in English until the following year, but his ideas were already well known in the United States, according to Everett. Von Neumann’s formulation is ‘the more common (at least in this country) form of quantum theory,’ he says in a letter.75 But Everett did not swallow the gospel whole.
Like Bell, who was also working towards his PhD on the other side of the Atlantic, he was dissatisfied with von Neumann’s approach to the measurement problem. The abrupt transition from quantum to classical that is implied by wave function collapse is ‘a “magic” process’, he wrote to Jammer in 1973, quite unlike other physical processes, which ‘obey perfectly natural continuous laws’. The ‘artificial dichotomy’ created by the ‘cut’, he said, is ‘a philosophic monstrosity’.
Everett first hit upon his solution ‘after a slosh or two of sherry’ with his flatmate Charles Misner and Aage Petersen, Bohr’s assistant, who was visiting Princeton at the time.76 Von Neumann’s approach had been to distil from the physics the bare mathematical principles required to explain quantum phenomena, then, using only these laws, infer whatever one could about the nature of the quantum realm. But Everett realized von Neumann had not done that in his treatment of measurement. Instead, von Neumann had noted that observers never see a quantum superposition of states, only a single classical state. Then, he had assumed that at some point, a transition from quantum to classical must take place. Nothing in the maths necessitated wave function collapse. What, Everett wondered, if we really follow the maths to its logical conclusion? What if there is no collapse at all?
Everett was led to a startling result. With no artificial boundary to constrain it, there is quantumness everywhere. All the particles in the universe are entwined in a single massive superposition of all possible states. Everett called this the ‘universal wave function’. Why, then, does an observer perceive only one outcome from a measurement and not some quantum fuzz of possibilities? That is where the ‘many worlds’ come in. Everett proposed that every time a measurement is made, the universe ‘splits’ to create a crop of alternative realities, in which each of the possibilities play out. (So Schrödinger’s cat is alive in one universe and dead in another. Or in several.) One of many objections raised to Everett’s ideas is that universes multiply like rabbits, a consequence that strikes some physicists as using an ontological sledgehammer to crack an epistemic nut.77 In some versions of the theory ‘measurement’ can mean any quantum interaction. Every time a nucleus emits an alpha particle or a photon interrogates an atom, a whole new universe springs into being.
Impressed with the mathematical lucidity of Everett’s thesis (if not entirely convinced by its substance), his PhD adviser, John Wheeler, took it to Copenhagen in the hope of winning Bohr’s approval. He failed. Everett, disappointed with the reception his theory received from the physics community, left academia to work in weapons research for the Pentagon. But the Copenhagenists’ grip on quantum physics was slowly loosening. In 1970, the American physicist Bryce DeWitt wrote an article for Physics Today, the membership magazine of the American Institute of Physics, and the theory began to make its way into the popular imagination, boosted by an article in the science fiction magazine Analog.
A welter of interpretations bloomed after Everett and Bohm highlighted some of the inadequacies of Copenhagen in the 1950s. The questions they raise are no longer of purely academic interest. Quantum mechanics now underlies a host of modern technologies, from fibre optics to microchips. The latest development is the quantum computer – still in its infancy, but potentially able to harness the power of quantum superpositions to do things conventional computers cannot handle, such as simulating the quantum processes behind chemical reactions. Most computers today work by manipulating binary digits – bits, which can each be either 1 or 0. A quantum computer instead works with a bit in a superposition of states. These quantum bits or ‘qubits’ each have a probability of potentially being 1 or 0 but are, in effect, both, until a measurement is made. A qubit, however, really comes into its own when it is entangled with others – ideally, with hundreds of others rather than the few dozen or so that have been corralled together to date. Physicists are probing the limits of quantum theory with experiments to find out whether such large assemblies of particles (which might be atoms, photons or electrons, for example) can be entangled and kept in quantum states long enough to do useful computing.
One result of the past few decades of experiments and theorizing is that most physicists now believe that there is no instantaneous wave function collapse. Instead, the wave function ‘decays’ in a small but finite amount of time into a classical state through a process called ‘decoherence’. How quickly this happens depends on how isolated the quantum system is from the environment and its size.
But there are other points of view. ‘Spontaneous collapse’, for example, posits that wave function collapse occurs on a time scale that is inversely related to the size of the object in question. The wave function of a particle such as an electron may not collapse for 100 million years or more, but a cat’s would collapse almost instantly. This solution to the measurement problem was put forward by Giancarlo Ghirardi, Alberto Rimini and Tullio Weber in 1986.78
Which of these many interpretations, if any, will turn out to be true? Von Neumann remained open-minded about the possibility of a deeper alternative to quantum theory for the rest of his life. ‘In spite of the fact that quantum mechanics agrees well with experiment,’ he says in his book, ‘one can never say of the theory that it has been proved by experience, but only that it [is] the best known summarization of experience.’ He was, however, more circumspect about the prospect of a future theory restoring causality. That events appear to be linked to each other in the familiar everyday world is irrelevant, von Neumann argued, because what we see is the average outcome of countless quantum interactions. If causality exists, then it needs to be found in the atomic realm. Unfortunately, the theory that best accounts for observations there appears to be in contradiction with it.
‘To be sure,’ von Neumann continues, ‘we are dealing with an age-old way of thinking that has been embraced by all mankind. But that way of thinking does not arise from logical necessity (else it would not have been possible to build a statistical theory), and anyone who enters the subject without preconceived notions has no reason to adhere to that way of thinking. Under such circumstances, is it reasonable to sacrifice a reasonable physical theory for the sake of an unsupported idea?’79
Dirac, on the other hand, felt that quantum theory was not the whole story. ‘I think,’ he told his audience during a lecture tour of Australia and New Zealand in 1975, ‘that it is quite likely that at some future time we may get an improved quantum mechanics in which there will be a return to determinism and which will, therefore, justify the Einstein point of view.’80
Today, we know that Dirac was almost certainly wrong, and the hopes of Einstein were misplaced. There may yet be a better theory than quantum mechanics, but thanks to Bell’s work and the experiments that followed, we know that non-locality will be part and parcel of it. Conversely, von Neumann’s cautious conservatism appears with hindsight the correct attitude. There is no proof yet for a deeper alternative to the quantum theory that von Neumann helped to forge more than a hundred years ago. All the experiments to date have revealed no hidden variables, nothing to suggest causality reasserts itself at some deeper level. As far as we know, it’s quantum all the way down.
Physicists now doff their hats to von Neumann’s Hilbert space theory, but Dirac’s approach is the one most often taught to undergraduates.81 Yet von Neumann’s formulation of quantum mechanics remains definitive. Wigner, who received a Nobel for his work in quantum mechanics, insisted that his friend Jancsi was the only person who understood the theory. While Dirac laid down many of the tools of modern quantum physics, von Neumann laid down a gauntlet. He presented the theory as coherently and lucidly as anyone could and by doing so exposed the limits of quantum mechanics to scrutiny. Without a clear view of those limits, interpreting the theory is impossible. ‘The historically most influential and hence for the history of the interpretations most important formalism’ was that of von Neumann, says Jammer.82 For physicists not content to shut up and calculate, von Neumann’s book remains required reading today, nearly a hundred years after its publication.
Von Neumann’s contributions to quantum theory did not end with his book. He helped Wigner with work that would win his friend a share of the Nobel Prize. While developing the maths of quantum theory, he became fascinated with the properties of operators in Hilbert space.83 Operators can, for example, be added, subtracted and multiplied and so are said to ‘form an algebra’. Operators connected to each other by similar algebraic relationships are dubbed ‘rings’.
For several years, von Neumann outlined the properties of these operator algebras and published what he found in seven monumental papers, a total of 500 pages in length – his most profound contribution to pure mathematics. He discovered three irreducible types of operator ring that he called ‘factors’. Type I factors exist in n-dimensional space, where n can be any whole number up to infinity. Von Neumann’s version of quantum mechanics was expressed in just this kind of infinite-dimensional Hilbert space. Type II factors are not restricted to a Hilbert space with a whole number of dimensions; they can occupy a fractional number of dimensions, ½ or π (don’t even try to visualize this). Type III factors are those that do not fit into the other two categories. The three together are now known as von Neumann algebras.
‘Exploring the ocean of rings of operators, he found new continents that he had no time to survey in detail,’ writes Dyson. ‘He intended one day to publish a grand synthesis of his work on rings of operators. The grand synthesis remains an unwritten masterpiece, like the eighth symphony of Sibelius.’84
Others have since explored a few of the archipelagos and peninsulas of von Neumann’s operator theory and returned with enormous riches. The mathematician Vaughan Jones, for example, was awarded the Fields Medal in 1990 for his work on the mathematics of knots, which emerged from his study of Type II von Neumann algebras. Jones had read Mathematical Foundations as an undergraduate. ‘His legacy is quite extraordinary,’ Jones says. A central aim of knot theory is to distinguish with certainty whether two tangles of string are genuinely different or if one can be turned into the other without cutting the string. Different forms of essentially the same knot are described by the same polynomial. Jones discovered a new polynomial that could distinguish between, for example, a square knot and a granny knot. The Jones polynomial now crops up in different areas of science. Molecular biologists, for instance, have used it to understand how cells uncoil the tightly knotted DNA inside the nucleus so that it can be read or copied.
Physicist Carlo Rovelli and mathematician Alain Connes, meanwhile, have used Type III factors in their effort to solve the ‘problem of time’: that though we feel time to flow ‘forwards’, there is no single unified explanation for why this is so (quantum theory and general relativity, for example, have radically different concepts of time).85 The pair speculate that the non-commutativity at the heart of quantum theory and embedded in Type III algebras may give time a ‘direction’ because two quantum interactions must occur in sequence, not simultaneously. This, they claim, determines an order of events that we perceive as the passing of time. If they are right, our perception of time itself is rooted in von Neumann’s maths.
The dark political clouds that had been gathering on the horizon quickly rolled in after von Neumann and Wigner left Germany in 1930. In September, the Nazi Party garnered more than 6 million votes to become the second-largest party in the Reichstag. At the next election, two years later, they received 13.7 million votes, and Hitler was appointed Chancellor of Germany in January 1933. When a fire gutted the Reichstag the following month, Hitler was awarded emergency powers. Freedom of speech, freedom of the press and the right to protest were suspended along with most other civil liberties. In March, he consolidated his power with the Enabling Act, which effectively allowed Hitler and his cabinet to bypass parliament. One of the first acts of the new regime was to introduce the ‘Law for the Restoration of the Professional Civil Service’, which called for the removal of Jewish employees and anyone with communist leanings. In Germany, university staff are officially appointed and paid directly by the government. About 5 per cent of all civil servants lost their jobs. But physics and mathematics departments were devastated: 15 per cent of physicists and 18.7 per cent of mathematicians were dismissed. Some lost more than half their faculty more or less overnight. Twenty of the ousted researchers were either already Nobel laureates or future recipients of the prize. Some 80 per cent were Jews.
Back in Princeton, Wigner faced a quandary. Princeton had extended his contract for five years along with von Neumann’s but he felt guilty about turning his back on Europe. He turned to his friend for advice. ‘Von Neumann,’ Wigner said, ‘asked me a simple question: Why should we stay in a part of the world where we are no longer welcome? I thought about that for weeks and came up with no good answer.’ Instead, Wigner focused his efforts on finding jobs for the scientists now desperate to leave Germany.
In June that same year, von Neumann wrote to Veblen: ‘If these boys continue for only two more years (which is unfortunately very probable), they will ruin German science for a generation – at least.’86 How right he was. By the end of 1933, Germany was a totalitarian dictatorship, and the trickle of scientists leaving the country became a flood. The economist Fabian Waldinger recently analysed the impact of the dismissals on German research.87 Scientific productivity dropped like a stone: researchers produced a third fewer papers than before. The ‘Aryan’ scientists recruited to replace those forced to leave were generally of a lower calibre. He found that university science departments that were bombed during the war recovered by the 1960s, but those that had lost staff remained sub-par well into the 1980s. ‘These calculations suggest that the dismissal of scientists in Nazi Germany contributed about nine times more to the decline of German science than physical destruction during WWII,’ Waldinger notes. By coincidence, his analysis indicated that the most influential scientists between 1920 and 1985, as measured by how often their research was cited by others, were Wigner among the physicists and von Neumann in maths.
In Göttingen, Born, Noether and Richard Courant, Hilbert’s de facto deputy, were among those who left as the mathematics and physics departments were decimated. Virtually all the founders of quantum mechanics emigrated en masse. Heisenberg stayed, only to be branded a ‘white Jew’ for his adherence to the theories of Einstein. Hilbert surveyed the scene with utter bewilderment. He hated chauvinism. Five years earlier, Germany had been invited to its first major international mathematics conference since the end of the First World War. Many of his colleagues tried to whip up a boycott, to protest their earlier exclusion. Ignoring them, Hilbert triumphantly led a delegation of sixty-seven mathematicians to the congress. ‘It is a complete misunderstanding of our science to construct differences according to people and races, and the reasons for which this has been done are very shabby ones,’ he declared. ‘Mathematics knows no races. For mathematics, the whole cultural world is a single country.’
As the sacked professors departed, the seventy-one-year-old mathematician accompanied them to the train station and told them their exile could not last long. ‘I am writing to the minister to tell him what the foolish authorities have done.’ The minister in question was, unfortunately, Bernhard Rust, who was instrumental in initiating the purges. Next year, when Rust attended a banquet at Göttingen, he asked Hilbert whether it was true that mathematics had suffered after the removal of Jews. ‘Suffered?’ replied Hilbert. ‘It hasn’t suffered, Herr Minister. It just doesn’t exist anymore.’88 Hilbert would die of natural causes a decade later in wartime Germany.
The golden age of German science was over. America was about to get an injection of talent that would transform its fortunes for ever. Von Neumann would soon be reunited with many of his Göttingen colleagues – not to discuss the finer points of quantum mechanics this time, but to design the most powerful bomb ever made.