2

Trust No One

Oh, the RAND Corporation is the boon of the world;

They think all day for a fee.

They sit and play games about going up in flames,

For counters they use you and me, Honey Bee,

For counters they use you and me.1

In the fall of 1948 most of the newly arrived postgraduate maths students at Princeton University in New Jersey were cocky, but one was even cockier. Still only nineteen, he was always boasting about his mathematical prowess. No one recalls him attending any of the regular classes; nobody saw him with a book. Partly it was because he was dyslexic, but it was also because he thought too much reading would stifle his creativity. He took regular detours down Mercer Street in the hope of catching sight of its most famous resident, Albert Einstein. One day he succeeded. But a few weeks into his first term he decided a remote glimpse was not enough. He made an appointment to see Einstein.

He told Einstein’s assistant that he had an idea about gravity, friction and radiation which he wished to discuss with the great man. Einstein listened politely, sucking on his tobacco-less pipe, while the twenty-year-old student wrote equations at the blackboard. The meeting lasted nearly an hour, at the end of which Einstein grunted: ‘you had better study some more physics, young man.’2 The student did not immediately follow Einstein’s advice but, years later, did go on to win the Nobel Prize – although for economics rather than physics. The student was John Nash, and his Nobel Prize-winning idea would become central to how we think today about interactions between people with conflicting interests.

To understand Nash’s brilliant idea – and the way it changed the direction of economics, as well as much of social science, biology, philosophy and law – we need to begin with the time, the place and the theory out of which it emerged.

The time and place was early 1950s Santa Monica, at the end of the Malibu Beach Crescent, just west of Los Angeles. The seaside promenade was lined with hotels and retirement homes, shades of cream and pink punctuated by bursts of vivid bougainvillea. The scent of oleander hung in the air. Santa Monica was an improbable setting for the offices of the RAND Corporation, a secretive think tank employing mathematicians and scientists to develop military strategy for potential nuclear war with the USSR. The Korean War had just begun and the Cold War was getting hot. The atmosphere at RAND combined paranoia, megalomania and a worship of abstract logic. Nuclear military technology was still in its infancy, and during the Second World War US generals had realized that they needed advice on the best way to deploy the latest weapons, from radar to long-range missiles, as well as the atom bomb. Such was the motivation for setting up RAND (an acronym for Research ANd Development) in 1948, initially as an offshoot of the Douglas Aircraft Corporation. RAND was described as ‘the Air Force’s big-brain-buying venture’.3 Its mission, in the words of influential RAND nuclear physicist Herman Kahn, was to ‘think the unthinkable’.

The intellectual framework for all this nuclear strategizing was game theory. It was the perfect tool for the RAND style of military thinking. Game theory assumes that humans are purely selfish and hyper-rational, in possession not only of all the information relevant to making decisions but of perfect and exhaustive powers of computation and logical reasoning.

John von Neumann is usually seen as the father of game theory. Nash may have been a genius, but he was almost a mathematical minnow in comparison to von Neumann.

DR STRANGELOVE AND THE KAISER’S GRANDSON

The 1964 film Dr Strangelove satirized the Cold War with its tale of impending Armageddon triggered by a crazy US air force general launching a nuclear first strike on the USSR. If you have seen the film, it is hard to forget Dr Strangelove himself, gesticulating wildly from his wheelchair in a strange Mitteleuropa accent. Bizarre though it may seem, the film drew heavily on real events. In 1956 President Eisenhower held regular secret meetings with a Hungarian mathematician confined to a wheelchair who would be taken back and forth by limousine to the White House from his bed at Walter Reed Hospital in Washington. The patient was under armed guard day and night because he would frequently descend into deranged babbling, so it was feared he might spill military secrets if an enemy agent could get to his bedside. The patient, in what was to be the last year of his life, was John von Neumann, undoubtedly one of the inspirations for Dr Strangelove. (At one point in the film, Strangelove refers to research by the ‘Bland Corporation’.)

Before his tragic decline ‘Johnny’ von Neumann’s genius was so overwhelming that it is hard to summarize. He was a mathematical prodigy: at the age of eight, when given any two eight-digit numbers, he could divide one by the other in his head. Although he was credited with the invention of game theory, mathematicians regard von Neumann’s work in pure mathematics as a greater achievement. He was undoubtedly one of the top few mathematicians of the twentieth century, often talked about as the greatest of all. Easier to describe is his memory: after just once reading any lengthy book, he could quote it verbatim in its entirety (he first performed this trick at the age of six, with pages from the Budapest telephone directory). One of the key people behind the invention of the atomic bomb and the modern computer, he could create and revise computer programs (involving forty lines of complex code) in his head alone. In the popular press he was seriously described as ‘the best brain in the world’. At Princeton, he acquired legendary status among colleagues, the joke being that he was not human but a demigod who had studied humans and learned how to imitate them perfectly. Notably, this joke was told about von Neumann rather than his Princeton contemporary Einstein.

Von Neumann’s opinion of Soviet Russia was as congenial to the RAND worldview as his game theory. Referring to the possibility of nuclear conflict with the USSR, he stated simply, ‘it is not a question of whether but of when’. Given this premise, there was a kind of game-theoretic logic to von Neumann’s advocacy of preventive nuclear war. Or, as he remarked in 1950: ‘If you say why not bomb them tomorrow, I say why not today? If you say today at five o’clock I say why not one o’clock?’4 For von Neumann, once the US had developed a hydrogen bomb (partly based on von Neumann’s ideas), the only way for them to maintain their advantage in the nuclear game was to bomb the Soviets before they had built a hydrogen bomb too. Secretary of State John Foster Dulles was convinced by von Neumann’s game-theoretic logic. Fortunately, President Eisenhower wasn’t so sure.

Von Neumann’s reasoning was straight out of the pages of Theory of Games and Economic Behavior, published in 1944, which von Neumann had written with Princeton economist Oskar Morgenstern. Like von Neumann, Morgenstern was an Austro-Hungarian émigré with a low opinion of the intellect of most people. Morgenstern cultivated an eccentric image: boasting that he was an (illegitimate) grandson of Kaiser Friedrich III, he rode around Princeton on horseback in bespoke three-piece suits. Morgenstern played an essential, but subordinate, role in developing the new theory, a kind of Dr Watson to von Neumann’s Sherlock Holmes.

At that time, the late 1940s, the orthodox view in economics still looked back to Keynes’s vision of the ideal economist as someone who was simultaneously a ‘mathematician, historian, statesman, and philosopher’.5 (Admittedly, this interdisciplinary economics often generated equivocal on-the-one-hand-but-on-the-other-hand advice, which led President Harry Truman to plead, ‘Give me a one-handed economist!’) Von Neumann and Morgenstern had no interest whatsoever in this Keynesian vision of economics; they bonded over their agreement that economics was in a mess. Von Neumann: ‘economics is simply still a million miles away from … an advanced science such as physics’.6 Morgenstern: ‘Economists simply don’t know what science means. I am quite disgusted with all this rubbish. I am more and more of the opinion that Keynes is a scientific charlatan, and his followers not even that.’7 But if economics was a mess, theirs would be the double act to fix it. Their plan was to use game theory to turn economics into a proper science.

Theory of Games and Economic Behavior opened with a suggestion that the effect of game theory on economics would be akin to that of Newton’s discovery of gravity on physics. Indeed, this claim underplayed the extent of von Neumann and Morgenstern’s ambition. They had originally planned to call their book General Theory of Rational Behavior, because their ultimate hope for game theory was that it would become the single underpinning framework for analysing human relations.

The initial reviews were ecstatic. In an instant, game theory was transformed from an obscure corner of pure mathematics into a new science of social interaction which caught the public’s attention: Theory of Games and Economic Behavior was a front-page story in the New York Times in March 1946.

This new science, however, contained a gaping hole. Von Neumann and Morgenstern’s book distinguished cooperative from non-cooperative game theory. In cooperative games, players can make agreements or contracts before the game itself begins. Non-cooperative game theory assumes such agreements are impossible, because they are unenforceable (players will make promises then break them). But the book did not discuss most non-cooperative games. It only covered one type: zero-sum games between two players.

Zero-sum games are those in which whatever is good for one player is bad for the other. This framing of the analysis can make a big difference. The nuclear stand-off between America and the USSR was a perilous, indisputably ‘non-cooperative’ game – but was it zero-sum? Should the strategists at RAND and the Pentagon rule out altogether, in the very set-up of their analysis, the possibility of an outcome in which neither side wins? And also rule out an outcome where both do (the origin of the term ‘win–win solution’)? By adopting von Neumann and Morgenstern’s zero-sum non-cooperative game theory, they could not consider such possibilities.

The lesson of this kind of game theory is simple. The best strategy is to calculate the worst-case outcome arising from each alternative you might choose, then choose the alternative that leads to the least bad of these worst-case outcomes. This minimax strategy is so called because you minimize your maximum possible loss. Effectively, you assume your non-cooperative opponent is trying to make you lose as much as possible (bad for you means good for them), so you minimize this risk. Von Neumann was widely regarded at the time as having ‘invented’ the minimax strategy,fn1 and it was exactly this reasoning which led to his conviction that America should drop a hydrogen bomb on Russia before the Soviets developed one too.

But the real and intellectual worlds moved on fast. In 1953 the Soviet Union conducted its first hydrogen-bomb test, making von Neumann’s advice redundant. And by this time minimax thinking had largely been superseded – by Nash himself, who had in the interim published a much more general approach to playing non-cooperative games which encompassed games involving more than two players and which were not zero-sum.

In 1950 Nash published the simple, elegant idea that made his name, nowadays known as the Nash equilibrium. Barely 300 words long, it had been accepted by the prestigious journal Proceedings of the National Academy of Sciences, a great achievement for a doctoral student. Von Neumann knew of this development in game theory but did not appreciate its significance. We know he knew, because Nash had arranged to meet von Neumann to tell him about it. Nash’s meeting with von Neumann was even less successful than his meeting with Einstein a year earlier.

Again, twenty-one-year-old Nash felt he had an idea worthy of the attention of a world-famous genius. But this time the rejection was more dismissive. Nash had barely uttered more than a few sentences outlining the mathematical proof he had in mind when von Neumann cut in: ‘That’s trivial, you know. That’s just a fixed point theorem.’8

In a sense, von Neumann was right. Nash’s equilibrium theorem was just a (mathematically straightforward) extension of a well-known theorem. Nash’s contribution was not as mathematically deep as any of von Neumann’s major mathematical achievements. But by providing a more general approach to playing non-cooperative games, Nash’s equilibrium idea effectively superseded von Neumann’s game theory. And it casts light on a central aspect of what it means to be human – interdependence.

Since our choices are interdependent, an individual’s best strategy depends on the strategies adopted by others. But in many situations – whether playing poker or competing against an arch-rival in business – you have to pick a strategy without knowing the strategies adopted by others. And, likewise, they will be choosing without knowing your strategy. Before Nash, reasoning in these situations seemed to fall ever deeper into a never-ending regress: ‘If you think I will choose X, then I will do better choosing Y. But if you think that I will choose Y because I think that you think I will choose X, then you will change your strategy, and I may be better off choosing X after all. But if you think that I have realized that, then …’ Put another way, chains of reasoning for a particular course of action often collapse once you realize that your opponent will probably be aware of that reasoning too.

Nash cut through this circular reasoning with the simple but brilliant insight that a particular pattern of social interaction will disappear as soon as anyone realizes they can do better by behaving differently. Hence, for a particular pattern of social interaction to persist, no one must have any reason to change their behaviour. And that must mean everyone has already adopted the best possible strategy given the strategies adopted by others. This is a Nash equilibrium. Even though, when making their decisions, no one knew what anyone else would do, it is as if everyone correctly guessed the strategy adopted by everyone else and picked their best response accordingly. The situation merits the ‘equilibrium’ label because it is stable: no one can do better by changing their behaviour.

Game theory has two obvious uses. First, as an aid to players in real-world games – a prescription telling you the best strategy to adopt in the situation you face. Second, as a tool for others to use in predicting what will happen – how the game will actually be played. The importance of this second use was obvious in the Cold War, as the entire world waited and wondered how the nuclear game between the US and the USSR would be played out. But it matters when the stakes are lower too: when Samsung and Apple play games with each other in the pricing and design of new smartphones, many outsiders try to predict what will happen – consumers, regulators and chip manufacturers all have an interest. In short, we look to game theory for an answer, a solution, comprising a prescription of how to play, or a prediction of what will be played, or both. Ever since Nash’s 1950 paper, Nash equilibrium has been the basis of that answer: simultaneously a prediction of what a stable outcome must look like and a prescription of how to play.

Nash equilibrium bears the mark of a real intellectual breakthrough – an idea that had not occurred to anyone before Nash yet one that with hindsight seems entirely obvious. Together, von Neumann, Morgenstern and Nash had triggered a revolution in our thinking about human interaction. What happened next?

NON-COOPERATION ABOUT NON-COOPERATION

At first, nothing. Economists did not adopt game theory; a few mathematicians elaborated the mathematics of game theory as a project in pure mathematics; and RAND doggedly pursued a game-theoretic approach to military strategy, with few results of practical significance.fn2 Despite the extravagant early praise lavished on their ideas by economists, there was virtually no progress in von Neumann and Morgenstern’s grand project of doing for social science what Newton did for mechanics. Some spoilsports had even pointed out a crucial flaw in the analogy: balls, planets and all the other subjects of Newtonian mechanics are not aware of being studied. Humans are aware – and may change their behaviour accordingly.

Meanwhile, the project was plagued with people problems: indeed, it almost didn’t get off the ground at all. Von Neumann’s dismissal of Nash’s equilibrium idea, combined with some relatively minor criticisms from his PhD supervisor, led Nash, who struggled to cope with intellectual criticism, seriously to consider abandoning research in game theory altogether. By the end of the 1950s the root of his problems would become clear: diagnosed with paranoid schizophrenia, he would spend ever-longer periods of time in hospital. Already, his condition dominated his behaviour: aggressively competitive, even by the standards of the elite young mathematicians he mingled with, he had almost no awareness of the brash abrasiveness which so alienated him from others.

While many of von Neumann’s greatest intellectual accomplishments emerged in collaboration with others, and most of the (collaboratively written) Theory of Games and Economic Behavior was devoted to cooperative game theory, Nash was a loner. Indeed, he argued (in another path-breaking paper published just a year after his paper setting out the Nash equilibrium idea) that von Neumann’s cooperative game theory was redundant. All cooperative games, Nash argued, should be understood as in fact non-cooperative: the seemingly cooperative phase, involving players making agreements before the game begins, should be seen as itself a separate, non-cooperative game. For Nash, in other words, what superficially looks like cooperation turns out to be nothing of the sort. This approach, which came to be known as the Nash program, was the first of many such programs across the social sciences from the 1960s onwards which ‘explain’ seemingly cooperative or altruistic behaviour as really non-cooperative and selfish underneath. Neither Nash nor his game theory did cooperation. From a comfortable post-Nobel Prize vantage point four decades later, Nash was phlegmatic about the initial rejection of his equilibrium concept by von Neumann and others, but his game-theoretic view of the world remains striking: ‘I was playing a non-cooperative game in relation to von Neumann rather than simply seeking to join his coalition. And of course it was psychologically natural for him not to be entirely pleased by a rival theoretical approach.’9

Nash’s descent into schizophrenia was surely one reason why game theory was not rapidly adopted by economists: the leading advocate of non-cooperative game theory went silent. Equally important was the attitude of von Neumann. In the mid-1950s he was busy with the development of the bomb and the computer. When he did have time for game theory, he reiterated his complaint that mainstream economic theory was mathematically primitive. In so doing he deeply antagonized many of the self-confessed ‘mathematical’ economists in academic economics at that time. Yet whatever von Neumann might have thought of their mathematics, these were the obvious people with sufficient mathematical skills to begin to incorporate game theory into social science. Von Neumann had alienated the academic audience likely to be most receptive to his game theory.

Given von Neumann’s mathematical ambitions for social science, it was ironic that what finally propelled game theory beyond RAND and university maths departments was not maths but a story.

Albert Tucker was John Nash’s PhD supervisor. In May 1950, just after persuading his wayward student not to abandon his PhD on game theory, Tucker was asked to talk about the new theory to a group of psychologists. Since his audience did not know the theory or the maths, Tucker decided to present a game he had learned about from some RAND researchers in the form of a little story. He called it the Prisoner’s Dilemma.

Two members of a criminal gang have been imprisoned separately. The police have enough evidence to convict them both of a minor crime, but not the major one that they suspect them of committing. So they offer each prisoner the following deal: confess and implicate your partner and you receive immunity from prosecution while your former partner will be hit with a ten-year sentence. If you both stay silent, you will each be given a two-year sentence for the minor crime. The snag, however, is that if you both confess, the immunity deal is off and you will both be sentenced to eight years.

Assuming each prisoner cares only about their own sentence, what should they do? Although they have no means of communicating with each other, the prisoners believe they have both been offered the same deal. However, they can reason their way out of the dilemma. ‘If my former partner in crime does not confess but I do, I get the lowest possible sentence. If he does confess, I will get a lower sentence if I confess than if I don’t. So, either way, I should confess.’ The trouble is, since both prisoners may reason alike, they may both confess and be given eight years in jail, significantly longer than if neither had confessed. Even if the prisoners could communicate and agree to stay silent, the outcome would surely be the same, because each prisoner would be tempted to break the agreement in the hope of a lower sentence.

In 1950 no one had any inkling that the Prisoner’s Dilemma would later become the most influential game in game theory. Unsurprisingly, RAND was interested in the game for military reasons: the nuclear arms race between the US and the USSR was a classic Prisoner’s Dilemma, both sides building more and better weapons in a futile attempt to gain an advantage. But the game’s structure (rather than its story) captures more than just Cold War rivalries. It elegantly expresses the conflict between private and collective interest in thousands of real-world contexts. Firms producing similar products – OPEC oil producers or Coke versus Pepsi – cut prices to gain market share, but because their rivals do the same all firms suffer a fall in profit. The Prisoner’s Dilemma describes this and many other ‘races to the bottom’. Likewise, the so-called Tragedy of the Commons describes a Prisoner’s Dilemma: given free access to a common resource, everyone will consume it regardless of what others do, leading to damage or destruction of the resource and making everyone worse off. The challenge posed by climate change is, too, widely agreed to be a Prisoner’s Dilemma: everyone is better off if global carbon emissions are reduced, but every country is reluctant to reduce emissions, regardless of what other countries do. You and the other spectators face a Prisoner’s Dilemma when deciding whether to stand up for a better view at a sports event: everyone stands, so everyone is worse off than if they had remained seated.

In the original Prisoner’s Dilemma story, the reasoning described earlier implies that both players should confess. And this outcome is a Nash equilibrium: if your partner confesses, you do best by confessing too. So the blame for the damaging non-cooperation nurtured by this reasoning may seem to lie with John Nash’s equilibrium idea. But – although millions of students in social science, philosophy, law and biology are today introduced to game theory via the Prisoner’s Dilemma and its Nash equilibrium ‘solution’ – the Nash equilibrium idea is not driving the outcome here. There is a more basic logic at work: regardless of what the other player does, your best action is to confess. Any predictions about, or agreements with, your opponent are irrelevant: you will always do better in Prisoner’s Dilemma situations if you respond with a ‘non-cooperative’ action.

TRUST WITHOUT TRUE TRUST

Given the set-up of the Prisoner’s Dilemma, this logic is unassailable. Rational players must play the Prisoner’s Dilemma this way – and suffer the consequences, from longer jail terms to nuclear arms races. This follows directly from game theory’s assumption that rational human behaviour is non-cooperative and distrustful. Von Neumann could not imagine it otherwise: ‘It is just as foolish to complain that people are selfish and treacherous,’ he asserted, ‘as it is to complain that the magnetic field does not increase unless the electrical field has a curl. Both are laws of nature.’10

It is easy to shudder or scoff at this simplistic view of humanity – and that is precisely what critics did in 1944, when von Neumann and Morgenstern published their magnum opus. The influential British anthropologist Gregory Bateson remarked that ‘premises of distrust are built into the von Neumann model’, as was ‘the more abstract premise that human nature is unchangeable’. There was, he concluded, nothing human about ‘von Neumann’s players’, who ‘differ profoundly from people and mammals in that those robots totally lack humour and are totally unable to “play” (in the sense which the word is applied to kittens and puppies)’.11

Players in game theory are unlike real humans. At best, they are partial and incomplete representations of humanity. What’s more, game theory misses a crucial part of what it means to play games, because its vision of rational behaviour rules out all the play, all the fun. But so what? Even if game theory leaves a lot out, it might still give us valuable insights into social interaction – for instance, in contexts where the selfish, ruthless, calculating side of humanity is to the fore.

But what contexts are those? From the 1960s onwards, as game theory began to creep out of its academic niche into wider discussion in social-science departments and beyond, it became clear that the biggest challenge to the Prisoner’s Dilemma was reality – the undeniable fact of cooperation in so many real-world contexts which look just like Prisoner’s Dilemmas. Returning to the alleged Prisoner’s Dilemmas mentioned earlier, firms often resist the temptation to cut prices, knowing a price war would harm them. Common resources are often sustainably managed, while countries have cooperated to limit carbon emissions. We don’t all stand up at sporting events, and nuclear-arms-control measures were eventually agreed upon. If the Prisoner’s Dilemma captures the essence of these interactions, why do we observe cooperation in the real world?

Real people in Prisoner’s Dilemma situations can cooperate by agreeing to do so before they play the game. They trust each other to keep their promises. For most people, keeping promises and trusting others is normal, default behaviour, because they have been raised and educated to behave that way and experience confirms it generally makes life more liveable. Put another way, we escape the destructive consequences of following game theory’s prescription of ‘rational’ behaviour in the Prisoner’s Dilemma by rejecting this definition of ‘rational’. As Nobel Prize-winning economist Amartya Sen put it in 1977, game theory’s advice on how to behave in Prisoner’s Dilemma situations shows us not how to be rational but how to be a ‘rational fool’. The overwhelming evidence of cooperation in real-world Prisoner’s Dilemmas suggested that not only was game theory bad at predicting how we do behave, its advice on how we ought to behave was suspect too. No one wants to be a rational fool.

But by the late 1970s game theorists had developed a response to this challenge with the theory of repeated games. In repeated games, people cooperate because they look to the future. People cooperate, even in Prisoner’s Dilemma situations, for the sake of maintaining beneficial relationships in the future. Cheats, promise-breakers and the selfish generally find themselves shunned; they miss out on the benefits of future cooperation. This is the same cold and calculating view of human interaction as that underpinning the Prisoner’s Dilemma – it is the Prisoner’s Dilemma, but played repeatedly over time. If you know that you will meet the same opponent in another Prisoner’s Dilemma situation in the future, you may cooperate now, for the sake of long-term benefits from cooperation – because if you break your promise or act selfishly now, you may find yourself punished by non-cooperative behaviour from your opponent in the future. The idea extends beyond two people: a group can hang on to some form of cooperation among its members by constant threats to punish selfish behaviour by those who fail to act in the group interest. The punishment is usually short and sharp – tit-for-tat, as the game theorists aptly describe it – but not too severe, as that would be too costly for those doing the punishing. One implication is that groups need not rely on external pressures such as law, coercion or social convention to sustain cooperation. Tit-for-tat will suffice, although to an observer it can look more like anarchy than stable society. The Mafia is a perfect example of tit-for-tat in action. Over a hundred years ago Pasquale Villari, a Neapolitan politician, observed, ‘The Mafia has no written statutes, it is not a secret society, hardly an association. It is formed by spontaneous generation.’12 In recent times, fans of Friedrich Hayek have invoked this aspect of game theory in an attempt to give mathematical credibility to Hayek’s idea that seemingly anarchic societies with little or no government can sustain themselves through ‘spontaneous order’.

Although the obvious political home for game theory seems on the Right – relentless competition between selfish individuals, self-organizing societies with no need for government – thinkers on the Left have pressed it into service too. They have argued that, contrary to appearances and von Neumann’s views, game theory is compatible with a nicer, more trusting picture of human relations. Indeed, game theory can explain why we trust each other: I trust someone when I know they have an incentive to keep their promises. We are playing a repeated game in which we both know that any immediate gain either of us can make from breaking promises now is outweighed by losses from future punishment. The punishment may be inflicted not just by the victim of broken promises, but by the wider community: it is in everyone’s interest that trust is possible and deals can be done. This way of thinking about trust may help us understand why some human relations and institutions work as they do. But it raises more questions than it answers.

To begin with, it implies that you can only trust those who have no incentive to break their promises to you. This game-theoretic perspective turns our normal understanding of trust upside down: it implies we can trust someone only when we don’t need to – because it is in their interest to keep their promises anyway. Real trust means having faith in someone to keep promises, to do the right thing, because we believe in their good character, even in circumstances where we know they could profit by breaking promises, stabbing us in the back. Even in the competitive world of business, people need real trust more than the ersatz game-theoretic variety. There are too many situations where businesspeople are trying to do deals without any expectation of future interaction: they cannot assume they are playing a repeated game. Instead, they unconsciously rely on facts about human psychology which game theory ignores. It is, for instance, much easier to decide whether someone can be trusted if you meet them face to face. This is why, even in the era of Skype, business leaders are still willing to fly across the world for a crucial meeting.

Clearly, some game-theoretic explanations of trust and long-term cooperation seem to miss the point. But there is a more basic problem with this kind of response to the Prisoner’s Dilemma. Even if strategies like tit-for-tat can help sustain cooperative behaviour when people interact repeatedly over a fairly long period, what about one-off interactions? Here the gap between theory and reality remains: game theory predicts that people will not cooperate in one-off Prisoner’s Dilemma situations, yet they often do. Game theorists did not face up to this problem for many years. Most of them didn’t accept that there was a problem. Instead they used a dodge which has often been used by economists when confronted with evidence of altruistic, cooperative or moral behaviour – reinterpret the evidence to make it go away. Thus, players who are apparently cooperating in a one-off Prisoner’s Dilemma are not really doing so, because they are not actually playing a Prisoner’s Dilemma. By definition, a pure Prisoner’s Dilemma is a game in which each player cares only about their own prison sentence. Players who also care about their opponent’s welfare, or believe in group solidarity, or know they will suffer the pangs of bad conscience if they break a promise to their opponent, and so on, are playing some other game. In the mathematical representation of the situation – which is, in the end, all that concerns game theorists – these other considerations would be captured in the single number representing the value or worth of a particular outcome for a player. These additional considerations would lead to a different number for most outcomes, compared to the pure Prisoner’s Dilemma. Hence the new game is not the same.

The drawback with this dodge is that, taken to extremes, it can be used to define away any evidence that conflicts with the theory. It certainly makes it exceptionally difficult to find real-world evidence that is not vulnerable to this kind of dodge. It took until the 1990s for enough evidence of cooperation to emerge in a form which game theorists could not bypass or ignore: carefully designed experimental games played under lab conditions in which the information provided to participants – and, by extension, their possible motives – was strictly controlled. But by then game-theoretic thinking was securely embedded in economics and wider society. Its influence has become so strong that we fall back on it in times of crisis, to help define our civilization and identity. Three days after the 11th September 2001 attacks on New York and Washington, a New Yorker commentator tried to capture its significance:

The calamity, of course, goes well beyond the damage to our city … it is civilizational. In the decade since the end of the Cold War, the human race has become, with increasing rapidity, a single organism … The organism relies increasingly on a kind of trust – the unsentimental expectation that people, individually and collectively, will behave more or less in their rational self-interest.13

Although, from the 1960s onwards, game theory began to influence everyday thinking, game theorists themselves were focusing on its limitations. In particular, they were becoming aware that in all too many contexts game theory seemed to have little to say.

CHICKEN

In 1955 the philosopher Bertrand Russell released an influential manifesto, co-authored by Albert Einstein, calling for nuclear disarmament. But Russell had an unintentionally greater impact on the disarmament debate just a few years later, by publicizing a game called Chicken. Painting a picture which could have come from the James Dean hit movie of the era, Rebel without a Cause, Russell imagined the US and the USSR as rival young drivers, speeding towards each other down the middle of a long, straight road. If neither swerved, both would die. But the first to swerve, the ‘Chicken’, would earn the everlasting contempt of his rival.

Chicken soon became a benchmark game in discussions among Cold War thinkers, game theorists and their students. In 1960 RAND strategist Herman Kahn adopted Chicken as a game to describe the nuclear stand-off in his influential and bestselling 652-page tome, On Thermonuclear War. Russell questioned why playing Chicken for nuclear high stakes seemed morally acceptable in RAND circles, while teenagers playing Chicken for much lower stakes were criticized:

As played by youthful plutocrats, the [Chicken] game is considered decadent and immoral, though only the lives of the players are risked. But when the game is played by eminent statesmen … it is thought that the statesmen on one side are displaying a high degree of wisdom and courage, and only the statesmen on the other side are reprehensible.14

In any case, Chicken was not a helpful bit of game-theoretic analysis, because it has two Nash equilibria, the first being ‘your opponent does not swerve, you do’ and the second being ‘you do not swerve, your opponent does’. Game theory here makes no prediction about what will happen, or what should. The importance of this limitation was clear in the context of the Cuban Missile Crisis two years later when, in October 1962, both the US and the USSR refused to back down in their confrontation over the placing of Russian nuclear missiles on Cuba. It was obvious to both sides that Chicken was the game being played. However, what they both wanted to know was: which Nash equilibrium? In other words, who would swerve first? A mistake could mean annihilation. Most historians agree that the world has never come closer to full-scale nuclear war than during the Cuban crisis.

To recap, game theory’s two obvious uses are to provide a prediction about how the players will behave, and/or a prescription about how they should. In games with more than one Nash equilibrium, like Chicken, game theory seemed to fail on both counts. Even game theorists began to ask: what’s the point?

Worse still, over the coming years it became clear that games with multiple Nash equilibria were not rare exceptions. They were ubiquitous. Game theory provided no guidance in these situations. And by the time the importance and ubiquity of this so-called multiplicity problem became clear, Nash was in no position to help.

EMPEROR OF ANTARCTICA WINS NOBEL PRIZE

Already, by early 1959, Nash’s descent into madness had begun to accelerate. He was offered a top professorship at the University of Chicago, but wrote back explaining he could not take up the post because he was about to become Emperor of Antarctica. This was not an isolated incident. Around that time, Nash gave his expired driving licence to one of his students, writing the student’s nickname over his own and telling the student furtively that it was an ‘intergalactic driver’s licence’.15 Von Neumann had died two years earlier, so the two founding fathers of game theory were now silent. After the excitement of the early years, game theory had slipped, in the eyes of most economists, from the best hope for a comprehensive science of society to an intellectual dead end, bogged down with the multiplicity problem, which would preoccupy game theorists for years to come. As for Nash himself, by the 1980s many younger game theorists assumed he was dead. Other rumours circulated that he had had a lobotomy or lived in a secure psychiatric hospital. Yet in 1994 Nash, along with two other game theorists, John Harsanyi and Reinhard Selten, won the Nobel Prize for economics. How did game theory make such a brilliant comeback?

There are two versions of the history of game theory in the forty-odd years that elapsed between von Neumann’s death and Nash winning the prize. Let’s begin with the official history. It is straightforward: Harsanyi, Selten and others made good progress on solving the multiplicity problem. Alongside other innovations like repeated game theory, the overall result was that game theory became useful again.

In the 1960s John Harsanyi explicitly set out the challenge game theory faced: solve the multiplicity problem and provide a determinate solution for every game, derived solely from general principles of rational behaviour. If this could be achieved, it would bring about the pure science of social interaction dreamed of by von Neumann, Morgenstern and Nash. The first major ascent in this Everest project was achieved by Reinhard Selten in 1965. To address the problem of multiple Nash equilibria, the obvious line of attack is to find grounds for ruling out some of these equilibria as inferior. Selten argued that some equilibria are inferior because they can only emerge when players make threats which are not credible. For example, the MAD (Mutually Assured Destruction) doctrine of nuclear deterrence relies on nuclear powers threatening catastrophic retaliation in response to a nuclear attack. But the threat is not credible if the recipient of the threat does not believe it would be carried out. In Dr Strangelove the Russians designed their Doomsday machine to trigger catastrophic retaliation automatically and irrevocably once an attack had been detected, removing any possibility of non-retaliation and hence making their threat completely credible.fn3 In business, a monopoly firm in a particular market will often loudly threaten a price war to any firm considering entering the market as a competitor. If the new firm believes the threat, it may stay out of the market, allowing the monopolist to continue reaping big profits.

A threat is credible only if the player making the threat won’t become worse off by carrying it out. Selten cleverly generalized and extended this idea, arguing that, at every stage in a game, players won’t make choices that make them worse off, regardless of what they have said beforehand. Since ‘what makes you worse off’ depends on what you and the other players do later in the game, you find your best strategy by deciding what will happen at the end of the game and reasoning backwards from that, until you have decided your first move. This backward induction procedure can yield a determinate solution, a prediction of how to play, just as the game theorists hoped. But it yields some big surprises too.

Imagine a TV game show with the following format. The two players (let’s call them Johnny and Oskar) know that the host has a maximum of $1,000 in prize money to hand out. The host starts by making an offer to the first player, Johnny – both players get to keep $100. If Johnny accepts the offer, both players leave with $100. If Johnny refuses, the host makes a new offer to the second player, Oskar: $50 for Johnny, $250 for Oskar – the total prize-money pot has increased by $100 but now it is to be split unequally. The game ends if Oskar accepts the offer, but if he refuses it is Johnny’s turn again. Again, the total prize money pot increases by $100, but now it is to be split equally: $200 for each player. Again, the game ends if Johnny accepts the offer, while if he refuses the pot is increased by another $100 but split unequally: $150 for Johnny, $350 for Oskar. And so on, if both players keep refusing, until the host offers $350 for Johnny, $550 for Oskar. If Oskar refuses that, they both get $500 and the game ends.

It seems that, with just a bit of patience, Johnny and Oskar can share the maximum prize pot of $1,000 between them. But Selten’s backward induction procedure implies otherwise. Both players know that Oskar will be better off accepting the host’s last offer (he receives $550) than by refusing (he receives $500). So both players know that Oskar would accept that offer. So, reasoning backwards, Johnny knows he is better off accepting the host’s previous offer ($400 each) than refusing it (he would receive $350 when Oskar accepts on the following round). The same reasoning applies to all previous rounds of the game too. Both players realize that they will always be better off accepting the host’s offer than letting the game continue and getting a smaller prize when their opponent accepts in the next round. So Johnny should accept the very first offer of $100 and end the game immediately. Backward induction reasoning prevents either player from getting a bigger prize, because neither can trust the other to let the game continue past the next round. It is another variation on the familiar story of game theory: in the pursuit of hyper-rationality, cooperation is subverted and everyone is worse off.

Of course, people rarely think like this: in a large number of experiments with people playing games like the one just described, very few of them behave as prescribed by backward induction reasoning. In an attempt to suggest that people do use backward induction reasoning, if only they are smart enough, economists have recently repeated these experiments using chess Grandmasters as players. The results were ambiguous: some Grandmasters play as prescribed by backward induction reasoning; some don’t. This brings us to a deep flaw in the logic of backward induction. In the game show just described, to conclude that you should accept the first $100 offer, you need to believe that your opponent is following backward induction reasoning and so will accept the first offer they receive. In other words, even if you are smart enough to understand backward induction reasoning, should you assume your opponent does too? The chess Grandmasters knew they were playing other Grandmasters, so for them it may have been a sensible assumption. But for most people, it isn’t. If you are on the game show and your opponent refuses the first offer they receive, then you know, from observing this refusal alone, that they are not following the rulebook of game theory, because backward induction implies they should accept the first offer. More generally, in real-life interactions beyond the game show, we frequently deal with people who break the rules of game theory. It would be unwise to assume they will follow the rules in future. Game theorists call the rule-breakers ‘irrational’ and insist we should assume everyone is rational. No: given good evidence of someone’s ‘irrationality’ in the past, it would be truly irrational to assume they will be ‘rational’ in the future.

The official history of game theory has largely ignored these problems. Some game theorists have always accepted that people often do not behave as game theory predicts. More modestly, they hold that game theory should not be understood as providing predictions, only prescriptions for how best to play. But games like the game show described above undermine even this modest view, because they show that following the prescriptions of game theory is not always the smart way to play a game after all. Which is where we must turn to the unofficial history.

A ZOMBIE SCIENCE OF HUMAN LIFE

Despite rumours to the contrary, Nash had been quietly working back at Princeton for some years before the Nobel announcement. On the afternoon of the announcement he gave a short speech. Nash’s odd, uneasy sense of humour was still there. He noted that Nobel laureates are supposed to say how pleased they are to be sharing the honour. But Nash said he would have preferred to win the prize alone, because he really needed the money. He finished his speech by comparing game theory to string theory in physics, both subjects that researchers find intrinsically fascinating – so they like to pretend that both subjects are actually useful.16 It was perhaps characteristic of Nash, especially after the Nobel announcement, to be playfully dismissive of game theory: around that time he described his own contribution to game theory as his ‘most trivial work’.fn4 But another of the laureates alongside Nash also raised concerns about the triviality of game theory. Reinhard Selten, who had used the paradoxical backward induction procedure in an attempt to solve the multiplicity problem, had turned away from such theoretical indulgences. From the late 1970s onwards Selten repeatedly stressed that game theory was too formal and mathematical to be a reliable guide to how people actually think in social interactions: ‘game theory is for proving theorems, not for playing games’.17

However, there seems to be a clear exception to this negative verdict. Game theory can tell us the smart way to behave in some economic and social contexts – that is, contexts in which every player knows that every player is well versed in game theory, as if they have a state-of-the-art textbook in hand. So if a chess Grandmaster is playing another chess Grandmaster, then both of them might reasonably assume their opponent has a sophisticated knowledge of game theory. Such a defence of game theory is (a bit) less useless than it seems. On 5th December 1994, the day John Nash left America for Stockholm to collect his Nobel Prize, Vice-President Gore was announcing the ‘greatest auction ever’ – an auction of airwave frequency spectrum licences to be used by mobile phones. Auctions are a type of game, and this auction was carefully designed using the latest game theory. When the auction closed in March 1995, the US government was delighted: it had received more than $7 billion in bids. The spectrum auctions, great revenue-raising successes for government, were hailed as a triumph of applied game theory. Here, at last, was a setting in which truly ‘rational’ players would interact – big corporations competing in an auction, each advised by a team of game theorists – leading to outcomes which could be predicted and tweaked by the game theorists designing the auction, on behalf of the government. Or so it seemed.

In reality, game theory did not provide the recipe for an ideal auction design to meet the government’s objectives, because the theory could not adjudicate between conflicting auction designs. Different game theorists made different recommendations. This was not surprising, given that these theorists were employed as consultants for competing corporations. Moreover, game theorists didn’t just provide advice on how to bid in a predetermined auction: they were employed from the very beginning of the process to help corporations lobby for particular auction designs, rules which would help skew the game in their favour. And the final outcome does not suggest hyper-rational corporate players after all. Many successful bidders defaulted on their payments, and the later rise in failures and mergers in the telecoms industry was widely attributed to the burden of excessive auction bids.18 The experience with a British spectrum-licence auction in 2000, also heavily influenced by game theorists, was similar: game theory did not tell the government how best to design the auction, nor did it adequately explain or predict the behaviour of bidders.

If game theory has limited use even in situations like these auctions, designed by game theorists as a potentially ideal playground for the theory, then why does it enjoy exalted status in economics today? There is no consensus answer, but there are common themes.

To begin with, the rise of game theory was not due to its successes or strengths but because economists turned to it to fix problems elsewhere in economics, or at least to find new techniques to break a stalemate in long-running debates. By the 1970s, for example, the regulation of big corporations was increasingly shaped by the ideas of lawyers and economists at the University of Chicago. This Chicago approach to ‘law and economics’ essentially argued that the less regulation there was, the better. Dominant corporations were dominant, they argued, because they offered better products at lower prices, not because of anti-competitive practices. Game theory gave opponents of the Chicago view a new framework which took anti-competitive behaviour seriously, a framework which impressed regulators and courts because of its high-status mathematical sophistication. Less politely, it was a new gimmick which might give you the upper hand in policy debate or legal argument. Elsewhere, ambitious economists were making imperialistic forays into aspects of life beyond the scope of markets and prices, and hence beyond the scope of the traditional tools of economic analysis. Game theory provided a new toolkit for these economists, who saw themselves as social engineers designing institutions and mechanisms to produce desired social outcomes. In their own terms, these academic users of game theory were remarkably successful: after Nash, Harsanyi and Selten in 1994, research essentially based on game theory led to Nobel prizes for eight more economists over the following twenty years.

In contrast, economists who questioned game theory faced ostracism. Rather than following the standard game-theoretic practice of making assumptions about human behaviour, Selten himself had become a firm believer in using lab experiments to study how people actually behave. As far as some game theorists were concerned, this reduced him to ‘a turncoat, who has lost (or even worse) has left the path to the “pure and true” cause of game theory’.19

But Selten was an exception among economists. Game theory was not short of pure and true believers. Its seductive power should not be underestimated. Despite the problems with the theory, the temptation of a pure science of society, a grand unification theory for social science to rival that hoped for in physics, has proved irresistible. This seduction was reinforced by what Robert Axelrod, a US political scientist turned game theorist, called the ‘law of the instrument’: give an academic (or a child) a hammer, and they will find things to hammer. So game theory was used to ‘explain’ trust, although it is far from obvious that before game theorists came along there was any mystery about trust that needed explaining. Game theory is a kind of zombie science, a vision of human interaction which, no matter how broken it seems, never dies. Many thinkers abandon the project, but new recruits revive the grand dreams. As one recent convert solemnly intoned, ‘game theory is a general lexicon that applies to all life forms. Strategic interaction neatly separates living from non-living entities and defines life itself.’20

These fantasies matter to the rest of us, in ordinary life. Game-theoretic ideas have spread out of academia to become part of common-sense thinking. But along the way, some subtleties have been lost. It is widely believed that cooperation is mostly for suckers and only the naïve rely on trust. In particular, game theory has been understood to prove, as a matter of irrefutable logic, that it is irrational to be altruistic, trustworthy or cooperative, even when the people you are dealing with are altruistic, trustworthy or cooperative. But this is a fundamental misunderstanding of the theory.

Yes, game theorists – especially in the early days of von Neumann, Nash and RAND – often assumed people are always selfish. But the circumstances under which game theory justifies or recommends selfishness are remarkably narrow. Nash’s equilibrium idea essentially implies that if everyone else is behaving selfishly, you should do so too: selfishness is your best response. And their selfishness is then their best response to yours: we can get locked into non-cooperative situations. But crucially, in many contexts, we cannot assume that everyone else is behaving selfishly in the first place. And without this assumption, the explanation for why we get locked into non-cooperative situations disappears.

Put another way, game theory says we will end up in a Nash equilibrium, but it does not explain which equilibrium – cooperative, non-cooperative or otherwise. It is a Nash equilibrium that everyone drives on the same side of the road, and there are two equilibria: everyone drives on the left, and everyone drives on the right. Game theory has little to say about which equilibrium will emerge, and why it differs across countries. Likewise, the QWERTY layout for keyboards is a Nash equilibrium: if everyone is using QWERTY to type, and almost all keyboards are manufactured with QWERTY, then you should learn to type using QWERTY too, and new keyboards will be made with that design. Therefore, the equilibrium will be maintained even though it is much slower to type in QWERTY than in rival layouts such as DVORAK: the equilibrium persists even though all keyboard users are worse off. But again, game theory does not explain why we got stuck in this slow equilibrium, with the slow QWERTY layout.

The key question, then, is often less about why a Nash equilibrium persists once the players are playing their equilibrium strategies, and more about whether we will reach that equilibrium in the first place: a question of history rather than game theory. (In the case of QWERTY, its convoluted form was precisely the point: it was invented to slow down typists in an era of mechanical typewriters with keys that were liable to jam when used at speed.) Most troubling of all for game-theoretic orthodoxy, even if a game has only one Nash equilibrium, it does not follow that we will reach it – that it will be the outcome when the game is actually played. Playing the Nash equilibrium strategy is only the best way for you to play the game if everyone else is playing the Nash equilibrium strategy too. But as we have seen, there are several good reasons why you might think others won’t be playing their Nash strategy – because they are not selfish, or because they don’t think like game theorists. This is a very basic hole in the theory, yet none of the textbooks mentions it.

At the climax of George Orwell’s Nineteen Eighty-Four, Winston and Julia are literally in a Prisoner’s Dilemma: each is held separately and tortured to try to force them to betray each other. But here, the prediction of game theory goes wrong. They don’t betray each other. Orwell’s understanding of what it means to be human makes love, friendship and loyalty paramount. These are concepts which have no place in traditional game theory. But why, exactly? Why can’t traditional game theory encompass a full understanding of what it means to be human?

As we saw earlier, when people in a seeming Prisoner’s Dilemma situation don’t behave as predicted by game theory, game theorists respond by arguing that these players cannot really be in a Prisoner’s Dilemma. The rules of that game specify that players must be narrowly self-interested, so by definition people such as Winston and Julia do not face a Prisoner’s Dilemma. Once we incorporate Winston and Julia’s concern for each other in the mathematical representation of the game, then it recommends that they should not betray each other. Their love, friendship and loyalty tilts the ‘best strategy’ calculation in favour of cooperation. More generally, the argument runs, anything a player cares about can be included in game theory, by adjusting the numbers representing the consequences which follow from each choice.

Yet game theory imposes one subtle but critical restriction: it is concerned not with the historical contexts of different choices, but solely with their consequences or outcomes.21 Consequences are inherently forward-looking, while our moral concerns about fairness and responsibility typically look backwards: to the history of who did what, and why. This focus on consequences alone means that game theory must inevitably operate with a restricted, partial understanding of what it means to be human, an understanding which insists our future is always more important than our past. In later chapters we will find a similar pattern arising repeatedly elsewhere: the attempt to incorporate moral concerns within standard economic theories, in the process restricting, distorting or subverting them.

In the last months of his life von Neumann did something which shocked all who knew him. Perhaps even he had begun to look beyond game theory’s limited view of humanity. Or perhaps it was just a side-effect of the mental and physical breakdown wrought by the cancer which now overwhelmed him. A firm agnostic throughout his life, he had himself baptized a Catholic. Confined to his hospital bed, he was regularly visited by Father Strittmatter, a Benedictine monk who received his confessions. It didn’t seem to help: as Strittmatter recalled, von Neumann remained terrified of death until the end. As he drove away in a cab after von Neumann’s burial, done with full Catholic rites, the Director of the Los Alamos Laboratory remarked to a physicist colleague: ‘If Johnny is where he thought he was going, there must be some very interesting conversations going on about now.’22