One entity in the world is completely free of gender prejudice and preconceived ideas about who is stronger, men or women, and that is the computer. Throughout the history of mechanical and digital computing, chess has been near the forefront of designers’ minds. As soon as man invents a machine, it seems the next step is to turn his creation into a chess player. One reason is that so many geniuses and inventors have played the game—maybe not brilliantly, but with passion and interest. Chess has always maintained a position as, in Goethe’s words, “a touchstone of the intellect.” Just about everyone who created a “thinking machine” was quick to put it to the test of mastering the world’s most respected game.
The first chess-playing “machine,” known as the Turk, was introduced to the general public in 1769. The Hungarian engineer Baron Wolfgang von Kempelen created it for the amusement of the Hapsburg empress Maria Theresa. This purely mechanical device was hidden beneath a majestic mannequin dressed as a Turk. Predictably, it was a fake: its outstanding playing strength was in fact supplied by a chess master cleverly secreted inside the device.
The main challenge of chess programming is the large number of continuations involved. In an average position there are about forty legal moves. So if you consider every reply to each move, you have sixteen hundred positions. After two moves there are 2.5 million positions, after three moves, 4.1 billion. The average game lasts forty moves, so the numbers involved are beyond astronomical.
Remarkably, the first computer program was written before a computer existed that could run it. Its creator was Alan Turing, the British mathematician who led the group that broke the German Enigma code during World War II and is widely considered the father of modern computer science. He developed a series of instructions for automated chess play, but since there was as yet no machine that could execute this first-ever chess code, Turing worked through it himself, on paper. Around the same time, in the United States, another great mathematical mind, Claude Shannon, was outlining several strategies computers could use to play chess.
In 1950 the nuclear laboratory of Los Alamos was the unlikely site of the next step forward in chess computing. When the gigantic machine MANIAC 1 was delivered, the scientists tested it by writing a chess program. After playing against itself and then losing to a strong player— despite being given an extra queen—the machine beat a young woman who had just learned the game. It was the first time a human had lost to a computer in a game of intellectual skill.
The next advances came in the form of smarter programming, in which the developers “taught” the computer how to avoid wasting time considering inferior moves. The mathematical “alpha-beta” chess algorithm was developed, which allowed the program to rapidly prune out weak moves and see further ahead. This is a brute-force method, now in universal use, in which the program stops evaluating any move that returns an evaluation score inferior to the score of the current first-move choice. The first programs that used this method, running on some of the fastest computers of the day, reached a respectable playing strength. By the 1970s, early personal computers could defeat most amateurs.
The next leap came from the famous Bell Laboratories. Ken Thompson, creator of the UNIX operating system, built a special-purpose chess machine with hundreds of chips. His machine, which he named Belle, was able to search about 180,000 positions per second. Supercomputers at the time could only manage 5,000. Seeing up to nine moves ahead during a game, Belle could play at the level of a human master and far better than any other chess machine. It won just about every computer chess event from 1980 to 1983, before it was finally surpassed by giant Cray supercomputers.
As the technology developed, new consumer chess programs with names such as Sargon, Chessmaster, and Fritz benefited, in particular from the faster processors engineered by Intel. Dedicated machines made a comeback thanks to a generation of chess machines designed at Carnegie Mellon University. Professor Hans Berliner was a computer scientist as well as a world champion at correspondence chess, an ancient form of playing the game through the mail. His machine, HiTech, was later surpassed by the creations of his graduate students Murray Campbell and Feng-hsuing Hsu. They took their computer champion, Deep Thought, and joined IBM, where their project was rechristened Deep Blue.
The Deep Blue machine that I faced in matches in 1996 and 1997— more on them in a moment—consisted of an IBM SP2 server equipped with a large number of special chess chips. This combination was capable of searching 200 million positions per second. Like all modern chess machines, Deep Blue also had access to a vast database of preprogrammed opening positions culled from human Grandmaster play. Containing millions of positions, these opening databases imitate and of course surpass human knowledge and memory of the openings. By accessing these databases of moves, a program will play well over a dozen moves according to a preset routine before it begins to compute for the first time. Without the benefit of this human knowledge in the openings, the programs would be considerably weaker.
Some databases are drawn into service only at the end of the game. These “endgame tablebases,” another creation of Ken Thompson, record every possible position with six or fewer pieces (some sets with seven now exist) and their most efficient solutions. With the aid of these oracles we have discovered positions that require over two hundred accurate moves to force a win, a level of complexity previously undreamt of—and still impossible for any human to master.
Fortunately, the two ends—opening research and endgame databases—will never meet, so it is highly unlikely that anyone will ever see a computer play its first move 1.e4 and announce checkmate in 33,520 moves.
My first experiences with computers were far more pleasant than the more famous encounters we’ll come to in a moment. In 1985 I was twenty-two years old and the recently crowned world chess champion. One of my new perks was an early personal computer, one of the few in my hometown of Baku. One couldn’t do a great deal with it as I recall, but it fascinated me just the same. One day I received a package in the mail from a stranger named Frederic Friedel, a chess fan and science writer based in Hamburg, Germany. He sent me an admiring note and a floppy disk containing several computer games, including one called Hopper.
Video games weren’t yet the phenomenon they had become in the United States, and I enthusiastically took up this new challenge. I spent much of my free time over the next few weeks practicing Hopper and setting ever higher record scores.
A few months later I traveled to Hamburg for a chess event, and I made sure to look up Mr. Friedel at his suburban home. I met his wife and two young sons, Martin, age ten, and Tommy, age three. They made me feel quite at home, and Frederic was eager to show me the latest developments on his own computer. I mentioned that I had completely mastered one of the little games he had sent me.
“You know, I’m the best Hopper player in Baku,” I said, omitting any mention of the total lack of competition.
“What’s your high score?” he asked.
“Sixteen thousand,” I replied, a little surprised that this extraordinary number failed to elicit at least a raised eyebrow.
“Very impressive, but that’s not such a big score in this house.”
“What? You can beat it?” I asked.
“No, not me.”
“Ah, okay, Martin must be the video game whiz.”
“No, not Martin.”
With a sinking feeling I realized the smile on Frederic’s face meant that the household Hopper champion was the three-year-old. I was incredulous. “You can’t mean Tommy!” My fears were confirmed when Frederic led his little boy over to the computer and sat him down next to us as the familiar game loaded. Since I was the guest, they let me go first, and I rose to the occasion with a personal best of nineteen thousand points.
My success was short-lived, however, as Tommy took his turn. His little fingers were a blur, and before long the score read twenty thousand, then thirty thousand. I figured I should concede defeat before we sat there watching through dinnertime. My cause was clearly hopeless.
Losing to a little kid at Hopper was easier on my ego than any loss to Karpov, but it still gave me food for thought. How was my country going to compete with a generation of little computer geniuses being raised in the West? Here I was, one of the few people in an entire Soviet city with a computer, handily outperformed by a German toddler. And what about the implications for chess? What if we could improve the way we studied chess the way we used our PCs to write letters and store records? This would be a powerful weapon, one that I shouldn’t be the last to have.
But my first opportunity to employ what I learned from this lesson wasn’t related to chess. When I signed a sponsorship deal with the computer company Atari, I took as payment over a hundred of their machines to bring back to a youth club in Moscow, the first of its kind in the Soviet Union. We couldn’t be left in the Stone Age while Tommy and his nimble-fingered compatriots took over the world.
I had also had the chance to address the other issue with Frederic— how a home computer could be turned into a chess tool. Our conversations led to the creation of the first version of ChessBase, a name now synonymous with professional chess software thanks to the company of the same name that Frederic cofounded in Hamburg. ChessBase was the result of embracing innovation and of being alert to the trends and the possibilities. (And while Martin and Tommy have so far failed to take over the world, both are successful computer design and programming professionals.)
My six-game matches against the IBM supercomputer Deep Blue in 1996 and 1997 received unprecedented attention around the world for a chess event. The official Web site of the 1997 rematch generated Web traffic similar to that of the Atlanta Olympic Games. Newsweek ran a cover story titled THE BRAIN’S LAST STAND, and a thousand subplots were developed. Was Deep Blue really artificial intelligence? Was I the defender of humanity? When it was all over, people debated the implications of my initial win in Philadelphia, my loss in New York a year later, and IBM’s refusal to play a third, deciding match.
For those who may not remember how the whole thing played out, here’s a brief summary. After my first match with Deep Blue in 1996 in Philadelphia received so much publicity, IBM threw its full weight behind a six-game rematch in May 1997 to be held in downtown Manhattan. In 1996, I lost the first game of the match, but after that wake-up call I came back to win three games and easily defeat the computer 4–2. For the rematch IBM claimed “Deeper Blue” was twice as fast and much smarter. They had hired human Grandmasters to work full-time “teaching” the computer by improving its evaluation function.
The media attention for the first game of the rematch was beyond that of a world championship. Over three hundred journalists were accredited, and the five-hundred-seat auditorium sold out at $25 per ticket. Of course I was nervous. Being human, I was unable to ignore all of these distractions, unlike my silicon opponent. But I played well enough to win the first game with white. The second game would, however, change the course of the match, and of human-machine competition.
I’ve talked about computers and their inability to make long-term plans. They calculate variations in a linear way, examining each move in turn, searching deeper and deeper. Even at Deep Blue’s 200 million moves per second, it took it a long time to see far enough ahead to play competent strategic chess. It would occasionally make silly moves no strong human would consider, and it did this in its loss in game one. Game two was an entirely different matter. The machine played with the subtlety of a Karpov, especially at one key moment when I was desperately hoping to gain counterattacking chances. I attempted to sacrifice material for activity, but uncharacteristically for a machine, Deep Blue declined to win material. Instead it played a quiet prophylactic move that ended my hopes, the sort of move no computer had ever before made. Instead of going for a short-term advantage, it closed in for the kill. Faced with a losing position and stunned by the godlike quality of the machine’s play, I resigned.
I soon received an even greater shock. It turned out that the final position of the second game was not a losing one for me after all. With its last move the computer had blundered, and I had resigned in a drawn position! It felt like being kicked in the stomach after already being knocked down. When I was shown the drawing line, I realized that I would have continued on against any human. But during the game I couldn’t imagine the machine making such a mistake, and I had assumed my position was hopeless. I had completely psyched myself out. My feelings of embarrassment and anger were quickly joined by doubt and suspicion. How could Deep Blue play so fantastically well and then in the same game make such an elementary (for a computer) blunder? My mind began to reel with thoughts about just how far IBM might go to win. Would not billions of dollars in “free favorable publicity”—IBM’s words—be worth giving the machine a little human help at a key moment?
Always one to speak my mind, I suggested this possibility in the press conference after the drawn third game. I should have considered that the uproar that followed would only heighten the tension, something that of course had no effect on my opponent. I missed a clear win in the fourth game, and by then I was exhausted and confused. Was something fishy going on, or was Deep Blue really so strong? In the fifth game I again missed good winning chances, so the match remained tied with one win apiece and three draws. Everything was set for a showdown in game six, but looking back today, I see my fate had already been sealed.
It took only nineteen moves for me to resign the worst game of my career and lose the match. I was simply in no condition to play chess by that point, and I made an infantile blunder in the opening. After a few more feeble moves the game and the match were over. I was ashamed of my performance and for letting the mysterious second game get to me.
Worse than the loss of the final decisive game was IBM’s blow to the scientific and chess communities by deciding to immediately shut down the Deep Blue project. For half a century chess had been considered a unique field for the comparison of the human and machine minds, of intuition versus calculation. To this day the six games I played with the multimillion-dollar machine are the only ones ever made public. It was as if they had gone to the moon and not taken pictures.
The tragedy of IBM’s hurried dismantling of Deep Blue overshadowed their disappointingly questionable behavior during the match. IBM was not only my opponent at the board in the 1997 rematch, but also the organizer of the event. There was so much antagonism, with so many unanswered questions about what was going on behind the scenes, that it was hard to avoid coming up with conspiracy theories. I don’t have any proof of foul play, but I feel they didn’t prove much either thanks to their decision to terminate their creation. I live in doubt.
Before I am accused of being a sore loser, I will plead guilty to the charge. I hate losing, especially when I don’t understand the reason for the loss. When we analyze those six games today, we find that on the whole Deep Blue was inferior to today’s PC programs. Only in a few key moments did the IBM computer play extraordinarily subtle moves, moves that even today make one question how they emerged from the same machine that lost game one and blundered at the end of game two.
We have discussed the importance of preparation, and this was another illustration of the incomplete nature of this famous chapter in the man-machine saga. Going into the match, Deep Blue was a complete unknown to me, a black box literally and figuratively. But they could, and did, analyze every one of my games and customize Deep Blue’s play to exploit that advantage.
The closed nature of the contest created the potential for human interference, although in the pre-Enron era it sounded like paranoid folly to suggest that a corporate giant might resort to subterfuge to gain publicity and a huge surge in its stock price. Despite these remaining sour feelings, I was amazed at the enormous appeal the match clearly had for the general public. I knew I wanted to continue the adventure, although in the future the environment would need to be much more open and scientific.
If You Can’t Beat ’Em, Join ’Em
My enthusiasm for finding new ways to use computer technology to promote the game of chess did not disappear when IBM pulled the plug on Deep Blue. In 1998 I turned to a new experiment dedicated to enabling humans to fight along with machines instead of against them.
Grandmasters play chess by combining experience with intuition, backed up with calculation and study. Computers play chess by brute calculation; their “study” consists of a gigantic database of opening moves. At present there is a rough equilibrium between these methods; the best computers play at around the same strength as the best humans. As microprocessors have got faster, humans have learned new tricks to expose the weaknesses of computer play. Inevitably the machines must win, but there is still a long way to go before a human on his or her best day is unable to defeat the best computer.
The concept of Advanced Chess illustrates the costs and benefits of human + computer collaboration. I developed this game as a way of answering the elusive but fascinating question, what would a combination of human intuition and computer calculation produce on the chessboard? Would they combine into an invincible centaur or an uncoordinated Frankenstein’s monster? In June 1998, with two powerful computers at their sides, two Grandmasters, Veselin Topalov and I, faced off across the board in the first match of its kind.
Although I had prepared for the unusual format, our six-game match was full of strange sensations. We all use computer programs in our analysis and training, so we know what they are capable of and what their weaknesses are. But having one available during play was as disturbing as it was exciting. Being able to access a database of a few million games meant we didn’t have to strain our memories nearly as much in the opening. But since we both had equal access to the same database, the advantage still came down to creating a new move at some point, and making sure it was better than what had been played before.
In the middle game, having a computer running meant never having to worry about making a tactical blunder. We could concentrate more on deep planning instead of the precise calculations that take up so much of our time in regular games. Again, since we were both using computers, it was a matter of how well we used them to check our plans and whose plan was more effective. As when I played against Deep Blue, there would be no way back if I made an error. The machine would not forgive any mistakes by making one of its own in return.
It was difficult to find the best way to utilize the machine’s abilities. I felt I was in a race to check the validity of the computer’s evaluation. It gives its opinion instantly, but its recommendation changes as its analysis goes deeper and deeper. Just as a good Formula One driver really knows his own car, so did we have to learn the way the computer program worked. There is a strong impulse to unquestioningly follow the machine’s evaluation if the move it recommends looks like something the computer would usually play well. That’s a dangerous habit.
Despite the human + machine formula, my games with Topalov were far from perfect, mostly due to the unforgiving clock and the intense time pressure we were under. Toward the end, we had no time on our clocks to consult the machines for more than a few seconds. Putting that flaw aside, the results were interesting. Just a month earlier I had defeated the Bulgarian 4–0 in a match of regular rapid chess. Our Advanced Chess match finished in a 3–3 draw.
An important benefit of Advanced Chess is that the computer creates a log of every variation the players examined during play. This leaves a diary of the players’ thoughts throughout the game, which is both fascinating for online spectators and immensely valuable as a training tool. Normally it is forbidden to take any notes during a game, but in Advanced Chess we provide a complete map of the path the game took through the players’ minds.
The experiment continued in León in later years with other players, and in 2005 the ethos of Advanced Chess found its true home on the Internet. The online site Playchess.com hosted what they called a “freestyle” chess tournament in which anyone could compete in teams with other players or computers—whichever they prefer. Lured by the substantial prize money, groups of strong Grandmasters working with several computers at the same time entered the competition.
At first, the results seemed predictable. The teams of human plus machine totally dominated even the strongest computers. The mighty chess machine Hydra, which is hardware-based like Deep Blue, was no match for a strong human using a relatively weak laptop. Human strategic guidance combined with the tactical acuity of a computer was invincible.
The surprise came at the conclusion of the event. The winner was revealed to be not a Grandmaster with a souped-up machine, but a pair of amateur American chess players using three computers at the same time. Their skill at manipulating and “coaching” their computers to look deeply into positions effectively counteracted the superior understanding of their Grandmaster opponents. Weak human + machine + superior process was greater than a strong computer and, remarkably, greater than a strong human + machine with an inferior process.
The “freestyle” winners had taken advantage of superior coordination of their contrasting methods and strengths. They understood their tools—human and machine—and figured out how best to get the most from them. A manager might say they built an effective team from a group of individuals with disparate skill sets. An army commander would recognize that a well-coordinated force will almost always triumph over a numerically superior enemy who lacks organization. A company with an efficient management structure, or assembly line, will often have better margins than a larger, less agile competitor. Process is critical, especially since its benefits multiply with each cycle.
Staying Out of the Comfort Zone
Opposite pairs working in harmony: this has become a theme of our quest to perfect decision-making. Calculation and evaluation. Patience and opportunism, intuition and analysis, style and objectivity. At the performance level these elements come together in management and vision, strategy and tactics, planning and reaction. Success comes from balancing these forces and harnessing their inherent power.
And as we’ve seen again and again in this book, the only consistent method for achieving such a balance is to constantly seek to avoid our comfort zone. It’s a bad habit to become overreliant on one skill or way of doing things just because it has in the past worked well for you. It’s better to throw yourself off-balance, as Topalov and I did in that first game of Advanced Chess. One of the lessons I took away from that match I think about almost every day: the one time you are surely learning something is when you are nervously attempting something new, even if it is simply solving a routine problem in a novel way. If you want an illustration of how deeply you are set in your routines, try brushing your teeth left-handed, or putting on your trousers left leg first. Our mental routines are no less ingrained—and they have more profound consequences.
Engaging with the weakest points in our game and drilling down so we really understand them is the best and fastest way to improve. Working to become a universal player—someone who can defend as well as attack and is at home in any type of position—may not always have an obvious immediate benefit, especially if you are in a specialized field. But in my experience working toward a universal style creates a rising tide that lifts all boats. Gaining experience in one area improves our overall abilities in unexpected, often inexplicable ways.
I was lucky in that I was virtually forced by Anatoly Karpov to become a more positional, strategic player. It was sink or swim for me: either I broadened my style and my understanding or I wouldn’t be able to beat him. The situation is not so clear for most people. We can go through our day-to-day lives without changing our habits and nothing terrible will happen to us. The problem is that it is also highly unlikely anything at all will happen to us—including good things. Successfully avoiding challenges is not an accomplishment to be proud of.
When I was in the fifth grade, the greatest mystery that school held for me was drawing. It seemed like an occult science; I just couldn’t do it, and to this day I’m lousy at it. Instead of working at drawing as I did my other subjects, I—cleverly I thought—convinced my mother to do my drawing homework. She was quite good; in fact, she was good enough to catch the attention of my teacher with a fine picture she made of a bird in a tree. I could no sooner have drawn that myself than I could have painted the Mona Lisa. My teacher asked me if I would be interested in entering a drawing competition, in which I would have to perform in front of judges, not at home. If you think this is the end of the story, you haven’t realized how proud and competitive I was even then.
Instead of confessing, I spent the next few weeks at home training myself to draw the picture of the bird exactly as my mother had. I spent hours on it, reproducing it line by line as if memorizing a chemical formula. Eventually I could manage a quite reasonable facsimile of the bird. Sweating nervously at the competition, I produced a creature that was almost identical to my mother’s original. I have no doubt that that bird was and is the only thing I could draw in the world.
Of course now I wish I had done my drawing homework myself and actually learned to appreciate and cultivate the skill it requires. But even if I can’t draw a picture, I did benefit from the lesson that experience offered. I stepped out of my comfort zone and pushed a bit at the boundaries of what I thought I could do. And it wasn’t such a bad bird, after all.
It has long been fashionable to talk about left-brain and right-brain activities, even left-brained and right-brained people. But it doesn’t require a discussion of brain science to understand how indulging our creative side and letting our minds wander in artistic pursuits can be enormously helpful in breaking us out of our problem-solving routines.
The great physicist Richard Feynman offers an inspiring example of a brilliant man who pushed the boundaries and refused to be defined by his achievements in one particular area. When Robert Oppenheimer was in charge of the Manhattan Project, which produced the atomic bomb, he described Feynman as “the most brilliant young physicist here.” But he was also the greatest troublemaker. He saw everything as a challenge, as a puzzle to be solved. Feynman enjoyed picking the locks in the top-secret offices of Los Alamos just to see if he could. He became a serious amateur painter and musician and loved to perform as a drummer at Brazilian carnival celebrations.
I have no doubt that Feynman’s free spirit and playful mind were assets to his scientific work, not detriments. In his popular books he insisted that science was a living subject, not just a cold set of formulas. He excelled in combining techniques and transforming a difficult problem into a comparable one that was easier to solve. This skill was directly related to his inclination to stay open to new ideas in every aspect of his life.
Today our society places great emphasis on specialization and focus. Students used to go off to university with the idea of broadening themselves; now it has become a mostly vocational experience. Students use higher education as a means to develop a skill that will make them attractive to employers. We place so much emphasis on being good at what we do that we fail to realize that getting better at what we do might be best achieved by getting better at other—and wildly different— things.
It sounds strange to say that being a better artist might make me a stronger chess player or that listening to classical music can make you a more effective manager. And yet this is exactly the sort of thing that Feynman had in mind when he said that being a drummer made him a better physicist. When we regularly challenge ourselves with something new—even something not obviously related to our immediate goals—we build cognitive and emotional “muscles” that make us more effective in every way. If we can overcome our fear of speaking in public, or of submitting a poem to a magazine, or learning a new language, confidence will flow into every area of our lives. Don’t get so caught up in “what I do” that you stop being a curious human being. Your greatest strength is the ability to absorb and synthesize patterns, methods, and information. Intentionally inhibiting that ability by focusing too narrowly is not only a crime, but one with few rewards.
My relationship with computers over the years has been contentious, but I readily acknowledge they have had a major impact on the way I think. Both playing against them and using them as an analytical tool forced me to recognize flaws in my decision-making. Like any tool, computers extend our reach and present us with new ways of solving old problems. They also present us with an entirely new set of problems, but this can be a blessing in disguise. Solving new problems is what keeps us moving forward as individuals and as a society, so don’t back down.