In 1922, the reigning world chess champion, the Cuban José Raúl Capablanca, simultaneously took on 103 opponents in Cleveland, Ohio. After seven hours, he had won 102 games and drawn one. But with so little time to calculate, how could this be possible?
It is all too natural to imagine that Capablanca must have been playing lightning chess with a lightning brain; that he must have been able to race through the forking paths of move and counter-move faster than his bemused opponents. If this was indeed his secret, then it would imply that Capablanca could calculate a hundred times faster than his hundred opponents, just to get a hundred draws (after all, Capablanca has just one hundredth of the time each of them has to think through the next move). Not only that, Capablanca – unlike his opponents – would have to spend a great deal of time moving from board to board (whereas his opponents would be able to focus relentlessly on a single game). Over the entire seven-hour period, he made, on average, about ten moves per minute: this pace is consistent with sidling up to a board, taking a quick look, making a move, and then shifting onto the next game And, of course, if Capablanca’s forte was lightning calculation, then he would need not just to match the calculation of his opponents, but to exceed it very considerably – after all, his opposition was comprehensively flattened. So, in short, one might suspect that, if Capablanca’s magic trick was lightning-fast calculation, then he must have been able to calculate several hundred times faster than anyone else – a sort of human supercomputer.
Now this story would, indeed, be a pretty good model for computer chess. Of course, the chess-playing ability of a computer program depends a great deal on how cleverly the program is designed. But a decisive factor in the dramatic rise in the quality of computer chess over the past few decades has been the equally dramatic increase in the raw speed of computer processing. Contemporary computer chess programs really do calculate at lightning speed – evaluating many millions of possible board positions per second. So a super-fast computer chess program could, indeed, simultaneously crush one hundred more sluggish chess programs, if it could calculate, say, 500 or 1,000 times faster. The super-fast computer would simply be able to look further into the forking tree of possible moves and counter-moves that its ponderous opponents could make.
Yet Capablanca did not have the brain of a lightning calculating machine – and he did not need it. A glance at the board would instead call up past games, and past good (and bad) moves – one clue is that experimental tests show that chess grandmasters have a phenomenal ability to remember chess positions from real chess games of the past. After five seconds, a top chess player is able to ‘read’ the structure of a chess position – searching out which pieces are threatening which others, noting familiar patterns of pieces (e.g. a castled king and rook behind a row of pawns; advanced central pawns defended by knights, and so on), finding, in short, what the position means. And this is usually enough, not just to decide on a high-quality next move, but to commit the position, aside from a few incidental details, completely to memory – indeed, a grandmaster may be able to recall that very position many minutes or even hours later. To the inexpert chess player, this astonishing feat suggests that grandmasters supplement fantastic calculating ability with stupendous powers of memory. But the feat is no more remarkable than our ability to commit to memory an enormously long string of letters, simply because they are arranged meaningfully, as in this sentence. Someone unfamiliar with the English language, or even the Roman alphabet, would find this ability equally astounding – precisely because they could impose no meaning upon it (consider your own ability to remember strings of characters in a distant language in an unfamiliar script). Or consider a skilled musician’s ability to reproduce a lengthy stream of musical notation, because it can be turned into a meaningful tune (while, to the rest of us, the pattern of notes on a stave is mere gobbledygook). In each case, memory is the by-product of understanding: what we cannot interpret, we cannot remember.
It seems, then, that grandmasters are special not because of their unusual mental powers, but because, through long experience, they have learned to find meaning in chess positions with particular fluency; and they can do this because they can link the current board position with their memory traces of past board positions, acquired through thousands of hours of chess-playing.
Two further observations reinforce this picture. The first concerns the nature of the memory mistakes that grandmasters make. They recall, more or less unfailingly, the pieces that matter to the progress of the game, but where some peripheral piece plays no active role its precise location need not be encoded exactly. By contrast, non-chess players make mistakes of all and every kind – to them, of course, the board is a mere jumble of pieces, not a subtly interlocking set of threats, counter-threats and defences.
The second observation is that grandmasters are no better than the rest of us at remembering random board positions – their superior memory skills evaporate as soon as they are faced with arbitrary chess positions, because they can impose no meaningful interpretation on these positions in relation to their huge repertoire of past chess experience.1 In the same way, readers of English will struggle to learn random sequences of letters, just as expert musicians will have no special ability to recall arbitrary forests of musical notation.
The ability to make sense of current chess positions, by linking them with the vast library of past positions, greatly simplifies the problem of choosing the right move (just as familiarity with English makes continuing an English sentence relatively straightforward, and musical expertise makes it possible to write out a plausible next bar or two of a simple tune). In chess, of course, the number of possible continuations explodes as we attempt to see further ahead in the game – but almost all of these moves are wildly implausible and can safely be ignored.
Grandmasters do not reach such impressive levels of performance by out-calculating their bemused opponents and seeing many more moves ahead. Instead, they look only a little further ahead than amateur chess-players, but their memory bank of past experience and, in particular, their library of meaningful analyses of chess positions, allows them to focus on only the best moves, and to ignore the rest.
Notice, too, that Capablanca had no ‘theory’ of chess, beyond his vast fund of experience. He wrote a number of celebrated chess books, one promisingly entitled Chess Fundamentals, with lists of ‘principles’ to guide the aspiring player.2 Yet these principles are, in reality, a series of helpful examples illustrating useful rules of thumb – there is no chess equivalent of Newton’s Laws! Tellingly, Capablanca is attempting to distil some of his knowledge not into principles, but into particularly helpful illustrations.
To learn to play chess is to learn to impose meaning on each board; and each board makes more sense in the light of the meaning imposed on previous boards. Perhaps expertise in any domain, however remarkable, is not based on superior mental calculating power, but on richer and deeper experience: Capablanca could play new chess positions because he had a vast library of precedents from imposing meaning on prior chess positions – and could use those precedents more creatively and effectively than anyone else. And perhaps that is how skills, learning, memory and knowledge always work. We layer each momentary thought on top of past momentary thoughts, tracing an ever-richer web of connections across our mental surface.
The interpretation of each new chess position depends on a vast battery of interpretations of prior chess positions; and, in the same way, the interpretation of each everyday scene depends on a vast hoard of past interpretations of everyday scenes. Indeed, perception works by relating, often in the most flexible and creative fashion, our sensory input with our memory of past experience. We do not interpret every sensory impression afresh, but in terms of the memory traces of past sensory impressions. Consider the rather delightful ‘found faces’ in Figure 37. In terms of superficial shapes and colours, these items – variously a leather bag, cheese-grater, a block of wood and the detail of a wash basin – are about as distant as possible from human form. Yet we see them, more or less immediately, not merely as faces, but as faces with personalities, expressions and even a measure of pathos that seems particularly incongruous given that each of these objects is patently inanimate.
The cycle of thought should, then, create an organization of the sensory input that depends not just on the input itself, but on ‘resonance’ with memory traces of past inputs (Figure 38) – for example, previous faces that we have encountered. The brain interprets one word, face or pattern at a time, but in doing so it simultaneously explores possible links between the current stimulus and a vast array of memories of interpretations of past stimuli. ‘Resonances’ between current and past stimuli are not defined by superficial similarity – otherwise the brain could only interpret a cheese-grater in terms of past silver metallic box-like objects, and never as a face. Yet the brain rapidly sees the ‘eyes and mouth’ pattern in this otherwise incredibly un-face-like object. The ‘found faces’ of Figure 37 are illustrations of just how flexibly the brain can map past memory traces (of human faces) to impose meaning on the current input.
The resonance between perception and memory must occur ‘in parallel’, however. Given our sluggish neurons, matching the current perceptual input with each of our vast repertoire of memory traces one at a time would be unfeasibly slow. And, when interpreting a new stimulus, the brain may have little idea which memories it needs to search – indeed, it seems able to draw on its entire stock of memory traces equally easily. Before the interpretation has been made, the brain can’t know which memories will be relevant – so it has to search them all.4
Notice that, from this viewpoint, each new perceptual interpretation is based on memories of past interpretations. We never see the world ‘with fresh eyes’. Each new interpretation is an amalgam and transformation of past interpretations. Consider what happens when you read a word, or ‘read’ a face, or a chessboard – those perceptual interpretations depend on years of past experience of our language and writing system, of our long history of interactions with other people, and the nature of past experience (if any) with the game of chess. As usual, of course, we have awareness only of the output of thought – the result of our current interpretation; all other mental processes leading to that output are never conscious. Thus we have no awareness of the memories activated, or how they were transformed and combined to interpret the current stimulus. Listening to our own native language, we ‘hear’ the speech sounds, words and pauses as if they are plainly observable aspects of the speech signal. But listening to an entirely unfamiliar language, we are confronted with a baffling, uncategorized and apparently chaotic flow of sound. The difference is that, for our own language, we can map new streams of sounds to a vast range of previously interpreted speech sounds, words, phrases and more. We can interpret new speech input in terms of a vast repertoire of memory traces from our past interpretations of past speech input. As with learning any other skill, we break down the code of our language piece by piece, over many months and years; and, as with any other skill, we are aware only of the results of our current competence, oblivious to the myriad of memory traces on which that competence depends.5
What are memory traces like? What information do they contain? The most natural answer is that they are nothing more than the remnants of past interpretations of past perceptual inputs. As far as we know, these remnants are not later reorganized, filtered, corrected or generally tidied up; and there is no internal librarian to carefully file and index each memory trace into a coherent archive. The remnants of each individual episode of perceptual processing lie, as it were, where they fall; the brain is immediately busy with the next cycle of thought, and the next.
The brain is therefore not a theorist trying to distil deep abstract principles from experience – it is, instead, focused on coping with the present, as far as possible, by relating the present to amalgams and transformations of the past. According to this viewpoint, then, memory traces are fragments of past processing – so that it is past interpretations that are stored in memory, rather than raw, disorganized sensory input. So, for example, if we see a cheese-grater as a face in one situation, then that interpretation will be stored in memory. When we next encounter a similar cheese-grater, we are more likely to see that grater, too, as a smiley face – we remember the interpretation.
Conversely, uninterpreted aspects of the sensory world will be forgotten: a piece of handwriting that is too difficult for us to read, a fragment of a language we don’t understand, or the outlines of a distant figure in the trees that we fail to notice, will not, by this account, be filed away in our memory to be subject to future analysis, or to shape later perceptions. They will be lost for ever. (In passing, note that this is reassuring news for those worried about subliminal messages that might be planted by advertisers or nefarious forces wishing to control our minds by stealth.)
Perception and memory are therefore intricately entwined. Recognizing a friend, a word or a tune requires not merely linking together different aspects of the perceptual input, but connecting these fragments to stored memories of faces, words and melodies. So, for example, a recognized face typically does not merely feel vaguely familiar: we can also access information about the corresponding person. The string of letters making up a word is typically also linked to its meaning, its sound, and much more; and recognizing a tune may conjure up the associated lyrics, singer, era in which we first heard it, and more. So the interpretation of the information flowing through our senses depends on a huge body of remembered information, but this information is, of course, nothing more than the memory of past interpretations of previous sensory information. Today’s memories are yesterday’s perceptual interpretations.
Successful perception, then, requires almost instant conjuring up, and deploying, of the ‘right’ memory traces to make sense of the current sensory input, just when we need them. This is remarkable in the light of the sheer number of such traces, accumulated over a lifetime. It is more remarkable still, given that the right memories can be very indirectly related to the perceptual fragments: that images ostensibly of handbags, cheese-graters, blocks of wood or hand basins can trigger memories of faces (see Figure 37); that a few scratches of ink on a sheet of paper can conjure the human figure in a landscape (Figure 39b); that the rearrangement of the same few geometric shapes can remind us of a rocket, a kneeling person, a rabbit and many other images (Figure 39a). And the same spectacular flexibility of thought arises of course throughout the metaphors that pervade our language and our thoughts – memories of one thing are fluidly and naturally linked to our memories of another thing. So, for example, we can ‘see’ our boss variously as a conductor, a general, a robot or a shark.
A helpful way to view how perceptual or memory information is analysed is to regard the given information (the clues from memory and perception) as providing parts of a pattern, with the brain’s task being to ‘fill in’ the gaps in the pattern. But this image underplays the spectacular flexibility of the brain in transferring knowledge from one domain to some apparently entirely unrelated topic. So, for example, taking some fragments of visual information about a handbag, the brain may indeed fill in various other details (inferring, perhaps, that the sealed container – the handbag – has an opening only at the top, or that it comes from a specific period or country, or that it has a certain monetary value), but equally the brain may well interpret it as an alarming, if slightly comical, snarl. The brain operates by the routine exercise of an astonishingly exuberant imagination.
Very much the same story applies when we consider consulting not the contents of the visual world (handbags, cheese-graters, actual human faces), but our own memories. We can have the feeling of immediate and simultaneous access to a vast repository of general knowledge, autobiography, likes and dislikes, moral and religious convictions, and more. But in truth we impose meaning on one set of memory traces at a time. And when we do lock onto those memory traces, we impose meaning on them with the same flexibility and urgency that applies in perception. The memories themselves are not thoughts: they are not, for example, beliefs, choices or preferences. And we cannot merely ‘read off’ their contents to know what we think, what we like, or what sort of person we are. Instead, they consist of mere fragments of past thoughts, to be reused, reconstituted and transformed by the cycle of thought.
What makes me, me? What makes you, you? Tempted, as ever, to peer into the darkness of our mental depths, we search for the attributes of character: bravery, stoicism, anxiousness, kindness or cruelty. What are we really like, deep down? It seems hard to be sure of our own nature – each of us is, of course, a mixture of momentary thoughts and feelings, at one moment brave, at another timorous; sometimes stoical, at other times anxious. But it is a seductive idea that our confusions and contradictions are merely the stuff of our turbulent surface, the vagaries of the moment pushing us here and there. Deep down, perhaps too deep for us to fathom, lies our inner self, replete with the virtues and vices of our true character. But mental depth is, we have seen, an illusion; there is no inner core, virtuous or venal.
Yet if the mind is an engine of precedent, continually reshaping past thoughts and actions to deal with the present, then each of us is not just a bundle of character traits, but a rich store of distinctive past experience: we are like corals layered, polyp by polyp, into infinitely diverse forms. What makes each of us unique is our individual and particular history – our own specific trail of precedents in thought and action. We are each unique, in short, because of the endless variety of our layered history of thoughts and actions.
This viewpoint might seem to imply that we are nothing more than ‘creatures of habit’. Not at all. As we shall see in the next and final chapter, it is our remarkable ability to make imaginative leaps, both large and small, that breaks us free of blind repetition. We can project our experience of everyday human faces onto the ‘found faces’ in handbags and sinks (remember Figure 37). The subtle and measured application of past precedents, not merely the sheer number of chess-playing hours, surely distinguishes the brilliance of great champions like José Raúl Capablanca, Bobby Fischer or Magnus Carlsen from the everyday run of Masters and Grandmasters. Or consider our ability to improvise dance, create songs, paint and draw, spin stories, or invent imaginary worlds – not from scratch, but rather through the reinterpretation and reconfiguration of the elements of the world we know.
Each of us is a tradition, guided and shaped by our past. Like traditions in music, art, literature, language or the law, we are capable of refinement, adjustment, reinterpretation and whole-sale reinvention. Our mental present is built out of our mental past, but our imagination need not be trapped in cells we constructed long ago; we continually build and rebuild ourselves. Re-routing our minds is always slow and difficult. But where we can purposefully shift our present, there is hope of reshaping our future.
Each turn of the cycle of thought leaves a trace, and that trace may shape future turns in the cycle. Thoughts are like water droplets finding their way from high ground to the sea, following the channels in the landscape, whether gullies, streams or river valleys. And, in its passing, each droplet cut those channels just a little more deeply. The landscape, then, is partly a history of past water flow, as well as a guide for how water will flow in the future. In the same way, our mental life follows channels carved by our previous thoughts, and traces of our present thoughts and actions will shape how we think and act in the future.
Droplets of water make their way downhill by the steepest available path, each heedless of the incalculable number of droplets that have created the landscape through which it runs, and heedless, too, of how its own imperceptible corrosive force may alter, ever so slightly, the path of the next droplet, and the next. Similarly, each cycle of thought lays down memory traces that may smooth, or obstruct, future cycles. Each momentary interpretation draws on and adapts our previous attempts to understand the world. So each cycle of thought maybe viewed as creating, over time, mental channels along which our thoughts most easily flow; and, indeed, each cycle of thought attempts, as far as possible, to make our interpretation of current sensory information cohere with our interpretations of prior sensory information.
Over a lifetime, the flow of thought shapes, and is shaped into, complex patterns: our habits of mind, our mental repertoire. These past patterns of thought, and their traces in memory, underpin our remarkable mental abilities, shape how we behave and make each of us unique. So, in a sense, we do, after all, possess some inner mental landscape. But this is not an inner copy of the outer world or, for that matter, a library of beliefs, motives, hopes or fears; it is, instead, a record of the impact of past cycles of thought – rather than, as it were, any mysterious subterranean geological forces.
The brain operates by precedents not principles. Each new cycle of thought makes sense of the information to which we are currently attending, by reworking and transforming remnants of past related thoughts. And the result of each cycle of thought becomes, itself, raw material for future thoughts.
So the failure of early artificial intelligence to discover the principles that underlie our knowledge of the physical and social worlds; the failure of linguistics to uncover the grammatical principles that generate language; and the failure of philosophy to articulate the principles underlying the true significance of truth, goodness and the nature of mind – all have a common source. The system of precedents underpinning human intelligence can be contradictory, highly flexible and open-ended – especially in relation to cases for which no nearby precedent has yet been set. But this very open-endedness is precisely what is required when dealing with a world that is still far too complex for us fully to understand.7