12
The Secret of Intelligence
In the 1950s, Canadian psychologist Craig Mooney created a remarkable array of black and white images (Figure 40). The faces were created for a rather narrow purpose, as a test of the development of face perception in childhood.1 Yet thinking about these faces, and how our brain understands them, provides an opportunity to encapsulate some of the key themes we have explored in this book – and provides a starting point for investigating the ‘secret’ of intelligence.
Initially, the display is rather mystifying, but spend a minute or two looking at these strange black and white patterns and the experience becomes more interesting. Somewhat miraculously, the initially inchoate patterns start to make sense. To my eye at least, each face creates a vivid sense of a particular person caught in a particular mood, with a specific expression, gender, age, personality, and perhaps even a historical period. Some of these simple arrangements of black and white patches seem to have as much life, pathos and a sense of human drama as the most detailed photographic image. They are also rather beautiful.
Some faces jump out at us almost right away; others remain enigmatic until, mysteriously, the baffling bumps, curves and blobs resolve into a portrait of a human being. You may not be able to see all the patterns as faces – indeed, there are still a few of these images that I can’t make sense of, having looked at them countless times over the years. But if you see even a few, you are still far ahead of any computer vision technology yet invented.
On reflection, finding faces in these incredibly simplified and stylized patterns is an astonishing feat: these strange ‘woodcut’ patterns are so utterly unlike the colourful, three-dimensional, moving people we see around us. Where are the eyes, nose and mouth that we might be expecting our brain to pick up on? These features appear to be wholly absent, at least until we crack the code revealing the entire face. The components of the image are just jumbled bumps, curves and blobs, until, of course, the delightful moment that a human face breaks through to us from the darkness. Once the ‘insight’ occurs, our interpretation of that image is permanently or near-permanently recorded in memory – a jumble has become a face, and will now be a face for ever.3
To make life more difficult still, try turning the page upside down. A few of the faces you initially identified will still make sense. Many, though, will return to seeming no more than abstract patterns. Give yourself time, though; look back at the face the right way up. Gradually, more and more images will snap into coherence.
There are several interesting conclusions to draw from looking at Mooney’s faces, both the right way up and upside down. Seeing the images upside down gives us a sense of how truly chaotic they are – that there are, indeed, no eyes, noses, mouths or ears that can be identified in isolation. When the faces are the right way up, some of these features can, sometimes sharply and sometimes only dimly, be discerned – we see the parts only when we are able to grasp the whole. The holistic way we perceive Mooney faces (i.e. the interdependence of perceiving a whole pattern and its component parts) reflects the general operation of the brain. Reading scrawled handwriting, we recognize the letters in tandem with understanding the whole word. We recognize speech sounds, word-breaks and so on at the same time as we decode the message that a person is attempting to convey (and recall that speech in an unfamiliar language is little more than a blur of sound). And we interpret a small wooden object being slid from a white square to an adjacent black square as advancing a pawn, threatening the opponent’s knight, or leading to checkmate in three moves, by interpreting a physical aspect of the world in terms of the rules of chess. Further, as we saw in Chapter 5, Lev Kuleshov demonstrated how the same facial expression can be interpreted as subtly conveying sorrow, hunger or lust; and Schachter and Singer showed that the way we interpret our own bodily states (rushes of adrenaline, racing heart) as anger or elation depends on our interpretation of the social interaction we are engaged in (e.g. whether the person we are interacting with is being annoying or amusing).
Note, too, that our sense of the entire face is remarkably rich, even if those individual features are difficult to discern; I suspect you feel, as I do, that you might be able to pick out some of these people if they appeared in a police line-up. The leap from a stark, black-and-white image to a three-dimensional real person is, of course, vast – the fact that we can make such remarkable perceptual leaps is a reminder of the astonishing flexibility and, one might even say, creativity of the perceptual system.
The human brain is wonderfully attuned to faces – so that we can detect and reconstruct them on the flimsiest of evidence, as the Mooney faces attest. And, indeed, we can see faces when the evidence is flimsier still. Recall the ‘found faces’ we saw in Figure 37. The snarling handbag at the extreme left looks, to my eye, at once angry and supercilious; the cheese-grater is youthful, eager to please, and perhaps a little anxious; the block of wood has a relaxed but slightly drunken look; while the taps and sink on the extreme right seems preoccupied with its own thoughts, and riddled with anxiety. Yet what is really remarkable about this array of faces is that we can see them at all. Each image is incredibly distant from any real human (or indeed animal) face we have ever encountered. Yet they not only seem face-like, but, like the Mooney faces, they convey emotional expressions and even, to a degree, personalities. The imaginative leap required to distort and stretch these everyday objects onto the templates of a prototypical human face is very great; our brain makes such leaps so rapidly and so naturally that we think of it, if we consider it at all, as entirely unremarkable.
These imaginative jumps are, I believe, at the very core of human intelligence. The ability to select, recombine and modify past precedents to deal with present experience is what allows us to be able to cope with an open-ended world we scarcely understand. The cycle of thought does not merely refer passively to past precedents – we imaginatively create the present using the raw materials of the past.
THE UBIQUITY OF METAPHOR
The wild inventiveness of the human brain is perhaps best exemplified by the ubiquity and centrality of metaphor in our thoughts. Beyond seeing cheese-graters as grinning stupidly and hand basins looking diffident, we are forever seeing one thing in terms of something else completely. We portray each other as bursting with (or just full of) emotion, letting off steam, feeling down or depressed, or on top of the world; being weighed down or buoyed up, feeling flat or bouncy. We can be mixed up, messed up, straightened out, put straight; thoughts can be crowding in on us, we can be teeming with ideas, or our mind can be blank, empty, a void; we can be light-hearted or our thoughts can turn dark or even black. Our ideas can be sharp, penetrating, incisive, sparky, sparkling, fizzing, bright, brilliant, illuminating, or simply dull and blunted; and our words can be cutting, barbed or pointed, or perhaps smooth, silky, silvery or even oily. Or consider our physical condition, which can be in shape, out of shape, full of – or drained of – energy, broken down or patched up, out of gas, or having a second wind. And, of course, the very idea of seeing one thing as another is itself a metaphor, using visual perception as a proxy for thought. Our language is thoroughly soaked in metaphor.
Metaphors also thoroughly permeate our thoughts. Consider the illusion of mental depth that we discussed in Part One. Once we begin to see how thoughts are ‘hidden’ below a mental ‘surface’, it seems only natural to try to ‘uncover’ them, ‘bring them to the surface’; to suspect that some people’s thoughts run deep, while others are shallow. And the very idea that the mind is flat is, of course, just one more metaphor, albeit, I hope, a helpful antidote to our standard repertoire. And, needless to say, antidote and repertoire are metaphors too!4
Just like faces, metaphors are everywhere, when we stop to notice them. And, indeed, it is hard to imagine how we could talk, or think, without them. If you have your doubts, try rewriting a few sentences of this book with the metaphors taken out, stripped away, removed, excised, deleted, cut out, missing, absent or eliminated, or whatever would be the appropriate, completely literal, phrasing, if one exists.
Some metaphors can of course become fossilized in the language (to deploy yet another metaphor), divorced (help, another one!) from any original meaning, and sometimes continue to be used where their origin is entirely forgotten. Presumably, riding a bicycle was once a metaphorical extension of riding a horse, but in a world in which bicycles are plentiful and horses are few, any such link has been lost. Indeed, our entire language (including this very sentence) is a graveyard of dead or half-dead metaphors.
Found faces and metaphors have a lot in common. Like found faces, metaphors embody three characteristics. First, a metaphor requires non-obvious, lateral thinking (another metaphor, of course), to link together two apparently unrelated domains (cheese-graters versus faces; real objects being hidden, buried, brought to the surface, and thoughts being ‘hidden’, ‘buried’, or ‘brought to the surface’). Second, metaphor, by its very nature, requires the transformation of past experience into present experience: between, say, resolving outstanding issues in a project and ‘tying up loose ends’ of string or rope. We can only see an unthreatening shark as a teddy bear if we have some experience or knowledge of teddy bears. And we can only see a person as a shark if we have some experience of, or at least some prior knowledge of, sharks. And, third, metaphors are just as apt to mislead as they are to inform. Just as a cheese-grater doesn’t really feel eager to please and a block of wood can’t get slightly inebriated, our inventive metaphorical leaps can be wildly off-target. For example, if the intricate mechanism of Nature is like the workings of a watch, as the eighteenth-century clergyman and theologian William Paley famously suggested, then it is easy to leap to the conclusion that it must have a designer, and one far more skilled and intelligent than any watchmaker. Evolutionary biology, of course, tells a very different story.
Battles of ideas are often fought over which metaphor is appropriate: is light made up of particles or waves? Are humans ‘risen’ apes or fallen gods? Is nature a harmonious society or a brutal war of all against all? Such metaphors are not marginal to thought, but its very essence. Our continual search for meaning is the struggle to find patterns in our present experience, in the light of the past. And so we see one thing in terms of another: a wash basin as a face; a mind as a container, a sea, or an inner world. Metaphors, too, are also employed to impose meaning upon one aspect of the world, by drawing on an understanding of another. We have lots of everyday experience with water waves (whether in puddles, ponds or the sea); and this helps us understand what it might mean for sound, light or even gravitation to be a type of wave (interfering, refracting, diffracting and so on). Or we can take our intuitions about water flow to help us understand the flow of heat or electricity.
Or, to consider a final, more whimsical, example, consider the game of charades. The title of a book, song or film is whispered in your ear; and you must communicate its identity to other members of your team with a sequence of improvised gestures and movements. One moment we may be transformed into a ravening dog for The Hound of the Baskervilles; the next we may desperately attempt to indicate the form of a rose flower, closing into mere bud, or, failing that, to stagger about with an imaginary stick, in an attempt to convey Citizen Kane. What is astonishing about charades is that we can play it at all: that a person can take on enough of the movement and gnashing teeth of a dog to convey the famous Hound; that a mime with the hands can imply the stalks, thorns, flower and even bud of a rose; and that we can walk as if supported by a stick, when the stick is patently absent. And we can do this naturally and spontaneously, and without years of dedicated preparation or rehearsal.
The team-mates attempting to decode our movements face a challenge somewhat akin to seeing found faces: they need to find some aspects of the communicator’s demeanour and actions which suggest, however remotely, the target idea (and they will, of course, have additional clues from what they know about the communicator, their shared cultural knowledge, and so on). The communicator has to go in a reverse direction, creating something akin to the ‘found hound’, ‘found rose’ or ‘found stick’ sensory input for her team-mates – and to do so on the spot.
The search for interpretation is centre-stage again: we are able to create and decode inventive transformations of past experience – all the way from the book, to a hound, to a hound engaged in a violent raging attack (we don’t attempt to portray the hound sleeping peacefully, or drinking from a bowl of water) – then map the imagined hound’s body movements onto our own pantomime actions. And all of this requires the ingenious transformation of a wealth of knowledge – of the book and its central theme, and some crude grasp of the motifs of a canine attack, and how this can map onto the human body (so that, for example, arms become front legs, and fingers become claws). Such coding and decoding is all too imperfect, as players of charades will be aware. A mime intended to portray the flailing claws and gnashing teeth of the famous hound can so easily be read as Jurassic Park.
IMAGINATION AND INTELLIGENCE
The whimsical imagination which creates faces from handbags or from apparently meaningless black and white patterns, which plays with metaphors and invents stories, and which creates and perceives songs and art, might seem charming but impractical. Such exuberant mental leaps may appear primarily to be the province of the arts. After all, the worlds of painting and sculpture are replete with incredibly sketchy, incomplete and distorted representations of the human face, or indeed of many other things; and literature is full of metaphors, one character or story representing some other character or story, and relies on our brains turning a text, a stage, or the movements of a TV or film screen into the semblance of an entire world.
Why has our species developed such rich imaginative powers? Survival and reproduction surely don’t depend on seeing faces in inanimate objects, viewing moods as spatial locations (up or down) or personalities as light sources (brilliant, sparkling, dazzling, bright on the one hand, or dull and dim on the other), or generating a continual stream of stories, whether credible or completely fantastical, to explain the world around us. Why, indeed, hasn’t the iron logic of evolution by natural selection eliminated such playfulness in favour of a relentless focus on, say, commando-style survival skills? And how could the infinite imaginative landscape of romance and love arise in a species shaped, one might think, to have a one-dimensional focus on sex for reproduction?
Putting evolution to one side, and simply considering the demands of modern life, one might also suspect that capricious flights of fancy, while perhaps entertaining and enriching, must be secondary to the core practical business of the mind: understanding the world around us, making decisions and planning actions, and sending precise instructions and observations to others. Years of formal education, and the experience of working, as most of us do, somewhere in the middle of a complex web of bureaucratic processes, might reinforce such suspicions. Isn’t the essence of thought discipline and control, rather than whimsical and wild connections? It is tempting to see imagination as no more than a mental ‘bauble’: conspicuous but entirely dispensable.
But we have seen that the opposite is true: imaginative leaps are essential for perceiving the world and making sense of each other. Such leaps allow us to project our past experiences onto an open-ended and continually surprising world. Any discipline of thought (learning to programme computers, play in an orchestra, prove mathematical theorems) restricts the direction in which our imaginations take us; and the ability to work within these restrictions is learned painfully and effortfully. Indeed, I suggest such discipline is difficult precisely because it requires taming and trammelling our sometimes unruly imagination. Our natural ‘mode’ of thinking is wildly flexible – we only think of discipline and control as the essence of thought because these require our conscious and careful attention. The sheer ubiquity of our imaginative flexibility renders it invisible.
Leaps of the imagination are, on reflection, evident even when solving what we might think of as ‘logical’ puzzles, which are central to intelligence. Spend a few minutes on the perhaps familiar IQ puzzles below:
| (1) Space is to ruler as time is to: | A: Metronome |
| B: Chronometer | |
| C: Clock | |
| D: Stopwatch | |
| (2) Sound is to echo as light is to | A: Shadow |
| B: Reflection | |
| C: Refraction | |
| D: Mirror | |
| (3) Replicate is to duplicate as divide is to | A: Split |
| B: Segment | |
| C: Fractionate | |
| D: Halve |
IQ questions of this type are tests of mental elasticity – but also precision. Considering (1), for example, we need to map between time and space. A ruler measures the distance between two points in space. So what measures the distance between two points in time? Aha! – a stopwatch (a clock – which is a good second choice – measures the time, not the difference between times). But finding this mapping is not easy (and not entirely uncontroversial, perhaps).
Or consider (2): when sound bounces back off a surface (e.g. the walls of a ravine or the roof of a cave) we sometimes hear a copy of the original sound: an echo. When light from an object bounces off a surface (e.g. a mirror, or the surface of a still pond), we sometimes see a copy of that object: a reflection. This line of thinking suggests that B is the right answer.
Finally, in (3), when we make one or many copies of a gene, a computer file or a sheet of music (perhaps many times), we replicate it. When we create precisely one copy, we duplicate the original: we end up with precisely two versions of our gene, file or sheet of music. When we divide a number, we break it into several equal-sized pieces; when we divide it to create precisely two equal sized pieces, we halve it. This line of thinking suggests that D is the right answer.
Notice that solving these puzzles is very different from solving Sudoku or squaring a large number. In open-ended IQ problems, imagination is required even to make the problem meaningful at all. What, after all, is the relationship between the words space and ruler? A ruler takes up space? A ruler measures spaces? Space has no ruler – i.e. it is not ruled by anyone? Rulers typically have lots of space to live in? These fairly daft possibilities didn’t even cross your mind, I suspect. One clue that they would be on the wrong track comes from the fact that we need to find a link between space and time. So presumably space and time must be considered quite abstractly. And then we might think of a ruler as a measure of space, but then several of the available options (chronometer, clock, stopwatch) are measures of time – so this doesn’t give us a unique answer. A bit more thought might lead to the idea that both rulers and stopwatches measure intervals (in space or time), whereas clocks just give an absolute value. At least this interpretation picks out a unique answer, stopwatch, to be paired with time. Of course there are endless other ways of picking out a unique answer – though some seem more natural than others.
Suppose, for example, that we reason: a ruler can be used to divide space into equal intervals – and a metronome can be used to divide time into equal intervals – and hence we plump for option A. This seems a bit of a stretch (another metaphor), because rulers can do many other things than divide space into equal intervals (in particular, a ruler has a measuring scale, which has no analogue in the metronome).
Or, even less convincingly, we notice that the first letters of space and ruler are consecutive letters in the alphabet: r is the letter before s. Now time begins with a t, so perhaps the related word should begin with the letter of the alphabet before t, namely s. And indeed, there is a unique option beginning with an s, namely stopwatch. So we get the same ‘right’ answer – but using rather tenuous and forced reasoning. After all, the fact that the words are all about time, space and measurement is purely coincidental.
So, unlike Sudoku or arithmetical calculations, for which a solution is precisely defined (and there are, indeed, methods for systematically finding such solutions), these IQ test problems are actually remarkably open-ended. They are capturing our ability to search in the vast space of possible metaphorical links between words (or the ideas they bring to mind), and judging our ability to find natural, sensible mappings rather than perverse and tortured links.
But what, precisely, counts as the right answer in analogy problems like these? It is not entirely clear what the right criterion is, but surely the best answer is something like the consensus answer – not the one most of us think of, perhaps, but the one that most of us think is the best, when it is pointed out.5
This is not so different from the problem of interpreting faces in found objects. These ambiguous images could be given all sorts of interpretations, at a stretch. The handbag in Figure 37 looks like a snarling face to most of us, but could also vaguely remind one of an owl, a rubbish bin, the mouth of a fish, or a beetle. But all interpretations are not equal – again, there is a clear consensus; indeed, once we’ve seen our ‘found faces’ as faces, it is difficult to see them any other way. Imagination – our ability to construct rich interpretations in complex, open-ended problems – seems to be just what many IQ tests are measuring. Thus the secret of intelligence is imaginative interpretation, rather than ‘cold logic’.
But intelligence requires us to harness our imagination in a disciplined way – intelligence requires more than merely blurting out the first interpretation or metaphor that comes to mind. It turns out to be highly productive to think of gases as consisting of clouds of tiny frictionless billiard balls continually bumping into each other in three dimensions – indeed, this is the standard model in physics. This insight proves to be crucial in understanding how the microscopic properties of trillions of molecules can lead to ‘macroscopic’ observations of pressure, temperature, volume and so on. Suppose, for example, sticking to two dimensions for a moment, we imagine expanding our billiard table dramatically – the billiard balls will continue to race about and bounce into each other as before, but now they will be much sparser – and hence far fewer balls will thump into the sides of the billiard table at any moment. And the reduced pummelling of the sides of the billiard table corresponds to lower pressure. Or suppose that we have two adjacent billiard tables, one in which a rather sparse set of balls is hurtling around at tremendous speed; and another where a much denser cloud of balls is moving about, on average, much more slowly. If we remove the partition between the two tables, then the density, and the average speeds, of the balls will gradually even out. The speed of the balls corresponds to temperature in the billiard-ball model of gas – so this means that the temperature in the two initially separate ‘gases’ will gradually become equal as they are joined together; and their density (i.e. the crowdedness of the billiard balls) and pressure (pummelling of the sides of the billiard tables) will reduce to an equilibrium as well.
The billiard-ball model of gases turned out to be incredibly useful, and now serves as the basis for capturing all sorts of aspects of known gas behaviour. It produces a beautiful link between the microscopic behaviour of, not billiard balls, but tiny gas molecules, whizzing about roughly according to Newton’s laws of motion (the very same laws we use for projectiles and planets), and the way gases work. Developing and utilizing an analogy like the billiard-ball model of gases requires great care and subtlety. The goal is to create a rigorous, mathematically precise model – not merely a fanciful metaphor – and to test it in experiments to see where it works, and where it fails. Such carefully developed analogies are the foundation of many areas of science.
But to do this, I suggest, doesn’t require switching to some entirely different style of thought – some imagined cold logic with which the products of our imagination can be soberly assessed. Instead, we need to direct successive thoughts to search for alternative interpretations, check that those interpretations make sense, and so on, by ‘locking onto’ different aspects of the information in the problem. Figuring out unexpected predictions from our analogy, deciding how to map them into known mathematics, and establishing which key experiments we need to conduct, all require ingenuity and inspiration, not mere handle-cranking. Both intelligence and analogy-making, even in science, are driven by our wonderfully elastic imagination, properly harnessed and directed.
THE DISTANT PROSPECT OF INTELLIGENT MACHINES
If our spectacular mental elasticity – our ability to imaginatively interpret complex, open-ended information into rich and varied patterns – is the secret of human intelligence, what does this imply for the possibility of artificial intelligence (which we began to examine in Chapter 1)?
My suspicion is that the implications are far-reaching. As we saw, the early attempts to extract and codify human ‘reasoning’ and knowledge into a computer database failed comprehensively. The hoped-for hidden inner principles from which our thoughts and behaviour supposedly flow turned out to be illusory. Instead, human intelligence is based on precedents – and the ability to stretch, mash together and re-engineer such precedents to deal with an open-ended and novel world. The secret of intelligence is the astonishing flexibility and cleverness by which the old is re-engineered to deal with the new. Yet the secret of how this is done has yet to be cracked.
The spectacular progress in computational intelligence over the last half-century has not been achieved by replicating the elasticity of the human imagination, which allows us to see, for instance, in a game of charades, a middle-aged man beating his chest and swatting the air with his fists as a representation of the film King Kong, or read rich emotion and humanity into Mooney’s black and white patches, or see the world through an endlessly shifting blizzard of metaphors. Computational intelligence has instead taken a very different tack: focusing on problems, like chess or arithmetic, that require no free interpretation at all, but which can be reduced to vast sequences of calculations, performed at lightning speed. In addition it has proved to be invaluable for things like speech recognition, machine translation and general knowledge tests, hoovering up solutions to almost unimaginably vast quantities of past problems to enable the machine to solve new problems, which are only a little different.6
Yet what is astonishing about human intelligence, and perhaps biological intelligence more broadly, is its spectacular flexibility. To us, it is unremarkable that our brains can map J. M. W. Turner’s smudges of paint, or Debussy’s sinuous orchestration in La Mer into a turbulent or calm seascape; that we can interpret the movements of cartoon characters, shadow puppets or ballet dancers as playing out human dramas; that we make instant mental leaps from properties of tensile materials to states of mind (being under strain, stretched, over/under-stretched, pulled in too many directions, spread too thinly, taut, tense, likely to snap, at breaking point, (in)flexible, rigid, stiff, fragile, shattered); that our brains are able to discern links between ripples on water, sound, light, radio, vibrations, the wriggling of a rope and perhaps even the force of gravity itself.
But my suspicion is that it is our mental elasticity that is one of the keys to what makes human intelligence so remarkable and so distinctive.7 The creative and often wildly metaphorical interpretations that we impose on the world are far removed from anything we have yet been able to replicate by machine.
To those who, like me, are fascinated by the possibilities of artificial intelligence, the moral is that we should expect further automation of those mental activities that can be solved by ‘brute force’ rather than mental elasticity – the routine, the repetitive, the well defined. This is part of a trend that started with the development of stone tools in the Olduvai Gorge in present-day Tanzania, more than two and a half million years ago and accelerated spectacularly in the Industrial Revolution: people and technology can achieve far more than people alone. It is a continual source of amazement to us that tasks that seemed to require the full power of human ingenuity can be solved, often far more efficiently, by processes of standardization and mechanization: the flexibility and dexterity of human hand-weaving could, in many cases, be replaced by the precision achieved by the hand-loom, and then the steam-driven Jacquard loom, controlled by punched cards around 1800, and on to the phenomenal productivity of the computerized power looms of today. At each step, the environment is made more precise and more standardized; and more of the task can be handed over to machines.
In the same way, the rise of digitization and big data can create an ever more frictionless and more precisely defined world in which computers can operate far better than we can. But the secret of human intelligence is the ability to find patterns in the least structured, most unexpected, hugely variable of streams of information – to lock onto a handbag and see a snarling face; to lock onto a set of black-and-white patches and discern a distinctive, emotion-laden, human being; to find mappings and metaphors through the complexity and chaos of the physical and psychological worlds. All this is far beyond the reach of modern artificial intelligence.
It is our wild imposition of meaning upon the world by the appropriation and transformation of past experience that is the essence of human thought, from which our more sober reflections can, with difficulty, be constructed. Watching the mind at play is our best guide to its natural mode of operation. This reveals the search for interpretation that drives us – the effort to find meaning that is channelled by, rather than replaced with, step-by-step conscious deliberation.
To those who fear the march of the machines, this should be of some comfort. If imagination and metaphor is the secret of our intelligence, then that secret may, perhaps, be safely locked away in the human brain for centuries and perhaps for ever.