Both deliberative and procedural decision-making require recognition of the “situation” we are in. Situation recognition is a categorization process in which we recognize that this moment shares similarities with previous situations we have experienced and differs from others. Although models exist of static situation-recognition processes, in actuality, situation recognition must be about recognizing sequences of events (narratives). The neural mechanisms underlying narrative remain unknown.
Animal psychology experiments tend to be run in cue-poor environments. A rat in a small box with nothing but a hole to poke its nose into and maybe a few levers to push, or a monkey in a small chair facing a blank screen, has few choices to decide between. When the tone sounds and food is delivered into the food port, there’s not a lot of ambiguity about what the new cue was. When it happens a few more times, the animal quickly recognizes that the tone precedes food delivery. When a dot appears on the screen, the monkey looks at it—it’s the only thing that has changed in the environment.
Even so, animal behavior has to be shaped to an environment.1 One of the most famous experiments used in neuroscience today is the Morris Water Maze,2 which consists of a large (several meters wide) pool of water made opaque with a small amount of water-soluble nontoxic white paint. In this task, rats or mice are trained to find a platform hidden under the water. This task is popular because it is particularly sensitive to damage to certain brain structures, such as the hippocampus.A From a decision-making perspective, the Morris Water Maze is very interesting because the task can be solved by either the deliberative or procedural (habit) systems, depending on the training sequence.3 But what is the first thing that rats do on this task? They swim around the edge of the pool, looking for a way out. Only after first thoroughly exploring this possibility do they start heading for the platform.
With humans, the important cues in psychology and neuroscience experiments are often explicitly identified to the subject: “You will see a set of faces. Try to identify which faces you have seen before … ” Even when they are not explicitly identified, although the environments for human psychology experiments are often cue-rich, the important cue is often obvious to the subject. An fMRI machine is a claustrophobic space with tight walls around you, a metal cage around your head, and extremely loud pounding thumping noises.4 And, yet, when the visual cue is turned on, we know to attend to it. In part, we know to attend to it because we have a storyline for what we expect to happen in one of these experiments.
But, as I’ve pointed out before, life outside of psychology experiments is made up of a plethora of cues impinging on our senses at all times. We are not often sitting, staring at a screen, trying to figure out if a small subset of a set of randomly moving dots tend to be moving more in one direction than another. Certainly, there are times that we see people peering into the distance or the darkness (think of a sailor standing on the crow’s nest on a ship, keeping a lookout for land, or of a hunter stalking a deer); however, most of the time, the issue is too many cues, not too few.
I first got interested in the question of situation recognition when talking to one of my colleagues (Steve Jensen) over lunch. Steve was fascinated by a phenomenon of one-trial learning in ICD patients.5 These patients have a risk of sudden heart arrhythmias (where the beating of the heart changes to the wrong rhythm) and have had an implantable cardioverter-defibrillator (ICD) attached to their heart. These devices detect the rhythm of the heart and, if that rhythm goes wrong, provide an electrical shock to restart the heart. (Just as an external defibrillator can shock the heart back into rhythm, so can an internal one.) Unlike a pacemaker, which provides a slow, steady rhythm to the heart, the ICD kicks in only when the rhythm is stopped or wrong. Also, unlike a pacemaker, the ICD hurts like hell when it kicks in. Unlike an external defibrillator, which is used only on unconscious patients, the ICD usually kicks in while the patient is conscious. In such situations, the patient is suddenly provided with an extremely negative experience (even if it potentially may have saved the patient’s life). These ICD patients have the strangest negative associations with things in their lives. Steve wanted to know what made the patient decide that the fork was the culprit and not the spoon, or that it was the act of sitting in a chair and not laughing with one’s friends. Clearly, it has something to do with what these subjects were attending to at the time, but it got us thinking about the question of how we decide what cues are important.
In the robotics, computer science, and psychology literatures that study reinforcement learning, this is called the credit assignment problem.6 Generally, the assumption in these models and experiments is that there is a small subset of potential cues that could be important, and the agent (the animal or the computer simulation) has to figure out which one is the key to making the right decision. But we know from other experiments (such as the Morris Water Maze and other spatial navigation experiments) that it is often a constellation of cues that informs the animal.7 A subset of the cues can be removed and the animal will still find its way correctly, as long as enough of the cues are available. No one cue is key.
We realized that most of the computer models of decision-making assumed that the agent knew which possible situations were available (technically known as the state space) and often knew how they changed from one to another (technically known as the transition matrix), but that in reality, animals created situations out of cues. Real animals (including humans) categorized cues into situations.8 Simple categorization in neural systems is well understood and can be modeled with content-addressable memories.9 In fact, there are very well-studied learning systems that automatically categorize inputs, even without supervision.10 These systems are called unsupervised learning algorithms and work by identifying regularities in the inputs.
Using a standard unsupervised learning algorithm, we built a model that incorporated a situation-categorization component into a standard reinforcement learning agent (an agent that learned what to do in what situation, essentially a procedural-learning agent in the terminology used in this book).11 We found that this situation-recognition component turned out to change the observed psychology of our agent in a number of important ways. Most importantly, disappointment (lack of delivered reward) became different from punishment (delivery of a negative stimulus). In our model, disappointment led to the recognition of a change in the situation, while punishment led to a new association. (We’ve talked about the differences between reward, reinforcement, and disappointment, and their negative counterparts [punishment, aversion, and relief] in Chapter 4.)
This model suggests that there is a significant context effect to recognizing changes in our expectations of reward or punishment. This can explain why, for example, addicts who become clean in hospitals relapse on being released back into their normal lives, or why students who learn the correct answers to give about electricity in physics class are still unable to correctly build circuits when given a set of real batteries and other electrical components to build with.12
We were also able to manipulate parameters in our model to produce simulations of problem gambling due to incorrect situation recognition, either because the agent thought it could control a random situation (the illusion of control, believing that two outcomes were due to two situations when they really occurred due to random chance within a single situation) or because the agent was unable to recognize a change in a situation (which led to the phenomenon of chasing losses, continued playing even in the face of devastating loss because the agent thinks the difference is just random chance).13
Although our model was able to explain a number of psychological phenomena (such as why behavioral extinction is not the same as forgetting or why animals are slower to extinguish variably delivered rewards) and a number of clinical phenomena (such as context-related relapse in addiction or the illusion of control in gambling),14 the unsupervised learning algorithm we used was a static cue-categorization mechanism. The algorithm took the currently available set of cues and determined whether it was part of a known situation or whether it had to be declared a new situation.15 The real world is, of course, much more complex.
Of course, situation recognition is not all or none: situations are both similar and dissimilar to other situations. Making correct decisions requires that we be able to recognize both the similarities and differences between situations. The usual statement attributed to the Greek philosopher Heraclitus is that “You never step into the same river twice,” but the fragments we have from the actual Greek suggest that a better interpretation is that some things stay constant through change.16 If you are going to learn, you have to have some constancy between your past and your future. You need to recognize the similarities between your past and your present in order to translate past successes and failures into present plans. To solve this paradox, we need a system that can categorize situations, recognizing both the similarities and differences between your past and present.
I give a lot of scientific presentations to lots of different audiences. When I build my slides for a presentation, I try to make sure I have the right slides for the given audience. I remember the first time I went to give a talk in Japan. My wife pointed out that most Japanese scientists could read written English much better than they could understand spoken English. So I added a section to each of my slides describing the basic results on that slide. (I have since found that this helps native English speakers as well.) But even though the situation was slightly different, I didn’t think of giving-a-talk-in-Japan to be a new situation, I defined it as a special case of my giving-a-talk category. This hierarchical nature is a key to human categorization.17 A penguin is a bird, but it is also a water-swimmer, and it doesn’t fly. Just as we categorize our concepts in a complex hierarchy with special cases that do not quite fit the categories, we also must categorize situations in complex ways.
Situations are also dynamic. We do not simply see a set of cues, categorize them as a situation, and respond to it. Instead, we recognize a sequence of situations and travel through it. I travel through the Minneapolis-St. Paul airport (MSP) approximately once per month (sometimes more, but I try to restrict myself to traveling less than once a month). Sometimes I have my laptop, sometimes I don’t. Sometimes I have my family with me, sometimes I don’t. Sometimes I have checked a bag, sometimes I take only carry-on. Going through security with my laptop is different from not. Similarly, security is different if it’s just me or if I have my kids, and it’s different if my toiletries are in my carry-on bag or checked through. And yet, I didn’t need to relearn how to go through security from scratch the first time that they told me I had to take my toiletries out of my carry-on and pass them through the X-ray machine separately. I was able to take what I knew about going through airport security and modify it to include taking my toiletries out of my carry-on suitcase.
Going through airport security is a good example of a dynamic situation, what might be called a script.18 The actions at each stage are very different, and yet they all go together in a recognizable sequence. First, you need your ID and boarding pass. Once those are checked, you then need to wait in line for the plastic tubs to put your stuff into. In American airports, you need to take off your shoes. (I once started to take off my shoes in a European airport and the security agent looked at me with quizzical surprise and asked, “What are you doing?”) Going through airport security is a sequence of subsituations and actions.
A script is a storyline, a categorization of a sequence that can be modified at its sub-stages. The important points here are threefold. First, we categorize the situations of the world in order to act within it (What are the important cues here?). Second, those categories are complex and hierarchical (Giving a talk in Japan is a special case of my giving-a-talk category). Third, these scripts are dynamic (Going through airport security is a complex, dynamic sequence of situations and actions). These scripts are narratives, stories we tell ourselves about our lives.
The success of great literature is usually due to the recognizability of the characters. We ask ourselves, “Is that what this character would do in this situation?” Playwrights often talk of “keeping things in character.” When a person does something that is “out of character,” everyone notices. As Fergus (played by Stephen Rea) explains to Dil (played by Jaye Davidson) in Neil Jordan’s The Crying Game, “It’s in my nature.” And then, to fully explain his actions to her, he tells her the fable of the frog and the scorpion. We recognize the self-consistency of his actions as a definition of his nature. This is true whether the stories be realistic or fantastical. Even explicitly unrealistic plays are based, fundamentally, on recognizable characters. I remember the first time I saw Samuel Beckett’s Krapp’s Last Tape and realized it was really about an old, alcoholic writer reminiscing over a long-past love affair. For all of its existentialist implications, Jean-Paul Sartre’s No Exit is a simple dramatic interaction between three very real people. Even the absurdist plays like Eugene Ionesco’s Rhinoceros have recognizable human interactions throughout.
In fact, throughout literature, the stories that survive are the ones in which we recognize human characteristics. Whether this be the remarkably human characteristics of the Greek or Norse gods, or the foibles of Biblical characters, the stories that survive are the ones that capture something “true” about human nature.
In large part the success of literature arises from the fact that human nature hasn’t changed in thousands of years. We recognize ourselves in the squabbling Greeks of Homer’s Iliad and the recovering soldier working his slow way home in Homer’s Odyssey. We recognize ourselves in Hamlet’s indecision, in the rashness of King Lear. The soap opera of Shakespeare’s A Midsummer Night’s Dream is recognizable in any high school clique. The fables of Africa, North America, and medieval Italy all speak to us reliably.19 We can even find ourselves in the imagined worlds of fantasy and science fiction, whether it be the hobbits learning to stand tall in J. R. R. Tolkien’s Lord of the Rings or the political machinations in Frank Herbert’s Dune.
It is important to recognize that this doesn’t mean that literature is a morality play; true morality plays tend to be boring and quickly forgotten. Successful literature is more like an experiment, a way of testing “what would happen if.” Literature is a form of fictive learning, a way to see what would happen if someone with a certain character were faced with a specific situation. It allows us to ask ourselves, “Would I do that?” Whether we ask ourselves if we would make the same mistakes or if we would have the strength to take the right action, literature is a way of trying out hypotheses about consequences, much like the Gedanken experiments (thought experiments) in science.
Writing instructors tell nascent writers to find the character, and then to put him or her into the most difficult situation one can.20 What makes a situation difficult depends on the character one is writing about. At each stage of a story, we ask ourselves, “What would this character do in this situation?” We are very good at recognizing forced fiction, where the characters do something that is out of character to further the plot.
Fundamentally, humans are narrative creatures.21 We tell ourselves stories of our lives and we define our lives in terms of those stories. We use narrative to explain our past, to recognize our present, and to predict the future. We plan the future by telling stories of what we think will happen (see Chapter 9). We recognize our present through the scripts we are passing through, and we try to understand our past actions and the things that have happened to us in terms of a narrative structure, by telling stories. Of course, as we saw in Chapter 9, that past is constructed—reconstructed, one might say. By changing the stories we tell ourselves, we reinterpret why we did what we did and we change who we are.
For example, the simple sentence “There is no such thing as one drink for an alcoholic” implies that the decision options you have in the future are not between going to the bar and having a single drink or staying home and being bored, but rather between going to the bar and getting really drunk (and thus regretting it) or staying home and not. This is a process called “bundling” in the neuroeconomics literature.22 By bundling future decisions together, one sees both the immediate and long-term consequences. Computational models of this show that these bundling effects can change decisions in ways that nonbundled decisions cannot.
At this point, we don’t know very much about the neurophysiology of how stories are represented within the brain. Clearly it must involve representations of categories and content-addressable memories. We know from the psychological literature that they must exist, and we know something of how static categories are represented within the brain, but the complexity of situation-recognition systems (such as the hierarchical nature) and the dynamics of those representations (such as the narrative structure and the effect of that narrative structure on decision-making) are still very open questions in the scientific literature today.
Rather than point you to scientific papers about the role of narrative, I’m just going to suggest reading fiction. Great authors are keen observers of the human condition.B