During the past twenty-five years, the field of decision making has concentrated on showing the limitations of decision makers—that is, that they are not very rational or competent. Books have been written documenting human limitations and suggesting remedies: training methods to help us think clearly, decision support systems to monitor and guide us, and expert systems that enable computers to make the decisions and avoid altogether the fallible humans.
This book was written to balance the others and takes a different perspective. Here I document human strengths and capabilities that typically have been downplayed or even ignored.
In 1985, I did my first study of how firefighters make life-and-death decisions under extreme time pressure. That project led to others—with pilots, nurses, military leaders, nuclear power plant operators, chess masters, and experts in a range of other domains. A growing number of researchers have moved out of the laboratory, to work in the area of naturalistic decision making—that is, the study of how people use their experience to make decisions in field settings. We try to understand how people handle all of the typical confusions and pressures of their environments, such as missing information, time constraints, vague goals, and changing conditions.1 In doing these studies, my research team and I have slept in fire stations, observed intensive care units, and ridden in M-1 tanks, U.S. Navy AEGIS cruisers, Blackhawk helicopters, and AWACS aircraft. We have learned a lot about doing field research.
Instead of trying to show how people do not measure up to ideal strategies for performing tasks, we have been motivated by curiosity about how people do so well under difficult conditions. We all have areas in which we can use our experience to make rapid and effective decisions, from the mundane level of shopping to the high-stakes level of firefighting. Shopping in a supermarket does not seem like an impressive skill until you contrast an experienced American shopper to a recent immigrant from Russia. Moving to the other extreme of high-stakes decisions, an example is a fireground commander working under severe time pressure while in charge of a crew at a multiple-alarm fire at a four-story apartment building. Our research concentrated on this high-stakes world. The fireground commanders seemed to be making effective decisions.
My research assistant, Chris Brezovic, and I are sitting in a fire station in Cleveland on a Saturday afternoon in the summer of 1985. We slept only a few hours in the station the night before since we had been up late interviewing the commander during that shift. He was going to stay up all night catching up on his work. We were assigned beds on the second floor. I was told to be ready to get down the stairs and onto the truck no more than twenty-five seconds after an alarm sounded. (No, we did not slide down the pole, although the station still had one. Too many firefighters had broken ankles that way, so they no longer used the pole.) I even slept with my eyeglasses on, not wanting to waste precious seconds fumbling with them. There was only one call, at around 3:00 in the morning. The horn suddenly began blaring, we all jumped out of bed, ran down the flight of stairs, pulled on our coats and boots, and climbed onto the trucks within the time limit. The fire was pretty small—a blaze in a one-car garage.
Chris and I are feeling a little sleepy that next afternoon when the alarm comes in at 3:21 P.M. for the emergency rescue team. Three minutes later, the truck is driving up to a typical house in a residential neighborhood. It is summer, and young women in bikinis who had been tanning themselves on their lawns are running over to their neighbor’s yard.
When we pull to a stop, we see a man lying facedown in a pool of blood, his wife crouching over him. As the emergency rescue team goes to work, the woman quickly explains that her husband had been standing on a ladder doing some home repair. He slipped, and his arm went through a pane of glass. He reacted foolishly by pulling his arm out and, in doing so, sliced open an artery. The head of the rescue team, Lieutenant M, later told us that the man had lost two units of blood. If he lost four units, he would be dead. Watching his life leak out of his arm, the man is going into shock.
The first decision facing Lieutenant M is to diagnose the problem. As he ran to the man, even before listening to the wife, he made his diagnosis. He can see from the amount of blood that the man has cut open an artery, and from the dishcloths held against the man’s arm he can tell which artery. Next comes the decision of how to treat the wound. In fact, there is nothing to deliberate over. As quickly as possible, Lieutenant M applies firm pressure. Next, he might examine whether there are other injuries, maybe neck injuries, which might prevent him from moving the victim. But he doesn’t bother with any more examination. He can see the man is minutes from death, so there is no time to worry about anything else.
Lieutenant M has stopped the bleeding and directs his crew to move the man on a stretcher and to the truck. He assigns the strongest of his crew to the hardest stretcher work, even though the crew member has relatively little experience. Lieutenant M decides that the man’s strength is important for quick movement and thinks the crew member has enough training that he will not drop the stretcher as it is maneuvered in through the back of the rescue truck.
On the way to the hospital, the crew puts inflatable pants on the victim. These exert pressure on the man’s legs to stabilize his blood pressure. Had the crew put the pants on the man before driving, they would have wasted valuable time. When we reach the hospital I look down at my watch: 3:31 P.M. Only ten minutes has elapsed since the original alarm.
This example shows decision making at a very high level. Lieutenant M handled many decision points yet spent little time on any one of them. He drew on his experience to know just what to do. Yet merely saying that he used his experience is not an answer. The challenge is to identify how that experience came into play.
We have found that people draw on a large set of abilities that are sources of power.2 The conventional sources of power include deductive logical thinking, analysis of probabilities, and statistical methods.3 Yet the sources of power that are needed in natural settings are usually not analytical at all—the power of intuition, mental simulation, metaphor, and storytelling. The power of intuition enables us to size up a situation quickly. The power of mental simulation lets us imagine how a course of action might be carried out. The power of metaphor lets us draw on our experience by suggesting parallels between the current situation and something else we have come across. The power of storytelling helps us consolidate our experiences to make them available in the future, either to ourselves or to others. These areas have not been well studied by decision researchers.4
This book examines some recent findings that have emerged from the field of naturalistic decision making. It also describes how research can be done outside the laboratory setting by studying realistic tasks and experienced people working under typical conditions. Features that help define a naturalistic decision-making setting are time pressure, high stakes, experienced decision makers, inadequate information (information that is missing, ambiguous, or erroneous), ill-defined goals, poorly defined procedures, cue learning, context (e.g., higher-level goals, stress), dynamic conditions, and team coordination (Orasanu & Connolly, 1993).
We like to study people under time pressure. We have estimated that fireground commanders make around 80 percent of their decisions in less than one minute.5 As in the case of the torn artery, most of these decisions take only seconds. We have studied chess players under blitz conditions, where the average move was made in six seconds.
Our results seem to hold even when there is not much time pressure. We get the same findings with design engineers who may have weeks or months to finish a project. They insist that they are working under extreme time pressure relative to their tasks, but in comparison to the fireground commanders and chess players, they are almost on vacation.
Naturalistic decision making is concerned with high stakes. When a fireground commander makes a poor decision, lives can be lost. When a design engineer makes a poor decision, hundreds of thousands of dollars can be lost.
We are interested in experienced decision makers since only those who know something about the domain would usually be making high-stakes choices. Furthermore, we see experience as a basis for the sources of power we want to understand. The fireground commanders we studied had an average of twenty-three years of experience as firefighters, and the chess masters we studied had played thousands of games. In contrast, in most laboratory studies, experience is considered a complicating factor. Subjects who know something about the task may have preconceived notions that could get in the way, or their strategies could distort the results. Therefore, subjects are given totally novel tasks to make sure all of them start with the same level of experience: zero.
We want to know how people carry on even when faced with uncertainty because of inadequate information that may be missing, ambiguous, or unreliable—either because of errors in transmission or deception by an adversary.
We are interested in tasks where the goals are unclear. Most of the time when we have to make difficult choices, we do not fully understand what we want to accomplish. For instance, when fireground commanders are called out to a blaze, they do not know what type of outcome they will be trying to reach: a fire needs to be extinguished, or the fire is so big that the best thing to do is to prevent it from spreading further, or they need to begin with search and rescue rather than fighting the fire, or they may have to call in a second or third alarm to get more resources, or the situation does not warrant extra resources and they can let the fire burn itself out. In contrast, laboratory studies concentrate on tasks with well-defined goals, since the achievement of a well-defined goal is easy to measure. With an ill-defined goal, you are never sure if the decision was right.
Naturalistic decision making is concerned with poorly defined procedures. Conventional laboratory studies, in contrast, prefer to keep decision making distinct from problem solving and do not require subjects to invent or modify procedures.
Cue learning refers to the need to perceive patterns and make distinctions. In laboratory studies we can present unambiguous stimuli: “If you choose option A, you have a 20 percent chance winning $100,000, whereas option B gives you a 100 percent chance of winning $15,000. Which do you pick?” For a task with ambiguous stimuli, consider a skilled racetrack handicapper, who notices that one of the horses in a race does better in the mud, examines the track that is slightly moist from an early morning drizzle, and tries to judge if the track is sufficiently moist to make a difference.
Most tasks are performed within a larger context that includes higher-level goals and different tasks with their own requirements, and this must be taken into consideration. Context also includes background conditions, such as noise, poor lighting, constant interruptions, and stressors.
Dynamic conditions (that is, a changing situation) are an important feature of naturalistic decision making. New information may be received, or old information invalidated, and the goals can become radically transformed. In our research with fireground commanders, we estimated that the situation changed an average of five times per incident. Our work with U.S. Navy commanders showed the same thing. Some changes were minor; they were elaborations on what the commanders already knew. Some were major, requiring a shift in the way the commanders understood the situation.
Finally, we want to know how people working in teams make decisions. For most of the domains we have studied, teams are involved: a fireground commander in charge of a fire company, a helicopter pilot working with a navigator or other helicopters, or a three-person airline cockpit crew. Rarely do we find a single decision maker, such as a chess player, who does not have to coordinate with anyone else.
The field of naturalistic decision making tries to understand how these features come into play. This book focuses on what we have learned about the way people think and decide in natural settings, using different sources of power. It explores the sources of power themselves, what they offer us, and where they fall short or get us in trouble. And it describes some ways of using these sources of power, whether for training or for designing better systems. Most of the chapters end with a section entitled Applications. One reason to discuss applications is that many applied researchers believe that there is nothing as practical as a good theory. If the things we learn do not have much practical value, perhaps we are investigating questions that are not important.
Some chapters end with a listing of key points and others do not. Some chapters seemed more straightforward, and the listing of key points seemed unnecessary. I added this listing for chapters that covered a lot of material and needed some summary.