Brain Rule #8
Stimulate more of the senses.
EVERY TIME TIM SEES the letter E, he also sees the color red. He describes the color change as if suddenly forced to look at the world through red-tinted glasses. When Tim looks away from the letter E, his world returns to normal, until he encounters the letter O. Then the world turns blue. For Tim, reading a book is like living in a disco. For a long time, Tim thought this happened to everyone. When he discovered this happened to no one—at least no one he knew—he began to suspect he was crazy. Neither impression was correct. Tim is suffering—if that’s the right word—from a brain condition called synesthesia. It’s experienced by perhaps one in 2,000 people; some think more.
Synesthesia appears to be a short circuiting in the way the brain processes the world’s many senses. But it also provides a strong clue that our sensory processes are wired to work together. In one of the strangest types of synesthesia—there are at least three dozen—people see a word and immediately experience a taste on their tongue. This isn’t the typical mouthwatering response, such as imagining the taste of a candy bar after hearing the word “chocolate.” This is like seeing the word “sky” in a novel and suddenly tasting a sour lemon in your mouth. A clever experiment showed that even when the synesthete could not recall the exact word, he or she would still get the taste from a general description of the word. Even when the brain’s wiring gets confused, the senses still attempt to work together.
Here’s another way we know the brain likes sensory integration. Suppose researchers show you a video of a person saying the surprisingly ugly syllable “ga.” Unbeknownst to you, the scientists have turned off the sound of the original video and dubbed the sound “ba” onto it. When the scientist asks you to listen to the video with your eyes closed, you hear “ba” just fine. But if you open your eyes, your brain suddenly encounters the shape of the lips saying “ga” while your ears are still hearing “ba.” The brain has no idea what to do with this contradiction. So it makes something up. If you are like most people, what you actually will hear when your eyes open is the syllable “da.” This is the brain’s compromise between what you hear and what you see—its need to attempt integration. It’s called the McGurk effect.
But you don’t have to be in a laboratory to see it in action. You can just go to a movie. The actors you see speaking to each other on-screen are not really speaking to each other at all. Their voices emanate from speakers cleverly placed around the room: some behind you, some beside you; none centered on the actors’ mouths. Even so, you believe the voices are coming from those mouths. Your eyes observe lips moving in tandem with the words your ears are hearing, and the brain combines the experience to trick you into believing the dialogue comes from the screen. Together, these senses create the perception of someone speaking in front of you, when actually nobody is speaking in front of you.
The process of sensory integration has such a positive effect on learning that it forms the heart of Brain Rule #8: Stimulate more of the senses at the same time.
A fire hose of sights and sounds
An incredible amount of sensory information comes at us in any given moment. Imagine, for example, that you’ve gone out on a Friday night to a dance club in New York. The beat of the music dominates, hypnotic, felt more than heard. Laser lights shoot across the room. Bodies move. The smells of sweat, alcohol, and illegal smoking mix in the atmosphere like a second sound track. In the corner, a jilted lover is crying. You step out for a breath of fresh air. The jilted lover follows you. All of these external physical inputs and internal emotional inputs are presented to your brain in a never-ending fire hose of sensations. Does the example of a dance club seem extreme? It probably holds no more information than what you’d normally experience the next morning on the streets of Manhattan. Faithfully, your brain perceives the screech of the taxis, the smell of the pretzels for sale, the blink of the crosswalk signal, the touch of people brushing past. And your brain integrates them all into one coherent experience.
You are a wonder. We in brain-science land are only beginning to figure out how you do it.
It’s mysterious: On one hand, your head crackles with the perceptions of the whole world—sight, sound, taste, smell, touch—as energetic as that dance party. On the other hand, the inside of your head is a darkened, silent place, lonely as a cave. The Greeks didn’t think the brain did much of anything. They thought it just sat there like an inert pile of clay. Indeed, it does not generate enough electricity to prick your finger. Aristotle thought the heart held all the action. Pumping out rich, red blood 24 hours a day, the heart, he reasoned, harbored the “vital flame of life.” This fire produced enough heat to give the brain a job description: to act as a cooling device. (He thought the lungs helped out, too.) Perhaps taking a cue from Aristotle, we still use the word “heart” to describe many aspects of mental life. Now we know that one of the brain’s major job descriptions is to handle all of the inputs that our senses pick up and allow us to perceive the world.
How we perceive something
During the Revolutionary War, the British—steeped in the traditions of large European land wars—had lots of central planning. The field office gathered information from leaders on the battleground and then issued its commands. The Americans—steeped in the traditions of nothing—used guerrilla tactics: on-the-ground analysis and decision making prior to consultation with a central command. These very different approaches are a good way to describe the two main theories scientists have about how the brain goes from sensing something to perceiving it. Imagine the sound of a single gunshot over a green field during that war.
In the British model of this experience, our senses function separately, sending their information into the brain’s central command, its sophisticated perception centers. Only in these centers does the brain combine the sensory inputs into a cohesive perception of the environment. The ears hear the rifle and generate a complete auditory report of what just occurred. The eyes see the smoke from the gun arising from the turf and process the information separately, generating a visual report of the event. The nose, smelling gunpowder, does the same thing. They each send their data to central command. There, the inputs are bound together, a cohesive perception is created, and the brain lets the soldier in on what he just experienced.
The American model puts things very differently. Here the senses work together from the very beginning, consulting and influencing one another quite early in the process. As the ears and eyes simultaneously pick up gunshot and smoke, the two impressions immediately confer with each other. They perceive that the events are occurring in tandem, without conferencing with any higher authority. The picture of a rifle firing over an open field emerges in the observer’s brain. Perception is not where the integration begins but where the integration culminates.
Which model is correct? The data are edging in the direction of the second model. There are tantalizing suggestions that the senses do help one another, and in a precisely coordinated fashion. We’ll talk about them in a couple of pages.
First sensing and routing, then perceiving
No matter which model is eventually declared the winner, the underlying processes are the same, and they operate in the same order: sensing, routing, and perceiving. Sensing involves capturing the energies from our environment that are pushing themselves into our orifices and rubbing against our skin. The brain converts this external information into a brain-friendly electrical language. Once the sensory information is encoded, it is routed to appropriate regions of the brain for further processing. As we discussed in the Wiring chapter, the signals for vision, hearing, touch, taste, and smell all have separate, specialized places where this processing occurs. A region called the thalamus—a well-connected, egg-shaped structure in the middle of your “second brain”—supervises most of this shuttling.
The information, dissected into sensory-size pieces and flung widely across the brain, next needs to be reassembled. Specialized areas throughout the brain take over from the thalamus to make this happen. They are not exactly sensory regions, and they are not exactly motor regions, but they are bridges between them. Hence, they are called association cortices. (“Cortices” is the plural of “cortex.”) As sensory signals ascend through higher and higher orders of neural processing, the association cortices kick in.
The association cortices employ two types of processors: bottom up and top down. Let’s walk through what they might be doing in your brain as you read the next sentence—a randomly chosen quote attributed to author W. Somerset Maugham.
The rank and file make a report
“There are only three rules for writing a novel,” Maugham once said. “Unfortunately, nobody knows what they are.”
After your eyes look at that sentence and your thalamus has routed each aspect of the sentence to the appropriate brain regions, “bottom-up” processors go to work. The visual system is a classic bottom-up processor. It has feature detectors that greet the sentence’s visual stimuli. These detectors, working like auditors in an accounting firm, inspect every structural element in each letter of every word in Maugham’s quote. They file a report, a visual conception of letters and words. An upside-down arch becomes the letter U. Two straight lines at right angles become the letter T. Combinations of straight lines and curves become the word “three.” Written information has a lot of visual features in it, and this report takes a great deal of effort and time for the brain to organize. It is one of the reasons that reading is a relatively slow way to put information into the brain.
Higher-ups interpret the report
Next comes “top-down” processing. This can be likened to a board of directors reading the auditor’s report and then reacting to it. Many comments are made. Sections are analyzed in light of preexisting knowledge. The board in your brain has heard of the word “three” before, for example, and it has been familiar with the concept of rules since you were a toddler. Some board members have even heard of W. Somerset Maugham before, and they recall to your consciousness a movie called Of Human Bondage, which you saw in a film history course. Information is added to the data stream or subtracted from the data stream. In plenty of cases, as we saw in the McGurk effect, the brain resorts to making something up.
At this point, the brain generously lets you in on the fact that you are perceiving something.
Given that people have unique previous experiences, they bring different interpretations to their top-down analyses. Thus, two people can see the same input and come away with vastly different perceptions. It is a sobering thought. There is no one accurate way to perceive the world.
Smell is a powerful exception
Every sensory system must send a signal to the thalamus asking permission to connect to the higher levels of the brain where perception occurs—except for smell. Like an important head of state in a motorcade, nerves carrying information about smell bypass the thalamus and gain immediate access to their higher destinations.
Right between the eyes lies a patch of neurons about the size of a large postage stamp. This patch is called the olfactory region. The outer surface of this region, the one closest to the air in the nose, is the olfactory epithelium. When we sniff, odor molecules enter the nose chamber, penetrate a layer of snot, and collide with nerves there. The odor molecules brush against little quill-like protein receptors that stud the neurons in the olfactory epithelium. These neurons begin to fire excitedly, and you are well on your way to smelling something. The rest of the journey occurs in the brain.
One of the neurons’ destinations is the amygdala. The amygdala supervises not only the formation of emotional experiences but also the memory of emotional experiences. Because smell directly stimulates the amygdala, smell directly stimulates emotions. Smell signals also beeline for a part of your brain deeply involved in decision making. It is almost as if the odor is saying, “My signal is so important, I am going to give you a memorable emotion. What are you going to do about it?”
Smell signals are in such a hurry, our receptor cells for smell aren’t guarded by much of a protective barrier. This is different from most other sensory receptor cells in the human body. Visual receptor neurons in the retina are protected by the cornea, for example. Receptor neurons that allow hearing in our ears are protected by the eardrum. The only things protecting receptor neurons for smell are boogers.
Pairing two senses boosts one
We’ve talked about the fact that the brain strives to integrate all of the senses, and we’ve touched on the regions of the brain involved in perceiving those senses. (We haven’t talked about exactly how the brain integrates the senses, because, well, no one knows how that works.) Now let’s look at those tantalizing hints that stimulating multiple senses at the same time increases the capability of the senses.
In one experiment, people watched a video of someone speaking, but with no sound. At the same time, scientists peered in on the brain using fMRI technology. The fMRI scans showed that the area of the brain responsible for processing the sound, the auditory cortex, was stimulated as if the person actually were hearing sound. If the subject was presented with a person simply “making faces,” the auditory cortex was silent. It had to be a visual input related to sound. Then, visual inputs influence auditory inputs.
In another experiment, researchers showed short flashes of light near the subjects’ hands, which were rigged with a tactile stimulator. Sometimes researchers would stimulate the subjects’ hands while the light flashed, sometimes not. No matter how many times they did this, the visual portion of the brain always lighted up the strongest when the tactile response was paired with it. They could literally get a 30 percent boost in the visual system by introducing touch. This effect is called multimodal reinforcement.
Multiple senses affect our ability to detect stimuli, too. Most people, for example, have a very hard time seeing a flickering light if the intensity of the light is gradually decreased. Researchers decided to test that threshold by precisely coordinating a short burst of sound with the light flickering off. The presence of sound actually changed the threshold. The subjects found that they could see the light way beyond their normal threshold if sound was part of the experience.
Why does the brain have such powerful integrative instincts? The answer seems a bit obvious: The world is multisensory and has been for a very long time. Our East African crib did not unveil its sensory information one sense at a time during our development. Our environment did not possess only visual stimuli, like a silent movie, and then suddenly acquire an audio track a few million years later, and then, later, odors and textures. By the time we came down out of the trees, our evolutionary ancestors were already champions at experiencing a multisensory world. So it makes sense that in a multisensory environment, our muscles react more quickly, our eyes react to visual stimuli more quickly, and our threshold for detecting stimuli improves.
A multisensory environment enhances learning
Knowing that the brain cut its developmental teeth in an overwhelmingly multisensory environment, you might hypothesize that its learning abilities are increasingly optimized the more multisensory the situation is. You might further hypothesize that the opposite is true: Learning is less effective in a unisensory situation. That is exactly what you find.
Cognitive psychologist Richard Mayer probably has done more than anybody else to explore the link between multimedia exposure and learning. He sports a 10-megawatt smile, and his head looks exactly like an egg (albeit a very clever egg). His experiments are just as smooth: He divides the room into three groups. One group gets information delivered via one sense (say, hearing), another the same information from another sense (say, sight), and the third group the same information delivered as a combination of the first two senses.
The groups in the multisensory environments always do better than the groups in the unisensory environments. Their recall is more accurate, more detailed, and longer lasting—evident even 20 years later. Problem-solving ability improves, too. In one study, the group given multisensory presentations generated more than 50 percent more creative solutions on a problem-solving test than students who saw unisensory presentations. In another study, the improvement was more than 75 percent! Multisensory presentations are the way to go.
Many researchers think multisensory experiences work because they are more elaborate. Do you recall the counterintuitive concept that more elaborate information given at the moment of learning enhances learning? It’s like saying that if you carry two heavy backpacks on a hike instead of one, you will accomplish your journey more quickly. But apparently our brains like heavy lifting. This is the “elaborative” processing that we saw in the Memory chapter. Stated formally, the extra cognitive processing of information helps the brain integrate the new material with prior information.
One more example of synesthesia supports this, too. Remember Solomon Shereshevskii’s amazing mental abilities? He accurately reproduced a complex formula 15 years after seeing it once. Shereshevskii had multiple categories of (dis)ability. He felt that some colors were warm or cool, which is common. But he also thought the number one was a proud, well-built man, and that the number six was a man with a swollen foot—which is not common. Some of his imaging was nearly hallucinatory. He related: “One time I went to buy some ice cream … I walked over to the vendor and asked her what kind of ice cream she had. ‘Fruit ice cream,’ she said. But she answered in such a tone that a whole pile of coals, of black cinders, came bursting out of her mouth, and I couldn’t bring myself to buy any ice cream after she had answered that way.”
Synesthetes like Shereshevskii almost universally respond to the question “What good does this extra information do?” with an immediate and hearty, “It helps you remember.” Most synesthetes report their odd experiences as highly pleasurable, which may, by virtue of dopamine, aid in memory formation. Indeed, synesthetes often have a photographic memory.
Smell boosts memory all by itself
I once heard a story about a man who washed out of medical school because of his nose. To fully understand his story, you have to know something about the smell of surgery, and you have to have killed somebody. Surgery can assault many of the senses. When you cut somebody’s body, you invariably cut their blood vessels. To keep the blood from interfering with the operation, surgeons use a cauterizing tool, hot as a soldering iron. It’s applied directly to the wound, burning it shut, filling the room with the acrid smell of smoldering flesh. Combat can smell the same way. And the medical student in question was a Vietnam vet with heavy combat experience. He didn’t seem to suffer any aversive effects when he came home. He was accepted into medical school. But then the former soldier started his first surgery rotation. When he smelled the burning flesh from the cauterizer, it brought back to mind the immediate memory of an enemy combatant he had shot in the face, point-blank, an experience he had suppressed for years. The memory literally doubled him over. He resigned from the program the next week.
This story illustrates something scientists have known for years: Smell can evoke memory. It’s called the Proust effect. Marcel Proust, the French author of the profoundly moving book Remembrance of Things Past, talked freely 100 years ago about smells and their ability to elicit long-lost memories. Why? Remember, smell neurons gain VIP access to the amygdala.
Smell has the unique advantage of being able to boost learning directly, without being paired with another sense. That’s because it is an ancient sense, not fully integrated with the rest of the brain’s sensory circuitry but instead closely wired to the emotional learning centers of the brain. In the typical experiment testing the effect of smell on remembering, two groups of people might be assigned to see a movie together and then told to report to the lab for a memory test. The control group takes the test in a plain room. The experimental group takes the test in a room smelling of popcorn. The second group blows away the first group in terms of number of events recalled, accuracy of events recalled, specific details, and so on. In some cases, they can accurately retrieve twice as many memories as the controls.
However, this is true for only certain types of memory. Odors appear to do their finest work when subjects are asked to retrieve the emotional details of a memory—as our medical student experienced—or to retrieve autobiographical memories. You get the best results if the smells are congruent. A movie test in which the smell of gasoline is pumped into the experimental room does not yield the same positive results as the smell of popcorn does.
Odors are not so good at retrieving declarative memory. Smell boosts declarative scores in only a couple of scenarios. It works if you’re emotionally aroused—usually, that means mildly stressed—before the experiment begins. For some reason, showing a film of young Australian aboriginal males being circumcised is a favorite way to do this. And it works if you’re asleep. Researchers used a version of a delightful card game my sons and I play on a regular basis. We use a deck of cards we purchased at a museum, resplendent with 26 pairs of animals. We turn all of the cards facedown, then start selecting two cards to find matches. It is a test of declarative memory. The one with the most correct pairs wins the game. In the experiment, the control groups played the game normally. But the experimental groups didn’t. They played the game in the presence of rose scent. Then everybody went to bed. The control groups slept unperturbed. Soon after the snoring began in the experimental groups, however, the researchers filled their rooms with the same rose scent. Upon awakening, the subjects were tested on their knowledge of where the matches had been discovered the previous day. The control group answered correctly 86 percent of the time. Those exposed and reexposed to the scent answered correctly 97 percent of the time. Brain imaging experiments showed the direct involvement of the hippocampus, that region of the brain deeply involved in memory. Smell, it appeared, enhanced recall during the offline brain processing that normally occurs during sleep.
Smell aside, there is no question that multiple cues, dished up via different senses, enhance learning. They speed up responses, increase accuracy, improve stimulation detection, and enrich encoding at the moment of learning.
More ideas
Multimedia presentations
Over the decades, Mayer has isolated a number of rules for multimedia presentations, linking what we know about working memory with his own empirical findings on how multimedia exposure affects human learning. Here are five of them, as he summarized in his book Multimedia Learning, useful for anyone giving a lecture, teaching a class, or creating a business presentation.
Multimedia principle: Students learn better from words and pictures than from words alone.
Temporal contiguity principle: Students learn better when corresponding words and pictures are presented simultaneously rather than successively.
Spatial contiguity principle: Students learn better when corresponding words and pictures are presented near to each other rather than far from each other on the page or screen.
Coherence principle: Students learn better when extraneous material is excluded rather than included.
Modality principle: Students learn better from animation and narration than from animation and on-screen text.
Sensory branding
Author Judith Viorst once said, “Strength is the capacity to break a [chocolate] bar into four pieces … and then to eat just one of the pieces.” No doubt, smell affects motivation. Can it also affect the motivation to buy?
One company tested the effects of smell on business and found a whopper of a result. Emitting the scent of chocolate from a vending machine, it found, drove chocolate sales up 60 percent. The same company installed a waffle-cone-smell emitter near a location-challenged ice cream shop: It was inside a large hotel and hard to find. Sales soared 50 percent, leading the inventor to coin the term “aroma billboard” to describe the technique.
Welcome to the world of sensory branding. Businesses are beginning to pay attention to human sensory responses, with smell as the centerpiece. For example, Starbucks does not allow employees to wear perfume on company time because it interferes with the seductive smell of the coffee they serve and its potential to attract customers.
Evidence for doing so comes from research by Dr. Eric Spangenberg, dean of the business school at Washington State University. Spangenberg knew from prior work that the male nose responds positively to the smell of rose maroc (spicy floral notes), the female nose to vanilla. What if he pumped rose maroc into the air of the men’s section at a clothing store and vanilla into the women’s section? Spangenberg hit pay dirt, generating twice the sales throughout the store. What if he then flipped the smells, introducing the male-preferred odor to the female section and vice versa? Spangenberg hit pay dirt again: Sales went down. “You can’t just use a pleasant scent and expect it to work,” Spangenberg explained in an interview with Fast Company. “It has to be congruent.”
Smell also can be used to differentiate a brand. Enter any Subway fast-food restaurant blindfolded and you’d instantly know where you were. In choosing a scent to represent your brand, one newspaper article advises, consider the aspirations of your potential buyer. Realtors sometimes employ the smell of freshly baked bread or cookies during an open house to remind buyers of the comforts of home, for example. Also match the odor to the “personality” of the object for sale, the article suggests. For potential buyers browsing an SUV dealership, the fresh scent of a forest or the salty odor of a beach might evoke a sense of adventure more so than, say, the scent of vanilla.
Research shows that the less complex the smell (the fewer interacting ingredients), the more likely it is to drive sales. Simpler smells drive sales 20 percent more than complex smells, or no smells at all.
Smells at work (not coming from the fridge)
I occasionally teach a molecular biology class for engineers, and I decided to do my own little Proust experiment. (There was nothing rigorous about this little parlor trick; it was simply an informal inquiry.) Every time I taught a section on the enzyme RNA polymerase II, I prepped the room by squirting the perfume Brut on one wall. In an identical class in another building, I taught the same material, but I did not squirt Brut when describing the enzyme. Then I tested everybody, squirting the perfume into both classrooms. Every time I did this experiment, I got the same result. The students who were exposed to the perfume during learning did better on subject matter pertaining to the enzyme—sometimes dramatically better—than those who were not. And that led me to an idea. Many businesses have a need to teach their clients about their products, from how to implement software to how to repair engines. For financial reasons, the classes are often compressed in time and packed with information—90 percent of which is forgotten a day later. (For most declarative subjects, memory degradation starts the first few hours after the teaching is finished.) But what if you pair a smell with each lesson, as in my Brut experiment? Teachers could do this for the class as a whole, or you could do it on your own. You could spritz a bit of the scent near your pillow before you go to sleep, too. Overnight, you could not help but associate the autobiographical experience of the class—complete with the intense transfer of information—with the scent. Back at your company, when you need to apply what you learned, you could review your notes in the presence of the smell you encountered during the learning. See if it improves your performance, even cuts down on errors.
Is this context-dependent learning (remember those deep-sea divers from the Memory chapter) or a true multisensory environment? Either way, it’s a start toward thinking about learning environments that go beyond our usual addiction to images and sounds.
Brain Rule #8
Stimulate more of the senses at the same time.
• We absorb information about an event through our senses, translate it into electrical signals (some for sight, others from sound, etc.), disperse those signals to separate parts of the brain, then reconstruct what happened, eventually perceiving the event as a whole.
• The brain seems to rely partly on past experience in deciding how to combine these signals, so two people can perceive the same event very differently.
• Our senses evolved to work together—vision influencing hearing, for example—which means that we learn best if we stimulate several senses at once.
• Smells have an unusual power to bring back memories, maybe because smell signals bypass the thalamus and head straight to their destinations, which include that supervisor of emotions known as the amygdala.
