Sensory Interaction
We have seen that vision and kinesthesia interact. Actually, none of our senses act alone. All our senses—seeing, hearing, tasting, smelling, touching—eavesdrop on one another, and our brain blends their inputs to interpret the world (Rosenblum, 2013). This is sensory interaction at work. One sense can influence another.
Consider how smell sticks its nose into the business of taste. Hold your nose, close your eyes, and have someone feed you various foods. A slice of apple may be indistinguishable from a chunk of raw potato. A piece of steak may taste like cardboard. Without their smells, a cup of cold coffee may be hard to distinguish from a glass of Gatorade. A big part of taste is right under your nose.
Thus, to savor a taste, we normally breathe the aroma through our nose. Like smoke rising in a chimney, food molecules rise into our nasal cavity. This is why food tastes bland when you have a bad cold. Smell can also change our perception of taste: A drink’s strawberry odor enhances our perception of its sweetness. Even touch can influence our taste. Depending on its texture, a potato chip “tastes” fresh or stale (Smith, 2011). Smell + texture + taste = flavor. Yet perhaps you have noticed: Despite smell’s contribution, flavor feels located in the mouth, not the nose (Stevenson, 2014).
Vision and hearing may similarly interact. A weak flicker of light that we have trouble perceiving becomes more visible when accompanied by a short burst of sound (Kayser, 2007). The reverse is also true: We can hear soft sounds more easily if they are paired with a visual cue. If I [DM], as a person with hearing loss, watch a video with on-screen captions, I have no trouble hearing the words I am seeing. But if I then decide I don’t need the captions and turn them off, I will quickly realize I do need them. The eyes guide the ears (Figure 21.7).
Figure 21.7 Sensory interaction
Seeing the speaker forming the words, which Apple’s FaceTime video-chat feature allows, makes those words easier to understand for hard-of-hearing listeners (Knight, 2004).
So our senses do not function in isolation; they interact. But what happens if they disagree? What if our eyes see a speaker form one sound but our ears hear another sound? Surprise: Our brain may perceive a third sound that blends both inputs. Seeing mouth movements for ga while hearing ba, we may perceive da. This phenomenon is known as the McGurk effect, after Scottish psychologist Harry McGurk, who, with his assistant John MacDonald, discovered the effect (1976). For all of us, lip reading is part of hearing.
We have seen that our perceptions have two main ingredients: Our bottom-up sensations and our top-down cognitions (such as expectations, attitudes, thoughts, and memories). In everyday life, sensation and perception are two points on a continuum. It’s not surprising, then, that the brain circuits processing our physical sensations sometimes interact with brain circuits responsible for cognition. The result is embodied cognition. We think from within a body. Some examples from playful experiments:
- Physical warmth may promote social warmth. After holding a warm drink rather than a cold one, people were more likely to rate someone more warmly, feel closer to them, and behave more generously (IJzerman & Semin, 2009; Williams & Bargh, 2008). Have hot tea with José and iced tea with his identical twin Juan, and you may perceive José to be the warmer brother.
- Social exclusion can literally feel cold. After being given the cold shoulder by others, people judged the room to be colder than did those who had been treated warmly (Zhong & Leonardelli, 2008).
- Judgments of others may also mimic body sensations. Sitting at a wobbly desk and chair makes others’ relationships, or even one’s own romantic relationship, seem less stable (Forest et al., 2015; Kille et al., 2013).
As we attempt to decipher our world, our brain blends inputs from multiple channels. But in a few select individuals, the brain circuits for two or more senses become joined in a phenomenon called synesthesia, where the stimulation of one sense (such as hearing sound) triggers an experience of another (such as seeing color). Early in life, “exuberant neural connectivity” produces some arbitrary associations among the senses, which later are normally—but not always—pruned (Wagner & Dobkins, 2011). Thus, hearing music may activate color-sensitive cortex regions and trigger a sensation of color (Brang et al., 2008; Hubbard et al., 2005). Seeing the number 3 may evoke a taste or color sensation (Newell & Mitchell, 2016). Those who experience such sensory shifts are known as synesthetes.
* * *
For a summary of our sensory systems, see Table 21.2.
| Sensory System | Source | Receptors | Key Brain Areas |
|---|---|---|---|
| Vision | Light waves striking the eye | Rods and cones in the retina | Occipital lobes |
| Hearing | Sound waves striking the outer ear | Cochlear hair cells (cilia) in the inner ear | Temporal lobes |
| Touch | Pressure, warmth, cold, harmful chemicals | Receptors (including painsensitive nociceptors), mostly in the skin, which detect pressure, warmth, cold, and pain | Somatosensory cortex |
| Taste | Chemical molecules in the mouth | Basic taste receptors for sweet, sour, salty, bitter, and umami | Frontal temporal lobe border |
| Smell | Chemical molecules breathed in through the nose | Millions of receptors at top of nasal cavities | Olfactory bulb |
| Body position—kinesthesia | Any change in position of a body part, interacting with vision | Kinesthetic sensors in joints, tendons, and muscles | Cerebellum |
| Body movement—vestibular sense | Movement of fluids in the inner ear caused by head/body movement | Hair-like receptors in the ears’ semicircular canals and vestibular sacs | Cerebellum |
* * *
To feel awe, mystery, and a deep reverence for life, we need look no further than our own perceptual system and its capacity for organizing formless nerve impulses into colorful sights, vivid sounds, and evocative smells. As Shakespeare’s Hamlet recognized, “There are more things in Heaven and Earth, Horatio, than are dreamt of in your philosophy.” Within our ordinary sensory and perceptual experiences lies much that is truly extraordinary—surely much more than has so far been dreamt of in our psychology.