How does the brain represent our sensory world in order to serve as the basis for perception? This is one of the oldest questions in philosophy and a central question for modern psychology and neuroscience. It is also central for neurogastronomy.
Modern neuroscience has shown that other sensory systems construct spatial representations of their stimuli. The body surface is represented by a body map, also called a homunculus. The visual world is represented by visual images. In audition, different frequencies are represented in spatial “tonotopic” maps. In all these cases, the spatial representations start with the sheet of sensory receptor cells and are maintained up to the cerebral cortex, where perception occurs.
Research over the past half-century has shown that the brain also represents smell molecules by spatial patterns. These patterns are not present in the receptor cell responses we have just considered, but are formed in the first relay station, the olfactory bulb. This is the core concept in our theory of smell, because these patterns function as what we call smell images for further processing by the brain as the basis for smell perception. It may seem strange that a nonspatial stimulus received by the receptors in the nose, as we have just seen, can be represented as a spatial pattern specific for that odor molecule. The way this comes about has been one of the best-kept secrets of the brain.
What is the advantage of forming a neural image of the information carried in smell molecules? The idea of a smell image seems at first so strange that it is difficult to answer this question. It will be an advantage to take a brief digression and learn what is known about the formation of a “visual image,” the most obvious image that we form of our sensory world.
Legions of neuroscientists and psychologists have studied how this happens in the brain. Our hypothesis is that by studying the neural mechanisms for setting up and processing a visual scene in the brain, we will learn principles that will apply to the neural mechanisms for setting up and processing a smell image in the brain. If this hypothesis holds water, we will gain a new perspective on how smells are represented in our brains and how these representations contribute to the perception of flavor.
You might assume that the best insights into how humans form visual images would come from studying humans themselves. However, as in many fields of biology, the best strategy when starting on a problem such as this is to find an animal in which the particular system exists in a simple form, so that experiments can be carried out that would not be possible in humans.
Discovering Lateral Inhibition
Such a system is found in the eye of the horseshoe crab (Limulus polyphemus). If you live near the ocean or vacation on a beach or visit an aquarium, you may see one of these creatures, like a low inverted bowl in the sand or in the shallow water. Look closely and you can just make out two small eyes in its hard shell. These simple eyes can detect only light or dark or shadowy moving objects, but that is enough to tell the crab the general time of day and warn it of predators in its neighborhood.
This creature does not seem to promise much insight into human vision, but it did to a biologist named H. Keffer Hartline, who worked at the Rockefeller Institute in New York City in the mid-twentieth century. Hartline really believed in the simple system approach. As quoted in “Horseshoe Crabs and Vision,” he used to tell his students who wanted to study the neural mechanisms of vision, “[A]void vertebrates because they are too complicated, avoid color vision because it is much too complicated, and avoid the combination because it is impossible.” Thanks to Hartline and many subsequent scientists, we have learned general principles about vision that are surprisingly relevant to the principles underlying other sensory systems, including smell.
Without going into the details of the receptor cells, we can summarize by saying that each one is contained in a microcartridge that takes in a small part of the visual scene. The combined array should therefore give a faithful representation of the pattern of light falling on the eye. Hartline tested this by moving a bright spot of light halfway over the eye, then abruptly changing it to a lower intensity in the other half. The responses were accordingly strong and weak. He then repeated the experiment, but instead stimulating simultaneously the two halves of the retina. Now he observed a dramatic effect: at the light–dark border, the cells responded more than expected on the light side and less than expected on the dark side. The neural image of this simple pattern did not represent the real pattern at all.
In summary, how strongly a cell fires depends on how active its neighbors are: a strong cell gets stronger, and a weak cell gets weaker.
Contrast Enhancement in Space and Time
This effect had actually been described in humans by a German physicist named Ernst Mach in the nineteenth century. He had noticed that when we view a light–dark border, such as a sharp boundary between two walls with different illumination, the contrast is enhanced by a lighter band on the light side and a darker band on the dark side. These came to be called Mach bands. You can see them yourself if you look for them. (The bibliography provides a site for you to look them up on the Internet.)
Hartline showed that Mach bands are present even in the primitive eye of Limulus. He further showed the mechanism that produces them: lateral inhibitory connections between the receptor cells. Through these connections, the strongly excited cells at the border more strongly inhibit the weakly stimulated cells, and the weakly stimulated cells more weakly inhibit the strongly excited cells. The mechanism is called lateral inhibition. The effect is called contrast enhancement, because the difference between the light and dark areas is enhanced at their boundary. In a general sense, contrast enhancement also is a kind of feature extraction, the enhanced response to specific spatial features in a visual scene.
This is contrast enhancement in space. Hartline’s laboratory also showed that there is contrast enhancement in time. When there is an abrupt step increase in illumination, a single cell responds with a large increase in impulse firing, which rapidly declines to a steady level somewhat higher than before. The overshoot in impulse frequency is called the phasic response, in contrast to the steady tonic response. It shows that the nervous system is sensitive primarily to a change in the environment rather than to an unchanging steady input. This contrast enhancement in time is the counterpart to contrast enhancement in space. After the initial increase in stimulation, lateral as well as self-inhibition comes on to counterbalance the higher level of steady stimulation.
Forming Images in the Brain
In addition to contrast enhancement and feature extraction, lateral inhibition has other functions. The visual image is blurred from the dispersion of light as it passes through a lens; lateral inhibition reduces this dispersion, a process called image reconstruction. Another function is gain control—that is, the adjustment of how much amplification there is for increases or decreases in sensory stimulation. For example, Limulus responds over a wide range of light intensities. To cover this wide range and still have mechanisms that enhance sensitivity for very weak stimuli, there must be a reduction in sensitivity as stimulation gets stronger. This process is called gain compression. Gain compression is built into electronic amplifiers through feedback suppression in a manner analogous to feedback and lateral inhibition in Limulus.
So the neural image formed by the eye is not a faithful image of the actual scene the way a camera image obtained at low contrast is. It is instead an abstracted image, a high-contrast image, in which edges in the visual scene are abstracted and enhanced and the nonchanging rest of the field is suppressed. Similarly, the stimulation is highest when a new scene appears, and it settles down if there are no changes. This is why a horseshoe crab is sensitive to a shadow or an edge of light or dark that moves, so that it can be alert to predators or prey.
Lateral inhibition has come to be recognized as one of the most important principles in the organization of sensory systems. The findings in Limulus were confirmed in the mammalian eye by a remarkable scientist, Stephen Kuffler. He was a refugee from Austria early in World War II, escaping by skiing over the Alps. Later, at Johns Hopkins University, inspired by Hartline’s results in Limulus, he set up similar experiments in the cat retina, recording from single ganglion cells—the cells that carry the output of the retina to the brain—while stimulating the retina in different places with a spot of light. Stimulation at the site of recording (in the center of the receptive field) would often excite the cell, whereas stimulation further away (in the “surround”) would inhibit it, or the other way around: central inhibition and surround excitation. These experiments showed that the principle of contrast enhancement applies to mammalian vision, in a form that came to be called center-surround antagonism.
Image Processing by Animals and Machines
When you get an eye examination and are asked to see letters of decreasing size, how well you do depends on the lens of your eye, which can be corrected with eyeglasses. It also depends on lateral inhibition, which heightens the contrast between the dark letters and the white background. This kind of visual scene, however, is highly artificial. Animals in the wild normally experience visual scenes full of objects at low contrast; survival depends on identifying prey or predator within those scenes. How the retina does it will give us clues to how the nose performs its task of identifying odors within low-contrast odor backgrounds, whether they are prey or predator odors in the environment or volatile food odors in the mouth.
Peter Sterling and Jonathan Demb, visual scientists at the University of Pennsylvania, explain how this works in The Synaptic Organization of the Brain. Creating the optical image of a high-contrast scene requires only light that is of relatively low intensity; this is why we can read books even by candlelight. However, for the retina to create an image of a low-intensity scene requires lots of light. An example would be the natural scene of a sheep in a forest as might be viewed by a predator at dusk. A photometer scan across a photograph of this scene shows the small fluctuations in intensity that occur on a relatively even background. Identifying these fluctuations in dim light is difficult. Why?
To answer this question, Sterling and Demb quote Albert Rose, an expert in video engineering, who compares the retina to a black canvas on which the individual quanta of light (photons) paint a kind of pointillist picture—that is, photons hitting single pixels of the picture. To make a high-contrast picture by turning on or off single pixels requires only approximately one photon per pixel, so it can be done in dim light, but making a low-contrast picture with many shades of gray requires gradations among many photons hitting many pixels. Thus, seeing clearly a low-contrast picture requires a lot of light, as we all have experienced.
Lateral inhibition in the retina, by the means we have discussed, is fundamental to increasing the contrast. It is also needed to combat noise. The more shades of gray exist, the more photons are needed and the noisier the picture. The problem then is to increase the signal-to-noise ratio, a fundamental function in all sensory systems (including smell). One way to increase the ratio is to have synchrony in the center of the receptive field that smoothes out irregularities; this is done in vision by having electrical connections between the photoreceptors that pool many receptor responses together. In smell there are other mechanisms, as we shall see. The other way is to use the lateral inhibitory mechanism to reduce redundancy. Much of a low-contrast natural scene has a relatively constant level of illumination. Much of the information in it is therefore redundant and can be removed, leaving the local sites of change more enhanced.
In the field of image processing, this is called predictive coding, because values near the center are predicted to be higher than the mean of the surround. As the illumination grows dimmer, the surround maintains its predictive value by broadening, as shown by experiment and also by theory.
Vision thus provides us with the most thorough analyses of the properties of image building and image processing in a sensory system. Although the details of the systems may vary, the underlying principles are quite general. We will see that they give new insights into how smell images are constructed and processed to provide the basis for smell perception.
Feature Extraction in Our Sensory World
In summary, the eye sets up a two-dimensional representation of the visual world—a visual image. The advantage of this representation is that the nervous system can form circuits that process the image. The processing involves a fundamental role of lateral inhibition and contrast enhancement that transforms the neural image into a form that is most appropriate to the operations of the brain in building visual perceptions and beyond. Mach himself explained this best:
[S]ince every (retinal) point perceives itself, so to speak, as above or below the average of its neighbours, there results a characteristic type of perception. Whatever is near the mean of the surround becomes effaced. Whatever is above or below is brought into disproportionate prominence. One could say that the retina schematises and caricatures. The teleological significance of this process is clear in itself. It is an analogue of the abstraction and the formation of concepts.
All the principles I have discussed—the initial image representation in a two-dimensional sheet, lateral inhibition, contrast enhancement, temporal transients, center-surround inhibition, and feature extraction—play essential roles in the formation of neural images in all sensory systems. In hearing, each nerve fiber from our ear that carries information has a “best” sound frequency. Lateral inhibition between fiber pathways helps sharpen that frequency response. In touch, our ability to discriminate between two closely spaced points (known as two-point discrimination) is better in our finger tips than on our abdomens. This is because of the higher density of innervation of the fingertip skin and the presence of lateral inhibition in the central pathways, which improves the discrimination. Similarly, by manipulating food in our mouths with our tongue, we carry out “feature extraction” on the mouth-sense of the food—whether it is smooth or rough, dry or moist, hard or soft, and so on.
And smell? We are now in a position to consider how all these principles apply to the representation of smells in our brains.