Figure 6.1 Place theory of pitch perception
Source: From HyperPhysics by permission of Rod Nave.
Studies of animal cognition are often about an evolutionary scala naturae – about the degree to which other animals approximate to mental functioning we know best in ourselves. Do they have a grasp of predication? Are they self-aware? Do they have a sense of the future?
In this entry, I am concerned with a different sort of question. All animals perceive, but they often perceive differently. My assumption is that perception serves nonhuman animals just as well as it does humans. Within the parameters set by this assumption, we can ask: What is the significance of a different way of perceiving? My example is colour, which I contrast with acoustic pitch. I will show that the colours animals perceive are different from those that humans do, but that pitch is the same across the board. I’ll try to explain this and to draw some conclusions about perceptual qualities in general.
Some animals sense novel colours (Thompson 1992, Matthen 1999) – colours that humans don’t perceive. For example, chickens are tetrachromats. They have four different classes of cone cells while humans have only three, three opponent processes while humans have only two (Osorio et al. 1999). (Details and explanations later.) The colours that result are quite different from those that humans experience. These colours are not merely more fine-grained – it’s not just that birds can distinguish by colour some things that look the same to us (though this is also true). Nor is it just that they see beyond the extremes of the spectrum visible to humans – it is true that birds are sensitive to ultraviolet, but this is not my focus here. The point of interest for me here is that bird colours are removed from such determinants of human colour experience as the red-green, blue-yellow hue palette. The colour properties birds see are different from those we see.
What colour does a bird see when it looks at turmeric powder? The point I want to make is that it can’t be orange, which is what trichromatic humans see. Orange is the colour we experience as reddish and yellowish in more or less equal proportions. But the red-green, blue-yellow hue dimensions thrown up by our three cone cells and two opponent processes do not define bird colour-experience.1 (These processes give the bird a richer palette of colour experience; on the flip side, colour deficiencies such as protanopia diminish colour experience in some humans.) And the same goes for other bird colour-experiences. Bird colours are defined by different hue dimensions. Consequently, birds don’t experience orange and other colours that humans do; conversely, humans don’t experience the colours that birds do. (For more on the structure of colour, see Matthen forthcoming.)
A caveat before we continue. I said that bird colour-experience is different from ours, and concluded that birds don’t experience the same colours. There’s a lacuna in this inference: it doesn’t rule out the possibility that birds have different experience of the same colours. Consider motion: when I throw a ball to you, you experience it as approaching and I as retreating. In other words, you and I experience the same motion differently. Could birds and humans, similarly, experience the same colours differently? Let’s hold this possibility in abeyance until the end of Section II, when we will have a bit more apparatus at hand.
Here, however, is a preliminary conclusion:
As an aside, Proposition 1 is a problem for those who would define colour as a disposition to evoke a specific kind of experience. For it invites the question: Why does the kind of experience that birds have count as colour experience? Since the colour experiences in question are different, the question arises: why are novel colours colours at all?
Colours are properties, commonalities among multiple concrete individuals. Orange is a colour that this mound of turmeric powder has; it is also present in that heap of turmeric. This is what it means to say it is a property.
Perception gives us direct visual experience of the colours – “direct” in the sense that we do not see them by seeing something else. Some argue that we see polygons indirectly, because we see them by seeing their sides. Whether this is correct or not regarding polygons, the point about colours is that there is no such intermediary. It is not by construction or learning that we see the colour of turmeric, but just by seeing.
The directness of colour perception has prompted certain well-known doctrines about the nature of colour. G. E. Moore (1903, 10) says that the colours are simple, undefinable qualities known by perception. This is not exactly correct. As I said earlier, colours have component structure: orange consists of the reddish and yellowish hue dimensions in more or less equal parts. In this sense, at least, they are not simple. Still, Moore points to an important truth: the colours cannot be defined except in chromatic terms, i.e., in terms of directly perceived qualities such as hue and lightness.
Bertrand Russell (1912) propounds a view similar to that of Moore. He thought that colours were sense data known by acquaintance. “I know the colour perfectly and completely when I see it, and no further knowledge of it itself is even theoretically possible,” he wrote (47). Russell’s point is that if colour is a perceptual property, then nonperceptual knowledge about it must originate in correlations that we discover. These correlations do not constitute “knowledge of [colour] itself.” Orange is reddish yellow; this is its directly perceived chromatic essence. Orange has a wavelength of around 600 nm; this is a contingently correlated physical fact not knowable by perception alone. We know a colour “perfectly and completely” when we directly perceive its chromatic essence.
The Moore-Russell doctrine stands on a simple foundation: perception is self-contained. We go beyond perception when we investigate physical causes. With colour, perception reveals a system of properties constituted by the hue and lightness. There is nothing else to know about its intrinsic nature.
Mark Johnston (1992) formulates another central doctrine: colour causally explains colour experience. He suggests that this sits uncomfortably with the Moore-Russell doctrine. Scientists causally explain colour experience by reference to physical quantities, such as wavelength or reflectance, or physically realized “dispositions to appear colored” (Johnston 1992: 224). It follows, Johnston says, that colour must be some such quality. But these qualities are not perfectly and completely known by perception (225). So, Johnston concludes, we should “abandon or weaken” the Moore-Russell doctrine (228).
This reasoning rests on a mistaken assumption. We explain colour experience by chromatic properties. For instance, we say that turmeric looks orange (an experience) because (causal explanation) it is orange and white light is shining on it. It is true that science amplifies this explanation in terms of wavelength and the like, but it does not do so by identifying colours with these physical quantities. Science only demands that there be law-like correlations between physical quantities and the colours. Psychophysics provides these correlations. Psychophysics goes beyond mere perception. There is no reason to weaken the Moore-Russell doctrine.
This gives us the argument that we earlier were missing about the novel colours. Perception serves chickens as well as it serves us. So we should assume that they have the same kind of perceptual access to the colours they directly experience as we do to ours – they, too, enjoy perfect and complete grasp of perceptual qualities. But bird colours have a different chromatic character than those that we perceive. Thus, we have:
Note that this argument does not simply go from different experience to different thing experienced. Rather, it uses the Moore-Russell doctrine as an additional premise.
Auditory pitch (high/low) and loudness are also perceptual qualities, this time of audition. The Moore-Russell doctrine implies that we know them by hearing them (though I don’t think we know either pitch or loudness completely2). But as I shall show in this section and the next two, colour and pitch are very different cases. Colour and auditory pitch are both wavelength-related property systems. But our perceptual systems relate to the underlying wavelengths in different ways. The way wavelength sensitivity works in the case of colour is what makes novel colours possible. There is no such thing as novel auditory pitch (at least not in cochlea-based auditory systems). (This should be somewhat qualified. Pitch has attending characteristics that can vary from organism to organism. My point is that its correspondence to wavelength cannot vary.)
Here’s how the auditory system measures wavelength. Sound waves are converted to pressure waves in the fluid of the middle ear; these are conveyed to the cochlea of the inner ear. According to Georg von Békésy’s now widely accepted “place theory” (http://tinyurl.com/okumo), different parts of the cochlea (and of its basilar membrane) are sensitive to different sound frequencies. (See Figure 6.1.) Taken as a whole, the cochlea is a coiled organ; each place on this organ corresponds to a frequency, and the strength of response at this place indicates the amplitude of the corresponding frequency. Thus, the cochlea is, as the jargon goes, “tonotopic”; its places correspond to tones. Cochlear hair cells transduce this information and send it through a number of subcortical relay stations through to the auditory cortex. The primary auditory cortex is likely not concerned with tone as such, but rather with characteristics of sound that enable us to identify environmental events of different kinds. Nevertheless, it does retain some of the tonotopic organization of the cochlea and reflects the wavelength composition of ambient sound. (For a recent review, see King and Nelken 2009.)
Our experience of pitch and volume derives from this mechanism. The auditory system and the speech perception system extract detailed information from the incoming acoustic signal. From the single amalgamated signal, they reconstruct the spatial array of individual sources of sound. For example, they separate out the many voices and other sounds at a cocktail party, and differentiate human voices from other kinds of sounds such as those of people walking about and wine glasses tinkling against one another. But whatever else we might experience in audition, we also perceive pitch and volume. It is not clear that the pitch and volume that we hear corresponds cleanly to the composition of sounds; for instance, one voice may be more high-pitched than another, even though both consist of many tones. Nonetheless, we perceive environmental sources of sounds as being, among other things, high or low and loud or soft.
Pitch and loudness are directly perceived in the way that Moore and Russell say colour and brightness are (though in this case the resulting knowledge is incomplete – see note 2). We know what we know of them by hearing them, and no other source of knowledge of them themselves is even theoretically possible.
Here, however, is something audition doesn’t tell us: because pitch and loudness trace back to the tonotopic organization of the cochlea, their auditory inter-relations mirror the physical structure of frequency and volume.3 This representational structure is dictated by the cochlea. (It is possible, in some sense, for auditory perception not to be cochlea based, but the resulting qualities would have a very different perceived character than pitch and loudness.)
This imposes a constraint on “novel” pitch and volume. Some animals may experience finer gradations of pitch; some may sense higher or lower frequencies than humans can. However that might be, high/low and loud/soft always correspond in the same way to the physical parameters of frequency and volume in sound. The physical structure of sound constrains the perceptual structure of auditory qualities. As we shall see, this marks a contrast between colour vision and audition.
Now let’s return to colour and the visual system. Colour is extraordinarily vivid to those of us who see it, and some think that it is the basic visual datum from which all visual objects are constructed – “If it be perceiv’d by the eyes, it must be a colour,” Hume (1739/1978: 16) wrote. Whatever Hume might have meant and however he might be vindicated, we should beware of assuming that colour is functionally primary for vision. The business of vision is space; colour is an add-on. And this has important implications for how colour is processed by the visual system.
Because vision is primarily for space, the retina is topotopically organized; each place on the retina corresponds to a place in the two-dimensional image produced by the corneal lens. Colour sensitivity would be improved by adding cone-cells, but this reduces spatial resolution, since more cells have to be packed into each location. Birds are tetrachromats; humans are normally trichromats – in each case, the limitations on colour resolution are an evolutionary compromise between the needs of spatial vision and those of colour vision.
This brings us to the cone-cells. Animals with colour vision – honeybees, birds, humans, etc. – possess several photoreceptor cell types, each differentially sensitive to light in a subregion of the visual spectrum. Figure 6.2 shows the spectral sensitivity of the three human cone-cells. Each place in the colour-sensitive portion of the retina contains these three cone-cells.
The cone-cells are sensitive to broad wavebands. The L (or long wavelength) cone emits a response when light of just about any frequency is incident upon it, but if we pick a threshold of 25% of peak response, then it responds to light between 500 and 650 nm, or about half of the visible spectrum. The output of each cone-cell is proportionate to the integrated sum of signal strength at each wavelength multiplied by cone-cell sensitivity at that wavelength. Cone-cells are distinguished one from another by a response curve more than by peak sensitivity. The M-cone is only slightly shifted from the L-cone, as it results from a genetic modification of the latter. Kainz et. al 1998 and Surridge et al. 2003 are good reviews.)
The output of the cone-cells is not wavelength-specific in the way that the output of the cochlea is. With regard to environmental discrimination, this has the negative consequence that there are signals that are different in wavelength composition, but nonetheless equivalent with respect to their effect on the cone-cells. For example, since light of 525 nm and 625 nm both affect the L-cone, we can get the same L-response by manipulating strength at one frequency to compensate for changes at the other. This kind of equivalence among distinct light signals is called “metamerism.”
Though wavelength discrimination is limited by metamerism, it is surprisingly fine. Consider two monochromatic beams of light at 575 and 525 nm, the second twice as strong as the first. These beams will have the same effect on the L-cone, since the strength of the second beam compensates for the lower response-sensitivity of this cone at 525 nm. However, the two beams have a completely different effect on the M-cone, which has roughly the same sensitivity at these two wavelengths, and thus responds much more strongly to the stronger 525-nm beam. Thus, the two beams are distinguished by their aggregate effect on the three cones.
Figure 6.2 Response curves of human cone-cells
Source: From the Wikipedia article on cone-cells under the Creative Commons licence.
In this manner, take the ordered triple of the three cone-cell responses, <x, y, z>. The responses to two signals is different if they produce a different effect on even one cell. In consequence, given just three cones, discrimination for monochromatic beams is close to perfect over a long-enough series of “same-different” discrimination trials (Hardin 1988). Cone-cell activation is always different for different monochrome lights. (Figure 6.3 maps these differences for equal-strength beams.)
The points made in the immediately preceding paragraphs fit oddly with each other. On the one hand, cone-based colour vision is capable of making a very large number of distinctions: over a long-enough series of discrimination trials, light of any given wavelength is discriminable from light of any other at equal strength. On the other hand, because of metamerism, colour vision fails to discriminate signals with very different wavelength-amplitude profiles. You could put it this way: there are very many colours – in some ways of looking at it, an infinite number of them – but in any given illumination, there are things of different spectral composition that look the same. The colours are a mash-up. Figure 6.3 makes it look as if colour difference is wavelength difference, but this is only because of background constraints.
Figure 6.3 Colour and the visible spectrum
Source: By User: PAR (Own work) (Public Domain), via the Wikimedia Creative Commons Licence.
Before we can figure out the significance of Proposition 4, we need to look a little deeper into the operations of the colour vision system.
As explained in the preceding section, a light beam hitting the colour-sensitive portions of the retina will elicit a response from cone-cells of each of three types. These three response values jointly define what one might call the system’s first colour response (FCR). Colour experience is extracted from the FCR by the operation of two functionally different processes, opponent processing and constancy processing.
Light reflected off natural objects is generally polychromatic, though not balanced across the spectrum. Consequently, it stimulates all three cones. A ripe tomato, for example, stimulates the L-cone more than the S-cone. (Red light is long wavelength; blue is short.) But this does not mean that it doesn’t stimulate the S-cone. Though it reflects 70% of the red light that falls on it, it still reflects 10% of the blue. In bright light, then, the red tomato might actually stimulate the S-cone more than a blue object does in dim light.
FCR reflects both the spectral composition and the brightness of light reflected off or emitted by environmental objects. Because of brightness, the three values tend to be correlated: bright polychromatic light produces a strong response from all; dim light evokes a weak response. So to gauge the tomato’s colour, as opposed to the brightness of the illumination, the important thing is not the response elicited from each cone, but the difference between cone-cell responses. In any level of light bright enough to elicit any cone-cell response, white light reflected off the tomato will elicit a stronger response from the L and M (long and medium wavelength) cones than from the S. To detect this, the visual system subtracts the S-response from the sum of the L- and M-responses. It also calculates the difference between the L- and M-cone responses. This is called “opponent processing.”
The hue dimensions – red-green and blue-yellow – correspond to these differences in cone-cell responses. For instance, something looks reddish if it elicits more response from the L- and M-cones than it does from the S-cone; it looks yellowish if the difference between the L- and M-cones is positive. It is important to note that opponent processing does not add information to the cone-cell responses; it merely extracts information that is already present. The function of opponent processing is to remove the correlated portions of cone-cell response, which are due to brightness, so as to arrive at the uncorrelated portion, which is due to wavelength.
One last fact: the two hue dimensions of colour derive from the opponent channels; brightness derives from the sum. Experienced colour is hue plus brightness.
The same object reflects different colours when differently illuminated. The visual system is, however, able to reduce the effect of these changes. Objects look more or less the same colour and lightness in good viewing conditions, i.e., when the illumination is more or less white and reasonably bright. Processing for this kind of constancy makes certain “assumptions” – for example, that the brightest object in a scene is white. These assumptions arise from successful processing strategies pursued in evolutionary time. Thus, there is added information in constancy processing (for example, the “information” that the brightest object in any scene is white).
One can think of the output of opponent processing in two ways.
One way is to think of it as a valiant attempt at wavelength readout, given not very good apparatus. Figure 6.3 suggests this picture: in that diagram, the <x,y> value of 700 nm is <0.74, 0.27>. So, one might think: if something matches this point, it looks 700-nm red, though the look might merely be metameric and thus misleading.
But this gets it backwards. The colours are calibrated by the <x, y> coordinates of Figure 6.3 (and an additional dimension that need not concern us here). (There are other systemizations of colour experience as well; for present purposes, the differences are not important.) Monochrome beams of light and other light beams can be placed in the <x, y> matrix; light of 700 nm is red. But it is the <x, y> matrix that is directly given; it marks the colour properties we perceive directly.
Here is a better way of thinking about colour properties. They are simply the values that opponent processing computes. Or to put it differently, each colour-vision system, with its proprietary system of cones and opponency functions, generates its own system of colour properties. This, I take it, is what lies behind Byrne and Hilbert’s (2003) characterization of colour representation:
Objects are represented as having proportions of “hue” magnitudes … if an object is perceived as orange, then it is represented as having a value of R that is approximately 50 percent of its total hue, and similarly with Y … If [it] is perceived as purple, it is seen as having R and B in a similar proportion.
(Byrne and Hilbert 2003, 14)
Byrne and Hilbert define the variables R, Y, etc., in terms of opponent values. The details are unimportant here, but let’s just say that the red-green, blue-yellow, and black-white dimensions of colour are hue “magnitudes” defined by functions of L-, M-, and S-cone responses. Colour properties are just triples of these magnitudes. Simplifying greatly: colour properties are functions of the three cone responses. Different animals (and perhaps even different individual humans) use differently tuned cones and different opponent functions, resulting in individually tailored colour schemes. This is what makes novel colours possible.
Think back to audition; when we perceive pitch and loudness, we directly perceive structural properties of frequency and volume. Byrne and Hilbert are proposing, in effect, that when we perceive colour, we directly perceive a property defined in terms of cone-cell response differences. Notice that by their proposal, they are committed to bird colours being different from human trichromat colours. This is, I believe, a revolutionary consequence. (They take their proposal in a physicalist direction, but I won’t go into that here.)
To clarify, looking a certain colour is not a matter of evoking a certain cone response. For as we noted earlier, constancy processing adjusts apparent colour so that it is more or less constant in varying illumination conditions. Consider a pale yellow object like a lemon. When it is under leaf colour, the light falling on it is tinged with green; when it is under a bright blue sky, the light falling on it is yellow-deficient; at sunset, it is irradiated with red-tinged light. Consequently, the light received from the lemon will vary from greenish under the tree, to darker under the blue sky, to more orange at sunset. In these different conditions, the lemon has a different effect on the cone cells. Constancy processing modifies and adjusts the cone-cell response so that it varies less than the light received changes. It looks more or less the same shade of yellow in all three conditions.
A closing remark. I have said what it is to appear a certain colour, but what is it to be a certain colour? Something that looks orange is orange if it actually is the colour it looks to be. But what is it to be this colour in actuality, as opposed to merely looking that way? I would urge that this is not perceptually given. Being a certain colour is to look that colour in circumstances C: for instance, in white light or in good light, etc. But surprising facts have been uncovered about colour appearance: things look different when in certain situations of contrast, or when viewed obliquely, etc. We learn how to make colour judgements, but artists and scientists can still surprise us.
Is turmeric human-orange or bird-orange (where the latter is the name I give to whatever colour a bird sees when it looks at turmeric)? The question is based on the idea that we have to choose. Perhaps we would have to choose if colour were defined in terms of an external quality such as wavelength; then different colour systems would arguably be incompatible. But we have seen that colour doesn’t do this. The colour properties humans see are defined by the chromatic dimensions that arise from their cone-cells and opponent functions; the colour properties birds see are similarly defined. These are simply different ways to partition the domain of surfaces. Turmeric is rightly classified as human-orange by humans and as bird-orange by birds. Human and bird visual systems have different ways of partitioning surfaces.
Sensory systems assign worldly objects to categories. There are two kinds of constraints on this function.
First, sensory systems must assign relevantly similar objects to the same categories. A coloured object sends different light to the eye in different conditions of illumination, and the visual system gets it wrong if it does not generate the same experience in all of these conditions. Getting this right is the function of constancy processing. (See section V above.)
The second kind of constraint has to do with the correspondence between sensory properties and the physical world. As we saw, high and low pitch corresponds to high and low acoustic frequency. An auditory system would be nonfunctional if it violated this correspondence. We have seen that colour does not correspond to physical properties in the way that pitch does. Consequently, it should not be thought of as violating its function if there is a failure of correspondence. Bird colours are just as much out of correspondence with physical wavelength-related properties as human colours are. But neither of these perceptual categories are nonfunctional as a result. As we said at the outset, perception serves animals differently from humans, but equally well.
1 It’s important to remember here that we are not talking about mixing pigments or filters, but rather about phenomenal components of colour experience. Orange is experienced as a mixture of reddishness and yellowishness. It is also important to distinguish between the colour, red, and the hue dimension, reddishness. Orange is reddish, but not red; in the Swedish Colour System, it is 50% R and 50% Y.
2 Presumably, only those with “perfect pitch” know it completely by hearing. Most of us hear pitch (and also loudness) only relationally – we don’t have a perceptual schema for all pitch qualities, within which we can place notes that we hear. Arguably, trichromats have “perfect colour,” or something close to it – we do have a perceptual schema for colour. (The Munsell and Natural Colour System are attempts to capture this schema; see Matthen forthcoming.) This is a difference between the two cases that I won’t explore further here.
3 There is constancy processing in sound analogous to that described in Section V for colour. Consequently, the auditory system may use pitch cues to adjust loudness and vice versa. This doesn’t add new structure to auditory properties; it is rather an adjustment of how particular sounds are assigned to those properties.
For more about colour vision in other animals, see the classic work by G. H. Jacobs, Comparative Color Vision (New York: Academic Press, 1981), and also J. F. Nuboer, “A Comparative View of Colour Vision” (Netherlands Journal of Zoology 36, 1986: 344–380), as well as Osorio et al. 1999. Important discussions of the evolution of colour vision are found in B. C. Regan et al., “Fruits, Foliage, and the Evolution of Colour Vision” (Philosophical Transactions of the Royal Society of London B 356, 2001: 229–283) and N. J. Dominy and P. W. Lucas, “Ecological Importance of Trichromatic Vision in Primates” (Nature 410, 2001: 363–366), as well as Surridge et al. 2003. A superb account of colour structure is found in H. Arnkil, Colours in the Visual World (Helsinki: Aalto ARTS Books, 2013), Chapter 7. Philosophical discussion of these issues is found in M. Matthen, Seeing, Doing, and Knowing (Oxford: Clarendon Press, 2005).
Byrne, Alex, and David Hilbert (2003) “Color Realism and Color Science,” Brain and Behavioral Sciences 26: 3–64.
Hardin, C. L. (1988) “Phenomenal Colors and Sorites,” Noûs 22: 213–234.
Hume, D. (1739/1978) A Treatise of Human Nature. L. A. Selby-Bigge and P. H. Nidditch (eds.) New York: Oxford University Press.
Johnston, Mark (1992) “How to Speak of the Colours,” Philosophical Studies 68: 221–263.
Kainz, Pamela M., Jay Neitz, and Maureen Neitz (1998) “Recent Evolution of Uniform Trichromacy in a New World Monkey,” Vision Research 38: 3315–3320. doi:10.1016/S0042-6989(98)00078-9
King, Andrew J., and Israel Nelken (2009) “Unraveling the Principles of Auditory Cortical Processing: Can We Learn From the Visual System?” Nature Neuroscience 12: 698–701. doi: 10.1038/nn.2308
Matthen, Mohan (1999) “The Disunity of Color,” Philosophical Review 108: 47–84.
Matthen, Mohan (forthcoming) “Unique Hues and Colour Experience,” Routledge Handbook of the Philosophy of Colour.
Moore, G. E. (1903) Principia Ethica Cambridge: Cambridge University Press.
Osorio, D., M. Vorobyev, and C. D. Jones (1999) “Colour Vision of Domestic Chicks,” Journal of Experimental Biology 202: 2951–2959.
Russell, Bertrand (1912) The Problems of Philosophy Oxford: Oxford University Press.
Surridge, Alison K., Daniel Osorio, and Nicholas I. Mundy (2003) “Evolution and Selection of Trichromatic Vision in Primates,” Trends in Ecology and Evolution 18: 198–205. doi:10.1016/S0169-5347(03)00012-0
Thompson, Evan (1992) “Novel Colors,” Philosophical Studies 68: 321–349.