Motivational Assumptions of Information Choice Theory
There are three key motivational assumptions of information choice theory. First, experts seek to maximize utility. Thus, the information-sharing choices of experts are not necessarily truthful. Second, cognition is limited and erring. Third, incentives influence the distribution of expert errors. I take up each point in turn.
Experts Seek to Maximize Utility
The information-choice assumption that experts seek to maximize utility is parallel to the public-choice assumption that political actors seek to maximize utility. I have used the word “seek” to avoid any suggestion that experts must be modeled as “rational.”
The assumption of utility maximization should not be given a narrowly selfish meaning. Agents who seek to maximize utility may seek praise or praiseworthiness as well as material goods. Truth may often be an element in the utility function, though it will usually be subject to tradeoffs with other values. While I think truthfulness is a value that should generally be treated as like any other, I do not wish to deny that some experts will find ways to constrain themselves to a corner solution. Hausman and McPherson (1993, p. 685, n. 21) relate an apposite story about Lincoln’s response to a man who tried to bribe him. (They report that they could not trace the origin of the story.) “Lincoln kept brushing him off genially and the briber kept increasing his price. When the price got very high, Lincoln clouted him. When asked why he suddenly got so aggressive, Lincoln responded – because you were getting close to my price!” On the other hand, we also have instances in which truth seems to be quite absent from an expert’s utility function, as in the fraudulent forensic science of Fred Zain, who simply lied in court about tests he had never performed (Giannelli 1997, pp. 442–9). Abe Lincoln and Fred Zain remind us that different experts will have different utility functions, in some of which truthfulness will have a higher marginal value than in others.
The assumption of utility maximization has created controversy for public choice theory. That theory has been criticized for assuming “egoistic rationality” (Quiggin 1987). There is some truth to the claim that public choice theorists have assumed people are selfish in some sense. For example, Dennis Mueller says “The basic behavioral postulate of public choice, as for economics, is that man is an egoistic, rational, utility maximizer” (Mueller 1989, pp. 1–2). Mueller’s 1986 presidential address to the Public Choice Society, however, already called on public choice theorists to draw on “behaviorist psychology” to account for the observation that people seem to cooperate more in the prisoners’ dilemma than “rational egoism” would predict. Since then, public choice and economic theory in general have moved toward more nuanced theories of individual action and motivation. In their 1997 “critical survey” of public choice, Orchard and Stretton say: “The assumption of exclusively rational, egoistic, materially acquisitive behaviour is now acknowledged to be unhelpful and to generate inadequate explanations of political performance” (p. 423). While behaviorist psychology has not figured prominently in the move toward a more nuanced image of human action, economists and public choice theorists are drawing from other parts of modern psychology, including cognitive psychology (Frohlich and Oppenheimer 2006) and evolutionary psychology (Congleton 2003).
The move away from egoistic rationality is a move back to the original core position of Buchanan and Tullock. In The Calculus of Consent, they say quite explicitly that the assumption of narrow self-seeking is not essential to their analysis. Their argument “does not depend for its elementary logical validity upon any narrowly hedonistic or self-interest motivation of individuals in their behavior in social-choice processes.” It depends only on the more mild “economic” assumption that “separate individuals are separate individuals and, as such, are likely to have different aims and purposes for the results of collective action” (Buchanan and Tullock 1962, p. 3).
The situation is the same for information choice theory. On the one hand, the assumption of egoistic rationality among experts is clearly exaggerated. On the other hand, it seems likely that in information choice theory, as in public choice theory, the assumption of egoistic rationality will sometimes be a serviceable first approximation that helps draw our attention to basic issues and relationships, while also pointing to empirical anomalies that can be resolved only by relaxing the assumption. And in information choice, as in public choice, we have clear early statements repudiating facile models of “selfish” behavior. Koppl and Cowan (2010) emphasize identity as an expert motive. Peart and Levy (2005, 2010) have emphasized sympathy, approbation, and praiseworthiness as expert motives. We should recognize, therefore, that the “economic” perspective of information choice theory does not require us to assume that an expert – or anyone else – is “selfish” in any substantive sense. We should repudiate both the naive view that experts are always and everywhere truth seekers and the cynical view that truthfulness has no value to any expert ever.
Information choice theory includes identity as a motive of experts. Akerlof and Kranton (2000, 2002) introduce identity to the utility function. Akerlof and Kranton (2005, 2008) put identity into the utility function of the agent in an otherwise standard principal-agent model. Cowan (2012) and Koppl and Cowan (2010) apply the principal-agent model of Akerlof and Kranton (2005, 2008) to forensic science.
Identity creates the risk of faction. An expert who identifies with the standards and practices of their expert group may be less likely to contradict the opinions of their fellow experts even when doing so would be correct or in the client’s interest. The fingerprint examiner hired for Mayfield, for example, missed evidence tending to exonerate Mayfield, and one may wonder whether a sense of identity with the fingerprint profession may have contributed to this oversight. An expert’s identity as a professional or sympathy for his or her fellow experts may compete with sympathy for the client. (See Levy and Peart 2017, pp. 197–209 on factionalized science.)
Information choice theory includes sympathy as a motive of experts. Adam Smith defined sympathy as “our fellow-feeling with any passion whatever,” including “our fellow–feeling for the misery of others” (Smith 1759, p. 5). Peart and Levy have emphasized the important role of Smithian moral sentiments motivating action. Peart and Levy (2005) note that sympathy, or its lack, may be an important motive. They cite both Adam Smith (1759) and Vernon Smith (1998).
Information choice theory includes approbation as a motive of experts. Levy (2001, pp. 243–58) contains a helpful discussion of approbation. He quotes Adam Smith saying “man” has “an original desire to please, and an original aversion to offend his brethren” (Smith 1759, as cited in Levy 2001, p. 244). Peart (2010) emphasizes approbation as an expert motive.
Information choice theory includes praiseworthiness as a motive of experts. Levy and Peart (2008a) extend the attention to Smithian moral sentiments by noting the importance of praiseworthiness. “Motivation by praiseworthiness may answer the question of how the two great reforms [Adam] Smith advocated, free trade and abolition of slavery, could be effected despite Smith’s pessimism to the contrary,” they note. “If a world without special privilege becomes a norm and action to move the world closer to the norm is a praiseworthy act, then the desire to behave in a praiseworthy manner can change the world” (p. 475).
We have seen Mandeville adopt a rather dark and satiric perspective on human motivation. But he too affirms the importance to each of us of the opinions of others. He describes the “Witchcraft of Flattery” (p. 37) as “the most powerful Argument that could be used to Human Creatures” (p. 29). Our feelings of honor and shame come from our susceptibility to flattery (p. 29). These two “passions” induce us to act as if our inner motives were better than they really are. He says, “So silly a Creature is Man, as that, intoxicated with the Fumes of Vanity, he can feast on the thoughts of the Praises that shall be paid his Memory in future Ages with so much ecstasy, as to neglect his present Life, nay, court and covet Death, if he but imagines that it will add to the Glory he had acquired before” (p. 237). It is, I think, just to roughly equate Mandeville’s “Vanity” or, in some passages, “Pride and Vanity” with Adam Smith’s “impartial spectator.”
Other writers have found some degree of equivalence between Mandevillean pride and Smith’s impartial spectator. Of the relationship between Mandeville’s Fable and Smith’s Theory of Moral Sentiments, Kerkhof says:
The central question is whether morality can be reduced to a “regard to the opinion of others”, in other words whether virtue ultimately appears to be a form of vanity. In the final analysis, this proves to be the case: “virtue” appears to be a more effective way of receiving applause from the audience – internalized as the “impartial spectator”. According to the Theory of Moral Sentiments morality is produced by something like a market of sympathetic feelings, although Smith as a moral person does not want to accept this. The flowery language Smith uses to cloak his findings, hides a darker, Mandevillean view of man as an animal living in constant anxiety about the opinion of others.
Kerkhof (1995, p. 219) draws our attention to Arthur Lovejoy’s letter to Kaye (editor of Fable) in which Lovejoy says the Theory of Moral Sentiments “carried out” in “more detail” the “general idea” from Mandeville that “pride” or “glory” is “the genesis of morality.” (Some passages of the letter are transcribed on page 452 of volume II of Kaye’s edition of the Fable.) Branchi (2004, n.p.) says “Smith’s effort to draw a clear distinction between love for praise and love for praiseworthiness” is “an attempt to trace a dividing line through the middle of that which Mandeville groups together as egoistic passions. Honor and the passions on which it is based rest in precarious balance on this boundary line.”
Mandeville even recognizes human compassion, which is close to what Smith called “sympathy.” We have seen Smith define “sympathy” as “fellow-feeling.” Mandeville defined “compassion” as “a Fellow-feeling and Condolence for the Misfortunes and Calamities of others” (1729, vol. I, p. 287). Mandeville and Smith both recognize and emphasize compassion. Unlike Smith, however, Mandeville places equal emphasis on malice as its ever-present dual. “Some are so Malicious they would laugh if a Man broke his Leg, and others are so Compassionate that they can heartily pity a Man for the least Spot in his Clothes; but no Body is so Savage that no Compassion can touch him, nor any Man so good-natur’d as never to be affected with any Malicious Pleasure” (p. 146). Compare Mandeville’s remarks with the opening paragraph of Smith’s Theory of Moral Sentiments:
How selfish soever man may be supposed, there are evidently some principles in his nature, which interest him in the fortune of others, and render their happiness necessary to him, though he derives nothing from it except the pleasure of seeing it. Of this kind is pity or compassion, the emotion which we feel for the misery of others, when we either see it, or are made to conceive it in a very lively manner. That we often derive sorrow from the sorrow of others, is a matter of fact too obvious to require any instances to prove it; for this sentiment, like all the other original passions of human nature, is by no means confined to the virtuous and humane, though they perhaps may feel it with the most exquisite sensibility. The greatest ruffian, the most hardened violator of the laws of society, is not altogether without it.
It is as if Smith could not bear to deploy Mandeville’s moral psychology without first stripping away or cloaking the darker elements. He stripped away the unpalatable truth that even the most “good-natur’d” person has at least some tincture of “Malicious Pleasure” in the suffering of others. And he cloaked vanity in the finery of the impartial spectator. With Mandeville, we have both Smithian “sympathy” and the Smithian desire of praiseworthiness. But we also have darker elements ever present. By muting the unharmonious notes in Mandeville, Smith gives us a more optimistic vision of human psychology. Mandeville placed greater emphasis on malice, self-deception, and the universality of hypocrisy. Smith seems to reassure us that we are not so bad, really, all things considered. Mandeville insists that we are far more wicked than we dare admit to ourselves. And yet the basic psychological mechanisms at work in the two theories are similar. Kaye is ambivalent on this issue, but as one point describes the differences between Smith’s psychology and Mandeville’s as reducible “mostly to a matter of terminology” (Mandeville 1729, p. cxlii, n. 3).
Cognition is Limited and Erring
The assumption of some sort of “bounded rationality” is no longer unusual in economics. The term has a fluid meaning, but always implies some idea of cognitive limits and the potential for error. While bounded rationality is not always assumed in economic analysis today, it is probably uncontroversial even among economists to say that the cognition of experts is limited and erring. Nevertheless, many economists seem somewhat disposed to neglect such limits when discussing different forms of expertise. Howard Margolis provides an illustration.
Margolis (2007) comments on Ferraro and Taylor (2005), who asked participants at the leading profession meeting for academic economists to identify the opportunity cost in a multiple-choice problem drawn verbatim from a successful introductory textbook, that of Frank and Bernanke (2001). The correct answer was the least popular. Ferraro and Taylor infer that pedagogy must be improved (p. 11). Margolis (2007), however, attributes the errors to a “cognitive illusion” and notes that “economists are human” and, therefore, “vulnerable to cognitive illusions” (p. 1035). Margolis (1998) identified a cognitive illusion that ensnared Tycho Brahe, influenced theory choice, and yet went undetected for 400 years. Thus, experts are human even when the stakes are high and many of them consider the same topics over long periods of time.
The assumption that experts seek to maximize utility might create the mistaken impression that information choice theory repudiates “bounded rationality.” It does not. If experts have “bounded rationality,” then they will not always be able to maximize their objective functions.
Felin et al. (2017) suggest why we should not limit our concept of “bounded rationality” to that found in the work of Herbert Simon or of Daniel Kahneman. They note that perception arises through an interplay of environment and organism and is, therefore, “organism-specific.” Color vision illustrates the point. Writers sometimes equate “color” with given light frequency. But people typically perceive an object’s “color” as constant when the conditions of viewing it are variable. The green of my shirt is the same at twilight and midday. Nor does it change when a cloud passes across the sun. The “spectral reflectance” (Maloney and Wandell 1987) arriving from the object to our retina is variable, and yet we perceive the object’s color as constant. Such perceived constancy is feature of human color vision and not a bug. Musacchio (2003) uses synesthesia to illustrate the claim that our phenomenal experience has no particular relationship to the external objects of that experience (p. 344). Felin et al. (2017) use the evocative phrase “mental paint.”
If we accept this sort of view, then, as Felin and colleagues emphasize, we should not imagine that “organisms (whether animals or humans)” correctly perceive an external environment and are bounded in their perception only because they “are not aware of, nor do they somehow perceive or compute, all alternatives in their environments.” They argue that this mistaken view of perception is characteristic of the leading economic models of bounded rationality, including those of Herbert Simon and Daniel Kahneman. They criticize this implicit idea of the “all-seeing eye” in theories of bounded rationality. We should recognize instead, they say, that the internal perceptual structure of the organism determines the sort of world it lives in. They favorably cite the biologist Jakob von Uexküll, who “argued that each organism has its own, unique “Umwelt” and surroundings.” Uexküll said that “every animal is surrounded with different things, the dog is surrounded by dog things and the dragonfly is surrounded by dragonfly things” (Uexküll 1934, p. 117 as quoted in Felin et al. 2017).
This argument by Felin et al. (2017) matters for us because something similar is true about perceptual differences among humans. Different persons live in different “Umwelten.”
Though rooted in a phenomenological perspective rather than natural science, Alfred Schutz (1945) makes a similar point. He speaks of the “multiple realities” we inhabit. William James’ Principles of Psychology, Schutz says, raises “one of the more important philosophical questions,” namely the nature of reality (Schutz 1945, p. 533). Schutz tells us that, as James explained,
The origin of all reality is subjective, whatever excites and stimulates our interest is real … [T]here are several, probably an infinite number of various orders of realities, each with its own special and separate style of existence. James calls them ‘sub-universes’ and mentions as examples the world of sense or physical things … the world of science … [and] … the various supernatural worlds of mythology and religion.
Schutz (1945) attempts to develop some of “the many implications involved” in James’s understanding of “reality” and “sub-universes” (p. 533). “Relevance” is a central concept in that effort.
Each of us has a different place in the division of labor. Therefore, each of us knows different things and has different sensibilities to events around us. We have different stereotypes and recipes to guide us in our daily lives. Each person’s “prevailing system of interests” determines which elements of his “stock of knowledge” are relevant to him (Schutz 1951, p. 76; see also Schutz 1945, pp. 549–51). This system of Schutzian “relevancies” guides the person and influences the sorts of discoveries they can make. Thus, “a barber is more likely to note details of a person’s haircut than a dentist, and a dentist more likely to note details about a person’s teeth” (Risinger et al. 2002, pp. 18–19). In a fairly literal sense, the division of knowledge in society produces a separate world for each person to inhabit.
Our “worlds” overlap, but each of us occupies a unique position in the system and lives, therefore, in a unique “Umwelt.” This important implication of the division of knowledge should warn us against adopting the view that “science” or some other expert perspective is “the” correct view. In some contexts, of course, the expert’s expertise trumps other views. The world is not a flat plate resting on the back of a turtle. And there is often only one correct answer to a question. Whether the defendant shot the victim is not a matter of perspective even if the available evidence does not show us who did it. Either Jones did shoot Smith or Jones did not shoot Smith. But the expert’s system of relevancies is not that of their clients or other nonexperts. And it is not generally desirable to impose the expert’s relevancies on others.
All perception comes from a point of view and no one point of view among the many is uniquely correct or superior to all others in all contexts. It is hard to imagine how there could be a division of knowledge in society and simultaneously one uniquely correct overarching point of view or one privileged system of relevancies. Totalitarian attempts to impose one unifying system of ideas and relevancies are utterly and necessarily futile. The heterogeneity and multiplicity of legitimate perspectives is a necessary consequence of the division of labor and its correlate, the division of knowledge. But this heterogeneity and multiplicity of perspectives implies that any one person will have a partial and limited model of the world. Such limits constitute a form of “bounded rationality,” but one very different from that of Herbert Simon and Daniel Kahneman. Call it “synecologically bounded rationality.” The bound on rationality is “synecological” because it is the necessary consequence of the fact that knowledge is synecological when there is a social division of labor.
Computability theory and related branches of pure mathematics imply a different sort of limit on cognition, at least if we are unwilling to assume that the agents in our model can compute the uncomputable. A mathematical function is (“Turing” or “algorithmically”) computable if a Turing machine, the mathematicians’ ideal type of a digital computer, can be programmed to solve it. (Often Turing computability is simply called “computability.”) Building on Lewis (1992), Tsuji et al. (1998) reveal just how pervasive (algorithmic) noncomputability is. They show that even finite games can be undecidable. A finite game is trivially decidable, of course, if we have a complete and explicit list of strategies and payoffs. We can list every strategy combination and its corresponding payoff vector. We can then simply run down this finite list and see which entries, if any, are Nash equilibria. But if the game is described in formal language, it may be impossible to solve by “brute force.” In that case the game may be undecidable. In a deep and important remark, Tsuji et al. (1998) note, “Formalized theories are about strings of symbols that purport to represent our intuitions about concrete objects” (p. 555). We use general terms to describe the world, only rarely employing detailed listings and the like. Thus, our account of the world is radically incomplete and constantly subject to limits of computability and decidability.
Velupillai (2007) shows that “an effective theory of economic policy is impossible” for an economy that can be modeled as “a dynamical system capable of computation universality (pp. 273 and 280). Policy makers would have to compute the uncomputable to know the future consequences of their policies. Velupillai links his important result to F. A. Hayek’s “lifelong skepticism on the scope for policy in economies that emerge and form spontaneous orders” (Velupillai 2007, p. 288).
Canning (1992) showed that a Nash game may not be algorithmically solvable if there is no “Nash equilibrium in which the equilibrium strategy is a best reply to all best replies to itself” (p. 877). He calls this condition “strict Nash.” This result is fundamental, but he curiously argues that his result implies only a “slight” adjustment to then current practice in social science and applied game theory. The requirement of “strictness” excludes many games, including two given considerable attention by von Neumann and Morgenstern (1953), namely, “Matching Pennies” and “Stone, Paper, Scissors.” It seems doubtful whether Canning’s restriction should be considered “slight.”
Algorithmic information theory, which “studies the size in bits of the smallest program to compute something” (Chaitin, da Costa, and Doria 2012, p. 50), reveals a sense in which it may be impossible to have a theory of some complex phenomena. To predict, explain, or even merely identify a sufficiently complex system may require a description so lengthy that no simplification of the original system is achieved. The behavior of a sufficiently complex system (one at the top of a Chomsky–Wolfram hierarchy) cannot be predicted ahead of time (Wolfram 1984; Markose 2005). We can do no better than watch it unfold. Wolpert (2001) showed that for any pair of computers it is impossible for each to reliably predict the output of the other, even if the computers are somehow more powerful than Turing machines.
An analog “hypercomputer” could theoretically compute functions that are not Turing computable (da Costa and Doria, 2009, p. 80). Opinions differ on whether hypercomputation is even theoretically possible (Cockshott et al. 2008; da Costa and Doria 2009). In any event, it is not a current reality. Moreover, Wolpert (2001) shows that computability problems would arise even in a world with hypercomputers, as long as they are physically realizable.
Theorists should not impute to the agents in their models the ability to compute uncomputable functions or compress uncompressible data. This methodological restriction imposes a minimal form of bounded rationality. As the contributions to Velupillai (2005) demonstrate, however, even this “minimal” form of bounded rationality is often violated in standard economic models.
Incentives Influence the Distribution of Expert Errors
There are two aspects to the assumption that incentives skew expert errors. First, experts may cheat or otherwise self-consciously deviate from complete truthfulness. They may do so to serve either an external master or an internal bias. Second, experts may unknowingly err in ways that serve an external master or internal bias. The first point is relatively straightforward, but not always consistently applied. I think the second point is quite important, but it seems to have been almost entirely overlooked by the economics profession.
In a standard principal-agent model, expert “errors” are the product of guile. In a standard model they must be the product of guile because cognition is unerring. In information choice theory, however, experts may make errors that are not fully or even remotely intentional. In this sense, they may make “honest” errors. These honest errors, however, are influenced by incentives. The lower the opportunity cost of making an error of a particular sort, the higher the incidence of such errors, independently of the degree of intention associated with the error. In other words, experts respond to bias. An expert may make honest errors in favor of their client because their client pays them or because they have sympathy for their client. Sympathy for someone else, other experts perhaps, may encourage an error that hurts the client. In all such cases, incentives skew honest errors.
The assumption that incentives skew honest errors is a natural assumption only if we are also assuming fallible cognition. When economists assume strong forms of rationality, the assumption may be unnatural or even impossible. In that case, one may be led to model expert errors as random draws from a symmetric distribution.
I have not been able to find a clear statement in the economics literature that honest errors are skewed by incentives. Gordon Tullock probably comes as close as anyone in his book The Organization of Inquiry. As Levy and Peart (2012, p. 165) explain, Tullock asked what science would be like “if the subjects of our discipline could trade with us, offering us things of value to bend our results.” His answer was that “we would have economics, a racket not a science.” The word “racket” appears twice in the book. It is seen first in a footnote to “Some Academic ‘Rackets’ in the Social Sciences” by Joseph Roucek (1963). “What it all amounts to,” says Roucek in his summary,
is this: the academic profession, as any other, carries on shady practices … It tries to prevent self-evaluation or an empiric description of its underground practices which might give the idea to the layman that, after all, even the professors are just human beings … and members of a profession which has to sell its wares to itself and especially to the unsuspecting public.
The second use of the word “racket” comes later in the book. Tullock notes that there is always a demand for supposed experts to defend tariffs and the like. The existence of demanders who do or might “need the assistance of either someone who believes in tariffs or an economist who is in this racket makes it possible for them to continue to publish, even in quite respectable journals. Thus a dispute which intellectually was settled over a century ago still continues” (Tullock, 1966, p. 158).
Tullock (1966), Levy and Peart (2010), Diamond (1988), Wible (1998), and others have made the “economic” assumption that incentives help shape the opinions of experts. Levy and Peart (2010) suggest that the assumption has not been well understood or widely adopted. In all these versions of the idea, however, the assumption is that the scientist or other expert knowingly serves his interests. Some evidence suggests, however, that the same thing can happen without the affected expert knowing it.
Some evidence has been discussed in the economics literature with the Allais paradox, preference reversal, and other cases in which people seem to contradict themselves. It is hard to square such phenomena with strong forms of economic rationality. Jakobsen et al. (1991) report on a patient with brain damage who could accurately describe small blocks, but had difficulty in grasping them because of an inability to appropriately direct the motions of her hand. Goodale and Milner (1995) use this case as part of an argument that the brain has two largely separate visual systems. They say: “the neural substrates of visual perception may be quite distinct from those underlying the visual control of actions. In other words, the set of object descriptions that permit identification and recognition may be computed independently of the set of descriptions that allow an observer to shape the hand appropriately to pick up an object” (Goodale and Milner 1995, p. 20). A difference in evolutionary time separates the “functional modules supporting perceptual experience of the world … and those controlling action within it” (Goodale and Milner 1995, p. 20). This study is but one of many tending to show that the idea of a unitary mind is no longer consistent with the established results of empirical science. If the hypothesis of a unitary mind cannot be sustained, then it is possible to consider evidence that incentives skew honest errors.
The literature on “observer effects” seems to show that our opinions may serve our interests even when we know it not. Robert Rosenthal is probably the leading theorist of observer effects. He was a coauthor on Risinger et al. (2002), which includes a masterful review and summary of the literature on observer effects. Risinger et al. (2002, p. 12) explain: “At the most general level, observer effects are errors of apprehension, recording, recall, computation, or interpretation that result from some trait or state of the observer.” We are generally more likely to observe, perhaps mistakenly, what we expect to see than what we do not expect to see, what we hope to see than what we do not hope to see. This tendency is strengthened by ambiguity. Thus, the three keys to observer effects are the observer’s state of expectation, their state of desire, and the degree of ambiguity in the material being observed. Krane et al. (2008) say that “Observer effects are rooted in the universal human tendency to interpret data in a manner consistent with one’s expectations. This tendency is particularly likely to distort the results of a scientific test when the underlying data are ambiguous and the scientist is exposed to domain-irrelevant information that engages emotions or desires” (p. 1006).
Figure 9.1 illustrates how errors of observations can be skewed in the direction of our expectations. If the central figure is seen only with the vertical elements, it will be seen as the numeral 13. If it is seen only with the horizontal elements, it will be seen as the letter B. Context creates expectations that influence perception.
Context influences perception.
Expectations influence observations without determining them. Risinger et al. (2002, p. 13) say: “The cognitive psychology underlying observer effects is best understood as a cyclical interplay between pre-existing schemata and the uptake of new information. Schemata are mental categories … that provide the framework for perception and reasoning.” They quote Ulrich Neisser saying “we cannot perceive unless we anticipate, but we must not see only what we anticipate” (Neisser 1976, p. 43, as quoted in Risinger et al. 2002, p. 14). This point merges with the point I made earlier that the division of knowledge implies a different perceptual framework for each person. The unique Umwelt of each person creates a different set of anticipations and, correspondingly, a different set of perceptions and perceptual possibilities. In other words, synecologically bounded rationality causes observer effects.
The synecological bound on rationality gives us insight into the supposed assumption of “selfishness” in economics. As I noted above, economists have sometimes assumed “egoistic rationality.” But the view that economists generally make such an assumption is mistaken. We have seen, for example, that the founders of public choice theory made no such assumption. And we have seen that even the cynical Mandeville recognized the existence of human compassion as well as psychological mechanisms that drive us to more prosocial behavior than our inner depravity might otherwise allow. But if our rationality is synecologically bounded, then the most altruistic person may be unable to transcend their local interests and perspectives. What may at first look like selfishness may simply be the fact that your Schutzian relevancies can enter my thinking at best only partially and imperfectly. We praise persons unusually able to apprehend and respond to the relevancies of others as highly empathetic and, perhaps, intuitive. Even the most empathetic and intuitive among us, however, will have little insight into the relevancies of strangers. The greater the social distance between thee and me, the less able I am to serve your interests directly. The epistemically necessary partiality in each person’s perspective makes us all look more “selfish” than we really are.
I have said that synecologically bounded rationality causes observer effects. Risinger et al. (2002 pp. 22–6) note that observer effects are pervasive and enhanced by “desire and motivation.” The selection of decision thresholds may be influenced by incentives, as Risinger et al. (2002, p. 16) briefly note. They point out that “observer effects manifest themselves most strongly under conditions of ambiguity and subjectivity” (p. 16, n. 62). Heiner (1983, 1986) imports signal detection theory to basic microeconomics. Whitman and Koppl (2010) give a rational-choice Bayesian model, discussed in Chapter 8, tending to support the claim of Risinger et al. regarding ambiguity.
Tversky and Kahneman (1974) discuss representativeness bias (p. 1124), availability bias (p. 1127), and adjustment and anchoring bias (p. 1128). These biases have since become familiar to economists. As far as I can tell, we can accept these “biases” as often present in human decision making even if we accept fully the critique of Felin et al. (2017) discussed earlier in this chapter.
Economists may be less familiar with role effects, which are important for information choice theory. Pichert and Anderson (1977) had their subjects read a story about two boys playing in a house. The story contained information about the house such as the presence of a leaky roof and the parents’ rule to keep a side door unlocked at all times. Subjects were instructed to read the story from either the perspective of a burglar or that of a realtor. Subjects had better recall of details relating to their assigned role rather than the opposite perspective. Anderson and Pichert (1978) and Anderson et al. (1983) found that when subjects were asked to switch roles in a second recall task, their memory improved for details relevant to the new role and degraded for details irrelevant to it. This effect was noted whether the second memory task was performed with a delay of five or ten minutes or a delay of about two weeks (Anderson et al. 1983, pp. 274–5).
Role effects reflect synecological (and thus SELECT) knowledge and, therefore, synecologically bounded rationality. To view the house from the burglar’s point of view we must imagine ourselves in the burglar’s Umwelt. Imagining life in that Umwelt directs our attention to burglar things. The experimental results of Anderson and his coauthors support the view that incentives skew the distribution of honest error. They induced observer effects with no more powerful an incentive than the injunction to read or recall from a given perspective. The perspective adopted determined what “errors” – in this case memory lapses – were more likely. These errors seem to have been unintended and unrelated to any scruples the subjects may have had about honesty or other virtues.
Risinger et al. (2002, p. 19) define conformity effects as “our tendency to conform to the perceptions, beliefs, and behavior of others.” They matter for information choice theory because experts may have relatively high status. Risinger et al. (2002) describe an experiment reported in Sherif and Shefir (1969) in which subjects were asked to say how far a light in darkened room had moved. The light was in fact stationary, but might seem to move under the optical conditions created by the experiment. Subjects were shown the light and asked to give their opinions in one another’s presence. “Although each person’s perceptions of motion range were influenced by the announced perceptions of the others, those of perceived lower rank were more influenced by those of perceived higher rank” (Risinger et al. 2002, p. 19).
Several experimenter effects have motivated protocols such as randomization and double-blind procedures in science. The Mendel–Fisher controversy, nicely summarized by Franklin (2008), illustrates the need for such precautions. Mendel (1865) worked out the elements of genetics in a study of peas. He carefully bred and crossbred two varieties of garden peas, noting five different properties of each, such as the color of the pod and whether the peas were wrinkled or smooth. He reported some results that seemed to have surprised him, but others that seem to have conformed to his expectations. In particular, he found that crossbred plants exhibited very nearly a 2:1 ratio of heterozygous to homozygous individuals. Fisher (1936) showed that this 2:1 ratio is not the one that should have been expected. We do expect the distribution of the true ratio to be centered on 2:1. But Mendel’s method for judging and recording heterozygosis required a correction factor that he neglected. To decide whether a crossbred plant was homozygous or heterozygous, Mendel planted seeds from each such plant to see if any of that plant’s offspring exhibited the recessive trait. Each seed from a crossbred and heterozygous plant has a one in four chance of exhibiting the recessive trait. Thus, it is necessary to plant enough seeds from each plant to be nearly certain that at least one offspring will exhibit the recessive trait. Seeing just one offspring with the recessive trait assures you that the crossbred parent plant was indeed heterozygous. Mendel, however, chose to plant only ten seeds per crossbred plant. That’s not enough. Fisher noted: “If each offspring has an independent probability, .75, of displaying the dominant character, the probability that all ten will do so is (.75)10, or .0563. Consequently, between 5 and 6 per cent. of the heterozygous parents will be classified as homozygotes” (1936, p. 125). Mendel should have recorded a ratio of about 1.7:1, not 2:1. His results were too good to be true. Fisher had difficulty accounting for the error. He said: “Although no explanation can be expected to be satisfactory, it remains a possibility among others that Mendel was deceived by some assistant who knew too well what was expected. This possibility is supported by independent evidence that the data of most, if not all, of the experiments have been falsified so as to agree closely with Mendel’s expectations” (Fisher 1936, p. 132).
Presumably, the Mendel–Fisher controversy will never be fully resolved, although the majority opinion seems be that Mendel probably did not commit a willful fraud. (Fisher cast his suspicions on an anonymous assistant, but not Mendel himself.) If we absolve Mendel of fraud, observer effects become a likely explanation. Mendel or his assistants were more likely to recheck data that violated expectation, which would induce bias. (Apparently, Sewell Wright was the first to make this point in 1966.) Similarly, knowing that he “needed” more wrinkled peas or more yellow pods may have caused Mendel or an assistant to see wrinkles that were not present or to mistake the color of a pod. The fact that Mendel’s data was too good to be true may help to suggest that the tendency of incentives to skew honest errors increases the risk of faction in science.
Whatever particular mechanisms may have been at work, we should take a lesson from the Mendel–Fisher controversy. Observer effects matter in science. Note also that Mendel’s apparent errors were not uncovered for seventy-two years. (While Fisher was not the first to see a problem, it was his article that caused a stir, albeit with a delay.) Once an error is recognized, it can be hard to correct. Richard Feynman gives another salient example in his famous lecture “Cargo Cult Science” (1974). Robert Millikan performed an experiment in which he measured the charge of an electron by observing how little oil drops emitted from an atomizer move in a field with and without an electrical charge. Feynman said “we now know” that the calculated value was “a little bit” too small: “It’s interesting to look at the history of measurements of the charge of the electron after Millikan. If you plot them as a function of time, you find that one is a little bigger than Millikan’s and the next one’s a little bit bigger than that, until finally they settle down to a number which is higher” (Feynman 1974, p. 12).
Berger, Matthews, and Grosch (2008, pp. 234–7) give three examples in which experimental precautions against observer effects are compromised by “inappropriate yet regimented research methods.” In the most striking of the three examples, “run-in bias” is created by deleting adverse events prior to randomization. “In randomized treatment trials,” they explain, “it is common to pre-treat the patients with the active treatment, evaluate their outcomes, and determine which patients to randomize based upon those outcomes. Bad outcomes (even deaths) prior to randomization do not make it into the analysis and do not count against the active treatment under scrutiny” (p. 234).
Dror and Charlton (2006) had experienced fingerprint examiners reexamining evidence from cases they had decided in the past. The evidence was presented in the ordinary course of work as real case evidence. The real case information was stripped away, however, and replaced with either no supporting information or supporting information that suggested a match when the earlier decision had been an exclusion (being told, for example, that the “suspect confessed to the crime”) or an exclusion when the earlier decision had been a match (being told, for example, that the “suspect was in police custody at the time of the crime”). A pair of experienced experts confirmed that the original decision was correct in each case. This determination by experienced experts participating as experimenters creates the presumption that the subject examiners’ original judgments were correct for those pairs of fingerprints used in the study. Dror and Charlton found that from forty-eight experimental trials, the fingerprint experts changed their past decisions on six pairs of fingerprints. The six inconsistent decisions (12 percent) included two from the twenty-four control trials that did not have any contextual manipulation. The fingerprint experts changed four of their past decisions from the twenty-four experimental trials that included the contextual manipulation. Thus, biasing context seems to have induced inconsistent decisions in 16.6 percent of the cases with contextual manipulation. Only one-third of the participants (two out of six) remained entirely consistent across the eight experimental trials.
The experimental results of Dror and Charlton (2006, p. 610) support the view that incentives skew the distribution of honest error. Of the six errors, four – in this case reversing an earlier opinion – occurred when the experimenters provided context information. In each of those four cases the error was in the direction suggested by the context information. It seems unlikely that these errors were willful. Each of the experimental subjects agreed to participate in the experiment. Each of them understood that they would at some point get experimental materials masquerading as real case files. And yet they agreed to participate in the study. It does not seem reasonable for someone to report judgments he or she knew to be erroneous after agreeing to such conditions. Incentives skewed the experts’ errors in spite of a conscious desire to avoid error.
Information choice theory assumes that errors are skewed by incentives. But incentives depend on institutions. Thus, information choice theory includes comparative institutional analysis. This institutional connection brings us to a point of distinction between information choice theory and standard principal-agent models that was noted earlier. Information choice theory studies the ecology of expertise.
The Ecology of Expertise
Recall that information choice theory does not presume an isolated principal-agent institutional structure. Instead, the theory recognizes that the larger institutional context may create different degrees and forms of competition among experts. In both its positive and normative aspects, information choice theory explicitly considers the ecology of expertise.
Some simple analytics of “epistemic systems” (Koppl 2005b) may help us to identify relevant features of the ecology of expertise. Epistemic systems are agent-based processes viewed from the perspective of their tendency to help or frustrate the production of local truth. (This definition generalizes that of Koppl 2005b.) In this context, “local truth” may mean true beliefs, correct expectations, appropriate behaviors, or something else depending on context. “Local truth” is “getting it right.”
It will sometimes be convenient to model an epistemic system as a game, perhaps a sender-receiver game, specifying a set of senders, S, a set of receivers, R, and a set of messages, M, together with payoffs and rules for their interactions. The senders may have a probability distribution over messages, showing the subjective probability that each message is true. The senders send messages to the receivers, who somehow nominate one message from the message set and declare it “true.” This is the judgment, of the receiver(s). Figure 9.2, adapted from Koppl (2005b), illustrates epistemic systems with monopoly experts.
Monopoly expert.
The oval marked “message set” represents the set of messages an expert might deliver. A forensic scientist, for example, might declare of two samples “match,” “no match,” or “inconclusive.” (NAS 2009, p. 21 discusses the variety of terms used to describe what is commonly thought of as “matching.”) The dashed arrow represents the expert choosing from the message set. The solid arrows represent the transmission to the receiver, for example the judge or jury at trial, and the “nomination” of a message by the receiver. The circle marked “judgment” represents the receiver’s judgment, for example the judgment of the jury in a criminal case that it was the defendant who left a latent print lifted from the crime scene.
When the receiver is able to compare the opinions of multiple experts we have the sort of situation represented in Figure 9.3, also adapted from Koppl (2005b).
Multiple experts who may be in competition.
Because each receiver gets messages from multiple senders, the experts are in a position of strategic interdependence, which may constrain their choice of message. Figures 9.2 and 9.3 do not represent all dimensions relevant to the ecology of expertise. For example, are the competing experts truly independent? If they are, the multiplicity of signals available to each receiver acts like a coding redundancy to reduce the receiver’s rate of error in judgment. If they are highly correlated, however, the appearance of redundancy may reinforce errors in the mind of the receiver, serving only to degrade system performance. Sympathy among experts or identification with an expert’s professional standards and practices may reduce independence among experts and cause expert errors to be correlated. In Chapter 3 I noted that professions may foster uniformity of professional opinion among their members. Professional “standards” and professional loyalty may reduce independence among experts of that profession.
In some applications, it is important to know whether sender messages are independent. A jury in a criminal trial may be presented with a confession, eyewitness testimony, and forensic evidence. It seems plausible that a jury in such a case might imagine that it has three independent and reliable sources of evidence pointing to the defendant’s guilt. Any one of these three forms of evidence would be sufficient for conviction if errors were not possible. When presented with multiple channels of incriminating evidence, a jury that is highly alert to the possibility of error within each channel considered individually might nevertheless convict because it underestimates the probability of innocence. If each of three forms of evidence has a 50 percent chance of falsely suggesting guilt, the chance that three independent forms of evidence would all point in the wrong direction is only one in eight, 12.5 percent. If each line of evidence has only a 10 percent chance of falsely suggesting guilt, the seeming probability of error falls to just one in a thousand, or 0.1 percent. But in some cases, these seemingly independent sources of evidence may be correlated, as suggested by some facts about the American system of criminal justice.
The case of Cameron Todd Willingham illustrates the risk that police investigators may unwittingly influence eyewitnesses to provide more damning testimony as the police investigation unfolds. Willingham was convicted in 1991 of the arson murder of his three young children based in large part on now discredited fire investigation techniques (Mills and Possley, 2004; Willingham v. State, 897 S.W.2d. 351, 357, Tex.Crim.App. 1995). Willingham’s case seems to be an unambiguous example of a false execution. Grann (2009) reports that one eyewitness’s “initial statement to authorities … portrayed Willingham as ‘hysterical,’” whereas her subsequent statements “suggested that he could have gone back inside to rescue his children” without much risk or courage. Another eyewitness switched from describing a devastated father to expressing a “gut feeling” that Willingham had “something to do with the setting of the fire.”
Drizin and Leo (2004) review 125 cases of “proven” false confessions. They provide evidence that the risk of false confession is higher for the young, the mentally retarded, and the mentally ill. Garrett (2010) reports that in 252 DNA exonerations, forty-two involved false confessions. He performed a content analysis of the first forty of them. “In all cases but two,” he reports, “police reported that suspects confessed to a series of specific details concerning how the crime occurred. Often those details included reportedly ‘inside information’ that only the rapist or murderer could have known” (Garrett 2010, p. 1054). Garrett infers that in many cases, “police likely disclosed those details during interrogations” (2010, p. 1054). As Garnett points out, we do not know whether the apparent information transfer was willful or unconscious, vicious or innocent.
Finally, the literature on forensic science errors shows that they are likely to be more common than popularly imagined and may be influenced by incentives at the crime lab. I have estimated that there are more than 20,000 false felony convictions per year in the United States attributable in part to forensic science testing errors or false or misleading forensic science testimony (Koppl 2010b). In many of these cases, the error or inappropriate testimony may be motivated by knowledge of the police case file, as illustrated by the Dror study cited earlier. Risinger et al. (2002, p. 37) give an example drawn from lab notes in a real case. “Suspect-known crip gang member – keeps ‘skating’ on charges – never serves time. This robbery he gets hit in head with bar stool – left blood trail. [Detective] Miller wants to connect this guy to scene w/DNA.” In another case, an examiner writes, “Death penalty case! Need to eliminate Item #57 [name of individual] as a possible suspect” (Krane, 2008). Such context information has the potential to skew the results of a forensic science examination, particularly under the frequently encountered condition of ambiguous evidence.
It seems likely that in at least some cases in the United States, false convictions are created when police investigators induce a false confession from a suspect, influence eyewitnesses to testify against that suspect, and provide the crime lab with case information that induces an incriminating forensic science error. In such cases, errors are correlated across seemingly independent evidence channels. The system is nonergodic, but the multiplicity of evidence channels creates for the jury a false picture of the reliability of the evidence in the case.
Ioannidis (2005) shows that similar problems can exist in research science. The ecology of tests in a field may drive the “positive predictive value” (PPV) of a given result below the “significance level” of the researchers’ statistical tests. He denotes by R the ratio of true relationships to objectively false relationships “among those tested in a field” (p. 0696). Thus, R depends on both the objective phenomena of the field and the ideas of researchers about the field. Those ideas determine which possible relationships are tested and the ratio, therefore, of true to false relationships “among those tested.” In this case the chance of a positive result being true (the PPV) is 
Synecological redundancy may reduce the chance of correlated errors. Synecological redundancy is “the ability of elements that are structurally different to perform the same function or yield the same output” (Edelman and Gally 2001, p. 13763). Edelman and Gally call it “degeneracy.” I prefer the label “synecological redundancy” because it is more descriptive. It suggests the synecological idea of a relationship between an environment and a community of interacting elements occupying it. These heterogeneous interacting elements produce the same or similar functions in the system through a variety of methods. Precisely because these elements are not the same, the function is robust to failures among the elements.
I have abandoned the term “degeneracy” in favor of “synecological redundancy.” Although my label is more cumbersome, it is more descriptive (as previously noted) and it does not suggest decline or seem otherwise pejorative. The term “partial redundancy” has also been used (Whitacre and Bender 2010, p. 144). It may suggest, however, the need to “complete” redundancy or that “partial redundancy” is inferior to “redundancy.” Wagner (2006) discusses degeneracy in economics. Martin’s (2015) essay on “Degenerate Cosmopolitanism” provides an unusually lucid discussion of degeneracy, robustness, and evolvability in the context of political economy. He helpfully relates these terms to the idiosyncratic vocabulary of Taleb (2012).
Edelman and Gally (2001) contrast synecological redundancy with simple redundancy, “which occurs when the same function is performed by identical elements.” Synecological redundancy, by contrast, “involves structurally different elements.” It “may yield the same or different functions depending on the context in which it is expressed.” A multiexpert system will exhibit synecological redundancy to the extent that each expert in it is unique. A priest, a psychologist, a Buddhist monk, and a bartender may give very different advice on relieving anxiety. When all experts are the same, expert errors may be correlated.
Our earlier discussion of the criminal justice system suggests that synecological redundancy is not sufficient to create independent errors. Eyewitness testimony, confessions, and forensic evidence seem to be structurally different elements. The synecological redundancy of the system is low, however, because each of the seemingly diverse evidence channels emerges from one integrated process of evidence construction. “The absence of degeneracy [i.e. synecological redundancy] indicates monopoly, for it means that a successful plan must include participation at a particular node in the network of plans” (Wagner 2006, p. 119). Cowan and Koppl (2010, p. 411) note that the “tight relationship” among prosecutor, police, and crime lab “creates a vertically integrated monopoly supplier of criminal justice.” Such integration limits desirable properties of an epistemic system, namely, redundancy, synecological redundancy, adaptivity, diversity, and resilience. The system has low error-correcting power because errors can easily cascade down from the top, inducing the failure of multiple elements within the system. These considerations lead us to the theory of expert failure, which I consider in the next chapter.