3
There Is Room for Conditioning in the Creative Process: Associative Learning and the Control of Behavioral Variability
J. S. Bach created impressive and beautiful canons and fugues through creative manipulations of musical scales. Archimedes invented new methods of geometry to determine that the exact value of pi lay between two fractions: 310/71 and 31/7. Organic chemist Friedrich August Kekulé von Stradonitz had a dream about a snake biting its own tail, an analogy that provided him with the insight that important molecular compounds have a ring structure. Einstein discovered the fundamental relationship between time, space, matter, and energy. Mandelbrot observed that complex structures of real-world objects (mountains, coastlines, snowflakes, etc.) conform to hierarchies of fractal patterns repeated at multiple scales of observation. Although great scientists, mathematicians, and artists are revered for their creative genius, ordinary people in their daily lives also perform creative acts. When a student has trouble understanding a difficult concept, the teacher may apply a simple analogy to relate the concept to something familiar to the student, thereby engendering new understanding. If you take off your shoe to pound a nail into the wall because you’ve misplaced your hammer, you have engaged in a creative act.
Yet, creativity is not the sole domain of Mankind, of da Vinci, of Beethoven, of Shakespeare and Jimi Hendrix. Although a humble pigeon, possessing a brain the size of a cashew, cannot write beautiful sonnets, it and other nonhuman animals are equipped with the psychological processes that contribute to at least nominally creative behavior. Animals face problems and challenges in finding food and mates, avoiding predators, and thwarting competitors; creative behavior may play an adaptive role in their survival. What are the defining characteristics of creative behavior, and how can it be studied in humans and animals alike? Creativity can be defined as the tendency to generate new ideas or behaviors that may be useful in solving problems. Thus, a creative act is defined by both its novelty and its value in some context. We present a view of creativity from the perspective of animal researchers studying behavioral processes fundamental to the creative act. Our position is based on several principles on the nature of creative action.
(1) Creativity is dependent on mechanisms related to the production of novel and variable behavior. Without mechanisms for novel action, creative behavior is impossible, as a creative act is by definition an action that is, on some level, new.
(2) A creative act is not mere noise, but serves a useful end. That is to say, the production of novel and variable behavior is not in itself “creativity,” but it is critically important to the creative act.
(3) Processes that generate novelty and variability in behavior can be accessed and controlled in associative conditioning procedures. The distribution of an animal’s behavior is controlled by prior learning about the predictive value of stimuli (as in Pavlovian conditioning) and the consequences of its actions (as in instrumental conditioning).
The first two principles have widespread popularity among both the lay population and creativity researchers. A behavior must be both novel and (at least potentially) adaptive in order to qualify as creative. We do not take these to be controversial statements, nor do we believe them to be anything other than obvious. However, the associative learning mechanisms that produce variability in behavior have received insufficient attention in the literature of creativity. Therefore, in this chapter, we will focus primarily on the role of conditioning as it pertains to novel and variable action (i.e., Principle 3). This is a relatively narrow focus on the topic of creativity, but we believe it to be extraordinarily important.1
Creativity researcher Margaret A. Boden (2004) wrote, “Chance with judgment can give us creativity; chance alone, certainly not” (p. 237). This statement is not to imply that chance is unimportant. Indeed, it is necessary (though insufficient) for creativity. It is important to note that this chapter deals more with the processes that produce novel behavior (i.e., “chance alone”), and is not meant to provide a complete account of the creative act in nonhuman animals. This chapter seeks to examine how the behavioral grist for the creativity mill is generated in the first place. We will seek to answer the question of what processes control the emergence of new simple behaviors.
Spontaneous Behavior and the Problem of Novelty
Classically, a behavior may be described as instrumental if it is emitted by an organism, rather than elicited by the stimulus circumstances within which the organism finds itself. A behavior that is under a great deal of control may properly be described as an instrumental (a.k.a. operant) action; behavior that emerges relatively uncontrolled from the animal’s context is something else (e.g., reflex, Pavlovian conditioned response). For example, kicking a soccer ball toward your friend as play behavior is an instrumental action, directly under your control; on the other hand, kicking your leg upward when your doctor raps your patellar tendon is a simple, uncontrolled reflexive response to a stimulus.
An interesting aspect of spontaneous behavior, and one that is often overlooked, is the fact that it is never exactly the same. Certainly, there are actions that differ only a very small amount from other, very similar actions that have been performed in the past. For example, the behavior of tying one’s shoes is unlikely to be very different from its prior instances. However, there is little doubt that on the level of the organism, there must be minute differences in examples of even this well-trained action: the eyes of the person may flutter in a new way; the position of the foot relative to the hands is likely to be different; the laces themselves subtly degrade with each passing day and with each successive knot, engendering slightly less resistance to the tying process; and so on. Even well-trained athletes show subtle variation in the execution of a well-practiced skill (Bartlett, Wheat, & Robins, 2007). There is a great deal of scientific evidence that behavior commonly thought to be invariant in fact has small amounts of variation from instance to instance (e.g., Bartlett et al., 2007; Brainard & Doupe, 2001).
A classic behaviorist view has a difficult time accounting for this truth (Epstein, 1991). Indeed, that current behavior is never precisely the same as prior action is a potential concern for a psychological position such as B. F. Skinner’s brand of radical behaviorism. Radical behaviorism rests on the notion that the genetic makeup and environmental history (i.e., reinforcement schedules) of the organism are fundamentally critical for the production of behavior. However, a piece of the puzzle is missing: where does a new behavior come from in the first place?
This is problematic. Speaking in the behaviorist tradition, an animal’s actions are selected by the consequences of those actions; should a response (e.g., lever-pressing) be followed by a favorable circumstance (e.g., food delivery), then the response will be more likely to occur in the future. At its heart this is a redescription of Thorndike’s (1927) law of effect, which remains a fundamental principle of instrumental learning (see Dennett, 1975). However, it is important to note that this kind of account assumes the response before learning occurs. This account can describe the selection of a particular action as becoming relatively stronger or relatively weaker, but it does so by accepting that the response is already part of the individual’s repertoire, ready to be selected by the environment. The law of effect is silent on the origins of new behavior; one or more of an animal’s many expressed behaviors may be selected and strengthened. Skinner himself reflected a lack of concern for the processes that generate new behavior, attributing novel creative action to “chance” and random “mutations” (Skinner, 1970). Furthermore, Skinner (1966) argued that the process by which an animal moves toward a solution to a problem “does not . . . necessarily reflect an important behavioral process” (p. 240). Epstein (1991) reflects on this curious dismissal of processes underlying novel action:
Skinner took generativity for granted, relying on broad-brush explanations of creativity or on no explanations at all—even suggesting that the creative process was “not important.” This fits his two-factor form of determinism. Nontrivial mechanisms of variation might have made the organism seem a little too autonomous for Skinner’s liking. (p. 365)
Surely the environment can select action, but clearly there must be an account for the emergence of new action (Epstein, 1990). As Epstein (1991) put it, “Skinner’s deterministic dyad always needed another factor: Behavior is determined by genes, environmental history, and certain mechanisms of variability” (p. 363). It is clear that we must now discuss some of the work that represents the recent study of the production of novel and variable behavior in animals. We devote the next sections to these mechanisms of variability generation and control.
Explicitly Reinforced Variability in Behavior
Many early studies indicated that variability could be modulated by reinforcement; it is uncontroversial to acknowledge that the schedule of rewarding outcomes has long been known to predictably produce differential levels of behavioral variability in animals. In a bar-pressing task, Schoenfeld, Harris, and Farmer (1966) reinforced rats when their successive interresponse times fell into two distinctive temporal class intervals. This restriction on rewarded actions resulted in a very low level of variation in the rats’ bar-pressing behavior. Bryant and Church (1974) reinforced rats for performing on a pair of levers; if reinforcement was contingent on the rats alternating their responses (e.g., left, then right) on only 50 percent of trials, rats tended to develop relatively stereotyped behavior to a single lever. If reinforcement was contingent on the rats alternating levers on 75 percent of trials, however, the rats’ bar-pressing behavior became so variable as to be indistinguishable from random. Blough (1966) conditioned pigeons to peck at a keylight for food, with reinforcement only being delivered immediately following unusual interresponse intervals. This training resulted in the pigeons responding to the keylight with interresponse times approaching a random (i.e., highly variable) distribution. Pryor, Haag, and O’Reilly (1969) performed a study where they specifically reinforced novel behaviors (e.g., jumps, flips) in porpoises; the porpoises responded to this training by producing behavior that had never been seen before. In this case, reinforcement of novelty, of uncommon behavior, increased behavioral variation in the porpoises. Indeed, their experiment concluded at the point where classification of the porpoises’ increasingly variable behavior became virtually impossible.
Page and Neuringer (1985) extended the evidence that suggests that variation is an instrumentally controlled component of behavior. Page and Neuringer were careful to point out that there are two possible sources of behavioral variation: (1) incidental variation due to the schedule of reinforcement (e.g., Schwartz, 1982), and (2) direct reinforcement of variation itself. This latter possibility is the suggestion that reinforcing an animal for behaving variably will, in and of itself, engender higher levels of variability in behavior (e.g., Pryor et al., 1969). In one experiment, Page and Neuringer directly investigated whether variability is a reinforceable dimension of behavior by comparing pigeons’ response variation on a Lag 50 schedule (i.e., a response sequence was only reinforced if it was different from each of the last 50 sequences) to responses by a control group of pigeons that received the same rate of reinforcement, but where delivery was not contingent on the novelty of their response sequences. They found that variation was significantly greater in pigeons under the experimental Lag 50 schedule than the yoked control procedure, clearly demonstrating that variation can be directly manipulated through reinforcement. These findings also suggest that data from previous studies (e.g., Blough, 1966; Bryant & Church, 1974) were not necessarily by-products of the experimental reinforcement schedules, but instead may have been due to specific reinforcement of variability itself. In another experiment, Page and Neuringer (1985) demonstrated that the reinforced variability of responding could be brought under stimulus control in a manner similar to other aspects of behavior (e.g., response force, rate of responding; see also Ross & Neuringer, 2002; Denney & Neuringer, 1998; Neuringer, 1993; Morgan & Neuringer, 1990). This evidence illustrates that variability can clearly be an operant (Neuringer, 2002)—that is to say, the novelty of the form of a response can be increased with reinforcement of novel performance itself.
There are two potential explanations regarding the underlying mechanism that animals use to produce variation (Neuringer, 2004). One possible explanation is couched in terms of retrospective memory, or memory of previous responses. Animals that can remember recent behaviors can learn to avoid repeating these actions if high operant variability is being reinforced. Strong evidence for this kind of explanation comes from studies of rats performing in a radial arm maze; in a standard eight-arm maze, rats quickly learn to run down each of the arms without repetitions in order to efficiently obtain food placed at the end of each arm. This behavioral pattern is clearly efficient, as revisits to arms are not rewarded. Memory of previously visited arms prevents revisits and enables requisite variation to maximize the efficiency of retrieving food rewards in the radial maze. The evidence suggests that many animals are capable of using retrospective memory processes to generate enough variability so as to navigate the maze and collect food efficiently (Cook, Brown, & Riley, 1985; see also Kesner & Despain, 1988).
A second mechanism that may account for operant variability is a stochastic (i.e., random) behavior-generation process (Brembs, 2011). Evidence has shown that when variability is explicitly reinforced it tends to approach a random distribution (e.g., Blough, 1966; Neuringer, 1986; Page & Neuringer, 1985). Pharmacological manipulation of memory in rats also supports the stochastic generator hypothesis of behavior. On the one hand, ethanol disrupts short-term memory processes; presumably, failure of performance on a task that requires repetitive behavior is due to a failure in remembering the response sequence and/or recent behavior. The administration of ethanol has a marked deleterious effect on performance in rats rewarded for a stereotyped sequence of lever presses (e.g., left-left-right-right; McElroy & Neuringer, 1990). On the other hand, injections of ethanol have virtually no effect on operant performance when rats are reinforced for high variability in behavior (Cohen, Neuringer, & Rhodes, 1990; McElroy & Neuringer, 1990). Memory therefore seems critical for operant repetition, but not for operant variability (Neuringer, 1991). The results of Page and Neuringer (1985) represent further evidence against the memory account of behavioral variation. Remember, in this study a group of pigeons’ response sequences were reinforced on a Lag 50 contingency. Although the birds quickly learned to respond variably under this schedule, it seems highly doubtful that they were able to remember and avoid each of their previous fifty response sequences (Neuringer, 2004; but see Cook, Levison, Gillett, & Blaisdell, 2005, for evidence of the prodigious capacity of pigeon memory). Another study by Neuringer (1991) directly compared memory and random processes in rats. Neuringer inserted retention intervals (RIs) of varying length between individual responses across two levers; response sequences in one group were reinforced on a Lag 5 schedule (e.g., Group VAR). Whereas these RIs should interfere with memory for prior responses, they should have no deleterious effect on behavioral variation due to a random generative process. Rats were not detrimentally affected by the retention interval; indeed, they actually demonstrated better performance (i.e., greater variation) with longer RIs. In a control group (e.g., Group REP) rewarded for a repetitive response sequence (e.g., LLRR), RIs interfered with performance proportional to the length of time the rat had to wait before response emission. At an RI of 20 seconds, rats in group VAR met the response criterion and were reinforced on approximately 65 percent of trials; at the same RI, rats in group REP were reinforced on only about 5 percent of trials. These results led Neuringer to hypothesize that memory for prior responses does not facilitate operant variation, but instead interferes with variation (see also Weiss, 1965). This hypothesis is consistent with earlier findings (Page & Neuringer, 1985), in which pigeons were more likely to meet a variability criterion if a required behavioral sequence was eight responses rather than four responses in length. These data are perhaps the strongest indicators for a stochastic process being responsible for operant variability. The fact that stochastic responding is controlled by its consequences is a strong indicator that it is functional (Neuringer, 2004).
Neuringer, Deiss, and Olson (2000) investigated whether the direct reinforcement of variable behavior facilitates selection of a highly rewarded target response. In Experiment 1, they trained three groups of rats in a bar-pressing task; across phases of the procedure, they arbitrarily assigned sequences of target responses across two levers, ranging from two (e.g., left-right) to five responses (e.g., right-left-left-right-left). The rats were reinforced with a food pellet anytime the target response sequence was delivered. The first group (VAR) was trained so that variable responses (i.e., relatively unlikely response sequences, of the same length as the target) were reinforced on a variable interval one-minute schedule (VI-1). The VI-1 schedule allows frequent reinforcement but includes on average a one-minute temporal gap so that reinforcement of the target sequence is detectable. A second group (ANY) was also rewarded for nontarget responses on a VI-1 schedule, but reinforcement was not contingent on variable response sequences; in short, they were permitted, but not required, to vary their sequence behavior in order to obtain food reinforcement. A third group (CON) was only reinforced for producing the target response sequence. Although rats in all groups learned the easy (i.e., two- and three-response) sequences equally well, only rats receiving the VAR treatment learned to produce the more difficult target behaviors (i.e., the five-response sequence). The rats in the CON group tended to respond at very low rates during the more difficult phases, presumably because they initially were rarely reinforced for their bar-pressing behavior; rats in the ANY group continued to bar-press at high rates identical to those produced in the VAR group. Despite their high response rates, the ANY rats’ acquisition of the target behavior was markedly slower than the learning demonstrated by the VAR group for four- and five-response sequences. This clearly indicates that direct reinforcement of variability facilitates acquisition of complex operant behavior (see also Grunow & Neuringer, 2002).
Arnesen (2000; as cited in Neuringer & Jensen, 2010) examined whether reinforcement of variability facilitates future problem-solving behavior. She trained a group of rats to respond variably (see Pryor et al., 1969) when they encountered various arbitrarily selected objects (e.g., a soup can). For example, a rat may have been initially reinforced for touching the soup can with its nose, but then was required to perform a different action for reinforcement (e.g., touching the top of the can with its forepaws). A separate control group was presented with the same objects, but was reinforced for interactions with the objects irrespective of behavioral variation. Rats were then tested by being individually placed in an open field with thirty objects, each of which had a small piece of food hidden on or within it. Animals that had been previously explicitly reinforced for variable behavior found and consumed significantly more hidden food than the control group. These data indicate that problem-solving behavior is made more effective when the subjects are reinforced for variable responding.
In summation, the generation of novel behavior can clearly be controlled by the direct reinforcement of variation. Animals that are reinforced for acting variably will increase the probability of engaging in highly novel acts, including what can potentially be labeled as innovative, creative behaviors (e.g., Pryor et al., 1969). Thus, although reinforcement of one or a specified set of actions produces decidedly noncreative behavior (e.g., steady-state performance of lever-pressing for food reinforcement, or any overtrained skillful act), the direct reinforcement of “novel” responding does in fact result in an increase in new actions.
Expectation and the Generation of Variable Behavior
Another way for associative conditioning to modulate behavioral variation is through generation of expectations. A multitude of evidence suggests that behavioral variation increases markedly during extinction. In other words, when a previously reinforced behavior (e.g., lever pressing) is no longer followed by reward, the action does not merely become less frequent; behavior also tends to become more variable in nature. Many researchers have taken note of this relationship (Antonitis, 1951; Balsam, Deich, Ohyama, & Stokes, 1998; Eckerman & Lanson, 1969; Herrick & Bromberger, 1965; Millenson & Hurwitz, 1961; Neuringer, Kornell, & Olufs, 2001; Notterman, 1959; Stebbins & Lanson, 1962).
There is strong evidence that this increase in the variation of behavior is dependent on the animal’s Pavlovian expectation of appetitive (i.e., positive) events. Gharib, Derby, and Roberts (2001) trained rats to lever-press on a task called the peak procedure (Roberts, 1981). A trial would begin with the delivery of a discrete stimulus (e.g., a tone) that would be presented continuously throughout the trial. On 80 percent of trials, rats were reinforced with a food pellet following the first response after 40 seconds from the start of the trial. The remaining trials were probe trials, which lasted 195 seconds and terminated without food reinforcement. Gharib et al. found that the variability of rats’ bar-press durations increased significantly following the point at which the animal could predict an omission of food reward on a given probe trial (i.e., after 40 seconds on a trial). These results led Gharib et al. to propose a rule: Reduction in reward expectation increases variation in the form of behavior. A subsequent study confirmed the rule: the interresponse times of bar-pressing behavior in rats were more variable in the presence of a discriminative stimulus that signaled a low probability of reward, as compared to a stimulus signaling a high probability of reward (Gharib, Gade, & Roberts, 2004).
Neuringer et al. (2001, Experiment 3) conducted a study to investigate operant variation and extinction. They trained rats to respond across three operanda (a key, a left lever, and a right lever). They trained two groups of rats: a VAR group was reinforced for performing response sequences that were relatively uncommon, while a REP group was required to respond in a single response sequence (e.g., left-key-right) in order to obtain reinforcement. An extinction phase was introduced following acquisition of the appropriate response strategy. Unsurprisingly, during the reinforcement phase, the VAR group responded with significantly greater variability across the operanda relative to the REP group. However, both groups were more likely to meet the variability criterion (i.e., they both responded with greater variability than in initial training) during the extinction phase. Interestingly, in extinction, there was no difference between the groups in the amount of elicited behavioral variability. An additional interesting note: the operant chambers in which the rats were trained were equipped with a pair of response keys that were never reinforced during the acquisition phase. During initial learning, rats in both groups learned that responses to these operanda were not reinforced, and quickly stopped responding. However, during the extinction phase, responding to these keys rapidly and dramatically increased. Neuringer et al. (2001) write:
Variability increases in extinction because of the relatively large increases in low-probability behaviors. When reinforcers are no longer forthcoming, subjects occasionally try something different. (p. 90, emphasis added)
Our recent work has extended the examination of expectation-controlled respondent variability in behavior to open-field foraging behavior in rats (Stahlman & Blaisdell, 2011a); spatial and temporal variability of instrumental key-pecking behavior in pigeons (Stahlman, Roberts, & Blaisdell, 2010; Stahlman & Blaisdell, 2011b); and pigeons’ Pavlovian conditioned key-pecking behavior (Stahlman, Young, & Blaisdell, 2010). The latter example is particularly interesting because we find that the variability of behavior is dependent on the likelihood of food delivery even when the behavior is entirely inconsequential. In this study, pigeons observed pairings between colored discs and grain delivery; some discs (e.g., red) were consistently followed with a relatively high probability of grain delivery (e.g., 100%), while other discs were followed with a relatively low probability of food (e.g., 1%). In pigeons, the pairing of a visual stimulus with subsequent grain delivery will typically result in an increase in pecking to the visual stimulus over cumulative training trials. Despite its impotence, pecking is acquired and maintained virtually indefinitely. In our experiment, we found that behavior was more variable in both spatial and temporal domains on trials signaling a low probability of reinforcement. This indicates that the novelty of behavior, or the distribution of behavioral outputs, is an inverse function of Pavlovian expectation of positive outcomes.
In humans, a compelling case can be made for the role of reward expectation modulating the novelty (creativity) of behavior. There is a strong association between mood disorders and human creativity. Anecdotally, some of the most creative and greatest artistic minds in recent history are those who were diagnosed with or thought to have suffered from major depressive disorder or bipolar disorder (e.g., Jackson Pollock, Sylvia Plath, Ernest Hemingway, Virginia Woolf). There is a great deal of quantified evidence for a link between mood disorders characterized by depression (i.e., major depressive disorder, manic-depressive disorder) and creative behavior (e.g., Akinola & Mendes, 2008; Andreasen, 1987; Jamison, 1989, 1997; Lauronen, Veijola, Isohanni, Jones, Nieminen, & Isohanni, 2004; Richards, Kinney, Lunde, Benet, & Merzel, 1988; Simeonova, Chang, Strong, & Ketter, 2005). An important and pervasive characteristic of depressed individuals is that they report feeling helpless and hopeless, that nothing good will happen regardless of their actions. In an important way, this seems analogous to a situation whereby a pigeon has learned that its pecking behavior will not result in a positive outcome. As the pigeon engages in highly novel and variable behavior induced by low reinforcement expectation, so too might depressed individuals. We will cover this in more detail below, where we discuss the reward circuitry of the brain as it pertains to variation in behavior.
It makes a good deal of sense for an animal to behave with greater variability if reinforcement is unlikely. Gharib et al. (2004) describe the adaptive functionality of the relationship between variability and reward expectation well:
If an animal’s actions vary too little, it will not find better ways of doing things; if they vary too much, rewarded actions will not be repeated. So at any time there is an optimal amount of variation, which changes as the costs and benefits of variation change. Animals that learn instrumentally would profit from a mechanism that regulates variation so that the actual amount is close to the optimal amount. (p. 271)
The results of our study in a Pavlovian task with pigeons support Gharib et al.’s (2004) assertion, or at least the part of it related to the production of variable behavior. (The data pertaining to individuals with mood disorders are similarly supportive of Gharib et al.’s position.) On trials with a specifically low probability of grain delivery, the pigeons occasionally pecked on the touchscreen at a location very far from the stimulus target. Let’s say that I selected a region of space (away from the touchscreen) to designate as a “secret cache” of reward, such that a peck to that off-target location produced a certain reward. Pigeons pecking with high variability to the screen would presumably be more likely to discover this secret cache than pigeons with a narrow distribution of response location (e.g., on high-probability trials).
Empirical evidence for the negative effect of reinforcement on behavioral variability with respect to creativity comes from studies showing that variability tends to decrease as an animal draws nearer to reward (Gharib et al., 2001; Neuringer, 1991; Schwartz, 1982). The observation that variability tends to decrease with approach to reinforcers certainly suggests that reinforcement interferes with production of novel behavior (Cherot et al., 1996). Reinforcement of variability tends to increase total levels of variability (e.g., Page & Neuringer, 1985), but as outcomes become more proximal on a given trial, variability decreases. Each of these opposing effects of reinforced variability may have implications for the production of creative behavior.
Instrumental learning requires the selection of an action from numerous alternative behavioral options. Action is selected through differential reinforcement. As the likelihood of reinforcement decreases, the selective force governing instrumental behavior tends to relax, resulting in more variety in behavior. This increase in the variety of output increases the likelihood that the animal will stumble upon a behavioral option that is more valuable in terms of its consequences, at which time the animal can reduce the variability of its response around the new action.
Neurobiology of Behavioral Variation
An examination of the neurobiology that underlies variability and sterotypy in behavior reveals a great deal of complexity in the neural architecture employed in both reward processing and in expectation generation. Nevertheless, an inspection of the literature indicates that the structures of the basal ganglia, specifically, are critical to learning and to the production of behavioral variability.
A great amount of research supports the role of the basal ganglia in variability production. Abnormalities of the cortico-basal ganglia circuits have been linked to both motor and cognitive repetition (e.g., Leckman, 2002). Furthermore, direct manipulation of brain chemistry with respect to the basal ganglia’s dopaminergic structures has been found to alter the levels of behavioral variation in animals; severe stereotypies can be induced by administration of dopamine and opioid agonists (Canales & Graybiel, 2000; Saka, Goodrich, Harlan, Madras, & Graybiel, 2004; Saka, Iadarola, Fitzgerald, & Graybiel, 2002), and can be reduced by delivery of dopamine antagonists, such as haloperidol (Devenport, Devenport, & Holloway, 1981). Saka et al. (2004) found a strong correlation between activation of the striatum (especially the putamen) and cocaine-induced behavioral stereotypy in squirrel monkeys. Other experiments have found that opiate agonist-induced disinhibition of nigrostriatal dopaminergic projections induces strong stereotypy in rats (Iwamoto & Way, 1977). Ultimately, drug-induced stereotypies seem to result from behavioral disinhibition caused by abnormal functioning of the dorsal striatum and, by extension, an imbalance of neural activation in favor of the striatonigral pathway of the basal ganglia. Specifically, pharmacological manipulations that increase the activity of the direct, striatonigral pathway of the basal ganglia increase motor stereotypy; similarly, lesions or pharmacological down-regulations of the activity of the indirect, striatopallidal pathway of the basal ganglia induce stereotypy (Garner & Mason, 2002; Lewis & Kim, 2009; Presti & Lewis, 2005). Conversely, the administration of dopaminergic agonists that selectively target the striatopallidal pathway increases variability in behavior (Longoni et al., 1991). J. P. Garner comments on the competition between basal ganglia pathways with respect to variable and stereotyped behavior: “Broadly speaking, stereotypy can thus be reduced by drugs that activate the indirect pathway or suppress the direct pathway; while stereotypy is selectively induced by drugs that suppress the indirect pathway” (in Lewis, Presti, Lewis, & Turner, 2006, p. 210).
Stereotypic behavior can be elicited in other ways besides through the administration of psychoactive stimulants; for example, captive animals frequently exhibit spontaneous stereotypic behavior. These “cage stereotypies” are thought to be the product of stress, coupled with low behavioral competition due to environmental conditions lacking in complexity. Recent work in deer mice indicates that animals with relatively low activity in the subthalamic nucleus (a component of the indirect pathway of the basal ganglia) show higher rates of cage stereotypies (Tanimura, King, Williams, & Lewis, 2011); environmental enrichment both reduces the amount of stereotypic behavior and normalizes the activity of the subthalamic nucleus. Studies indicate that dysfunction and abnormalities of the basal ganglia are also related to spontaneous stereotypic behavior in laboratory rats (Garner & Mason, 2002), horses (McBride & Hemmings, 2005), parrots (Garner, Meehan, & Mench, 2003), and humans (e.g., Mink & Pleasure, 2003). The neurobiological correlates of stereotypic behavior seem to be conserved across phylogenetically distant species.
There has been a recent recognition of the role of the reward circuitry of the brain and its relationship to depression in humans. For example, Nestler and Carlezon (2006) present data that the behavioral symptomology of depression is, at least in part, due to malfunctioning of the striatum and ventral tegmentum, both of which are critical components of the basal ganglia circuit. There is mounting evidence that the structures of the basal ganglia are implicated in behavior relevant to depressive disorder in humans (Krishnan et al., 2007; Krishnan & Nestler, 2008). As we discussed above, an individual who displays depressive symptomology (as in major depressive disorder or bipolar disorder) is more likely than the general population to be involved in creative work (e.g., as an artist). It is therefore not surprising to find that the neural regions that are implicated in the production of stereotypic and variable behavior are also implicated in depression.
Study of spontaneous and drug-induced stereotypic behavior is important for our understanding of the role of the basal ganglia in producing stereotypic behavior. However, our primary focus in this chapter has been on behavior in associative preparations; we are interested in variation produced by direct reinforcement of behavioral variation (e.g., Page & Neuringer, 1985) and variation driven by reductions in the predicted amount of reinforcement (e.g., Stahlman et al., 2010a). Recently, a large amount of research has been devoted to investigating the role of the basal ganglia in instrumental behavior; normal functioning of the components of the ganglia appear to be critical for normal functioning in instrumental learning procedures. The nucleus accumbens (e.g., Hernandez, Sadeghian, & Kelley, 2002; Koch, Schmid, & Schnitzler, 2000; Salamone, Correa, Farrar, & Mingote, 2007; Wyvell & Berridge, 2000) and the striatum (e.g., Wiltgen, Law, Ostlund, Mayford, & Balleine, 2007; Yin & Knowlton, 2006; Yin, Knowlton, & Balleine, 2006) seem to be important for the acquisition and performance of instrumental actions.
A rapidly increasing body of literature suggests that the basal ganglia are of principal importance both for expectation and in regulating behavioral variability particularly in associative preparations. In rhesus macaques, information regarding the size of an expected reward is encoded by neurons in the anterior striatum (Cromwell & Schultz, 2003); other studies have demonstrated that motor behavior in monkeys is shaped by incentive value encoded in the basal ganglia circuit (Pasquereau et al., 2007), and that neural activity in the caudate nucleus accurately predicts both rewarded and unrewarded action (Watanabe, Lauwereyns, & Hikosaka, 2003). There is evidence to support the role for prefrontal cortical structures in modulating the behavior in the striatum during reward encoding (Staudinger, Erk, & Walter, 2011). Graybiel (2005), among others, has suggested that reinforcement signals (e.g., magnitude and likelihood of reward) are instantiated in the basal ganglia. In addition, she suggests that the basal ganglia are critically important in maintaining the balance of exploration and exploitation in conditioned animal behavior, thereby optimizing response output to the expected conditions of reward. It is important to note the confluence of this suggestion with Gharib et al.’s (2004) argument that variability in behavior must be appropriately modulated as the costs and benefits of variation change.
Neurobiological evidence from songbirds indicates that the functionality of the basal ganglia with respect to production of variation is conserved across even phylogenetically distant relatives. Brainard & Doupe (2000) discovered that lesions of the lateral magnocellular nucleus of the anterior nidopallium (LMAN, an avian cortical-basal ganglia circuit) result in unusual stereotypy in song in male zebra finches. Recent studies (Brainard & Doupe, 2001; Kao & Brainard, 2006; Kao, Doupe, & Brainard, 2005) have corroborated the existence of a positive correlation between variation in song and activity of the LMAN in male zebra finches. As a zebra finch male ages, the activity of the LMAN, the variation in its song output, and its ability to modulate its song all decrease (Kao & Brainard, 2006). These findings support the hypothesis that the basal ganglia are critical for the production and modulation of song variation; importantly, this variability in song is adaptive (i.e., not mere noise), allowing finches to rapidly learn to shift the pitch of their songs to avoid an external disruptor (Tumer & Brainard, 2007). Dopaminergic connections within the circuitry of the basal ganglia are critically important to modulate song variability in adult songbirds (Leblois, Wendel, & Perkel, 2010). The adaptability of the control of variability in behavior is not confined to songbirds; indeed, research with mice (Tanimura, Yang, & Lewis, 2008), voles (Garner & Mason, 2002), bears (Vickery & Mason, 2005), and humans (e.g., Neuringer, 2002) confirms the relationship of variability production with the generation of adaptive action.
Conclusions
The topics covered in this chapter would seem to be important pieces of the creativity puzzle. The individual must be able to engage in novel action in order for creativity to be a possibility. As behavior must be in some way novel to be described as creative, the animal must possess mechanisms related to the production of novel action. In this chapter, we have described associative processes as being heavily involved in the induction of novel actions. Theoretical accounts of expectation-induced variability (and stereotypy) in behavior suggest that animals engage in novel or unusual behavior in situations where they have learned that positive reinforcement is unlikely (Gharib et al., 2001). Similarly, direct reinforcement of variation in behavior will produce an animal that has a wider range of action, approaching a random distribution of the response measure (Neuringer, 2004). Indeed, when the variability of its behavior itself is reinforced, the animal may engage in such novel, random, and/or complex actions that they become functionally impossible to keep track of (Pryor et al., 1969). Importantly, operant variability appears to reflect instrumental control of a stochastic process, rather than the dynamics of a memory process.
In summary, we have much to learn about the production of novel and variable action by examining nonhuman animals. Associative conditioning preparations can engender high variability in behavior, which provides the animal with additional options for action. The engine that produces novelty in behavior is related to the interplay of the animal’s ability to predict (1) whether reinforcement will occur, and 2) what manner of response will be followed by reinforcement. Any complete account of creative behavior should incorporate these generative processes as a foundation upon which other processes can be built.
Acknowledgments
Support for the production of this chapter was provided by NIH Grant NS059076 (A. P. Blaisdell). Requests for reprints should be addressed to Aaron P. Blaisdell, UCLA Department of Psychology, 1285 Franz Hall, Box 951563, Los Angeles, CA 90095–1563, USA; e-mail: blaisdell@psych.ucla.edu.
Note
1. There are many other facets relevant to a discussion of creativity in animals (including the human animal) beyond the mechanisms discussed in this chapter: the directed combination of separately learned behaviors or “insight” (e.g., Epstein, Kirshnit, Lanza, & Rubin, 1984; Kohler, 1925); inferences and the spontaneous integration of spatiotemporal maps (Blaisdell, 2009; Leising & Blaisdell, 2009; Leising, Sawa, & Blaisdell, 2007; Sawa, Leising, & Blaisdell, 2005); and causal reasoning (Blaisdell, Sawa, Leising, & Waldmann, 2006), to just name a few.
References
Akinola, M., & Mendes, W. B. (2008). The dark side of creativity: Biological vulnerability and negative emotions lead to greater artistic creativity. Personality and Social Psychology Bulletin, 34, 1677–1686.
Andreasen, N. C. (1987). Creativity and mental illness: Prevalence rates in writers and their first-degree relatives. American Journal of Psychiatry, 144, 1288–1292.
Antonitis, J. J. (1951). Response variability in the white rat during conditioning, extinction, and reconditioning. Journal of Experimental Psychology, 42, 273–281.
Arnesen, E. M. (2000). Reinforcement of object manipulation increases discovery. Unpublished undergraduate thesis, Reed College.
Balsam, P. D., Deich, J. D., Ohyama, T., & Stokes, P. D. (1998). Origins of new behavior. Needham Heights, MA: Allyn & Bacon.
Bartlett, R., Wheat, J., & Robins, M. (2007). Is movement variability important for sports biomechanists? Sports Biomechanics, 6, 224–243.
Blaisdell, A. P. (2009). The role of associative processes in spatial, temporal, and causal cognition. In S. Watanabe, A. P. Blaisdell, L. Huber, & A. Young (Eds.), Rational animals, irrational humans (pp. 153–172). Tokyo: Keio University.
Blaisdell, A. P., Sawa, K., Leising, K. J., & Waldmann, M. R. (2006). Causal reasoning in rats. Science, 311, 1020–1022.
Blough, D. S. (1966). The reinforcement of least-frequent interresponse times. Journal of the Experimental Analysis of Behavior, 9, 581–591.
Boden, M. A. (2004). The creative mind: Myths and mechanisms. New York: Routledge.
Brainard, M. S., & Doupe, A. J. (2000). Interruption of a basal ganglia-forebrain circuit prevents plasticity of learned vocalizations. Nature, 404, 762–766.
Brainard, M. S., & Doupe, A. J. (2001). Postlearning consolidation of birdsong: Stabilizing effects of age and anterior forebrain lesions. Journal of Neuroscience, 21, 2501–2517.
Brembs, B. (2011). Towards a scientific concept of free will as a biological trait: Spontaneous actions and decision-making in invertebrates. Proceedings of the Royal Society of London, Series B: Biological Sciences, 278, 930–939.
Bryant, D., & Church, R. M. (1974). The determinants of random choice. Animal Learning & Behavior, 2, 245–248.
Canales, J. J., & Graybiel, A. M. (2000). A measure of striatal function predicts motor stereotypy. Nature Neuroscience, 3, 377–383.
Cherot, C., Jones, A., & Neuringer, A. (1996). Reinforced variability decreases with approach to reinforcers. Journal of Experimental Psychology: Animal Behavior Processes, 22, 497–508.
Cohen, L., Neuringer, A., & Rhodes, D. (1990). Effects of ethanol on reinforced variations and repetitions by rats under a multiple schedule. Journal of the Experimental Analysis of Behavior, 54, 1–12.
Cook, R. G., Brown, M. F., & Riley, D. A. (1985). Flexible memory processing by rats: Use of prospective and retrospective information in the radial maze. Journal of Experimental Psychology: Animal Behavior Processes, 11, 453–469.
Cook, R. G., Levison, D. G., Gillett, S. R., & Blaisdell, A. P. (2005). Capacity and limits of associative memory in pigeons. Psychonomic Bulletin & Review, 12, 350–358.
Cromwell, H. C., & Schultz, W. (2003). Effects of expectations for different reward magnitudes on neuronal activity in primate striatum. Journal of Neurophysiology, 89, 2823–2838.
Dennett, D. C. (1975). Why the law of effect will not go away. Journal for the Theory of Social Behaviour, 5, 169–188.
Denney, J., & Neuringer, A. (1998). Behavioral variability is controlled by discriminative stimuli. Animal Learning & Behavior, 26, 154–162.
Devenport, L. D., Devenport, J. A., & Holloway, F. A. (1981). Reward-induced stereotypy: Modulation by the hippocampus. Science, 212, 1288–1289.
Eckerman, D. A., & Lanson, R. N. (1969). Variability of response location for pigeons responding under continuous reinforcement, intermittent reinforcement, and extinction. Journal of the Experimental Analysis of Behavior, 12, 73–80.
Epstein, R. (1990). Generativity theory and creativity. In M. A. Runco & R. S. Albert (Eds.), Theories of Creativity (pp. 116–140). Thousand Oaks, CA: Sage Publications.
Epstein, R. (1991). Skinner, creativity, and the problem of spontaneous behavior. Psychological Science, 2, 362–370.
Epstein, R., Kirshnit, C. E., Lanza, R. P., & Rubin, L. C. (1984). “Insight” in the pigeon: Antecedents and determinants of an intelligent performance. Nature, 308, 61–62.
Garner, J. P., Meehan, C. L., & Mench, J. A. (2003). Stereotypies in caged parrots, schizophrenia, and autism: Evidence for a common mechanism. Behavioural Brain Research, 145, 125–134.
Garner, J. P., & Mason, G. J. (2002). Evidence for a relationship between cage stereotypies and behavioural disinhibition in laboratory rodents. Behavioural Brain Research, 136, 83–92.
Gharib, A., Gade, C., & Roberts, S. (2004). Control of variation by reward probability. Journal of Experimental Psychology: Animal Behavior Processes, 30, 271–282.
Gharib, A., Derby, S., & Roberts, S. (2001). Timing and the control of variation. Journal of Experimental Psychology: Animal Behavior Processes, 27, 165–178.
Graybiel, A. M. (2005). The basal ganglia: Learning new tricks and loving it. Current Opinion in Neurobiology, 15, 638–644.
Grunow, A., & Neuringer, A. (2002). Learning to vary and varying to learn. Psychonomic Bulletin & Review, 9, 250–258.
Hernandez, P. J., Sadeghian, K., & Kelley, A. E. (2002). Early consolidation of instrumental learning requires protein synthesis in the nucleus accumbens. Nature Neuroscience, 5, 1327–1331.
Herrick, R. M., & Bromberger, R. A. (1965). Lever displacement under a variable ratio schedule and subsequent extinction. Journal of Comparative and Physiological Psychology, 59, 392–398.
Iwamoto, E. T., & Way, E. L. (1977). Circling behavior and stereotypy induced by intranigral opiate microinjections. Journal of Pharmacology and Experimental Therapeutics, 203, 347–359.
Jamison, K. R. (1989). Mood disorders and patterns of creativity in British writers and artists. Psychiatry: Journal for the Study of Interpersonal Processes, 52, 125–134.
Jamison, K. R. (1997). Manic-depressive illness and creativity. Scientific American (February): 44–49.
Kao, M. H., & Brainard, M. S. (2006). Lesions of an avian basal ganglia circuit prevent context-dependent changes to song variability. Journal of Neurophysiology, 96, 1441–1455.
Kao, M. H., Doupe, A. J., & Brainard, M. S. (2005). Contributions of an avian basal ganglia-forebrain circuit to real-time modulation of song. Nature, 433, 638–643.
Kesner, R. P., & Despain, M. J. (1988). Correspondence between rats and humans in the utilization of retrospective and prospective codes. Animal Learning & Behavior, 16, 299–302.
Koch, M., Schmid, A., & Schnitzler, H. U. (2000). Role of nucleus accumbens dopamine D1 and D2 receptors in instrumental and Pavlovian paradigms of conditioned reward. Psychopharmacology, 152, 67–73.
Kohler, W. (1925). The mentality of apes. London: Routledge & Kegan Paul.
Krishnan, V., Han, M. H., Graham, D. L., Berton, O., Renthal, W., Russo, S. J., et al. (2007). Molecular adaptations underlying susceptibility and resistance to social defeat in brain reward regions. Cell, 131, 391–404.
Krishnan, V., & Nestler, E. J. (2008). The molecular neurobiology of depression. Nature, 455, 894–902.
Lauronen, E., Veijola, J., Isohanni, I., Jones, P. B., Nieminen, P., & Isohanni, M. (2004). Links between creativity and mental disorder. Psychiatry, 67, 81–98.
Leblois, A., Wendel, B. J., & Perkel, D. J. (2010). Striatal dopamine modulates basal ganglia output and regulates social context-dependent behavioral variability through D1 receptors. Journal of Neuroscience, 30, 5730–5743.
Leckman, J. (2002). Tourette’s syndrome. Lancet, 360, 1577–1586.
Leising, K. J., & Blaisdell, A. P. (2009). Associative basis of landmark learning and integration in vertebrates. Comparative Cognition & Behavior Reviews, 4, 80–102.
Leising, K. J., Sawa, K., & Blaisdell, A. P. (2007). Temporal integration in Pavlovian appetitive conditioning in rats. Learning & Behavior, 35, 11–18.
Lewis, M., & Kim, S. (2009). The pathophysiology of restricted repetitive behavior. Journal of Neurodevelopmental Disorders, 1, 114–132.
Lewis, M. H., Presti, M. F., Lewis, J. B., & Turner, C. A. (2006). The neurobiology of stereotypy. In G. Mason & J. Rushen (Eds.), Stereotypic animal behaviour: Fundamentals and applications to welfare (2nd Ed., pp. 190–226). Cambridge, MA: CABI.
Longoni, R., Spina, L., Mulas, A., Carboni, E., Garau, L., Melchiorri, P., et al. (1991). (d-Ala2)deltorphin II: D1-dependent stereotypies and stimulation of dopamine release in the nucleus accumbens. Journal of Neuroscience, 11, 1565–1576.
McBride, S. D., & Hemmings, A. (2005). Altered mesoaccumbens and nigro-striatal dopamine physiology is associated with stereotypy development in a non-rodent species. Behavioural Brain Research, 159, 113–118.
McElroy, E., & Neuringer, A. (1990). Effects of alcohol on reinforced repetitions and reinforced variations in rats. Psychopharmacology, 102, 49–55.
Millenson, J. R., & Hurwitz, H. M. B. (1961). Some temporal and sequential properties of behavior during conditioning and extinction. Journal of the Experimental Analysis of Behavior, 4, 97–106.
Mink, J. W., & Pleasure, D. E. (2003). The basal ganglia and involuntary movements: Impaired inhibition of competing motor patterns. Archives of Neurology, 60, 1365–1368.
Morgan, L., & Neuringer, A. (1990). Behavioral variability as a function of response topography and reinforcement contingency. Animal Learning & Behavior, 18, 257–263.
Nestler, E. J., & Carlezon, W. A., Jr. (2006). The mesolimbic dopamine reward circuit in depression. Biological Psychiatry, 59, 1151–1159.
Neuringer, A. (1986). Can people behave “randomly”? The role of feedback. Journal of Experimental Psychology: General, 115, 62–75.
Neuringer, A. (1991). Operant variability and repetition as functions of interresponse time. Journal of Experimental Psychology: Animal Behavior Processes, 17, 3–12.
Neuringer, A. (1993). Reinforced variation and selection. Animal Learning & Behavior, 21, 83–91.
Neuringer, A. (2002). Operant variability: Evidence, functions, and theory. Psychonomic Bulletin & Review, 9, 672–705.
Neuringer, A. (2004). Reinforced variability in animals and people: Implications for adaptive action. American Psychologist, 59, 891–906.
Neuringer, A., Deiss, C., & Olson, G. (2000). Reinforced variability and operant learning. Journal of Experimental Psychology: Animal Behavior Processes, 26, 98–111.
Neuringer, A., & Jensen, G. (2010). Operant variability and voluntary action. Psychological Review, 117, 972–993.
Neuringer, A., Kornell, N., & Olufs, M. (2001). Stability and variability in extinction. Journal of Experimental Psychology: Animal Behavior Processes, 27, 79–94.
Notterman, J. M. (1959). Force emission during bar pressing. Journal of Experimental Psychology, 58, 341–347.
Page, S., & Neuringer, A. (1985). Variability is an operant. Journal of Experimental Psychology: Animal Behavior Processes, 11, 429–452.
Pasquereau, B., Nadjar, A., Arkadir, D., Bezard, E., Goillandeau, M., Bioulac, B., et al. (2007). Shaping of motor responses by incentive values through the basal ganglia. Journal of Neuroscience, 27, 1176–1183.
Presti, M. F., & Lewis, M. H. (2005). Striatal opioid peptide content in an animal model of spontaneous stereotypic behavior. Behavioural Brain Research, 157, 363–368.
Pryor, K. W., Haag, R., & O’Reilly, J. (1969). The creative porpoise: Training for novel behavior. Journal of the Experimental Analysis of Behavior, 12, 653–661.
Richards, R., Kinney, D. K., Lunde, I., Benet, M., & Merzel, A. P. C. (1988). Creativity in manic-depressives, cyclothymes, their normal relatives, and control subjects. Journal of Abnormal Psychology, 97, 281–288.
Roberts, S. (1981). Isolation of an internal clock. Journal of Experimental Psychology: Animal Behavior Processes, 7, 242–268.
Ross, C., & Neuringer, A. (2002). Reinforcement of variations and repetitions along three independent response dimensions. In Behavioural Processes, Special Issue: Proceedings of the Meeting of the Society for the Quantitative Analyses of Behavior (SQAB 2001), 57(2–3), 199–209.
Salamone, J. D., Correa, M., Farrar, A., & Mingote, S. M. (2007). Effort-related functions of nucleus accumbens dopamine and associated forebrain circuits. Psychopharmacology, 191, 461–482.
Saka, E., Goodrich, C., Harlan, P., Madras, B. K., & Graybiel, A. M. (2004). Repetitive behaviors in monkeys are linked to specific striatal activation patterns. Journal of Neuroscience, 24, 7557–7565.
Saka, E., Iadarola, M., Fitzgerald, D. J., & Graybiel, A. M. (2002). Local circuit neurons in the striatum regulate neural and behavioral responses to dopaminergic stimulation. Proceedings of the National Academy of Sciences, 99, 9004–9009.
Sawa, K., Leising, K. J., & Blaisdell, A. P. (2005). Sensory preconditioning in spatial learning using a touch screen task in pigeons. Journal of Experimental Psychology: Animal Behavior Processes, 31, 368–375.
Schoenfeld, W. N., Harris, A. H., & Farmer, J. (1966). Conditioning response variability. Psychological Reports, 19, 551–557.
Schwartz, B. (1982). Interval and ratio reinforcement of a complex sequential operant in pigeons. Journal of the Experimental Analysis of Behavior, 37, 349–357.
Simeonova, D. I., Chang, K. D., Strong, C., & Ketter, T. A. (2005). Creativity in familial bipolar disorder. Journal of Psychiatric Research, 39, 623–631.
Skinner, B. F. (1966). An operant analysis of problem solving. In B. Kleinmuntz (Ed.), Problem solving: Research, method, and theory (pp. 225–257). New York: John Wiley.
Skinner, B. F. (1970). Creating the creative artist. In A. J. Toynbee, et al. (Eds.), On the future of art (pp. 61–75). New York: Viking Press.
Stahlman, W. D., & Blaisdell, A. P. (2011a). Reward probability and the variability of foraging behavior in rats. International Journal of Comparative Psychology, 24, 168–176.
Stahlman, W. D., & Blaisdell, A. P. (2011b). The modulation of response variation by the probability, magnitude, and delay of reinforcement. Learning and Motivation, 42, 221–236.
Stahlman, W. D., Roberts, S., & Blaisdell, A. P. (2010a). Effect of reward probability on spatial and temporal variation. Journal of Experimental Psychology: Animal Behavior Processes, 36, 77–91.
Stahlman, W. D., Young, M. E., & Blaisdell, A. P. (2010b). Response variability in pigeons in a Pavlovian task. Learning & Behavior, 38, 111–118.
Staudinger, M. R., Erk, S., & Walter, H. (2011). Dorsolateral prefrontal cortex modulates striatal reward encoding during reappraisal of reward anticipation. Cerebral Cortex, 21, 2578–2588.
Stebbins, W. C., & Lanson, R. N. (1962). Response latency as a function of reinforcement schedule. Journal of the Experimental Analysis of Behavior, 5, 299–304.
Tanimura, Y., King, M. A., Williams, D. K., & Lewis, M. H. (2011). Development of repetitive behavior in a mouse model: Roles of indirect and striosomal basal ganglia pathways. International Journal of Developmental Neuroscience, 29, 461–467.
Tanimura, Y., Yang, M. C., & Lewis, M. H. (2008). Procedural learning and cognitive flexibility in a mouse model of restricted, repetitive behavior. Behavioural Brain Research, 189, 250–256.
Thorndike, E. L. (1927). The law of effect. American Journal of Psychology, 39, 212–222.
Tumer, E. C., & Brainard, M. S. (2007). Performance variability enables adaptive plasticity of crystallized adult birdsong. Nature, 450, 1240–1244.
Vickery, S. S., & Mason, G. J. (2005). Stereotypy and perseverative responding in caged bears: Further data and analyses. Applied Animal Behaviour Science, 91, 247–260.
Watanabe, K., Lauwereyns, J., & Hikosaka, O. (2003). Neural correlates of rewarded and unrewarded eye movements in the primate caudate nucleus. Journal of Neuroscience, 23, 10052–10057.
Weiss, R. L. (1965). Variables that influence random-generation: An alternative hypothesis. Perceptual and Motor Skills, 20, 307–310.
Wiltgen, B. J., Law, M., Ostlund, S., Mayford, M., & Balleine, B. W. (2007). The influence of Pavlovian cues on instrumental performance is mediated by CaMKII activity in the striatum. European Journal of Neuroscience, 25, 2491–2497.
Wyvell, C. L., & Berridge, K. C. (2000). Intra-accumbens amphetamine increases the conditioned incentive salience of sucrose reward: Enhancement of reward “wanting” without enhanced “liking” or response reinforcement. Journal of Neuroscience, 20, 8122–8130.
Yin, H. H., & Knowlton, B. J. (2006). The role of the basal ganglia in habit formation. Nature Reviews: Neuroscience, 7, 464–476.
Yin, H. H., Knowlton, B. J., & Balleine, B. W. (2006). Inactivation of dorsolateral striatum enhances sensitivity to changes in the action-outcome contingency in instrumental conditioning. Behavioural Brain Research, 166, 189–196.