2 Classical Conditioning: Effects of Regularities in the Presence of Multiple Stimuli
After reading this chapter, you should be able to:
- Indicate under what conditions classical conditioning (as an effect) will occur.
- Give an overview of the core assumptions of the most important mental process theories of classical conditioning.
Introductory Task
Pairing two stimuli together will sometimes lead to changes in behavior—but sometimes not. Make a list of factors that you think might influence whether classical conditioning (as an effect) will occur. Do you think that the same factors always have the same impact regardless of the nature of the behavior that changes?
2.1 Some Basic Terms and Procedures
2.1.1 Basic Terms
In this chapter we discuss classical conditioning, or the impact of regularities between stimuli on behavior (see http://www.youtube.com/watch?v=Eo7jcI8fAuI for a real-life example). The prototypical example of classical conditioning is Pavlov’s dog salivating whenever it hears a bell because the bell was always followed by food. The bell is called the conditional stimulus (CS). The food is called the unconditional stimulus (US). The increased salivation after hearing the bell is called the conditional response (CR). Research into classical conditioning aims to understand when (functional approach) and how (cognitive approach) the regularity in the presence of the CS and the US leads to a CR toward the CS.
First, we must note that it is not always clear whether a stimulus is functioning as a CS or a US. We choose to make the distinction by defining a stimulus as a CS if one checks whether reactions to this stimulus are conditional (i.e., dependent on the relation between that stimulus and other stimuli). The US is the stimulus that is presented together with the CS and that, through its relation with the CS, comes to influence reactions to the CS. If a certain reaction to the CS depends on the relation between the CS and the US, we call that reaction a CR. The response to the US is called the unconditional response (UR). The advantage of these definitions is that they are not dependent on mental process theories about conditioning and do not exclude any type of stimulus or reaction from the realm of classical conditioning research (De Houwer, 2011a).
Figure 2.1
Ivan Pavlov.
One last terminological point. If there are different CSs present in a given situation, they are often referred to by different letters of the alphabet (e.g., A, B, X, Y). The presence of a US is often expressed by the + sign and its absence by the – sign. Therefore, the notation A+, for example, refers to a situation where CS(A) and the US are both present, and the notation AX- refers to a situation where both CS(A) and CS(X) are present but the US is absent.
2.1.2 Procedures
Although Pavlov’s procedure for conditioning a salivation response is perhaps the most famous example of classical conditioning, it is worth noting that it is rarely used in contemporary conditioning research (see Van Gucht et al., 2008, for one of the few studies on conditioning of salivation responses in humans), mainly because it is highly impractical to measure how much an organism salivates. To do so we would have to surgically operate on the person (as in Pavlov’s research) or work with cotton balls in the mouth to catch the saliva and weigh it afterward (as in Van Gucht et al.’s research). Researchers have therefore developed procedures that are much more practical. To illustrate, let’s consider two types of classical conditioning procedures: conditioning of the eyeblink reflex and fear conditioning (see Bouton, 2016, pp. 85–90, for a more extensive overview).
In studies on the conditioning of an eyeblink reflex, a device is used in humans or nonhuman animals that administers a short pulse of air to the eye. Such an air blast causes an eyeblink reflex. In this case, the air blast is the US and the blink reflex is the UR. Now imagine that prior to each blast of air we present a CS such as a tone or light. As a result of the joint presentation of the CS and US, the CS will also elicit a blink reflex over time (e.g., Gormezano, Kehoe, & Marshall, 1983; Thompson & Steinmetz, 2009). One can record the blinking of the eyelid automatically via electrodes placed on the eyelids, and the presentation of the CS and US can be controlled by a computer.
In studies on fear conditioning, researchers are concerned with changes in a variety of reactions that are assumed to index fear or anxiety (see Lonsdorf et al., 2017, for a methodological overview). For example, sweating (e.g., moist palm) may be an indication of arousal that is part of an anxiety reaction. The moister the skin (e.g., through sweating), the greater the skin conductance (i.e., conductance of electricity over the skin). Hence, skin conductance is often used as an index of fear in fear conditioning. In studies on skin conductance conditioning, aversive stimuli such as unpleasant electric shocks (US) and neutral stimuli (CS) such as shapes (e.g., a triangle), sounds, or lights are used. Following the joint presentation of the CS and US, skin conductance will increase upon presentation of the CS.
In yet other studies on fear conditioning, behavioral suppression is used as an index of anxiety. The first step is to ensure that the organism has a stable rate of behavior. For example, rats can be taught to regularly press a lever to obtain food (see figure 2.2). When an aversive US is presented (e.g., an electrical shock), the animal will stop pressing the lever. The behavior is thus suppressed (hence the term behavioral suppression). When the presentation of the US is preceded by a CS (e.g., a light), the presentation of the CS will also suppress that behavior over time. This is called conditioned suppression. It is usually seen as an index of the degree to which the animal experiences fear of the upcoming US (see Arcediano, Ortega, & Mature, 1996, and Meulders, Vervliet, Vansteenwegen, Hermans, & Baeyens, 2011, for a conditioned suppression procedure in humans).
A rat in a Skinner box, where pressing a handle is followed by the delivery of food. An electric shock can be delivered via the rods in the floor (US). Because of that shock, the rat will temporarily stop pressing the lever (suppression). Stimuli that repeatedly precede the shock (e.g., a light) will also result in a temporary reduction of the frequency with which the rat presses the lever (conditioned suppression).
2.2 Functional Knowledge
In this section we discuss the most important functional knowledge about classical conditioning (i.e., the most important knowledge about the conditions under which regularities in the presence of two or more stimuli have an influence on behavior). Again, we organize our overview according to the type of moderator that influences conditioning.
2.2.1 The Nature of the Stimuli
2.2.1.1 Classical conditioning is a general phenomenon Research shows that classical conditioning is a very general phenomenon in that it can occur with all kinds of stimuli. The CS can be very simple (e.g., the ringing of a bell), but even a very complex event can be a CS (e.g., receiving information about the food that someone else has eaten; see studies on human contingency learning discussed in section 2.2.2.2). The US can also vary from something very simple (e.g., a piece of food) to something very complex (e.g., information about the occurrence of a particular allergic reaction in a patient). In each case, the pairing of two stimuli can lead to a change in behavior. It is precisely because of this broad applicability that classical conditioning is such an interesting phenomenon.
As we noted in the introductory chapter, some researchers talk about classical conditioning only if the US is a biologically relevant stimulus (e.g., food or a painful shock). They then use terms such as associative learning to refer to other situations in which the combination of (nonbiologically relevant) stimuli leads to changes in behavior. However, we do not see any substantive reasons for restricting classical conditioning to situations with biologically relevant USs (De Houwer, 2011a). Moreover, such a distinction is difficult to make because it is not always clear a priori which stimuli for which organisms are biologically relevant in which situations. Because our definition of the different types of learning is based on different types of environmental regularities, there is no need for us to restrict classical conditioning to certain types of stimuli.
There are two types of conditioning in which two special types of stimuli are used: observational conditioning using social stimuli and conditioning through instructions using verbal stimuli. In observational conditioning research, an observer looks at a model that shows a reaction to a stimulus. For example, a young monkey (observer) is shown a video in which an older conspecific (model) shows a fearful reaction to a snake (e.g., Mineka, 1987). The young monkey does not show any fear toward the snake prior to watching the video but does show fear toward the snake after watching it. In this instance, the snake can be seen as the CS, the fear reaction of the older monkey is the US, and the learned fear reaction of the young animal is the CR that is the result of the co-occurrence of the CS and US. Many studies support the idea that emotional reactions in humans are often based on observational conditioning (e.g., Dunne & Askew, 2018; see Debiec & Olsson, 2017, for an overview).
Observational conditioning is just one instance of a much bigger class of social learning phenomena (see Heyes, 1994; Olsson & Phelps, 2007). Starting from our definition of learning, social learning can be defined as the impact of a particular type of regularity on the behavior of an observer—namely, those regularities that (a) involve the behavior of a model as one of its elements (which is why the learning qualifies as being “social”) and (b) do not involve the observer’s behavior as one of the elements (which disqualifies learning about others, such as how to influence others, and restricts social learning to learning from others). For instance, in observational conditioning there is a regularity between two events, one of which is the behavior of a model (e.g., a fearful response) and the other is the presence of a stimulus (e.g., a snake). One could also imagine situations in which one element of the regularity is the behavior of a model (e.g., a rat pushing a lever) and the other element is the outcome of the behavior of the model (e.g., a food pellet that is presented to the model after pressing the lever). If the behavior of an observer changes as the result of being exposed to such a regularity (e.g., the observer starts pressing the lever), one could describe this as an instance of observational operant conditioning. Hence, just as different types of learning can be distinguished on the basis of the type of regularity that causes the change in behavior, so too can different types of social learning be distinguished on the basis of the type of regularity that changes behavior, though our focus is now on changes in the behavior of an observer, and all regularities involved in social learning must involve a model but not the observer. Just as classifying changes in behavior as instances of particular types of learning involves a hypothesis on the part of the researcher about the causes of the change in behavior, so too do different types of social learning involve different hypotheses about the causes of changes in behavior, with the difference that the research now seeks to explain the behavior of an observer on the basis of the behavior of others. The study of social learning can thus provide unique insights into the social origins of behavior.
Conditioning through instructions is also an important source of behavior (Rachman, 1977). From the work of Cook and Harris (1937), we know that instructions about a CS-US relation are sufficient to establish a CR. For example, if subjects are told that the presentation of a triangle (CS1) may be followed by an electric shock and that a circle (CS2) will never be followed by an electric shock, then during a subsequent test phase we will observe more fear after the presentation of the triangle than the circle. This effect occurs even when the triangle and shock have never been presented together. More recent research shows that the moderators of conditioning through instructions are very similar to the moderators of standard conditioning that involves actual pairings of the CS and US (for an overview, see Mertens, Boddez, Sevenster, Engelhard, & De Houwer, 2018). Just as social learning can involve different types of regularities, so too can instructions relate to several types of regularities (e.g., regularities in the presence of one stimulus or in the presence of stimuli and responses; see Van Dessel et al., 2015, 2017). As we will argue in chapter 4, however, we believe that the effects of observed regularities and instructions about regularities are not simple effects of the regularities themselves but instead are instances of a complex form of learning involving many different regularities.
2.2.1.2 The influence of the properties of the CS or US, and the relation between CS and US on classical conditioning effects Although classical conditioning can occur with many different stimuli, properties of the CS and US can still have an influence on the strength of the behavioral change that occurs as a result of the pairing of stimuli. An obvious factor is the intensity of the stimuli. Few readers will be surpised to learn that regularities involving more salient, intense, or biologically relevant stimuli are learned more quickly (i.e., they require fewer presentations of the CS and US in order to produce a fixed level of behavioral change) and will have more impact on behavior than less intense stimuli. For example, a regularity involving the ringing of a bell and food will impact behavior more quickly if the ringing of the bell is clearly audible than if one can barely hear it. It also seems logical that a regularity involving the ringing of a bell and a biologically relevant stimulus such as food will have a greater impact on behavior than a regularity involving the ringing of a bell and a biologically irrelevant stimulus such as the appearance of a light.
What is less obvious is that in addition to the influence of CS and US properties, there is also an influence of the intrinsic relation between those stimuli. By intrinsic relation we mean the combination of or “match” between the two stimuli. In other words, effects of the intrinsic relation between stimuli are concerned not with the properties of the CS or the US (e.g., how intense the CS is or how important the US is), but rather with the interaction between the properties of those two stimuli. Observing these interactions shows that learning can be selective in the sense that certain relations are learned more quickly than other relations. The importance of the intrinsic relation between the CS and US was demonstrated for the first time by Garcia, in the context of research on conditioned food aversion (see Freeman & Riley, 2009, for a historical analysis of Garcia’s research). Garcia studied a phenomenon called aversion learning. What we mean by aversion learning is probably best explained using the following anecdote by Seligman (see Seligman & Hager, 1972). After eating “filet mignon and béarnaise sauce,” Seligman became ill later that same night. It subsequently turned out that this nausea was actually the beginning of a bout of the flu. But Seligman had already attributed it to the béarnaise sauce, and since then, he cannot tolerate the sight, let alone the taste, of this sauce. This anecdote raises several questions (see Bouton, 2016, pp. 206–220, for an overview), one of which is related to the selectivity in learning. Why did his reaction to the béarnaise sauce change? And why did he not show an aversive reaction to the steak, the dessert, the drink, the restaurant, or even the waiter?
Similar questions emerge from the work of Garcia and colleagues. Garcia and Koelling (1966) placed rats in a cage, and during a learning phase, they could drink from a tube of water that was infused with sugar. Each time the tube was touched, a light and a noise were also automatically presented. So in this case, there were two CSs: the sweet taste of the water and the light-sound stimulus. The nature of the US was manipulated between groups. In a first group, lithium chloride was delivered after drinking, which led to nausea; in a second group, an electric shock was delivered two seconds after drinking. This was followed by the test phase, in which the two CSs were presented separately: either the light + sound stimulus was delivered during the drinking of pure water (no taste), or the water had the sugar taste but the light-sound stimulus was not presented. In the lithium group, the animals drank very little of the water with the sugar taste; it was as if that taste had become aversive for them. However, the light-sound stimulus did not affect drinking the tasteless water (see figure 2.3). In the group with electric shocks during the learning phase, the animals drank little of the water that coincided with the light-sound stimulus but much of the water with the sugar taste (see figure 2.3). The key point here is that there is an interaction between the nature of the CS (taste or light-sound) and the nature of the US (lithium or shock): when the US is lithium (a poison that leads to nausea), the taste of the water during test has more impact on drinking than the presence of the light-sound stimulus; when the US is an electric shock (which leads to pain), the light-sound stimulus has more influence on drinking than the taste of the water. Note that these differences did not occur during a pretest that took place before the learning phase. It is therefore the learning phase that was responsible for the observed interaction (see Domjan, 2015, for a discussion of the limitations of Garcia’s research and of subsequent research that has circumvented these limitations).
Idealized results from Garcia and Koelling (1966). With lithium as a US (left), the taste stimulus but not the light-sound stimulus reduces the amount of liquid consumed after the conditioning phase (post). With shock as a US (right), less liquid is consumed after conditioning in the presence of the light-sound stimulus, whereas the taste does not reduce drinking.
2.2.1.3 The impact of the nature of the US on the nature of the CR
Sometimes a distinction is made between appetitive and aversive USs. Both types of USs are stimuli that elicit an (unconditional) response. With appetitive USs, this unconditional response (UR) is positive and often related to the fulfilment of a particular biological need (e.g., hunger or thirst). The prototypical example of an appetitive US is food. Salivation is an example of an appetitive UR because it is directed at consuming food. Aversive USs elicit negative URs (e.g., fear responses). The prototypical example of an aversive US is an electric shock, whereas freezing is a common example of an aversive UR. Freezing refers to the finding that because of the presentation of an aversive stimulus, the animal will not move for a period of time (i.e., it “freezes”; Hagenaars, Oitzl, & Roelofs, 2014). Although both types of USs can lead to changes in responses to the CSs with which they are paired, there seems to be a difference between appetitive and aversive conditional responses (CRs). A CS coupled with an appetitive US often provokes positive (appetitive) responses, whereas a CS accompanied by an aversive US usually provokes a negative (aversive) reaction. This demonstrates that the properties of the US can determine the properties of the CR.
That said, there is still a great deal of uncertainty about the relation between the US (and more specifically, the UR that the US evokes) and the CR. For example, it was long thought that the CR was identical to the UR that is elicited by the US. This is indeed the case in the example of Pavlov’s dog: initially, only food elicits salivation (salivation is therefore a UR that is elicited by the food), and later, as a result of pairing the bell and food, the bell also comes to elicit salivation (salivation is therefore a CR to the bell). To this day, classical conditioning is often defined as a transfer of reactions as a result of pairing stimuli. This idea is depicted in figure 2.4.
The traditional representation of classical conditioning. (1) The US initially elicits a UR. By pairing the CS and US (2), the CS will also trigger the UR (3). The UR therefore becomes a CR.
However, research shows that the relation between the CR and US is far more nuanced than initially thought. Recent work shows that the CR can in some cases be opposite to the UR; for example, whereas morphine causes a reduction in pain, stimuli that occur repeatedly with morphine administration seem to evoke more sensitivity to pain (Siegel, 1975; see Bouton, 2016, pp. 187–193, for a discussion).
Importantly, classical conditioning not only is sensitive to the properties of the US during conditioning but also to changes in the US after conditioning. This phenomenon is called US revaluation. In a typical US revaluation study, a tone is repeatedly presented with food and this leads to the tone eliciting an appetitive CR (e.g., salivation). Thereafter, the appetitive nature of the food is changed (e.g., by pairing it with nausea). The US is thus “revaluated” and becomes negative instead of positive. Now, presentations of the tone will no longer elicit appetitive responses and sometimes even elicit aversive responses (e.g., Holland & Rescorla, 1975). As we shall see in section 2.3, this finding has important implications for mental process theories of learning.
Finally, it is worth mentioning one last phenomenon known as counterconditioning (e.g., Pearce & Dickinson, 1975). In studies on counterconditioning, the nature of the US changes during CS+ presentations. During an initial learning phase, a tone is repeatedly followed by an aversive electric shock (causing the tone to elicit fear). Next, the same tone is followed by an appetitive stimulus such as food. This change in the nature of the US that a CS is paired with also leads to a change in the nature of the CR (e.g., fear responses to the tone will weaken and gradually be replaced by appetitive reactions).
2.2.2 The Nature of the Observed Behavior
2.2.2.1 Influences on involuntary behavior It is a common misconception that regularities involving multiple stimuli can have an impact only on autonomic reactions (i.e., behavior driven by the autonomous nervous system). This misconception may be due to the fact that Pavlov’s dog is regarded as the prototypical example of classical conditioning. In Pavlov’s procedure, an autonomic reaction (i.e., salivation) was indeed measured to investigate learning. Research on the conditioning of anxiety responses (which has been very influential, see box 2.1) has also made use of autonomic (physiological) reactions.
Box 2.1 On the Relation between Anxiety Disorders and Classical Conditioning
Research on the classical conditioning of fear and anxiety reactions contributed to the rise of behavior therapy, one of the most important traditions in psychotherapy (see Craske, Hermans, & Vansteenwegen, 2006). The starting point was a functional analysis of anxiety disorders in clinical practice. Such functional analyses refer to the possibility that anxiety disorders are an example of classical conditioning (i.e., fear that results from regularities in the occurrence of two or more stimuli). For example, a man’s phobia of dogs may be due to the fact that he was bitten by a dog during childhood. Likewise, a woman’s fear of elevators may be due to the fact that she experienced a hyperventilation attack when riding an elevator. Based on this functional analysis, we can investigate whether procedures that influence conditioned anxiety reactions in the lab also have an impact on anxiety reactions in clinical practice. This approach has led to various treatment techniques such as exposure therapy and systematic desensitization (see also box 2.6). Although it is generally accepted that at least certain anxiety disorders are examples of classical conditioning, it has also become clear that fears are not always the result of regularities in the occurrence of stimuli in the environment. For example, research shows that many people develop phobias even though they have never had a negative experience with the object they fear. Likewise, many people experience panic attacks when confronted with objects they have rarely or never been directly confronted with (e.g., snakes) or situations they are already afraid of when they are confronted with them for the first time (e.g., traveling in an airplane). It seems that anxiety disorders are often the result of observing others and/or hearing stories from others (Poulton & Menzies, 2002; Rachman, 1977). Note, however, that learning through observation and instructions can also be studied within learning psychology. We will return to this issue in section 2.2.1 and in chapter 4.
However, it is clear that not only autonomic reactions but also controlled behavior can be influenced by stimulus pairings. A good example of this is the phenomenon of autoshaping. Brown and Jenkins (1968) placed pigeons in a Skinner box and let a key light up just before the grain appeared in the food dispenser. After several presentations of the stimuli (light and food) the pigeon began to peck at the key (see http://www.youtube.com/watch?v=cacwAvgg8EA). So there is a change in controlled behavior (i.e., the pigeon starts to peck at the key) as a result of a regularity in the presence of two stimuli (light and food). This behavior is apparently formed in an “automatic” way (i.e., without the behavior being related with a certain result); hence the name autoshaping.
This change in behavior counts as an example of classical conditioning only if we are certain that the change is due to the relation between two stimuli and not to some other factor. A possible alternative cause of the pigeon’s pecking on the metal plate is that this behavior somehow leads to the presence of an appetitive stimulus (e.g., food). In that case, the behavior would be an example of operant conditioning (a change in behavior due to the relation between that behavior and a stimulus in the environment). The results of a study by Hearst and Jenkins (1974) suggest that such an alternative explanation in terms of operant conditioning is probably not correct. The setup can best be explained with reference to figure 2.5. A pigeon is placed in a long cage. On one side of the cage there is a light that always lights up before food appears in a dispenser that is on the other side of the cage. When the food appeared, it was accessible for only four seconds. In other words, the pigeon had to be at the dispenser within four seconds. An extremely strange behavior developed in their study. After a while, the pigeon spontaneously ran to the light that was paired with food, pecked on it, and then hurried to the dispenser. Given the distance, the bird often came too late—the food had already disappeared. And yet, the pigeons seemed to persevere in this completely useless and even counterproductive behavior. Because pecking on the key apparently has no appetitive consequences and even leads to a negative result (arriving at the dispenser too late to eat the food), it seems unlikely that the behavior is under the control of the stimuli that follow that behavior (food). A more likely explanation involves the relation between the light and the food. Moore (1973) showed that in an autoshaping trial, the behavior of the pigeon (CR) differs depending on whether food or water is used as the US. In both cases an illuminated image is presented, but in one case this image leads to behavior that shows the topography of “eating behavior,” and in the other case “drinking behavior” (see https://www.youtube.com/watch?v=50EmqiYC9Xw). The nature of the reaction with regard to the lighting of the key (pecking as if it is eating vs. pecking as if it is drinking) is thus determined by the nature of the stimulus that accompanies the illumination of the key, which suggests that the change in behavior is indeed a consequence of the CS-US relation (i.e., it is an instance of classical conditioning).
Longbox autoshaping with pigeons. When the light (left side) always precedes availability of the food in the food dispenser (right side), the pigeon will approach and peck on the light.
2.2.2.2 Three types of behavior Although a large variety of behaviors can be influenced by stimulus pairings, past research has largely concentrated on changes in three categories of behavior. First, in most experiments, changes in preparatory responses are examined (see Mackintosh, 1983, and Bouton, 2016, for an overview). This concerns responses that prepare the organism in a certain way for the arrival of a certain stimulus. Both appetitive and aversive responses can be seen as preparatory responses. For example, one can assume that an appetitive response such as salivation prepares the organism for the arrival of food and enables it to consume the food more efficiently (which is why such responses are sometimes also called consummatory responses). An aversive response such as freezing can facilitate the detection of aversive stimuli and thereby reduce the impact of those stimuli. Note, however, that preparatory responses are not always adaptive (in the sense of being useful for fulfilling the goals of the organism). In autoshaping (see previous section), for example, we see that a pigeon will peck at a key when the lighting of that key is followed by the delivery of food. This behavior does not actually help the bird get food and therefore it is not adaptive (if anything, it is sometimes counterproductive). Nevertheless, because the topography of the behavior depends on the nature of the US (food or water), one could argue that the behavior is preparatory in that it is an attempt to prepare the organism for the ingestion of food or water, albeit a misguided attempt.
Second, studies on human contingency learning examine changes in judgments that are due to regularities involving stimuli (see De Houwer & Beckers, 2002, and Shanks, 2010, for an overview). In these studies, participants are shown situations in which certain cues (which could be seen as CSs) and certain outcomes (which could be seen as USs) are either present or absent.1 On the basis of the information provided, the participant is asked to make a contingency judgment—that is, a judgment about the strength of the relation between the presence of a cue and the presence of an outcome. If such contingency judgments are influenced by the actual relation between the two stimuli, this qualifies as an instance of classical conditioning because behavior (contingency judgments) changes as the result of stimulus pairings (the regularity involving the presence of the cue and the presence of the outcome). The food allergy paradigm is one specific task that is often used in studies on human contingency learning. Participants are told that they will receive information about a patient who is allergic to certain types of food. During each trial, participants are told what the patient ate during a certain meal and whether the patient showed an allergic reaction after the meal. Each trial therefore corresponds to a separate meal. On the basis of that information, participants indicate the extent to which a food will lead to an allergic reaction in that patient. Results show that judgments about the (causal) link between foods and allergic reactions are influenced by the information about the co-occurrence of those foods and allergies.
Box 2.2 Why Does Classical Conditioning Receive Less Attention from Functional Psychologists?
One of the striking differences between handbooks on learning research concerns the extent to which attention is paid to classical versus operant conditioning. In the handbooks on learning written by functional psychologists (e.g., Catania, 2013; Pierce & Cheney, 2018), often there is only one short chapter devoted to classical conditioning, whereas it is covered extensively in textbooks written from a cognitive approach (e.g., Bouton, 2016; Schwartz et al., 2002). One reason for this difference is that only the cognitive approach devotes attention to theories about the mental mechanisms that mediate classical conditioning. A second is that classical conditioning is conceptualized in a more restrictive manner in the functional approach. In this box, we focus on the difference in conceptualization.
In the functional approach, the term respondent conditioning is often used instead of classical conditioning. The term respondent refers to “respondent behavior”—that is, behavior elicited by a stimulus. In contrast to operant behavior, which is a function of its consequences (e.g., whether an animal presses a lever depends on whether food follows that behavior or not), respondent behavior is dependent only on the antecedents of the behavior (e.g., seeing a lemon elicits salivation). Thus, you can change operant behavior by changing the consequences of behavior, but respondent behavior can be changed only by manipulating the stimulus that elicits that behavior. In functional psychology, classical conditioning is sometimes seen as learning respondent behavior, while operant conditioning is seen as learning operant behavior (e.g., Pierce & Cheney, 2018). The type of conditioning (classical vs. operant) is thus defined in terms of the type of behavior that changes (respondent vs. operant). Once it has been determined that a behavior is dependent on a certain outcome of that behavior, it can be concluded that the behavior is an operant and therefore that it falls outside the scope of classical conditioning. For functional psychologists, most behavior that humans and nonhumans emit is operant in nature (dependent on its consequences). This may explain why classical conditioning is not so interesting for functional psychologists: it only tells us something about the less interesting and less prevalent class of respondent behavior.
For cognitive psychologists, research on classical conditioning is important because it can provide them with insights into how organisms learn to anticipate events in the environment and how they manage to adapt to that environment (e.g., Bouton, 2016, p. 28). For them, it is not so important whether the organisms learn relations between stimuli or relations between their behavior and outcome stimuli. In both cases, a mental mechanism is needed that detects the relation between elements in the environment and allows the organism to adjust its behavior. When cognitive psychologists choose to study classical conditioning it is a largely pragmatic choice: the researcher has complete control over the stimuli presented during a study on classical conditioning. It is much more difficult for them to exercise control over operant conditioning because the presence or absence of stimuli is dependent on the behavior of the organism (e.g., the rat receives food only when it presses the lever, so the presentation of food depends on the behavior of the rat). The study of classical conditioning is therefore the most convenient way for a cognitive psychologist to investigate mental mechanisms of learning.
In this book, we also give classical conditioning greater weight than what is typical in functional learning psychology handbooks. The reason for this is that we define classical conditioning only in terms of regularities in the environment and not in terms of types of behavior. Take our previous example of changes in contingency judgments. Strictly speaking, contingency judgments are not respondent behavior, because making judgments clearly depends on the consequences of that behavior. For example, participants will provide judgments only because doing so is required by the task (e.g., the task can be terminated only if judgments are given). Contingency judgments are therefore an operant behavior because they depend on the outcomes of that behavior (e.g., the fact that they bring the end of the task closer). Nevertheless, the change in the content of the judgments (e.g., whether people believe that there is a positive or negative contingency between a certain food and allergic reactions) can be influenced by a regularity in the occurrence of two stimuli (e.g., the co-occurrence of the food and an allergic reaction). For us, this impact of a regularity on the judgment is sufficient to view the change in the judgment as an example of classical conditioning. In other words, if one aspect of the behavior is influenced by a regularity in the co-occurrence of stimuli (e.g., the judgment about the strength of a contingency), the change in that aspect of the behavior can be considered an example of classical conditioning. For our definition of classical conditioning, it is not important that there are other aspects of the behavior that are dependent on the consequences of the behavior (e.g., why the participant provides a judgment). Some instances of classical conditioning might even depend on (extensive training that leads to) verbal capacities (see chapter 4). From this broad perspective, classical conditioning is very important because many changes in behavior are determined by the co-occurrence of two stimuli. All of this does not mean that each instance of classical conditioning is necessarily moderated by the same variables or mediated by the same mental processes. Yet, there is merit in unifying a wide range of phenomena under the umbrella of classical conditioning because it allows one to describe them using the same concepts (e.g., CS, US) and thus to compare them in terms of the role of moderators (e.g., impact of CS-only trials) and mediators (e.g., associations, propositions).
A third type of behavior examined in classical conditioning research is evaluative behavior. Although it is difficult to pinpoint the exact difference between evaluative and nonevaluative behavior, at least intuitively it makes sense to classify certain behaviors as evaluative in the sense that they imply a certain liking or disliking of a particular stimulus. Most often, explicit ratings are used to measure changes in liking (e.g., selecting a number on a scale ranging from -100 to +100 to express one’s liking of a stimulus, with -100 being the rating for extremely negative stimuli and +100 the rating for extremely positive stimuli). The term evaluative conditioning is typically used to refer to changes in liking that are the result of stimulus pairings (see De Houwer, Thomas, & Baeyens, 2001, and Hofmann, De Houwer, Perugini, Baeyens, & Crombez, 2010).2
The prototypical way of studying evaluative conditioning is a picture-picture procedure originally developed by Levey and Martin (1975) and elaborated by Baeyens (e.g., Baeyens, Eelen, Crombez, & Van den Bergh, 1992). During a first phase, participants encounter a series of images (e.g., faces of unknown individuals or abstract geometrical shapes). For each of these images, the participant must indicate how pleasant or unpleasant he or she finds the image (e.g., by rating it on a scale of -100 [very unpleasant] to +100 [very pleasant]). On the basis of these ratings the researcher selects images that are considered neutral (e.g., a rating of 0, +10, or -10), positive (e.g., a rating of +80 or more), or negative (e.g., a rating of -80 or less). The experimenter then creates a series of stimulus pairs consisting of a neutral (CS1) and positive image (USpos) and another set of pairs of neutral (CS2) and negative images (USneg). In the second phase, participants see a pair of images on each trial. The neutral picture usually appears first, followed by a brief pause, and then the positive or negative image. The next trial starts several seconds later. In this way, certain neutral images are repeatedly paired with a positive image, whereas other neutral images are paired with negative images. Afterward, participants have to reassess the valence of the images on the same scale as before. In most evaluative conditioning studies, the neutral images that were paired with positive images are subsequently evaluated more positively than the neutral images paired with negative images. In other words, the valence of the neutral images changes in the direction of the valence of the images with which they were paired. It should be clear that the procedure used in evaluative conditioning studies corresponds to a classical conditioning procedure: the neutral stimuli can be regarded as CSs, the positive and negative stimuli as USs, and the change in valence as the CR. The only unique procedural element of evaluative conditioning is that it focuses on changes in evaluative responses to the CSs instead of preparatory responses or contingency judgments (De Houwer, 2007).
2.2.2.3 Unconscious learning: The relation between different conditioned changes in behavior Earlier in this chapter we argued that changes in contingency judgments about the relation between stimuli can be regarded as an instance of classical conditioning (i.e., a change in behavior that is due to the pairing of stimuli). This functional perspective differs from the typical cognitive view in which verbal judgments are seen as providing a direct index of knowledge (i.e., of mental representations). From a cognitive perspective, judgments about relations can tell us if the knowledge that mediates conditioning is consciously accessible. The answer to this question is not just interesting as such but also relevant for the much broader question of whether there are unconscious influences on behavior.
Box 2.3 Applications of Evaluative Conditioning to Social and Consumer Psychology
Our likes and dislikes determine many aspects of our behavior. For example, we tend to avoid the things we do not like and approach things we do like. This applies to objects, products, places, and people. Hence, a better understanding of how our likes and dislikes come about can lead to better prediction and control of other types of behavior. Research on evaluative conditioning has important implications for other research domains in and outside psychology. Take social and consumer psychology. In those domains, people want to learn more about how attitudes toward persons and products are created and how these can be changed. Research on evaluative conditioning can easily be applied to the domain of advertisements (e.g., Hütter & Sweldens, 2018). For example, in advertisements, a product (e.g., Coca-Cola) will often be presented together with positive stimuli (e.g., a smiling person). The product can be considered the CS and the smiling person the US. The aim is to transfer the positive valence of the US to the CS as a result of the joint presentations of the CS and US; this is a classically conditioned change in evaluative responding.
However, when we look at this research from a purely functional point of view, we must abandon the idea that a contingency judgment or any other behavior is a direct reflection of underlying knowledge. A judgment is also a behavior that is a function of certain factors in the environment, including regularities such as the pairing of stimuli. From a functional perspective, instances of unconscious learning illustrate that regularities can influence different types of behavior (e.g., judgments, skin conductance) in different ways (e.g., no impact on judgments, impact on skin conductance). This functional perspective does not exclude the possibility that this research may have implications for cognitive theories about the mental mechanisms that mediate conditioning, but as with other research, it is important to keep the effects (changes in judgments or skin conductance as a result of regularities in the presence of stimuli) separate from the mental processes (whether the mediating knowledge has to be consciously accessible).
Empirical research on unconscious conditioning strongly suggests that relations between stimuli in the environment must have an influence on judgments about that relation in order to influence other behavior. To illustrate this more clearly, let’s consider the work of Dawson and Schell (see Dawson & Schell, 1987, for an overview, and Mertens & Engelhard, 2020, for a review of related evidence). Their participants were exposed to a regularity involving the presentation of a light (CS) and an electric shock (US). Dawson and Schell investigated the conditions under which these stimulus pairings resulted in a change in skin conductance during CS presentations. They also asked participants to judge during each CS presentation the extent to which they expected the US to occur. They found that a change in skin conductance occurred only after a change in US expectancy had occurred. In other studies, Dawson and Schell found that factors that prevent changes in expectancies from occurring (e.g., making the CS less conspicuous and thus making the relation between the CS and the US less conspicuous) also prevented changes in skin conductance from occurring. Research into evaluative conditioning has also shown that the strength of changes in evaluative responding (how positive or negative you consider the CS to be) are to a large extent dependent on judgments about which CS went together with which US (contingency awareness). For example, a meta-analysis (Hofmann et al., 2010) showed that differences in contingency awareness was by far the most important determinant of the strength of the changes in valence (i.e., the better one’s conscious knowledge about the CS-US pairings, the stronger the evaluative conditioning effect). From a cognitive perspective, this implies that conscious knowledge is very important for classical conditioning effects.
Although it is clear that contingency awareness is an important factor in classical conditioning, there is still a debate about whether there are certain instances of conditioning that occur in the absence of contingency awareness. In other words, are there certain conditions under which conditioning can occur unconsciously? Whereas the existence of unconscious conditioning is self-evident for certain researchers (e.g., Clark, Manns, & Squire, 2002), others continue to question the evidence for unconscious learning (e.g., Lovibond & Shanks, 2002; Mertens & Engelhard, 2020; Mitchell et al., 2009; Vadillo, Konstantinidis, & Shanks, 2016). Although we do not exclude the possibility that conditioning can occur unconsciously under certain conditions (e.g., Greenwald & De Houwer, 2017), it seems to us that conditioning most often requires contingency awareness. In functional terms, it will usually be possible to observe changes in contingency judgments before one can observe other effects of stimulus pairings. The big question for future research is, what are the conditions that determine whether conditioning can take place unconsciously? To answer this question, however, we first need procedures that allow us to demonstrate unconscious conditioning in a reliable and unambiguous way. There is still no consensus about which procedures are suitable for achieving that aim.
2.2.3 The Properties of the Organism
Effects that result from stimulus pairings seem to occur in almost all animal species. For example, classical conditioning has been demonstrated in worms, fruit flies (see https://www.youtube.com/watch?v=-dPfZE5adYg), snails, bees, fish, birds, rats, and people. Given the large differences between these species, it seems unlikely that, at the cognitive level, classical conditioning is always based on a single (cognitive or neuronal) mediating mechanism. It also highlights the need for caution when we generalize knowledge about conditioning in one animal species to another species. Even if there are major similarities between the moderators of classical conditioning in, for example, bees and humans (e.g., Bitterman, 1996), this does not mean that the same mental mechanism is responsible for conditioning in both animal species. The presence of classical conditioning effects in so many species is probably due to the fact that different animal species are confronted with similar problems in their world; the principle of convergent evolution holds that animal species evolve independently of each other but still display similar characteristics because of similarities in their interactions with the environment (Van Horik, Clayton, & Emery, 2012). Every animal has a greater chance of survival and reproduction if it can adapt to environmental regularities (e.g., if it can predict where food can be found or when dangerous situations will occur). Almost all animals will thus show classical conditioning when doing so increases their chances of survival and reproduction. However, these similarities do not imply that the underlying mechanisms (at either the mental or neuronal levels) are always the same.
Moreover, there are also clear differences in the conditions under which classical conditioning occurs in different animal species. For example, research on the influence of intrinsic relations shows that one animal species can be more strongly influenced by certain regularities than another animal species. For example, mammals will learn more quickly about relations between the taste of food and nausea than about relations between the color of food and nausea (whereas there is no such difference in learning the taste-shock relation and the color-shock relation). Birds, on the other hand, seem to learn a relation between color and nausea more quickly than a relation between taste and nausea (whereas there is no such difference in learning the taste-shock relation and color-shock relation). This could be due to the fact that the selection of food in mammals is determined mainly by the taste of the food, whereas in birds this is determined mainly by the visual characteristics of food. It also seems that certain instances of selectivity in learning are already present from birth, indicating a genetically determined influence on learning (e.g., Gemberling & Domjan, 1982).
In light of these differences, it is therefore useful to also view conditioning from the perspective of the specific organism being studied (for an overview, see Domjan, 2005, and Bouton, 2016, pp. 193–200). From this perspective, conditioning is primarily a function of the survival and reproduction of the organism. Conditioning (as an effect) is thus seen as an adaptive phenomenon that occurs in natural situations. This perspective has two important implications for how conditioning research will be carried out. If you want to learn more about conditioning in natural situations, then you need to take stimuli that can occur together in the natural environment of an organism. There is little point in using CSs that normally do not occur with USs in the environment of an organism (e.g., a light flash followed by an electric shock), as is the case in the majority of the existing conditioning experiments. Instead, it is better to use CSs that often occur together with the US in the organism’s environment, because they are potential causes of the US (e.g., food with a certain taste [CS] and nausea [US]) or because the CS is an integral part of the US (e.g., the visual characteristics of a sexual partner [CS] preceding copulation with that partner [US]). Domjan (2005) calls such CSs ecologically valid.
Domjan (2005; Domjan, Cusato, & Krause, 2004) reviewed research that indicates that conditioning with ecologically valid CSs has different characteristics than conditioning with arbitrary CSs. He did research into sexual conditioning with quails (small birds that live predominantly in grasslands, see figure 2.6). Domjan and his colleagues repeatedly showed male quails a fake (i.e., taxidermic) female quail (CS) just before they were given access to a real female quail and had sexual contact with it (US). As a result of these “pairings” (i.e., the joint presentation of the CS and the US), the number of times the males attempted to mate with the fake female (the CS) increased. Domjan (2005) also discusses aversion learning (see section 2.2.1.2) as an example of conditioning with ecologically valid CSs. One can say that there is a “natural relation” between the taste of food and gastrointestinal sensations (including nausea).
A Japanese quail.
A second consequence of an ecological perspective on classical conditioning is that extra attention is given to adaptive conditioning effects (i.e., conditioning effects that help the organism to survive and reproduce; also see note 5 in the introduction). An important adaptive effect may be that the organism deals with the US in a more efficient way. The emphasis here is not on how conditioning as a procedure changes reactions to the CS (i.e., changes in the CR), but how conditioning influences reactions with respect to the US (i.e., changes in the UR). Domjan (2005) provides an overview of various findings that show that conditioning as a procedure can lead to changes in the UR. Perhaps the most imaginative is his own research into sexual conditioning in quails (see Domjan & Gutiérrez, 2019, for a recent review on sexual learning).
Again, Domjan showed a fake female quail to male quails (CS) just before they had access to a real female quail with which they could have sexual contact (US). During a subsequent test phase, the researchers evaluated not responses to the fake quail (CS) but rather actual sexual contact with the real female (US). They checked whether prior presentations of the CS had an effect on the efficiency of sexual contact and found that after the presentation of the fake quail (CS), sexual contact with the live quail was more efficient. For instance, the male needed less time to initiate sexual contact, semen contained more sperm, and the chance of conception of the egg was greater. These effects were obtained only when the fake quail was presented prior to the real female quail during the test phase (i.e., prior to sexual contact with a female) and when during a previous learning phase, the fake quail was paired with the presentation of a real female. The changes in the UR (the response to the sexually available female) were therefore the result of prior pairing of the CS and the US. Another example is conditioned drug tolerance (see section 1.2.2). Stimuli (CSs) associated with the use of a drug (US) will reduce the response to the drug (UR) and thus reduce the likelihood of death by overdose. In sum, conditioning plays an important adaptive role in dealing with many important events.3
2.2.4 The Influence of the Broader Context
Most studies that are relevant to the impact of the broader context on classical conditioning deal with the effect of secondary (additional) tasks. The results of these studies can be summarized as follows: a secondary task that draws the attention away from the relation between the CS and the US will weaken classical conditioning effects. For example, McKell Carter, Hofstötter, Tsuchiya, and Koch (2003) presented numbers as a tone was systematically followed by an electric shock. When subjects were given the task of repeating the numbers presented to them in a sequence, the tone subsequently elicited a smaller CR than when no (or a simpler) secondary task was given. In short, the presence of a secondary task that directs attention away from the CS-US relation seems to interfere with classical conditioning.
2.2.5 Characteristics of the CS-US Relation and Changes in Those Characteristics
A regularity involving two stimuli encompasses several aspects: the number of times they occur together, the number of times they do not occur together, the time between stimulus presentations, and so on. In this section we discuss the importance of these different aspects for classical conditioning effects. Here we mainly discuss studies on changes in preparatory responses, because research on those responses was popular from the very start of learning research and provided the inspiration for research into the classical conditioning of other behaviors. First, we discuss the influence of the nature of the spatiotemporal relation: how exactly is the spatiotemporal presence of the stimuli related? Next, we look at the impact of changes in the spatiotemporal relation itself.
2.2.5.1 The nature of the spatiotemporal relation We have defined classical conditioning as an effect of regularities in the spatiotemporal occurrence of stimuli. In this section we discuss whether the properties of the regularity are important. A key point to appreciate is that stimuli can occur together in time and space in different ways. There is a contiguous relation when stimuli are presented together in time and space at least once (e.g., they are presented next to each other on a computer screen at the same moment in time). There is a contingent relation when there is a reliable statistical relation in the presence of the two stimuli: the probability that one stimulus is present depends on the presence of the other stimulus. In this section, we investigate what kind of relation needs to exist in order for classical conditioning to occur.
a) Contingency is more important than contiguity Early in the psychology of learning, as outlined by Pavlov and the associationist tradition in philosophy (philosophers such as Locke and Hume), the coexistence of two events or “ideas” in time and space (i.e., the spatiotemporal contiguity) was often regarded as necessary and sufficient for organisms to learn a relation (“association”) between stimuli. Later, however, it was argued that a contingency between stimuli is more important than mere contiguity. Contingency refers to a logical or statistical relation between the presence of the stimuli. This depends not only on the co-occurrence of stimuli but also on situations in which the stimuli do not occur together. Logically speaking, there is a relation between the presence of two stimuli if the probability of the presence of one stimulus depends on the presence of the other stimulus. If the presence of the US is more likely in situations where the CS is present than in situations where the CS is absent, one speaks of a positive relation, or a positive contingency. If the presence of the US is less likely in the presence (than in the absence) of the CS, one speaks of a negative relation or a negative contingency. Statistically, the probability of the US given the presence of the CS is expressed as p(US/CS), and the probability of the US in the absence of the CS is expressed as p(US/~CS). Both probabilities depend on the frequency of four possible events: (a) both the CS and the US are present, (b) the CS is present but the US is absent, (c) the CS is absent but the US is present, and (d) both the CS and the US are absent. These four events correspond to the four cells of the four-field table shown in figure 2.7.
Figure 2.7
The four-field contingency table.
The p(US/CS) is equal to the frequency of cell (a) divided by the summed frequency of cells (a) and (b). The p(US/~CS) is equal to the frequency of cell (c) divided by the summed frequency of cells (c) and (d). The strength of the relation (indicated by the notation ΔP or delta P) is thus determined as follows:
ΔP = p(US / CS) − p(US / ~CS) = (a / (a + b)) − (c / (c + d))
Therefore, all other things being equal, when the CS and the US co-occur more often (the value of cell a increases) or are both absent more often (the value of cell d increases), the contingency between the two will become more positive. The more often that only the CS (cell b) or only the US (cell c) occurs, the more negative the contingency becomes. ΔP thus reflects the extent to which the presence or absence of the US is correlated with the presence or absence of the CS. A positive contingency indicates that the stimuli tend to be present together or absent together. A negative contingency indicates that the presence of one of them signals the absence of the other, and vice versa.
Rescorla (1966) was one of the first to systematically study the role of contingency in classical conditioning. In various experiments, he showed that CRs are dependent not only on the value of cell a, but also on the value of the other cells in the four-field table. He determined that excitatory conditioning occurs as soon as there is a positive contingency, or when p(US/CS) > p(US/~CS). The term excitatory refers to the finding that there is an excitation (i.e., an increase or intensification) of a certain behavior (e.g., an increase in anxiety). Inhibitory conditioning occurs as soon as there is a negative contingency. In this case, presenting the CS will lead to an inhibition (i.e., decrease or weakening) of a certain behavior (e.g., decrease of anxiety; see Sosa & Ramírez, 2019). Rescorla thus showed that there is a clear relation between the value of ΔP and the nature of the change in behavior (see figure 2.8). When ΔP = 0 and there was therefore no contingency between the occurrence of the CS and the US, there was no change in the behavior, even if the CS and US sometimes co-occur (i.e., even if cell (a) is greater than zero). This suggests that contiguity (CS and US sometimes occur together) is not a sufficient condition for conditioning (but see Papini & Bitterman, 1990, for a criticism of this conclusion).
The X-axis represents the conditional probability that the US occurs together with the CS. The Y-axis represents the probability that the US occurs without CS. On the diagonal where both probabilities are equal, there is no contingency between the stimuli (after Seligman, Maier, & Solomon, 1971).
Although contingency therefore plays an important role in conditioning, it should be noted that it is not easy to determine the degree of contingency between two stimuli in an unambiguous way. Take the example shown in figure 2.9:
The influence of the way in which time intervals are defined on the calculation of contingency. The slash (/) represents the beginning or end of a time interval.
If the time intervals are broadly defined as in figure 2.9A, one will conclude that there is a perfect contingency between the CS and the US (in each time interval they occur together or are absent together). If one works with short time intervals as in figure 2.9B, however, one must conclude that the relation is not perfect because there are six time intervals in which only the CS or the US occurs. The calculation of the contingency between two stimuli is possible only if an artificial unit is created in an artificial way—when does a situation start and when does that situation end? But defining situations is always arbitrary, so one can never be absolutely certain how to determine the start and end of a situation. Different time formats lead to a different interpretation of the four-field table and thus to different statements about contingency (see Gallistel, Craig, & Shahan, 2019, for a recent discussion of this issue).
b) Conditional contingencies are more important than simple contingencies There are a number of findings that show that even the presence of a statistical contingency between two stimuli is not sufficient to observe classical conditioning. The two most important findings are overshadowing and blocking.
– Overshadowing
In studies on overshadowing, two conditions are compared. In one condition, X is always the only CS on a trial and it is always followed by the US (X+). In the other condition, X is always presented together with another CS—namely, CS A—and both are followed by the US (AX+). When at the end of both conditions X is presented on its own during a test trial, X evokes a stronger CS in the first condition (X+) than in the second condition (AX+). It seems that the presence of A in the second condition “overshadows” the effect of the (perfectly contingent) X-US relation. This phenomenon was extensively described by Pavlov (1927).
– Blocking
The blocking effect is without doubt one of the most important findings in cognitive learning psychology (but see Maes, Vanderoost, D’Hooge, De Houwer, & Beckers, 2017, for a number of critical considerations). Kamin (1968) was the first to bring the phenomenon to light. One stimulus A (e.g., a visual signal) is always followed in the first phase by an electric shock (A+). In a second phase, stimulus A is presented along with stimulus X (e.g., a sound). This compound of both stimuli is also followed by shock (AX+). In a subsequent test phase, only X is presented. In a well-designed blocking experiment, there are also control conditions in which the first phase is omitted or a third stimulus is presented during the first phase (B+). So the design can look like this:
Phase 1 |
Phase 2 |
Test |
||||
|---|---|---|---|---|---|---|
Experimental condition: |
A+ |
AX+ |
X? |
|||
Control condition 1 |
AX+ |
X? |
||||
Control condition 2 |
B+ |
AX+ |
X? |
Blocking refers to when the CR elicited by X during the test phase is weaker in the experimental condition than in the control conditions. There is complete blocking if X does not elicit any CR in the experimental condition (but does do so in the control conditions). Blocking is interesting because X is always followed by the US in all conditions. The only thing that is manipulated across conditions is the relation between A and the US (but see Think It Through 2.1).
Think It Through 2.1: Control Conditions for Determining Blocking
In your opinion, what is the use of the second control condition (i.e., those in which B+ trials are presented in the first phase?)
Blocking shows that the existence of a positive contingency between X and the US is not a sufficient condition for conditioning effects to occur. The name blocking is poorly chosen because it refers to a possible explanation for the effect: it seems that the existence of the A-US relation “blocks” (prevents) the learning of the X-US relation. But this is not entirely correct, as indicated by the fact that backward blocking can also occur. Here, the CR to X is weakened by presenting A+ trials after the AX+ trials (i.e., AX+ followed by A+). With backward blocking, it is not possible for a strong A-US relation to block the learning of the X-US relation because the A-US relation is only strengthened after the AX + trials (but see Miller & Matute, 1996, for evidence that backward blocking occurs only under certain conditions).
Findings such as overshadowing and blocking suggest that the influence of a spatiotemporal relation between stimuli on behavior (i.e., classical conditioning) depends on the extent to which there is a conditional contingency between two stimuli. Conditional contingency refers to the contingency between two stimuli in situations where a certain condition is met: the situations differ only with regard to the presence of the CS. Take the example of blocking. In a blocking experiment A is always present when X is present (AX+). There is a perfect contingency between X and the US: the probability of the US if X is present [p(US/X)] is equal to 1, while the probability of the US if X is absent [p(US/~X)] equals 0. However, the calculation of this contingency is based on a comparison of situations where both A and X are present (the AX + trials) and situations where no CS is present. The difference in the probability of the presence of the US in those situations could be an indication not only of the strength of the relation between X and the US but also of the strength of the relation between A and the US. A correct estimate of the strength of the relation between X and the US can only be made by comparing the probability of the US between situations that differ only with regard to the presence or absence of X. In a blocking experiment this can be done by comparing situations where A and X are present (AX+ trials) to situations where only A is present (A+ trials). If we take into account only those situations (and not situations where both A and X are absent), then we can conclude that the probability of the US in both situations is equal. The US is always present, regardless of whether X is present (AX+ trials) or absent (A+). The conditional contingency is therefore zero, so the relation between X and the US will not affect the response to X. Conditional probability can therefore be represented as follows (see Cheng & Novick, 1990, 1992; Cheng & Holyoak, 1995):
ΔPc = p(O/A.X) − p(O/A.~X)
Box 2.4 Blocking in Real Life
When developing their products, companies sometimes use a technique called rebranding. A new brand name will be related to an existing product. For instance, in Europe, the famous snack Twix used to be called Raider (see figure 2.10).
Figure 2.10
Rebranding: in some countries, Twix used to be called Raider.
Although the name changed, the packaging remained largely the same. In many cases, companies hope that during rebranding, the relation between the old product and the new name will be learned quickly. But research on blocking suggests that learning this new name-product relation will be more difficult if the packaging remains unchanged. Prior to the rebranding, there was already a contingency between the golden packaging of Raider and the product. If we consider the packaging as CS (A), the old name as CS (B), and the product itself as the US, then we can consider consuming the product in its original packaging as AB+ trials. After the rebranding, the golden color of the packaging remains constant, but the name changes. If we use the letter X to refer to the new name, then we can describe the new situation as AX+. The golden package (A) can delay the learning of the relation between the new name and the product (X-US) because the package was previously presented together with the product. The fact that people did indeed require a lot of time to learn the new name of the product can thus be seen as an example of blocking. It is worth noting, however, that blocking does not occur under certain conditions, and even the opposite effect can occur: the CR for X can, under very specific conditions, be stronger after A+ and AX+ trials than after only AX+ trials (e.g., Liljeholm & Balleine, 2009). Thus, learning research can provide inspiration to optimize the effects of rebranding (see Van Osselaer, 2008, for an overview of various implications of conditioning research for marketing and product development).
In this equation, p(O/A.X) stands for the probability of the US when both A and X are present and p(O/A.~X) represents the probability of the US when only A is present. You can calculate ΔPc via the four-field table if you take into account only the trials in which A is present (i.e., determining the number of AX+, AX-, A+ and A- trials; see figure 2.11). In other words, you calculate the contingency between X and the US conditional on the presence of A.
The four-field table for situations where X always occurs with A.
At the functional level (i.e., purely in terms of factors in the environment or behavior), one can describe blocking as an example of the impact of conditional contingencies on behavior: it shows that in situations where X always occurs together with other CSs, the CR for X is determined by the degree of conditional contingency and thus not the degree of contingency per se. Note that conditional contingency is important only when X always occurs together with A. If there are situations where X occurs without A, then the strength of the relation between X and the US can be determined by comparing situations in which only X is present and situations where no CS is present [ΔP = p(O/X) − p(O/~X)].
c) Indirect stimulus relations Neither contingency nor conditional contingency is absolutely necessary for the occurrence of classical conditioning. Even when there is no (conditional) contingency or even contiguity between stimuli, conditioning can still take place. After all, stimuli can also be indirectly related. To illustrate, imagine that a tone (CS1) is always followed by a light (CS2) and that the light (CS2) is always followed by an electric shock (US) (i.e., CS1-CS2; CS2-US). In this case, there is no direct (first-order) relation between the tone and the electric shock (i.e., between CS1-US): they never co-occur in space and time. There is, however, a second-order relation between the two: both the tone and the shock are related to the light. Studies in sensory preconditioning and higher-order conditioning have shown that such an indirect relation between the tone and the shock can lead to changes in the response to the tone. Sensory preconditioning refers to the procedure in which the two neutral stimuli (e.g., tone and light) are first presented together during a first phase and the second neutral stimulus is presented with the US only during a subsequent second phase (e.g., light and shock in Phase 2; see also figure 0.2). In higher-order conditioning, the order of the two phases is reversed (e.g., first light-shock and only then tone-light). Note, however, that sensory preconditioning and higher-order conditioning depend on the (conditional) contingency of the underlying (first-order) relations. If there is no (conditionally) contingent relation between the tone and the light or between the light and the electric shock, the indirect relation between the tone and the shock has no influence on the reaction to the tone. Thus (conditional) contingencies are necessary for the effect of indirect relations. Also note that the effects of indirect relations can be seen as examples of complex forms of learning in which different regularities together determine the behavior. We will therefore return to sensory preconditioning and higher-order conditioning in chapter 4 on complex learning.
Think It Through 2.2: Overshadowing and Conditional Contingencies
Problem: How can you explain overshadowing based on the assumption that conditioning is determined by conditional contingency?
2.2.5.2 Changes in the nature of the spatiotemporal relation The spatiotemporal relation between stimuli (i.e., the way in which they occur in space and time) is not necessarily fixed or unchanging. Sometimes there is no relation between two stimuli at first, but later there is. At other times there is initially a relation and then it disappears. In yet other situations two relations can be present simultaneously in different contexts. We will discuss each of these three situations separately.
a) No relation followed by a relation: CS pre-exposure, US pre-exposure, and the absence of contingency There are three ways in which a CS and US can occur in an unrelated manner: the CS always occurs alone, the US always occurs alone, or both CS and US are presented in a noncontingent way. When these events occur before the pairing of the CS and US, they all reduce and delay the effect of the relation between the CS and US on behavior.
– Effects of CS pre-exposure
The term CS pre-exposure refers to procedures in which the CS is repeatedly presented before a relation is established between the CS and US (see Byrom, Msetfi, & Murphy, 2018, for a recent review). For example, in the first phase of an experiment, a tone is repeatedly presented (CS pre-exposure phase) to an organism; in a second phase, the tone always precedes an electric shock (CS-US conditioning phase). Findings show that the organism will find it much more difficult to learn the relation between the tone and shock than another organism that did not receive CS pre-exposure trials. This effect due to pre-exposure to the CS is often called the CS pre-exposure effect. In some cases, the term latent inhibition is used. The problem with the latter term is that it refers to a possible (but not necessarily correct) mental explanation of the CS pre-exposure effect. Because we choose to strictly separate the description of effects from possibly explanatory mental processes, we will refer to it as the CS pre-exposure effect. The pre-presentation of the CS is thus a way to weaken future learning about that CS.
At the functional level, we might also wonder why pre-exposure to the CS has a detrimental effect on classical conditioning effects. As we already know, when conducting a functional analysis, one does not search for mediating mental processes; instead, we want to identify other known environmental moderators of conditioning to which the CS pre-exposure effect could be related. Therefore, the question to ask in a functional analysis is the following: can I describe a particular phenomenon (e.g., the CS pre-exposure effect) in terms of another known phenomenon (e.g., the impact of other moderators on classical conditioning)? There are two potential moderators that may play a role here: (1) the intensity or conspicuousness of the CS and (2) the statistical contingency between the CS and the US. First, it is possible that a CS that is presented frequently is experienced as being less intense or important than one that is not presented frequently. This will reduce the effect of the CS-US relation simply because the CS is less conspicuous (see the section on the effects of CS properties). Second, each CS pre-exposure is a CS-only trial. These trials will therefore reduce the contingency between the CS and US. In other words, the CS pre-exposure effect might also be an example of the impact of statistical contingency on classical conditioning.
The CS pre-exposure procedure can be used to study situations where conditioning can have negative effects on people’s well-being. In the Surwit study (1972) on fear of dentists in young children, the fear can be seen as a CR that results from a pairing of the dentist (CS) with a painful treatment (US). Hence, fear of the dentist can be reduced by allowing the child to meet and interact with the dentist before the first treatment (CS pre-exposure). Similarly, some findings on the occurrence of conditioned nausea in cancer patients undergoing chemotherapy point to the role of CS pre-exposure in preventing or at least attenuating such conditioning (Bernstein, Webster, & Bernstein, 1982). Cancer treatment involves the administration of drugs that evoke nausea (US). Because the drug is administrated in a hospital setting, the nausea co-occurs with many hospital-related stimuli such as the sight of nurses in white uniforms and the taste of food eaten while in the hospital (CSs), which might lead to an aversion (CR) to those stimuli (e.g., white uniforms, certain foods). Allowing a patient to visit the hospital ward before the start of the treatment (CS pre-exposure) can weaken conditioned aversion.
Think It Through 2.3: The CS Pre-exposure Effect and Habituation
What do you think is the link between the effect of CS pre-exposure and habituation?
– Effects of US pre-exposure
Based on the previous section, it may already be clear what a US pre-exposure procedure involves: the US is repeatedly presented on its own before the CS is paired with that US. US pre-exposure also leads to a slowdown and weakening of the effect of the CS-US relation. For example, in a first phase, an organism receives an electric shock repeatedly (US). In the second phase, the shock is always given after a tone (CS). The question is whether the conditioning effect for the tone now differs from what we see in a second organism that was not pre-exposed to the US. Results show that this is indeed the case: the tone elicits less fear if the shock was previously presented alone compared to when the shock was not presented alone.
As in the case with the CS pre-exposure effect, when performing a functional analysis of the US pre-exposure effect, two potential explanations of the effect in terms of other moderators of conditioning present themselves (see Randich & LoLordo, 1979). First, it is possible that the US-only presentations reduce the intensity of the US (habituation), which will weaken the effect of the CS-US relation (because the CR is weaker when the US is weaker). On the other hand, the effect of the US-only presentations can also be attributed to the reduction in the statistical contingency between the CS and the US. Additional research is needed to distinguish these two possibilities.
–Effects of the absence of a relation
A period during which there is a statistical relation between a CS and US can be preceded by a period in which both the CS and the US occur in a noncontingent way (i.e., in the absence of a statistical link between the two stimuli). For example, during a first phase you can present both a tone and a shock in such a way that the chances of encountering the US do not depend on the presence of the tone [p(US/CS) = p(US/~CS)]. If there is then a (conditional) contingency between the CS and the US [p(US/CS) > p(US/~CS) or p(US/CS) < p(US/~CS)] this contingency will have less influence on the CR. This finding suggests that it is not only the presence of a relation that can have an influence on the behavior—so too can the absence of a relation. In other words, it seems that organisms can also learn that there is no relation between stimuli (which makes them learn more slowly that there is a relation; Mackintosh, 1975). We will return to this issue later in chapter 3 in the context of a phenomenon known as learned helplessness.
b) A relation followed by no relation: CS postexposure, US postexposure, and the absence of a contingency
– Effects of CS postexposure
Many studies in learning psychology use CS postexposure procedures (i.e., present the CS alone after the [contingent] pairing of that CS with a US). Researchers do so because they want to know not only how behavior can be established as a result of stimulus pairings but also how it can be modified after it has been established. At first sight, it is very easy to modify conditioned behavior: it is enough to repeatedly present the CS by itself. Imagine that we first pair a tone with an electric shock. As a result of this contingent relation, the tone will come to elicit a conditioned anxiety response (CR). If the tone is then repeatedly presented without the shock, the fear response to the tone will become weaker each time the tone is presented by itself. This effect (the weakening of a CR as a result of CS-only presentations) is called extinction.
Interestingly, however, research has shown that extinction is not permanent and sometimes does not occur at all. Pavlov (1927) showed that an extinguished CR can sometimes spontaneously return after a period of time. This phenomenon is called the spontaneous recovery of the CR (see figure 2.12). Another finding is that CS postexposure does not lead to extinction when it takes place in a context different from that in which the CS-US relation was initially learned and ultimately tested (e.g., Bouton, 1993, 1998). Suppose, for example, that an animal is exposed to the pairing of a tone (CS) and a shock (US) (i.e., CS+ trials) in a blue room. Across trials, the CR gradually becomes stronger. The animal is then transferred to another (green) room, where the CS is presented without the US (CS trials). This leads to the disappearance of the CR. If, however, the animal is then returned to the original learning context (i.e., the blue room) or to a new context (e.g., a yellow room), presenting the CS will immediately provoke a CR again, as if extinction in the green room never happened. This phenomenon is known as renewal (see figure 2.12).4
A schematic overview of spontaneous recovery (A) and renewal (B).
There are also studies in which CS postexposure does not lead to extinction, even when CS-only trials take place in the same context as the CS+ trials. For example, evidence suggests that extinction is less rapid or even does not occur with biologically relevant CSs. In studies by Öhman (e.g., Öhman, Fredrikson, Hugdahl, & Rimmö, 1976; also see Neumann & Longbottom, 2008), images of biologically relevant CSs (e.g., spiders and snakes) or biologically less relevant CSs (e.g., flowers) were first repeatedly paired with electric shocks. Thereafter, all CSs elicited an anxiety reaction. During the second phase, the CSs were presented without any electric shocks. The decrease in the CR was less pronounced for the biologically relevant compared to biologically less relevant CSs.
The nature of the observed behavior also seems to have an influence on the extent to which extinction occurs. For example, conditioned changes in valence (evaluative conditioning) seem much less sensitive to the effects of CS postexposure (e.g., Baeyens, Crombez, Van den Bergh, & Eelen, 1988; see De Houwer et al., 2001, for an overview, but Hofmann et al., 2010, for a meta-analysis that suggests that extinction does occur in evaluative conditioning, albeit only to a small extent). When a photo X has become negative because it was paired with a negative photo, X will retain that negative valence even if it is no longer followed by a negative photo.
– Effects of US postexposure
Although effects of US postexposure have been identified (e.g., Hammerl, Bloch, & Silverthorne, 1997), they often do not occur (e.g., Miller, Hallam, & Grahame, 1990). If, after pairing a tone and a shock, only the shock is presented, then the CR to the tone will sometimes weaken. As with other effects of CS or US (pre- or post-) exposure, one may ask whether the effects of US postexposure are due to changes in the characteristics of the US in itself (e.g., the shock is less negative after it is presented repeatedly; see US revaluation) or to changes in the relation between the CS and US (the contingency between the two decreases when the US is presented repeatedly on its own).
– Effects of the absence and reversal of a relation
We do not know of studies in which the presence of a contingency between a CS and US is followed by a phase in which both the CS and the US are presented but in a noncontingent manner (i.e., contingency equals zero). However, there have been many studies designed to reverse relations (contingency reversal). During the first phase of those studies, a certain CS (A) is followed by a US, whereas a second CS (B) is not followed by the US. During the second phase, A is no longer followed by the US, whereas B is followed by the US. Afterward, researchers check whether the CR with respect to A and B is determined mainly by the regularity in the first phase or by the regularity in the second phase. One speaks of a primacy effect if the first regularity is more important (i.e., if the CR to A is stronger than the CR to B) and a recency effect if the second (most recent) regularity has more of an influence (i.e., CR to B is stronger than the CR to A). Several factors determine whether primacy or recency effects occur. For example, in human contingency learning (see section 2.2.2.2), evidence shows that recency effects become stronger as participants are more frequently asked to judge the strength of the contingency (e.g., Collins & Shanks, 2002; Matute, Vegas, & De Marez, 2002).
c) The presence or absence of relations depends on the context Often, a regularity between two stimuli will exist only under certain conditions. For example, seeing a clock that indicates twelve o’clock will go together with hearing twelve strokes of the bell only if the clock is equipped with a (correctly working) mechanism for producing clock strokes. The effect of these conditional relations has been studied also in the lab (see Holland, 1992, and Schmajuk & Holland, 1998, for an overview). For instance, Rescorla (1987) used procedures that lead to autoshaping. In half of the trials, a background noise was presented for fifteen seconds. During the last five seconds of the presentation of the noise, a button illuminated. Immediately after the end of the presentation of the noise and the illuminated button, food was delivered. In the other half of the trials, the illuminated button was presented without any sound or food. Over time, the pigeon came to peck at the illuminated button (which is an example of autoshaping), but only if the noise was also present.
There are two functional analyses that can potentially explain this effect, which are based on a different description of the same objective situation. First, you can say that there is one CS-US relation—namely, the regularity between the illumination of the button and the delivery of food. However, this relation is valid in only one particular situation (i.e., after the background noise has been presented). The background noise, then, is an occasion setter: a signal that indicates when a regularity involving a CS and a US is present. In this case, the noise is a positive occasion setter because the presence of the sound indicates the presence of the CS-US relation. Negative occasion setters are stimuli that indicate that the CS-US relation is absent in the presence of that stimulus. The fact that the CS (e.g., the illumination of the button) elicits a CR only if the occasion setter (e.g., background noise) is present can be seen as a consequence of the fact that the regularity involving the CS and US is present only when the occasion setter is present. The effect of occasion setters on CRs is called occasion setting.
However, a second functional analysis is also possible. One can say that there are two different relations, each involving a unique CS. The background noise and the lighting of the button can be viewed as a compound CS that is more than the sum of its two elements (just as the letter T is more than the sum of a horizontal and a vertical line). In that case, there are two different relations with two different CSs: CS (A) (the compound stimulus) is followed by the US, while CS (B) (only the illumination of the button) is not followed by the US. The fact that CS (A) does elicit a CR can be explained by the presence of a relation between CS (A) and the US. The fact that CS (B) does not elicit a CR can be explained by the absence of a relation between CS (B) and the US (see Pearce, 2002, for a discussion of how many phenomena in conditioning can be explained by the idea that learning is about compound stimuli instead of individual stimuli).
It is very difficult, if not impossible, to determine which of these two functional analyses is most useful. After all, the statements refer to the same aspects of the procedure (characteristics of situations in which the US occurs and situations in which the US does not occur) but describe them in a different way (but see Bonardi, Robinson, & Jennings, 2017, for arguments in support of the view that not all findings on occasion setting can be explained in terms of compound stimuli). Despite this difficulty in studying occasion setting, there has been much research on this topic (see Leising & Bonardi, 2017, for a brief history of this research, and Trask, Thrailkill, & Bouton, 2017, for a review). The interest in occasion setting is partly due to the parallel between occasion setting and the context sensitivity of extinction (e.g., Bouton, 1993).
Think It Through 2.4: The Relation between Renewal and Occasion Setting
Question: What do you think is the connection between the phenomenon of renewal in extinction and the phenomenon of occasion setting? Moreover, what is the relation between spontaneous recovery to occasion setting?
2.2.5.3. The way that the CS-US relation is presented The same regularity involving two stimuli can be presented in different ways. First, one can vary the time when the CS and US are presented. For instance, the presentation of the CS can begin before, together with, or after the initial presentation of the US. Procedures in which the CS comes before the US are called forward conditioning procedures. When CS and US are presented at the same time, this is called simultaneous conditioning. When the US comes before the CS, this is called backward conditioning. Researchers have observed that backward conditioning effects are usually smaller than forward conditioning effects (but see Prével, Rivière, Darcheville, & Urcelay, 2016, and Prével, Rivière, Darcheville, Urcelay, & Miller, 2019, for evidence that reliable backward conditioning effects can be found). Another general finding is that conditioning effects become smaller as more time passes between the end of the presentation of the CS and the start of the presentation of the US. When, however, there is an intrinsic relation between the CS and US (e.g., food and nausea), the duration of this interstimulus interval will have less impact, and conditioning effects can be observed even with very long intervals (Etscorn & Stephens, 1973).
2.3 Mental Process Theories
In the second part of this chapter we will investigate which mental processes mediate the impact of regularities in the presence of stimuli on behavior (i.e., classical conditioning effects). Specifically, we will investigate the heuristic and predictive value of several mental process theories—that is, the extent to which they are able to explain existing functional knowledge (heuristic value) and predict new functional knowledge (predictive value). We focus on two classes of theories. The first class is defined by the assumption that the effect of stimulus pairings on behavior (classical conditioning) is mediated by the formation of associations between representations in memory. We call this class of theories associative models. A second, more recent class of propositional models assumes that classical conditioning is mediated by the formation and evaluation of propositions about stimulus relations in the environment.
2.3.1 Associative Models
This broad class of models has dominated the study of learning since the emergence of the discipline (see Pearce & Bouton, 2001, and Bouton, 2016, for an overview). A common feature of all associative models is the assumption that stimulus pairings under certain conditions lead to the formation of associations in memory. Associative models differ with regard to their assumptions about the conditions under which associations are formed (figure 2.13, Step 1), the elements involved in the association (figure 2.13, Step 2), and the conditions under which associations influence behavior (figure 2.13, Step 3).
Schematic representation of associative models of conditioning. All associative models are based on the assumption that classical conditioning is mediated by the formation of associations in memory, but they differ with regard to assumptions about: (1) the conditions under which associations are formed, (2) the elements involved in the associations, and (3) the conditions under which associations in memory affect behavior.
With regard to the first point, we make a distinction between S-R (stimulus-response) models that assume that learning is based on the formation of associations between stimuli and responses and S-S (stimulus-stimulus) models that postulate the formation of associations between the representations of stimuli. Within the S-R and S-S models, there are several models that each make unique assumptions about the conditions under which associations are formed and under which they influence behavior.
2.3.1.1 S-R models
a) The core of S-R models According to S-R models, an association is formed between the representations of the CS and the UR (see figure 2.14). As a result, presentation of the CS can also trigger the UR, and the UR thus becomes a CR. Let us apply this idea to the case of Pavlov’s dog. Initially, the dog salivates only when presented with food (UR). By pairing the bell and food (and thus also the occurrence of the bell and the salivation that is elicited by the food), the bell will also trigger salivation (the UR becomes a CR). A prototypical S-R model therefore starts from the assumption that contiguity between CS and UR is a sufficient and necessary condition for forming S-R associations. Once the S-R association is formed, the CS can activate the UR via this association, leading to the CR. This assumption states that the US is important only in order to elicit the UR and thus ensure that the CS and UR occur together. So there is no learning about the US but only learning through the US.5
Schematic representation of S-R models. (1) The co-occurrence of CS and UR is sufficient to form an association between the representation of the CS (S) and the UR (R) in the memory (2). After forming the S-R association, presenting the CS is a sufficient condition for the change in behavior: the CS then elicits a CR that is identical to the UR (3).
b) General evaluation of S-R models Although S-R models are able to explain that presenting two stimuli together can lead to changes in behavior, the prototypical S-R model is contradicted by several findings. Some of these findings go against the basic assumption that the combination of CS and UR is a necessary and sufficient condition for conditioning to occur. Other findings show that conditioning is effectively mediated by knowledge about the US. Next, we review some of the failures of S-R models (also see Rescorla, 1988).
– Not contiguity but (conditional) contingency determines whether classical conditioning will occur (see section 2.2.5.1)
Most S-R models are based on the assumption that the formation of S-R associations is determined by contiguity (i.e., by the degree to which and number of times that the CS and UR occur together in space and time; e.g., Guthrie, 1946). Yet this idea conflicts with the finding that it is not contiguity but rather (conditional) contingency that seems to be the most important determinant of classical conditioning effects. In other words, research clearly shows that contiguity is not a sufficient condition for conditioning, contrary to what is assumed in a prototypical S-R model.
– Sensory preconditioning (see section 2.2.5.1)
The influence of indirect stimulus relations on behavior also cannot be explained by S-R models, because the CS then elicits a CR despite the fact that the CS and UR have never occurred together. Take the example of sensory preconditioning: in a first phase, two neutral stimuli (e.g., a tone and a light) are presented together; in a second phase, one of the two stimuli (e.g., the light stimulus) is followed by a US until a CR is established; in a third phase, the other neutral stimulus from the first phase also provokes a CR (e.g., the tone). Such a finding shows that the coexistence of a CS and UR is not a necessary condition for the occurrence of conditioning: the crucial CS (in our example, the tone) was paired only with another neutral stimulus that did not elicit a response at the time of those pairings.
– US revaluation (see section 2.2.1.3)
Studies on US revaluation have shown that changes in the US after conditioning can influence behavior. For example, if you first pair a bell with food and then make the food negative (e.g., by pairing it with nausea), the bell will no longer trigger salivation. This cannot be explained on the basis of S-R models. According to an S-R model, an association between the bell (CS) and salivation (UR) is learned during the bell-food trials. Nothing is learned about the US. The US serves only to establish the UR so that an association can grow between the CS and UR representations. Changing the US after forming the CS-UR association should therefore have no influence.
– The CR may differ from the UR (see section 2.2.1.3)
Because S-R models assume that classical conditioning is the result of an association between the CS and UR, these models cannot explain why the CR can be different from the UR. After all, according to S-R models, the only difference between the CR and the UR is that one speaks of an UR if the reaction is elicited by a US, while the term CR is used to refer to the same reaction when it is elicited by the CS. Research shows, however, that usually the CR seems to be a reaction that prepares the organism for the arrival of the US. This preparatory response (e.g., fearful anticipation of a shock) can be very different from the UR (e.g., the pain evoked by the shock). This therefore suggests that knowledge about the US is very important in the creation of classical conditioning.
– Conclusion
It seems that S-R models cannot explain many aspects of functional knowledge about classical conditioning. We must admit, however, that we have assumed a caricature of S-R models, especially with regard to the assumption that the co-occurrence of the CS and US is a sufficient condition for forming associations. After all, it is easy to make variants of S-R models that state that factors other than contiguity are also important (e.g., attention). Nevertheless, it is not an accident that we focused on a model in which contiguity is seen as a sufficient and necessary condition. This model remains the prototypical S-R model of conditioning, which we see discussed in many psychology textbooks and which many people have in mind when they think of conditioning (e.g., Byrne & Bates, 2006). As Rescorla (1988; also see Eelen, 1980/2018) remarked a long time ago, it is high time we realize that these types of simplistic S-R models are highly problematic.
That said, we cannot exclude the possibility that S-R associations could mediate conditioning effects under very specific conditions. For example, Rescorla (1982) conducted experiments in which CS1 was first paired with a US. Then CS2 was repeatedly followed by CS1. After that phase, it was determined that CS2 also triggers a CR. This is an example of second-order conditioning (see section 2.2.5.1). If a US revaluation procedure was then applied (e.g., the food is now coupled with nausea), this did not appear to affect the extent to which CS2 provokes a CR. The CR with respect to CS2 thus seems to be based on S-R associations. It is indeed quite possible that different conditioning effects are mediated by different mechanisms and that some effects (e.g., second-order conditioning) rely on an S-R mechanism. Note, however, that if one accepts this view, then one must also try to find out the specific conditions under which the mechanism operates. This is not an easy task because it is already difficult to determine which mental mechanism is effective in a given situation (one cannot directly observe mental processes; see De Houwer, 2011b).
2.3.1.2 S-S models
a) The core of S-S models According to S-S models, the pairing of the CS and US results in an association between the representations of the CS and US in memory (see figure 2.15). Presenting the CS will lead to an activation of the CS representation. This activation spreads via the CS-US association to the US representation and thus elicits the UR (which is part of the US representation or associated with the US representation). For example, you could say that Pavlov’s dog salivates when he hears the bell because the bell reminds him of food, and thinking of food leads to salivation.
Schematic representation of S-S models. (1) The pairing of CS and US leads, under certain conditions (e.g., if attention is paid to the CS and US), to (2) the formation of an association between the representation of the CS (S) and the representation of the US (S) in memory. After forming S-S associations, CS presentations will lead, under certain conditions, to the CR (3).
A crucial difference between S-S and S-R models is that S-S models view conditioning as being dependent on knowledge about the US. Another crucial difference is that most S-S models assume that the pairing of the CS and US is not a necessary and sufficient condition for the occurrence of conditioning. Usually it is assumed that certain “cognitive conditions” must be met so that the pairing of the CS and US leads to the formation of associations (e.g., attention to the CS and/or US; see De Houwer, 2018b). Because of these differences, S-S models can explain certain findings that are problematic for S-R models. Below, we first provide an overview of how S-S models (in general) can explain some important functional properties of classical conditioning, then we discuss a number of specific S-S models in greater detail.
b) General evaluation of S-S models
– US revaluation (see section 2.2.1.3)
According to S-S models, a CR can occur only after the US representation is activated. Changes in the US representation can therefore lead to changes in the CR. Let’s return to the example of Pavlov’s dog. The repeated pairing of the bell and food leads to salivation because the bell is reminiscent of the tasty food. Thinking about the food elicits salivation. After US revaluation (e.g., food is accompanied by nausea) the dog will still think of food, but the food is no longer tasty, so thinking of that food will not elicit salivation. Hence, US revaluation will eliminate the conditioned salivation response to the bell.
– The impact of secondary tasks on conditioning (see section 2.2.4)
In S-S models, contiguity is not a sufficient condition. For example, most S-S models start from the assumption that S-S associations are formed only if attention is paid to the CS and US (e.g., Wagner, 1981; see below). Such models are consistent with the finding that secondary tasks have a detrimental effect on classical conditioning because these secondary tasks divert attention from (the pairing of) the CS and US. In box 2.5, we discuss how one can also explain other findings such as blocking based on the assumption that attention is important for forming associations.
Box 2.5 The Role of Attention in Classical Conditioning
In various S-S models, we see that a major role is reserved for attention. Pairing CS and US together is not a sufficient condition for forming associations; sufficient attention must also be paid to the US (and the CS) so that associations can be formed. We can already recognize the role of attention in the Rescorla-Wagner model and Wagner’s SOP model (1981). After all, it can be assumed that the degree of expectation discrepancy determines how much attention is paid to the presence or absence of a stimulus (see Dickinson, 1980, chapter 4, for an excellent discussion). If the US is unexpected, much attention will be paid to the US, and much is “learned” (in the cognitive sense of knowledge acquisition). In the Wagner model (1981) we see that the formation of associations depends on the extent to which the CS presentation is expected. If the presentation of the CS is unexpected, much attention is paid to the CS and associations can be formed. The idea that attention to the CS is important provides an elegant explanation for the effects of CS pre-exposure on classical conditioning. When the CS is repeatedly presented on its own (CS-only trials) and only afterward the CS is presented together with the US (CS-US trials), classical conditioning will be less pronounced than when there are only CS-US trials. Wagner’s model attributes this to the fact that, as a result of the CS-only trials, the presence of the CS is expected in that context and therefore receives little attention. Thus, context-CS associations are formed that determine the extent to which the CS presentation is expected, which in turn determines how much attention is given to the CS, and thus how well the CS-US relation is “learned.”
There are many other S-S models in which attention also plays an important role. These models often differ with regard to the factors that determine how much attention is paid to the CS and US. As we have seen earlier, attention is determined by the degree of expectation discrepancy in the Rescorla-Wagner and Wagner models. Yet, in the model of Mackintosh (1975), attention to the CS is determined by the extent to which the CS is a good predictor of the US. It indeed seems worthwhile to pay more attention to stimuli that help you predict important events than to stimuli that give little (additional) information about events in the environment. One can explain many findings about classical conditioning on the basis of this idea. The effects of CS pre-exposure, for instance, fit perfectly within the Mackintosh model. If you repeatedly present only a CS, this implies that the CS is not a predictor of important events, which will reduce attention to that CS. As a result, the CS-US relation is not noticed as quickly afterward. Note that both Wagner (1981) and Mackintosh (1975) offer an explanation for CS pre-exposure effects in terms of attention to the CS, but that they differ in the way that CS pre-exposure leads to lower attention to the CS (for Wagner, this is because of context-CS associations, whereas for Mackintosh, this is because the CS does not predict anything). Blocking can also be explained on the basis of the Mackintosh model. As we discussed previously, blocking refers to the finding that CS X elicits a weaker CR when AX+ trials are preceded by A+ trials than when there are only AX+ trials. Mackintosh explains this by assuming that as a result of the A+ trials, a lot of attention is paid to CS A. During the AX+ trials, the organism will determine that A remains a perfect predictor of the US, while X does not provide any additional information. This will reduce the attention for X, and the X-US association will be weaker after A+ and AX+ trials than after only AX+ trials.
On the other hand, one could also argue that there is no need to continue to pay a lot of attention to CSs that you already know are important. It indeed seems more important to pay attention to stimuli that you are uncertain about, compared to stimuli that you have certainty about with regard to how they act or what they predict. If you notice a CS and you know that the CS is a predictor of the US, then you can direct your attention away from that CS. However, if you see a new, unprecedented CS in an environment in which unpredictable USs occur, it is important to pay a lot of attention to the new CS because the CS could help you in predicting these USs. This idea lies at the core of the Pearce and Hall model (1980): if you are confronted with unpredictable USs, it is better to pay attention to the stimuli that you do not yet know the meaning of than to those you already know the meaning of. This model also offers an explanation for the effects of both CS pre-exposure and blocking. For instance, after CS-only trials, the meaning of the CS becomes clear: it does not predict anything. That is why less attention is paid to this CS and the CS-US relation will be noticed less quickly. In blocking, the A+ trials ensure that the organism learns that the USs are predictable (by A). As a result, the US is also expected on the AX+ trials (i.e., there are not unpredictable USs) and there is therefore no need to pay attention to new (X) or old (A) CSs.
The fact that models with diametrically opposed assumptions can explain the same phenomena can be very confusing. It shows how careful we must be when drawing conclusions about mediating mental processes on the basis of functional knowledge. As Bouton (2007, p. 123) points out, both the Mackintosh (1975) and Pearce and Hall (1980) models have contributed to understanding the role of attention in classical conditioning (see Le Pelley, Mitchell, Beesley, George, & Wills, 2016, for an overview). The core assumption of both models (and also the models of Rescorla-Wagner, 1972, and Wagner, 1981) is that attention is a crucial determinant of classical conditioning. On the other hand, research into the role of attention in conditioning also shows that conditioning is important in determining attention: attention to the CS varies depending on whether it is a predictor of other events. In this way, conditioning research has also contributed to a better understanding of the determinants of attention (see Le Pelley et al., 2016). Despite the importance of the concept of attention in the cognitive approach (of learning psychology), it remains unclear what attention actually is. Like concepts such as learning and conditioning, attention is often seen as a mental process that can explain behavior, but there is little consensus about what that mental process looks like. We also agree with Anderson (2011) when she doubts the extent to which the concept of attention can really play an explanatory role in cognitive theories. In our opinion, like learning and conditioning, attention is better defined as an effect (namely, the selective response to stimuli in the environment).
– Sensory preconditioning (see section 2.2.5.1)
For S-S models, contiguity is not a necessary condition. For example, one can explain sensory preconditioning based on S-S models. It is possible that during the first phase, an association is formed between the representations of the two neutral stimuli. During the second phase, an association is formed between the second neutral stimulus and the US. When the first neutral stimulus is subsequently presented, this leads to activation of the representation of that stimulus. This activation spreads to the representation of the second neutral stimulus and then to the representation of the US through which the UR/CR occurs (see figure 2.16).
Schematic representation of how S-S models provide an explanation for sensory preconditioning. The co-occurrence of CS1 and CS2 during the first phase results in the formation of an association between the representations of CS1 and CS2 in memory. The joint presentation of CS2 and the US in the second phase creates an association between the representation of CS2 and the US. When CS1 is presented during the test phase, this will lead to activation of the CS1 representation. This activation can then spread via the CS1-CS2 association to CS2 and so on, via the CS2-US association and the US-UR association up to the representation of the UR. Once the representation of the UR has been activated, this will lead to a CR.
– Conclusion
In sum, S-S models can explain a range of findings. Despite these successes, one could also ask questions about the basic idea underpinning S-S models. For instance, take the idea that the CR is the result of activation that spreads through a CS-US association in memory. This idea implies that the CR must always be identical to the UR because the CS activates the UR components that are connected to the US. It is therefore not entirely clear how S-S models explain how the CR and UR may differ from each other (see Bouton, 2016, p. 187, for a more in-depth discussion of the relation between the CR and the UR).
Instead of supposing that the activation of the CS ultimately leads to the activation of the UR, one could also assume that the activation of the CS leads only to the activation of the US representation. This, in turn, would lead to the expectation that the US will occur. According to this version of S-S models, the CS is a signal for the arrival of the US. The expectation of the US can then give rise to (controlled and involuntary) preparatory responses (i.e., responses that help the organism prepare for the arrival of the US; e.g., salivation that allows it to consume food better; see Kirsch, Lynn, Vigorito, & Miller, 2004, for a discussion). This could then explain why the CR (e.g., fear that an electric shock will occur) can differ from the UR (e.g., pain in response to the actual presence of an electric shock). But even that statement leaves a number of questions unanswered. It is especially unclear how the activation of a US representation can give rise to an expectation of the US. The mere activation of the US may lead to thinking about the US (e.g., the bell is reminiscent of food), but this is not the same as an expectation that the US will be effectively presented. After all, there are many situations in which one thinks of a stimulus without expecting that this stimulus will occur (Baeyens et al., 1992; Jozefowiez, 2018). It is not clear how one and the same process of US activation through S-S associations can lead to such fundamentally different cognitive states (i.e., thinking of something and expecting the presence of something).
Also, the importance of conscious knowledge about the CS-US relation in classical conditioning cannot be explained by S-S models in a straightforward manner. According to some S-S models, the S-S association must first give rise to a conscious expectation of the US before a CR can occur. The conscious expectation of the US is thus a “necessary gate”—that is, a necessary intermediate step that must be present before the S-S association can lead to a CR (Dawson & Schell, 1987). However, this does not explain why, in many situations, awareness of the CS-US relation seems necessary for conditioning to occur (see Mitchell et al., 2009). After all, a conscious expectation of the US is not the same as conscious knowledge of the CS-US relation.
c) The Rescorla-Wagner model as a prototypical S-S model The Rescorla-Wagner model (Rescorla & Wagner, 1972) is often considered a prototypical example of S-S models. It differs from other associative S-S models with regard to the assumptions about the conditions under which associations are formed and influence behavior. The core assumption in the Rescorla-Wagner model is that the extent to which CS-US associations are modified (see Step 1 in figure 2.15) depends on the extent to which the presence or absence of a US is expected or unexpected. If the presence or absence of the US is expected, no new associations will be formed and the strength of existing associations will not change. If the presence or absence of the US is unexpected or surprising, new associations will be formed and the strength of existing associations will change. One can also say that according to the model, “learning” (i.e., forming and changing associations) is dependent on expectation discrepancy. The second crucial assumption is that the strength of the CR to the CS is more or less a direct consequence of the strength of the association between the representation of the CS and US (see Step 3 in figure 2.15). In other words, the translation of associations to the behavior is simple. Initially, little attention was paid to this second assumption; researchers focused on testing the first assumption. Nevertheless, it will become apparent that this second assumption is crucial and also can be questioned.
The assumption that the formation of associations is driven by expectations was captured by Rescorla and Wagner (1972) in the following mathematical formula:
ΔVA = αA β (λ − VA)
The symbols used represent the following:
ΔVA = the change that occurs in the associative strength of CS (A); that is, the strength of the association between the representation of CS (A) and the US. Associative strength is often seen as the degree to which the CS leads to expectations that the US will occur.
αA = a parameter that reflects the salience or intensity of the CS.
β = a parameter for the salience or intensity of the US.
λ = the asymptote of conditioning; this is the maximum associative strength that is possible for a particular US with a certain intensity.
VA = the existing associative strength of stimulus A.
From the formula, it can be concluded that conditioning will increase in strength as the discrepancy between λ and VA becomes larger. Let’s demonstrate this point with an example in which we assign the following values: αAβ equals .50; λ equals 10; and VA initially equals 0.
Trial 1: ΔVA = .50 (10 − 0) |
= |
5 |
||
Trial 2: ΔVA = .50 (10 − 5) |
= |
2,5 |
||
Trial 3: ΔVA = .50 (10 − 7,5) |
= |
1,25 |
||
Trial 4: ΔVA = .50 (10 − 8,75) |
= |
0,625 |
||
VA after four A+ trials |
= |
9,375 |
In this example, after four trials the associative strength has become 9.375; with further pairings it approaches the asymptotic (i.e., maximum) value 10. We also see that the associative strength changes more during the first than during later trials. This is because the presence of the US is more surprising (ΔVA is greater) at the beginning than at the end of the learning phase.
If after the acquisition phase (CS-US trials) an extinction procedure is implemented (repeatedly presenting the CS without the US), the CR will systematically decrease in intensity. As we noted earlier, this effect is called extinction. According to the Rescorla-Wagner model, extinction as an effect is due to a decrease in the associative strength of the CS. After all, during the first trials of the extinction procedure, the presentation of the CS leads to a pronounced expectation that the US will follow (VCS > 0). Because the US is not presented during an extinction trial, λ will have a value of 0 on that trial. As a result, (λ-VCS) is less than zero and the associative strength of the CS decreases (ΔVCS = αCS β (λ − VCS)). Therefore, according to the Rescorla-Wagner model, extinction (as an effect) is due to the “unlearning” or forgetting of a CS-US association. To illustrate, let’s continue the numerical example listed above, and add four trials with only CS A:
Trial 5: Δ VA = .50 (0 − 9.375) |
= |
− 4.69 |
||
Trial 6: Δ VA = .50 (0 − 4.69) |
= |
− 2.34 |
||
Trial 7: Δ VA = .50 (0 − 2.34) |
= |
− 1.17 |
||
Trial 8: Δ VA = .50 (0 − 1.17) |
= |
− 0.58 |
||
VA after four A+ and four A-trials |
0.58 |
Another important assumption of the Rescorla-Wagner model is that the expectation of the US is determined by all CSs present at that time (see Witnauer, Urcelay, & Miller, 2014, for a critical discussion of this assumption). This is presented as follows:
ΔVA = αA β (λ − VAX)
ΔVX = αX β (λ − VAX)
VAX stands for the sum of the existing associative strength of A and X (i.e., VAX = VA + VX). Therefore, the change in associative strength for CS X on an AX+ trial (both A and X will be presented and followed by the US) will depend on not only the expectation elicited by X but also the expectation elicited by A. This assumption allows the Rescorla-Wagner model to explain phenomena such as blocking. Blocking refers to the finding that the CR for X after AX+ trials is weaker when these trials are preceded by A+ trials. According to the Rescorla-Wagner model, this is because an A-US association is formed during the A+ trials, as a result of which A already elicits the expectation that the US will be presented on the first AX+ trial. Consequently, there is little expectation discrepancy (difference between what is expected and what actually occurs) and the X-US association will not be formed, or will be weak. In more formal terms, we can say that VA has a high value as a result of the A+ trials. As a result, VAX will also have a high value (because it is equal to the sum of VA and VX) and the difference between λ and VAX is also small, as a result of which the associative strength of X changes little (ΔVX is small). Note that according to the Rescorla-Wagner model, blocking occurs because the association between the blocked stimulus X and the US is not formed. Blocking thus points to the failure of learning (in the cognitive sense of acquiring knowledge).
Interestingly, because the expectation of the US is determined by all CSs present at a certain point in time, associative strength can sometimes become negative. Imagine that you present Y+ trials intermixed with YA- trials. As a result of the Y+ trials, Y will get a positive associative strength. Hence, the US will be expected also on the YA- trials. However, because the US is always absent on YA- trials, the change in associative strength on those trials will always be negative, which means that the associative strength of A will drop below zero. If this happens, then the presence of A will lead to the inhibition of the US representation, which is assumed to result in the expectancy that the US will not occur (which is not the same as having no expectancy of the US, as would be the case when the associative strength is zero). Hence, the Rescorla-Wagner model captures the idea that organisms can also actively learn to predict the absence of stimuli.
The Rescorla-Wagner model has both high heuristic and high predictive value. It has led to the discovery of a number of phenomena, such as superconditioning, whereby you can make conditioning extra strong by pairing a CS together with a CS that has a negative associative strength. Again imagine that you first present Y+ and YA- trials. As noted in the previous paragraph, this will result in inhibitory learning about A; that is, A will develop a negative associative strength. If you subsequently present AX+ trials, the associative strength of X will be greater than if you only present AX+ trials (i.e., no Y+ and YA- trials).
Think It Through 2.5: Rescorla-Wagner (Example 1)
Create a numerical example showing how the Rescorla-Wagner model explains superconditioning.
Think It Through 2.6: Rescorla-Wagner (Example 2)
Create a numerical example in which you show how the Rescorla-Wagner model explains the effect of contingency— that is, the fact that conditioning depends not only on the co-occurrence of CS and US (cell (a) in the four-field table) but also on the occurrence of the CS or US on its own (cell (b) and cell (c) in the four-field table).
Because of its high heuristic and predictive value, the Rescorla-Wagner model has been very influential both in and outside of learning psychology. For example, it has been important in the development of so-called connectionist models (Rumelhart & McClelland, 1986), research into reinforcement learning as it is now conducted in computer science (e.g., Lee, Seo, & Jung, 2012; Sutton & Barto, 1998), and related theories of predictive coding (e.g., Clark, 2013; Friston, 2009). Yet it was clear very quickly that the Rescorla-Wagner model also had important limitations. A critical analysis of the theory was published by Miller, Barnet, and Grahame (1995), who described no fewer than twenty-three “failures” of the model (i.e., findings that cannot be explained by the Rescorla-Wagner model). That is why researchers have looked for alternative associative models that make other assumptions about the way in which associations are formed and influence behavior. We discuss a number of these models, always drawing on findings that the Rescorla-Wagner model cannot explain.
d) Extinction is not due to the removal of associations: The models of Wagner and Bouton As we noted above, the Rescorla-Wagner model explains extinction in terms of reduction in associative strength. Therefore, according to the Rescorla-Wagner model, extinction (as an effect) is due to the “unlearning” or forgetting of a CS-US association. In section 2.2.5.2, however, we discussed evidence that shows that extinction is not due to unlearning or forgetting. For example, CS postexposure trials (CS-only trials after acquisition) have no influence when presented in a separate context (renewal). Extinguished CRs can also reappear spontaneously over time (spontaneous recovery). These effects should be impossible if CS postexposure trials lead to the disappearance of an association.
Wagner (1981; see also Wagner & Brandon, 2001, and Vogel, Ponce, & Wagner, 2019, for a recent review) presented the sometimes opponent processes (SOP) model, which was an important first step in explaining phenomena such as renewal. In line with earlier proposals (e.g., Konorski, 1967; Pavlov, 1927), Wagner’s SOP model postulated that two types of S-S associations can be formed: excitatory associations and inhibitory associations. Excitatory associations will increase in strength when a CS is followed by the unexpected presence of a US, while inhibitory associations will increase in strength when a CS is followed by the unexpected absence of the US. So you could say that the strength of an excitatory association is a reflection of the extent to which the CS helps predict the presence of the US, while the strength of an inhibitory association is a reflection of the extent to which the CS helps predict the absence of the US. When a CS and US representation are connected by an excitatory association, the delivery of the CS will lead to an increase in the activation of the US representation. When an inhibitory CS-US association is formed, the CS presentation will lead to a reduction in the activation of the US representation. The same CS and US can be connected by both an excitatory and an inhibitory association. The strength of both determines the effect that CS presentations will have on the activation of the US representation and thus on the strength of the CR.
We will not go into the precise way in which inhibitory and excitatory relations come about (see Bouton, 2016, pp. 144–150), but we do want to note that Wagner’s SOP model is still one of the most elegant and influential models in cognitive learning psychology. In contrast to the Rescorla-Wagner model, which is essentially no more than a mathematical formula and can be considered cognitive only because the different elements in the formula can be interpreted as mental states (e.g., ΔV as expectation discrepancy), the SOP model is firmly entrenched in the cognitive approach to learning psychology. Although the idea of inhibitory associations was not well received by everyone (e.g., Miller & Matzel, 1988), it continues to be very influential. For instance, it has had a big impact on the development of techniques for the treatment of anxiety disorders (e.g., Craske, Treanor, Conway, Zbozinek, & Vervliet, 2014) and obesity (Epstein et al., 2009). In addition, the SOP model makes interesting predictions about the interaction between habituation and conditioning (see also section 1.2.2 on the role of conditioning of opponent processes). It is not for nothing that Bouton (2016, p. 144) describes Wagner’s SOP model as “the single most complete account of conditioning and associative learning that is available.”
Because excitatory and inhibitory associations can exist simultaneously, the meaning of a CS can be ambiguous: it can be at the same time a signal for the presence of the US and a signal for the absence of the US. Bouton (1993, 2004; see also Rosas, García-Gutiérrez, & Callejas-Aguilera, 2006) noted that this ambiguity can be solved by taking the context into account. It is indeed possible that a CS in a given context is a signal for the presence of the US and in a different context, a signal for the absence of the US. This assumption is in line with effects such as renewal (see section 2.2.5.2). In studies on renewal, the CS is followed by the US in a certain context (e.g., a blue room). According to Bouton, this leads to the formation of an excitatory association between the CS and the US (see figure 2.17, solid line). This association would be context-independent because at that moment in time, the meaning of the CS is not yet ambiguous (see Rosas et al., 2006). Afterward, the CS is presented alone in a different context (e.g., a green room). This leads to the formation of an inhibitory association (see figure 2.17, dashed line) that is context-dependent (see figure 2.17, dash-and-dot line). In other words, the organism first learns that the CS is a predictor of the US (which is reflected in the strength of the excitatory association) and then learns an exception to that rule—namely, that the CS is sometimes (e.g., only in the green room) followed by the absence of the US. Because the inhibitory association is context dependent, it will only play a role in the context in which it is formed (i.e., in the context in which extinction took place; the green room). Based on this model we can explain why the CS does not elicit a CR in the extinction context (e.g., green room; both associations have an influence on the US representation and therefore on behavior) but does subsequently do so in the original context (e.g., blue room; only the excitatory association has an influence on the US representation).
How the model of Bouton (1993) explains extinction. During CS-US presentations an excitatory association is formed (full line; +). During the extinction procedure, an inhibitory association (dashed line, −) is formed. The inhibitory association is modulated by the context so that the inhibitory association is active only when the context is present (dash-point line, +).
The essence of Bouton’s model is that extinction as a procedure does not lead to the unlearning or forgetting of an association, but to the acquisition of new knowledge about the CS-US relation (namely, the inhibitory association). Extinction does not involve “unlearning” but “learning” (in the cognitive sense of changing knowledge). Spontaneous recovery can also be explained in this way if one considers time as a kind of context. During extinction, the animal “learns” that the CS is no longer followed by the US at that moment in time. If time passes, the animal is in a different time context and it can therefore no longer be certain that the CS will still not be followed by the US. It still knows that at some point in the past the CS was not followed by the US, but it is possible that this period is over.
Box 2.6 Implications of Bouton’s Model
Bouton’s (1993) model has important clinical implications (Vervliet, Craske, & Hermans, 2013). As mentioned earlier (box 2.1), behavior therapy is derived directly from research on classical conditioning. The basic idea here is that psychological complaints such as anxiety disorders are examples of classical conditioning (i.e., changes in behavior that result from stimulus pairings). This functional analysis implies that anxiety disorders can be treated in the same way that conditioned anxiety is modified in the laboratory. This has led to the development of exposure therapies where patients are repeatedly exposed to the stimulus that elicits fear (e.g., a spider). These exposure treatments are equivalent to the extinction procedures developed in the laboratory. Although exposure therapy is very effective in the short term, it appears that the original complaint can sometimes reemerge even after treatment (e.g., the client becomes frightened by spiders once more). This relapse can be understood from the literature on extinction. The work of Bouton (1993) implies that exposure to spiders will not lead to the disappearance of the associations that initially led to the phobia. Instead, during exposure, the patient will “learn” that under certain circumstances (e.g., in the therapist’s treatment room, in the presence of the therapist, during that particular period) seeing or touching spiders does not lead to unpleasant consequences. This additional knowledge is inherently context dependent. It is therefore possible that the patient will still have a fear of spiders when he or she comes home (renewal) or that the fear will return spontaneously after a certain period of time (spontaneous recovery). To reduce the probability of relapse after successful treatment, the therapist can apply the exposure treatment in different environments, including environments in which the patient is often confronted with the phobic object (see Craske et al., 2014). From the above it can also be understood why relapse occurs so often in drug use (also see discussion of the opponent-process theory of Solomon in chapter 1, section 1.2.2).
The idea of context-dependent learning has also played an important role in research on our first impressions of other people. When we meet another person, we often immediately feel good or bad about this person. Research shows that this spontaneous impression can be an example of evaluative conditioning. This means that this first impression is the result of an earlier event in which the person co-occurred with something positive or negative (e.g., the person did or said something good or bad). Further research suggests that the first experience with a certain person is extremely important in the long term. Imagine seeing a new colleague at work for the first time in the hallway on the way to your desk. You greet him but he does not respond, or he looks angrily at you. As a result of that first experience, you probably develop a negative impression of your new colleague. Afterward, your boss presents the new colleague to you. At that moment, the new colleague is very friendly. Probably that second experience will do little to change your negative impression of the colleague and you will only learn that the colleague is friendly when your boss is present. One possible way to explain this is that the first experience with someone results in context-independent knowledge: you assume that your new colleague is unfriendly. A second experience that contradicts the first experience will only result in context-dependent knowledge: your new colleague is friendly when your boss is there. This idea is very similar to that of Bouton: the first thing you learn (the CS is followed by the US) is context-independent, but the second thing you learn (the CS is no longer followed by the US) is context-dependent (see Rosas et al., 2006). Because your original impression is based on context independent knowledge, it will be applied to all new contexts (e.g., when you meet your new colleague at a party) while the second experience will only have an impact in one context (i.e., when your boss is present). The context independence of initial learning might thus explain why first impressions can be so important (see Gawronski, Rydell, De Houwer, Brannon, Ye, Vervliet, & Hu, 2018, for an overview of this research).
e) Blocking is not due to the failure to “learn”: The comparator model of Miller We previously clarified that blocking—according to the Rescorla-Wagner model—is due to a “failure to learn”: because the presence of the US is expected on A+ and AX + trials the X-US association is not formed. Backward blocking, however, cannot be explained by the Rescorla-Wagner model. In studies on backward blocking, the organism is first confronted with AX+ trials and only then with A+ trials. According to the Rescorla-Wagner model, the X-US association should be formed on the AX+ trials. The subsequent A+ trials should have no effect on the strength of the X-US association simply because X is not present on those trials. However, several studies show that the CR triggered by X is smaller if the AX+ trials are followed by A+ trials than if only AX+ trials are presented. Such backward blocking effects indicate a fundamental error within the Rescorla-Wagner model. Within this model, it is assumed that all information about a CS-US association is abstracted (summarized) in a single parameter (i.e., the strength of the CS-US association [VCS]). Once information (e.g., a CS-US pairing) has had its influence, this information is forgotten. One can only learn about a stimulus at the moment at which it is presented. The fact that backward blocking can occur, however, suggests that the organism retrospectively revises the implications of the AX+ trials in the light of the A+ trials (see Miller & Witnauer, 2016, for an overview of research into backward blocking and other forms of retrospective revaluation).
Box 2.7 Can Emotional Memories Be Erased?
Over the past decade much attention has been focused on the idea that emotional memories can be erased from memory (see Beckers & Kindt, 2017, for a review). The starting point of this research is that memories of emotional events in the brain must be “consolidated” (strengthened) before they can have a long-lasting effect. Furthermore, it is assumed that even old, already consolidated memory tracks have to be consolidated again (reconsolidation) each time they are activated (e.g., every time the memory of a traumatic event comes up). Finally, it is assumed that both consolidation and reconsolidation can be weakened by administering certain chemical substances (e.g., propranolol). If these assumptions are correct, then one could make memory traces unstable and even erase them if one administers these chemicals at times when (re)consolidation is necessary, for example, when people think back to a traumatic event. This prediction has been tested in the context of classical conditioning (e.g., Nader, Schafe, & LeDoux, 2000). A CS (e.g., a tone) was presented together with an aversive US (e.g., an electric shock). The next day only the tone was presented. The idea was that this would activate the memory trace of the CS-US pairing. A chemical was then used in a first condition to prevent the reconsolidation of that memory trace while in a second condition an inactive substance was administered. On the third day it was found that the CS elicited less fear in the first condition than in the second condition. However, later research has shown that such effects only occur (at best) under very specific conditions. Even if the effects do occur, it is unclear whether these are due to the deletion of memory traces or to other, already known processes such as context-dependent learning as described by Bouton (1993, for example, renewal where the drug provides a special feeling and thus a unique context that differs from the context during acquisition and test). It would of course be very useful therapeutically if one could erase a traumatic memory from a patient’s memory, but as is often the case with sensational ideas, this idea might be “too good to be true.” The problem with such “too good to be true” ideas is that scientists also seem to fall prey to a confirmation bias (i.e., the tendency to see their own ideas as true). One consequence is that they publish research data more quickly when they confirm their ideas than when they contradict their ideas (see Simmons, Nelson, & Simonsohn, 2011, for an overview of various reasons why research can lead to false conclusions). Even if scientists act in good conscience, false conclusions can still be drawn. It therefore remains important to be critical when you consult the literature, especially if research results seem too good to be true.
Backward blocking is consistent with an alternative associative model which is called the comparator model (Miller & Matzel, 1988; Ghirlanda & Ibadullayev, 2015; Stout & Miller, 2007). According to this model, expectation discrepancy plays no role in establishing or changing associations. The only thing that has an influence on the strength of the CS-US association is the number of times that two stimuli occur together in time and space (i.e., contiguity is the driving force behind associations). Also essential to this model is the assumption that CRs with respect to a CS are not a direct reflection of the strength of the CS-US association. According to the comparator model, CRs depend on a comparison of the strength of different associations. Take the example of blocking. Because of the AX+ trials, an A-US and an X-US association are formed, regardless of whether there are additional A+ trials and regardless of whether the A+ trials come before or after the AX+ trials. If additional A+ trials are delivered, this results in a strengthening of the A-US association, but the X-US relation remains unaffected. Because X always occurred together with A, the CR with respect to X will not only be determined by the strength of the X-US association, but also by the strength of the A-US association. The CR with respect to X is in fact a function of the X-US association strength relative to the strength of A-US association. If A+ trials are presented in addition to the AX + trials, the strength of the X-US association will be weak in comparison to the strength of the A-US association. When there are only AX+ trials, the X-US association will be as strong as the A-US association (all other things being equal). Because the CR is dependent on the X-US association strength relative to the A-US association strength, the CR for X will be weaker when both A+ and AX+ trials have been presented than when only AX+ trials were delivered. This prediction holds irrespective of whether the A+ trials precede or follow the AX+ trials.
Given that our book is designed to be an introduction to learning psychology, it is not so important to know the details of the comparator model but it is important to understand its essence: (a) “learning” (i.e., association formation) takes place in a fairly simple and unconditional way (the only thing that counts is the extent to which two stimuli occur together in time and space), and (b) associations are not directly translated into behavior but only indirectly after a comparison is made with other associations. Blocking in the comparator model is due to the fact that the learned X-US association is not reflected in behavior because it is counteracted by a stronger A-US association.
Note, therefore, that blocking according to the comparator model is not due to a failure to form X-US associations but to the fact that the formed X-US association has no impact on behavior. The model therefore makes a clear distinction between the formation of associations and behavior: the fact that no CR occurs does not necessarily mean that no association has been formed. In this respect, the comparator model is much more realistic than the Rescorla-Wagner model, which makes virtually no distinction between the formation of associations and performance (i.e., CRs in the Rescorla-Wagner model are a direct reflection of associative strength). Although the comparator model cannot explain all existing evidence, it has led to new findings and offers an interesting alternative perspective on classical conditioning (see Miller & Witnauer, 2016, and Stout & Miller, 2007, for reviews).
2.3.2 Propositional Models
2.3.2.1 The core of propositional models Associative models have dominated learning research for more than one hundred years now, basically right from the start. As a result, for some, classical conditioning as an effect is almost synonymous to association formation as a mechanism (see De Houwer, 2018b, for an historical review). It is only by clearly separating the functional level of explanation (including classical conditioning as an effect) from the cognitive level of explanation (including association formation as a mechanism) that one can take seriously the idea that classical conditioning might be mediated by processes other than association formation.
It is only recently that a second class of mental process theories on classical conditioning has been proposed (e.g., De Houwer, 2009, 2018c; Mitchell et al., 2009; Waldmann & Holyoak, 1992). What they have in common is the assumption that the effect of stimulus pairings on behavior is mediated by the (typically nonautomatic) formation of propositions about relations in the environment. Propositions are units of information that specify assumptions about the nature of events in the world. For instance, a proposition could specify that the ringing of a bell is always followed by food. Propositions have two unique characteristics: (1) A proposition has a truth value: it is possible, at least in principle, to evaluate whether the assumptions about events in the world are right or wrong (Strack & Deutsch, 2004). (2) Propositions contain relational information, that is, information about how events are related (e.g., bell predicts food, smoking causes cancer; Lagnado, Waldmann, Hagmayer, & Sloman, 2007; Waldmann & Holyoak, 1992). Suppose you establish that people with a certain disease always have a certain chemical in their blood and that this substance is not present in people who do not have the disease. One possible proposition about this relation is that the chemical in the blood causes the disease. Another possible proposition is that the disease causes the chemical in the blood. According to both propositions, there is a relation between the chemical and the disease. The propositions differ, however, with regard to the nature of the relation between the two (Waldmann & Holyoak, 1992). Note that propositions are not necessarily verbal (i.e., expressed in words). It seems fair to assume that nonverbal organisms also have knowledge about how events in the world are related. Nevertheless, the exact nature of and the flexibility in using propositions might vary greatly depending on whether an organism is verbal or not (see De Houwer, Hughes, & Barnes-Holmes, 2016, for a discussion). We will revisit this issue in chapters 3 and 4.
Associations in memory are not propositions because they do not have the two characteristics of propositions: (1) An association in memory does not specify any assumption about events in the world. It is a hypothetical state in memory that is assumed to have been created as the result of a spatiotemporal relation in the environment. It is pointless to say that an association is right or wrong. (2) An association does not encode relational information, that is, information about how events are related.6
What do we mean when we say that learning is the result of nonautomatic formation of propositions about relations in the environment? First of all, this means that people (and certain nonhuman animals) form hypotheses about relations in the environment and try to determine which hypothesis is correct. When doing so, they deploy all information that can be useful to discover and evaluate a relation. This is not only information about when stimuli occur, but also knowledge that people already have in memory and knowledge that they acquire through observations and instructions. They can deploy this knowledge not only when they experience the events that constitute the regularity (e.g., when a tone and a shock are paired) but also when changes in behavior are assessed (e.g., when the conditioned fear for the tone is measured). Secondly, they often do this in a nonautomatic way, that is to say (among other things) that they are aware of the propositions they form and that they have to make an effort to form those propositions. The effect that a relation in the environment will have on behavior is determined by what the person consciously thinks about that relation (i.e., what proposition about the relation people perceive as being true).
How does all of this relate to classical conditioning? Let’s return to the example of Pavlov’s dog. According to propositional models, presenting an important stimulus such as food will encourage the dog to actively (purposefully) search for predictors or causes of the food. Because the bell is a salient stimulus, the dog will soon consider the possibility that there is a relation between the bell and the food. The fact that the bell is indeed always followed by food offers support for the hypothesis that the arrival of the food can be predicted on the basis of the bell. Behavior can be influenced by the bell-food relation only after the dog has formed a proposition about that relation. More specifically, the proposition specifying that the bell is followed by food will lead to the expectation that food will be delivered after hearing the bell. That expectation results in salivation when hearing the bell.
2.3.2.2 General evaluation of propositional models Can propositional models offer an explanation for available functional knowledge on classical conditioning? We will now discuss a number of important findings.
– Influence of stimulus characteristics and intrinsic relations (see section 2.2.1.2) How fast one discovers a particular relation depends on the properties of the CS, US, and the intrinsic relation between the two. People will be more motivated to discover relations with important USs. One will detect relations that contain striking CSs more quickly. A relation is also easier to detect if there are pre-existing reasons to suspect that such a relation would exist. For example, when people become nauseated, they will tend to think of food as a cause of that nausea because they know from experience that you can become nauseous from food (see Testa, 1974, for a precursor to this idea). Organisms therefore use available and past knowledge to discover new relations in the environment.
– Classical conditioning can also influence involuntary behavior (see section 2.2.2.1) In principle, propositions can influence all kinds of behavior. Contrary to what is sometimes thought (e.g., Shanks, 1990), behavior does not have to be a rational, logical consequence of a proposition about the relation between stimuli. Let us consider the fact that in autoshaping studies, pigeons move toward and peck on the illuminated key when doing so reduces their chance of contacting food. From the perspective of propositional models, it is possible that the propositional belief that the illumination of the key is followed by food, creates a tendency to walk toward the key. Propositional models of learning do not in themselves say anything about why certain propositions have certain effects on behavior. That is, they do not provide a full specified theory of behavior. What they do say is that relations in the environment can have an effect on behavior only after a proposition has been formed about that relation. It is, however, not possible to predict behavior perfectly on the basis of propositional knowledge unless one has a perfect theory of behavior (see Mitchell et al., 2009).
– Contingency awareness is important (see section 2.2.2.3) In contrast to associative models, propositional models can explain why learning usually occurs only after people are aware of the relation. At least in humans, forming and evaluating propositions happens in a nonautomatic and therefore conscious manner. The existence of conditioning without awareness of the CS-US relations would, however, be difficult to explain on the basis of propositional models (but see De Houwer, 2018c).
– Classical conditioning is a general phenomenon that occurs in different organisms (see section 2.2.3) At first sight, this observation seems to contradict propositional models. If all learning is based on propositional processes, then one would have to assume that all animal species are capable of forming and evaluating propositions in a conscious, nonautomatic way. This criticism is correct in the sense that it is unlikely that learning in simple animal species such as snails and bees is based on propositional processes. Yet, there are indications that learning in certain nonhuman animals such as rats is indeed based on the formation and evaluation of propositions (e.g., Blaisdell, Sawa, Leising, & Waldmann, 2006; Beckers, Miller, De Houwer, & Urushihara, 2006; see also Mitchell et al., 2009).
– Secondary tasks have an important impact on classical conditioning (see section 2.2.4) This conclusion fits well with the idea that learning is determined by propositional processes. After all, it takes a lot of effort to formulate and test hypotheses. If you have to invest energy in performing secondary tasks, there is less energy left for forming and evaluating propositions. If, however, attention is focused on the relation between the CS and the US (see Baeyens, Eelen, & Van den Bergh, 1990), then one will more quickly form hypotheses about that relation and evaluate them as being present.
– Conclusion In sum, from the above it appears that propositional models are capable of explaining many aspects of the existing functional knowledge about classical conditioning. They therefore have a high heuristic value. Nevertheless, there are also findings that seem to challenge a propositional account of classical conditioning. Perhaps the most intriguing challenge comes from research on the so-called Perruchet effect. Perruchet (1985) presented a series of trials in which a tone could be followed by an air puff delivered to the eye of the participant. He registered eye blink responses after presentation of the tone and asked participants to rate the extent to which they expected that an air puff would be delivered after hearing the tone. As the number of consecutive trials on which the tone was followed by the air puff increased, the likelihood of an eye blink response increased whereas the expectancy of the air puff after the tone decreased. The expectancy results are in line with the so-called gambler’s fallacy, which refers to the fact that gamblers believe that a loss is more likely after a series of wins, even when the chance of winning is equally high with each gamble. This clear dissociation between conscious expectancies and conditioned eye blink responses seems to suggest that the latter are not based on propositional knowledge of the tone-air puff relation but that they might reflect the operation of a separate, nonpropositional learning system. Although this conclusion is still being debated (e.g., Weidemann, McAndrew, Livesey, & McLaren, 2016), the Perruchet effect is widely regarded as a problem for the idea that all conditioning effects are mediated by propositions.
The predictive value of propositional models is also high. Propositional models have led to a better understanding of the conditions under which conditional contingency is important. More specifically, these models have led to a number of important studies on blocking (see Mitchell et al., 2009, and Boddez, De Houwer, & Beckers, 2017, for an overview). As indicated earlier, blocking refers to the finding that a cue X elicits a less strong CR when AX+ trials are presented together with A+ trials than when only AX+ trials are offered. According to propositional models, this result can be the consequence of causal reasoning. Suppose A and X are seen as two possible causes of the US. The fact that the US is just the same when only A is present than when both A and X are present, suggests that X has no causal influence on the US. After all, causes normally have additive effects. The effect of two causes should therefore be stronger than the effect of one cause in itself. However, we note that A and X together have just the same effect as A alone. We can therefore conclude that X is not a cause of the US. If blocking is indeed the result of this reasoning, then it should only occur if A and X are presented as possible causes of the US. Waldmann and Holyoak (1992) confirmed this prediction. In human contingency learning studies in which participants had to assess the strength of the relation between X and the US, blocking was only established when A and X were described as chemical substances in the blood and the US as a disease that could be caused by A and X. However, they found no blocking if A and X were described as chemical substances in the blood and the US as a disease that can cause A and X. So blocking only occurred if A and X were possible causes of the US but not if the US was a possible cause of A and X.
Even if A and X are considered as possible causes of the US, blocking should only occur if one assumes that the effects of two causes are additive. De Houwer, Beckers, and Glautier (2002) investigated the role of this assumption by providing test subjects with information about the maximum intensity of the US. In the submaximal condition, the subjects were told that the US had an intensity of 10 out of 20 on both the A+ trials and the AX+ trials. Given this information, one can be pretty sure that X is not a cause of the US because the US is just as intense on the A+ trials as on the AX+ trials. If X was a cause, the US should have been stronger on the AX+ trials than on the A+ trials. In this condition blocking was observed: test subjects believed that there was no causal relation between X and the US. In the maximal condition, participants were told that the intensity of the US on the A+ and AX+ trials had a value of 10 out of 10. Because A alone has the maximum effect, it is no longer possible for X to strengthen this effect. The fact that the US is the same on the A+ trials than on the AX+ trials therefore does not say anything about the effect of X on the US. It is possible that X does have an effect on the US, but that this extra effect does not show up because A alone already has the maximum effect. In other words, there is a “ceiling effect” that makes it impossible to draw conclusions about the relation between X and the US. No blocking was found in this condition. Beckers et al. (2006) showed that blocking in rats also depends on how likely it is that the effects of different causes are additive.
On the basis of such findings, most researchers now agree that at least some of the conditioning effects in humans are due to propositional processes (e.g., McLaren et al., 2014). Most researchers, however, continue to assume that associations can also lead to conditioning. Such dual-process models are currently very popular because they can explain more data than single-process models. But if there is more than one learning system, the question arises as to how these different learning processes relate to each other (e.g., when do the different processes guide behavior). Attention is increasingly being paid to this difficult question by proponents of dual-process models (see Mitchell et al., 2009, for a critique of dual-process models).
Despite the successes of propositional models of classical conditioning, their impact on learning research has been relatively limited. In part this can be attributed to the lack of precision in formulating these models. Whereas associative models are often formulated in very precise mathematical terms (e.g., Rescorla & Wagner, 1972; Stout & Miller, 2007), propositional models are often not more than a few verbally formulated ideas about the nature of the mental processes and representations that mediate learning. This lack of precision not only renders it difficult to derive precise predictions from propositional models but also to falsify those models on the basis of empirical evidence. Although these criticisms are valid, it is important to realize that propositional models as they are currently described in the literature, are a class of models that share the core idea that learning is mediated by propositional knowledge. It is indeed difficult to falsify a whole class of models. What is often forgotten, however, is that it is also impossible to falsify the class of all possible associative models of learning (Miller & Escobar, 2001). For every result in the learning literature, it is possible to find an associative model that can explain that result. At the same time, it would also be possible to find another (version of an) associative model that predicts the opposite result. More generally, we believe that the possibility to formalize or refute a theoretical model is not an essential criterion for the quality of the model (De Houwer, 2018c). What is more important is the ability of the model to explain existing functional knowledge (i.e., its heuristic value) and to predict new functional knowledge (i.e., its predictive value). From that perspective, we continue to believe in the value of propositional models of classical conditioning.
Notes
1. Some readers might object to equating outcomes to USs because only the latter are biologically relevant. Note, however, that we are drawing on a broad definition of classical conditioning that concerns only the impact of stimulus pairings on behavior, regardless of the nature of the stimuli or behaviors that are involved.
2. Although the literature on evaluative conditioning is extensive, surprisingly little is said about what “liking” actually entails. Intuitively, most of us have a sense of what it means to like or dislike something, but it is more difficult to delineate which behaviors are evaluative in nature and what makes them evaluative.
3. Conditioning that involves changes in the UR could also be interpreted in terms of enhanced priming. Just as the prime doctor speeds up responses to the target nurse in a lexical decision task (see introduction, note 3), the presentation of a biologically relevant CS (e.g., fake female quail) could be said to change responding to the US (e.g., sexual contact with an actual female). It is possible that under certain conditions, these priming effects become stronger with repetition—that is, the more often the prime (or CS) precedes the target (or US). Interestingly, such enhanced-priming-by-repetition effects have also been observed in research on induced resistance in plants (e.g., Song & Ryu, 2018). From this perspective, CS-US trials do not directly change the UR but they change the extent to which the CS primes the UR.
4. Spontaneous recovery can be seen as an instance of renewal in which time rather than spatial location functions as the context that modulates responding. More specifically, the time period during the CS-US pairings (acquisition phase) can be seen as Context A, the time period during CS-only (extinction phase) as Context B, and the time period after the delay as a new Context C.
5. Often (and also originally), S-R associations were conceived of in physiological terms as involving neural connections between stimulus and response centers in the brain. Given our focus on the mental level of explanation in the psychology of learning, we conceptualize an S-R association in mental terms—that is, as a link between mental representations. Note that the two perspectives overlap if one assumes that stimulus and response centers in the brain are the physiological basis for mental representations of stimuli and responses.
6. Some have argued that complex networks of associations can encode relational information and thus assumptions about the state of events in the world (e.g., Gawronski & Bodenhausen, 2018). However, we do not know any current associative model that captures the intricacies of relational information processing. Moreover, there are good reasons to doubt that associative models could ever achieve this (Hummel, 2010). But if an associative model could be constructed that encodes truth-evaluable relational information, we would consider it to be a propositional model because for us, propositions are about the content of information, not about the structure that is used to store that information. Hence, any representational structure that encodes truth-evaluable relational information would qualify as a proposition (De Houwer, 2018c).