The kind of stimuli which are usually called unpleasant, annoying, or, more technically, aversive are not distinguished by particular physical specifications. Very strong stimuli are often aversive, but some weak stimuli are aversive also. Many aversive stimuli damage tissue or otherwise threaten the well-being of the individual, but this is not always true. Painful stimuli are generally aversive, but not necessarily so—as a counterirritant shows. Stimuli which have acquired their aversive power in the process of conditioning are especially unlikely to possess identifying physical properties. A stimulus is known to be aversive only if its removal is reinforcing. In Chapter V such a stimulus was called a negative reinforcer. We define both positive and negative reinforcers in terms of the strengthening of a response. What happens when a positive reinforcer is withdrawn or a negative reinforcer presented will not be considered until the following chapter.
Behavior which is followed by the withdrawal of an aversive stimulus is called escape. We weaken an aversive noise by putting fingers in our ears, by moving away from the source, by closing intervening doors or windows, by stopping it at the source, and so on. Similarly, we escape from a bright light by shutting our eyes, turning our head away, or turning off the light. We cannot say that these responses are positively reinforced with “freedom from” noise, light, and so on, since it is the change from one situation to another which is effective, and this is the reduction of a condition prevailing before reinforcement.
In the laboratory we condition a rat to press a lever by reducing the intensity of a light when it does so. The level of illumination is critical. A weak light may be ineffective and a very strong light may lead to aversive behavior acquired earlier in the history of the rat, such as shutting the eyes or covering the head with other parts of the body. A loud noise or a light shock delivered through the floor of the box is less likely to evoke previously established aversive behavior, but the use of such stimuli is complicated by other factors. Aversive stimuli elicit reflexes and generate emotional predispositions which often interfere with the operant to be strengthened. It is then difficult to observe the effect of negative reinforcement alone.
Aversive stimuli are often used, both in the laboratory and in the practical control of behavior, because of the immediacy of the result. When we present an aversive stimulus, any behavior which has previously been conditioned by the withdrawal of the stimulus immediately follows, and the possibility of conditioning other behavior is immediately provided. The presentation of the aversive stimulus therefore resembles a sudden increase in deprivation (Chapter IX); but since deprivation and satiation differ in many respects from the presentation or removal of an aversive stimulus, it is advisable to consider the two kinds of operations separately. We study aversive behavior in accordance with our definition: by presenting an aversive stimulus, we create the possibility of reinforcing a response by withdrawing the stimulus. When conditioning has already taken place, the aversive stimulus provides an immediate mode of control.
Hunger pangs are a possible source of confusion between deprivation and aversive stimulation. Since hunger is the commonest drive, we have tended to model our formulation of all drives upon it. But we have seen that pangs are not representative of drives in general and that, even in the case of hunger, they call for a separate formulation. Insofar as one eats in order to reduce hunger pangs, the behavior is aversive. Whether pangs ever lead to eating before negative reinforcement has taken place would be difficult to determine, since pangs are produced by the very conditions which produce a heightened probability of eating regardless of the presence or absence of pangs. It is possible, however, to separate the producing of pangs from the increase in likelihood that eating will take place. When stimulation which resembles a hunger pang arises from other sources—for example, an inflammation—aversive eating may take place without deprivation. On the other hand, when we drink water, chew an indigestible substance, or take certain drugs in order to reduce hunger pangs, we are emitting behavior which otherwise may not vary with food deprivation. Similarly, although an individual may engage in certain sex practices because they reduce the time wasted in other sexual preoccupations, it does not follow that this result or the reduction of any other aversive consequence is essential to the normal variation in sexual behavior with deprivation or satiation.
Just as we did not define a positive reinforcer as pleasant or satisfying, so in defining a negative reinforcer in terms of its power to reinforce when withdrawn we do not assert that the stimulus is unpleasant or annoying. It would be as difficult to show that the reinforcing power of an aversive stimulus is due to its unpleasantness as to show that the reinforcing power of a positive reinforcer is due to its pleasantness. The arguments given in Chapter V on this point could be repeated step by step for the negative case. There is also a parallel explanation in terms of biological significance. It is not difficult to show that an organism which is reinforced by the withdrawal of certain conditions should have an advantage in natural selection.
Conditioned aversive stimuli. The formula of stimulus substitution applies to the function of negative reinforcement. Neutral events which accompany or precede established negative reinforcements become negatively reinforcing. Thus we move to escape from an annoying or offensive person even though he is not annoying or offensive at the moment. The so-called cures for smoking and drinking mentioned in Chapter IV follow this formula. By pairing the taste of tobacco or alcohol with a condition of nausea, the aversive behavior appropriate to nausea, perhaps including vomiting, is transferred to the tobacco or alcohol.
We use negative reinforcement in several different ways. An aversive stimulus which has already been withdrawn to reinforce a desired operant offers, as we have seen, an immediate mode of control. One boy holds another on the ground until the victim cries “Uncle.” An arm is twisted until a gun is dropped. A horse is whipped until it moves at a given speed. We use conditioned aversive stimuli in the same way—when, for example, we “shame” someone into acting. The boy who does not dive from the high board is called a sissy; and he can escape from this conditioned verbal stimulus only by diving. His companions present the stimulus to increase the probability that he will dive. “Daring” is a similar practice. (The inverse case, as we shall see in the next chapter, is to prevent behavior from occurring by branding it shameful. Shameful behavior is behavior which one may be “shamed out of.” Escape from the aversive shame is achieved by not engaging in the behavior or, more obviously, by engaging in conspicuously incompatible behavior.)
We extend the effectiveness of the technique when we condition behavior so that future aversive stimuli will have an effect. We may plan to present these stimuli upon later occasions, or we may simply prepare for them whenever they may occur. Conditioning is an important stage in the exploiting of aversive control in ethics, religion, and government, as we shall see in Section V.
We also condition aversive stimuli in order to provide for negative reinforcement. A neutral stimulus which is likely to occur on some later occasion is made aversive by being paired with aversive stimuli. Escape is then automatically reinforced. For example, the spread of venereal disease is to some extent controlled by educational programs which provide for the future reinforcement of aversive behavior to prostitutes or “easy pickups.” Descriptions or pictures of such people are paired with aversive information about venereal disease. One result is a strong emotional response at the sight of a prostitute which may be effective from the point of view of the educational program by being incompatible with sexual behavior: the individual may be too frightened to participate. To this extent the effect is emotional rather than aversive. Another object of such a program, however, is to guarantee the reinforcement of aversive behavior. When the individual looks away, turns away, or walks away from the prostitute, his behavior will be reinforced by the reduction of a conditioned aversive stimulus.
An important example of this use of aversive conditioning is the practice of branding an act wrong or sinful. Any behavior which reduces the stimulation arising from the early stages of such an act is then negatively reinforced. A single pairing of two stimuli may be sufficient to transfer aversive power, and a conditioned reinforcer may continue to be effective long after the basic unconditioned reinforcers have disappeared from the environment. Many problems in psychotherapy arise from the strength and duration of this effect, as we shall see in Chapter XXIV.
The withdrawal of a positive reinforcer has by definition the same effect as the presentation of a negative. Taking away privileges is not very different from establishing aversive conditions. We occasionally remove a positive reinforcer for practical purposes. What is removed is, more precisely, a conditioned positive reinforcer—a discriminative stimulus or, in other words, the occasion for successful action. There are several subtle distinctions here which are perhaps more important for the theory of behavior than for its practical control. Suppose we have deprived a man of permission to leave a military camp until a certain task has been performed, and suppose that upon past occasions the performance of similar tasks has been followed by the restoration of this privilege. Have we generated a state of deprivation, in which behavior which has been reinforced by the return of privileges will be strong, or have we presented an aversive condition from which the individual can escape only by performing the required task? It is possible, of course, that we have done both. Practically, the distinction may appear to be of little importance, but certain collateral results depend upon the extent to which each is involved.
Escaping from an aversive condition is clearly not the same as avoiding it, since the aversive condition which is avoided does not directly affect the organism. Although avoidance suggests that behavior may be influenced by an event which does not occur, we may account for the effect without violating any fundamental principle of science with the concept of conditioned negative reinforcement. In avoidance the conditioned and unconditioned aversive stimuli are separated by an appreciable interval of time. The required temporal relation is commonly encountered in nature. A rapidly approaching object precedes painful contact. The sputter of the fuse precedes the explosion of the firecracker. The sound of the dentist’s drill precedes painful stimulation in the tooth. The interval separating the two stimuli may be definitely fixed, or it may vary widely. In any case, the individual comes to execute behavior which prevents the occurrence or reduces the magnitude of the second stimulus. He dodges the object, puts his fingers in his ears to soften the sound of the explosion, and jerks his head away from the drill. Why?
When stimuli occur in this order, the first stimulus becomes a conditioned negative reinforcer, and henceforth any action which reduces it is strengthened through operant conditioning. When we avoid the painful stimulation of the tooth, we merely escape from the sound of the drill. That the behavior of avoidance appears to be “directed” toward a future event may be explained as in operant behavior in general: it is always past occurrences of conditioned negative reinforcers and past instances of their reduction which are responsible for the probability of the escape response. The fact that the future event does not occur when the behavior is emitted would raise a puzzling issue if the behavior did in fact continue in strength. But if an occasion for avoidance behavior arises often enough, the conditioned aversive situation grows progressively weaker. The behavior is no longer reinforced, and eventually not emitted. When this happens, the primary negative reinforcer is received. A single instance may suffice to recondition the reinforcing power of the earlier stimulus. Thus when certain visual stimuli generated by a rapidly approaching object are followed by injury, any behavior which converts the stimuli into more harmless forms will be strengthened. Moving out of the way, dodging, and putting up a guard are cases in point. By virtue of these responses the individual is successful in avoiding injury, but he is reinforced only in escaping from the conditioned aversive stimuli which we call the “threat” of injury. If injury is always avoided, the threat grows weaker, and the behavior is less and less strongly reinforced. Eventually a response is not made, an injury is received, and the visual pattern is re-established as a negative reinforcer. Similarly, if the stimulation incidental to the eating of a particular food always precedes a severe allergic headache, it may become aversive. As a result the food is not eaten, the headaches do not occur, and the original conditioning of the negative reinforcer suffers extinction. Eventually the food is no longer aversive. When it is eaten again, another headache occurs, the conditioned negative reinforcer is again established, and another cycle is begun. The “absence of a headache” has had an effect upon behavior only in furthering the extinction of the conditioned aversive stimulus.
The practical use of a “threat” is familiar to everyone. The bandit threatens his victim by creating a condition which has preceded physical injury, and the victim reduces this threat by turning over his pocketbook. Escape—running away—may also be highly probable, but it is only the behavior with respect to the pocketbook which fits the present formula. A threat is something more than daring or shaming because of the special temporal relation between conditioned and unconditioned negative reinforcers. Nothing else happens if a dare is not taken; the aversive condition simply continues.
Any stimulus which consistently precedes the aversive withdrawal of a positive reinforcer may come to act as a conditioned negative reinforcer. We avoid an aversive condition when we act to reduce any indication that an entertaining program will be brought to an end or that any advantage in a competitive affair will be reduced or that we shall lose the affection or love or services of someone who is important to us. The behavior reinforced by reducing such threats will not necessarily be the same as that which is positively reinforced by the entertaining program, by the advantage, or by love, affection, or services.
A stimulus which characteristically precedes a strong negative reinforcer has a far-reaching effect. It evokes behavior which has been conditioned by the reduction of similar threats and also elicits strong emotional responses. The bandit’s victim not only turns over his pocketbook and displays a high probability of running away, he also undergoes a violent emotional reaction which is characteristic of all stimuli leading to avoidance behavior. One who has been severely seasick will tend to escape from conditioned aversive stimuli which occur while planning a trip, while going up the gang-plank, and so on—for example, he will tend to cancel his trip or turn and rush off the ship. He will also exhibit strong conditioned reflexes transferred from the original stimulation of the ship in motion. Some of these may be simple gastric responses which we should not call emotional. Others may be of the sort commonly seen in fear. Operant behavior will be also markedly changed. The individual may seem “preoccupied”—which may mean nothing more than that he is not normally occupied. He may find it impossible to engage in normal conversation or to attend to the simplest practical affairs. He may speak curtly and show none of his usual interests. These are emotional effects upon probability which might have been considered in Chapter X. They can occur, however, only when a stimulus characteristically precedes an aversive stimulus by an interval of time sufficiently great to permit behavioral changes to be observed. The condition which results is usually called anxiety.
Almost every strong aversive stimulus is preceded by a characteristic stimulus which may come to generate anxiety. Contingencies of this sort are arranged in the practical control of behavior, often in connection with punishment. Although the biological advantage of avoidance is obvious, the emotional pattern of anxiety appears to serve no useful purpose. It interferes with the normal behavior of the individual and may even disorganize avoidance behavior which would otherwise be effective in dealing with the circumstances. For this reason anxiety is an important problem in psychotherapy, as we shall see in Chapter XXIV. In the design of controlling techniques the possibility of generating anxiety as an unfortunate by-product must constantly be kept in mind.
Since conditioning may take place as the result of one pairing of stimuli, a single aversive event may bring a condition of anxiety under the control of incidental stimuli. The sudden death of a close friend, for example, is sometimes followed by a sustained depression which may be verbalized as a feeling that “something is going to happen,” as a “feeling of doom,” and so on. It is hard to deal with such a case. When we say that a death was sudden or occurred without warning, we mean that no prior stimulus was particularly associated with it. The stimuli which received the force of the conditioning were therefore the undistinguished elements of daily life. It is not likely that there are any successful forms of escape appropriate to these stimuli, although other forms of escape may, through induction, be strengthened. Conditioned emotional reflexes, as well as conditioned emotional predispositions, may be almost constantly activated. In the case of an “expected” death—for example, the death of someone who has been ill for a long time—the event may be equally aversive, but the anxiety is conditioned to the specific stimuli which precede it. Anxiety is not so likely to arise again unless these stimuli are reinstated—for example, through the illness of another friend.
Although the emotional aspect of anxiety may be distinguished from the conditioned aversive effect responsible for avoidance behavior, it is possible that the emotion is also aversive. Avoidance responses may be interpreted as in part an escape from the emotional components of anxiety. Thus we avoid the dentist’s office, not only because it precedes painful stimulation and is therefore a negative reinforcer, but because, having preceded such stimulation, it arouses a complex emotional condition which is also aversive. The total effect may be extremely powerful. A problem of great military importance is the behavior of avoiding battle. Malingering, desertion, or a “nervous breakdown” may reach a very high probability. Successful preparation of the fighting man requires a clear understanding of the effect of the stimuli which precede the more aversive stimuli of combat. The man may be avoiding, not merely battle, but his own reactions of anxiety.
A counterpart of anxiety arises when a stimulus precedes a positive reinforcement by an appreciable interval. If an envelope received through the mail contains bad news, a similar envelope received later will, before it is opened, generate the anxiety just described. But envelopes also contain good news—perhaps a check or the offer of a good job. Here the avoidance behavior strengthened by bad news—turning away from the mail box, throwing the envelope down unopened, losing the envelope before it is opened, and so on—has its parallel in the increased probability of looking in the box, opening the letter in haste, and so on. Emotional reflexes in response to the unopened envelope will be appropriate to bad news in the one case and to good news in the other. Instead of responses commonly observed in grief, sorrow, or fear, we may observe responses characteristic of elation or joy. Emotional predispositions also stand in the same polar opposition: the general depression of activity in the one case is matched by a general heightening of activity in the other. Instead of growing silent and reserved, our subject speaks to everyone, reacts in an exaggerated fashion, walks faster and seemingly more lightly, and so on. This is particularly obvious in the behavior of young children—for example, on the eve of a holiday or festival.
The effect of stimuli which characteristically precede positive reinforcement may be chronic in a world in which “good” things frequently happen. It is not seen in the clinic because it is not troublesome. Anxiety, which is chronic in a world in which “bad” things frequently happen, has resulting disadvantages both to the individual and to society.
Anxiety, as a special case of emotion, should be interpreted with the usual caution. When we speak of the effects of anxiety, we imply that the state itself is a cause, but so far as we are concerned here, the term merely classifies behavior. It indicates a set of emotional predispositions attributed to a special kind of circumstance. Any therapeutic attempt to reduce the “effects of anxiety” must operate upon these circumstances, not upon any intervening state. The middle term is of no functional significance, either in a theoretical analysis or in the practical control of behavior.