One of Ivan Pavlov’s dogs, showing container for collecting saliva.
Stimulus and Response
One of Ivan Pavlov’s dogs, showing container for collecting saliva.
In Russia, Ivan Pavlov discovered conditioning. In America, John Watson proposed that psychology should be the science of behavior; he went on to influence advertising, not to give information, but to offer stimuli designed to cause consumers to respond by making purchases. B. F. Skinner further developed behaviorism. In a Skinner box, a rat or pigeon that exhibited some piece of behavior that was followed by a reinforcement such as delivery of food, increased its probability of producing that behavior. Are we humans like dogs, rats, and pigeons, as behaviorists describe, or might we have minds, and make decisions?
The first clues that it was possible to study learning in a way that was both simple and significant came from an unlikely source: experiments by a Russian physiologist on digestion in dogs. The researcher was Ivan Pavlov. He performed surgeries on dogs to move the duct of a salivary gland from the inside to the outside of the mouth and connect it to a tube, as shown in the chapter opening photo, so that salivation could be observed and measured.
Pavlov discovered that saliva started to flow not just when food entered a dog’s mouth, but when the dog saw food. This is a useful mechanism because salivary juices, which are important to digestion, can start working immediately when food enters the mouth. Pavlov’s most famous finding was that, when on several occasions he showed the dog food and at the same time sounded a buzzer, saliva would start to flow when the buzzer was later sounded alone, without food being offered. The original reflex, in which the sight of food caused salivation, had a learned reflex grafted onto it.
With this discovery, the terms “stimulus” and “response” first introduced by Descartes in his description of how the nervous system worked to produce reflexes, entered psychology. Many psychologists use the term “stimulus” today to mean a pattern of sensory events that can affect the nervous system, and they use the term “response” to mean a movement of muscles or glands in a particular circumstance.
Pavlov’s dogs learned that the buzzer meant that food was about to arrive. This kind of learning is called “conditioning.” The new stimulus is the conditioned stimulus. The dog has learned what event in the environment signals that something important is about to occur. The idea was that the nervous system becomes rewired, so that now the signal causes the response. Since Pavlov’s time, there has been intense research to discover what kinds of rewiring in the nervous system happen during conditioning.
Ivan Pavlov was born in 1849 in the small town of Ryazan, 250 miles southeast of Moscow.1 His father was a priest who inspired his sons with a love of learning. In his youth, Ivan was also strongly influenced by his godfather, abbot of a nearby monastery, who was known for his simple life and devotion to duty. Ivan Pavlov seems to have taken on these traits. He lived with an almost other-worldly concern for his work. Although at first he studied theology, Pavlov came to believe in science, and took up physiology. In 1861, he married Sara (Seraphima Karchevskaya). Their early years together were difficult because they had almost no money. Sara’s first pregnancy ended in a miscarriage, and the couple’s first child died while still very young, but they later had three children who survived to adulthood. From 1890, when he was invited to direct the Department of Physiology in the Institute of Experimental Medicine in St. Petersburg, Pavlov’s career thrived, but he did not become interested in psychology until ten years later when, at the age of fifty, he discovered conditioning. In 1904, he was awarded the very first Nobel Prize in physiology.
Watson and Behaviorism
Ivan Pavlov had laid the foundation of behaviorism. For its next phase we travel from St. Petersburg in Russia to Baltimore, Maryland in the United States. Here a psychologist called John B. Watson became the protagonist.2
Influentially and independently of Pavlov’s work (which he seemed not to have known about at that time), in 1913 Watson wrote an article for the prestigious journal, Psychological Review, entitled “Psychology as the behaviorist views it.” In it, he made three points. First he argued that psychology had lost its way with the idea that its chief subject matter was consciousness and that its chief method was introspection. He said this had yielded nothing of importance in nearly fifty years. Watson’s second point was that, as the proper alternative to introspection, psychology should use the observable data of behavior. In this way, the study of humans would become objective, and no different from the study of animals. Indeed, he said, humans should be regarded as animals from the point of view of psychology. Watson’s third point was that, by observing and analyzing behavior, psychology would place itself among the sciences.
Following the establishment of psychological experimentation by Wilhelm Wundt, and following Wundt’s principle that psychology should be a science, behaviorists tended to take physics as their model, with its theories such as gravitation, and with its laws such as Newton’s laws of motion. Psychology, too, they said, should have its theories and laws.
All three of the points that Watson stressed in his article continue to be influential. Most psychologists now regard introspection as of minor interest. Many think that behavior, along with physiological observations, constitute the principal data of psychology. Most remain keen that psychology is a science.
It came to seem for the behaviorists that the best domain within which to search for psychological laws was learning. Learning by conditioning, in the way that Pavlov had demonstrated, came to be regarded as a fundamental mechanism. We humans, it came to be thought, are characterized not by instincts like the lower animals, but by our superior ability to learn. Unlike lower animals, we guide our lives by what we learn. The behaviorist idea included the promise that if we could find out how fundamental kinds of learning occur, we would discover how these processes are elaborated in humans. This would reveal the secret of our success, help us cure what goes wrong for human beings, and help us to become even more successful.
In 1919, a nineteen-year-old student, Rosalie Rayner, became John Watson’s research assistant. It was with her that he performed a famous experiment on an eleven-month-old boy, who came to be known as Little Albert.
Albert was found to be curious about a friendly white rat, as you may see in figure 9. In the experiment, when the boy reached toward the rat, Watson banged an iron bar with a hammer so loudly that it startled the child. Albert withdrew his hand and whimpered. Over the next weeks, Watson and Rayner showed the rat to Albert several more times, and paired the boy’s attempts at reaching toward it with the loud sound of striking the iron bar. Albert stopped reaching toward the rat. In those days there were no ethical considerations about what kinds of experimentation might be regarded as acceptable.
Figure 9. Rosalie Rayner holds Little Albert, who takes interest in a white rat that is being shown to him by John Watson. Source: “Rosalie Rayner holds Little Albert.” http://www.nscpsychology.com/blog/who-was-little-albert. Archives of the History of American Psychology, The Drs. Nicholas and Dorothy Cummings Center for the History of Psychology, The University of Akron.
The experiment by Watson and Rayner, with Little Albert, published in 1920, began the American study of learning as conditioning. They called Albert’s change of behavior toward the white rat a conditioned emotional response. From this time, they started to take an interest in the conditioning that occurs in the development of children, and how this can shape their behavior. They also realized that conditioned emotional responses could be the basis of anxiety disorders. This idea that long-lasting anxieties can have been set up in this way is central today in behavior therapy and cognitive-behavior therapy.
As well as being the year of publication of a study that would make him famous, 1920 was the year in which John Watson’s academic career ended. He had started an affair with Rosalie Rayner. Watson’s wife, Mary, discovered letters between him and Rosalie, and the letters found their way to the president of Johns Hopkins University, who summoned Watson and required him to resign on the spot. Despite the influence he had won in psychology, Watson was never offered another university position. In describing the concept of “response” in his popular book Behaviorism, Watson would write: “Under sex excitement the male may go to any length to capture a willing female.”3 At the end of 1920, Mary and John Watson divorced, and at the beginning of 1921 Rosalie Rayner became the second Mrs. Watson. Along with a new wife, Watson needed a new career. He found one in advertising, at the J. Walter Thompson Company, where he worked until 1935.
Watson was responsible for bringing psychology into the advertising business. As David Cohen put it: “Watson … shifted the focus of American advertising. The potential buyer was a kind of machine. Provide the right stimulus and he will oblige with the right reaction, digging deep into his pocket.”4
The ingenious Watson took to advertising as an opportunity for a new kind of learning experiment. One had to choose stimuli with a view to eliciting certain kinds of responses. One successful campaign was for Maxwell House coffee. Cohen describes how Watson “decided to appeal to the snob in the customer.”5 Under Watson’s direction, Maxwell House advertisements pictured splendid historical scenes. At high society balls, what did the butler bring on a tray? Maxwell House. In elegant Southern mansions, beautiful women asked for Maxwell House. Cohen relates how in 1928 the New Yorker observed that Watson had “conditioned housewives and commuters into all sorts of prejudices about coffee.” Cohen continues:
It was not a drink that the advertisements were selling, it was a dream … Drink Maxwell House and project yourself into a world of superb elegance and gorgeous glamour. Sip Maxwell House. Slip into the dream.6
Watson’s third career was as a writer of popular books. In Behaviorism, Watson described his experiments on Little Albert and introduced, too, a series of other experiments on children to show, in perhaps the first examples of behavior therapy, how fears could be removed by methods he called “reconditioning” or “unconditioning.”
Watson was successful as a popular writer because what he said was outspoken and often contrary to received belief. He became an authority on childcare. He argued that too much mother-love was bad for children. There was often something missing from the lives of parents, he said. What they did in kissing and coddling their children was largely for their own gratification. The effect of this was to make the children hopelessly dependent, and it destined them for neurosis in adulthood. He proposed that parents should regard their children as experimental subjects and work out what enabled them to learn whatever might be necessary for purposes the parents might have. All this, of course, both enraged and engaged the public.
Skinner and His Boxes
In the middle of the twentieth century, if you opened an American textbook on psychology, you were likely to read that “Psychology is the science of behavior.” B. F. Skinner was a significant influence on this view. He promoted behaviorism so effectively that one might have thought that it was all there was to psychology.7
Among Skinner’s contributions was the invention of the Skinner box: a cage in which a rat or a pigeon could be rewarded for some particular piece of behavior. A Skinner box designed for rats contains a food hopper into which pellets of food can be delivered. In the box, too, is a lever which, when it is pressed, causes a food pellet to be delivered. When a hungry rat is first put into the box, it tends to wander about, sniff here, climb up there, until by accident it stands on the lever. Then, with a click, a pellet of food arrives in the hopper. The rat finds it and eats it, then tends to hover around the food hopper, then wander about the box some more until, perhaps accidentally again, it stands on the lever. Another click, another food pellet. If you watch sequences of this kind, you see the rat spend more time near the lever, and then go to sniff around the food hopper, go back toward the lever, perhaps climb on it so that another click is heard, and another pellet arrives. As the rat starts to go back and forth between lever and food hopper, it becomes more efficient until it presses, goes to the food hopper, presses again and so on, to produce for itself a steady supply of food pellets. You might then say to yourself: “Now it’s got the idea.”
According to Skinner, you’d be wrong. In his behaviorism, anything like an idea was prohibited. The proper way would be to describe the observed behavior. You might say that the rat’s behavior had been shaped to press the lever. You might describe the delivery of each food pellet into the food hopper. You wouldn’t, however, call this a reward. You would call it a positive reinforcement. If you took a class with Professor Skinner and in a test you wrote “reward” instead of “positive reinforcement,” you would not be reinforced. You would lose points.
Most important of all, according to Skinner’s psychology, you would describe relationships between responses and reinforcements. So, you could say: “The rat is on a schedule of reinforcement such that when it presses the lever a food pellet is delivered.” Whereas Pavlov’s kind of learning came to be called “classical conditioning,” Skinner called his kind of learning “operant conditioning,” where “operant” meant an operation on the environment. Rather than responding to the pairing of a signal and an event that was significant for the animal (as in Pavlov’s experiments), operant responses had to be emitted. Learning consisted of increasing the probability of emitting a particular response to which the experimenter had chosen to give positive reinforcement.
Skinner started to point out the parallel between operants selected by reinforcement and Darwin’s idea of evolution by means of heritable variations being selected. This was a shrewd move. If natural selection enabled species to evolve across generations, learning by reinforcement enabled the individual to evolve within a generation. Skinner thus elevated his idea of positive reinforcement as the shaping of organisms to contingencies of the environment to a grand principle. He insisted that what he was doing was Science, with a capital “S,” and he tended to belittle the work of other psychologists.
The principle of trying to shape children’s behavior by positive reinforcement became important especially in schools and in therapy for children who were delayed or disordered in their development. Punishment, the delivery of some painful event after a response, does diminish the probability of that response, but it’s not effective if it is delayed and, more importantly, Skinner found that it tends to disrupt behavior, so it is to be avoided if possible. If undesirable responses occur, instead of punishing children who emit them, these children are to be put on a timeout, comparable to when, for a rat in a Skinner box, the mechanism by which the lever causes food to be delivered is turned off. During a time-out period, children are not responded to at all. Instead, after the time-out interval, the teacher or therapist waits until the child does something thought to be appropriate and then delivers a positive reinforcement: a word of praise or something else the child is motivated to acquire.
Some principles based on Skinner’s ideas have become useful. For instance, if parents want their baby to sleep at night on its own, the principle of not responding to crying when the baby goes to bed is critical. If a child is put to sleep at night, and the parent leaves the room, the child is likely to cry. Distressed by the crying, the parent tends to come back to pick up the infant, then hug and soothe it until it stops crying. Perhaps the parent takes the child into the parental bed. What the child learns is that crying is reinforced; when the child is put down to sleep, the crying increases. Many parents have despaired in this situation. The answer is to stop reinforcing the crying behavior. It is anguishing for parents to hear their infant howl for thirty minutes, which is likely to happen on the first night it has to sleep on its own. But if the child is left, it is likely to fall asleep after crying. Next night, it may cry for twenty minutes. The following night, it cries perhaps just for five minutes. The next night it may fall asleep without crying. If we weren’t describing behavior in Skinner’s terms, we might say that in this way the child finds its own resources for comforting itself and going to sleep.
Despite great professional success, Skinner aroused hostility. One target was his use, for his younger daughter, Deborah, of a temperature-controlled crib that was padded and comfortable. He called it the “baby tender.” The idea was that a baby in the tender was in a completely safe environment, without aversive stimuli. If the baby cried, this response was not reinforced. Skinner saw his invention as a labor-saving device for mothers. In 1945, he published an article on this invention in the Ladies’ Home Journal. He wrote that the device offered mothers better opportunities to schedule naps and feedings and that, in the tender, babies could be more active and mothers were freer to love their babies who, because they were happier, were indeed more lovable. There was no suggestion that children were to be kept in the tender all the time. The tender’s function wasn’t much different from that of a playpen, a device that had been in use since the beginning of the twentieth century. But an editor at the Ladies’ Home Journal, perhaps intending to hint that Skinner was a bit of a crackpot, gave his article the title, “Baby in a box.”8 This was the second box Skinner had invented and, because some people thought he was experimenting on his children in a Skinner box, criticism became intense. In 2004, Lauren Slater published a book entitled Opening Skinner’s Box, in which she recounted rumors that Deborah was irreparably damaged from being experimented upon as a baby. When the book was published, Deborah, who was an artist at the time, married and living in London, happened to read a review of it, and wrote an article for The Guardian newspaper to say that no experimentation and no ill-effects had occurred. Skinner seems to have been a kind parent to both his daughters.
Skinner’s biggest challenge to conventional thinking was to propose that mind and free will are both wrongheaded, both dangerous. He pointed out that we humans continue to go to war with each other, and that social inequality is pervasive. The conclusion that individuals are autonomous beings has not led to happy, well-regulated societies, and not led to much care for our environment. What we should do, Skinner argued, is to live in a simpler way, as he had outlined in his novel, Walden Two. He proposed that the environment should be carefully engineered to ensure a flow of positive reinforcements.
One might say that Skinner was not an innovator who made striking discoveries and challenging pronouncements about how to manage people and societies. One could say there was merely a history of environmental contingencies that reinforced certain responses in an organism called B. F. Skinner. But that would be unfair.
In 1971, Skinner appeared on the cover of Time Magazine. That year marked, perhaps, the peak of his influence. What are the lasting effects of Skinner’s work? He seems not to have persuaded us that we can’t make decisions about our lives. Among lasting effects of behaviorism are that principles of learning-by-reinforcement have become part of the language of psychology, in rats, in pigeons, and even in us humans. Nowadays, however, psychologists also talk about rewards, and even about whether one can experience pleasure in accomplishing our goals. This issue has become critical, for instance, in understanding addictions. Schoolteachers and children’s therapists have rightly become wary of punishments, and think about how to offer positive reinforcements to children for their learning and for behavior that shows success in achievement, or demonstrates kindness and consideration for others.
What was discovered by Skinner? He showed, indeed, that learning often does occur by positive reinforcement, and that learned connections between stimuli and responses are determined by contingencies, and by whether responses are reinforced by certain events in the environment. He established too that if every response of a particular kind—a rat pressing a lever in a Skinner box, or a child correctly answering a teacher’s questions at school—is positively reinforced, then the schedule of reinforcement can be called continuous, and the probability of that response is increased. If positive reinforcements are discontinued then, after a while, responses are no longer emitted: the probability of response is diminished. Don’t even think that the rat or the child may have given up. One needs to say the response was extinguished. If positive reinforcements are re-established, re-learning is quicker than for the original learning. If a response is positively reinforced only intermittently, the schedule is called partial reinforcement. Here, responding becomes more rapid than for continuous reinforcement, and when reinforcement is discontinued, extinction takes longer. Negative reinforcement is the removal of aversive stimuli. Application of aversive stimuli, usually called punishment, is unreliable for learning and is to be avoided, since it disrupts behavior and generalizes too widely. There are a few more wrinkles, but this paragraph might be regarded as about 70 percent of the principles and findings of Skinner’s behaviorism.
It was thought by Skinnerian behaviorists that the operations of 86 billion neurons, in processes of human perception, learning, emotion, development, and imagination, could be summed up in a paragraph like the previous one. Some processes of reinforcement remain important, for instance in special education and behavior therapy, but behaviorism as a movement drained psychology of meaning. It seemed almost like a disease. As discussed in the next chapter, the remedy would come from an unlikely source—from linguistics. But further into the future, as we see in chapter 8, learning processes not unlike those outlined by Skinner would make a comeback in artificial intelligence.
