THE FACE CAN be a valuable source for the lie catcher, because it can lie and tell the truth and often does both at the same time. The face often contains two messages—what the liar wants to show and what the liar wants to conceal. Some expressions serve the lie, providing untrue information. Yet others betray the lie because they look false, and feelings sometimes leak despite efforts to conceal them. False but convincing expressions may occur one moment and concealed expressions leak the very next moment. It is even possible for the felt and the false to be shown in different parts of the face within a single blend expression. I believe that the reason most people fail to detect lies from the face is that they don’t know how to sort out the felt from the false expressions.
The true, felt expressions of emotion occur because facial actions can be produced involuntarily, without thought or intention. The false ones happen because there is voluntary control over the face, allowing people to interfere with the felt and assume the false. The face is a dual system, including expressions that are deliberately chosen and those that occur spontaneously, sometimes without the person even aware of what emerges on his own face. There is a ground in between the voluntary and the involuntary occupied by expressions that were once learned but come to operate automatically without choice, or even despite choice, and typically without awareness. Facial mannerisms and ingrained habits that dictate the management of certain expressions, such as being unable to show anger toward authority figures, are examples. My concern here, however, is with the voluntary, deliberate, false expressions, recruited as part of an effort to mislead, and the involuntary, spontaneous, emotional expressions that may occasionally leak feelings despite a liar’s attempt to conceal them.
Studies of patients with different kinds of brain damage dramatically show that the voluntary and the involuntary expressions involve different parts of the brain. Patients who have damage to one part of the brain, involving what is called the pyramidal neural systems, are unable to smile if asked to do so but will smile when they hear a joke or otherwise enjoy themselves. The pattern is reversed for patients who have suffered damage to another part of the brain, involving nonpyramidal systems. They can produce a voluntary smile but are blank-faced when enjoying themselves. Patients with pyramidal system damage—those who cannot make expressions deliberately—should not be able to lie facially, for they should not be able to inhibit or put on false expressions. Patients with nonpyramidal system damage—those who do not show expressions when they do feel emotion—should be very good facial liars since they won’t have to inhibit any true, felt emotional expressions.1
The involuntary facial expressions of emotion are the product of evolution. Many human expressions are the same as those seen on the faces of other primates. Some of the facial expressions of emotion—at least those indicating happiness, fear, anger, disgust, sadness, and distress, and perhaps other emotions—are universal, the same for all people regardless of age, sex, race, or culture.2 These facial expressions are the richest source of information about emotions, revealing subtle nuances in momentary feelings. The face can reveal the particulars of emotional experience that only the poet can capture in words. The face can show:
• which emotion is felt—anger, fear, sadness, disgust, distress, happiness, contentment, excitement, surprise, and contempt can all be conveyed by distinctive expressions;
• whether two emotions are blended together—often two emotions are felt and the face registers elements of each;
• the strength of the felt emotion—each emotion can vary in intensity, from annoyance to rage, apprehension to terror, etc.
But, as I said, the face is not just an involuntary emotional signal system. Within the first years of life children learn to control some of these facial expressions, concealing true feelings and falsifying expressions of emotions not felt. Parents teach their children to control their expressions by example and, more directly, with statements such as: “Don’t you give me that angry look”; “Look happy now when your aunt gives you a present”; “Don’t look so bored.” As they grow up people learn display rules so well that they become deeply ingrained habits. After a time many display rules for the management of emotional expression come to operate automatically, modulating expression without choice or even awareness. Even when people become aware of their display rules, it is not always possible, and certainly never easy, to stop following them. Once any habit becomes established, operating automatically, not requiring awareness, it is hard to undo. I believe that those habits involving the management of emotion—display rules—may be the most difficult of all to break.
It is display rules, some of which differ from culture to culture, that are responsible for the traveler’s impression that facial expressions are not universal. I found that when Japanese watched emotion-arousing films their expressions were no different than those shown by Americans, if the Japanese were alone. When another person was present while they watched the films, a person in authority, the Japanese much more than most Americans followed display rules that led them to mask any expression of negative emotions with a polite smile.3
In addition to these automatically operating habitual controls of facial expressions, people can and do choose deliberately, quite consciously, to censor the expression of their true feelings or falsify the expression of an emotion not felt. Most people succeed in some of their facial deceits. Nearly everyone can remember being totally misled by someone’s expression. Yet, almost everyone has also had the opposite experience, realizing that someone’s words were false by the look that passed across the face. What couple cannot remember an instance in which one of them saw on the other’s face an emotion (usually anger or fear), that the other was unaware of showing, and even denied feeling? Most people believe they can detect false expressions; our research has shown most cannot.
In the last chapter I described our experiment in which we found that people were not able to tell when the student nurses were lying and when they were telling the truth. Those who saw just the nurses’ facial expressions did worse than chance, rating the nurses as most honest when they were, in fact, lying. They were taken in by the false expressions and ignored the expressions that leaked the true feelings. When people lie, their most evident, easy-to-see expressions, which people pay most attention to, are often the false ones. The subtle signs that these expressions are not felt, and the fleeting hints of the concealed emotions, are usually missed.
Most researchers have not measured the liar’s facial expressions but instead have focused on easier-to-measure behaviors, such as body illustrators or speech errors. The few who have measured the face have examined only the smile, and they measured smiling too simply. They found that people smile just as often when they lie or tell the truth. These researchers did not identify the kind of smile. Not all smiles are the same. Our technique for measuring the face can distinguish more than fifty different smiles. When the nursing students lied we found that they smiled in a different way than when they told the truth. I will describe those findings at the end of this chapter.
It is just because there are so many different expressions to be distinguished that those interested in nonverbal communication and lying have avoided measurement of the face. Until recently there was no comprehensive, objective way to measure all facial expressions. We set out to develop such a method because we knew, after looking at our videotapes of the student nurses lying, that uncovering facial signs of deceit would require precise measurement. We spent nearly ten years developing a technique to measure facial expression precisely.4
There are thousands of facial expressions, each different one from another. Many of them have nothing to do with emotion. Many expressions are what we call conversational signals, which, like body-movement illustrators, emphasize speech or provide syntax (such as facial question marks or exclamation points). There are also a number of facial emblems: the one-eye closure wink, the raised eyebrows-droopy upper eyelid-horseshoe mouth shrug, the one-eyebrow-raised skepticism, to mention a few. There are facial manipulators, such as lip biting, lip sucking, lip wiping, and cheek puffing. And then there are the emotional expressions, the true ones and the false.
There is not one expression for each emotion but dozens and, for some emotions, hundreds of expressions. Every emotion has a family of expressions, each visibly different one from another. This shouldn’t be surprising. There isn’t one feeling or experience for each emotion, but a family of experiences. Consider the members of the anger family of experiences. Anger varies in:
• intensity, from annoyance to rage;
• how controlled it is, from explosive to fuming;
• how long it takes to begin (onset time), from short-fused to smoldering;
• how long it takes to end (offset time), from rapid to lingering;
• temperature, from hot to cold;
• genuineness, from real to the phony anger an amused parent shows a naughty, charming child.
If one includes the blends of anger with other emotions—such as enjoyable anger, guilty anger, self-righteous anger, contemptuous anger—there would be even more members of the angry family.
No one yet knows whether there are different facial expressions for each of those different anger experiences. I believe there are and more. Already we have evidence that there are more different facial expressions than there are different single words for any emotion. The face signals nuances and subtleties that language does not map in single words. Our work mapping the repertoire of facial expression, determining exactly how many expressions there are for each emotion, which are synonyms and which signal different but related internal states, has been under way only since 1978. Some of what I will describe about facial signs of deceit is based on systematic studies using our new facial measurement technique, and some on thousands of hours inspecting facial expressions. What I report is tentative, because no other scientist has yet tried to repeat our studies of how voluntary and involuntary expressions differ.
Let’s begin with the most tantalizing source of facial leakage, micro expressions. These expressions provide a full picture of the concealed emotion, but so quickly that it is usually missed. A micro expression flashes on and off the face in less than one-quarter of a second. We discovered micro expressions in our first study of clues to deceit, nearly twenty years ago. We were examining a filmed interview with the psychiatric patient Mary, mentioned in chapter 1, who was concealing her plan to commit suicide. In the film, taken after Mary had been in the hospital for a few weeks, Mary tells the doctor she no longer feels depressed and asks for a weekend pass to spend time at home with her family. She later confesses that she had been lying so that she would be able to kill herself when freed from the hospital’s supervision. She admits to still feeling desperately unhappy.
Mary showed a number of partial shrugs—emblematic slips—and a decrease in illustrator movements. We also saw a micro expression: using slow-motion repeated replay, we saw a complete sadness facial expression, but it was there only for an instant, quickly followed by a smiling appearance. Micro expressions are full-face emotional expressions that are compressed in time, lasting only a fraction of their usual duration, so quick they are usually not seen. Figure 2 (see next page) shows the sadness expression. It is very easy to interpret, because it is frozen on the page. If you were to see it for only one-twenty-fifth of a second, and it was covered immediately by another expression, as it would be in a micro expression, you would be likely to miss it. Soon after we discovered the micro expression other investigators published their discovery of micros, saying they are the result of repression, revealing unconscious emotions.5 Certainly for Mary the feelings were not unconscious; she was painfully aware of the sadness shown in her micro expressions.
Figure 2
We showed excerpts containing micro expressions from Mary’s interview to people and asked them to judge how she was feeling. Untrained people were misled; missing the message in the micros, they thought she felt good. It was only when we used slow-motion projection that these people picked up the sadness message. Experienced clinicians, however, didn’t need slow-motion. They spotted the sadness message from the micro expression when they saw the film at real time.
With about one hour’s practice most people can learn to see such very brief expressions. We put a shutter over a projector lens so that a slide could be exposed very briefly. At first when an expression is flashed for one-fiftieth of a second, people claim they can’t see it and never will. Yet very quickly they learn to do so. It becomes so easy that sometimes people think we have slowed down the shutter. After seeing a few hundred faces, everyone has been able to recognize the emotion despite the brief exposure. Anyone can learn this skill without the shutter device by flashing a photograph of a facial expression very rapidly, as fast as they can, in front of their eyes. They should try to guess what emotion was shown in the picture, then look carefully at the picture to verify what is there, and then try another picture. Such practice has to be continued for at least a few hundred pictures.6
Micros are tantalizing, because rich as they are, providing leakage of a concealed emotion, they don’t occur very often. We found few micro expressions in the experiment in which the student nurses lied. Much more common were squelched expressions. As an expression emerges the person seems to become aware of what is beginning to show and interrupts the expression, sometimes also covering it with another expression. The smile is the most common cover or mask. Sometimes the squelch is so quick that it is hard to pick up the emotion message the interrupted expression would have conveyed. Even if the message does not leak, the squelch can be a noticeable clue that the person is concealing feelings. The squelched expression usually lasts longer but is not as complete as the micro. The micro is compressed in time, but the full display is there, shortened. The squelched expression is interrupted, the expression does not always reach a full display, but it lasts longer than a micro and the interruption itself may be noticeable.
Both micro and squelched expressions are vulnerable to the two problems that can cause difficulty in interpreting most clues to deceit. Recall from the last chapter the Brokaw hazard, in which the lie catcher fails to take account of individual differences in emotional expression. Not every individual who is concealing an emotion will show either a micro or squelched expression—so their absence is not evidence of truth. There are individual differences in the ability to control expression, and some people, what I call natural liars, do it perfectly. The second problem, what I called the Othello error, is caused by a failure to recognize that some truthful people become emotional when suspected of lying. Avoiding the Othello error requires that the lie catcher understand that even when someone shows a micro or squelched expression, that is not sufficient to be certain the person is lying. Almost any emotion leaked by these expressions can be felt by an innocent trying to conceal having those feelings. An innocent person might feel afraid of being disbelieved, guilty about something else, angry or disgusted at an unjust accusation, delighted at the opportunity to prove the accuser wrong, surprised at the charge, and so on. If that innocent person wanted to conceal having those feelings, a micro or squelched expression could occur. Ways to deal with these problems in interpreting micro and squelched expressions are discussed in the next chapter.
Not all of the muscles that produce facial expression are equally easy to control. Some muscles are more reliable than others. Reliable muscles are not available for use in false expressions; the liar cannot gain access to them. And, the liar has a difficult time concealing their action when trying to hide a felt emotion, as they are not readily inhibited or squelched.
We learned about which muscles cannot be easily controlled by asking people to move deliberately each of their facial muscles, and also to pose emotions on their faces.7 There are certain muscle movements that very few people can make deliberately. For example, only about 10 percent of those we have tested can deliberately pull the corners of their lips downward without moving their chin muscle. Yet, we have observed that those difficult-to-control muscles do move when the person feels an emotion that calls forth the movement. For example, the same people who cannot deliberately pull their lip corners down will show this action when they feel sadness, sorrow, or grief. We have been able to teach people how to move these difficult-to-control muscles deliberately, although it usually takes hundreds of hours for people to learn. These muscles are reliable because the person does not know how to get a message to the muscle to deploy it in a false expression. I reason that if a person can’t get a message to a muscle for false expression, then the person will have a hard time getting a “stop” or squelch message to interfere with that muscle’s action when an emotion is felt that calls the muscle into play. If you can’t deliberately move a muscle to falsify an expression, you won’t be able to readily inhibit the muscle from moving to conceal part of an emotional expression.*
There are other ways to conceal a felt expression without being able to inhibit it. The expression may be masked, typically with a smile, but this won’t cover the signs of the felt emotion in the forehead and upper eyelids. Alternatively, antagonistic muscles can be tightened to hold the real expression in check. A smile of pleasure, for example, can be diminished by pressing the lips together and pushing the chin muscle up. Often, however, the use of antagonistic muscles may itself be a deception clue, since the melding of the antagonistic muscles with the muscles involved in the expression of the felt emotion may make the face look unnatural, stiff, or controlled. The best way to conceal a felt emotion would be to inhibit the actions of the muscles involved in its expression totally. And that may be difficult to do if the emotion involves the reliable facial muscles.
*I have discussed this idea with a number of neuroscientists knowledgeable about the face or emotion, and they believe this is a reasonable and probable notion. It has not yet been tested and must be regarded as a hypothesis.
The forehead is the chief locus for reliable muscle movements. Figure 3A shows the reliable muscle movements that occur with sadness, grief, distress, and probably also with guilt. (It is the same expression as shown in figure 2, but it is easier in Figure 3A to focus just on the forehead since the rest of the face is blank.) Note that the inner corners of the eyebrow are pulled upward. Usually this will also triangulate the upper eyelid and produce some wrinkling in the center of the forehead. Less than 15 percent of the people we tested could produce this movement deliberately. It should not be present in a false display of these emotions, and it should appear when a person feels sad or distressed (or perhaps with guilt), despite attempts to conceal those feelings. This and the other drawings of facial expression show an extreme version of the display to make the shape of the expression clear despite not being able to show the action move on and off the face. If a sad feeling was weak, the appearance of the forehead would be the same as in Figure 3A but it would be smaller. Once the pattern of an expression is known, even slight versions are detectable, when, as in real life, the movement, not a static representation, is seen.
Figure 3B shows the reliable muscle movements that occur with fear, worry, apprehension, or terror. Note that the eyebrows are raised and pulled together. This combination of actions is extremely difficult to make deliberately. Less than 10 percent of the people we tested could produce it deliberately. The drawing also shows the raised upper eyelid and tensed lower eyelid that typically mark fear. These eyelid actions may drop out when a person attempts to conceal fear, for these are not difficult actions to control. The eyebrow position is more likely to remain.
Figures 3C and 3D show the eyebrow and eyelid actions that mark anger and those for surprise. There are no distinctive eyebrow and eyelid actions that mark other emotions. The eyebrow and eyelid movements shown in figures 3C and 3D are not reliable. Everyone can do them, and therefore they should appear in false expressions and easily be concealed. They are included to round out the picture of how the eyebrows and eyelids signal emotions, so that the contrast in appearance with the reliable actions shown in figures 3A and 3B will be more evident.
Figure 3A
Figure 3B
Figure 3C
Figure 3D
The eyebrow actions shown in figures 3C and 3D—raising or lowering—are the most frequent facial expressions. These eyebrow actions are often used as conversational signals to accent or emphasize speech. Brow raises are also deployed as exclamation or question marks, and as disbelief and skepticism emblems. Darwin called the muscle that pulls the brows down and together the “muscle of difficulty.” He was correct in asserting that this action occurs with difficulty of any kind, from lifting something heavy to solving a complex arithmetic problem. Lowering and drawing the brows together is common with perplexity and concentration as well.
There is another reliable facial action in the mouth area. One of the best clues to anger is a narrowing of the lips. The red area becomes less visible, but the lips are not sucked in or necessarily pressed. This muscle action is very difficult for most people to make, and I have noted it often appears when someone starts to become angry, even before the person is aware of the feeling. It is a subtle movement, however, and also one easily concealed by smiling actions. Figure 4 shows how this action changes the appearance of the lips.
The Othello error—failing to recognize that a truthful person suspected of lying may show the same signs of emotion as a liar—can complicate the interpretation of the reliable facial muscles. An innocent suspect may show the reliable fear display shown in Figure 3B because he is afraid of being falsely accused. Worried that if he looks afraid people will think he is a liar, he may try to conceal his fear so that the signs of fear remain only in his eyebrows, which are difficult to inhibit. The liar afraid of being caught, who attempts to conceal his fear, is likely to show the same expression. Chapter 6 explains ways for the lie catcher to deal with this problem.
Figure 4
The Brokaw hazard—failing to take account of individual differences that may cause a liar not to show a clue to deceit while a truthful person does show it—also has to be avoided in interpreting the reliable facial muscles. Some people—both psychopaths and natural liars—have an extraordinary ability to inhibit facial signs of their true feelings. For them, even the reliable facial muscles are not trustworthy. Many charismatic leaders are such extraordinary performers. Pope John Paul II reportedly showed his skill during his visit to Poland in 1983.*
Just a few years earlier, the shipyard strike in Gdansk sparked the hope that the communist rulers in Poland might allow some political freedom. Many feared that if Lech Walesa, the labor union Solidarity’s leader, pushed too far or too fast Soviet troops would march in, as they had years before in Hungary, Czechoslovakia, and East Germany. For months Soviets troops engaged in “military exercises” close to the border with Poland. Finally, the regime that had tolerated Solidarity resigned, and the Polish military, with Moscow’s approval, took over. General Jaruzelski suspended the activity of labor unions, restricted the activity of Lech Walesa, and imposed martial law. Now, after eighteen months of martial law, the visit of the pope, himself a Pole, could have important consequences. Would the pope show support for Walesa, would his presence rekindle a strike, catalyze rebellion, or would he give his blessing to General Jaruzelski? Journalist William Safire described the filmed meeting between the general and the pope: “. . . the pontiff and puppet leader showed smiles and handshakes. The pope understands how public appearances can be used, and calibrates his facial expressions at such events. Here the sign was unmistakable: church and state have reached some secret agreement, and the political blessing sought by Moscow’s chosen Polish leader [Jaruzelski] was given to be played and replayed on state television.”8
*Our disapproval of lying is so strong that my use of the term liar for any who is respected seems wrong. As I explained in chapter 2, I do not use the term liar in a pejorative fashion, and as I will explain in the last chapter, I believe some Vies are morally defensible.
Not every political leader can so skillfully manage his expressions. The late president of Egypt, Anwar Sadat, wrote about his attempts as a teen-ager to learn how to control his facial muscles: “. . . my hobby was politics. At that time Mussolini was in Italy. I saw his pictures and read about how he would change his facial expressions when he made public addresses, variously taking a pose of strength, or aggression, so that people might look at him and read power and strength in his very features. I was fascinated by this. I stood before the mirror at home and tried to imitate this commanding expression, but for me the results were very disappointing. All that happened was that the muscles of my face got very tired. It hurt.”9
While not able to falsify his facial expressions, Sadat’s success in secretly forging a joint Syrian-Egyptian surprise attack on Israel in 1973 shows that he was, nevertheless, skillful in deceit. There is no contradiction. Deceit does not require skill in falsifying or concealing facial expression, body movement, or voice. That is necessary only in intimate deceits, when the liar and victim are in face to face, direct contact, as in the meeting during which Hitler so ably misled Chamberlain. Reportedly Sadat never tried to conceal his true feelings when he met directly with his adversaries. According to Ezer Weizman, the Israeli minister of defense who negotiated directly with Sadat after the 1973 war: “He is not a man to keep his feelings to himself: they are immediately evident in his expression as well as in his voice and gestures.”10
There is another, more limited way in which individual differences interfere with the interpretation of the reliable facial muscles. It involves the conversational facial signals I mentioned earlier. Some of the conversational signals are much like hand illustrators, providing emphasis to particular words as they are spoken. Most people either lower their brows or raise their brows (as shown in figures 3C and 3D). Very few people use either the sadness or fear (figures 3 A and 3B) brow movement to emphasize speech. For those who do, these movements are not reliable. The actor-director Woody Allen is a person whose brow movements are not reliable. He uses the sadness brow movement as a speech emphasizer. While most people raise or lower their brows to emphasize a word, Woody Allen instead usually pulls the inner corner of his eyebrows up. This is part of what gives him such a wistful or empathic look. Others who, like Woody Allen, use the sadness brow as an emphasizer are easily able to make these actions deliberately. Such people should be able to use these movements in a false expression and conceal them when they choose to. They have easy access to muscles most people can’t reach. The lie catcher can tell that he can’t rely on these muscles if the suspect frequently uses such actions as emphasizers.
A third problem can complicate the interpretation of reliable facial muscles and other clues to deceit: a theatrical technique can be used to bring these muscles into action in a false expression. The Stanislavski acting technique (also known as method acting) teaches the actor how to accurately show emotion by learning how to remember and re-experience an emotion. I mentioned near the end of the last chapter our use of this acting technique to study the autonomic nervous system. When an actor uses this technique, his facial expressions are not made deliberately but are the product of the re-experienced emotion, and as our study suggests, the physiology of emotion can be awakened. Sometimes when people cannot make the actions shown in figures 3A or 3B, I have asked them to use the Stanislavski technique, instructing them to re-experience sad feelings or fearful ones. The facial actions they could not make deliberately often then will appear. The liar could also use the Stanislavski technique, and if so there should be no signs that the performance is false, because, in a sense, it won’t be. The reliable facial muscles would appear in such a liar’s false expression because the liar feels the false emotion. The line between false and true becomes fuzzy when emotions are produced by the Stanislavski technique. Even worse is the liar who succeeds in deceiving herself, coming to believe her lie is true. Such liars are undetectable. It is only liars who know they are lying when they lie who are likely to be caught.
So far I have described three ways in which concealed feelings may leak: micro expressions; what can be seen before a squelch; and what remains on the face because it was not possible to inhibit the action of the reliable facial muscles. Most people believe in a fourth source for the betrayal of concealed feelings—the eyes. Thought to be the windows of the soul, the eyes are said to reveal the innermost true feelings. The anthropologist Margaret Mead quoted a Soviet professor who disagreed: “Before the revolution we used to say: ‘The eyes are the mirror of the soul.’ The eyes can lie—and how. You can express with your eyes a devoted attention which, in reality you are not feeling. You can express serenity or surprise.”11 This disagreement about the trustworthiness of the eyes can be resolved by considering separately each of five sources of information in the eyes. Only three of them provide leakage or deception clues.
First are the changes in the appearance of the eyes produced by the muscles surrounding the eyeballs. These muscles modify the shape of the eyelids, how much of the white and iris of the eye is revealed, and the overall impression gained from looking at the eye area. Some of the changes produced by these muscles are shown in figures 3A, 3B, 3C, and 3D, but, as already mentioned, the actions of these muscles do not provide reliable clues to deceit. It is relatively easy to move these muscles deliberately, and to inhibit their actions. Not much will leak except as part of a micro or squelched expression.
The second source of information from the eye area is the direction of gaze. The gaze is averted with a number of emotions: downward with sadness; down or away with shame or guilt; and away with disgust. Yet even the guilty liar probably won’t avert his eyes much, since liars know that everyone expects to be able to detect deception in this way. The Soviet professor quoted by Mead noted how easy it is to control the direction of one’s gaze. Amazingly, people continue to be misled by liars skillful enough to not avert their glance. “One of the things that attracted Patricia Gardner to Giovanni Vigliotto, the man who may have married 100 women, was ‘that honest trait’ of looking directly into her eyes, she testified yesterday [at his trial for bigamy].”12
The third, fourth, and fifth sources of information from the eye area are more promising sources of leakage or deception clues. Blinking can be done voluntarily, but it is also an involuntary response, which increases when people are emotionally aroused. Pupils dilate when people are emotionally aroused, but there is no voluntary pathway that allows anyone the option to make this change by choice. Pupil dilation is produced by the autonomic nervous system, which also produces the changes in salivation, respiration, and sweating mentioned in chapter 4 and some other facial changes described below. While increased blinking and dilated pupils indicate a person is emotionally aroused, they do not reveal which emotion it is. These may be signs of excitement, anger, or fear. Blinking and pupil dilation could be valuable leakage only when evidence of any emotion would betray that someone was lying and the lie catcher can rule out the possibility that they are signs of an innocent person’s fear of being wrongly judged.
Tears, the fifth and last source of information in the eye area, are also produced by autonomic nervous system activity, but tears are signs of only some, not all, emotions. Tears occur with distress, sadness, relief, certain forms of enjoyment, and uncontrolled laughter. They can leak distress or sadness when other signs are concealed, although I expect that the eyebrows would also show the emotion, and the person, if the tears began, would quickly acknowledge the concealed feeling. Tears of enjoyment should not leak if the laughter itself has been suppressed.
The autonomic nervous system produces other visible changes in the face: blushing, blanching, and sweating. Just as with the other facial and bodily changes produced by the autonomic nervous system, it is difficult to conceal blushing, blanching, or facial sweating. It is not certain whether sweating is, like increased eyeblinks and pupil dilation, a sign of the arousal of any emotion, or instead specific to just one or two emotions. Very little is known about blushing and blanching.
Blushing is presumed to be an embarrassment sign, occurring also with shame and perhaps with guilt. It is said to be more common in women than men, although why this might be so is not known. Blushing could leak that a liar is embarrassed or ashamed about what is being concealed, or it could be embarrassment itself that is being concealed. The face also turns red with anger, and no one knows how this reddening might differ from the blush. Presumably both involve dilation of the peripheral blood vessels in the skin, but the red of anger and the embarrassment or shame blush could differ in amount, areas of the face affected, or duration. I expect that the face reddens only when anger is not being controlled, or when a person tries to control anger that is verging on exploding. If that is so, then usually there would be other evidence of anger in face and voice, and the lie catcher would not have to rely just upon face coloration to pick up this emotion. In more controlled anger the face may whiten or blanch, as it also may with fear. Blanching might leak even when the expressions of anger or fear are concealed. Amazingly, there has been very little study of tears, blushing, reddening, or blanching in relation to the expression or concealment of specific emotions.
Let us turn from how the face may betray a concealed emotion to facial signs that an expression is false and that emotion is not really felt. One possibility, already mentioned, is that the reliable muscles may not be part of a false expression, as long as there is no Woody Allen or Stanislavski problem. There are three other clues that suggest an expression is false: asymmetry, timing, and location in the conversational stream.
In an asymmetrical facial expression, the same actions appear on both sides of the face, but the actions are stronger on one side than the other. They should not be confused with unilateral expressions, those that appear on only one side of the face. Such one-sided facial actions are not signs of emotion, with the exception of the contempt expressions in which the upper lip is raised or the lip corner is tightened on one side. Instead unilateral expressions are used in emblems such as the wink, or the skeptical raise of one eyebrow. Asymmetrical expressions are more subtle, much more common, and much more interesting than unilateral ones.
Scientists interested in the findings that the right hemisphere of the brain seems to specialize in dealing with emotion thought that one side of the face might be more emotional. Since the right hemisphere controls many of the muscles on the left side of the face, and the left hemisphere controls many of the muscles on the right side of the face, some scientists suggested that emotion would be shown more strongly on the left side of the face. In my attempt to figure out inconsistencies in one of their experiments, I discovered, by accident, how asymmetry can be a clue to deceit. Crooked expressions, in which the actions are slightly stronger on one side of the face than the other, are a clue that the feeling shown is not felt.
The accident happened because the first team of scientists who claimed to find that emotion is shown most strongly on the left side of the face didn’t use their own materials but borrowed facial photographs from me. I examined their findings more closely than I otherwise would have and was able to learn things they didn’t see because of what I knew as the photographer of the faces. Harold Sackeim and his colleagues cut each of our facial pictures in half to create a double-left photograph and a double-right photograph, each a full-face picture composed of a mirror image of one or the other side of the face. People rated emotion as more intense when they saw the double-left than the double-right pictures.1’ I noticed that there was one exception—there was no difference in the judgments of the happy pictures. Sackeim had not made much of this, but I did. As the photographer, I knew that the happy pictures were the only real emotional expressions. The rest I had made by asking my models to move particular facial muscles deliberately. I had shot the happy pictures by catching the models off-guard when they were enjoying themselves.
Putting this together with the studies on brain damage and facial expression I described early in this chapter suggested a very different interpretation of facial asymmetry. Those studies had shown that voluntary and involuntary expressions involve different neural pathways, for one may be impaired but not the other, depending upon where the brain is damaged. Since voluntary and involuntary expressions can be independent of each other, if one was asymmetrical the other might not be. The final bit of logic was based on the well-established fact that the cerebral hemispheres direct voluntary, not involuntary, facial movement; the latter are generated by lower, more primitive areas of the brain. Differences between the left and right hemispheres should influence voluntary expressions, not involuntary emotional expressions.
Sackeim had found, according to my reasoning, just the opposite of what he thought he had proven. It was not that the two sides of the face differ in emotional expression. Instead, asymmetry occurred just when the expression was a deliberate, voluntary, pose, one made on demand. When expression was involuntary, as in the spontaneous happy faces, there was little asymmetry. Asymmetry is a clue that the expression is not felt.14 We conducted a number of experiments testing these ideas, comparing deliberate with spontaneous facial expressions.
Scientific argument about this matter has been intense, and only recently has partial agreement emerged—just about the actions involved in the positive emotional expressions. Most investigators now agree with our finding that when the expression is not felt, the principal muscle involved in smiling acts more strongly on one side of the face. When we asked people to smile deliberately or pose happiness we found asymmetry, as we did when we examined the smiles people sometimes show when watching one of our gory films. Typically, the action was slightly stronger on the left side of the face if the person was right-handed. In genuine, felt smiles we have found a much lower incidence of asymmetrical expressions, and no tendency for those that are asymmetrical to be mostly stronger on the left side of the face.15
We also have found asymmetry in some of the actions involved in the negative emotions, when the actions are produced deliberately, but not when they are part of a spontaneous display of emotion. Sometimes the actions are stronger on the left, sometimes they are stronger on the right, and sometimes there is no asymmetry. In addition to the smile, the brow-lowering action that is often part of the anger display usually is stronger on the left side of the face when the action is made deliberately. The nose-wrinkling action involved in disgust and the stretching of the lips back toward the ears found in fear are usually stronger on the right side of the face if the actions are made deliberately. These findings have just been published, and it is not yet certain whether they will convince those, like Sackeim, who proposed asymmetry in emotional expressions.16
I did not think it would matter much to the lie catcher. Asymmetry is usually so subtle that I thought no one could spot it without precise measurement. I was wrong. When we asked people to judge whether expressions were symmetrical or asymmetrical, they did far better than chance, even though they had to make this judgment without slow-motion or repeated viewing.17 They did have the benefit of not having to do anything else. We don’t know yet whether people will be able to do so well when they also have to contend with the distractions of seeing the body movements, hearing the speech, and making replies to the person they converse with. It is very difficult to devise an experiment to determine that.
If many facial expressions are asymmetrical it is likely they are not felt, but asymmetry is not certain proof that the expression is unfelt. Some felt expressions are asymmetrical; it is just that most are not. Similarly, the absence of asymmetry does not prove that the expression is felt; the lie catcher may have missed them, and apart from that problem, not every deliberate, unfelt expression is asymmetrical; only most are. A lie catcher should never rely upon one clue to deceit; there must be many. The facial clues should be confirmed by clues from voice, words, or body. Even within the face, any one clue shouldn’t be interpreted unless it is repeated and, even better, confirmed by another type of facial clue. Earlier, three sources of leakage, or ways the face betrays concealed feelings, were explained—the reliable facial muscles, the eyes, and autonomic nervous system changes in facial appearance. Asymmetry is one of another set of three clues, not of leakage of what is being concealed but of deception clues that the expression shown is false. Timing is the second source of deception clues.
Timing includes the total duration of a facial expression, as well as how long it takes to appear (onset), and how long it takes to disappear (offset). All three can provide deception clues. Expressions of long duration—certainly ten seconds or more, and usually 5 seconds—are likely to be false. Most felt expressions don’t last that long. Unless someone is having a peak experience, at the height of ecstasy, in a roaring rage, or at the bottom of depression, genuine emotional expressions don’t remain on the face for more than a few seconds. Even in those extreme states expressions rarely last so long; instead, there are many shorter expressions. The long expressions are usually emblems or mock expressions.
There is no hard and fast rule about deception clues in the onset and offset times except for surprise. Onset, offset, and duration all must be short, less than a second, if the surprise is genuine. If it is longer it is mock surprise (the person is playing at being surprised), a surprise emblem (the person is referring to being surprised), or false surprise, in which the person is trying to seem surprised when he isn’t. Surprise is always a very brief emotion, lasting only until the surprised person has figured out the unexpected event. While most people know how to fake surprise, few could do so convincingly with the fast onset and offset that a natural surprise must have. A news story showed how valuable a genuine surprise expression can be. “A man wrongfully convicted of armed robbery was freed after a prosecutor—noticing the man’s reaction to the guilty verdict—dug up new evidence that proved Wayne Milton innocent. Assistant State Attorney Tom Smith said he knew something was wrong after seeing Milton’s face drop when a jury convicted him last month of the $200 holdup at the Lake Apopka Gas Co.”18
All the rest of the emotional expressions can be very short, flashing on and off in a second, or they may last for a few seconds. The onset and the offset may be abrupt or gradual. It depends upon the context in which the expression occurs. Suppose a subordinate is faking enjoyment when hearing a dull joke told for the fourth time by an intrusive boss, who has no sense of humor and a poor memory. How long it should take for the smiling actions to appear depends upon the build-up to the punch line—whether it is gradual, with slightly humorous elements, or abrupt. How long it should take for the smiling actions to disappear depends upon the type of joke—how much recycling or redigesting of the story would be appropriate. Everyone is able to make some kind of smile to falsify enjoyment, but a liar is less likely to correctly adjust the onset and offset timing of that smile to the particulars demanded by the context.
The exact location of an expression in relation to the flow of speech, the voice changes, and the body movements is the third source of deception clues that an expression is false. Suppose someone is falsifying anger and says, “I’m fed up with your behavior.” If the anger expression comes after the words it is more likely to be false than if the anger occurs at the start, or even a moment before, the words. There is probably less latitude about where to position facial expression in relation to body movement. Suppose during the “fed up” the liar banged a fist on the table. If the anger expression followed the fist bang it is more likely to be false. Facial expressions that are not synchronized with body movement are likely to be deception clues.
No discussion of facial signs of deceit would be complete without considering one of the most frequent of all the facial expressions—smiles. They are unique among the facial expressions. It takes but one muscle to show enjoyment, while most of the other emotions require the action of three to five muscles. This simple smile is the easiest expression to recognize. We found such smiles can be seen from further away (300 feet) and with a briefer exposure than other emotional expressions.19 It is hard not to reciprocate a smile; people do so even if the smile they reciprocate is one shown in a photograph. People enjoy looking at most smiles, a fact well known to advertisers.
Smiles are probably the most underrated facial expressions, much more complicated than most people realize. There are dozens of smiles, each differing in appearance and in the message expressed. There are many positive emotions signaled by smiling—enjoyment, physical or sensory pleasure, contentment, and amusement, to name just a few. People also smile when they are miserable. These aren’t the same as the false smiles used to convince another that positive feelings are felt when they aren’t, often masking the expression of a negative emotion. We recently found that people are misled by these false smiles. We had people look only at the smiles shown by the student nurses in our experiment and judge whether or not each smile was genuine {shown while a nurse watched a pleasant film), or false (shown while a nurse concealed the negative emotions aroused by our gory film). People did no better than chance. I believe the problem was not just a failure to recognize deceptive smiles but stemmed from a more general lack of understanding of how many different kinds of smiles there are. The false can’t be distinguished from the felt without knowing how each resembles and differs from all of the other principal members of the smile family. Following are descriptions of eighteen different kinds of smiles, none of them deceptive smiles.
The common element in most members of the smile family is the appearance change produced by the zygomatic major muscle. This muscle reaches from the cheekbones down and across the face, attaching to the corners of the lips. When contracted, the zygomatic major pulls the lip corners up at an angle toward the cheekbones. With a strong action this muscle also stretches the lips, pulls the cheeks upward, bags the skin below the eyes, and produces crow’s-feet wrinkles beyond the eye corners. (In some individuals this muscle also pulls down slightly the tip of their nose; in still others there will be a slight tug at the skin near their ears). Other muscles merge with the zygomatic major to form different members of the smile family; and a few smiling appearances are produced not by the zygomatic but by other muscles.
The simple action of the zygomatic major muscle produces the smile shown for genuine, uncontrolled, positive emotions. No other muscles in the lower part of the face enter into this felt smile. The only action that may also appear in the upper face is the tightening of the muscle that circles the eyes. This muscle produces most of changes in the upper face that also can be produced by a strong action of the zygomatic major—raised cheek, bagged skin below the eye, and crow’s-feet wrinkles. Figure 5A (see next page) shows the felt smile. The felt smile lasts longer and is more intense when positive feelings are more extreme.201 believe that all of the positive emotional experiences—enjoyment of another person, the happiness of relief, pleasure from tactile, auditory, or visual stimulation, amusement, contentment—are shown by the felt smile and differ only in the timing and intensity of that action.
The fear smile in Figure 5B (see next page) has nothing to do with positive emotions, but it is sometimes so mistaken. It is produced by the risorious muscle pulling the lip corners horizontally toward the ears so that the lips are stretched to form a rectangular shape. Risorious is from the Latin word for laughing, but this action occurs principally with fear, not laughter. The confusion probably arose because sometimes when risorius pulls the lips horizontally the corners will tilt upward, resembling a very widely stretched version of the felt smile. In a fear facial expression the rectangular shaped mouth (with or without an upward lip corner tilt) will be accompanied by the brows and eyes shown in figure 3B.
The contempt smile is another misnomer, for this expression too has not much to do with positive emotions, although it is often so construed. The version of contempt shown in Figure 5C involves a tightening of the muscle in the lip corners, producing a muscle bulge in and around the corners, often a dimple, and a slight angling up of the lip corners.* Again, it is the angling up of the lip corners, a shared characteristic with the felt smile, that causes the confusion. Another shared element is the dimple, which sometimes appears in the felt smile. The chief difference between the contempt smile and the felt smile is the tightened lip corners, which are present in contempt and absent in the felt smile.
Figure 5A Felt smile
Figure 5B Felt smile
Figure 5C Felt smile
In a dampened smile a person actually feels positive emotions but attempts to appear as if those feelings are less intense than they actually are. The aim is to dampen (but not suppress) the expression of positive emotions, keeping the expression, and perhaps the emotional experience, within bounds. The lips may be pressed, the lip corners tightened, the lower lip pushed up, the lip corners pulled down, or any combination of these actions may merge with the simple smile. Figure 5D (see next page) shows a dampened smile with all three dampening actions merged with the felt smile action.
The miserable smile acknowledges the experience of negative emotions. It is not an attempt to conceal but a facial comment on being miserable. The miserable smile usually also means that the person who shows it is not, at least for the moment, going to protest much about his misery. He is going to grin and bear it. We have seen this miserable smile on the faces of people when they were sitting alone in our laboratory watching one of our gory films, unaware of our hidden camera. Often it appeared early when they seemed to first become aware of just how terrible our films are. We have also seen miserable smiles on the faces of depressed patients, as a comment on their unhappy plight. Miserable smiles are often asymmetrical. They are often superimposed on a clear negative emotional expression, not masking it but adding to it, or they may quickly follow a negative emotional expression. If the miserable smile is acknowledging an attempt to control the expression of fear, anger, or distress, the miserable smile may appear much like the dampened smile. The lip pressing, lower lip pushed up by the chin muscle, and corners tightened or down may be serving to control the outburst of one of these negative feelings. The key difference between this version of the miserable smile (shown in figure 5E) and the dampened smile is the absence of any evidence of the muscle around the eyes tightening. The action of that muscle—pulling in the skin around the eye and crow’s-feet wrinkles—is part of the dampened smile because enjoyment is felt and absent from the miserable smile because enjoyment is not felt.-The miserable smile may also show in the eyebrows and forehead the felt negative emotions being acknowledged.
*Contempt can also be shown by a unilateral version of this expression in which one lip corner is tightened and slightly raised.
Figure 5D Dampened smile
Figure 5E Miserable smile
In a blend two or more emotions are experienced at once, registered within the same facial expression. Any emotion can blend with any other emotion. Here we are concerned just with the appearance of the emotions that often blend with the positive emotions. When people enjoy being angry, the enjoyable-anger blend will show a narrowing of the lips and sometimes also a raising of the upper lip, in addition to the felt smile, as well as the upper face appearance shown in figure 3C. (This could also be called a cruel smile, or a sadistic smile.) In the enjoyable-contempt expression the felt smile merges with the tightening of one or both lip corners. Sadness and fear can also be enjoyed, as those who make horror and tear-jerking films and books. In enjoyable-sadness the lip corners may be pulled down in addition to the upward pull of the felt smile, or the felt smile may just merge with the upper face shown in Figure 3 A. The enjoyable-fear blend shows the upper face in Figure 3B together with the felt smile merged with the horizontal stretching of the lips. Some enjoyable experiences are calm and contented, but sometimes enjoyment is blended with excitement, in an exhilarating feeling. In enjoyable-excitement the upper eyelid is raised in addition to the felt smile. The film actor Harpo Marx often showed this excited, gleeful smile, and at times when pulling a prank, the enjoyable-anger smile. In enjoyable-surprise the brow is raised, the jaw dropped, the upper lid raised, and the felt smile shown.
Two other smiles involve merging the felt smile with a particular gaze. In the flirtatious smile the flirter shows a felt smile while facing and gazing away from the person of interest and then, for a moment, steals a glance at the person, long enough to be just noticed as the glance shifts away again. One of the elements that makes the painting of the Mona Lisa so unusual is that Leonardo depicted her caught in the midst of such a flirtatious smile, facing one way but glancing sideways at the object of her interest. In life this is an action, with the gaze shift lasting but a moment. In the embarrassment smile the gaze is directed down or to the side, so that the embarrassed person does not meet the other’s eyes. Sometimes there will be a momentary upward lift of the chin boss (the skin and muscle between the lower lip and the tip of the chin) during the felt smile. In still another version, embarrassment is shown by combining the dampened smile with a downward or sideways gaze.
The Chaplin smile is unusual, produced by a muscle that most people can’t move deliberately. Charlie Chaplin could, for this smile, in which the lips angle upward much more sharply than they do in the felt smile, was his hallmark. (See figure SF, next page.) It is a supercilious smile that smiles at smiling.
The next four smiles all share the same appearance, but they serve quite different social functions. In each the smile is deliberately made. Often these smiles will show some asymmetry.
The qualifier smile takes the harsh edge off an otherwise unpleasant or critical message, often trapping the distressed recipient of the criticism into smiling in return. The smile is set deliberately, with a quick, abrupt onset. The lip corners may be tightened and sometimes too the lower lip pushed up slightly for a moment. The qualifier smile is often marked with a head nod and a slightly down and sideways tilt to the head so that the smiler looks down a little at the person criticized.
The compliance smile acknowledges that a bitter pill will be swallowed without protest. No one thinks the person showing it is happy, but this smile shows that the person is accepting an unwanted fate. It looks like the qualifier smile, without that smile’s head position. Instead, the brows may be raised for a moment, a sigh may be heard, or a shrug shown.
Figure 5F Chaplin smile
The coordination smile regulates the exchange between two or more people. It is a polite, cooperative smile that serves to smoothly show agreement, understanding, intention to perform, or acknowledgment of another’s proper performance. It involves a slight smile, usually asymmetrical, without the action of the muscle orbiting the eyes.
The listener response smile is a particular coordination smile used when listening to let the person speaking know that everything is understood and that there is no need to repeat or rephrase. It is equivalent to the “mm-hmm,” “good,” and head nod it often accompanies. The speaker doesn’t think the listener is happy but takes this smile as encouragement to continue.
Any of these four smiles—qualifier, compliance, coordination, or listener—may sometimes be replaced by a genuine felt smile. Someone who enjoys giving a qualifying message, who takes pleasure in complying, listening, or coordinating, may show the felt rather than one of the unfelt smiles I have described.
Now let’s consider the false smile. It is intended to convince another person that positive emotion is felt when it isn’t. Nothing much may be felt, or negative emotions may be felt that the liar tries to conceal by using the false smile as a mask. Unlike the miserable smile that acknowledges pleasure is not felt, the false smile tries to mislead the other person to think the smiler is having positive feelings. It is the only smile that lies.
There are a number of clues for distinguishing false smiles from the felt smiles they pretend to be:
False smiles are more asymmetrical than felt smiles.
The false smile will not be accompanied by the involvement of the muscles around the eyes, so that the slight to moderate false smile will not show raised cheeks, bagged skin below the eyes, crow’s-feet wrinkles, or a slight lowering of the eyebrow that will appear in the slight to moderate felt smile. An example is shown in figure 6; compare it to figure 5A. If the smile is larger in size, the smiling action itself—the zygomatic major muscle—raises the cheeks, gathers the skin below the eyes, and produces crow’s-feet wrinkles. But it will not lower the eyebrow. If you look in a mirror and slowly make a larger and larger smile, you will note that as the smile increases the cheeks rise and crow’s-feet appear; but your eyebrow will not be pulled down unless the eye muscle also acts. The lack of eyebrow involvement is a subtle cue, but a crucial one for distinguishing felt from false smiles when the grin is broad.
The offset time of the false smile may appear noticeably inappropriate. The smile may drop off the face too abruptly, or there may be a stepped offset, in which the smile decreases, and then is held, before either disappearing or going through another stepped decrease as it leaves the face.
Figure 6 False smile
Figure 5A Felt smile
When used as a mask the false smile will only cover the actions of the lower face and the lower eyelid. The reliable muscles that appear in the forehead to signal fear or distress may still appear. Even in the lower face, the false smile may not succeed in completely covering the signs of the emotion it is intended to conceal; instead, there may be a merging of elements so that some trace still appears, as if it is an emotion blend.
Our first test of these ideas was to measure the smiling expressions shown by the student nurses in our experiment. If my ideas about smiling are correct, they should have shown the felt smile in the honest interview, when they had watched a pleasant film and described their feelings frankly. They should have shown false smiles in the deceptive interview, when they had watched a very unpleasant film but tried to appear as if they were seeing another pleasant film. We measured only two of the signs that a smile is false—the absence of the muscle around the eyes and the presence of signs of disgust (nose wrinkling) or contempt (tightening of the lip corners). The results were exactly as predicted, and very strong: in the honest interview there were more felt than false smiles and no smiles that leaked either disgust or contempt; in the deceptive interview the leakage smiles appeared and also there were more false than felt smiles. I was amazed that these two clues to deceit worked so well, especially since I knew that people don’t seem to make use of them when judging others. In earlier studies we have shown the very same videotapes of facial expression and asked people to judge when the nurses were lying. People do no better than chance. Are we measuring something too subtle to see, or is it that people don’t know what to look for? Our next study will find out by telling people how to recognize when the eye muscle is acting, and the leakage smiles are occurring, and then checking on whether they can more accurately spot lying.
The face may contain many different clues to deceit: micros, squelched expressions, leakage in the reliable facial muscles, blinking, pupil dilation, tearing, blushing and blanching, asymmetry, mistakes in timing, mistakes in location, and false smiles. Some of these clues provide leakage, betraying concealed information; others provide deception clues indicating that something is being concealed but not what; and others mark an expression to be false.
These facial signs of deceit, like the clues to deceit in words, voice, and body described in the last chapter, vary in the precision of the information they convey. Some clues to deceit reveal exactly which emotion is actually felt, even though the liar tries to conceal that feeling. Other clues to deceit reveal only whether the emotion being concealed is positive or negative and don’t reveal exactly which negative emotion or which positive emotion the liar feels. Still other clues are even more undifferentiated, betraying only that the liar feels some emotion but not revealing whether the concealed feeling is positive or negative. That may be enough. Knowing that some emotion is felt sometimes can suggest that a person is lying, if the situation is one in which except for lying the person would not be likely to feel any emotion at all. Other times a lie won’t be betrayed without more precise information about which concealed emotion is felt. It depends upon the lie, the line taken by the person suspected of lying, the situation, and the alternative explanations available, apart from lying, to account for why an emotion might be felt but concealed.
It is important for the He catcher to remember which clues convey specific and which convey only more general information. Tables 1 and 2, in the appendix, summarize the information for all clues to deceit described in this and the previous chapter. Table 3 deals with clues to falsification.