20

Filmed Interactions: The Behavior of an Invisible Visible Body

INTRODUCTION: A NEW WAY TO APPROACH THE ORGANISM

Behavior Can Adapt Itself Simultaneously to a Multitude of Heterogeneous Stimulations

Behavior is often reduced to a body phenomenon or the functioning of the psyche. In the following sections, I show that the study of behavior requires particular methods and theories that are well differentiated from those used to study the psyche or the body. It is useful to clarify the epistemological status of behavior for all forms of psychotherapy because their dynamics are ceaselessly solicited by psychotherapists. In body psychotherapy, the nonverbal behavior is permanently related not only to what is being said but also to the reactions of the body and the vegetative system.

The field of nonverbal communication studies organized itself around the theoretical formulations of Gregory Bateson (1972). One of the first discoveries in Bateson’s generation is that an event does not convey a particular meaning, or does not have a particular function, but it is almost always polysemous and multifunctional:

A vignette on a hand that takes a cup of tea.

 

  1. The integration between gesture and physiology. By taking up a cup of tea, my hand is included in a series of dynamics that coordinate various parts of my nervous system, cardiovascular system, respiration, and the rapport of my body with gravity. This gesture may sometimes be activated by a thirst, itself activated by metabolic activity. The style of this movement may also convey an impression of tension or relaxation to another person.
  2. The gesture as an indicator of personal dynamics. The particularities of my gesture also depends on my sex, age, cultural origins, mood, way of analyzing what is going on around me, and the impression I want to make on others. I can take the cup politely, by its handle, while holding the saucer under the cup; or I can grab the cup as if it were a bowl to warm my hands. My hand may tremble because I am nervous or because of my age. I can set the cup on the table loudly to express my displeasure, and so on.
  3. The gesture as a stimulus for the other. A gesture may be carried out with a particular intention and have an unforeseen impact on the other. The typical example is that of intercultural interactions where one gesture changes meaning from one culture to another. An Arab may embrace a U.S. citizen out of politeness while the latter receives it an excess of familiarity.1 The way the other reacts to the gesture can also define its meaning and function. The others decide whether my way of handling the cup of tea appears to be relaxed, elegant, insulting, irritating or amusing.
  4. The gesture as an object of attribution. Those who see me take the cup of tea and drink will apply an index of possible readings to my way of behaving that have been constructed in their organism. If several individuals are present, they could each have a different index of possible meanings. One individual may find my gesture sexy, another find it disgraceful, and another still polite.

You may have noticed that my hand, in this example, can participate in a large number of heterogeneous personal and interpersonal regulations at the same time. I have only mentioned a few of the many possible ways of analyzing what my hand does when it lifts a cup of tea. This movement is influenced by a set of inner dynamics that are often relatively independent from how others perceived it. In this last example, nothing I do or want to do will change anything because it is in the mind of the other that some details of my hand take on a particular meaning. It is also possible that everybody is talking about a painting hanging on a wall and no one notices what I am doing with my hands. This last remark raises the following question: are all of the particularities of the hand gesture necessarily perceived by the nervous system of my protagonists? The answer is probably that many are, but not all. The items selected by a given conscious system may vary from one second to another, but this requires a sustained attention that mobilizes a great quantity of energy. Most of the time, a conscious system prefers to handle the first items that hit it for as long as possible. Only in the case of disequilibrium or a crisis in the interaction does a conscious system feel obliged to look at complementary information.

To scientifically study this type of phenomenon, it is useful to begin by making a list of the concrete options that present themselves. As it happens, we are only at the beginning of a long list that shows that at any given moment, a gesture may be part of a great variety of regulatory systems qualitatively different one from the other. It sometimes happens that a gesture inserts itself into a specific mode of functioning and then acquires a standardized semiotic function within a species or a particular cultural environment. Only in these cases (relatively few with regard to the entirety of mechanisms that can animate a gesture) does it become possible to relate a gesture to a meaning or a particular function. However, even in these cases, a body position can also have a multitude of other functions.

The researchers who analyzed nonverbal behavior in the 1950s where astonished when they observed that almost each item of behavior can simultaneously participate in a variety of regulatory systems. This is what allows the body to adapt itself simultaneously to a multitude of more or less heterogeneous stimulations. In the previous examples, we have seen that the movements of a hand can be simultaneously associated to a variety of phenomena, such as furniture, the shape of objects, one’s sex, age, etiquette, character, health, weather forecast, and more. These causal systems have not necessarily adapted to each other before influencing the organism. An individual system must continuously react to a multitude of solicitations that have their particular exigencies. It is this multiplicity of behavioral dimensions that allows the human organism to accommodate itself in such a refined way to such a large number of environments and of variables. This particularity of human behavior partly explains the adaptive potential of our species. Once this analysis is understood, it becomes easier to understand why I had emphasized the following quotes by Spinoza:

 

In proportion as a body is more capable than others of doing many things at once, or being acted on in many ways at once, so its mind is more capable than others of perceiving many things at once. (Spinoza, 1677a, II, Proposition XIII, scholium, p. 40)

 

The human mind is capable of perceiving a great many things, and is the more capable, the more its body can be disposed in a great many ways. (Spinoza, 1677a, II. proposition XIV, p. 44)

Some think that nonverbal communication is too young a science to be able to contradict what is evident to human perception. We will see that they are not entirely wrong. This science has not yet been able to produce a methodology and theory sufficiently robust to be convincing. Many times, in this volume, we have seen that a science often needs centuries to establish itself. Psychology is about 130 years old, and the study of nonverbal communication is barely 60 years of age. Nevertheless, we saw that this discipline has already demonstrated that most gestures cannot be explained by a single causal chain, such as a conscious intention or an emotion. As this thesis is central to my analysis of how human organisms function, I will now try to give tangible and concrete examples to support it. Because this field has not yet been able to discover precise general laws, I have to become a bit more technical than elsewhere to make my point.

The first researchers in nonverbal communication research, like Ray L. Birdwhistell, Edward T. Hall, and Adam Kendon, thought that there was a body language. They believed that gestures could be perceived, like words, as signs that represent a meaning. They explicitly referred to diverse linguistic theories, which they attempted to apply to the analysis of movements. However, as soon as they tried to analyze gestures with well-conceived experimental plans, they experienced a deluge of problems. They did not succeed in finding reliable methods that would allow them to describe a body event, analyze what had been observed, and gather the data into a theoretical whole. As gestures are spontaneously managed by all humans, they thought that analysis of them would be relatively easy. They were surprised to discover all the methodological difficulties raised by their project. These difficulties are a mine of challenging information. This is what I focus on in this chapter.

By definition, nonverbal communication studies everything that can be discerned by the nose, ears, and eyes. I concentrate on the study of bodily behavior because it is the visual modality that has been studied the most up to this day and I know this domain particularly well.

A Systemic View of Interaction

 

Interest in Ballet becomes more comprehensible if we realize that the most deeply moving moments of our lives usually leave us speechless, and in such moments our body carriage may well be able to express what otherwise would be incomprehensible. (Rudolf Laban, 1950, The Mastery of Movement, IV, p. 87)

 

Single movements are, of course, only like the words or letters of language; they do not give a definitive impression or a coherent flow of ideas. (Rudolf Laban, 1950, The Mastery of Movement, IV, p. 87)

For Thelen and Smith,2 reality is observed with a microscope that can be adjusted to provide a broad general view of reality. The researcher then perceives a relatively coherent form of reality with large patterns and clearly delineated rhythms. “As we turn up the magnification of our microscope, we see that our vision of linearity, uniformity, inevitable sequencing and even irreversibility break down” (Thelen and Smith, 1994, Introduction, p. xvi). Once the details become increasingly apparent, reality appears so varied and disparate, that it is impossible to categorize into great schemes (e.g., all fingerprints are different). The organization of causal chains now appears to be messy, heterogeneous, and fuzzy. Thus, psychologists situate the distinction between organism, body, and psyche in some global schemes, which they forget as soon as they observe a specific behavior. The researcher and the practitioner prefer to remain as close as possible to the details of what is perceived and measured instead of blinding oneself by following a theory that is only partially relevant.

This change of perspective often trivializes the notion of a global well-organized organism, as described by the System of Dimensions of the Organism. It becomes a meeting place within which heteroclite mechanisms pile up. When a researcher analyses the details of an event, he does not preoccupy himself too much with their status within the organism. The typical researcher will study different aspects of a dancer’s gesture without knowing what is psychological, of the body, behavioral or physiological. Certain variables will be collected at all these levels to find some interesting correlations. The questions regarding the way these variables interact are left aside for the day when we have a more consistent theory of the organism. For the moment, researchers tend to assume that there must be a web of connecting interfaces that allow for the multiple correlations that can be observed.

Psychologists often study an interaction between a patient and his therapist by using three kinds of data:

 

  1. What is going on inside the patient’s organism.
  2. What is going on inside the therapist’s organism.
  3. The impact of each one’s behavior on their interior world and on that of the other.

To coordinate these three pieces of data, researchers in nonverbal behavior tend to use three types of data that can be simultaneously viewed on three screens. The first shows the physiological functioning of an individual. The second films the two interacting organisms. The third shows the physiological functioning of the other individual. These screens give information on what is going on inside and on the surface of each organism in the interaction.

The psyche is situated somewhere in this frame. By following the procedures of artificial intelligence, psychologists then research algorithms that may be able to explain in an economical fashion the way the inside of an organism coordinates with the external surface. They then reduce their definition of the psyche to these hypothetical algorithms. An experimental psychologist can observe three types of phenomena:

 

  1. The way the behaviors of several organisms coordinate to accomplish common tasks (like piloting a sailboat).
  2. The way the behavior of an organism influences its physiological functioning and that of other organisms.
  3. Individual thoughts.

The content and form of a communicated intention are experienced as being shared material. This aspect is often discussed when members of a family expect that their qualities and efforts would be recognized and appreciated by everyone.

This impression that there exists shared content between humans is exquisitely described by Daniel Stern (1990) in Diary of a Baby. In his more rigorous research, Stern (1985) tries to associate a description of behavior and a more intuitive attempt to grasp the intimate sentiments of the subjects. He demands of the experimental psychologist not only a rigorous analysis of behavior but also an attempt to construct for oneself an intuitive impression of the intimate experience of the subjects in interaction. In other words, for Stern, psychologists need to keep an eye on the psychological dynamics of organisms in interaction. Thus, in his studies of the interactions between infants and their mothers, Stern sometimes uses an experimental design with two phases:

 

Phase 1. The researchers film an interaction between the mother and the infant following a research protocol.

Phase 2. The researcher reviews this film with the mother and asks her what she was feeling at each moment and what she is feeling viewing the film. He also asks her what her infant was feeling according to her. Sometimes this second interaction is also filmed and analyzed.

The proposition organized around Gregory Bateson is commonly known as systemic therapy. It is one of the foundations of family therapy. This type of therapy assumes the behaviors that bring a patient to a psychotherapist are the symptoms of group dynamics rather than only those of an individual. Therefore the group is the focus of the team of psychotherapists. Systemic therapy functions particularly well when other colleagues observe the interactions between the psychotherapist and the family. In the 1960s, the rest of the team observed the session hidden behind a one-way mirror. Today, the teams often use video recordings. As soon as several organisms are perceived on an image, they form an aesthetic whole for the viewer. He rarely differentiates the verbal from the nonverbal or the roles that each one plays. The looks, words, gestures, and postures blend in his mind in many different ways. A hand relates to the words of one and to the crossing of the legs of another. The mind of the viewer seems to isolate some items in the image, without really understanding how, and explores all sorts of combinations that often seem to have some meaning to him, when he does not analyze the details of what he perceives too closely. The impression that the gestures of others speak to us is a necessary illusion of communication, which, as we will see, can never resist the probing analysis of a film.

NONVERBAL COMMUNICATION, OR THE BODY IN COMMUNICATION

 

The positions of the head and of the arms being countless, I will not try to state any rule. (Da Vinci, 1942, The Notebooks II, p. 265; translated by Marcel Duclos)

Nonverbal communication designates all forms of behavior that do not require the analysis of the content of words. This definition implies that all behavior can have (but not necessarily will have) a communication function. The difficulty with this definition is to accept that the organism which activates a behavior, even consciously, is not able to know ahead of time how its behavior will be perceived. The writer often admits with difficulty that his readers perceive some thoughts in his work that he never had and that they will never be able to know the intellectual and affective forces that pushed him to write.3

Bodily nonverbal communication studies analyze all parts of the organism that appear on a visual recording. Such a recording may use all sorts of support, like a film, a video tape, a DVD. Because these distinctions are not important for the following sections, I will use the term film to designate a visual recording capable of capturing the contours of the body and the visible aspects of its texture, whatever the technology that is used. From the point of view of the large categories of this textbook, this means that body and behavior are not always differentiated. On the other hand, it is clear to all researchers that it consists of a partial approach of a whole that is the organism.

An Invisible Visible Body
Structuring How We Look

The individual who can draw knows that the viewer has the impression of seeing everything, but that is not the case. It suffices to try to draw a tree without leaves in winter to discover this. The beginner may try, but he soon discovers a discrepancy between what he sees and what he creates. In the case of the branches of a tree, an art teacher often teaches the student to draw the shape of the space between the branches to be able to situate them. The student is then able to pencil a more realistic drawing. This is an example of why we all need to learn how to look as soon as we need to integrate what we see in a precise motor action.

The study of how eyes scan a visual field is called pupillometrics. This field analyzes pupil size and movement (Hess, 1965). For example, researchers who wanted to understand how children learn to look asked them to count the number of windows on a photo of a large building. The very young child focuses on one window, then on another, and then another. His eyes jump from one window that attracts his attention to another, but he does not have a strategy. After a while, he no longer knows which window has already been counted and which has not. Most of the older children have acquired an automatic scanning procedure—for example, counting by going from the highest to the lowest row, or going from left to right in each column. This type of strategy allows one to know immediately which windows remain to be counted. In this example, we see that the cognitive performance depends on sensorimotor strategies used by the movements that allow the retina to sweep across a surface (movement of the eyes, of the head, sometime of the bust, etc.).4

Necessary Illusion: I See Everything

Researchers like Eckhard H. Hess5 have established a device that allows one to know what an eye is looking at, at an exact moment. He has shown that pupils scan just a small percentage of the surface of an object. Yet a human has, in every instance, the impression of perceiving a continuous visual field. There are therefore editing mechanisms between the retina and a conscious visual perception. They fill in the visual field with information that did not enter through the retina. Francis Crick calls this editing process a form of filling in that is based on “unconscious inferences.”6 This effect becomes tangible when a person discovers some new details by concentrating longer and more attentively on an object. The implication of this finding is that a large part of our conscious thoughts are fabrications: sensory data is so partial that it requires considerable editing before it can form a conscious perception,7 This filling-in plays an important role in conscious perception. Imagine that you could only perceive those parts of the surface swept by the central fovea of your retina. You would only see a few pieces of the surface surrounded by empty space, which would be troubling and would stop you from running when it was necessary.

Pierre Bourdieu (1997) describes another illusion that he calls knowing by body.8 This illusion functions in two steps:

 

  1. Our organism (which Bourdieu calls body) is part of the physical, biological, and social world, but the mind has the impression that it understands (and includes) the world (pp. 156f and 162f).
  2. Our perceptions construct themselves inside the cranium, but we have the impression that they exist outside of the body where the perceived objects are thought to be. This illusion has already been described by Kant: “By means of the external sense (a property of the mind), we represent to ourselves objects as without us, and these all in space” (Kant, 1787,1.1.1.2, p. 43)

These phenomena have a well-known clinical relevance because they allow for the activations of not only projections but also the variants, such as the expulsion of conflicts observed by Racamier in psychotics and perverse narcissists:9 The notion of expulsion which thus appears will reveal itself to be central in the discovery of the concept of narcissistic perversion. It signals a kind of falling-out of a perspective centered on a suffering individual who dumps his suffering on another. In these mechanisms, there are forms of pouring out from the inside to the outside that allow an individual to have the illusion that he can become a parasite to the soul of the other, settle in there, and feed on it. To the extent that this conscious illusion is accompanied by a nonconscious know-how on the prowl, the perverse narcissist can develop a capacity to manipulate and destroy the other that can become terrifying.10

Constructing a Representation of Body Communication

In the previous section, I showed that the perception of the other is not a simple registration of reality. It is at least as complex as what happens in a movie theater. The spectator has the impression of seeing a succession of images, whereas sometimes, between two images, there were months of reflection and crises in the filming crews.

I have already mentioned that the first researchers11 in nonverbal communication believed that to describe the movements of people would be easy. They thought it would suffice to find a system to note the gestures to describe what everybody sees.12 An arm displaces itself in a geometric space; a smile is more or less lengthy, and so on. They even hoped to have found a low-cost way to obtain crucial scientific data on human functioning. They were quickly disillusioned. By reviewing the difficulties they encountered, we are able to begin to question ourselves, in a more precise manner, about the way a perceptual system resolves these problems.

How Does a Brain Record Visual Information?

The first practical challenge that a researcher in bodily nonverbal communication must deal with is to obtain images that he can work with. Even when one only studies an interaction between two people (a dyad), it is horribly complicated, if only for the following reasons:

 

  1. The number of cameras: There must be cameras that film each body face and profile, from head to toe; and a camera that films, from further away, how bodies are situated in relationship to each other; and sometimes, cameras that film parts of the body that one wants to study in greater detail (e.g., the face and the hands).
  2. The coordination of the cameras: The cinema has shown that human perception needs at least 25 images per second to be comfortable with a film. The disjointed movements of the early films of Charlie Chaplin used fewer than 20 images per second. To study an individual with several cameras, the images must be coordinated with a precision of at least one one-hundredth of a second. Again, the solutions to this problem require technical means and competent personnel.
  3. The quality of the film: To observe the large movements of the body does not require high-performance films. On the other hand, as soon as a researcher wants to analyze the varied muscles of the face, he needs to be able to see each muscle. This requires good lighting, the right angle, excellent recording equipment (equivalent to that used by EV cameras). Finally, certain visible body indices are difficult to perceive on film, even when the best equipment is at the disposal of the researcher. I think of the splendor of a glance and the skin, which, after my experience, has an important impact on the way that a person is intuitively perceived. It is rare that a researcher in nonverbal communication can afford equipment that meets these criteria because the opinion persists that studying nonverbal communication is inexpensive. Nobody wants to believe that what he perceives every day in an apparently comfortable manner needs a refined technology to be rigorously analyzed.

These issues highlight the exquisite refinement of sensorimotor circuits, which can deal with such issues rapidly and more efficiently than the most sophisticated laboratories. For the moment, we do not know how nonconscious processes manage this type of information.

How Does the Brain Code Visual Information?

As soon as researchers began to look at a sequence of images many times over, they discovered a lot of things going on in all parts of the body—much more than they thought. They then set about to find ways of codifying everything that could be distinguished in a film. One of the first series of studies was dedicated to the analysis of a film 18 seconds long that showed a family therapy session by Bateson. In this sequence, Bateson lights the cigarette of his female patient.13 Several researchers (e.g., Birdwhistell) studied this sequence in the course of 12 years. They collected so much material that they were never able to publish all of their observations.14 Their system of notation often consisted of written and sketched descriptions of what they had observed, all of which generated an enormous pile of unmanageable data.

This problem is partially resolved since the propositions of Ekman and Friesen (1978) relative to the face and Frey (1985) with regard to the whole body. Their solution is to assign a number to each body part and a number for each possible position of that body part and note the position of a body item (the corner of the lips or a hand) in each image.15 This allows for the completion of a matrix16 in which each line is a moment and each column, a body item. At the intersection of a line and a column, we have a value that localizes the behavior of a part of the body in space and time. Here is an example.

 

Vignette on smiles. A smile: has a moment tl, the corners of the lips stretch a bit. In Ekman and Friesen’s system, a smile is unit 12, and a weak intensity that is barely discernible has an intensity of A. Thus, at the moment tl, the face being analyzed has a 12A.

With this system, it becomes possible (a) to know exactly how many distinctions a coding grid can distinguish, and (b) to deal with this information with a computer program.

The basic matrix contains the data coded by looking at films image by image. This is the coded reality. From there, a series of variables can be defined. Table 20.1 shows a matrix that isolates the coordination between eyebrows and a smile observed on one of the subjects filmed in the Affect and Communication Laboratory (LAC) of Geneva.

An individual who perceives this smile for the first time rarely notices the following details:

 

  1. Only three facial units are mobilized.
  2. The temporal profile of the expression begins by a mobilization of the eyebrows, then the smile.
  3. These details are discerned by the nervous system in a nonconscious way, but are often too detailed to be perceived consciously. Brevity is one of the criteria that permit us to affirm that an activity is nonconscious. If the details of the temporal profile of a facial expression correlate with an aspect of the behavior of other persons present, the researcher assumes it is highly probable these details have had an impact on the perceiver’s nervous system.

TABLE 20.1. Coordination between Smile and Movement of the Eyebrows

Image

Notes: The time scale is of 20/100th of a second. This table describes how eyebrows and smiles were coordinated over eight frames. The units are coded following the FACS of Ekman and Friesen: 1: rising of the internal eyebrow, 4: narrowing of the eyebrows, and 12: spreading of the corners of the lips. S indicates that the movement is symmetrical. The second letter indicates the intensity of the movement, going from A (a hardly visible activity) to E (maximum intensity).

A researcher who applies this type of descriptive system to each part of the body in every film image of two persons in an interaction soon discovers that he needs a matrix capable of distinguishing between millions of possibilities. Only some of these possibilities are used in a particular interaction, but in comparing several interactions, the researcher soon finds himself with more data than he can consciously manage. Such a coding procedure allows a researcher to analyze the density of information exchange between interacting organisms in the course of a one-hour psychotherapy session between two persons.

Statistical Issues

After having discussed issues that relate to data gathering, I now say a few words about the difficulty of managing data on nonverbal behavior. My examples continue to be the analysis of how a psychotherapist interacts with a patient in individual sessions.

Once it became possible to introduce data into computers (in the 1970s), researchers were able to generate two types of matrixes of descriptive data.

 

  1. A basic matrix, conceptualized like the smile, contains everything that has been coded.
  2. Derived matrixes calculate the variables starting from the basic matrix.

For some psychotherapists, the statistical issues I address now may be difficult to grasp. However, I do not have any other means to demonstrate what I must explain. All I can do is to assume that some of my readers have not studied statistics.

THE BASIC MATRIXES

The Structure of the Basic Matrixes. We have just seen a basic matrix of the behavior of an individual. There are as many columns as there are body items, which can be distinguished in a reliable17 way by anybody who has learned how to code body items. It is evident that some individuals perceive a behavior in greater detail than others. Only what can be reliably perceived by most trained coders can be integrated in the research protocols. This implies that small nuances which may be detected only by some cannot be included in this procedure.

In spite of certain inevitable simplification, the researcher is thus able to gather a considerable amount of data and distinctions. Utilizing Ekman and Friesen’s FACS, the analysis of the face allows one to distinguish at least 50 items. If I distinguish everything that is going on in the body with as much detail as I have just described with regard to the movements of the hand, I easily find myself confronted with 103 identified body items. This accounts for the number of columns in a matrix of individual data for one person.

In what pertains to an image by image analysis, every 5/100th of a second, we have 20 lines per second, 1,200 for a second, 720,000 for a minute, up to 43,200,000 for an hour. With 200 columns, that gives us a matrix of 8,640,000,000 cells18 to describe in a detailed way the behavior of one person during a psychotherapy session. If we analyze an interaction between two individuals, we have at least tripled the number of cases:

 

  1. 8,640,000,000 cells to describe the behavior of the second person.
  2. At least as many to describe the relationship between the two persons for each body item (same position, different position, etc.). But this can be done by a computer.

Nonetheless, this gives us 25,920,000,000 cells to form the basic matrix of a body psychotherapy session. Take a study that wants to compare individual psychotherapy sessions with depressed patients and those with nondepressed patients (I will discuss some of these studies in a moment). Statistically, it would require at least 1,000 examples of each type of therapy to be able to complete an analysis that would have a minimum of statistical validity. The researcher would then have 51,840,000,000,000 cells of raw data to analyze.

No research as thorough as this has been carried out in the last decades. I believe I am one of the last to try to carry out a detailed coding of a complete interaction. I had focused on posture, that is, an aspect of behavior that moves relatively little. Before I detail what I have to say on the data management of coded behavior, I need to define derived data matrixes.

The Content of the Basic Matrix. I content myself in this section with evoking what happens when a researcher wants to analyze all the aspects of a body behavior of his patient during a one-hour psychotherapy session. There are then 43,200,000 cells to complete. For the moment, the difficulty is that every cell of a basic data matrix must be typed in manually. To continue with the hand movement example, one has to study each dimension of the movement of the hand, image by image.19 Here are some of the dimensions distinguished to analyze the position taken by a hand at a given moment:

 

  1. Sagittal. A hand can go up or down in space. It can, for example, be above the head, at the level of the face, near the knees, or on the floor.
  2. Lateral. A hand can move from right to left. It can touch a wall at the right of the body, be in front of the torso, or touch an object at the left of the body.
  3. Depth. The hand can be in front of the torso, can touch the torso, or find itself behind the torso.
  4. Rotation. The palm of the hand can be oriented toward the ceiling, toward a wall, or toward the floor.
  5. Closure. The palm of the hand can be more or less open.
  6. Touch. The hand can touch another part of the body, a part of another’s body, an object, the sofa, or nothing at all.
  7. Details difficult to include in one of the preceding categories, like knowing if the hand is trembling or still.

In other words, there would be seven columns in our matrix to identify the behavior of the hand. The coder codes what he sees, one dimension at a time. He codes the position of the hand in the sagittal dimension at the beginning of the film, and waits until there is another movement of the hand in the sagittal dimension. He then stops the image, and notes the new position on the line that corresponds to the moment inscribed on a timer which has been set on each image (e.g., line 10 for the tenth image). When an individual moves very little, the work is quickly done; when he moves his hand all the time, the work of coding can last a week, doing it part-time (no one can code full-time because the work demands too much attention).

A coder thus develops an impression that a person who moves all the parts of the body all the time will require much more work than a person who moves very little. This creates an impression that a person who moves a lot is necessarily tiring for all the brains that perceive his behavior and for the brain that emits all of these gestures. Because of this impression, he ventures the hypothesis that to move all the time is therefore necessarily a mobilization of the attention of all those concerned. In real life, we notice that, faced with an individual who moves all the time, most people distance themselves and quiet their senses to protect themselves, or they are obliged to be completely awake, as happens sometimes with a clown or a comedian. These relational issues are independent of the content transmitted by the gestures.

The first robust conclusions that his type of research allows is that there is so much more going on at the level of behavior than is consciously detected. This is the daily experience of someone who codes films. Each day, he has to concentrate on a few behavioral dimensions; each day he discovers other manifest aspects in the gestures of his subjects that he has not previously perceived.

Today, the researchers hope that new technologies will make it possible to complete these analyses automatically. While waiting for this technology, most researchers focus on a few body dimensions a few moments at a time (typically during a few minutes), selecting samples in a film that seems crucial in a given theoretical frame.

THE MATRIXES DERIVED FROM THE BEHAVIOR OF AN INDIVIDUAL

We are not at the end of our troubles. We must, in effect, construct some derived matrixes220 that will permit us to obtain a set of useful variables that can be extracted by a computer program from the basic matrixes. Here are a few derived variables that are currently being used:

 

  1. Mobility: The software counts the number of position changes for each item of the body (in each column). It will, for example, count the number of times that there was a change in the sagittal position of the hand. The derived matrix will then furnish this result for each subject and show that some individuals have moved their hands from high to low more often than others. It is also possible to generate compound variables, like knowing how many times there was a change in at least one of the dimensions used to described the behavior of the hand, or how many times at least one item of the body moved.
  2. Thickness: The software counts how many parts of the body change position simultaneously. For example, when an individual only moves his hands, the thickness of his behavior is 1; when he moves, at the same time, the hand + the head + a foot + his back, the thickness of his movement has a score of 4. The programs can register in the derived matrix the average thickness of an individual’s mobility. A program may also provide a curve that shows the variations of thickness in time. If I want to analyze an interaction between a therapist and a patient, I can ask the program to produce two curves: the variations in the thickness of the gestures of the patient and the variations in the thickness of the gestures of the therapist.
  3. Complexity: The software counts, for each part of the body, how many dimensions of one part of the body are mobilized. In the preceding section, I identified seven dimensions of the movement of the hand in space (high/low, right/left, in front/in back, etc.). When the movement of the hand is carried out in a dimension (moving upward, for example), the movement has a complexity of 1. If in moving the hand upward, the palm of the hand closes and turns toward the floor, the movement has three dimensions.
  4. Time spent in motion (TSM): The program computes how much time there is a movement in a set of body items (e.g., the activity of the top of the body, of the hand, of the eyes, etc.).
  5. Rhythm. This variable is defined as the computation of the average length of time between two actions. In these studies, rhythm is defined less rigorously than in music. It is defined as the average duration during which no movement occurs.
  6. Individual contingencies: The correlations between the parts of the body of an individual. For example, the software allows one to discover that each time a person smiles, he moves the hand in a certain way in the minute following the smile.
  7. Interpersonal contingencies: The correlations between the body parts of two individuals. For example, the software discovers that nearly every time that a patient smiles the therapist taps his fingers on the chair.

THE ANALYSIS OF THE DERIVED MATRIXES

The Statistical Analysis of the Derived Matrixes. Once a researcher has a series of derived matrixes at his disposal, he finds himself faced with an enormous quantity of data, which his consciousness is incapable of managing. He therefore uses statistical programs that try to extract, in function of certain rules of research, particularly characteristic schemas that correlate with known variables. By “known variables,” I mean variables obtained independently of the films. For example, some relatively reliable tests allow one to determine a patient’s degree of depression and of anxiety. Some software can then search out which behaviors, in the patient and therapist, correlate with the results of the tests of depression and anxiety of the patient.21

The difficulty (not yet resolved) is that the current statistics have been constructed to find significant correlations and differences when the researcher has, at his disposal, a small sample of data on a large number of subjects. In nonverbal communication, we have the opposite situation: an enormous amount of data obtained on a ridiculously little cohort of subjects. Therefore, when one asks a statistical program to find correlations and differences, it often finds a great quantity of highly significant results. The only problem is that the mathematical bases foresee that when the statistical tests analyze a great amount of data obtained on small samples, it will find a large number of apparently very significant results that are, in fact, due to chance.

We find this difficulty in many domains that manipulate a large amount of data. This is the case in the new explorations in the neurosciences that generate photos of brain activity or of the research that finds correlations between cancer and just about anything. Evidently, new technologies allow us to scan reality in a more refined and automatic fashion. To deal with this type of data, we need statistical theories and procedures that do not yet exist. This is one more reason not to throw ourselves into immense coding projects of nonverbal behavior. Even if we have the data, we are incapable of using them other than as sources of inspiration that allow us to present apparently convincing beautiful graphics. The same problem is evident in the politics of health care. Some statisticians gather an impressive amount of data from practitioners (by imposing routines that are often a burden). Most of the time, once these data have been gathered, the only thing they can do with it is produce beautiful graphs that confirm the political position of the health systems. These statistics are presented to some political parties and the population as “scientific” proof of the soundness of the stated policies. In fact, this is often nothing other than data manipulation, which shows that with statistics an expert is able to demonstrate many things.

To get out of this bind, the hard sciences rely on the replication of specific results to render a finding as robust as possible. Once a computer program has generated a list of “highly significant” results, research laboratories can try to replicate one of these results. They can use only a small portion of these results, using longer film samples and more subjects. This requires the participation of several laboratories that are as independent of one another as possible.

Paranoia. The research procedures that I have just described can be used as a highly simplified version of what a brain must be able to accomplish before it can generate a conscious representation of what is happening in an interaction. A clinician may also draw inspiration from these procedures to revisit the notion of paranoia. The paranoid individual (with delusions of persecution) is traditionally considered as someone who “hallucinates” connections between events that allow him to think that everybody (or a group of individuals) wants to persecute him. However, the statistical procedures we have just taken into consideration show that relatively coherent procedures make it possible to lead one to believe that there are coherent schemas that are actually products of the mind. There is no need to project or hallucinate to find schemas that feed the fear of being persecuted. It suffices to take as fact some rationally constructed observations. Furthermore, it is also true that some groups do try to persecute or manipulate other groups.

A true researcher is someone who knows all that and is not content to find some facts that seem convincing. That is why the ethics of science has developed a series of procedures (like having the replication of one’s experiments conducted by somebody one does not necessarily agree with) to try to find, in the plethora of “truths” produced by the human imagination, a few islands of formulations that are as robust as possible. The notion of objectivity implies that a formulation is confirmed by heterogeneous procedures, one different from the others.

Situations and Dyads

Social psychology, at the beginning of the 1950s, sought to show that a group of interacting persons will generate behaviors that the individual members would not have if they were in another group. In other words, “groups have properties different from those of their parts taken separately” (Asch, 1952, p. 141).22 This type of research showed that the dynamics and organization of a group could influence the behavior of a member independently of the specific traits and conscious will of the individual.

THE SITUATION: AN INTERPERSONAL SPACE

 

We start with the bare observation that a number of persons will, in a given situation, perceive objects and happenings within it in a similar way; and that modes of action in the situation will also have a basic similarity. The tree that I see others see too; what I hear they hear. Indeed, to every change and gradation of my perception there correspond similar changes in the perception of others. (Solomon Eliot Asch, 1952, “The Individual and the Group,” p. 146)

Imagine that you want to film five persons about to have tea together, and you have no impact on what is going on because you and your cameras are invisible to them. You will nonetheless have to take the shape of the room, the lighting, and the objects present into account. When we later view what you have filmed, we notice a variety of events. Some objects are touched but do not move (like the carpet on which the individuals have their feet and the chairs on which they are sitting). This forms the world of the support and basic postures.23 Other objects move, like the door and the windows when they are opened or closed, the tea cups and the sugar, the cigarettes (if there are any). Then there is the visible and audible surface of the behaviors, as well as clothing. After having coded an hour’s worth of interactions, frame by frame, taking into account all the visual and auditory information, you would have acquired tens of millions pieces of information. Some of the gestures and postures you observe are easily understood: like the fact of putting or not putting sugar in one’s tea, certain gestures that emphasize what is being said, the orientation of the gaze toward the person to whom one speaks, to stand up to air out the room, and certain expressions that ostensibly express a feeling. Other data are more difficult to identify but are perceived intuitively by all who look at the film, such as behavior styles. Some hold the saucer with the cup on it; others take hold of the cup directly. Some are more extroverted than others; some interrupt more often; some agree with what is being said while others prefer to contradict.24 Then there are all sorts of gestures whose meaning is more difficult to grasp, like habits and gestures that mostly look like random discharges of the nervous system: to scratch, blink, lick one’s lip or tap a foot. As I have already explained, you are not able to assign a meaning or a robust function to more than 1 percent of what is coded; and the explicit consciousness of the individuals who participate in the gathering will not have registered more than 1/lOOOth of what you see and hear in the film. The whole of these phenomena form what we can call an interpersonal space or a situation.

It then becomes evident that the totality making up the situation is structured relatively independently of the dynamics of each organism. A situation has a bit of the same structure as that of the stage direction of a play: it can be maintained for years on end, even if the actors change. Some actors are better than others, but the nature of the play remains. A situation is animated by a multitude of dynamics quite distinct from the dynamics of the psyche. Psyche and situation interact in a continuous fashion by going through the behavioral dynamics of each one but also by the dynamics induced by the size of the room, the lighting, and the objects. What emerges from all of this (that is, the situation) will also have properties that are more than the sum of each individual behavior. To understand the dynamics of a situation requires more than the analysis of each organismic dynamic.

THE DYADIC DERIVED MATRIXES

Given the cost of conducting a study of a situation, several researchers limit themselves to the study of the situation involving a dialogue between two individuals, currently called dyads. I have already indicated that this reduction is essentially due to methodological limitations, like the difficulty of coordinating the filming of several cameras, the limits of statistical methods, and the lack of methods capable of automatically coding all of the variables that can be obtained from the filming of an interaction.

These methodological constraints encouraged the study of the interactions that occur spontaneously in didactic fashion, like individual psychotherapy25 or the mother-infant interaction.26 In this context the derived dyadic variables were created. The basic notion is quite simple. Consider a typical research issue as an example: the hypothesis that depression creates a decrease in motor activity.27 The researcher who wants to verify this hypothesis often employs three types of experimental strategies that compare individual psychotherapy with manifestly depressed patients (DP), and individual psychotherapy with patients who are no longer depressed (HP):

 

  1. Influence of pathology on the patient. To know whether there are some common points in the behavior of depressed patients, the researchers will, for example, compare the number of movements made by DP patients, and the number of movements made by HP patients. If the first patients move less, the hypothesis that depression slows motor activity is confirmed.
  2. The influence of the patient on others. The researcher can also study the impact of the patient’s depression on the behavior of those with which he interacts. He will then compare the therapists of the DP dyads with those of the HP dyads. If the first therapist moves less than the second one, the researcher is able to speak of some kind of contagion from the depression on the therapist. To understand this contagion, he can then establish more exact correlations between the mobility of the therapist and that of the patient. If he observes that the less a patient moves, the less the therapist moves, the researcher is able to conclude that the therapist has a certain way of reacting to the slowing down of the patient’s motor activity. Whereas if he observes that the less a patient moves, the more the therapist moves, the researcher is able to conclude that the therapist has another way of reacting to the slowing down of the patient’s motor activity.
  3. The influence of the patient on the structure of the interaction. Finally, the researcher is able to create some dyadic variables, as the sum of the movements made by both protagonists. He then compares the mobility in the DP dyads with those in the HP dyads. Since some researchers28 have not observed clear differences in making the previous comparisons, they sought to discover if the inhibition of the body mobility influences at least the therapeutic dyad, taken as a whole. They can then see whether there were dyadic differences. If they find a clear difference with these data, they are going to conclude that the depression creates an atmosphere in the room that can slow down the motor activity of the therapist, or of the patient, or of both. The impact may vary from one dyad to another, but in all cases, the dyadic mobility will be lower in the DP dyads than in the HP dyads. This result reinforces the impression that depression influences the emergence of the structure of the dyad as much as the individuals who are part of it. If only the results of the third type are observed, they are difficult to explain by the individual variables alone. There is then necessarily an interaction between the state of the patient and the organization of the interaction.

An example of dyadic data was published by Siegfried Frey and his team.29 It consists of a study that analyzes the stability of basic postures and hierarchical status. In this study, the researcher compared situations where female students were in a discussion with other female students and situations where the female students were in a discussion with university teaching assistants.30 In the dyads with the assistants, the postures were somewhat more stable than in the dyads with other students.31 This difference was not created by the students or assistants, because it was not always the same protagonist who had a particularly unstable posture. An exact formulation would be that in the dyads with the inequality in status, the probability is greater that one of the two protagonists at least would have a stable posture, or that in the dyads with equal status, the protagonists change posture more easily.

Researchers who have been influenced by systems theory have tried to further understand this third type of data by speaking of dyadic effects. They hypothesize that the connection between the behaviors of several individuals can be as indirect as the relationships between the dimensions of an organism. A gesture does not always directly influence the other individuals in the group. It primarily influences the structure of an interaction; that change then influences the other person. The difficulty with this type of model is that no one knows if there is a group entity capable of perceiving a gesture. We must then suppose that the protagonists of a group seek to maintain group cohesion, and they will therefore react to a gesture that risks reinforcing or weakening the coherence of the group. Let me be more explicit. I can react to an individual who stands up because he is expressing a feeling while rising (there is then a direct interaction between two organisms) or because he is about to end a discussion (the change of ritual provokes the reactions).

Having specified what is meant by the organization of an interaction, researchers were able to refine their analysis. They are no longer content to simply add up individual behaviors. They move, above all, to analyzing the way an individual behavior inserts itself into the organization of a dyad. I will provide examples in my discussion of Stern, Beebe, and Tronick (chapter 21). Here are two typical dyadic variables:

 

  1. Software is able to compute how many times the protagonists look into each other’s eyes.
  2. A program is able to calculate the number of times a person interrupts the other.

In these two examples, it is the relationship between the two persons that is studied. These data could not have been analyzed if the camera had only filmed one of the two protagonists. Another kind of dyadic data is the temporal organization of the behaviors of individuals in interaction. For example, Joseph Jaffe and his collaborators (2002) analyzed the rhythm in the exchange of sounds between newborns and adults. Certain relationships have a faster rhythm than others.

In the case of studies on the relationship between a mother and a newborn, the infant does not have the means to structure the relationship in a conscious way. He reacts in a local manner to each event, without being able to become aware of what constructs itself between him and his close ones. Research shows that even in adults, setting up the organization of an interaction is mostly non-conscious. This organization is structured by a few conscious variables for the adults (e.g., cultural norms) and by a succession of events that organize themselves without anyone really understanding how. Individuals have the tendency to feel that they are making a particular gesture for a conscious reason and another gesture for another reason without being able to understand what is organizing itself (in them and around them) while all of these local actions are taking place. Mary Catherine Bateson (1994) analyzes this type of schema between behaviors and thoughts by showing that people often have the impression that they are improvising. In systemic therapy for couples in crisis, this lack of mastery is particularly striking.

An example of the dyadic variable often used in these studies32 is called the measure of contingency by Beatrice Beebe. It consists in finding what gesture often correlates to another gesture: if we see a body item appear in a given context, can we predict which other body item will appear at the same time? Beebe distinguishes between two types of contingencies:

 

  1. A self-contingency: A body item is often associated with another body item in the same body. For example, almost every time a subject smiles at another person, she frowns.
  2. An interpersonal contingency: A body item of a subject predicts the activation of a body item in another person. For example, it often happens that when a subject smiles, the other also smiles.

This allows for the distinction between self-regulation and interpersonal regulation, but it also describes a type of auto-regulation that may have an impact on those who perceive it. This type of auto-regulation is ostentatious to the extent that even if it is activated in a nonconscious way, it is part of the mechanisms that regulate the interaction. The research studies that Beebe et al. (2010) refer to and quote report forms of contingencies between a baby and mother that construct themselves in less than half a second.

The difficulty with the dyadic effects is that they have become central in the current research for purely methodological reasons. We will see, in speaking about Stern, Beebe, and Tronick, that certain researchers believe dyadic interactions are a basic building block of human interaction. There is a kind of theoretical bias in this. It uses the methodological limitations that make the dyad central to promote an ideology according to which the mother is uniquely responsible for the welfare or discontent of the infant. This point of view is not false, but it can be more or less relative. In psychoanalysis, the importance given to the mother-child dyad is special to the researchers in the United States; in Europe, and in Freud’s work, the father often predominates: “This father, whom he could not help hating as a rival, was the same father whom he had always loved and was bound to go on loving, who had been his model, had been his first playmate, and had looked after him from his earliest infancy: and this is what gave rise to his first conflict” (Freud, 1909b, III, p. 134).

DOES A DEPRESSED PATIENT MOVE LESS OFTEN?

I now give the example of the correlation between the complexity of movements and depression, which will show the advantage for psychotherapy of distinguishing between behavioral dimensions, such as they are computed to generate derived matrixes. In the preceding section, I described hypothetical studies regarding the connection between depression and mobility. Here, I mention a few studies on this phenomenon to show what research can bring to psychotherapy.

For a century (at least), it has been recognized in psychiatry that patients suffering from a major depressive disorder with melancholic features move less than most other persons, except those in a catatonic crisis. Some psychiatrists think the reduction of motor activity can be observed with most depressive patients,33 whereas speeding up motor activity is a sign of anxiety or mania. This phenomenon has been effectively observed by some researchers, but not by all. It is therefore possible that depression does not act directly on mobility but on something else.

In a series of research studies on depressive behavior, Heiner Ellgring (1989, 1990) observed a drop of mobility linked to depression, but this drop is often local. Certain parts of the body move less in almost every case, but it is not always the same ones: less smiling (61 percent), smaller number of gazes oriented toward the other (58 percent), less facial activity (31 percent), or less verbal activity (33 percent). Ellgring did not observe systematic differences when he analyzed the variety in the facial repertoire, the expressions used, or the gestures associated to speech. There is a correlation between depression and slowing down in at least two of these dimensions among 70 percent of patients. Consciousness generally seeks some solid associations between behavior and psychopathology. Most psychotherapists would very much like researchers to show that each time a person does not move very much, one can conclude that there is depression. Research confirms that the link between behavior and psychopathology is always fuzzy. For example, a patient perhaps smiles less than usual when he is depressed, but nonetheless he smiles more than other persons. This leads to two types of differences between groups:

 

  1. There is an absolute difference when most of the members of a group behave differently than all the other members of another group. This kind of difference is observed more often in the studies on cultural differences than on the differences between psychiatric diagnoses.
  2. There is a relative difference when it is not possible to differentiate all of the members of one group from the members of another group but that, on average, the difference goes in the same direction. The relationship between athletic performance and gender is a very well-known example. It is possible to state that on average men have a more powerful body (e.g., from the point of view of respiratory capacity) than women do. It is also manifest that there are many women who have a more powerful body than that of many men. It is probable that the best male tennis player will always have a more powerful game than the best female tennis player, but I could never compete with a female tennis champion. The difference is relative because in each group, the variance is enormous. In clinical research, the behavioral differences are generally relative, as in Ellgring’s study. These relative differences are difficult to grasp consciously, especially when they become as diffused as in the observations I have just summarized.

A similar result to those of Heiner Ellgring was found by Beatrice Beebe et al. (2010) in a study on the interaction between four-month-old babies and their depressed mothers. These results concern the analysis of the contingency of movements. She noticed a polarization between the auto-contingency and the interpersonal contingency, which are both particularly elevated and particularly low in depressed mothers. This observation contradicts the hypothesis that the more there is contingency between a mother and her baby, the more constructive the relationship. The nondepressed mothers that Beebe and her team observed have, on average, a moderate variable contingency.

In research by Siegfried Frey and his team34 on the correlation between the behavior of a patient and the seriousness of the depression, he distinguished the impression of a slowing down in the consciousness of the psychotherapist and a real slowing down in the patient.35 The patients observed suffer from a “major depression,” which is considered less severe that melancholia. Software generated tables for two distinct variables, computed for each part of the body:

 

  1. The time (TSM) during which at least a part of the body moves.
  2. The average complexity of the mobility, defined as the number of dimensions mobilized by each part of the body, on average.

For this study, Frey and his team coded the position of the head in function of the following dimensions:36

 

  1. Sagittal. The head moves vertically upward and downward.
  2. Rotation. The head moves horizontally toward the right or the left.
  3. Lateral. The head leans to the right or to the left.

When a movement of the head mobilizes one of these dimensions, its complexity is 1; when a movement of the head simultaneously mobilizes three dimensions, there is a complexity of 3. Someone who uses only one dimension in a continual fashion with each part of the body has an average complexity of movement of 1; a person in whom all the body parts simultaneously use all of the conceivable dimensions has an average complexity of movement of 3. Most subjects displayed behaviors that are situated between these two extremes and displayed a distribution of the complexity of movements that varied for each part of the body.

In distinguishing the duration during which a movement occurred and its complexity, Frey and his team were able to demonstrate that the depressed patient moves about as many times as his physician does, and sometimes as often as after he is healed.37 On the other hand, when they were depressed, the behavior of the patients was less complex. For Frey and his collaborators (1983), the interest in this observation is twofold. First, they introduce the notion of complexity into the psychiatric clinic and allow for a greater precision in the analysis of what is happening. Second, they show how the consciousness of a psychotherapist simplifies what he observes. Only the experts in the study of movement are able to consciously perceive the degree of complexity in the movements of the body. Even for them, this analysis requires an intense attention that rapidly becomes exhausting. However, the nonconscious processes of the therapist (neurological, for example) seem to be sensitive to this complexity. They transform this information into an impression of slowness for the conscious perception. Thus, even when the whole community of psys agrees that they perceive a motor retardation in most depressed patients, they can be suffering from an identical illusion. It is worth noting that most of the time this impression that there is motor retardation suffices for the current needs of the clinician.

These results show why psychiatrists have had so much trouble identifying the bodily and behavioral contours of a psychopathology without the help of systematic studies such as those mentioned here. This type of result supports the suspicion I have often expressed in this book with regard to the belief that there exists direct connections between style of behavior, body shape, and mental problems. The greater the advances in research, the more it becomes evident that conscious observation is incapable of understanding the details of a motoric (or enacted) pattern. Conscious observation can only content itself with simplified data to elaborate its own reflection.38 This explains the proliferation and the popularity of simplistic models that make everybody believe they are capable of mastering this complexity. The connections are so complex that today’s researchers are only beginning to elaborate a theoretical stance that could allow them to understand the dynamics of nonverbal communication. Other resources are necessary to transform these studies into a domain of robust knowledge. On the other hand, research confirms that the therapist can often have confidence in his intuition that when a person is depressed something is slowing down in him and the interaction he has with depressive patients. This fuzzy impression would be a summary of what the brain discovers in a nonconscious way. On the other hand, as soon as the therapist attempts to explicitly identify and explain what causes this impression, he often has recourse to simplifications that do not stand up to rigorous observation.

The Body without Sound

When a researcher shows his films without any sound to a group of people who do not know his research, each one proposes personal observations that could explain the meaning attributed to gestures. Each individual notices immediately that the others are making different associations. As soon as the researcher turns on the sound, the meaning of the words varied much less from one person to another. When the researcher reveals the setting in which the film was shot, these individuals immediately distinguish a series of details that hitherto had meant nothing to them and now give a meaning that allows them to support their representation of the situation. Often, the researcher must then explain why this new series of associations are frequently false, even if all the members of the group had spontaneously agreed with each other.

This typical observation teaches us many things. I isolate three of them for the moment.

Only a Few Highly Standardized Gestures Are Understood by Several Persons the Same Way

Research indicates that the body signals to which it is possible to assign a robust meaning are rare.39 Perhaps less than 1 percent of the gestures observed in a film during a spontaneous interaction can be included in that category. But they are the gestures that are retained in the conscious thoughts of the observer.

Speech Often Helps One Understand What Is Happening at the Level of Behavior or at Least Makes it Possible to Think that a Motor Behavior Has a Precise Meaning

Video analysis allows us to observe how a mother acts differently when she makes a mechanical gesture with the baby without talking to him compared to when she accompanies her gestures with appropriate words. If she puts a spoon in his mouth while watching television, the gestures are often awkward; the coordination between the movements of the baby and the movements of the mother is poor. If she takes the trouble to speak to him, by describing to him what she is about to do while feeding him (e.g., by saying something like “here is something to eat” or “I am bringing the spoon to your mouth”), she becomes more attentive. By speaking with the baby, something in her is better coordinated. As a conscious thought may be used more or less in a central way by a propensity, a gesture may be related in a more or less intense manner with consciousness. By addressing herself to her child, the mother’s behavior acquires a better coordination with the infant’s responses.

A striking example of movements that take on a meaning after the sound is added, is the research by Siegfried Frey and his collaborators (1980) on psychotherapy with depressed patients (e.g., in which he analyses the complexity of movements). These researchers analyzed interviews between medical psychiatrists and depressed patients in psychotherapy. Each therapeutic process is filmed twice: at the first and last sessions. Frey, Jorns, and Daw (1980) notice a striking difference between the first and last session. In the first, the patients shake their head in the rotational dimension (side to side) and the physicians mostly shake their head in the sagittal dimension (up and down). At the last session, the effect is inversed. The patients mostly move their head up and down while the physicians move it from right to left. How would you explain this difference? Having asked this question of several persons, the researchers noticed the probability that someone, even an expert, would arrive at the right answer almost by chance. By turning on the sound, everything becomes clear:

 

  1. The first session:

    a. The therapist does not cease saying “yes” that life is worth living to give hope to the patient.

    b. The patient does not cease saying “no” to the encouragements of the therapist by denying that there could be any positive event on this horrible planet dominated by a cruel human species.

  2. The last session:

    a. The patient does not cease saying “yes, you have been successful in treating me” to his therapist. He shows a profound admiration for his therapist, characteristic of a slightly manic state a patient can be in when he comes out of his depression.

    b. The therapist does not cease saying “no” to this enthusiasm, emphasizing that he has only carried out his duty.

To show that this apparently insignificant aspect masks profound mechanisms, consider the following:

 

  1. The fact that software programs more easily manage coded behavior than conscious perception has made it possible to demonstrate that these moments have a particularly convincing correlation with the amount of patients getting well. The analysis of coded behavior during the interaction demonstrates that these are key moments.
  2. This observation shows very well that these key moments are dyadic. We observe a change not only in the patient but also in the therapist. Not only is the patient better, but his impact on the physician is different because the physician reacts automatically, without knowing it, in a relatively standardized manner to this improvement.
  3. This observation can be quite useful to demonstrate a well-known phenomenon by clinicians in the training of psychotherapists. The tendency to console depressed patients and to have a fear of compliments is an often mentioned mechanism. Having said that, I must compare two types of observations: (a) the experienced psychotherapist often refuses to console a depressed patient and finds it important to accept compliments, and (b) young therapists often have better results than more experienced ones. It is possible that in falling into some traps, beginners allow some relational mechanisms to establish themselves in such a manner that that they are more effective for reasons that no one yet understands. It would be interesting to take up this experiment by differentiating between experienced therapists and those still in training.
A Gesture Rarely Attempts to Convey a Meaning, but It Often Has an Impact

 

He who will know fully the vanity of man has only to consider the causes and the effects of love. The cause is a “je ne sais quoi” (Corneille), and the effects are dreadful. This “je ne sais quoi,” so small an object that we cannot recognize it, agitates a whole country, princes, armies, the entire world. Cleopatra’s nose: had it been shorter, the whole aspect of the world would have been altered. (Blaise Pascal, 1669, “Pensées,” 162, Kindle 1083)40

To introduce you to the realm of questions I wish to discuss in this section, try to understand the meaning of the gestures displayed in figure 20.1.

The first row of photographs in the figure shows “a small girl sitting in the audience of a theatrical performance.” In the second row, “a girl watches a woman with a baby.” These gestures seem to make immediate sense. However, if several experts try to explain the need to behave in this way in such circumstances in Bali, they will soon discover that such images are like a Rorschach test. Their spontaneous explanation provides more information on how each expert thinks than about the actual event. For example, as soon as these behaviors are presented as typical in a culture, experts may develop a different set of explanations than if they were asked to explain individual motives.

Image

FIGURE 20.1. “The sensory titivation and fidgeting with the hands in the area of the mouth and nose” when observing other people moving, in Bali. Source: Bateson and Mead (1947, plate 25, p. 104).

When one uses video for psychotherapy, one constantly needs information from the patient to understand the gesture. Then one uses a first approximate analysis of the gesture to explore new aspects of the patient’s thoughts. Going back and forth between a video, the impressions of the patient, and those of the therapist can become incredibly fruitful, if one is willing to accept that this is a process, an exploration. The threads that move a marionette can be attached to hundreds of different mechanisms. Balinese Character (Bateson and Mead, 1947) is an excellent place to begin, if one wants to acquire the type of perception that is required to nonverbal interaction.

GESTURES RARELY “SPEAK” . . .

We have just seen that, isolated from their context, gestures, mimics, and postures rarely have an obvious meaning for conscious thoughts. They mostly serve as support for an associative system that rapidly transforms itself in to an interpretive delirium. The more we question a person concerning the impression a gesture makes on him, the more he discovers that his first impression is nothing other than a “filling in,” and this filling in is a mental construction that reinforces the necessary illusion that everything has been seen and understood. Therefore, people who observe a gesture have the tendency to attribute meaning to it that is part of their current beliefs. Let us consider an example.

 

Vignette on suicidal behavior. At the end of the 1990s, most psys believed that the more an individual expressed himself, the healthier he was becoming. At that time, the laboratory where I was working41 was analyzing the behavior of suicidal patients in the hope of finding behavioral indices that would permit us to predict the risk of future suicide attempts (Heller et al., 2001). When I presented our results to colleagues, they were astonished that the individuals at a high risk of suicide were more expressive than the individuals at low risk. Yet this is easily explainable. These individuals had been brought to the hospital after a suicide attempt and had been filmed during their interview with a psychiatrist in the emergency department. Some felt that their attempt was a mistake and that they would not do it again. This was an episode to forget. They did not feel any need to develop a connection with the psychiatrist. Those who still experienced a need to kill themselves felt that the physician could possibly help them. The patients in the second group therefore had a greater need to interact with the psychiatrist.

Most of the gestures that are made influence the nonconscious processes of the individual with whom an organisms interacts. This impact can influence the protagonists in less than 250 milliseconds without them realizing it.42 Even if the subject does not perceive the details of these stimulations, they often create in him a first impression that influences the way he spontaneously reacts. Such a first impression of another person can persist for years.43 When an individual has the impression of being able to evaluate a candidate for the presidency based on television images, he has the impression of expressing a well-founded personal opinion. Siegfried Frey’s research shows that such an intuition is often as strong, powerful, and lasting as it is unreliable.44 The power of this impression allows citizens to think that there is no need to inquire any further to know the worthiness of a candidate before voting for or against him.

. . . BUT THEIR IMPACT CAN BE MORE OR LESS PREDICTABLE

An individual is entitled to expect that certain behaviors would have a relatively predictable effect on self and his entourage. For example, recent research has shown that children need to find islands of predictable communication that permit them to have the impression of acquiring a form of mastery over what is happening. Some psychotherapists who defend the notion of a self relate what is predictable to the notion of meaning. This meaning is not necessarily conscious. It may even be nonconscious in the case of automatic interactive exchanges. In the domain of those schools of psychotherapy that take the body into account, this perspective can be found in the writings of psychotherapists such as Beatrice Beebe45 and Rubens Kignel.46 In both cases, the idea is that the more contingency there is, the more the impact of a gesture acquires meaning for the mind. It is therefore possible to speak of a fuzzy or soft semiotics for ordinary gestures, in contrast to signs that are explicitly associated to a meaning.

This meaning of a gesture constructs itself in interaction with the person’s surroundings.47 A newborn may produce a reflex movement that the adult (e.g., the mother) perceives as a smile. The newborn then experiences an affective response from the mother that is accompanied by a mimic of what she feels to be a smile. The infant thus learns, by imitation and positive reinforcement, what may happen when he smiles.

Rubens Kignel (2010) refers to Pierce and Ecco to assume that such habits, constructed with an interactive frame, are the basis on which semiotics are co-constructed. As in Piaget’s theory of intelligence, these constructions may at first be nonconscious sensorimotor schemas, but the more co-conscious they become the more psychological meaning they generate. This position is not necessarily in contradiction with the notion that some emotional forms of expression have an innate core function. Nevertheless, it imposes the idea that even innate emotional mechanisms must be calibrated in function of contextual, relational, and cultural dynamics. One of the by-products of this calibration process would be the emergence of increasingly meaningful forms of habitual interactive skills, which can sometimes acquire a symbolic configuration.

In his example of the smile, Kignel quotes researchers for whom positive reinforcement is a key motivator for the development of these co-constructed skills. I have noticed with some of my patients that this is not necessarily the case. The prime motivator was, in those cases, a need for predictability. In my experience, this need is deeper than a need for kindness. In some families, the only predictable interactive skills that could be constructed were negative. Because the infant needs to create some form of attachment to his parents and construct an honorable self-image, he will then take these skills as proprieties of his self-image. I have noticed how attached these patients can be, once they are adults, to these predictable interactive co-constructed schemas . . . even when they are destructive for themselves as well as for those they live with. The best known example is that of children who had been beaten and then beat their own children.48 It is then extremely difficult to modify these habitual “meaningful” habits, because they are the unstable sand on which such persons have built their sense of identity. The person grew up investing a huge amount of energy to find ways of surviving in such an environment.

I then observe an internal conflict between (a) an immense aggression against abusive patents, and (b) an attempt to preserve an honorable loyalty to the habits and customs of the abusive family. Having become adults, these individuals may not have enough energy to change their functioning style when they discover that they are able to live in other less hostile environments.

Kignel (2010, p. 18) also discussed gestures produced by a body without any intent to communicate. As soon as they are explicitly perceived by another person, a meaning is often automatically attributed to them. Even when the attributed meaning has no connection to the mechanisms that produced the behavior, it may become one if the entourage begins to incorporate certain repetitive signs into their common communication network. In my research studies on nonverbal communication, I have noticed that most of the bodily activities I coded fit into this category. In our study on the risk of suicidal recidivism (Heller et al., 2001), we observed that patients made a large variety of movements of the lips, which had no discernible communicative relevance. But as soon as our studies demonstrated that there was a correlation between the frequency of these lip movements, very different from each other, and the risk of suicide, we gave them meaning for our psychotherapeutic colleagues. When I perceive them on a video, I think immediately of the person’s potential risk of suicidal recidivism. This does not mean that these movements have been generated to communicate a risk of suicide to others. We have been able to establish that the therapists of these patients do not notice these little rapid and often insignificant gestures, that they do not consciously correlate these gestures with suicidal risk.

It often occurs (a) that a patient’s familial milieu had contributed a meaning to these multiple gestures that the body ceaselessly produces, and (b) the patient then attributes this meaning each time the gesture happens. There is then a loss of the freedom of mobility. This loss can have a negative effect on the organism when the inhibited gestures are part of the regulatory systems. In other cases, this inhibition permits a calibration of the behavior that allows for a better social integration. Often, as we have already seen, the different dimensions of the organism have some heterogeneous goals that can become contradictory. What is good for social integration is not necessarily good for the health of the organism, and vice versa.49

The psychotherapist who is confronted by the “false meanings” the patient has introjected often has the need to sow doubt in the patient, to show him that some of the explanations he attributes to these gestures freeze certain regulatory mechanisms of his organism, notably in the mind. To free consciousness of false explanations about self is as important as showing that some behaviors have an implicit meaning. My patients often have an entire network of false explanations concerning being imprisoned by their body—their weight, their appearance, the grace of their gestures, and so on. To allow them to rediscover their body independently of these aesthetic criteria and of the auto-evaluation that have structured their super-ego is also a task of psychotherapy. One of the strategies used in these cases is to relearn to feel the impact of one’s behavior on one’s vegetative sensations. The patient is thus able to reflect on the conflicts within him between his somatic needs and those of a social image. He may then search for compromises that he would be better able to take on than before.

This way of defining the meaning of gestures by their predictable impact has a certain but limited usefulness.50 A gesture may have a function for self, another function for someone else, and a function in dynamic situations. It is only in the case where the three functions are congruent (or at least complementary) that this type of meaning could become useful and constructive. As soon as each one perceives the meaning of a gesture differently, the notion that the function of a gesture can be used to grasp its “meaning” loses its relevance. If the notion of familiarity with certain behaviors and their impact is an important key, it must still be deepened before being able to lead us anywhere. We will see, for example, that it is also possible to take predictable sequences of behavior in a relationship as a relevant functional clue. It is always good to remind ourselves that even a professional actor needs a director to know how his behavior is perceived by others.

In the course of a session, a psychotherapist is bombarded with information. The therapist tends to focus on the behavioral items of the patient that resonate with his affective impressions and clinical preoccupations. He may notice that he experiences sadness when the patient talks in a certain way and anxiety when the patient moves in a certain way. Noticing that the patient is enraged when he mentions certain events of his life is one thing, deciding if exploring this rage is therapeutic is another. By comparing several sessions given by several therapists with numerous patients, research studies make it possible to calibrate the attention of the psychotherapists on the elements that seem truly pertinent. Some of these are not expressive. They belong to the category of apparently meaningless gestures. In the case of our research on suicidal patients, the therapist experienced a variety of emotional impressions, such as anger and sadness, that did not correlate with suicidal risk. We were able to show that the impressions that correlated with suicidal risk were mostly the feelings of comfort and discomfort, self-confidence and lack of confidence.51 This study needs to be replicated before these findings can be used in a reliable way by psychotherapists.