IMMERSION
A Tale of Two Meanings
Immersion as Illusion
In their effort to assemble the different theories of immersion into a joint framework, Frans Mäyrä and Laura Ermi have proposed the SCI model (2005), where three types of immersion are defined and intersect in the gameplay experience: sensory immersion, challenge-based immersion, and imaginative immersion. Although there is some overlap between the elements of this segmentation, it will be a useful entry point into the semantic web of immersion.
Of Mäyrä and Ermi’s three types, “sensory immersion” is the one that corresponds the most to the original meaning of the word. “Immersion” comes from the Latin “immersio,” meaning to submerge a body in water. It has been used metaphorically in the context of cultural and linguistic exchanges, referring to the feeling of being enveloped by different social norms and engaged in an intense learning situation. It is also associated with the feeling of being transported into a non-immediate reality in the context of mediated representations. In these cases, it is generally linked causally to the degree of vividness or credibility of the represented reality. The development of new interfaces in the military and scientific contexts of the 1970s onward brought the term to prominent use, along with relative concepts such as telepresence and presence. In their literature review, Matthew Lombard and Theresa Ditton (1997) highlight this overlap between the various usages; the fourth definition of presence is synonymous with immersion. “Perceptual immersion,” the authors note following Biocca and Delaney, “can be objectively measured by counting the number of the users’ senses that are provided with input and the degree to which inputs from the physical environment are ‘shut out’” (1997). This definition of immersion or presence is rather mechanical: a “reality engine” produces illusions, and the perceptual saturation—the number of senses that are addressed, as well as the quality of the illusion—exhibited by this device determines the user’s immersion. In this view, IMAX movies are very immersive, interactive virtual reality apparatuses are leaps ahead, and a comic book is not very engaging.
This type of immersion and the media that were created to maximize it have been studied extensively by Oliver Grau in his book Virtual Art: From Illusion to Immersion (2003). Grau inspects a variety of art practices that are all based on the same principle: surrounding users completely in a space of visual illusion. This immersive strategy is the main focus of the book; it leads, according to the author, to a different mental state, “characterized by diminishing critical distance to what is shown and increasing emotional involvement in what is happening” (2003, p. 13). The most interesting aspect of this study comes from the realization that human cultures have tried to submerge the senses completely since antiquity at the very least. “Landscape chambers” were discovered in the ruins of wealthy villas. For instance, all vertical walls of the Villa dei Misteri (60 AC) were fully painted in order to depict a continuous scene, most likely a bacchanal celebration. The main type of immersive strategy studied in the book is 360-degree visual illusions, with other significant additions, such as linear perspective in the famous Sala delle Prospettive (Baldassare Peruzzi, 1516) or in baroque churches. Grau’s ambition is to establish a link between these exceptional illusion spaces and the more recent development of virtual reality apparatuses, which also seek to submerge the senses, with the addition of direct adaptability of the virtual images to the users’ movements. It is because of this common ground that the author speaks of all the objects in his corpus as “Virtual Art.”
It would be easy to read the evolution of video game as an “arms race” toward evermore powerful processors that are dedicated to the creation of photorealistic virtual worlds. As Aki Jarvinen (2002) has pointed out, photorealism is but one of many visual styles used by video game creators. But clearly, it is an obsession to create ever more realistic and senses-luring special effects, through complex geometry, high-resolution photographic textures, and lighting and shadowing effects. On a purely perceptual basis, contemporary video games can create very convincing illusions, but expert eyes can still perceive imperfections—jagged lines due to poor resolution, blurry textures, frame rate drops, geometry pop-in, pixelated shadow maps, etc. Although newer products don’t always focus on the audiovisual capabilities of games, the fascination with illusion-making is likely to stimulate the constant renewal of technological props, until—if ever—a satisfactory configuration emerges.
Immersion as Psychological Engagement
The second most common usage of the word “immersion” is associated with a particularly engrossing state of mind, a concentration of mental resources in the course of a specific activity. In Mäyrä and Ermi’s model, it corresponds more closely to the “challenge-based immersion.” This definition is associated with the empirical studies conducted by Mihaly Csikszentmihalyi in the 1970s. Csikszentmihalyi set out to better understand the structure and dynamics of autotelic activities (i.e. activities that are gratifying in and of themselves). He interviewed several alpinists, chess players, rock climbers, as well as practitioners of a demanding profession (surgeons). All participants related a similar experience: in the course of their hobby, periods of intense absorption emerge and eventually seem to blur the limits of the self and the world around. For instance, one alpinist declared:
One tends to get immersed in what’s going on around him, in the rock, in the moves that are involved … search for handholds … proper position of the body—so involved that he might lose consciousness of his own identity and melt into the rock.
(1975, p. 43)
The defining aspect of this flow experience, according to Csikszentmihalyi, rests on the optimal usage by an individual of specific skills. The structural elements highlighted by the original study are: attention focussed on the task, limitation of the stimulus field, balance between challenge and skills, clarity of the goals and of retroaction; all these can explain the autotelic nature of the experience, and eventually the transcendence of the self’s limits through action.
The key element of Csikszentmihalyi’s theory is the balance between skills and challenges. This is what allows the individual to enter an ideal flow channel, where the challenges are increased at the right pace, mimicking the learning experience and development of operational schemata in the individual. The flow channel refers to the ideal progression through the experience, where states of anxiety or frustration are avoided. However, challenges that have been mastered repeatedly are prone to boredom, and cannot sustain interest. The flow experience is thus linked causally to the idea of an adequate balance between skills and challenges:
If there were complete congruence, there would be nothing that was not already known and the object would hold no interest. If there were no overlap whatsoever, there would be no point of entry, nothing to allow viewers to exercise their skills.
(Csikszentmihalyi and Robinson, 1990, p. 134)
Needless to say, many aspects of ludic activities bear a striking resemblance with the structural aspects put forward by Csikszentmihalyi: limitation of the spatial field of play through arenas and boards; classification of players and challenges to favor “fair play,” etc. Video games have developed this aspect to a great extent through many assistance systems. Clear instructions on arcade cabinets and in game manuals have been around since the first days, and are now integrated dynamically in the first moments of the interactive experience. Algorithms can detect specific performance aspects and provide tailored information to players; recent games provide textual hints even before the player shows any sign of struggle or failure. Adjustable difficulty settings have also been around since the early days, and recent offerings such as Left 4 Dead (Valve, 2008) or The Elder Scrolls IV: Oblivion (Bethesda, 2006) dynamically adjust the challenge covertly while the player is performing. Spatio-narrative guidance systems are omnipresent in open worlds as well as linear virtual environments; arrows, lines, and “golden trails” indicate clearly the path to follow in order to progress. All these design elements seek to keep the player in a certain comfort zone and avoid frustration. Yet, some expert players are complaining that they take away much of the challenge.
Immersed in Fictional Worlds
The evolution of sensory illusion as envisioned by scientific experiments or in science fiction is the starting point for Janet Murray’s conception of immersion. In Hamlet on the Holodeck, the famous Star Trek device is used as a clear example of what a future medium could achieve. The holodeck feeds first and foremost on the ideal of a perfect “reality engine,” an illusion-producing black box that can create holographic—yet tangible—realities, able to fool many of the senses perfectly. This vision of immersion as a consequence of illusion-making has been criticized by many scholars, including Salen and Zimmerman (2003). But beyond illusion-making, Murray’s take on the holodeck also shifted the attention from sensorial dupery to the actual believability of the depicted world:
[it proposes] an illusory world that looks and behaves like the actual world […] The Star Trek holodeck is a universal fantasy machine […] a vision of the computer as a kind of story-telling genie in the lamp. [Users] participate in stories that change around them in response to their actions.
(1997, p. 15)
In the context of mediated immersion, the goal is ultimately to visit another world, and Murray highlights the challenges—and lays out some potential solutions—to realize the interactive medium’s promise to adapt to the users’ action in this world.
Moving from the illusionistic qualities of the medium to the virtual world itself, we reach Mäyrä and Ermi’s third type: imaginative immersion. They describe it as the feeling of transportation that can happen in literary and cinematographic media, which became ideal vehicles for portraying expansive narrative worlds. Even though the focus has shifted, discussions regarding immersive worlds often focus on the realism of the depicted events and characters, and thus can be seen as a continuation of the illusory definition of immersion. This is laid out directly in the literature about presence. Lombard and Ditton have gathered many uses of the term that refer to the idea of “social realism”: “Social realism is the extent to which a media portrayal is plausible or ‘true to life’ in that it reflects events that do or could occur in the nonmediated world” (1997). The authors present a rather simplistic view of this aspect: “a world with a green sky, flying trains, and misshapen animals that speak Chinese would surely seem more surreal than real, and therefore would be less likely to evoke presence” (1997). Similarly, Thomas Pavel claimed that “make-believe efforts” vary according to the relative socio-cultural proximity of the fictional world and the user’s actual world experience (1988). Yet, fantasy literature is full of these otherworldly aspects, while enjoying incredible popularity; the intertextual knowledge developed by fans in their consumption of this literature and the strength of the involvement they experience seem to contradict the restrictive definition of presence put forth by the authors. As Marie-Laure Ryan points out,
There is no point in denying that the worlds of the stereotyped texts of popular culture are the most favorable to immersion: the reader can bring in more knowledge and sees more expectations fulfilled than in a text that cultivated a sense of estrangement.
(2001, p. 97)
A discussion about the coherence of character and plot twists in fictional worlds would be a relevant avenue to inspect in this regard.
In Narrative as Virtual Reality: Immersion and Interactivity in Literature and Electronic Media (2001), Marie-Laure Ryan seeks to explain the journeys into any kind of narrative fictional world as a virtual reality experience. Building on possible worlds theory, she conceives immersion as the transportation of one’s consciousness from the actual world to a non-actual possible world. This transportation can be facilitated by the specific nature of each media and, in each narrative type, various representational strategies. For instance, she discusses what types of discursive constructions facilitate this act of mental relocation on the part of the reader. This is obviously a continuation of immersion as illusion-making, but the most illusionistic strategies are not always the best in her study of the “text as world.” This aspect has also been pointed out by Jean-Marie Schaeffer’s Pourquoi la fiction? (1999); this seminal effort sums up many of the propositions laid out in other theories, and opens up to the realm of cognitive and neurological sciences.
Immersive Paradoxes
Jean-Marie Schaeffer’s theory of fictional immersion is ambitious: founded on the mimesis principle, it encompasses the illusion-making aspect of mediated worlds, while acknowledging the cognitive distance implied by what we call “fiction.” The framework also seeks to include all media practices that are associated with fiction, from pictorial arts to contemporary video games. Since the introduction of interactivity has often been presented as being at odds with traditional world-building techniques in movies and literature, Schaeffer’s position is boldly transmedial. It states that all fictional apparatuses are built on illusion-making, a “key” to accessing fictional worlds, and that these access keys trigger a corresponding immersive posture. This posture, insists Schaeffer, is similar to one or many of our ways to relate to the world around us on a daily basis. The pretend speech acts laid out in a novel are read and understood as any other narrative; the moving images at the theatre are perceived just like any documentary shot would be, and might even bring about a strong visual identification (as was stipulated by Christian Metz with the concept of primary identification); the simulation of retroaction between an agent and a world in virtual reality and video games makes us identify with an alter ego, in a way that implicitly refers to the way we interact in real life. The premises of the theory can be seen as a reversal of the classic “willing suspension of disbelief” associated with Coleridge; at a very fundamental level of the experience, users don’t need to actively overlook the mediated nature of the representations, but are treating the illusion just as any other similar stimuli. Their basic perceptual and linguistic systems are prone to give credibility to perceptions/assertions; users have to fight to suspend this natural tendency to “believe” and determine the referential level of the objects they are perceiving.
Schaeffer’s theory insists on the various illusion-keys that let us access a world model. The basic overarching posture is one of mimetic immersion, which is triggered naturally in human beings in non-fictional contexts; Schaeffer highlights the fundamental importance of learning through imitation in children. Fictional apparatuses, then, are a subgenre of this mimetic immersive activity in that they call upon another mental aptitude: a cognitive framing of the illusions that prevents users from reacting inappropriately or to acquire false beliefs. This cognitive activity is itself encouraged by framing operations typical of make-believe activities: the frame of the stage or movie shot, various editorial strategies, etc. Thus, the fictional version of mimetic immersion is based on a “decoupling” between the illusion and its potential effects:
its potential consequences in terms of beliefs, in terms of motor reaction, and even in actantial terms, are neutralized by the pragmatic frame of shared make-believe—even when fictional immersion is actantial (as that of the actor [and of video game players]).
(1999, p. 136, freely translated)
This formulation is especially interesting in order to better understand the complex posture of a video game player: the depicted events evoke certain knowledge from our daily experience, but even the most recent natural interfaces don’t use our daily motor knowledge perfectly, and part of the fun is that we are freed from some of the constraints of physical and social life. Racing in a virtual world allows us to be much bolder and adventurous than we could ever be in real life. It is no wonder that the thrills of morally reprehensible actions are also explored in many games. Moreover, Schaeffer’s conceptualization of immersion highlights the fluidity of immersive postures in the course of the experience, just like children in games of make-believe go in and out of the world they create all the time. According to him, readers switch from the position of a narratee to partial identification with the figure of the narrator; moviegoers identify with the perceptual flows, while still being addressed by verbal narrators. This idea of variability of immersive posture goes against the simple illusionistic conception, and is especially suited to talk about the video game experience. Players control an avatar or a point of view as one would remote control an electric puppet or car, yet they get to incorporate these controls on a visceral level; they are looking at images that mimic to some extent our natural perception, but the typical screen is cluttered with arbitrary signs—such as the various assistance features highlighted earlier—that represent a new form of visual narration, and complicate further the immersive posture. As Schaeffer points out: “the variability of the modalities of an immersive posture is one of the most important factors in the cognitive richness of artistic fictions, since it allows the creation of multiple perspectives (or access points) of fictional worlds” (1999, p. 258, freely translated).
Flowing Forward
Schaeffer’s theory of fictional immersion is certainly one of the most ambitious and complete conceptualizations. It highlights the illusion-making aspects of the phenomenon while acknowledging the essential work of cognitive framing in the experience of fiction. All aspects of the theory are based on age-old propositions from Plato and Aristotle, which are connected with more recent findings in linguistics and biology. Contemporary research in neurosciences tends to corroborate Schaeffer’s vision of a “gullible” perceptual system easily fooled by illusions. For instance, Joseph Ledoux’s study on the emotional unconscious/innate fear system highlights how the body’s reactions can be triggered by very simple stimulus inherited from our long evolution (1996). Research on mirror neurons allows us to explain the contagiousness of emotional faces, the perception of pain and of certain hand gestures (Iacoboni et al., 2005). As Torben Grodal observes: “Via mirror neurons, the facial expressions’ emotions resonate in the onlooker, and that explains the emotional contagion emanating from close-ups” (2009, p. 187). It is these recent developments of cognitive and neurological sciences that have led Grodal to create his PECMA flow framework, in order to better understand the different elements at play in the reception of mediated worlds.
PECMA stands for Perception, Emotion, Cognition, Motor Activation. Grodal acknowledges the entanglement of all these processes in our daily experience and media consumption, but the acronym is still supposed to represent a logical “progression” in our perceptive-cognitive system, with different type of artistic works making the flow “stop” at certain “stations.” For instance, the abstract films of Norman McLaren such as Dots (1940), Lines: Vertical (1960), or Lines: Horizontal (1962) are especially appealing to the visual cortex and can be engaging solely on that particular level. More typical narrative films or novels involve semantic memories and world-building capabilities (agent intentions, ordering of events, anticipation of future developments, etc.), which occur within the prefrontal cortex. Movies are able to trigger the premotor and somatosensory cortex, for instance via mirror neurons, and thus can help viewers feel part of the action in a visceral way. But in a very literal sense, only interactive media trigger our motor cortex directly.
As I’ve pointed out, even natural interfaces require learning new motor schemata. So in a paradoxical way, interactive media require the assimilation of more intertextual knowledge in order to get immersed in the experience. As such, even video games require us to use an “inhibiting function,” similar to what Schaeffer refers to as the cognitive framing of fiction: “Those inhibiting functions develop in children in tandem with their ability to understand pretend behavior and false belief” (Grodal, 2009, p. 150). But contrary to Schaeffer, Grodal seeks to highlight the very distinct nature of the interactive experience:
Interactive media such as video games have given rise to new types of experience that allow for the fusion between the roles of spectator and participant. These interactive media games offer the possibility of an entirely new type of immersion, involving even the element of concrete motor action in the PECMA flow.
(Grodal, 2009, p. 187)
In the end, one can still wonder why novel enthusiasts, moviegoers, and game players are willing to invest so much cognitive, affective, and motor efforts in the enjoyment of their favorite fictional worlds. Here, Dolff Zillmann’s theories of suspense are revealing (1996), and allow us to make a clear link with Csikszentmihalyi’s theories. For Zillmann, one of the motivations to assess cognitively and enjoy dysphoric emotions in suspenseful episodes can be summed up by the concept of “affective overreaction”; once users are aroused—here in a negative way—by a representation, the resolution of the tension will be enjoyed even more; excitation overlaps from dysphoric to euphoric feelings. Thus, “the investment of efforts” acquires an autotelic nature, since users have built expectations on the gratifying nature of the experience. And that is certainly a trait of popular fiction across a variety of media and practices.
As we have seen, the various theories of immersion propose a particularly rich framework to analyze and address the complexity of the video game experience, and are able to accommodate very recent findings from a variety of disciplines. Semiotics and aesthetic theories are augmented with propositions from cognitive sciences and recent findings in neurology in an effective way, shedding light on the common yet very complex phenomenon of fictional immersion. The broadness of this cultural practice—ranging from stage acts, to pictorial arts, to spoken and written narratives, and contemporary video games—necessitates more than an transmedial framework; it can only be addressed meaningfully by adopting a multidisciplinary approach.
References
Csikszentmihalyi, M. (1975). Beyond boredom and anxiety: The experience of play in work and games. Washington: Jossey-Bass Publishers.
Csikszentmihalyi, M. and R. E. Robinson (1990). The art of seeing: An interpretation of the aesthetic encounter. Malibu, CA: The J. Paul Getty Trust.
Grau, O. (2003). Virtual art: From illusion to immersion. Cambridge, MA: The MIT Press.
Grodal, T. (2009). Embodied visions: Evolution, emotion, culture, and film. New York: Oxford University Press.
Iacoboni, M. et al. (2005). Grasping the intentions of others with one’s own mirror neuron system. PLoS Biology, 3(3). Retrieved September 16, 2012, from http://biology.plosjournals.org/.
Jarvinen, A. (2002). Gran Stylissimo: The audiovisual elements and styles in computer and video games. In F. Mäyrä (Ed.), Proceedings of Computer Games and Digital Cultures Conference (pp. 113–128). Tampere: Tampere University Press.
Ledoux, J. (1996). The emotional brain: The mysterious underpinnings of emotional life. New York: Simon & Schuster.
Lombard, M. and T. Ditton (1997). At the heart of it all: The concept of presence. Journal of Computer-Mediated Communication, 3(2). Retrieved September 16, 2012, from http://jcmc.indiana.edu/vol3/issue2/lombard.html.
Mäyrä, F. and L. Ermi (2005). Fundamental components of gameplay experience: analysing immersion. DiGRA Conference 2005 Changing Views—Worlds in Play. Retrieved September 16, 2012, from www.digra.org/dl/db/06276.41516.pdf.
Murray, J. (1997). Hamlet on the holodeck: The future of narrative in cyberspace. Cambridge, MA: The MIT Press.
Pavel, T. (1988). Univers de la fiction. Paris: Éditions du Seuil.
Ryan, M.-L. (2001). Narrative as virtual reality: Immersion and interactivity in literature and electronic media. Baltimore: The Johns Hopkins University Press.
Salen, K. and E. Zimmerman (2003). Rules of play: Game design fundamentals. Cambridge, MA: MIT Press.
Schaeffer, J.-M. (1999). Pourquoi la fiction? Paris: Éditions du Seuil.
Zillman, D. (1996). The psychology of suspense in dramatic exposition. In P. Vorderer, H. J. Wulff, and M. Friedricksen (Eds.), Suspense, conceptualizations, theoretical analyses, and empirical explorations (pp. 199–232). Mahwah, NJ: Lawrence Erlbaum Associates.