4.1 Changing Actively and with Care
As established in the previous chapter, cinematic writing is a system of transition from an implicit mind-cinema to its embodiment
in digital texts. In cinematic writing, this process is understood as the intermesh of mental activity with the sequential physical manipulation of material elements, or bricole
s. This can also be explained using the analogy of clockwork, where every cog is enmeshed with the others in transmitting rotational movements.
Stimulus
response
,
emotioning
reasoning
circularities are remixed in dynamic circular interactions, resulting in a representation of meaning.
Vygotsky (2012 [1934]) described mind processes as ‘autonomous’, ‘idiomatic’ and a ‘speech function’, which is not an activity that can be translated into a direct vocalised form (p. 263). ‘Inner speech is to a large extent thinking in pure meanings. It is a dynamic, shifting, unstable thing, fluttering between word and thought’ (p. 264). In other words, inner speech becomes accessible as a sequentially rolling mind-cinema only through the process of its articulation into a mode of meaning comprehensible to others.
[have] varying value and weight within complexly articulated social, cultural, political, educational, religious, economic, familial, ecological political, artistic, affective, technological webs (you can name others, I am sure); we know that, in our places and times, writing is one of many operations by which we compose and understand ourselves and our identities and our abilities to live and work with others. (p. 2)
Wysocki elaborates on the importance of going along with changes in writing practices into multimodal systems of expression ‘actively and with care’. She calls for keeping ‘the old rug’ rather than replacing it with a new one ‘as though that rug ever existed as anything but an imaginary comfort’ (p. 2). She argues that in terms of theoretical concepts and practical strategies for multimodal writing, we currently have ‘the equivalent of carpet scraps, some tentative weaves, bits and pieces of matting and colourful materials for you to consider and, if they seem at all useful, to arrange as they fit you for now’ (p. 2).
In a similar manner, Hayles (2012) believes that education would not benefit from breaking with traditional ways of conveying meaning, ‘leaving behind millennia of thought, expression, and practice that no longer seem relevant to its concerns’ (p. 7).
Taking a position similar to that expressed above, cinematic writing advocates a methodology for knowledge production using writing as a framework component in the articulation of knowledge, while progressively remixing it with non-linguistic modes of meaning. The key point in this integration is not the production of decoratively impressive texts, but finding ways for other modes to play their own significant role in the production of meaning.
[…] we run the risk of overlooking the fundamentally multimodal aspects of all communicative practices. If we acknowledge that literacy and learning practices have always been multimodal and that ‘communication has always been a hybrid blending of visual, written and aural forms’ (Hill, 2004, 109), the challenge becomes one of finding ways to address – in our scholarship, research, and teaching – the multimodal, technologically mediated aspects of all communicative practice. (loc. 370)
In acknowledging the challenges, Shipka also cautions against facilitating changes that result in the substitution of one set of sign systems, technologies and limitations for another, or downplaying any of them (loc. 382).
Cinematic writing proposes a system of communication for education that in the words of Wysocki is, ‘raised on alphabetic literacy’. This suggests one of the many possible ways of closing the gaps between fragmented systems of meaning articulation and is a method that can be used for the construction of hybridised bridges between writing and other representational modes. Cinematic writing, as one of them, is a budding technology for a mind-cinema to stream out of its confinement into material representations, making things visible and audible and enabling learners to explore themselves within the structures of surrounding world.
4.2 Recovering the ‘Atmosphere’
McLuhan (1964) suggests that with the advent of the alphabet and invention of other technologies, people have drifted away from tribal traditions and isolated themselves from being spatially connected to each other. Scholarly writing, to tailor McLuhan’s words, has assumed the role of ‘an aloof and dissociated literate Westerner’ (p. 4).
Returning to oral culture and considering a campfire storyteller, for example, we can imagine them delivering a tale not entirely through words. Everything matters: sometimes, not uttering a word, the storyteller just shrugs. Sometimes, he/she unexpectedly pokes the person next to him/her in the ribs, making the whole crowd burst into peals of laughter when the person starts in surprise. Sometimes, the storyteller throws a log onto the fire and for a few moments, simply watches the sparks in the rising smoke. There are sounds of crackling fire, chirping crickets, rustling leaves, people’s breathing, the movement of the wind—together, it all creates a memorable atmosphere. It does not simply form the background for the story being told; it is a classroom where people make connections between the story and the world around them.
These ‘atmospheric’ aspects of storytelling are coded in the words and hidden deeply between the lines. Cinematic writing suggests that with the use of digital media, it is possible to release these ‘atmospheric’ elements from their imprisonment in text-boxes and return them to the physicality of space and time. This craft can be developed through the expansion of mental grasps, the most vivid or essential moments in the process of the syntactical articulation of the vaporous fluctuations of mind. Catching glimpses of sound, motion and images that accompany the synthesis of data acquired in the process oriented towards the intended purpose of a knowledge acquisition task, we can interpret them through other modal analogies. By doing so, we infuse the experience of writing with new physicality, or as it may be, the storytelling returns to its original form.
Selfe (2009) argues for the integration of aural and writing modes in her essay titled, The Movement of Air, The Breath of Meaning. With this title, she aptly encapsulates the significance of atmospheric presence in the formulation of meaning. Selfe writes: ‘We need to better understand the importance that students attach to composing, exchanging, and interpreting new and different kinds of texts that help them make sense of their experiences and lives—songs and lyrics, videos, written essays illustrated with images, personal Web pages that include sound clips’ (p. 642).
The students’ atmospheric experience of life as an aesthetic property is widely multimodal. ‘Human normal perception [of such a multimodal environment] gravitates toward holistic integration, with all sensory streams able to receive some degree of mental representation at the same time’ (Tucker 2007, p. 67). Traditionally, in education, the meaning of any abstract concept is conveyed by squeezing its multidimensional essence inside into one representational modality at a time.
[…] the rights and responsibilities that students have to identify their own communicative needs and to represent their own identities, to select the right tools for the communicative contexts within which they operate and to think critically and carefully about the meaning that they and others compose. (p. 618)
The assertion of the right of free individual expression within the framework of ideological constructions, or rigid cultural or educational traditions, is one of the central threads in the fabric of learning with new media. Every individual must be given an opportunity to choose his/her own communicative techniques that are congruent with fulfilling this right of self-expression.
Vygotsky (2012 [1934]) maintains that in inner speech, ‘a single word is so saturated with sense that it becomes a concentrate of sense. To unfold it into overt speech, one would need a multitude of words’ (p. 342). It is impossible to translate through ‘a direct transition from thought to word’ (p. 346). It can be assumed the impossibility of direct translation results either from the insufficient scope of the representational sign system or a lack of human ability to read ‘between the lines’ (around the signs) of symbolic representation. Vygotsky calls the meaning hidden ‘around the signs’ a subtext and notes that in order to understand another’s speech, one must be able to decode the subtext by delving deeper than the speaker’s thought, by reading from ‘beyond the semantic plane’ (p. 317).
In defining the ‘semantic plane ’, Vygotsky refers to the meaning constructed by words. Accordingly, he makes a case for the limitations of language as writing or speaking in decoding a subtext . Cinematic writing suggests that the subtext can be translated through an interplay of other modes; thereby, one can read ‘beyond the semantic plane’ and ‘between the lines’.
4.3 Gestalt—An Interplay of All Modes
- a)
a mode is a resource for making meaning;
- b)
every mode has its own affordances and limitations; and
- c)
modes carry certain socially assigned meanings (Kress 2010, p. 84).
In giving examples of what can be included in the category of mode, Kress lists image, speech, gesture and writing (p. 84). Elaborating on this, he adds facial expression (p. 87). Kress then poses questions: is font a mode? Is layout a mode? Is colour a mode? (p. 87). After considering these ‘candidates for mode status’, Kress concludes that ‘meaning can be made through the affordances of font as it can through colour’ (p. 88).
Shipka’s (2011) approach is to see modes as spatially inclusive forms of representation. She considers sights, sounds, scents and movements as viable communicative modes (loc. 493). Ball et al. (2013) refer to modes of communication as linguistic, aural, visual, spatial, gestural expressions, and combinations thereof (p. 18).
[…] a set of organising principles and resources (e.g., image, music and gesture are modes) that is an outcome of the cultural shaping of a material. Each mode consists of a set of semiotic resources , which have meaning potential, based on their past uses, and affordances based on their possible uses. (p. 184)
[…] everyone working in multimodality uses the term ‘mode’: some prefer to talk about ‘resource’, or ‘semiotic resource’, and generally avoid strong boundaries between different resources: highlighting instead the significance of the multimodal whole (‘gestalt’ ). (loc. 202)
- 1.
Meaning is made with different semiotic resources , each offering distinct potentialities and limitations.
- 2.
Meaning making involves the production of multimodal wholes.
- 3.
If we want to study meaning, we must attend to all the semiotic resources being used to make a complete whole (loc. 192).
It can thus be concluded that meaning-making in a multimodal artefact is achieved by striving for a gestalt, wherein all modes attuned into one harmonious composition.
A theatre production is a perfect example of multimodality. In it, we can observe a gestalt of attuned modes of communication: language as written and spoken word and a variety of visual, auditory, movement and lighting modes. Gestalt as a multimodal whole can also be compared to Eisenstein’s (1949) notion of overtonal montage . In this regard, the gestalt of expressive modes can be considered as a device for merging the semantic plane with its subtext (s).
With a multimodal whole understood through a variety of representational modes, the question that arises here is what exactly is a cinematic writing mode and how does it relate to semiotic resource?
4.4 Cinematic Writing
In relation to meaning-making, the definitions of the modes presented in the section above are concerned with culturally constructed systems of representation. They are known as semiotic systems.
In a semiotic sense, signs take the form of words, images, sounds, gestures and objects. Contemporary semioticians study signs not in isolation but as part of semiotic ‘sign-systems’ (such as a medium or genre). They study how meanings are made and how reality is represented. (p. 2)
In cinematic writing, the acts of knowing are understood as taking place through the representation of mind-cinema. This can be a reflection, observation and representational unpacking of the mind-cinema. The mind-cinema evolves through the process of representing
meaning-making of the given events or phenomena. Here, we can observe a circuity within a larger circuity of the process. As alphabetic writing unfolds, it becomes a canvas into which non-linguistic mode
s of communication are interwoven in a gestalt fashion, a multimodal whole.
In cinematic writing, the meaning is constructed with images, sounds and motions. Individually, however, they are not necessarily separate modes as they are often intertwined with other elements to constitute a new mode or perhaps genre. For example, sound can be used in the form of a song, music or recorded speech as a background for the alphabetic text, but it can also be part of an animation or video-recording.
Visualisation of cinematic writing
The dynamic circular movements of the lines signify recursive feedback loop s of representing
Non-linguistic modes of expression are not just symbolic elements enmeshed into the fabric of writing, but the ligaments – connective tissue bundles – that join the sequences and provoke and determine the movements of meaning-making.
Borrowing the principle from the previous quote, for the moment means that currently, cinematic writing is not concerned with the theoretical and methodological complexity of semiotic modes. Cinematic writing is established on the relationships between representational modes; however, some clarity is still required regarding the understanding of what is a mode .
At the foundation of cinematic writing is a notion of production in which lines of alphabetically articulated thoughts, as shown in the diagram above, give rise to a full mind-cinema. McLuhan’s (1964) concept of typography as a movie projector is seen as alphabetic writing that provokes thoughts and evokes memories that are saturated with meaning. The process of interpreting the meaning with alphabetic writing alone loses the potency of its saturation; borrowing from Vygotsky (2012 [1934]), it: ‘[…] miscarries—when, as Dostoevsky put it, a thought “will not enter words”’ (p. 264). In this case, cinematic writing operates through the mental grasps, catching the ‘frames’ of the mind-cinema and presenting their saturation with the variety of modes. The representation of the mental grasps with the verbal mode alone is limited. Incorporation of non-linguistic modes—images, sounds, motions and interactions—facilitate the conversion of mental grasps into a much richer physical form.
For example, if there is a need to compose a cinematic writing with a dance element in it, the dance is recorded digitally. Placed inside the digital page, the dance-recording is no longer an isolated representational mode. It is disassembled into representational modules that are now compose a new choreography together with linguistic elements, static or moving images within spatial and temporal arrangements of the page. They can be short, a series of glimpses, or materialised mind frames represented as interwoven modalities into new patterns of meaning. In other words, the video is not an artefact on its own but an element of a symbiotic interplay, or gestalt, of all modes.
From this perspective, cinematic writing is concerned with time as a dimension, which renders the experience of another dimension, space, in a completely new way. For example, the sound of an author’s humming can start ten seconds after a reader commences reading a text. At this moment, an object can move across the page, rain can start pouring over the written text, a dog can run into the page and bark, and so on.
In relation to what is discussed above, we can conclude that cinematic writing is a system of signs and principles organised to make meaning, that is, a semiotic resource. To reiterate, the interdependence between signs and principles is known as structural relations involving two dimensions, paradigmatic and syntagmatic, as discussed earlier. Paradigmatic components function as the signifiers and signified . They are an explicit aspect of cinematic writing presented through such modes as alphabetic writing, images, sounds, and motions. In cinematic writing, they are identified as database elements.
Syntagmatic dimension is the implicit stimulus
response process that leads to the explicit formation of the multimodal choreography within given parameters of time and space.
According to Chandler (2002), ‘Syntagms and paradigms provide a structural context within which signs make sense; they are the structural forms through which signs are organised into codes’ (p. 80). In cinematic writing, the paradigmatic dimension is embodied in database collections of digital objects and data, or signs, and the syntagmatic dimension is seen as a narrative system of patterns organisations, or codes.
Cinematic writing employs a variety of digitised modes. Physical objects and their surroundings are photographed, drawn and scanned, or computer-generated. Sounds, movements, gestures and facial expressions are digitally recorded. Thoughts and feelings are expressed through a wide range of modalities including alphabetic writing, shapes, colours, sounds and movements. All these modes have their own structural principles of signification that in cinematic writing are assembled into a larger unified system that works within time and space structures.
Overlapping representational modes in cinematic writing
Overllaping modes of cinematic writing | |||||
|---|---|---|---|---|---|
Linguistic mode | Visual mode | Audible mode | Kinaesthetic mode | Interactive mode | |
Language | Alphabetic writing | Text, signs, symbols | Speech, songs, exclamations, expressive interjections | Gestures, moving text | Moving images, interactive buttons, links, diagrams, touch screen |
Images | Signs, symbols | Signs, symbols, photos, graphics, drawings, diagrams, shapes, colours, layouts | Animations, videos | Gestures, dance, animations, videos | |
Sounds | Speech, songs, exclamations, expressive interjections | Dance videos, music videos | Sounds of nature, sounds of objects and sounds made by living beings, music | Dance/music videos | |
Motions | Visual mode components manipulated within time/space flow | The above components manipulated within time/space flow | The above components manipulated within time/space flow | The above components manipulated within time/space flow | Interactions with the elements of multimodal text |
4.5 Convergent Points
Cinematic writing is a digital genre. It operates within a system of digital modes in which database material elements are manipulated by the principles of multimodal choreography. The functionality of cinematic writing is based on structural relationships
between the paradigmatic
syntagmatic dimensions, which are parallel to database material elements and fundamental principles of organisation adopted from a range of representational modes. Therefore, cinematic writing can be considered as a complex semiotic resource with an enhanced modal capacity for meaning-making.
The augmented remixability of the modal elements and principles opens up the possibility for constructing learning activities in more personalised ways, thus fostering a considerable degree of autonomy for learners. In this case, remixability lies at the heart of cinematic writing as it determines the multimodal nature of this semiotic resource. In other words, the elements—database resources, and principles—and the ways in which the database elements are organised are remixed not for the purpose of embellishing or decorating the text, but to achieve an organic choreography, to tell a story with the unified whole.
response circuity. This can also be represented as database
narrative interactions.Key terms | |
|---|---|
Gestalt | –An interplay of all modes in the construction of unified meaning |
Rhetorical sovereignty | –The full rights and responsibilities of students over the articulation of their identity in the production of knowledge, which includes the choice of the modes of communication and methods of exploration |
Semantic plane | –The framework of meaning |
Semiotics | –The study of signs and symbols and their use in meaning-making |
Semiotic resource | –The system of signs and symbols that promotes the translation of meaning into organised code |
Signifier | –A sign that stands for something other than itself |
Signified | –Something that is signified by a certain sign |
Structural relations | –The interdependence of elements and principles of various modes within one digital space/time construct |
Subtext | –The implicit meaning |
4.6 DOING
KNOWING: The Ripples Pedagogy in Practice
4.6.1 Learning Task Three: Making Your Own Meaning
Learning math and spelling is far less important than learning the act of learning. Arithmetic is an outdated life skill, like swordplay or horse riding. Four hundred years ago, those were vital life skills; today they are relics of bygone world and primarily enjoyed as sports. (Mitra, as cited in Rshaid 2017, p. 6)
- 1.
The quote above is from Sugata Mitra, winner of the 2013 TED Prize award.
Watch his award-winning talk in a group of two or three people:
https://www.youtube.com/watch?v=zpcEpmNbHds
- a.
Create a new document using desktop publishing software, graphic design software, Power Point or any other software where you can write with inclusion of images, sounds, motion or interactivity.
Imagine that going from page to page in your digital document is similar to going from one room to another. The rooms are the spaces in your workshop. Think about ‘the walls, furniture, windows, doors and so on’ because they represent spaces where your mind-cinema will roll on through shapes, photos, objects and typefaces. The workshop needs to be as conducive as possible for the mind to feel ready to produce the cinema. What layout creates a presence that makes you feel comfortable? Experiment. Do not worry about it looking aesthetically good. Think about it feeling right, even if it might look absolutely wrong to someone else.
‘Design is spoiled more often by the designer’s having been overly cautious rather having been overly bold. Dare to be bold’ (White 2002, p. 73).
- b.
Write and explain five points on whether you agree or disagree with the main ideas of Sugata Mitra’s talk in the video and in his quote presented above.
- c.
Discuss your points with others. Before the discussion starts, turn on your mobile audio recording. The main focus of the dialogue is ‘learning about the act of learning’. What life skills that students must learn at school today do you think are the most important? In what ways can your own envisioned act of learning facilitate learning about these life skills?
- d.
Working on your own, listen to the discussion again. Extract the most salient aspects and moments of the discussion. Think how the dialogue with your peers influenced your initial opinion. Describe the process of its transformation. If you believe there was no any transformation in your thinking, think more critically and start writing about the most heated moment of the discussion. Support your thoughts with extracted fragments of your audio recording. Try to make a meaningful composition, expressing your thoughts and emotions with writing, shapes, colours, letterforms, audio elements, photos (not necessarily of the participants), images, and sketches in such a way that there is an obvious synergy between them—as if you are preparing the elements for the mind-cinema to roll on a page.
- a.
- 2.Collect three opinions from other people, e.g., parents, other students or educators on the ‘act of learning’.
- a.
Based on the meaningful compositions you compiled on the two pages of your cinematic writing document, as well as the three opinions of others you have collected, make a synthesis (evaluate and integrate the data into a new personal conceptualisation) of how you envision ‘the act of learning’ for the students of the twenty-first century.
- b.
Present your conceptualisation expressed in words, images, sounds or short videos.
- a.
- 3.Collect three opinions from people on social media or other websites.
- a.
Repeat the process by making a synthesis of what you thought before and how your opinion has been modified.
- b.
Write a reflection of the feedback loops and how they affected (or did not affect) your opinion and why.
- a.