3.1 Mind-Movie Projector
Cinematic writing has been developed as a practice of meaning-making through the process of multimodal representations. It is rooted in the digital cultural trend of ‘deep remixability’ (Manovich 2013, p. 272). At the heart of cinematic writing —writing with images, sounds, motions and interactions—is a thesis of the non-linear dynamics, a cybernetic principle of feedback loops . The circuity of the feedback loops is positioned around two ‘choreographic’ movements of the ideational process: (a) the act of representing, or writing with images, sounds, motions and interactions, and (b) as explained by Postman and Weingartner (1969), the act of ‘thinking’ and ‘emotioning’, ‘minding’ or ‘making meaning’ (p. 84).
The principle of digital media that ‘can be called the fractal structure of new media’ (Manovich 2002, p. 30)—in other words,
modularity
—allows the reconstruction of various elements of digital objects without affecting their integrity. Another principle of digital media—
variability
—opens up new possibilities for the
deep remixability
of these discrete samples into numerous, novel representational variations. The activity of fragmenting and remixing generated data such as facts, images, songs or videos into new patterns of organisation promotes the recognition of new relationships between the components, while minding—emotioning
reasoning about the process reveals new meaning.
The characterisation of multimodal writing as cinematic was inspired by McLuhan’s (1962) notion of typography as a movie projector. McLuhan drew an analogy between translating the movements in the production of scribal art into typography with taking a series of static shots and seeing them placed within the frames (p. 124). He suggests that: ‘typography bears much resemblance to cinema, just as the reading of print puts the reader in the role of the movie projector’ (p. 124). He continues, ‘The reader moves the series of imprinted letters before him at a speed consistent with apprehending the motions of the author’s mind’ (McLuhan 1962, p. 124).
The word cinematography comes from two Greek roots: kinesis (the root of cinema), meaning movement, and grapho, which means to write or record. […]Writing with movement and light – it’s a great way to begin to think about the cinematographic content of motion pictures. (Sikov 2010, loc. 953)
In the quote above, cinematography is conceptualised as writing with movement and light. Using the same principle, cinematic writing can be defined as writing with movement and sound. Digital media affordances —the representational capacity of digital media—provide new techniques for obtaining and manipulating data by means of written, photographic, graphic, audio and video recording, thereby turning digital effects into ‘meaningful artistic language’ (Manovich 2002, loc. 257). Writing with movements and sounds also implies writing with images because for a movement to be actualised, an object, i.e., a sign, symbol, icon, photograph or drawing, must be present. Therefore, images are added to the definition of cinematic writing: cinematic writing is writing with images, sounds, movements and interactions.
In cinematography, the term mise-en-scene denotes literally everything that has been put into the scene.
Mise-en-scene consists of all of the elements placed in front of the camera to be photographed: settings, props, lighting, costumes, makeup, and figure behaviour (meaning actors, their gestures, and their facial expressions). In addition, mise-en-scene includes the camera’s actions and angles and cinematography, which simply means photograph for motion pictures. (loc. 308)
The cinematographic elements of mise-en-scene correspond with the term relevant to the processes of cinematic writing , bricoles . This can imply: theoretical concepts, methodological approaches, or data fragments (conceptual or material), such as existing ideas, scholarly quotes, screenshots from social media or websites, and images, audio or video recordings. In other words, bricoles are the material, conceptual elements or fragments which create a deep remixability that makes meaning out of experience.
3.2 Looping for Meaning
By the nature of digitality, the production of cinematic writing, as an embodiment of meaning, is realised through layered systems of creative software. The database elements such as alphabetic text, images, sounds and interactive components are distributed on software layers. As McLuhan suggests, they can be compared to shots placed within the frames of the film.
The layers or frames in creative software are organised not in ‘a film-thread’ but in a stacking order, thereby affecting each other through automated unified projection. The producer of the representation is minding by applying sequential, narrative logic: the stimulus materialises as a result of the layers’ unified projection. Through thinking
emotioning, the producer responds to the stimulus by applying associative
bricole
s either retrieved from a database or generated anew with the help of the database’s automated items and codes. The producer’s minding and behaviour then results in a production of new stimulus and therefore triggers a new response, and so on. With respect to time, the producer works in a sequential, linear order. In relation to the direction of the process, the stacking order of the layers alters its linearity. The progression is realised through recursive feedback loops and due to a non-linear, stacking order, ‘the choreography of looping’ (Ingold 2015, p. 19) is determined by a stimulus
response
correlation.
In cinematic writing , the digital page is constructed by the overlapping of ‘the frames of the rolling mind-cinema ’. The producer composes a choreography of moves by interlacing the written text with appearing or disappearing images, sounds and moving objects. The reader’s ‘mind-cinema’ is activated by their personal ‘typographic movie projector’. However, it ‘stumbles over’ other superimposed modes of expression, such as sounds, objects, movements. They alter the linearity of the mind-cinema, leading the reader to see subsequent ‘frames’ according to their experiences of the previous ones. In other words, the reader adapts the choreography of psychological moves proposed by the author.
Together, representing
meaning-making are, borrowing from Ingold (2015), ‘a wreath of entwined lines, a whirl of catching and being caught’ (p. 7). They are two interlaced streams in a vortex of recursive merging and separations resulting in the articulation of evolving meaning.
- a.
considerations of modal affordances and rhetorical requirements;
- b.
orientations of modes and their different ‘takes’ on the world;
- c.
the organisation of modes if it is ‘a multimodal ensemble’ (p. 93).
Let us more closely examine the instance in a continuous mind-cinema that Kress refers to as a ‘halt’. According to Kress, for the ‘halt’ to occur, the mind-cinema must be ‘framed’. This brings about a ‘prompt’ to make the internal process visible (p. 93).
The Recyculator. Author: J9 Stanton, Alice Springs, Australia (Found-objects sculpture representing the recycling process [Personal photograph])
In this sculpture, I saw a visual analogy of
deep remixability
, a metaphor for the process of representing
meaning-making. The flow of incoming information never stops and the mind is always busy ‘churning’ and reconstructing the data arriving within the frames of its own making.
‘Ongoing unconscious assistance to our conscious life occurs whenever we speak aloud or only to ourselves, write or type, play a musical instrument, perform athletic routines, drive, or simply set a table’ (Edelman and Tononi 2013, p. 182). The memories evoked by ongoing experience emerge from the unconscious, mingle with the conscious ‘emotioning-reasoning’, and halt it to form a framed episode before starting the whole process again, turning the flow into a mind-cinema.
Semiosis , the making of meaning, is ongoing, ceaseless. Occasionally there is a ‘prompt’ to make that internal process visible, and there is then an ‘utterance’, an outward material sign-complex, always as a response to the prompt. It is a punctuation of semiosis: the ceaseless process of inner meaning-making is halted for a moment. It is ‘fixed’ and it is ‘framed’. Kress (2010, p. 93)
Kress proposes that ‘occasionally there is a prompt’ in the mind-cinema that must be uttered, that is, to be embodied into some material mode . I will refer to Kress’ concept of the ‘prompt’ as a ‘mental grasp ’ or ‘grasp’.
We think in images – the things we see, snapshots, and moving pictures. We think in sounds – the background sounds at a circus or crowded mall, the music of a concert, the foreground sounds when standing at the foot of a waterfall, and the words and statements said. We think in sensations – the tactile sensations of touch, perhaps smoothness of silk, the wetness of water, the rhythmic vibration of a heart […] We think in smells and tastes – strawberries, hot bread from the oven, a chicken coup on a farm, the fragrance of an aftershave lotion. (Hall 2006, loc. 350, italicised by author)
Parole Intérieure, M.V. Egger remarked that ‘before speaking, one barely knows what one intends to say, but afterwards one is filled with admiration and surprise at having said and thought it so well.’
The woman in E.M. Forster’s novel Howards End said memorably: ‘How can I know what I think till I see what I say?’ (p. 182)
In relation to cinematic writing, semiosis is achieved through the process of representation with alphabetic text, images, sound and movements. The meaning emerges through a unified projection resulting from the process of emotioning
reasoning
of the rolling frames of the mind-cinema.
In this view, the technicality of the layered representational production appears to be illustrative of the metacognition of semiosis.
3.3 Narrative and Database Through the Process of Layered Production
Working with layers can demonstrate the circuity between stimulus
response as well as emotioning
reasoning of semiosis on a metacognitive plane. Furthermore, a layered production can be employed as a device to understand the relationship between two cultural mechanisms of expression: narrative
and database
.
In Manovich’s (2002) theory, the relationship between database and narrative is categorised as an opposition (p. 225). In arguing for this, Manovich draws on the semiological theory of syntagm and paradigm (p. 229). To articulate the syntagmatic dimension, Manovich gives an example of the speaker producing ‘an utterance by stringing together elements, one after another, in a linear sequence’ (p. 230).
The paradigmatic dimension is explained as a set(s) of related categories. ‘All nouns can be one set; all synonyms of a particular word form another set’ (p. 230). Syntagmatic units are evident through the modes of representations, either speech or writing. Paradigmatic units exist only as abstract categories that facilitate a system of operation.
Borrowing from Capra and Luisi (2014), the paradigmatic dimension can be said to be a study of matter that answers the question ‘What is it made of?’ The syntagmatic dimension answers the question ‘What is the pattern?’ (p. 4) or, ‘What is the organisation and relationship?’ According to Manovich, ‘Database (the paradigm) is given material existence, while narrative (the syntagm) is dematerialised’ (p. 230). In relation to cinematic writing
, the composed texts are represented as patterns of materialistic bricole
s that are organised by means of the behaviour, such as stimulus
response. Digitised bricoles are the elements taken from various electronic databases. The approach of organising the bricoles in meaningful patterns is a string of events and experiences that can be characterised as the narrative.
reasoning, which can be classified as the mind-cinema progression—are implicit. Nevertheless, in the process of meaning-making, the progression of the inner mind-cinema is as an important component as material objects or embodied concepts.Database and narrative elements in cinematic writing
Database Dimension (explicit) Material Digital Elements—their study answers the question ‘What are they made of?’ | |||
|---|---|---|---|
Writing | Images | Sounds | Movements |
Citations Alphabetic text Annotations | Personal photos Internet stock images Self-generated graphics and visual elements Database graphics Illustrations Diagrams | Mobile sound recordings Internet stock sounds | Mobile video recordings Internet videos Animated elements |
Narrative Dimension (implicit/explicit) Organising Material Elements—their study answers the question: ‘What are the patterns and relations?’ | |||
Emotioning
Stimulus
Representing
| Mind cinema (implicit) | ||
Emergent patterns and representations (explicit) | |||
In cinematic writing, explicit patterns of meaning are represented with a gestalt of modes. An explicit thought is expressed with a remix of modalities (writing, images, sounds and movements). It becomes an embodiment of the implicit mind-cinema, where narrative and database are seen, in the words of Hayles (2012), as ‘natural symbionts’ (p. 175).
Because a database can construct relational juxtapositions but is helpless to interpret or explain them, it needs narrative to make its results meaningful. Narrative, for its part, needs the database in the computationally intensive culture of the new millennium to enhance its cultural authority and test the generality of its insights. If narrative often dissolves into the database […] the database catalyses and indeed demands narrative’s reappearance as soon as meaning and interpretation are required. (p. 175)
As cultural mediators, the narrative and database are both important component of contemporary system of communication. In fact, in the process of meaning construction, the boundaries between their roles are often merged, making it often hard to determine whether it was a database or narrative that led to a certain development.
3.4 Overtonal Montage
Cinematography, claimed Eisenstein (1949), ‘is, first and foremost, montage’ (p. 28). ‘Primo: photofragments of nature are recorded; secundo: these fragments are combined in various ways. Thus, the shot (or frame), and thus, montage’ (p. 3, italicised by the author). If we compare the layered construction of representations with the principle of montage, we can see that each layer correlates to primo—a shot; and their relationship, to secundo—‘combination in various ways’. As we apply different filters, the relationship between layers is changed and unified projection is therefore altered.
In describing Eisenstein’s theory of montage, James Gibson (1979) relates montage to a similar art genre—collage. He writes about painters’ practice of collage: ‘The associating of scraps, pieces, pictures, or forms not previously associated was thought to yield a fresh insight, or unexpected gestalt’ (p. 286). Comparing painting to film, Gibson maintains that the film is different in that it ‘is composed of events and superordinate events, of episodes, happenings, and history’ (p. 287). As illustrated above, in cinematic writing, the events or episodes in the form of text(s), photos, generated graphics, sounds, recorded videos, coded movements and so forth (primos) are positioned not on the frames of a film-thread but on the representational layers. Thus, the final outcome is produced not in a linear order of the rolling frame-montage (secundos) but in a collapsed projection of the layers. The layers can be projected into each other in diverse ways, enabling the effect with various degrees of distortion. The result is similar to how Eisenstein (1949) describes the outcome of a film-montage. It can ‘fluctuate from exact naturalistic combinations of visual, interrelated experiences to complete alterations, arrangements unforeseen by nature and even to abstract formalism, with remnants of reality’ (p. 3). The features of the montage can be altered and those alterations intensified according to the intended purpose and ‘social premises of the maker of the film-composition’ (p. 3).
In Eisenstein’s theory of montage, the intensifying or distorting combinations that take place within the frames are called the ‘overtonal complex of the shot’ (p. 78). These collateral vibrations become merely “disturbing” elements, the ‘same vibrations in music—in compositions, become one of the most significant means for affect by the experimental composers of our century such as Debussy and Scriabin’ (p. 78).
[It] is built, not on particular dominants, but takes as its guide the total stimulation through all stimuli. That is the original shot, arising from the collision and combination of the individual stimuli inherent in it.
These stimuli are heterogeneous as regards their ‘external natures’, but their reflex-physiological essence binds them together in an iron unity […] behind the general indication of the shot, the physiological summary of its vibrations as a whole […]. (p. 67)
Overtonal montage in cinema produces an analogous effect to one generated by a unified projection of the layers in cinematic writing.
3.5 Cinema Thinking and Sociological Imagination
The process of composing and reading cinematic text analysed through the lens of the mind-cinema can be described, borrowing from Eisenstein (1949, p. 17), as ‘cinema-thinking’ (p. 17), which for him was a ‘synthesis of all the best that has been done by our silent cinematography, towards a synthesis of these with the demands of today’ (p. 17). Ironically, Eisenstein’s words bear sad historical and ideological symbolism, the synthesis of two things. The first is ‘silent cinematography’, which can be seen not only as a sequential stage in cinematic evolution, but as an attribute that marks ‘the epoch […] of socialist realism’. The second is ‘the demands of today’ (p. 17) in which artists, including Eisenstein himself, had to keep silent about the atrocities of Stalin’s regime. These unintentionally offer an acute account of the inhuman reality of Soviet times. According to the demands of Eisenstein’s day, ‘cinema thinking’ had to be kept silent within the confines of an individual mind.
Theorising about cinema-thinking, Eisenstein asserts that ‘in mastering the technique of frame, and the theory of montage, we have another credit to list—the value of profound ties with the traditions and methodology of literature’ (p. 17). The film is language, he continues, and the film-language is called upon by cinema-thinking ‘to embody the philosophy and ideology of the victorious proletariat’ (p. 17).
Cinematic writing is actuated by digital media. It is a language that allows individuals to reach the recesses of their own mind, to bring up that which for various reasons has been kept silent for a period of time and engage with these aspects through cinema-thinking. Representing the objects from memory and direct observation, by overlaying written descriptions with sounds or music associated with them and enabling them with movements, the producer is presented with an opportunity to make sense of concepts that were previously obscured. Thus, cinematic writing is imbued with the potential for what Giddens (1991) defines as ‘a corrective intervention […] not merely a chronicle of […] events’ (p. 72).
In information societies, ‘the demands of today’ lie in sharp opposition to those forged by the architects of the coercive regimes, where mind-cinema had to remain silent. To address the critical issues of ecology, economy, politics, and social and personal relations, digital media allows individuals to express their voices, and provides them with the opportunity to be released from the constraints of fear or apathy.
‘Neither the life of an individual nor the history of a society can be understood without understanding both’, argued Mills (1959, p. 3). In cultivating a free selfhood ‘in accordance to cherished values’ (p. 4), today’s individuals are given personalised tools of production to conduct self-reflective and self-representational practices. Due to the affordances and wide availability of digital media devices, individual cinema-thinking is moving ‘towards a synthesis with the demands of today’, that is, from a silent cinematography of the mind to acquiring a voice in representing autobiographies, thus asserting personally established social positions.
This can be observed in the realm of social media. Arguing from positions of personal experiences, individuals enter into dialogue with other people, often those whom they do not know and whose responses are therefore not biased by the need to maintain a good rapport with them. Such a condition allows the responses and comments to be more straightforward than in discussions conducted face-to-face, and they thus have the potential to provoke a more rigorous reassessment of positions and individual world views.
In continuing to observe autobiographic representations and the dialogues evolving around them on social media, it can be noticed that visual accounts become more prominent while written words are reduced to their commentaries. This bears a likeness to cinematic writing. In social media, cinematic writing is only emerging; what can be observed, however, are indications that cinematic writing may evolve into a genre for social media. An indication of this, for example, is the ‘live’ image option available on iPhone cameras, which by its mere existence encourages the photographer to engage with the aesthetics of figure/ground and motion/fixed-frames. With this technique, people can represent not only the events from their lives but also fleeting moments of themselves being situated at a certain intersection of the space/time continuum and their observable existence among others. Such an activity may become a stage for developing what Mills terms the ‘sociological imagination’ (p. 5). Sociological imagination provides individuals with the ability to juxtapose various elements from their biographies with the relevant social conditions of a particular historical moment, making the individuals actively conscious of their personal qualities and worldviews. Through representing, understanding and establishing their own selfhood, individuals cultivate a capacity ‘to shift from one perspective to another […]; from examination of a single family to comparative assessment’ (p. 7) of a range of sociological issues.
The sociological imagination can be considered one of the key principles in composing cinematic writing. As Altglas (2014) argues, the postmodern world ‘involves fragmentation of information and knowledge, the collapse of boundaries between reality and representations’ (p. 4). In reflexive societies, Altglas asserts, people construct identities through personal choices. In creating these identities, individuals remix the fragments of their personal biographies with social events, utilising the styles of ‘patchwork, pick and mix, and the pastiche’ (as mentioned earlier) to do so. Following Giddens’ (1991) notion that ‘we are not what we are, but what we make of ourselves’ (p. 75), we can say that individuals in the digital world ‘deliberately juxtapose seemingly contradictory styles derived from immensely different sources’ (Grenz 1996, loc. 438) to remake themselves through their new representations. Grenz continues: ‘This technique not only serves to celebrate diversity but also offers a means to express subtle rejection of the dominance of rationality in a playful or ironic manner’ (loc. 438).
3.6 Convergent Points
Cinematic writing is informed by the notion of typography as a movie projector. McLuhan’s suggestion to see letters assembled in words, sentences and lines like a movie film running inside someone’s mind inspired the concept of remixing writing with images, sounds, motions and interactions.
The inauguration of such a concept would not be possible without the creative affordances of new media that empower the writer not only with personal publishing production tools but also with the possibility of remixing various representational modalities, which was not possible before.
A layered system of creative software plays a significant role in the process of modal remixability and a document compilation for publishing. In addition, it enhances our understanding of the representation
meaning-making interactions as two moving cogs of one mechanism. Observing the circuity of these functions on a metacognitive level, a layered system further enables the identification of such categories in the processing of the mind-cinema as emotioning
reasoning. This is another circuity through which the mind-cinema unfolds, giving rise to stimulus
response gears, which in turn, are inseparable elements of the whole production mechanism.
Representation
meaning-making of a mental grasp can be compared to the overtonal montage
in cinematography. As in overtonal montage, the mental grasp presented in a unified projection of the layers encompasses all the sensory ‘collateral vibrations’ associated with the grasp. It creates an effect of conflict, which forces the reader to look at the expressed meaning in a new way.
meaning-making experiences to the collective allows the development of sociological imagination in learners and fosters participation in the global, cosmopolitan community.Key terms | |
|---|---|
Affordances | – Capacity of the environment and the tools of production |
Bricole | – Material object such as a photograph, scanned image, computer generated graphic, embodied text, audio or video fragment that is used in the construction of patterns of meaning |
Cinematic writing | – Digital writing with the inclusion of images, sounds, motions and interactions |
Deep remixability | – Remixing a wide range of elements from heterogeneous resources: environments, digital, hand-crafted, objects, techniques and concepts |
Emotioning
| – Circularity of two interdependent actions within the process of representing
|
Mental grasp | – A ‘halt’ perceived in a ‘mind-cinema’ that is distinct and is determined to be made materially evident |
Mind-cinema | – An emergent property that results from the process of representation
|
Mode | – Material representational system for making meaning evident |
Overtonal montage | – Mental grasp that is represented by the use of various representational modalities that are meant to work as one whole |
Database (paradigmatic dimension—bricole s) | – Collections of categories of material elements used in the process of meaning-making |
Semiosis | – Making of meaning |
Sociological imagination | – Ability to position one’s personal biography within a larger historical/socio-cultural framework |
Stimulus
| – A circularity of two interdepended actions within a process of representation
|
Narrative (syntagmatic dimension) | – System of organisation of paradigmatic (database-bricoles ) elements |
3.7 DOING
KNOWING: The Ripples Pedagogy in Practice
3.7.1 Learning Task Two: The Shower of Experiences
A thought may be compared to a cloud shedding a shower of words. Precisely because thought does not have its automatic counterpart in words, the transition from thought to words leads through meaning. (Vygotsky 2012 [1934], loc. 346)
- 1.
Read Vygotsky’s quote closely
- (a)
mind-cinema and mental grasp
- (b)
overtonal montage;
- (c)
a thought shedding a shower of words.
Draw a conceptual map in whatever form you can—visualisation, symbolic/schematic representation, diagram, table, graph, and so on—to exemplify how these three concepts relate to each other. Put some creative thoughts into your map. Draw it with coloured pencils and multi-ink ballpoint pens to make a comprehensive embodiment of your thinking.
- 2.
Observe your own mind-cinema
To observe your own mind is not an easy task. As the famous film theorist and psychologist Rudolf Arnheim (1962) noted: ‘[…] the person’s own mind tends to shrink when it is watched’ (p. 1). The more something is concealed, however, the more resolute is the force by which the learner’s curiosity and thrill for discovery is triggered. In observing your own mind, turn yourself into ‘a mastermind’.
Use analogies, metaphors or similes. What is your mind-cinema like? Is it like a train swishing by so fast that the instant you notice something interesting, you can only see the tail of it disappearing? Is it as if someone is flicking through TV channels? Is it like ripples in a pond, with one image being wiped up by another? Is it like lottery balls jumping in a container? Once you get a feel for your mind-cinema’s nature, you can work out a strategy for ‘catching’ its frames and observe them more closely.
Open a desktop publishing program, graphic design program, Power Point or any other software that allows you to write with inclusion of images, sounds, motion or interactivity. Create a new document. Write a freestyle account of your experiment. You don’t need to write about any specific or explicit details if you do not want to—observe the phenomenon as a whole. For example, do your feelings change with the changes of the images in your mind, and if so, how? Maybe you don’t really see any images but instead perceive concepts through your other senses? Be creative in describing your mind-cinema; and be investigative in making meaning of it.
Write approximately 300 words.
- 3.
Place the photo of your mind-map inside the Cinematic Writing document
- 4.
Watch the YouTube video ‘Handlebars’ by Flobots:
https://www.youtube.com/watch?v=HLUX0y4EptA
- (a)
talk about the audience—pre-service teachers;
- (b)
challenge your readers either by promoting this video for education or disapproving its use for learning;
- (c)
include areas such as: content, relevance to today’s young people, style of presentation, music and animation, effectiveness of the multimodal communication in conveying the message.
- 5.
Watch the video again
- 6.
Record a short video of your surroundings
With either your mobile or tablet, record a short video of your surroundings. It could be an outdoor or indoor setting, in your room, or any other rooms in the house. It can be a university yard or lecture theatre. Wrap your mind-cinema song together with your video recording. You can do a remix of your surroundings to create a more exciting video clip.
Write a reflection of this task. Make links to the theory material presented in this chapter. What did you learn in this task? In what ways can you identify this task as a knowledge-production activity?