The Materiality of Writing

Trace-making is the material inscription of graphic traces spatially arranged on a surface. The articulation of material inscriptions is a record of the trace-maker’s activity, arising through the movement of a graphic tool on a surface. Cultural constructions of graphic trace-making based on literacy have induced us to think about graphic traces in terms of writing. Writing is a second-order cultural construct that transforms our first-order experience of making, perceiving and responding to graphic traces. Consequently, much of current thinking about graphic trace-making is shaped by second-order constructs and theories of writing that have little to do with and little to say about the material dynamics of our interactivity with trace-making (Thibault, 2011). To use Linell’s (2011/2005) felicitous term, it is time to move beyond the ‘written language bias’ that informs not only the linguist’s understanding of spoken and written language, but, I submit, of graphic traces as well. With reference to some examples of young peoples’ trace-making, I examine their first-order interactivity with graphic traces in order to focus more clearly on the links between learners’ first-order experience of graphic traces and the second-order demands of writing. I look at the tensions and disharmonies as well as the complementarities between the two, including the ways in which the graphic traces and writing mutually inform each other in the trace-making activities and experiences of young people.

I start with the empirical fact that no learning and no knowledge can take place without movement (Held and Hein, 1963) and apply it to the learning and knowledge that arises in graphic trace-making. Empirical studies of interactivity by experimental psychologists (Vallee-Tourangeau et al., 2011) show that persons do not interact with artefacts as already given wholes. Rather, sequences of exploratory movements that unfold in time give rise to experiences of aspects of the affordance layout of the artefact. The experiences so generated have irreducible kinaesthetic and sensory dimensions. The basis of our coming to know an artefact, and hence its cognitive constitution as a particular form (ideation), lies in the emergence of a form from an unfolding trajectory of movement sequences and thereby in discovering how an aspect changes into another aspect along this trajectory.

The production and reception (perception) of graphic traces are thus brought forth by agents’ sensory-kinetic experience of bodies-in-movement (Sheets-Johnstone, 2010, 2011; Stuart, 2010; Stuart and Thibault, 2015) when they couple to and explore the affordance layouts of graphic traces. These movements have their distinctive “kinetic melodies” (Luria, 1973) that link movement sequences into distinctive wholes with their characteristric feel and emotional tone. Drawing on the idea of “haecceity” in the work of Deleuze and Guattari (2004/1980: 287–292), I consider how the diverse trajectories of movement and becoming engendered by graphic traces are extensions of the agent’s world and its being-in-the-word and the means whereby it threads and loops its way into the world as a self who has perspectives on the world. Graphic traces afford the possibility of the self’s engaging with them as objects of perceptual exploration that can be returned to, reflected on, and talked about thereby enabling the self to become self-aware of its own perceptual activity.

2. Trace-Making and Gibson’s Concept of the Fundamental Graphic Act

In this section, I clarify some basic principles of trace-making in the ecological framework first developed by Gibson (1986/1989) with reference to children’s early trace-making. Gibson defined a picture as a surface treated by photographic or chirographic means so that the surface makes available to observers “an optic array of arrested structures with underlying invariants of structure.” (1986/1979: 272). The visual invariants referred to by Gibson are formless. The array of optical information that is installed on a surface is also a record of the trace-maker’s activity. Gibson is at pains to point out that a picture is not typically a record of perception, though it can be as in the case of photography. Rather, it is a record of perceptual (visual) invariants that the observer has extracted from the optic array and recorded on a surface by the producer’s trace-making activity. The recording of the extracted invariants can thus be stored, retrieved, and inspected by the same observer or by different observers on different occasions. A picture is not necessarily or even usually a record of perception because the optic array and the invariants that can be extracted from it persist in time and do not depend on perception from a point of observation in the here and now. Instead, we can have non-perceptual awareness of invariants experienced in the past as well as those projected into some imaginary future (Gibson, 1986/1979: 274).

The fundamental graphic act is a permanent visual record of the movement of a tracing tool over a surface. This requires the coordination of a graphic tool (pencil, pen, crayon, etc.) with the hand–eye system. Gibson proposes that both depiction and handwriting develop in the child from what he calls the fundamental graphic act. Gibson defines this as follows: “the making of traces on a surface that constitute a progressive record of movement” (Gibson, 1986/1979: 275). He identifies the following visual invariants in the trace-making activities of young children: the quality of being straight; the quality of being curved; opposite curves; a trace can have a beginning and an ending, or it can be continuous; a continuous trace can change direction with a jerk or a zigzag; a trace can connect two marks; two marks can be lined up; a trace can exhibit closure when a continuous trace comes back to where it began; a trace can connect with other traces to produce an intersection (Gibson, 1986/1979: 276). Gibson observes:

Figure 3.1 shows an example of one child’s early trace-making in which fundamental visual invariants such as those identified by Gibson are featured. The child’s drawing in Figure 3.1 evidences many of the visual invariants listed above. There are curved lines, lines that suddenly change direction, intersections of lines, parallel lines, lines that curve back on themselves, nestings of shapes within other shapes and so on.

We also recognise that it is a drawing of a face featuring hair, eyes, a mouth, and so on. The child has a perceptual awareness of the idea of a face and manages to extract and record on a surface through her trace-making a record of the visual invariants she as extracted from the optic array and selected for attention. The child in question drew this picture of her father’s face one day in March 1987 when she was at the ‘asilo nido’ (nursery school) she attended in Bologna. Her father was not present at the time she drew the face therefore she did not produce the drawing on the basis of a here-and-now perception of her father’s face. Instead, she has selectively extracted visual invariants in the optic array which she has noticed, most probably across multiple occasions of attending to faces, and organised the invariants so selected in order to produce the idea of a face. Her ability to select invariants is in theory unlimited and depends on her perceptually honed skills in making ever finer discriminations in the optic array. As Gibson (1986/1979: 243) points out, there is no limit to perception in this sense. Perception depends upon our learned ability to deploy the skills required to make ever finer discriminations in the perceptual array.

The visual invariants referred to above always occur one dimension lower than the image—for example the face—which one perceives. Using the mathematical variable n to indicate the number of dimensions, I suggest that the selected invariants always have n − 1 dimensions that are unfolded as dynamic trace-making events that interact with the material substrate of a surface (Johannessen, 2016: 5–6) along a time-extended trajectory, as also recognised in Gibson’s account of the fundamental graphic act. Instead of saying that the observer of the picture in Figure 3.1 imposes an abstract schema or a form on it in order to make sense of it, a materialist metaphysics of historically constituted processes of the kind envisaged by Deleuze and Guattari (2004/1980: 23) argues that the structure of the possibility space is always immanent, not transcendent. The imposition of a transcendent face-form or face-schema would be an external imposition existing on some higher dimension, n + 1, as in Aristotle’s highly influential and enduring ontology of formal essences.

According to Aristotelian realism, essences are the transcendent forms that generate real entities as copies or instantiations of them. In Aristotle’s Metaphysics (1996/1933), the generation or coming into being of a substance is seen as a hylomorphic combining of form and matter: both form and matter pre-exist their combination, but it is the form which shapes and defines the resulting hylomorphic combination of form with matter to create a particular substance (entity) (Aristotle, 1996/1933: 355). Form is essence in this sense.¹ One of Aristotle’s examples is the building of a house (Aristotle, 1996/1933: 339–347). The builder has in mind a pre-existing plan or form, which is then instantiated as a material house. The essence of the house is defined by the pre-existing form. A given entity is thus defined by a particular form of substance, which defines the conditions for that entity being a substance of a particular essence or form (see also DeLanda, 2010: 123). For Aristotle (1996/1933: 353–357), the “formula” (“definition”) of a syllable contains the formula of the letters (in contrast to the segments of a circle, which are not contained in the circle’s formula) because “the letters are parts of the formula of the form; they are not matter;” whereas in the case of the circle “the segments are parts in the sense of matter in which the form is induced” (Aristotle, 1996/1933: 357). Letters for Aristotle are formal essences that are separate from the materiality of trace-making activity.²

Deleuze’s ontology of immanent, not transcendent, material processes is realist, process-based, and historical in contrast to the ontology of substances, entities and the formal causes that are used to explain the coming into existence of the entities that are assumed to populate the world. It seeks to explain the historical emergence of the particular organisations of process on different strata, for example geological, biological, social, cultural, linguistic, that constitute the world. In the Deleuze-inspired ontology I am developing here, the n − 1 dimension is an immanent, not transcendent, structure of possibilities—a virtual possibility space or a virtual field of differential relations (the n − 1 dimension)—that can be actualised in potentially many divergent ways in a n-dimensional space of perceived forms. The idea of a face is just such an n-dimensional multiplicity.

When I perceive a face in the child’s drawing in Figure 3.1, I am actualising in the n-dimension the potentialities of the virtual field of differential relations constituted by the selections of invariants that constitute the delimited optic array which she has recorded on a surface. The idea or concept of a face is not then an abstract form that is imposed from the outside on an inert and recalcitrant matter; it is not a higher-order essence or form that unifies under the one idea all possible instantiations. There is no need to postulate a unitary n + 1 dimension from which all possible instantiations of a formal essence are derived (Deleuze, 2004/1968: 230–232; Deleuze and Guattari, 2000/1980: 23). Instead, the idea or concept of a face is a structured space of possibilities that can be actualised in divergent ways rather than being the material realisation of a preexisting abstract form or schema. For example, the tendency to perceive a face in the child’s drawing is a real tendency even if it is not always actualised on any given occasion by a particular observer. The visual invariants on n − 1 are topological invariants in many different dimensions and organised in many different ways that structure a concrete possibility space. The perception of a (picture of a) face is an actualisation of a virtual structure that is immanent in the space of possibilities; it is not imposed on it from the outside. Aristotle’s hylomorphic account, on the other hand, explains the generation of form in matter in terms of a unitary n + 1 principle that is externally imposed. Both Aristotle’s original account and its modern counterparts are unable to explain the self-organising capacities of the complex matter, energy and information flows that form human culture.

3. De-Stratifying Trace-Making

On the basis of the arguments developed in the preceding section that focus on the immanent potentialities of a concrete possibility space, we can in effect de-stratify the child’s record of her trace-making. Rather than seeing it as a stratified finished product in which a given selection and recording of visual invariants on a surface expresses or signifies a picture of a face, we can de-stratify this view and focus not on the finished product but on the production processes of the observer/perceiver. The first view—the stratified finished product, the picture of a face—takes the perspective of the designer and the designer’s finished product.

Many aspects of the world in which we live are defined by spaces that have extensive boundaries. The familiar ‘entities’ of our everyday world are defined in this way. They have extensive boundaries and definite properties such as shape, volume, size and so on. They can be segmented as perceived entities. On the other hand, many of the processes that drive and sustain the existence of these bounded ‘entities’ are driven by intensive processes (Deleuze, 2004/1968: chap. 5; DeLanda, 2010: 116). In graphic trace-making, intensive processes include, for example, differences in, for example, the speed and the pressure with which a trace-making tool is applied in the act of trace-making to a surface, the flow and viscosity of the ink or other substance that leaves the trace on the surface, the manner of its physical and chemical interaction with the surface and so on.

Articulatory processes entail matter and energy flows that can be modulated in ways that leave a trace in the products that are inscribed or traced onto a surface. The n − 1 dimension reminds us of these intensive processes based on flows of matter and energy. If we attend to the end products, that is the forms on the n-dimension, we cancel out the significance of these form-creating or morphogenetic intensive processes that leave their trace in the articulatory processes of the n − 1 dimension. Media are not shapeless, inert or passive lumps of lifeless matter on which we impose a form from the outside, as in Aristotle’s hylomorphic model of creation. Rather, materials have their intrinsic properties, capacities, and tendencies that interact with the articulatory tool and the articulatory actions of the trace-maker. The interactions of these factors spontaneously give rise to objective material processes and possibilities that are intrinsic to the objective material properties of the trace-maker–surface–graphic tool interaction system. What we conventionalise, describe and formalise as ‘forms’ are second-order informational constraints that function like future attractors: they guide and give shape to the matter-energy flows whereby the immanent possibilities of material entities are actualised. It is not the case that a pre-existing plan or form is simply instantiated as a material copy of that form. This is not to deny that trace-makers have plans and intentions. My point is that plans and intentions are immanent in the trace-makers’ activity rather than transcendently standing above or outside of it. Thus, the initial idea or plan one might have before starting guides and modulates the entire process along its trajectory from start to completion under the influence of positive and negative feedback loops that modify the initial idea or plan in concert with the development with the unfolding trajectory. Ideas or plans do not predetermine the final outcome in the manner of an efficient case that gets the process going and then stops once the process has started.

Processes of graphetic articulation (Johannesssen, 2010) on the n − 1 dimension involve intensive matter-energy flows in which the material properties and capacities (affordances) of the surface (e.g. paper), the trace-making tool (e.g. pencil, crayon) and the skills and capacities of the trace-maker interact in ways that change each other and thus give rise to emergent form-giving properties that were intrinsic to the material system and its interacting components. The interaction of all these factors during the time-extended production of a graphic trace is a live gradient that orders and gives pattern to it. This pattern has a specific identity that is informational and serves to identify the trace. The completed trace, the final product on a surface, is thus expressive in the sense that raw physical pattern (colour, texture, etc.) expresses its identity. Every phenomenon in the universe has this capacity: properties of the phenomenon interact with something in their environment in ways that leave an informational trace that tells a potential observer about the identity of that phenomenon. Graphic traces in the human ecology build on and exploit this basic physical fact.

Physical information in this sense is not to be confused with form. Forms, however, are not imposed from outside. Instead, they spontaneously emerge when all of the component parts of the relevant system (surface, graphic tool, hand–eye–brain system of tracemaker and his or her past experience, and sociocultural continuities of practice) interact with each other. If we focus exclusively on the end products, as seen from the perspective of the n-dimension, we lose sight of the immanent form-creating processes and tendencies on the n − 1 dimension that give rise to these forms qua finished products (n-dimension) and which leave their material traces in them. Forms qua informational constraints guide, shape and emerge from the intrinsic and dynamical potentialities of these matter–energy flows; they are not imposed from the outside on an inert matter. Every craftsman and artist already knows this.

Finished products are defined by their extensive properties (DeLanda, 2010: 128). The visual invariants and the interrelations between them have been sorted into a definite final form that yields the picture of a face in our present example. Final products have extensive properties that can be segmented in definite ways (Kanizsa, 1980). The face in Figure 3.1 is a volume that occupies a definite area, it has a particular shape, and so on. These properties are the final outcomes of prior production processes. In that sense, semioticians may say that a particular organisation of visual invariants signifies or realises a picture of a face. The finished product is a stratification of the dynamical field of virtual potentialities that generated the form, seen as content that is expressed by a particular arrangement of visual invariants. The resulting product—the visual sign—is accordingly stratified in terms of its expression and content planes of organisation in accordance with principles laid down by Saussure (1993/1907, 1910–1911) and elaborated by Hjelmslev (1954) in their theories of the linguistic sign (see Thibault, 2007).

Perceivers are also producers though in different ways from the trace-maker who creates the drawing. Gibson showed that perception is an active and time-extended process of exploration and discovery rather than a passive inputting of sensory data by the senses (Gibson, 1986/1979: 240). When we focus on the production processes of both the trace-maker and the perceiver, we are in the domain of intensive differences, flows of materials and energy, and the singularities or critical thresholds that change quantity into quality when a quantitative change becomes a qualitative change that has a determinate value in the system. Perceivers do not register an instantaneous perceptual form. Perception is not a snapshot of an already-given reality. Instead, it is, as Gibson (1986/1979: 240) showed, a continuous stream of exploratory activity that takes place in time. Perceivers engage with and follow the flows of the visual invariants and the connections between these invariants as they follow, with their eyes, hands, voices and so on, the patterns of flow that the spatial arrangements of visual invariants on a surface enable. The perceiver’s interactivity with the flow of invariants just is the production process. It is through these production processes that the potentialities of the virtual possibility space—the n − 1 dimension—are actualised as manifest forms that are experienced on level n.

In other words, it is the pattern of the perceiver’s interactivity with the affordances of the graphic trace that differentiates and construes the form that is interacted with. On this view, there is no need for the imposition from outside of a transcendent form that pre-existed its realisation as a particular material manifestation of the transcendent form and its properties. The regularities in both the product and the processes that yielded the product can be accounted for in a purely immanent way without recourse to transcendent form. Instead, forms emerge as a consequence of our material interactivity with the dynamical properties and flows that are intrinsic to the material world. They are informational constraints that enable us to manage and direct our interactivity in productive ways that develop solutions to the problematics that our interactivity with, for instance, material artefacts encounters.

With these considerations in mind, it is now possible to rethink the structuralist idea of stratification in more dynamic terms as organisation and integration of process across different ecological scales (see Section 8). A virtual structure is a dynamical field or network of differential relations. Virtual possibility spaces are not static structures that generate static forms. When the perceiver interactively explores the visual invariants as a time-extended flow, his or her activity sets up and catalyses an organisation of interactions among processes that is dependent on: (1) the relationships of the visual invariants with each other, (2) the neural and bodily processes of the perceiver and (3) the relationships between the visual invariants and the perceiver’s activity. An organisation of process thus depends on how the perceiver interactively explores in time the visual invariants. The perception of a visual form like the face in Figure 3.1 is thus an emergent consequence of how a given interaction of processes are organised, which will in turn generate their properties and their functional capacities and affordances. The perception of a particular visual form—for example a face—is then a particular functional organisation of processes, that is an organisation that serves a particular function. The idea of a face (the n-dimension) is then dependent on the organization of the visual invariants into dyamic patterns that play particular functional roles in the larger organisational whole.

Thus, in Figure 3.1, some dynamical patterns are perceived as the eyes, the hair, the mouth and so on—all functioning in relation to the larger whole (the face). Some of the processes involved like the visual invariants recorded by the trace-maker on a surface are persisting whereas others like the perceiver’s neural and bodily activity are fleeting. The significance of the persisting patterns lies not in the mere fact that they are material marks or traces on a surface, but in the ways in which a form of functional organisation persists which makes possible certain kinds of interactivity (i.e. perceptual exploration) with it and in ways which enable the interactivity of the fleeting and the persistent to give rise to functional organisations of process that serve particular functions for the perceivers and users of graphic traces. For the child producer of the drawing in Figure 3.1, we might say, following Gibson (1986/1979: 274–276), that her tracings serve as persisting visual records which afford their exploration as stimulants to further perceptual exploration and discovery.

Figure 3.2 sets out the n- and n − 1 dimensions that are involved in the production of forms. It would however be mistaken to see the two dimensions as distinct levels, as in structuralist accounts of the biplanar organisation of the linguistic sign. Instead, the two dimensions are co-extensive with each other; they inhabit the same scale and are intertwined with each other rather than being on separate levels. The dotted line simply serves to highlight that the two dimensions are not two distinct levels that are related to each other as in the Saussurean theory of the linguistic sign. Instead, it is intended to show that (1) the properties of the whole on the n-dimension are not transcendent with respect to the n − 1 dimension but are immanent in the possibility space of the latter and (2) the possibility space of the n − 1 dimension is immanent in the divergent actualisations of this possibility space that are experienced as manifest forms defined by the n-dimensional parameters that actualise and give structure to a determinate experience of the form. The arrow head highlights the temporal and processual dimensions of sign-making as an activity that occurs in time.

Rather than a relationship between two levels of abstract form, as postulated by Saussure’s (1993/1907, 1910–1911) account of the linguistic sign, graphic trace-making is a material activity that co-articulates hand movements, eye movements, graphic tool (e.g. a pen, brush), a surface (e.g. paper), a medium (light) and writing materials (e.g. ink, paint, chalk). These behaviours of the intensive flows generated by the co-articulation of these factors exhibit abrupt changes in form at critical thresholds such as variation in the speed of the movement of a tool over the surface, variations in pressure, changes in direction, straightness, curvature and so on. These are intensive properties that are marked by critical points or thresholds at which a qualitative change is perceived to occur. Deleuze and Guattari (2004/1980: 435–458) argue that the essential relation is not between abstract form and homogeneous matter but between materials and forces. Heterogeneous materials with their variable properties are enlivened by forces of tension and energy that give rise to the forms of the world. Trace-making is, then, less a matter of projecting a pre-existing form onto inert matter than a working with and intervening in materials, their fields of force and their flows, in order to give rise to forms (see also Ingold, 2011: 210–212).

With reference to “Un-straight shape features” (in contrast to “straight” ones), Johannessen (2016: 9–10), points out that the difference between ‘convex’ and ‘concave’—that is Un-straightness directed into or out from a graphic region (Johannessen, 2016: 9)—that yields the following “five possible permutations of Shape choices: Straight, Un-straight>Convex/Curve, Unstraight> Concave/Curve, Un-straight>Convex>Angle, and Un-straight>Concave/Angle” (Johannessen, 2016: 9). Johannessen contrasts the descriptive adequacy of these distinctions to their “observational adequacy” (2016: 10), owing to the difficulties encountered by perceivers in judging when a difference makes a difference that matters. The critical points at which perceivers make such judgements cannot, however, be located on the n-dimension but is always located on the n − 1 dimension. Johannessen takes the distinctions referred to here to be features, which at the least delicate level fall into one of two categories: Straight or Un-straight.

In my view, the focus on these categories instates a higher level of transcendent determination, n + 1, in conformity with Aristotelian essences, that reside on a higher ontological plane than the more specific differences seen to derive from them. In my view, these distinctions are better seen as variables on the n-dimension that describe the number of dimensions (differences that make a difference) on that dimension. By the same token, the intensive thresholds always have n − 1 dimensions. What Johannessen takes to be a question of judgement, I consider to be part of a still more general theoretical problem concerning the transformations that segment an exterior shape into different topological regions at which point a quantitative difference becomes a qualitative one.³

The standard idea of the sign is a purely formal relation between two abstracta, seen as a pairing of a signifier with a signified, an expression with a content, or a form with a meaning This view cuts the sign off from experience and treats it as a purely formal relation between the two levels held to constitute the sign. Figure 3.2 shows that the sign is a temporally unfolding process rather than a frozen pairing of the two levels of ‘form’ and ‘meaning’. Instead, the sign is an unfolding selection of process in a virtual field of differential relations (the n − 1 dimension), the selection of which alters the relations of the actualised organisations of process on the n dimension of the form which is apprehended. Moreover, the sign is embedded in and is constitutively inseparable from the experiential invariants that it acts on and transforms, including evaluations, affects, feelings and so on. A sign is then a transformation of the intensities associated with the selected processes and their relation to the informing ‘background’ experiences and their invariants and their transformations in the processes of graphetic articulation. For example, in the production process described earlier, the perceiver’s perception of the graphic trace resonates with its changing intensities and affects together with the perceiver’s changing intensities, affects and evaluations (e.g. interest, curiosity, fear, joy, etc.) as these interact, too, with the unfolding trajectory and the experiential invariants and their transformations that the trajectory activates.

The n −1 dimension is not a set of micro-level features that are combined to form a larger-scale entity on the n dimension. The n − 1 dimension is defined as a population of dynamic interacting micro-processes and their trajectories that are defined by intensive differences, that is by flows of matter, energy and information that enter into the production of the emergent finished products on the n dimension. The dynamic processes of graphetic articulation leave their traces in the finished product and are constitutive of that product, which is defined by an actualisation in the n-dimension of the virtual different field of n − 1 possibilities. When the finished products are reified as abstract forms that are then typologised as having distinctive features that define and categorise them, the resulting static typologies are turned into timeless essences that are said to belong to transcendent systems or codes that are the conditions of possibility of the reified formal abstracta. Pace social constructivist accounts of knowledge and the construction of knowledge, the tracemaker is not a disembodied mind who imposes disembodied forms or conceptual schema onto a passive and inert material world ‘out there’. Trace-making is not a selection of disembodied forms from abstract systems. Trace-making is a material practice that feeds off and is sustained by other material practices at the same time that it gives rise to material artefacts that become embedded in other cycles of material practices that are distributed over diverse times, places, institutions, and persons.

The trace-maker (and the perceiver) is a living human being who harnesses his or her neural and bodily resources in concrete, embodied and situated practices in concert with the intrinsic tendencies and properties of the materials with which he or she works when a trace-maker–graphic tool–ink or other substance–surface interaction system is set up in trace-making activity. All the components mentioned here that combine to form the interaction system are themselves the products of prior practices and their histories. There is no passive and inert raw matter on which order is imposed by a mental schema that is external to it, as in social constructivist accounts. The trace-maker and his or her materials are not separate poles of subject (mind) and object (matter). Instead, the trace-maker and his or her materials are entangled with each other in the activities that produce graphic traces. Furthermore, this activity is attuned to and resonates with relevant sociocultural continuities of practice and the norms and values that derive from them and which inform and guide the trace-maker’s always-embodied and situated trace-making activity.

Where then does the cultural idea of a face come from, if not from a transcendent and eternal domain of timeless forms externally imposed on matter? The answer lies in Ingold’s (2010: 94) insight that skilled performance is not a matter of imposing a preexisting mental schema or representation onto external matter. Instead, forms emerge in skilled practice and its historical continuities. A skilled performance is not the faithful execution of a pre-existing schema, but the application of bodily skills that have been honed and perfected through practice, observation and the education of attention. The novice observes and is guided by more skilled experts. Novices, in concert with skilled practitioners, participate in joint acts of observation, attention directing, bodily performance and imitating, among others, in which they learn what is important, what needs to be remembered, which discriminations are useful, and so on (Melser, 2009). In this way, the novice develops the perceptual, manual and other skills for engaging with materials, for following their flows, and for becoming more attuned to environmental discriminations that make a difference to individual and collective praxis.

Learning, as Gibson (1986/1979: 258–263) pointed out, is a process of the ‘education of attention’, of attuning the perceptual systems to the environment, of making ever more refined perceptual discriminations, and knowing how to use them. The development of these bodily skills of perception and action gives rise to sociocultural histories of skilled practices that become sedimented in communities as tradition and custom. Individuals orient to and reference these continuities in ways which both constrain and enable the individual’s skilled performance. By the same token, individuals do not mechanically reproduce these skills. Instead, they adapt, modify and transform them in ways that contribute to the renewal of the tradition. These “socioculturally constituted continuities of practice” (Linell, 2011/2005: 216) do not take the form of representations or mental schemas which are passively copied and transmitted from one generation to another. Instead, they are fields of practice in which capacities, experience, expertise, knowledge and skills are collectively and individually developed, maintained and renewed (Ingold, 2011: 15–18). The capacity to detect, orient to and make use of the picture of a face in the example under discussion is then the result of the individual’s participation in the relevant sociocultural continuities of practice.

These continuities unfold over the longer cultural-historical time scales on community and population scales and play out in situated and embodied performance on particular occasions. Rather than the outcome of a pre-existing code or plan that is transcendent with respect to and imposed on matter from the outside, the individual’s exercise of his or her skills in detecting and making use of the affordances of graphic traces is immanent in both the sociocultural continuities and regularities of practice mentioned here and the virtual field of differential relations that constitutes the possibility space of trace-making. When an individual trace-maker or perceiver engages with this possibility space, he or she draws on and orchestrates in real-time past experience, performance skills and knowledge and expertise that resonate with and draw on situation-transcending traditions of practice. In this sense, the skilled activities of making and perceiving graphic traces and for perceiving forms in them are immanent in these historical continuities of practice rather than originating from a transcendent domain of timeless forms.

In the following section, I consider some further aspects of how a four-year-old child engages with the affordances of a publicly displayed campus map at the University of Hong Kong.

4. Visual Displays as Prompts for Interactivity: Explorations of a Campus Map

One day in early April 2010 I took my four-year-old daughter, Catherine, outside so she could ride her bike. We went to an area of the University of Hong Kong (HKU) campus where I worked at that time—a small secluded piazza beside the Chong Yuet Ming Amenities Centre and which is an ideal place for children to play in. In the piazza there is a campus map. During our stay there on that afternoon, my daughter, who was not quite four at the time, got interested in the map. She climbed up onto a low retaining wall to better access the map and began tracing with her finger and eyes along the coloured markings indicating roadways, buildings and other features of the map. In order to do so, given her age and size relative to the map, she had to move back and forth along the low retaining wall placed (conveniently for her) immediately in front of the map.

A map is embedded in and enmeshed with complex networks of interactive possibilities. Typically, we might say that the map ‘represents’ or ‘stands for’ the campus. On this view, the different features of the visual display that is installed on the surface that is displayed in the square beside the Chong Yuet Ming Amenities Centre at HKU stand in a relationship of correspondence to specific features of the campus. For example a particular visual icon or image ‘stands for’ the library, another for the lily pond, another for Starbucks and so on. The idea that one thing stands for something else has little or no explanatory value. It is generally accepted that this is how representation works though we are rarely given any explanation as to how representation might work in this way. I would say that such a view is unhelpful and fails to explain the functionality of the map for people who use the affordances of the map to find their way around the HKU campus.

My then-four-year-old daughter did not at that time perceive the visual display as a map. She did not have the cultural concept or idea of a map to organise her perception of the visual display in a particular way. She did not have access to the non-local cultural resources that would have enabled her to orient to and interact with the visual display as a means of finding her way around the campus. As the video recording shows, her interaction with the map was local and focused exclusively on the patterns she discovered in the visual display through her first-order interactivity with the display. She sees things like lines, connections and intersections of lines, shapes and their colours, nestings of shapes within other shapes and so on. She does not see pictures of particular buildings on campus, their location relative to each other, the pathways leading from one building to another or her own location relative to these items. In other words, she does not perceive the map and her relation to it and to the surrounding campus as consisting of n dimensions in the form of the conceptual (visual-spatial) structures that would enable her to configure her perceptions in the culturally legitimated way that would enable her to interact with the map as a resource for finding her way around the campus; instead, she perceives the map as having n − 1 dimensions.

At four years old, she lacked the cultural resources for seeing the items she explores as the interrelated components of a campus map that in turn relates to features of the surrounding campus. I cannot say exactly how she configured her perceptions of the map, but we can obtain some clues from the video recording. She explores the immanent structure of the possibility space of the display that is installed on the surface without embedding it in a non-local reference space. She explores the variables that constitute the visual display itself without performing the culturally legitimated transformational operations that relate the visual display to the HKU campus. Her perceptual activity explores different dimensions or variables that constitute the degrees of freedom of the possibility space consisting of n − 1 dimensions. She explores and tracks de-stratified intensive flows of visual-spatial variables with her eyes, hands and voice that correspond to the distributions of visual-spatial invariants. These invariants are zones of stability that can be explored over time by bodily movements which the given invariant affords.

The idea or concept of a map that relates to its surroundings is not something that is externally imposed on it in the way that is supposed by social constructivism. There is no imposing of an abstract and transcendent schema or form on the visual display. The variables and the relations between these variables that give rise to the idea of a map are, as Deleuze (2004/1968: 230–231; Deleuze and Guattari, 2004/1980: 23) suggests, an n-dimensional space that is emergent from the space of virtual possibilities on the n − 1 dimension. The possibility space on n − 1 is defined as a field of reciprocally defining and co-articulated differential relations that are entirely dependent on each other and which can be actualised in a diversity of spatiotemporal relations at the same time that they are embodied in a variety of forms when agents couple to and explore the possibility space in and through their own bodily movements.

If we stratify the map as some kind of visual image or ‘representation’ of something, we are focused on the finished product. On this view, an expression stratum of visual invariants and the relations between them realises or expresses the idea or the conceptual structure of the map on the content stratum. We are looking at the map as a final product, defined by its extensive properties, seen as constitutive of actual entities, that is its segmentation into definite shapes with their properties, their sizes and colours, their relations to each other, the lines that connect one shape to another and so on. Moreover, these shapes and their interrelations are said to signify or stand in a particular relationship to features of the HKU campus (buildings, locations, pathways, etc.). However, Catherine’s encounter with the map allows another complementary view to come more clearly into focus. Here we see the production process defined by intensive differences and the flows of bodily movements that are driven by these differences and the critical thresholds that change quantity into quality (DeLanda, 2010: 128).

We also see the virtual structure of the possibility space on n − 1 which accounts in a purely immanent way for the regularities in the production processes and the products derived from them. Rather than an external schema or form that is imposed by an external mind on the visual display. We see how the idea of a map emerges from the virtual possibility space n − 1 by means of the production processes that actualise and give form to its potentialities through the exploratory activities of agents themselves. We see in other words Catherine’s “effort after value” (Reed, 1996: 104) as she explores the affordance layout of the visual display she has encountered. The effort after value, as Reed explains, organises directs, and motivates the values-realising activities she performs in her efforts to discover the affordances of the display. These activities are bodily movements that couple her to the display and its affordance layout. They constitute the production processes referred to above as she seeks information in the stimulus array of the display. The seeking after information, not value per se, is the way in which knowledge is created. Knowledge is not created in the way constructivists suggest, that is by imposing a form or a schema on the external world. Rather, it is created through our interactivity with the affordance structures of the world that we couple with. In doing so, we learn to explore and discover the virtual potentialities of these affordances and to actualise them through our own bodily activity. As Bergson (1896/1911: 260–261) showed, perception is virtual action. What we perceive as a result of our own time-extended exploratory activity affords the individual organism potentialities for action.

Empirical studies of interactivity by experimental psychologists (Vallee-Tourangeau et al., 2011) show that persons do not interact with artefacts as already-given wholes. Rather, sequences of exploratory perceptual movements that unfold in time give rise to experiences of aspects of the affordance layout of the artefact. The experiences so generated have irreducible kinaesthetic and sensory dimensions. The basis of our coming to know an artefact and hence its cognitive constitution as a particular form lie in the emergence of a form from an unfolding trajectory of movement sequences and thereby in discovering how an aspect changes into another aspect along this trajectory.

Things look, feel and sound the way they do because of what Noë (2004: 109) calls sensorimotor dependencies. Each perceptual modality has its own distinctive sensorimotor dependencies. Thus, how things look depends on and co-varies systematically with the ways in which we move our eyes, our head, body and so on relative to the perceived entity. Perceptual exploration is an activity in time which is characterised by “reliable patterns of sensorimotor dependence” (Noë, 2004: 109). However, our phenomenological experience of graphic traces that is generated by our exploration of their sensori-motor contingencies is not freestanding (see Noë, 2007), but is embedded in and entangled with many other ecological scales which, together, constitute the human cultural meshwork (see Section 8).

We are in touch with the world not through representations that interface between agents and their worlds, but through embodied and situated cycles of perception-action that are active, exploratory and always in movement, all the while continually adjusting, focusing and refocusing, attending to specific aspects and not others, and so on. Gibson (1986/1979: 245, 249, 263) used the term “resonance” to describe the way an organism’s perceptual systems continually adjust themselves in time in order to tune themselves in order to achieve more finely differentiations of the relevant environment. Perception is not something that is captured at an instant in time like a camera snapshot, but is experienced as a temporally extended flow of movement as the organism explores and adjusts its perceptual systems to an everchanging affordance layout.

When we examine Catherine’s video recorded interactivity with the visual display more closely we see that it is segmented into movement sequences (Gatewood, 1985: 213–214) that have a temporal continuity, a characteristic emotional tone, a characteristic tempo and a characteristic rhythm that constitute a unified mode of action or a single kinetic melody (Luria, 1973; Sheets-Johnstone, 2010, 2011; Stuart, 2010; Stuart and Thibault, 2015. I briefly describe three such segments that occur as a sequence of movements in the first twelve seconds of the video recording, as follows:

Figures 3.3 to 3.5 illustrate some of the movement segments Catherine performed in relation to different features of the HKU campus map and corresponding bilingual legend in English and Chinese. The three movement segments are part of her efforts to change quantity into quality—her efforts after value and meaning. The three segments are examples of how she segments the map into differential zones of stability. Each segment displays a relative consistency of its internal patterning that is demarcated from its predecessor by an abrupt transition. The moment of transition coincides with the crossing of a critical point or threshold that sets off a shift to a qualitatively different phase (a new movement segment). In the present example, the thresholds appear to be: (1) her reaching the top of the column of coloured circles (the transition from Segment 1 to 2) and (2) her being pulled back to one particular cluster of the coloured circles lower down (the transition from Segment 2 to 3).

The segmentation into differential zones of stability illustrated by the three examples does not correspond to conventionalised reading paths (Kress and Van Leeuwen, 1996; Baldry and Thibault, 2006) for working out the location of something on the campus or for getting from one location to another. Conventionalised reading paths enact qualitative changes in accordance with the n-dimension—seeing and using the visual display and its invariants as a map with its extensive properties—definite objects, locations, pathways and so on. Catherine does not do this, but she does nonetheless segment the visual invariants she encounters on the n − 1 dimension in ways that correspond to an intensive (not extensive) map. This intensive map corresponds to the different phases of her efforts after value and meaning in her exploration of the visual display. She explores the immanent structure of its possibility space though in ways that do not yield the required n-dimensional concept or idea of a map of the HKU campus (Deleuze, 2004/1968: 230–231).

In each of the three movement sequences described here, Catherine aligns her finger and hand movements, her gaze and her vocalising so that these actions resonate with and are synchronised with a movement sequence that constitutes a vector of attention as she explores an aspect of the affordance layout of the display. She enacts and explores a pathway through the display that orients and shapes her awareness. In each case, she abruptly transitions to a second and a third movement sequence that initiate and explore new pathways and the vectors of attention and awareness that they constitute. Clearly, she is not attending to the map in the way that a skilled practitioner would in order to find out how to get from one place to another in the terrain specified by the map. What she is doing, in my view, is putting on display and contextualising for self and others co-present her capacity to engage with the affordances of the display, to make sense of them and to regain control in the face of a momentary lapse as is the case when she stumbles while climbing onto the retaining wall.

In the episode under consideration here, Catherine performs a solo act that is however embedded in what Reed called a field of culturally promoted action. In this field, she learns about visual displays not through acquiring an inner store of ever-more precise representations that she can impose on the world. Instead, she develops in concert with more skilled practitioners the culturally shaped and attuned perceptual skills for attuning to and interacting with her environment in ever more precisely articulated and differentiated ways. As Bergson (1896/1911: 12) pointed out, the perceiver is a “centre of determination” that is organised for action in a field of virtual potentialities. As Ingold has shown, knowledge construction is not the imposing of abstract representations on externally derived sense data but a process of learning to resonate with the world in the way Gibson hypothesised and which was later confirmed by neurophysiologists such as Berthoz (2012/2009: 66–67).

Even though, on this occasion, Catherine’s performance is in one sense a solo one, no doubt motivated in the first instance by the effort after value identified by Reed (see the earlier discussion), it nonetheless takes place in a field of culturally promoted actions. In this field, she is never, in fact, alone; her activities are guided and promoted along the culture’s preferred pathways by more skilled practitioners who, through showing and telling, and the rewarding of her successes, draw her attention to those patterns which the culture considers to be useful, relevant, important, worth remembering and so on. She learns in other words the skills for discovering information in the arrays of stimulus information which she encounters and on that basis how to shape, guide and hone her own perceptual activity in ways that will enhance her efforts after meaning. Knowledge is thus understood to be the discovery of meaning in the affordance layouts encountered in the world, the developing ability to refine the skills for doing so, and for using the meanings so discovered to act in culturally appropriate ways in the field of culturally promoted and legitimated action. But does the map ‘represent’? I turn to this question in the next section.

5. Graphic Traces: Affordances for Interactivity or External Representations?

Many researchers (e.g., Clark, 1997, 2006; Dennett, 1996; Clowes, 2007; Clowes and Morse, 2005) would claim that the map is an external representation in contrast to an internal mental representation. The term external representation does not in my view explain how the map ‘represents’. It is simply taken for granted that in some unspecified way the graphic traces recorded on a surface enable or do the representing of something else that is not on the map. I want to start with the idea of the map as a material artefact that is positioned in a particular location. Moreover, it is there to serve a purpose of some kind. With these considerations in mind, I propose the following factors that need to be part of the explanation:

If a person wants to find his or her way to some place on the campus, he or she must have ways of selecting and orienting to those resources that will enable the agent to achieve the goal of finding the library or Starbucks or the lily pond. To achieve the goal of finding Starbucks rather than the lily pond, the person will need to be able to differentiate the relevant environment in ways that are relevant to particular projects or anticipated courses of action. The affordance layout of the campus map enables the map reader to differentiate the relevant environments in which it is appropriate to engage in one kind of activity rather than some other. The ability of the map reader to detect and make use of visual-graphic tracings does not mean that these correspond to or stand for something in the real campus. The icon of the lily pond in Figure 3.6 does not stand for the real lily pond. It is not an encoding of a perception of the lily pond. It does not correspond to the lily pond. The graphic trace that depicts the lily pond on the campus map is a perceptual invariant—a line that returns to where it started to create the effect of closure (see Section 2)—that has been extracted from the optic array, recorded as a graphic trace on a surface and stored thereon in ways that permit multiple inspections by multiple persons on multiple occasions.

In some other context, the same trace could be construed as an entirely different visual form. The icon of the lily pond is an aspect of a differential field of relations that serves not to represent something ‘out there’ but to affect the map reader in such a way that he or she will orient to it as something that functionally serves to help him or her set up an appropriate future course of action, for example getting from one’s current location to the lily pond. It is integrated with other items that comprise the visual display and with which it contrasts in this particular field of relations. Its construal as an image or icon of a lily pond is also grounded in its capacity to evoke map readers’ prior experiences of lily ponds and the perceptual invariants that are associated with them. The icon therefore evokes in the current interaction a non-perceptual awarenesss of lily ponds that is grounded in and constrained by past information dependencies that are based on prior experience of the relevant perceptual invariants. This is not representation in any standard sense. Rather, the past-informational dependencies that are activated in the current situation by the visual icon serves in the apperceptive updating of the current situation image (Bickhard and Ritchie, 1983: 23–24).

Past-informational dependency, as Bergson (1896/1911: 260–261), Gibson (1986/1979: 253–255) and Bickhard and Richie (1983: 25) have all shown, is a form of memory that is inseparable from and constitutive of perceptual experiencing. It is continuous with and hence an aspect of perceiving and therefore of how we perceive. As Gibson’s theory of affordances shows, perception is, in the first instance, directed at the functional capacities of that which is perceived. The perception of graphic traces is no exception to this general principle. The functional capacities of graphic traces—their affordances in the Gibsonian parlance—are what are most directly perceived. The perception of the graphic trace as an icon of a lily pond is not a copy or representation of something else but, instead, an interaction outcome grounded in past-informational dependencies that serves to indicate potential future interactions with the relevant environment.

The iconicity of immediate recognition of lily ponds qua percept and the iconicity of the image of the lily pond on the map are thus correlated so as to form an indexical sign relation that puts the two icons in a co-occurrence relation (Deacon, 1998/1997: 73–75) that has future interactive potential. The map reader recognises the two recognitions—that is the evocation of the non-perceptual awareness of lily ponds grounded in prior interaction outcomes and the iconicity of the immediate recognition of the lily pond icon on the map—as occurring together in an ordered or patterned way: the two icons co-contextualise each other in a potentially regular way and are accordingly re-organised on a higher level as an indexical sign that is anticipatory, not backwards-looking. The skilled perceiver thus makes use of the visual invariant of ‘closure’ (see the earlier discussion) that is common to the icon of the lily pond and past perceptions of lily ponds to create an analogy between past informational dependencies and current perception that is projected into the future interaction possibilities with the locally relevant environment.⁴ The resulting indexical sign just is the second-order contextual ‘rule’ or principle which correlates the two icons on the first level, in the process contextually integrating their ‘separate’ iconic properties, both to the iconic properties of the correlation between them and to the newly emergent indexical properties, and functions of the higher-level contextualising principle, whereby the indexical relationship is established.

The perceiving of the lily pond icon as such is not the perception of the icon as a representational stand in for the lily pond ‘out there’ on the HKU campus. Instead, the perception of this perceptual invariant as a ‘lily pond’ is the detection of an affordance that informationally constrains and changes the perceiver’s neural and bodily processes in ways that prepare the perceiver for potential future interaction outcomes. The skilled perceiver perceives functionally useful information—affordances—that can be appropriated and used in the service of one’s future activity. The fact of perceiving the lily pond icon at a particular position on the map is a skilful performance that is saturated with value and significance that are internally related to the anticipated future course of action, viz. getting to or finding the lily pond. The perception of that visual shape with that significance in that context in relation to the anticipated future course of action is what constitutes the content of the image, not the fact that in this context it is construed as an image of a lily pond as the interactive outcome of the perceiver’s exploration of the visual invariants recorded by the graphic trace. The anticipated future situation is the content, not any purported correspondence or stand in relation to the actual lily pond. Content is not encoded in the trace, but is constituted as a future-oriented outcome of the perceiver’s interactivity with the affordances of the trace.

The visual image of the lily pond is of course a conventional image that is linguistically anchored (see Figure 3.6). Its conventional shape serves to affect the map reader, for example, by (1) catching his or her attention perhaps by reminding him or her of lily ponds by virtue of the iconic likeness to lily ponds that the image sets up in this context and by (2) differentiating the relevant field of relations in the process of distinguishing one visual shape from another in relation to the overall field of relations in which they are embedded. However, its function is not one of corresponding to or standing in for the real lily pond, but of integrating a (possible) future course of action with the information that the map makes available about the location of the lily pond and how to get to it. The image of the lily pond on the HKU campus map is a visual form that informationally constrains the perception-action of the map reader with respect to anticipated future courses of action and how to implement them. It offers or affords possible solutions to the map reader’s current engagement with the problematic field of relations posited by the presence of the map in its particular location.

The duality of perception noted earlier in point (5) means that the viewer sees both the arrested invariants of the displayed optic array this is traced onto the surface and the invariants of the nonpictorial surface on which the pictorial surface has been traced. The perceptual discrepancy between the surfaces of the picture and the surfaces in the picture (Gibson, 1986/1979: 282) gives rise to two kinds of awareness. In many visual genres, the invariants in the picture are not connected to the invariants in the place where it is displayed. The invariants of photo of my daughter in my room are not connected to the invariants of the surface layout of the room in which the photo is displayed (see Gibson, 1986/1979: 282). The two kinds of awareness are as follows. First, there is the viewer’s awareness of his or her deictically situated relationship to the surfaces of the picture, for example the icon of the lily pond or of Starbucks. Second, there is his or her relationship to the displayed artefact itself, for example his or her physical distance from it. The HKU campus map, like all maps of its kind, is subtly and importantly different from the cases mentioned earlier, for example the photo of my daughter hanging on a wall in my room. In the case of the HKU campus map, the surfaces of the picture (its visual invariants) are connected to the layout of the surfaces in the surrounding campus. The campus map is thus viewed in the context of the surfaces that constitute the affordance layout of the surrounding campus. In other pictorial genres, the relation between the treated surface of the picture and the non-pictorial surface that surrounds it will be different, for example a portrait photograph or painting hanging on a wall in one’s home or in an art gallery.

The discrepancy noted by Gibson between the perception of the real surfaces of the environment, including the non-pictorial surfaces of the displayed artefact embedded in the environment, and the perception of the virtual objects that are made possible by the information that has been traced by a trace-maker onto a non-pictorial surface gives rise to a duality of perception. This duality of perception that is characteristic of depiction promotes and makes possible two kinds of awareness, simultaneously: (1) a direct perception of the affordance layout of the non-pictorial surface and (2) an indirect awareness of a virtual surface (Gibson, 1986/1979: 283). The map affords a virtual awareness of the campus that is embedded in and connected to the affordance layout of the surfaces of the real campus, starting with the non-pictorial surface of the displayed map itself qua material artefact that is physically installed at a particular location in the real campus. Observers dually orient to and switch between their perception of the invariants of the non-pictorial surfaces of the campus and the invariants of the graphic traces recorded on the nonpictorial surface.

The deictic switching that this duality of perception makes possible serves to organise and set up a duality of the map reader’s point of view. The map reader is thus situated in a network of potential interaction outcomes that he or she may enact in and through his or her interactivity with the interconnected affordances of (1) the forms of virtual awareness made available by the pictorial surfaces traced on the non-pictorial surface of the display and (2) the affordances of the surfaces of the real campus. These networks of potentialities for interaction are contentful because the content is internally related to the functional relationships between points (1) and (2) mentioned earlier and (3) the observer’s situated point of view. When the map reader encounters the HKU campus map, he or she is placed before a web of potential interactions and interaction outcomes which he or she may carry out on account of the map reader’s deictically situated and reflexive relationship to the web of potential interactions which engagement with the map makes available.

6. Moving With and Being Moved by Graphic Traces

As I showed in Section 5, what we somewhat unhelpfully call ‘representations’ of, say, the HKU campus are constituted as certain patterns of invariants within the overall web of interactive potentialities in which the map is embedded. The map is not arbitrarily related to these potentialities. Rather, these webs are organised in terms of how some interactive possibilities are reached via interactivity with intermediary affordances and potentialities. In time, Catherine will learn in concert with more skilled practitioners how her interactivity with selected aspects of the affordance layout of the campus map will enable her to reach and interact with the affordances of more remote possibilities. If she wants to interact with the affordances of the university library but doesn’t know how to get to the library, she can interact with the relevant affordances of the campus map in ways that will scaffold her getting to the library. She will learn and deploy the culturally promoted skills for actualising a specific set of spatiotemporal relations and embodying them in order to make her body mesh with and correspond to the flows of movement.

Graphic traces are never given all at once to the perceiver. As a consequence of visual, vocal, haptic (and other) exploration of their affordance layouts by perceivers, something is constituted as a whole that transcends the kinaesthetic and sensory dimensions of the experience of the trace as a result of the sequence of movements that unfolds in time (see the previous discussion) and their associated kinaesthetic and sensory experiences. In this way, the agent is moved by the trace to form an idea that both transcends the kinaesthetic/sensory experience at the same time that the idea is prompted or induced to emerge by the kinaesthetic/sensory experience of the graphic trace. Ideation is anticipated future interactive potential in the given situation. The forming of an ideation is not therefore the projecting of a pre-existing schema onto the trace in the manner of constructivism. Deleuze and Guattari (2004/1980: 449–451), Ingold (2011), and Roth (2014) have all stressed how what Ingold (2011: 6) calls “intransitive actions” (cf. Deleuze and Guattari’s “lines of flight”) yield pregnancy and excess that set up future potentialities of interaction; new knowledge cannot be anticipated on the basis of what is already known. Instead, the originary movement-based contact with the trace becomes, over time, abstracted from it as an ideation (a form) that constitutes its own memory (Roth, 2014: 76) as past interaction outcomes guide future activity. The setting up of ‘correspondence relations’ (Ingold, 2011: 241–243) between the performer and perceiver of a graphic trace sets up potentialities for following the movement of the trace—movement that correlates with the perceiver’s kinaesthetic and sensory experiences of the trace—not only in the present but also into anticipated futures. This movement yields situation-transcending ideational knowledge about the trace and about its embedding in and connection to flows of cultural-ecological relations and processes across multiple timescales that we follow and are moved by.

As the analysis of the three movement segments in Section 4 shows, Catherine engages in online mental rehearsing that is prompted and guided by the material artefact with which she interacts and the action-perception cycles that this interactivity engenders. Her mental rehearsing is a form of pretend activity that is stimulated and sustained by the sensory-kinetic dynamics of the kinetic melodies that she enfolds into her efforts after value and meaning in relation to the artefacts, cultural concepts and practices of the field of culturally promoted actions in which she participates. She is both anticipating potential future interactions as well as practising or rehearsing what she thinks is the right way to interact with the artefact she has encountered (Bogdan, 2013: 62). The map and the patterns she explores on it thus form the basis for the forming of reflections on it. These reflections will in time form the basis on which further reflections are projected—reflections moreover that she will in the course of further development learn to manipulate in order to take her beyond the here and now and into the realm of possible future interaction outcomes and the means to attain them.

The HKU campus map is embedded in a temporally extended and spatially defined field of relations that is constrained by populations of recurrent productions processes whose statistical tendencies, in time, become hardened into socially legitimated and promoted conventions of map reading. These conventions set limits on the temporal duration and the spatial extent of the map reader’s encounters with the map. The perceiver participates in sense-making that involves and is inseparable from bodily (haptic, optical, spatial, etc.) orientation. He or she becomes a mobile map because he or she senses his or her body in relation to the orientational field afforded by the map and employs it to move his or body along the pathways—the spatiotemporal relations—that are actualised by the perceiver’s body in seeking to access and engage with the more remote affordances—the knowable unknowns—that interactivity with the affordances of the map makes possible. The map is at the centre of a larger field of virtual potentialities—a field of potentialities for extended action and perception—that can be actualised in divergent ways when one follows one of the lines that thread out from the map and into the world. The map is, then, much more than a graphic tracing of a spatial terrain; it is also a guide to the webs of interactive potentialities of that terrain and how to reach them. Figure 3.7 models a simplified selection of these interactive potentialities.

Learning and development on their respective timescales involve the agent’s continual updating of these webs and their interactive potentialities, drawing on the agent’s interactive perceptions of its environment and its past successes and failures in interacting with these potentialities and enacting the potential interactions functionally indicated as a result of the affordances one has detected. Parts and aspects of the web will change in accordance with the interactions the agent engages in. Other changes will occur whether or not the agent engages in specific interactions. These webs constitute the agent’s meaning potential and hence its practical know-how with respect to its environment. The agent’s maintenance and development of such a network of potential environmental interactions constitutes its basic awareness of its world. While knowledge of this kind is practical, grounded in embodied–embedded interactivity, it is the agent’s very embeddedness that enables it to update its webs of interactive potentialities and to extend the semiotic-cognitive reach of the agent beyond the here and now.

These webs of possibilities are contentful: they are concrete, interconnected and provide information to agents as to what they can do (Bickhard, 2005). Content is internally related to the functional relationship involved and necessarily is from the agent’s point of view. The agent’s embeddedness in such webs of interactive potentialities is also the basis for its selfhood.⁵ Selfhood involves a perspectival and reflexive relationship to the webs of interactive potentialities in which the agent is embedded. At the same time, the spatial location of the reader also implicates time as it invites the reader to participate, both virtually and actually, in the laying down of traces in the world as one extends one’s movement from the virtuality of those afforded by the graphic traces inscribed on a surface to those that one lays down when one follows the pathways indicated by these traces and ventures out into the world. The graphic-spatial organisation of the map organises time in two senses. First, it takes time to explore the affordance layout of the map itself. Second, it structures along determinate pathways the ways we encounter the terrain specified by the map. It takes time to move down these pathways and to encounter and engage with the remote affordance layouts of, for example, the campus library or Starbucks that are presupposed by our interactivity with the affordance layout of the map itself.

What we loosely and often without clear definition call ‘representations’ and mistakenly assume that they are stands in for something else that pre-existed them are, in fact, culturally sedimented histories of past interaction outcomes and their informational dependencies that constrain and enable our ability to anticipate, contentfully, the future interaction potentials of the affordances we encounter in the present.

In the following section, I consider some aspects of the ways in which Catherine’s trace-making activities give rise to the phenomenology of the writerly self and thus the kinds of experiences that graphic trace-making makes possible when selves engage with, explore and develop their potentialities for self-becoming.

7. The Writerly Self

The artefactual properties of graphic traces afford possibilities of reflective abstraction that scaffold the processes of analysis and the construction of levels of knowing. A graphic trace is an object-event whose physical properties afford and promote (1) reflection on those lower-level properties and (2) the differentiation of some smaller set of properties out of the potentially infinite set of implicit properties and its abstraction to a next level of knowing where the abstracted property or property set is given its own explicit characterisation (Bickhard & Campbell, 1986: 85). In turn, the new level and its properties enable further possibilities of reflection on some selection of that level’s properties and the abstraction of those properties to a new level of knowing where the abstracted properties are explicitly differentiated and characterised and so on.

Trace-making is a participation in and a creation of flows of activity that stimulate self-reflexivity—the agent creates, enacts and engages with traces that can be attended to and contemplated from some point of observation. They stand out from the world yet are enmeshed with the world and its flows. They afford the possibility of the self’s engaging with them as objects of perceptual exploration that can be returned to, reflected on, talked about, and they thus enable the self to become self-aware of its own perceptual activity. Graphic traces thus afford processes of self-stimulation (Thibault, 2012: 35–40). They afford and promote possibilities of reflective abstraction that scaffold the agent’s processes of self-exploration in the process of eliciting reflection on the perceptual and other properties of the graphic trace.

In his essay “Art as Technique” (1965/1917), Viktor Shklovsky argues that as perception becomes habitual, it becomes automatic (1965/1917: 778). The resulting “over-automatization of an object” (1965/1917: 778) serves the function of permitting the “greatest economy of perceptive effort” (1965/1917: 778). However, this economy comes at a cost: “Either objects are assigned only one proper feature—a number, for example—or else they function as though by formula and do not even appear in cognition” (1965/1917: 778). The technique of art, on the other hand, “is to make objects ‘unfamiliar,’ to make forms difficult, to increase the difficulty and length of perception because the process of perception is an aesthetic end in itself and must be prolonged” (1965/1917: 778). In this way, “art removes objects from the automatism of perception in several ways” (1965/1917: 779). It does so by de-familiarising our habitual perceptions of familiar objects and events so that we perceive and cognise them in new, unfamiliar ways. Whilst Shlovsky was concerned with art, broadly defined, and the aesthetic experiences that it enables us to experience when we interact with it, I would suggest that perceptual de-familiarisation is at work in the way we interact with and experience the affordances of graphic traces in relation to the surfaces on which they are installed.

As Gibson showed, trace-making affords points of observation from which the trace can be observed, attended to, reflected on, talked about and so on. Trace-making stimulates the self to become self-aware of its own perceptual activity by virtue of the way in which the trace that is traced onto a surface of some kind is, dually, a part of that surface at the same time that it specifies something other than that surface (Gibson, 1986/1979: 273). For example, a line running down the centre of a roadway is both a part of the surface of the roadway at the same time that it specifies something other than the roadway. The centre line on the roadway makes visible and invites drivers to see the roadway in ways that the absence of a centre line would not necessarily promote. The centre line has an artefactual status; it is seen as both part of the road and yet distinct from it. It functions as a scaffold for experiencing and cognising the road and one’s relations to other vehicles in ways that make one self-aware of one’s perception of the road. It at least minimalistically de-familiarises the experience of the road and asks drivers to reflect on that experience and its significance—be attentive, stay on the right side of the line, do not cross the line into oncoming traffic and so on.

Table 3.1 presents a transcription of a short excerpt from a more extended writing event that took place one evening between Catherine and her father when they were living in Hong Kong from mid-2009 to early 2012. The event was video recorded by the father using a handheld video camera as they participated in the activity of experimenting with, making, and construing the graphic traces which they produced together. The recording was made on 10 April 2010 just a few days before Catherine turned four years.

The transcription in Table 3.1 is divided into a sequence of numbered Frames. Each Frame is a snapshot of a distinctive phase of the entire transcribed sequence. The two participants are identified as C. and F., respectively. The verbal activity of the two participants is shown in italics. Nonverbal activity appears in regular font. The ‘+’ sign indicates two or more strands of co-synchronised activity.

Frame 1 shows the father proposing and then drawing the letter b for Catherine. Her father’s trace-making is an occasion for observation of and reflection on her father’s trace-making in Frames 1 and 2 of the transcription. In Frame 2, Catherine leans across the desk to examine carefully her father’s trace-making. On completing the letter, her father comments on the quality of his trace-making (“a bit rough”) in response to Catherine’s question, “How can we draw b?” Her father then encourages Catherine to try for herself.

In Frame 3, Catherine leans back and resumes a more or less upright sitting position. From the altered point of observation which her adjusted posture affords, she articulates a particular stance on her father’s letter b. She says, “That’s not good,” in an animated way that secures the agreement of her father. In Frame 4, Catherine traces b and while doing so she says to her father, “You gotta go like this.” During this phase, her father twice comments on the letter b produced by Catherine: “That’s very good … that was good.” In Frame 5, Catherine points to the letter b with the writing pencil and comments: “I told you you gotta go longer”, referring to the length of the upstroke in her father’s b. She offers a normative judgement on the appropriate length of the upstroke of her father’s b.

In Frame 6, she traces over her father’s b in order to lengthen the upstroke. Her father responds with the normative judgement “Yes you’re right”, which encourages Catherine to trace over the lower part of her father’s b. In this phase, Catherine increasingly shifts into the teacher role. In Frame 7, she says to her father, “I’ll help you”, while tracing over the lower part of her father’s b and then returning to the upper part. In Frame 8, she sits upright to allow her father to view her retracing of his b and comments, “Like that”, to which her father replies, “Very good”. In Frame 8, we see both how Catherine positions herself as the source of normative judgements about her own and her father’s trace-making, in the process positioning her father as the learner. By the same token, her father offers the normative judgement praising her efforts. He says, “Very good” (Frame 8). In praising Catherine’s efforts, the father both articulates a normative stance on Catherine’s efforts, suggesting that her b is in conformity with the cultural standards at the same time that his praise also articulates the flow of affect and its fluctuating intensities that is important in the parent–child relationship. In Frame 9, these two interwoven aspects of the phase of the activity focused on in the transcription reach a peak of intensity in the flow of affect when Catherine turns from contemplation of the letter b she has traced while saying, “You gotta go like that Daddy if you don’t go like that I will …”. At this point, she turns to her father, points her left index finger at him and says, “I will teach … I will show you how to …”, before then shifting her posture to resume her trace-making. In Frame 10, she proceeds to produce an incorrect h, which her father questions, “That’s not h”. Catherine responds by making a second attempt, which is again negatively evaluated by her father. Catherine then switches to p, which Catherine proceeds to change when her father points out it is not h.

In the brief episode recorded in the transcription, Catherine and her father draw on and embed the products of their prior experiences of letter shapes into the very short timescales of looking, touching, moving, talking, tracing and so on in the process of articulating and reflecting on the graphic traces that they make. Their interactivity with tools (the pencil), a writing surface (sheet of paper), and articulatory movements of the eye–arm–tool–surface trace-making system drives the trace-maker’s lived experience of making, construing and reflecting on graphic traces. The activities of both articulating and perceptually exploring the graphic tracings catalyses further talk, thinking, action, and affect. In the process, Catherine hones and develops skills in making graphic marks on a surface and in reflecting on and evaluating them. In doing so, she, in concert with her father, brings into play and articulates what we may call, distantly echoing Walter Benjamin (1998/1966: 90) in a very different context, the writerly self (Cowley, personal communication), who not only writes but also comments on, evaluates, reflects on and prescribes what they write.⁶ The writerly self

Catherine’s trace-making is shaped by her own decisions and choices together with the honing of her skills of hand–eye–pencil–paper coordination and co-articulation. The graphic act is an articulatory act in which the materiality of tools, materials, surfaces and bodies are manipulated and intentionally modulated to achieve some difference which makes a difference by modulating in the real-time micro-genesis of graphic traces at least three sets of affordances: (1) the acting body, (2) the materials acted on and (3) the graphic tool with which the body acts. In the very short time scales involved, graphic trace-making activities are integrated with perceptual exploration, relationships, the flow and fluctuating intensities of affect, (self-)reflection and thinking.

The choices are constrained by normative cultural patterns to which she is attempting to orient and the constant negotiation we see here between Catherine’s trace-making activity and normative standards and evaluations and writerly conventions which she seeks to orient to and master. Whilst many of the meanings are specific to the father–daughter dyad, we see how cultural constraints cast their shadow on these events and begin to shape the products that result from them. Accounts of literacy and its development in children must therefore eschew reductive views that reduce literacy to the mechanistic acquisition of formal second-order abstracta and see literacy as the development of the cognitive skills and capacities, the forms of socially distributed agency and the creation and articulation of graphic and other traces that give voice to persons as writerly selves who can perform the complex forms of distributed cognitive activity that enable them to participate in culturally relevant and normatively appropriate ways in the entangled scales of the extended human ecology and its flows of human becoming.

As the transcription in Table 3.1 shows, there are constant shifts of attention among the performance of a given graphic act, the tracings inscribed on the paper and the two persons involved in this distributed cognitive event as a form of co-action between two agents, both of whom contribute to the creation and maintenance of the activity. Acts of trace-making and their perceptual exploration are treated as fully integrated into acts of feeling and thinking. Cycles of activity are created in which writing is integrated with flows of thinking, action, affect, which, in turn, prompt further cycles of activity in a recursive and self-reflexive process. Prior experiences, affect and feeling act as prompts for writerly activity just as the graphic acts (the tracings) which are produced prompt integration of experience that is synthesised with the articulatory act in the creation of the self-reflexive writerly self.

Both Catherine’s self-instigated and self-guided engagement with the HKU campus map just a few days prior to her fourth birthday and her early trace-making activities (see Table 3.1) takes place in what Reed (1996: 149–151) called “the field of promoted action”. She is a social agent who has already become a person: she has already developed a range of skills and capacities for interacting with and managing both other persons and aspects of the world within a single, coordinated frame of interaction (Halliday, 1975; Reed, 1996: 136). In coordinating both persons and aspects of the world within a single interaction frame, she has also developed the ability to take stances on the world, to coordinate her stances with those of others and to recognise that others may have different stances than hers. The field of promoted action refers to the ways in which a culture makes available and organises a range of resources for structuring everyday life (Reed, 1996: 149). These resources include artefacts of many kinds, activities, books, computers and other digital devices, games, toys, graphic traces, pictures, time-telling, visual displays and so on.

Reed points out that the child is immersed in a background field of relations that is constantly present but that different individuals and social groups may orient to and select from these resources in different ways according to factors such as economic possibilities, education, family preferences, social class and so on. The first social institution that the child learns to participate in is the family. Humans develop into self-reflective agents (persons) through their participation in the interactions of social groups and institutions, starting with the family. It is in the context of social groups, not as isolated individuals, that the child discovers that others may have different values and viewpoints and that these may contrast with or conflict with one’s own. This heteroglossic diversity of viewpoints acts as a selection pressure on the developing infant.

The child learns, through his or her development of the “language stance” (Cowley, 2011), to draw on and orient to the meaning potentials of utterances in order to articulate and to respond to the utterance activity of the social group. In achieving this cultural-cognitive milestone, the child develops the capacity to participate in and to modulate the flow of languaging activity, in the process developing and exercising the skills and capacities for managing and interacting with other persons and with the world within a single, coordinated interaction frame, that is the resources of human languaging. Catherine, at that age, was already a member of another social institution, a Cantonese-speaking kindergarten in Hong Kong. Like many other children of her age in different parts of the world, the range of social groups and institutions in which she was a regular participant is rapidly expanding beyond the immediate family circle. Consequently, the field of promoted action is also expanding.

Graphic trace-making is one of the culturally promoted activities that children around the age of three and four years begin to explore and undertake before they have developed any autonomous capacities and skills to interact with the affordances—the meanings and value—of trace-making (see Reed, 1996: 149). Nonetheless, Catherine, like children of her age, is aware that visual displays and graphic traces are resources that promote culturally desired forms of interactive exploration and discovery. The child perceives patterns in the tracings on a displayed surface like the HKU campus map that motivate and give rise to perceptual exploration of these patterns (Section 4). The child has learned from experience that such patterns have meaning and value in the field of promoted action even though the child has not yet developed the capacities for detecting the culturally preferred and legitimated ones.

These early attempts to ‘fill in’ the patterns that they explore with meaning and value are functional in bootstrapping the child into the preferred patterns and meanings of the particular field of promoted action in which the child participates (Reed, 1996: 150). Meaning is not ‘in’ the graphic trace as a property that can be extracted but emerges in an interactive field that is distributed across the entire trajectory that coordinates persons, their brains and bodies and the other ecological scales that are entangled whenever an agent sets up and enacts an interaction with other persons and with aspects of the wider ecological field of relations. The self is not confined to the physical body of the organism but loops out into and extends across and is entangled with diverse ecological scales.

The interactivity between the many fine-grained differences that are the traces of the graphic agent–tool–materials–surface interaction system catalyses flows of matter, energy and information through the system. These differences interact with each other in mutually in-forming ways that affect and make a difference to each other. The stochastic character of the interactivity and the mutual in-formativeness that it fosters yield self-generating iterations that begin to sprout forth and engender more complex forms of organisation. As Gibson also showed, this does not take place independently of viewpoints and observational perspectives. Instead, it engenders them. The mutual in-formativeness of the different component processes that constitute the graphic agent–tool–materials–surface interaction system gives rise to subjective viewpoints, naturally.

8. Scalar Entanglement, Assemblages and the Human Cultural Meshwork

Graphic traces, the surfaces on which they are installed, the materials and tools used by the trace-maker to deposit a graphic trace on the surface, the artefact on which a surface is placed, the physical location or setting of the artefact, and the materials and agencies involved in the manufacture of the artefact are not separate, hierarchically organised levels, each with their distinctive attributes and qualities. They are not independent of each other. Instead, they are meshworks of entangled flows of matter, energy, and information across different ecological scales. Meshworks co-articulate interconnected and interacting heterogeneous components into a larger-scale emergent whole without however subordinating the components to a single unifying principle that homogenises them (DeLanda, 1997: 38–39). Meshworks are heterarchical rather than hierarchical: no single component or principle dominates or controls the others in the way that the top level of a command hierarchy controls and subordinates its lower levels to its principles of organisation. The emergent properties and capacities of meshworks are neither reducible to nor explainable in terms of unitary formal essences. Instead, the components retain a relative autonomy at the same time that they have complementary functions in relation to each other.

Graphic traces, surfaces, artefacts, and settings form meshworks of entangled flows that integrate with each other; they cannot be seen as functionally independent of each other. By the same token, the different scales manifest different forms of materiality, expressivity, agency, affordances and flow. In order to illustrate this point more concretely, I propose four scales for the purpose of understanding the interactivity between Catherine’s embodied skills and capacities and the affordances of the HKU campus that I discussed in Section 4. The four scales are as follows:

Catherine’s interactivity with the HKU campus map does not take place as an isolated event even though it is a one-off event that was never repeated. As shown in Section 4, she uses the resources and action possibilities of her body—body movements as she moves along the retaining wall to reach the right height to access the display, haptic exploration with her hands, talk (in Cantonese), visual scanning and vocalising (non-linguistic)—to explore and probe the patterns she discovers in the traces displayed on the surface. In doing so, she reconfigures the affordance layout of the displayed material in the process of actively engaging with and exploring it. The real-time deployment of her bodily activities constitute and enact specific sensorimotor capacities and skills that cause changes in the affordance layout of the displayed patterns (see Chemero, 2009: 151). Changes in the affordance layout, in turn, give rise to feedback loops that will, in time, modify and bootstrap the perceptual and cognitive capacities and skills of Catherine and how these will be used in future.

The interactivity between Catherine and the campus map is between Catherine’s skills and capacities as a developing social agent and the affordance layout of the graphic traces displayed by the campus map. Rather than say that this event takes place ‘in’ a context or ‘in’ a surrounding environment, we may speak of the following entangled ecological scales that the plugging of Catherine’s skills and capacities into the affordance layouts of the campus map bring into direct interaction with one another:

Cultural artefacts like the HKU campus map are persisting organisations of and organisers of process embedded in culturally promoted fields of practice that set up and orchestrate Person–Artefact–Environment Interaction systems and the forms of interactivity they give rise to as well as specify the kinds of operations and the sequencing of these operations that the artefact enables. The map qua cultural artefact is a component in an assemblage that includes meta-level control structures, visual stimulus information qua inputs and agents’ goals, purposes and motivations. The identity of the assemblage is defined dynamically by the interactions between all of its component processes. The functional whole that is dynamically constituted and maintained over time by the interactions between its components thus depends on the exercise of the capacities of its components. However, assemblages are not organic totalities in which the identity of the parts of the whole are inextricably tied up with the relations of the parts to the whole and subsumed by the whole, as in Saussure’s account of la langue. According to assemblage theory, the parts retain a relative functional autonomy: they can be unplugged from one whole and plugged into and form part of some other functioning whole. The display as a whole consists of a map and its accompanying legend. The graphic traces that are displayed therefore consist of a heterogeneity of diverse elements. These heterogeneous elements have the capacity to enter into co-functioning relations of symbiosis to form what Deleuze and Guattari (2004/1980: 435–458) called an “assemblage”. An assemblage is a multiplicity of heterogeneous component processes that retain their relative autonomy. These component processes can be detached or unplugged from one assemblage and plugged into some other without losing their relative autonomy (Deleuze and Guattari, 2004/1980: 435–458; DeLanda, 2010: 10–13).

As the name suggests, the diverse components of an assemblage have been brought together and assembled by historical processes as well as by the interactions between the components that maintain the whole. Assemblages are historically contingent organisations of interactions of process on some scale. Smaller scale assemblages can enter into larger scale ones as functioning parts of these. The Person–Artefact–Environment Interaction System that is constituted by the interactivity between a person and the HKU campus map is an interaction of component parts that is not defined by the existence of unchanging formal essences that are instantiated in material form by the assemblage. Two different occasions of interaction between persons and the map are not instantiations of some higher-order schema or category but are unique historical individuals. If the two occasions have things in common, as is quite likely, or even appear to be similar in many respects, to the extent that we may say they are interactions of the same kind, this does not change the fact that the two events are the outcomes of historically unique processes that individuate that assemblage. However, each unique occurrence can belong to a population of similar assemblages that reflect recurrent production processes such that many individuals actualise in divergent ways. DeLanda explains recurrence as follows:

Catherine’s interactivity with the campus display allows us to de-familiarise our usual view of ‘text’ as a stable and relatively homogeneous entity that fixes the parameters of the social transactions between the graphic traces and the map reader. Instead, we see more clearly how an entire meshwork of entangled ecological scales is at work. Instead of saying that a ‘text’ is a self-contained semiotic object that we place ‘in’ its context, the focus is on the role that graphic trace-making plays in a systemic view of the extended human ecology. On this view, we see how, on and across diverse ecological scales, graphic traces are woven into the fabric of the living of human life. Their flows are coordinated with the flows of the bodies whose movements become entangled with the flows of the graphic traces they explore and interact with. The resulting entanglements of these flows give rise to orchestrations of materialities and agencies across ecological scales that are functional in organising and coordinating ecological, socially organised flows of matter, energy and information that shape and give form to (aspects of) the living of human life.

9. Conclusion

Graphic traces afford processes of self-stimulation (Thibault, 2012: 36–40) that afford and promote possibilities of (self-)reflective abstraction. The capacity for reflective abstraction scaffolds the agent’s processes of self-exploration in the process of eliciting reflection on the perceptual properties of the graphic trace. Trace-making promotes and enhances an experimental logic that scaffolds the processes of perceptual self-stimulation that are required for the taking up of stances from specific points of observation. Whereas representation is concerned with the ‘representation’ of how things are, experimentation seeks to manipulate material processes and forces and to provoke them into behaving in new ways. Selves emerge through interactive engagements with and transformations of the affordances of their worlds, not through static representations of the already given. Stance taking in turn enables self-reflection on the interactive processes in which the self is engaged, in the process supporting and scaffolding the further development of the self.

Notes

1Aristotle argues that form is not generated but is the essence which is “induced” in matter by some external agency: “It is therefore obvious that the form (or whatever we should call the shape in the sensible thing) is not generated—generation does not apply to it—nor is the essence generated; for this is that which is induced in something else either by art or by nature or by potency. But we do cause a bronze sphere to be, for we produce it from bronze and sphere; we induce the form into this particular matter, and the result is a bronze sphere” (Aristotle, 1996/1933: 345).

2Aristotle (1996/1933: 357) further points out, “But there is a sense in which not even all the letters will be contained in the formula of the syllable; e.g. particular letters on wax or sounds in the air; for these too are parts of the syllable in the sense that they are its sensible matter.” Here, Aristotle refers to “the concrete object” as distinct from “the form, or that to which the formula refers”. The sensible matter of the concrete object is thus seen as external to the formal essence which induces form in matter.

3Johannessen (2016: 10) uses the formalism of the system network developed over many decades in systemic-functional linguistics to map a network of systemic choices in ‘shape’. In my view, this formalism retains the logic of formal essences that characterises the hylomorphic view of creation formulated by Aristotle. On this view, the least delicate categories are like formal essences that both define the characteristics of lower level or more delicate options at the same time that the latter are instantiations of the former. However, we can reverse this logic and say that the forms do not exist in some higher-order n + 1 domain as transcendent formal essences that are then selected and instantiated as a material form (graphic substance). Instead, they are future attractors or tendencies that are immanent in the intensive flows of materials and their critical thresholds on the n-n − 1 dimension. This thought bubble here suggests a major rethinking of the logic of system networks that I will try to undertake in future work.

4Citing Kirsh’s (2009) idea of projection as a way of seeing something extra in what is perceived, Malafouris (2013: 101–102) develops a similar argument (see also Thibault 2005a, b, on language as an anticipatory system; see also Section 6 of this chapter).

5This accords with Vygotsky’s observation that the social is the source, not the setting, for the developmental emergence of the self (Veresov, 2005: 42; Vygotsky, 1978/1930). It also raises the question as to how we understand the ‘social’. Do we theorise it as a reified generality or abstracta that stands ‘above’ individual persons, or in terms of the webs of interdependent first-order interactions in which persons interact with other persons, with cultural institutions, with cultural artefacts and with second-order meaning potentials (see also Jones, 2009: 180)?

6Benjamin (1998/1966: 90) writes, “The reader is always prepared to become a writer, in the sense of one who describes or prescribes.” Anne Bostock, the English-language translator of Benjamin’s text comments in a footnote on this sentence as follows: “Benjamin makes a play on words here with Schreibender (one who writes), Beschreibender (one who describes) and Vorschreibender (one who prescribes).” Translator’s footnote, p. 90. Cowley’s term “the writerly self” is perhaps a distant echo of this discussion but captures, in my view, the stance-taking that writing affords selves and which Benjamin appears to acknowledge in the sentence quoted here, especially when seen in the light of the translator’s comment, albeit in a radically different historical context and in relation to very different concerns with respect to mine in this chapter.

References

Aristotle. (1996) [1933]. Metaphysics: Books I–IX. Loeb Classical Library (H. Tredennick, Trans., G. P. Goold, Ed.). Cambridge, MA and London, England: Harvard University Press.

Baldry, A., and Thibault, P. J. (2006). Multimodal Transcription and Text Analysis: A Multimedia Toolkit and Coursebook (with associated online course). Foreword by Jay Lemke. London and Oakville, CT: Equinox.

Benjamin, W. (1998) [1966]. The Author as Producer. In Anna Bostock (trans.), Understanding Brecht (pp. 85–103). Introduction by S. Mitchell. London and New York: Verso.

Bergson, H. (1896) [1911]. Matter and Memory (N. M. Paul and W. S. Palmer, Trans.). London: Swan Sonnenschein & Co., LIM and New York: Palgrave Macmillan.

Berthoz, A. (2012) [2009]. Simplexity: Simplifying Principles for a Complex World (G. Weiss, Trans.). New Haven and London: Yale University Press and Paris: Éditions Odile Jacob.

Bickhard, M. H., and Campbell, R. L. (1996). Topologies of learning and development. New Ideas in Psychology, 14(2), 111–156. doi: http://dx.doi.org/10.1016/0732-118X(96)00015-3

Bickhard, M. H., and Richie, M. D. [with the assistance of Robert Hughes and James Dannemiller]. (1983). On the Nature of Representation: A Case Study of James Gibson’s Theory of Perception. New York and Eastbourne, UK: Praeger.

Bogdan, R. J. (2013). Mindvaults: Sociocultural Grounds for Pretending and Imagining. Cambridge, MA and London, UK: MIT Press.

Brotton, J. (2013). A History of the World in Twelve Maps. London and New York: Penguin Books.

Chemero, A. (2009). Radical Embodied Cognitive Science. Cambridge, MA and London, UK: MIT Press.

Clark, A. (1997). Being There: Putting Brain, Body, and World Together Again. Cambridge, MA: MIT Press.

Clark, A. (2006). Material symbols. Philosophical Psychology, 19(3), 291–307.

Clowes, R. W. (2007). Semiotic symbols and the missing theory of thinking. Interaction Studies, 8(1), 105–124.

Clowes, R. W., and Morse, A. (2005). Scaffolding Cognition With Words. In L. Berthouze, F. Kaplan, H. Kozima, Y. Yano, J. Konczak, G. Metta, J. Nadel, G. Sandini, G. Stojanov, and C. Balkenius (Eds.), Proceedings of the 5th International Workshop on Epigenetic Robotics (pp. 102–105). Nara, Japan: Lund University Cognitive Studies, 123. Lund: LUCS.

Cowley, S. J. (2011). Taking a language stance. Ecological Psychology, 23(3), 185–209.

Deacon, T. (1998) [1997]. The Symbolic Species: The Co-Evolution of Language and the Human Brain. London and New York: Penguin.

DeLanda, M. (1997). A Thousand Years of Nonlinear History. New York: Swerve Editions.

DeLanda, M. (2010). Deleuze: History and Science. New York and Dresden: Atropos Press.

DeLanda, M. (2011). Philosophy and Simulation: The Emergence of Synthetic Reason. London and New York: Continuum.

Deleuze, G. (2004) [1968]. Difference and Repetition (P. Patton, Trans.). London and New York: Continuum.

Deleuze, G., and Guattari, F. (2000) [1980]. A Thousand Plateaus: Capitalism and Schizophrenia (B. Massumi, Trans.). London and New York: Continuum.

Dennett, D. C. (1996). Kinds of Minds. New York: Basic Books.

Gatewood, J. B. (1985). Actions speak louder than words. In J. Dougherty (Ed.), Directions in Cognitive Anthropology (pp. 199–219). Urbana: University of Illinois Press.

Gibson, J. J. (1983) [1966]. The Senses Considered as Perceptual Systems. Boston: Houghton Mifflin.

Gibson, J. J. (1986) [1979]. The Ecological Approach to Visual Perception. Hillsdale, NJ: Erlbaum.

Halliday, M. A. K. (1975). Learning How to Mean: Explorations in the Development of Language. London: Arnold.

Held, R., and Hein, A. (1963). Movement-produced stimulation in the development of visually guided behaviour. Journal of Comparative and Physiological Psychology, 56(5), 872–876.

Hjelmslev, L. (1954). La stratification du langage. Word, 10(2–3), 163–188.

Hodges, B. H. (2007). Good prospects: Ecological and social perspectives on conforming, creating, and caring in conversation. Language Sciences, 29, 584–604. doi: 10.1016/j.langsci.2007.01.003

Ingold, T. (2010). The textility of making. Cambridge Journal of Economics, 34, 91–102.

Ingold, T. (2011). Being Alive: Essays on Movement, Knowledge and Description. London and New York: Routledge.

Johannessen, C. M. (2010). Forensic Analysis of Graphic Trademarks: A Multimodal Social Semiotic Approach. Unpublished Ph.D. thesis, Institute of Language and Communication, University of Southern Denmark, Odense.

Johannessen, C. M. (2016). Experiential meaning potential in the Topaz Energy logo: A framework for graphemic and graphetic analysis of graphic logo design. Social Semiotics, 27(1): 1–20. doi: 10.1080/10350330.2016.1187880

Jones, P. E. (2009). From “external speech” to “inner speech” in Vygotsky: A critical appraisal and fresh perspectives. Language & Communication, 29, 166–181.

Kanizsa, G. (1980). Grammatica del Vedere: Saggi su percezione e gestalt. Bologna: Il Mulino.

Kirsch, D. (2009). Projection, problem space and anchoring. In N. A. Taatgen and H. van Rijn (Eds.), Proceedings of the 31st Annual Conference of the Cognitive Science Society (pp. 2310–2315). Austin, TX: Cognitive Science Society.

Kress, G., and Van Leeuwen, T. (1996). Reading Images: The Grammar of Visual Design. London and New York: Routledge.

Linell, P. (2011) [2005]. The Written Language Bias in Linguistics: Its Nature, Origin and Transformations. Oxford and New York: Routledge, Taylor & Francis Group.

Luria, A. R. (1973). The Working Brain: An Introduction to Neuropsychology (B. Haigh, Ed.). London: Allen Lane.

Malafouris, L. (2013). How Things Shape the Mind. Cambridge, MA: MIT Press.

Melser, D. (2009). Verbal communication: From pedagogy to make-believe. Language Sciences, 31(5), 555–571.

Noë, A. (2004). Action in Perception. Cambridge, MA: MIT Press.

Noë, A. (2007). The critique of pure phenomenology. Phenomenology and the Cognitive Sciences, 6, 231–245.

Reed, E. S. (1996). Encountering the World: Toward an Ecological Psychology. New York and London: Oxford University Press.

Roth, W.-M. (2014). On the pregnance of bodily movement and geometrical objects: A post-constructivist account of the origin of mathematical knowledge. Journal of Pedagogy,5(1), 65–89.

Saussure, F. de (1993/1907, 1910–1911). Cours de Linguistique Générale: Premier et troisième cours d’après les notes de Riedlinger et Constantin (E. Komatsu, Ed.). Tokyo: Université Gakushuin.

Sheets-Johnstone, M. (2010). Body and Movement: Basic Dynamic Principles. In S. Gallagher and D. Schmicking (Eds.), Handbook of Phenomenology and Cognitive Science (pp. 217–234). Dordrecht, The Netherlands: Springer.

Sheets-Johnstone, M. (2011). The Primacy of Movement (2nd ed.). Amsterdam/Philaelphia: John Benjamins.

Shklovskij, V. (1965) [1917]. Art as technique. In L. T. Lemon and M. J. Reis (Eds.), Russian Formalist Criticism: Four Essays (pp. 3–24). Lincoln: University of Nebraska Press.

Stuart, S. A. J. (2010). Enkinaesthesia, biosemiotics, and the ethiosphere. In S. J. Cowley, J. C. Major, S. V. Steffensen, and A. Dinis (Eds.), Signifying Bodies: Biosemiosis, Interaction and Health (pp. 305–330). Braga: The Faculty of Philosophy of Braga Portuguese Catholic University. ISBN 978-972-697-191-7.

Stuart, Susan A. J., and Thibault, P. J. (2015). Enkinaesthetic polyphony: The underpinning for first-order languaging. In U. M. Lüdtke (Ed.), Emotion in Language: Theory—Research—Application (pp. 113–133). Amsterdam/Philadelphia: John Benjamins.

Thibault, P. J (2005a). What kind of minded being has language: Anticipatory dynamics, arguability, and agency in a normatively and recursively self-transforming learning system, Part 1. Linguistics and the Human Sciences, 1(2), 261–335.

Thibault, P. J. (2007). Writing, graphology, and visual semiosis. In T. Royce and W. Boucher (Eds.), New Directions in the Analysis of Multimodal Discourse (pp. 111–145). Mahwah, NJ: Lawrence Erlbaum Associates.

Thibault, P. J. (2011). First-order languaging dynamics and second-order language: The distributed language view. Ecological Psychology, 23(3), 210–245. http://dx.doi.org/10.1080/10407413.2011.591274

Thibault, P. J. (2012). Hypermedia selves and hypermedia stories: Narrativity, writing, and normativity in personal blogs. In M. Cambria, C. Arizzi, and G. Magazzù (Eds.), Web Genres and Web Tools: Contributions From the Living Knowledge Project (pp. 11–52). Como-Pavia: IBIS. Volume 4 in the series, Multimodal Text Studies in English, Anthony Baldry, series editor. http://mcaweb.unipv.it/mtse/

Vallée-Tourangeau, F., Euden, G., and Hearn, V. (2011). Einstellung defused: Interactivity and mental set. Quarterly Journal of Experimental Psychology, 64(10), 1889–1895.

Veresov, N. (2005). Marxist and non-marxist aspects of the cultural-historical psychology of L. S. Vygotsky. Outlines, 7(1), 31–49.

Vygotsky, L. S. (1978) [1930]. Mind in Society: The Development of Higher Psychological Processes. Introduction by Mike Cole and Sylvia Scribner. Afterword by Vera Steiner and Ellen Souberman. Cambridge, MA: Harvard University Press.

3 Graphic Trace-Making as Articulated-Expressive Trajectories of Movement

De-Textualising and De-Stratifying Graphic Traces

1. Introduction