The Routledge Handbook of Translation and Pragmatics

Multimodal deictic enrichment during in-vision news sign language translation

Christopher Stone

Introduction

The Broadcasting Act 1996, chapter 55, section 20, placed a legal obligation on broadcasters in the UK to include British Sign Language (BSL) in their programmes, either via presentation in, or translation into, sign language. This has included the rendering into BSL of current affairs programmes, popular programmes and soaps with Deaf and hearing translators/interpreters (T/Is) being employed to undertake this work. In-vision translation, however, is not new, and existed before 1996 (Ladd, 2007), but little attention has been paid to the multimodal nature of the translation and the pragmatics of delivering a seen translation, that is, with the translator viewed by the audience, presenting a translation that interacts with other elements on the television screen. This involves the representation of the news and other current affairs to ensure that sign-language-using-deaf people have access to the news in their first or preferred language.

This chapter examines the presented in-vision translation of spoken English current affairs programmes rendered into BSL by both deaf and hearing professionals. The output examined is from the BBC, although there are other broadcasters that provide in-vision programmes. There is also an organisation called the British Sign Language Broadcasting Trust (BSLBT) that receives contributions from “narrowcasters” (i.e., those with less than 1 per cent audience share, Ofcom, 2105) and commissions BSL programming made in BSL and often produced by Deaf community members.

The chapter examines decisions made by in-vision professionals to ensure that Deaf people have access to current affairs. I draw on Relevance Theory (Sperber & Wilson, 1995) as the theoretical framework to analyse the multimodal environment that the viewer is watching and the ways in which the in-vision professionals manage that environment as they craft their renditions. This builds on earlier work (Stone, 2007, 2009) in which I was the first scholar, to my knowledge, to apply Relevance Theory to sign language translation and interpreting; furthermore, following Sequeiros’ (1998, 2002) work on enrichments in Spanish to English literary translation, I examine the rendering of English in-vision news into BSL.

First, I introduce Relevance Theory, before explaining the environment within which the translator provides the rendition and how this manifests in the viewed product. I then describe the shifts from pointing (allowing the audience to watch the programme as information is provided by the images on screen alone), to telling (using BSL to render information), to showing (using depicting strategies that are isomorphic with the images on screen) information to a BSL using audience to ensure an optimally relevant target language. I conclude that these decisions are motivated by the T/Is wanting Deaf appropriate communicative norms to be seen by mainstream audiences while simultaneously providing optimal access to information for the Deaf audience.

1 Relevance Theory

This theory of pragmatics is grounded in information processing and cognitive theories of linguistic communication. Its central tenet is that the aim of information processing is to recover as many contextual effects as possible for the least cost of processing (Blakemore, 1992: 34). As human beings we use certain “behaviour which makes manifest an intention to make something manifest – ostensive behaviour or simply ostension” (Sperber & Wilson, 1995: 49). A general property of human interaction is a desire to point out information and to communicate that this information has been intentionally pointed out. The hearer uses his inference system to understand that there was an ostension; a coded communication such as language can be used to strengthen this ostensive-inferential communication, where ostensive-inferential communication can be defined as follows:

Ostensive-inferential communication: the communicator produces a stimulus which makes it mutually manifest to communicator and audience that the communicator intends, by means of this stimulus, to make manifest or more manifest to the audience a set of assumptions.

(Sperber & Wilson, 1995: 63)

In linguistic communication, the processing effort that the hearer makes to understand an utterance needs to be worth making. This effort is seen as worth making when overt communication is occurring; ideally in this situation, the speaker is deemed by the hearer to be optimally relevant. That is to say that the speaker intends the hearer to believe she (the speaker) is being optimally relevant when she speaks.

Relevance Theory is bound by two principles: Cognitive Principle of Relevance, according to which human cognition tends to be geared to the maximisation of relevance (Wilson & Sperber, 2002: 254), and Communicative Principle of Relevance, according to which every ostensive stimulus conveys a presumption of its own optimal relevance (Wilson & Sperber, 2002: 256). Relevance Theory does not ignore the possibility the speaker might not be being optimally relevant, but rather accepts that there are risks present in linguistic communication.

This overt linguistic communication occurs in the context of a shared cognitive environment that stems from several sources: the immediate environment or information, expectation or general cultural assumptions (Blakemore, 1992; Sperber & Wilson, 1995). Thus, a central problem for pragmatic theory is to describe how, for any given utterance, the hearer finds a context which enables him to understand it adequately (Sperber & Wilson, 1995:16).

Relevance theorists strive to identify how contexts are actually selected and used in utterance comprehension. These contexts can be cumulative, such that environmental factors, cultural assumptions, expected future outcomes and previous and present linguistic code could be used collectively to understand an utterance. For our purposes these environmental factors also include the multimodal nature of in-vision translation and the use of these visual resources to ensure that a covert translation occurs, i.e., that the product is domesticated (as much as possible) for the minority language community (BSL users) rather than maintaining mainstream norms (Stone, 2009). Here we can conceive of a covert translation (House, 1997) as one that is “not marked pragmatically as a translation” (p. 69). Even though the translations have equivalent functions “the translator has to take different cultural presuppositions in the two language communities into account” (ibid.: 70) and this is made manifest in the ostensive stimulus the T/Is render to the audience via explicatures and implicatures.

1.1 Explicatures and implicatures

One of the most important underpinning ideas of Relevance Theory is that of language underdeterminacy. This is the idea that language never fully encodes the information that we wish to communicate. Some of the information is communicated explicitly (by explicature) and some of it is communicated implicitly (by implicature). Explicature is taken to be an “explicitly communicated assumption”, that is, an assumption communicated by an utterance U is explicit if and only if it is a development of a logical form encoded by U (Sperber & Wilson, 1995: 182). This means that by combining the logical form of an utterance (decoding the linguistic information) with assumptions (pragmatic inferring from the context), the hearer is able flesh out the semantic representation (Blakemore, 1992).

In the example “it’s snowing [IN KATHMANDU]” (Carston, 2002: 323), we see that the logical form is understood and then the utterance enriched as part of an inferential pragmatic process with a location [shown in parentheses]. The semantic representation then becomes that “it is snowing in Kathmandu”. If this is what the speaker intended to communicate and what the hearer understands the speaker to have intended to communicate to them this ostensive communication is successful. And, depending upon the contextual assumptions in the shared cognitive environment (the speaker and the addressee being in Kathmandu or having previously mentioned Kathmandu), successful communication is possible.

There are varying degrees of explicitness within explicature: how explicit the linguistic code is affects the level of inference, making the utterance (or explicature in this case) more or less explicit. In the example above (it’s snowing), there could also be a time as well as location explicature, as in “It’s snowing [IN KATHMANDU] [AT THE PRESENT TIME]”. The quantity of information present in the linguistic code reduces the amount of explicatures needed and increases the degree of explicitness. We need to bear in mind that there is a: “possible difference between the proposition expressed by the speaker and her explicature(s): the proposition expressed may or may not be communicated; only when it is communicated is it an explicature of the utterance” (Carston, 2002: 117).

That is to say that if the speaker intends to communicate P by saying utterance U, P is an explicature if and only if the hearer understands U to have communicated P. This constitutes successful ostensive communication without which the explicature would not exist. Implicature on the other hand is: “when the speaker could not have expected his utterance to be relevant to the hearer without intending him to derive some specific contextual implication from it, then, and only then, that implication is also an implicature”(Sperber & Wilson, 1981: 284). This is expressed well in the following example from Blakemore (1992: 123–124), where the hearer has to access the context (5) and to deduce the contextual implication (4):

(3) A: Did I get invited to the conference? B: Your paper is too long.

(4) Speaker A did not get invited to the conference.

(5) If your paper is too long for the conference you will not be invited.

(Original numbering)

We still see that the answer in (3) B is enriched by an explicature: your paper is too long [for the conference]. There is the specific contextual implication (the implicature) (5). It is not clear whether the enrichment happens prior to or subsequent to the implicature, but both are present in this utterance. Sperber and Wilson argue that no language utterance is ever completely explicit and more contemporary works support this (Fauconnier, 1997; Talmy, 2000a, 2000b; Fauconnier and Turner, 2002). In other words, the linguistic code of an utterance always underspecifies the assumptions associated with that utterance.

Wilson (2005: 1130) gives useful examples of both explicatures and implicatures and the categories into which they fall:

Identification of explicit content (explicatures):

(1) John left the party. (“political groups”, “festive gathering”)

(2) The teachers told the students they needed more holidays. (reference resolution)

(3) I met no one in town. (“no one I knew”, “no one interesting”)

(4) Your father will be here soon. (resolution of vagueness)

(5) The sky is blue. (“partly/totally”, “blue of a certain shade/blueish”)

(6) You will be there tomorrow. (request, bet, prediction)

Implicit context (implicatures):

(7) This book is as good as any the author has written. (good? mediocre? bad?)

(8) Some of the lectures were interesting. (scalar implicatures)

(9) a. Jim: Have you read Relevance? b. Sue: I don’t read difficult books. (indirect answers)

(10) I’m hungry. (indirect speech acts)

(11) Bill is a giant. (literal/metaphorical/ironical)

Clearly these two types of inferences affect translators as they work. It is necessary for translators not only to access the full propositional form in the Source Language (SL) but also to decide how to represent that in the Target Language (TL). If we then consider SL produced in current affairs or news programme this is spoken (from an autocue) while other information appears within the shared cognitive environment, i.e., the language is produced as other information is made available to the watcher. The audience hear English while watching further information, including maps of areas being discussed (e.g. weather news), images of people being shown (e.g. those being discussed such as politicians), video footage of processes occurring while being discussed (e.g. live footage of an area suffering a natural disaster such as flooding or bush fires), etc.

For a present/seen in-vision translator using a visual language (which has visually motivated depicting strategies encoded in the language code) such as BSL, these are also resources that can be drawn upon to enable an effective TL rendition. In BSL some nouns and verbs are differentiated according to visual motivations (Sutton-Spence & Woll, 1999). This builds upon the work of Mandel (1977) and is explored in greater depth in ASL by Taub (2001) who discusses visual motivation in ASL and other signed languages:

the meaning of tree and the associated visual image do not determine the signs’ forms, as they are all different – but neither are the forms unrelated to the meaning. Instead the forms all bear different types of physical resemblance to the image of a tree. The nature of these forms, given their meaning, is neither arbitrary nor predictable but rather motivated.

(Taub, 2001: 8)

Similarly, gestural interactions are commonplace in deaf communities during face-to-face interaction (Kusters, 2017). The (cultural) presupposition of gesture-rich interaction also comes into play and can change the level of underdeterminacy within in-vision situated language use. It is possible to point to images that will appear on the television screen such that the English newsreader who says “the plane” as scripted in the autocue while the video footage of a plane is being displayed is expecting that the viewer will understand the implicature that “the plane” under consideration is the plane that is the plane being viewed in the footage. For a present BSL translator the script could be rendered in a similar manner with the lexical sign AIRPLANE but is typically rendered with an index finger point to the video footage of the plane followed by the lexical sign AIRPLANE.

SL:	. . . the plane
TL	. . . INDEX-POINT-(to-video footage-of-airplane) AIRPLANE

The enrichment of the implicature, the index point, is ambiguous in terms of its linguistic or gestural nature, but for this analysis this is somewhat irrelevant in that ostensive communication can include both language and gesture to make manifest the meaning. And for the BSL translators this fulfils the desire for a covert translation (House, 1997) as it adheres to BSL cultural communicative norms (Cormier etal., 2015).

2 Relevance Theory and translation

For Gutt, Relevance Theory can enable “an empirical account of evaluation and decision-making” (1991: 21) with respect to equivalence and the relationship between the source and target texts. What is of interest here is Gutt’s treatment of covert translation, which he describes as occurring “where the translated text is intended to function like a target language original” (1991: 45) as with House (1997). That is to say that the TL has all traces of the SL removed from it.

Gutt (1991) suggests that many acts performed by bilinguals are called translations, but the main difference is whether they use the SL descriptively or interpretively. Blakemore and Gallai (2014, see also Gallai, this volume) have, however, criticised Gutt’s notion that interpretation is only ever part of attributed use, i.e., that “the rendered text is an interpretation of the translator’s/interpreter’s thoughts which are themselves an interpretation of the thoughts of the original speaker” (p. 107). In their account of discourse markers used by interpreters in police interviews they observe that discourse markers “ensure that the audience hears the “voice” of the original speaker (S1) even though his interpretation of S1’s thoughts is based on the evidence provided by another speaker (S2)” (p. 109); they refer to this as mutuality.

The concept of mutuality does appear to describe the desires of the deaf T/Is to engage in a covert translation, i.e., to be the newsreader for the Deaf (Stone, 2005, 2009) and so although Gutt’s suggestion of attributive use can be applied to many instances of interpreting it does not mean that when there is a present in-vision translator/interpreter mutuality does not occur. As Blakemore and Gallai (2014) observe, interpreters sometimes use linguistic resources (in their data discourse markers) to make salient the thoughts of the original speaker and so “any contextual assumptions she [the audience] accesses along that inferential route will be attributed not to the interpreter but to the original speaker” (p. 116). The types of resources the sight translators and interpreters use in their renditions appear to do this by engaging with the visual information made manifest in the cognitive environment.

Gutt also discusses the different socio-cultural needs of different receptor language audience, which can include the assumptions made on the part of the author about the type of information that is known by the audience. In other words, in some situations the original text could be used as a guideline rather than a source text that must be followed faithfully; a fully enriched semantic representation can drive these decisions and Relevance Theory can account for that.

Unlike people hearing the SL and watching the broadcast images, sign language users watch the language and also watch the broadcast images. This single channel for comprehending both language and the shared cognitive environment require the management of language (telling), depiction (showing) and viewing control (pointing) from the translator to ensure that Deaf community cultural viewing norms are observed. That is to say that the everyday language practices of deaf people should inform the interaction we see of the in-vision translator with the broadcast images on the screen, and indexing these phenomena, using a variety of strategies that we expect to see in ostensive BSL communication.

2.1 Interlingual translation enrichments

For the Deaf community, often failed by mainstream education where education is accessed via interpreters (often untrained and unqualified historically), the translator may assume that lack of access to information throughout life requires certain contextual assumptions to be made explicit, i.e., that some explicatures and some implicatures need to be explicitly coded in the language used.

Sperber and Wilson’s (1995) Relevance Theory can be applied to the analysis of the TL and the shifts that occur in the translation process because the TL has to be relevant enough to make it worth the addressee’s while to process the ostensive stimulus. If the Deaf audience has to spend too much cognitive effort on understanding the TL, then the T/Is are not fulfilling their purpose, i.e., translating the English SL into a (covert) BSL TL that creates equality of access for the Deaf audience. These texts are understood because of the multimedia environment they are produced in and so language that is produced must be contextually appropriate, drawing upon contextual assumptions that the viewing audience will make by virtue of what they are seeing. These viewed assumptions can be represented by depicting handshapes (Zwitserlood, 2012) which are visually motivated by the shape of an entity, e.g. a person is a long thin cylinder and represented by an upright index finger, which can be moved in space to represent a person moving about. This then can be used in a rendition to move with respect to the screen so that it is isomorphic with the screen, e.g. like a weather person using the screen to gesturally depict how rain clouds might move across a geographical area.

Sequeiros (1998, 2002) analyses pragmatic additions and omission in Spanish to English literary translations with respect to the TL. He defines enrichment as:

A process of completion of the logical form (i.e. the semantic representation encoded by the utterance) whose aim is to arrive at the proposition expressed, which may or may not be one of the set of thoughts explicitly communicated by the utterance.

(Sequeiros, 2002: 1070)

Initially it is useful to look at intralingual examples, i.e., examples of the process of the completion of a logical form to arrive at the propositional form within a language. Wilson and Sperber’s notion is that “If the linguistically encoded information is too vague, or too incomplete, to yield an adequately relevant interpretation, it will be enriched using immediately accessible contextual assumptions, to the point where it is relevant enough” (Wilson & Sperber, 1993: 293). This idea is used by Sequeiros to further expand upon the idea of pragmatic enrichment. The ultimate point is that not all information is linguistically encoded, e.g. in a conversation the speakers could say:

Speaker A:	Will Aoife be long?
Speaker B:	She is with Richard.

Here the logical form of the utterance made by Speaker B is not sufficient to answer the Speaker A’s question. If, however, the situation is such that Speakers A and B know that Aoife is a student, Richard is her tutor and Richard only ever spends a short time with his tutees, then Speaker A can use this implicature to make a relevant interpretation of the logical form to create the propositional form. The pragmatic enrichment is connected to the main premise of pragmatics, which is that we use language to semantically encode representations that are only partial representations of the thoughts that we intend to communicate. The decision of the T/I then has to be how to represent the enriched logical form (the propositional form) in the TL so that it is as relevant to the audience as the speaker intended.

If we now look at interlingual enrichment, then according to Sequeiros (2002: 1078): “An utterance is a case of interlingual enrichment if its semantic representation is the intended enrichment of the semantic representation of an utterance from another language.” Sequeiros builds upon Gutt (1991), noting that if the translator explicates the TL in relation to the full propositional form rather than following the logical form then this would be a case of interlingual enrichment. Furthermore, Sequeiros (2002: 1077) states that the logical possibilities between the two languages seem to allow four different cases as regards explicitness:

A Translation more explicit because of (enrichment):

i Linguistic differences between two languages

ii A choice of the translator on some other grounds

B Translation less explicit because of (impoverishment):

i Linguistic differences between two languages

ii A choice of the translator on some other grounds

Sequeiros further details four areas of enrichment: temporal enrichment, thematic enrichment (agent, source and possessor), enrichment based on discourse relations, and enrichment based on implicatures. These four areas of enrichment build on his previous work on impoverishment (Sequeiros, 1998) and give a useful taxonomy of the types of pragmatic shifts that may occur in translation.

Of interest is when the translators make decisions to represent the information in a relevant way to the Deaf audience so that deaf people can watch both the rendition of the SL and at the same time view the images and video footage being shown during programmes so that they see a relevant TL in line with Deaf culturally relevant viewing behaviours. These decisions have been previously identified as visual incorporation enrichment (Stone, 2009) but it is important to note that this descriptor covers a subset of the multimodal interaction between the seen translator, producing a translation in-vision and drawing upon resources available in the shared cognitive environment of the viewer and the translator. These translation acts then create a TL that meets both the socio-cultural needs of BSL users and the socio-political desires of the translators to create a BSL space for news and current affairs building upon the traditional translation practices of the UK Deaf community (Adam etal., 2011).

2.2 Relevance Theory, BSL translation and depiction

When we consider the properties of British Sign Language and translation enrichment we must consider the difference in BSL and English. British Sign Language is a language that uses hands, torso and face to produce language and, more importantly, it is an unwritten language with many unwritten language features (Ong, 1984). This does not mean, however, that is it impossible to write BSL. There are various conventions used to represent BSL (Stone & West, 2012), although, as with all writing systems, these conventions move away from the live production of face-to-face language use.

However, BSL can now be recorded using video technology, and these recordings can be sent either as messages via a variety of video messaging apps, or of course more formally broadcast or streamed to provide direct information (i.e., BSL news created in BSL) or accessible news and current affairs. As this technology is now ubiquitous (and cheap) there does not appear to be any desire to create a standardised written system for BSL. Some sign languages do have a written system, but they are very much in the minority. As such, BSL translations are a hybrid form in that although the translation is prepared there is still a performance element to it (Stone, 2007). Even so, “prepared sight translation” or sign language translation is becoming more widespread in the UK for the translation of museum guides, information videos, NHS epidemic warnings and broadcast news and current affairs. And with more sophisticated video editing techniques becoming more widespread, this hybrid form is in some instances moving closer towards its written translation analogue.

When we consider contextual assumptions, as noted above we need to consider contextual assumptions made due to visual information and the ability of BSL to linguistically encode the shape and/or size of an entity, the movement characteristics of an entity and the locational information (relative or otherwise) of an entity, as in the following example:

SL:	The car was driven down the road
TL:	CAR DRIVE CL-(flat-hand)-MOVE-ALONG-LEFT-HAND-SIDE-OF-ROAD
Translation:	the car was driven down the left-hand side of the road

In this example, the interlingual enrichment of the SL in the TL includes more specific locational information, which is visually motivated (as described above). The implicature of driving in the UK contextually assumes that cars drive down the left-hand side of the road but this is explicitly encoded in the BSL rendition. I would argue that this is linguistically driven as it is encoded spatially in BSL and serves as an example of ostensive communication where the intention to communicate that the car was driven down the left-hand side of the road is made manifest and so satisfies Relevance Theory principles. The language, or language and gesture fusion (Liddell, 2003), utterance points to a phenomenon that the translator wishes to communicate; it is more explicit than the SL via an enrichment of implicature that is spatially expressed.

In Relevance Theory, the concept of relevance is assessed in terms of cognitive effects and processing effort:

Relevance of an input to an individual

Other things being equal, the greater the positive cognitive effects achieved by processing an input, the greater the relevance of the input to the individual at that time.

(Wilson & Sperber, 2002: 252)

The goal of the translator is to maximise the relevance of the TL for the target audience and this in turn means that the TL is constructed in such a way that it is relevant to the audience rather than just seeming relevant to the audience. Optimal relevance then becomes:

An ostensive stimulus is optimally relevant to an audience if [if and only if]:

It is relevant enough to be worth the audience’s processing effort;

It is the most relevant one compatible with communicator’s abilities and preferences.

(Wilson & Sperber, 2002: 256)

In Relevance theoretical terms, ease of understanding is viewed within a framework such that comprehension is described as below:

Relevance-theoretic comprehension procedure:

a Follow a path of least effort in computing cognitive effects: test interpretive hypotheses (disambiguation, reference resolutions, implicatures, etc.) in order of accessibility

b Stop when your expectations of relevance are satisfied.

(Wilson & Sperber, 2002: 259)

In the example of “the plane” described earlier, the ostensive communication is made manifest in the TL by using the index point which disambiguates which plane is being discussed without further cause for reference resolution. The index point in the TL more explicitly draws the audience’s attention to the information that is being shown on the screen and so the BSL TL uses the multimodal resources as part of its explicature, as opposed to the implicature that is used in the SL.

3 Data analysis

To further elucidate this Relevance theoretical account of BSL rendered news let us consider some data. The analysis concerns news and current affair programmes rendered by deaf sight translators/interpreters working from an English autocue (teleprompter) when presenting prepared renditions to video camera ready for later broadcast in the UK. These programmes formed part of the live broadcast and re-broadcast programmes that fulfilled the legal obligations placed on broadcasters by the Broadcasting Act (1996) and Communications Act (2003) in the UK. In total nine hours of footage was analysed using the ELAN¹ video annotation tool. Instances of interacting with the screen were noted and then later categorised. The findings were then presented at a workshop of the sight interpreters for respondent validation.

For the live broadcasts the sight interpreters (reading a live subtitling autocue rather than listening to the spoken word) were able to watch the news to familiarise themselves with the news that was broadcast, its format, the scripting and the footage that was used to support the news reading and news reporting. For the rebroadcasts, the sight translators (reading from a pre-prepared autocue) had the opportunity to view the programmes, read transcripts of the programmes already prepared for subtitling (captioning) and rehearse their BSL rendition while also viewing the images and footage that were to be broadcast.

Viewing the images enabled the sight translation/interpreting professionals to select appropriate visually-motivated lexical items, but also to ensure that any other forms of depiction such as reconstructing conversations between several interlocutors, e.g. interviewer and interviewee, were presented isomorphically with the screen for example twisting the torso so that the plane of the torso was facing in the same direction as the interviewer to re-present interviewer talk, and in the plane of the interviewee to represent interviewee talk (see Quinto-Pozos & Mehta, 2010). This is an analogue of constructed dialogue as described by Tannen (2007) for spoken English and described by various authors for sign languages (see Lillo-Martin, 2012 for an overview). My argument is that these explicitations of implicatures ensure that the BSL conforms to the norms of a covert translation, and in so doing reduces the cognitive effort of the BSL viewing audience by increasing the optimum relevance of the BSL TL.

Before we consider the categories that emerge from the data it is worth us considering the environment in which the translations are crafted. In Figure 8.1 we see the studio in which the Deaf translators work.

The translators have various sources of information available to them in the studio when they are representing their prepared and rehearsed translation. First, they can see the image that is being broadcast, next to this is the autocue script which delivers the SL. If they wish, they can also look at the broadcasts superimposed upon (see Figure 8.2), which reminds the translator of any visual motivations that can be found in the footage.

They can also see a screen with them superimposed on to the footage (which is the view that the audience or receiver of the TL will watch). The video footage and the superimposed video footage allow the sight translator to judge any moments when visually-motivated multimodal explicature can be used in the TL such that it conforms linguistically or culturally to BSL communication norms (Cormier etal., 2015). The types of visually-motivated decisions can be divided in three different categories: pointing at things appearing on the screen and then telling; telling the audience the information in the TL while using visually-motivated lexical signs to index the relevant information co-occurring one screen; showing by using visually-motivated lexicon while also indexing other on-screen activities through visually-motivated verb inflections and other depicting strategies. We will now consider some examples to further understand these phenomena.

3.1 Pointing

There are several examples of pointing that can be seen and these are phenomena that I label as pointing without any other deictic function being co-produced. These are all explicatures of the source language, although they could be considered gestural explicitation or strategies that make manifest context assumptions within the shared cognitive environment. These fall into watching and index pointing, which I shall now describe.

3.1.1 Watching

The first example is pervasive throughout sight translation and interpreting and could be considered a type of translation by illustration (Baker, 2011), and yet manifests in the covert translation by watching (see Figure 8.3). Here we see the rendering professional watching the video images that are presented on the screen and allowing the sign language viewer to see the images before the professional then engages in a covert translation that tells the viewer the information that is being presented by the news reader. This is a part of the functional ostensive communication of the covert translation that makes manifest to the viewer that there is footage to watch. This also then provides the context for the rendered language and makes explicit the shared cognitive environment that the linguistic communication is being presented within.

This also, however, forms part of the translation process. The sight interpreter chooses to look at the autocue that is next to the superimposed image rather than the image near the camera. This gives the sight interpreter access to the live subtitling of the spoken text produced by the newsreader, thus reading (i.e., “listening to”) the SL. This dual function of providing the SL while directing viewer gaze has an important role in sight interpretation and translation for (re-)broadcast television. It enables a covert translation by appearing to the viewing audience as an invitation to engage in culturally-appropriate watching behaviours while hiding, or at least reducing, the rendering process. This also then appears to be an invitation from the original speaker rather than attributed to the interpreter, potentially contributing to mutuality. The positioning of the various resources within the studio can support the covert translation norm by enabling a process that permits this layering of two functions in one seen action.

3.1.2 Index pointing

Of all of the forms of multimodal deictic enrichment, index finger pointing is the most transparent. Not only transparent to those reading this chapter but also to those viewers of the broadcast programmes who do not understand BSL and yet can clearly view the pointing that the BSL translators engage in as seen in Figure 8.4.

In this example, the BSL translator points to the screen using an index finger (there are other ways of pointing in BSL which are less specific, i.e., using a full hand with the palm facing upwards) and, in so doing, disambiguates the noun that is subsequently produced. In this example the sight interpreter then tells the audience what is said in the SL but also reproduces an isomorphic representation of the graph. In this way the notion that inflation is increasing, or decreasing, is understood within the context of the graphic on the screen. Whereas the news presenter discusses inflation and points to the info-graphic, the BSL covert translation becomes more overt using the linguistic elements of the tracing of the graph to refer to these elements.

The pointing behaviour presupposes that the deaf BSL-using audience will want to see culturally-appropriate behaviours. It also points towards the graphic that the presenter has chosen to represent the information he is also talking about. When the sight interpreter depicts the graph after pointing to it, the viewing audience is given the impression that they not only have access to what the presenter says but also to the thoughts and thought processes behind drawing the graph. This shift to depict the graphic, then, brings about some sense of mutuality for the audience.

In a fully covert translation the sight interpreter would further interact with the on-screen images by also pointing at the info-graphic, although this would then shift away from mutuality to attributive use, i.e., this is my thoughts on their thoughts. The preliminary norm (Toury, 1995), however, is such that even though the interpreter could move across the screen and engage in index pointing there is an expectation that the interpreter will stay on the right-hand side of the superimposed broadcast image (Ofcom, 2015). In this regard even though the translation goal is a covert translation there is still the need to be mindful of mainstream sensibilities and so to constrain the translation to index pointing and linguistic explicitation.

A well-known example of offending mainstream sensibilities in the Deaf community occurred the first time the Queen’s Christmas Day speech was rendered into BSL. The non-signing audience felt that the sight translator did not represent the dignity and gravitas of the Queen appropriately, even though the (grammatical) facial expressions that were used by the sign interpreter were linguistically appropriate. A number of complaints were received by the BBC, which in turn led to choosing professionals who would be mindful of this in later years. We could imagine similar complaints from the non-signing public if the sight interpreter encroached further on the screen.

3.2 Telling while pointing

There are, however, moments when sight translators engage in telling and showing. Some of the linguistic resources available in BSL can be pointed to things. This can be locations in space that are either to real world objects or to locations in space that are established as discourse referents (Perniss, 2012). One of the resources that can be pointed is the possessive index (a clenched fist in BSL where the orientation of the palm points to the possessor).

In Figure 8.5, just prior to the image shown we see a gaze towards the boy shown on the video footage. Then we see the use of the possessive sign (shown in the white circle) and this is then pointed towards the boy on screen while the gaze of the sight translator returns to the camera (i.e., the viewing audience). The translator’s gaze directs the viewers to the discourse referent, which in this case is the boy in the footage. Thus, there is complex interplay between being in the studio with no present audience and using BSL in a culturally appropriate way for the audience, i.e., in a pre-supposed face-to-face interaction, which directs the audience to the relevant information in the shared cognitive environment. The sight translator is looking at the camera and so does not see the superimposed image of themselves on the broadcast footage and yet knows that optimal relevance can be achieved by making manifest to the viewer the discourse referent from the viewer’s perspective.

From the professional’s perspective the image is seen on the screen in front of them and to the side of them. The pointing of this sign in the studio is not to any discourse referent. Only in the superimposed image does this pointing make sense, or at least add the explicature of something belonging to the specific boy shown in the footage. This disambiguates the pronoun explicitly as there is no ambiguity for the watching audience. Here this disambiguation perfectly marries the pointing behaviours of BSL users and the constraints of the in-vision operational norm such that a covert translation can be created.

3.3 Showing

Finally, the data contain instances of showing in BSL renditions. In Figure 8.6 we see the use of a depicting verb, that linguistically encodes the visual form of an event. This language use as mentioned above is visually motivated. This can be seen by the use of the handshape to represent the water/retardant being dropped onto the fire. Here the handshape is informed by the video footage and we can see that the geometric plane of the hands in the BSL is on the same geometric plane of the water being dropped. This isomorphism moves away from a telling strategy to a visually mimetic showing strategy. While the sight interpreter could tell the audience what is happening, the embodied language use also involves a tilt of the head while simultaneously articulating an adjectival mouth morpheme that represents “large quantity”. Again this showing satisfies the covert translation expected by the audience.

In the final example in Figure 8.7 we see several images of a sight translation from a current affairs programme describing the organisation of dabbawala (the lunch delivery system in Mumbai). In the first still on the left-hand side we see an index point to the screen where we see the dabbawala carrying lunches on his head. Next the sight translator looks to the image while producing the sign HEAVY (evoking mutuality). In the next still we see the “heavy load” indexed with the eye gaze looking away from the viewer and away from the screen (maintaining the mutuality); here the sight translator is constructing dialogue by positioning her head in the plane of the interviewer who had previously appeared on the screen.

In the final still the sight translator uses visually-motivated depicting handshapes to show the type of container that the “heavy load” is filled with. This is then moved along the horizontal plane to depict some of the cylindrical objects on the heavy load. This chain of pointing, telling and showing strategies demonstrates the complexity of the explicatures that the sight interpreters and translators engage in. By using a variety of visually motivated depicting strategies while looking at the video footage and ensuring the language is presented to the audience in an optimally relevant way a covert translation is often achieved, even within the operational norm constraints of a televised seen and presented translation.

Many of the language behaviours described above are now noticed by non-signing audiences in the UK; stand-up comedians will use this as material for their comedy shows to mainstream audience. That this becomes the subject of mainstream humour (often but not always achieved in a non-offensive way) demonstrates the pervasiveness of these strategies; the “otherness” of the ostensive communication norms of BSL users that are made manifest during in-vision sight translation and interpreting; and the creation of a covert translation that creates a “deaf space” within the news and current affairs which is culturally and politically motivated (Stone, 2007, 2009; Adam etal., 2011).

Concluding remarks

In this chapter the case of BSL translators has been considered via a Relevance Theory lens. The translation of broadcast television into BSL is required by legislation in the UK and some of these programmes are news and current affairs programmes. Most of these programmes include spoken information that is delivered in a multimodal multimedia environment where interviewees are shown, or video footage is talk about or voiced over while information is understood within the context of that multimedia environment. Scripted and spontaneous speech is uttered within the context of other information being made available in the shared cognitive environment of the newsreaders/presenters and the viewers, and this shared cognitive environment is also available to the BSL sight translators and interpreters.

The BSL sight translators and interpreters engage in three different forms of multimodal deixis: pointing, telling and showing. These are visually motivated by the shared cognitive environment and satisfy the linguistic requirements of BSL and the cultural norms of using depicting strategies to communicate using the single channel of vision. Depicting deixis may be from the gestural repertoire, linguistic repertoire or a fusion of both linguistic and gestural repertoires, and yet satisfies the ostensive communicative requirements in making manifest optimally relevant multimodal resources to the viewer.

Relevance Theory helps us to explain the decisions that the sight translators and interpreters make which take into account the non-present audience. The presence of the translator/interpreters on screen makes those professionals present much like interpreters in the community such as those described in Blakemore and Gallai (2014). Many of the strategies in trying to achieve the covert translation may well be striving to achieve mutuality rather than the attributive use conceived by Gutt (1991).

Further analysis needs to be undertaken to better understand whether the distinction between attributive use and mutuality help to explain in-vision interpreting. It may also be a function of genre, in that when programmes other than current affairs and news, i.e., factual programmes, are translated or interpreted mutuality may or may not play a part. Similarly, it would be interesting to explore how pointing, telling and showing strategies are used in face-to-face interpreting or sight translation when the professional and the audience is in the same location and there is interaction.

The presentation of a prepared sight translation or interpretation by a seen translator or interpreter in a visual unwritten language (BSL) necessitates appropriate depicting strategies to be used if a covert translation is to be achieved. This in and of itself is a socio-political act, rather than shying away from language practices that can be mocked by the mainstream (and have been in the past, Ladd, 2007) the presence of the seen translator or interpreter gives BSL a place in the national cultural landscape. In being witnessed by a larger mainstream audience the BSL translators, while satisfying the face-to-face communicative norms of BSL users, demonstrate different ways of engaging in communicative practices to non-signers that also creates awareness of sign language and sign language communication. In so doing translation, communication and political practices are made manifest.

Note

1 https://tla.mpi.nl/tools/tla-tools/elan/.