Arithmetic in the Bilingual Brain

Nicole Y. Wicha *; Danielle S. Dickson *; Amanda Martinez-Lincoln † ^* The University of Texas at San Antonio, San Antonio, TX, United States
^† The University of Texas at Austin, Austin, TX, United States

Abstract

In the United States, approximately 1 in 5 children enters school with a first language other than English (U.S. census data). The impact that this has on learning and using simple arithmetic is not well understood. Models of arithmetic learning argue that arithmetic facts are encoded in a verbal memory store specific to the language in which the facts were learned (e.g., Dehaene, S., & Cohen, L. (1995). Toward an anatomical and functional model of number processing. Mathematical Cognition, 1(1), 83–120.) or that bilinguals maintain separate representations of mathematical facts for each language (e.g., Campbell, J. I. D., & Xue, Q. (2001). Cognitive arithmetic across cultures. Journal of Experimental Psychology: General, 130(2), 299–315.). Authors of other chapters in this book provide evidence that bilingualism impacts symbolic number knowledge in preschool children and that bilingualism may also impact nonsymbolic processing in bilingual adults. In this chapter, we discuss the behavioral and brain evidence for how young adults and children process simple arithmetic, mainly focusing on multiplication, and how monolinguals and bilinguals differ. We present evidence from our own work using scalp-recorded event-related brain potentials showing that both the language of learning math facts and the use of math facts in a language can influence the efficiency and the way in which bilinguals access these facts in each of their languages.

Keywords

Bilingual; Arithmetic; Multiplication; Congruency; Children; N400; P300; Event-related potentials; ERP

Introduction

Pertinently, simple multiplication problems (e.g., “2 × 4 = 8”) have traditionally been memorized through mental repetition and rote verbal rehearsal, generating a language-based representation of these arithmetic facts, much like other overlearned verbal expressions (De Smedt, Noël, Gilmore, & Ansari, 2013; El Yagoubi, Lemaire, & Besson, 2005). Perhaps unsurprisingly, then, fluent use of verbal retrieval strategies for these problems is associated with higher levels of performance (faster and more accurate responding) for these problems (Prado et al., 2013), although procedural approaches to solving problems are sometimes used (e.g., LeFevre, Sadesky, & Bisanz, 1996).

Bilingualism and Retrieval of Mathematical Facts

In this chapter, we address how the implementation of verbal retrieval strategies to efficiently access arithmetic-related information might be affected by the ability to speak more than one language. Bilingualism is growing in the United States, both from changes in the population demographics and the rising popularity of bilingual education in U.S. schools. Approximately 1 in 5 children enters school with a language other than English as their first language, and in parts of the United States, like South Central Texas, bilinguals can comprise over 50% of the population (2010 U.S. Census). Much research has been devoted to determining the effects of bilingualism on aspects of a speaker's language acquisition (e.g., with respect to receptive and productive vocabulary; Hoff et al., 2012; Thordardottir, Rothenberg, Rivard, & Naves, 2006; Umbel, Pearson, Fernández, & Oller, 1992) and the influences of bilingualism on broader cognitive abilities (e.g., executive function, for which evidence is mixed and contentious; Bialystok, Craik, & Luk, 2008; Hilchey & Klein, 2011; Paap & Greenberg, 2013; Paap, Johnson, & Sawi, 2015). Given this (sometimes heated) backdrop, it is important to be mindful that the most critical feature of the bilingual experience is still the ability to speak and comprehend in more than one language—a communication advantage that is undeniable.

In educational settings, however, bilingual children typically learn arithmetic facts in only one language—the language that is spoken in their schooling environment. When challenged to solve problems with number words in their other language, bilinguals are able to do so (i.e., they can produce or recognize the correct solution). Nevertheless, they do show differences in accessing math facts across their languages (e.g., Bernardo, 2001). Critically, retrieval fluency for simple arithmetic predicts later higher math competence, which in turn can lead to lifelong consequences on earning capacity and quality of life (e.g., Geary, Hoard, Nugent, & Bailey, 2013; Price, Mazzocco, & Ansari, 2013; Siegler et al., 2012). We do not know whether bilingual children make this critical transition to retrieval fluency as successfully as monolinguals do. Further, if they do transition to using fluent retrieval strategies, we do not know whether this skill is limited to the language in which they were instructed. Given this context, 20% of U.S. children who are bilingual are at a potential disadvantage in early math learning. Characterizing the nature of how this population processes math facts is therefore an important facet of minimizing their potential educational challenges.

How Bilinguals Process Arithmetic

Other chapters in this book indicate that bilingualism may impact symbolic number knowledge in preschool children (Chapter 9 by Sarnecka et al.) and nonsymbolic processing in bilingual adults (Chapter 8 by Salillas and Martínez). In this chapter, we focus on the behavioral and brain evidence for how young adult and child bilinguals process arithmetic, specifically looking at multiplication as an example of exact arithmetic processing. Our lab approaches this research from a bilingualism perspective, informing our questions and paradigms with our studies of language in the bilingual brain. We review bilingual arithmetic processing research, focusing on studies with adult bilinguals that report electrophysiological measures of brain activity during the processing of arithmetic problems. We present evidence from our own work using scalp-recorded event-related brain potentials showing that the language of learning math facts and the frequency of using math facts in a given language can influence the efficiency and the manner in which bilinguals access this information. We conclude with preliminary findings from our investigations of the developmental trajectory for multiplication fact learning in elementary school children. This research has laid the groundwork for understanding bilingual arithmetic, a building block for achieving higher math fluency in 11.2 million (and growing) U.S. children (Aud et al., 2011).

Relevant Models of Mathematical Cognition

Before considering math in the bilingual brain, it is appropriate to discuss the relevant models of math cognition that have set the framework for thinking about arithmetic processing. As mentioned above, simple multiplication problems are solved primarily through direct retrieval from long-term memory, although there is evidence that people use other strategies to solve these problems as well (e.g., repeated addition), especially earlier in development (LeFevre et al., 1996). The details of the primary models of simple arithmetic processing differ somewhat in how they account for different phenomena but largely agree on core principles (see Ashcraft (1992) for a review of differences and similarities in the network-retrieval model (Ashcraft, 1987), the network-interference model (Campbell & Graham, 1985), and the distribution of association model (Siegler, 1988)).

In particular, consistent with modern theories of how semantic information is thought to be retrieved from linguistic representations, simple arithmetic information is accessed through spreading activation across a network of interrelated facts. Differences in response time (RT) to specific problems are then accounted for by the strength of the association between the memory representation for the given problem and its answer, which is strongly influenced by the problem solver's frequency of exposure to these problems and experience with retrieving the answers (see Ashcraft, 1995, for review; c.f., Barrouillet, Fayol, & Lathulière, 1997; Kaufmann, Lochy, Drexler, & Semenza, 2004; Lemaire, Abdi, & Fayol, 1996). Importantly, then, not all information in the network is retrieved with the same level of ease; the problem-size effect, for instance, where problems with smaller solutions are retrieved faster and are less error-prone than larger problems, suggests that some parts of the network are more accessible than others, perhaps due to less interference from other stored facts (Campbell & Graham, 1985).

This spreading of information across the arithmetic network leads to variable RTs in verification tasks, wherein subjects must respond with a confirmation or rejection of the provided answer (e.g., 4 × 3 = 12?), based on the strength of the connection between the provided answer and the correct solution. In initial work using this type of verification task (instead of production tasks that had dominated the literature, wherein the answer is not provided and people produce their own answer), Campbell (1987) observed delays in RT when participants needed to reject an answer as incorrect if that answer shared properties with the (not presented) correct solution. For example, if not only the provided answer is incorrect (e.g., 3 × 4 = 16) but also it happens to be the correct solution for a table-related problem (4 × 4 = 16), then RTs are longer than if the provided answer were completely unrelated to the problem (e.g., 3 × 4 = 17). Typically, then, the fastest RTs are observed for verification of correct answers (due to familiarity and/or retrieval success), slower RTs are observed for incorrect answers with low association to the correct solution, and the slowest RTs are observed for incorrect answers that are associated with the problem itself or with properties of the correct solution (e.g., table-related numbers, Ashcraft & Battaglia, 1978; Lemaire et al., 1996). This pattern of results is broadly inconsistent with early views in which answer retrieval and verification for a given problem unfolded without influence from other known arithmetic facts (for more on this history, see Brysbaert, 2005; Fayol & Seron, 2005).

The exact structure of the representations in this network has been a matter of some debate, especially given that the same facts can be presented in different forms (e.g., Arabic digits and number words). In addition, the extent to which language itself affects access to or retrieval of these facts is debated (Campbell & Xue, 2001; Dehaene & Cohen, 1995). In particular, within the context of multilingualism, there are potentially multiple avenues to retrieve arithmetic-related information from long-term semantic memory. Although the theories of fact representation most relevant to this discussion propose a representation of arithmetic facts in some form of verbal memory, they disagree about the nature of the relationship between learned arithmetic facts and verbal memory in each language. Specifically, one class of theories presupposes that arithmetic facts are encoded in a verbal memory store restricted to the language in which the facts were learned (e.g., Dehaene & Cohen, 1995), whereas the alternative views suggest that these same facts exist in separate verbal memory stores for each language (as well as digit formats), even though the facts were primarily learned only in one of the languages (e.g., Campbell & Xue, 2001).

The dominant example of the former theory is the triple-code model (TCM), which suggests that multiplication tables are “syntactically organized sequence(s) of words” and that the language processing system is the basis of arithmetic fact retrieval (Dehaene & Cohen, 1995, p. 85; Dehaene, Piazza, Pinel, & Cohen, 2003). According to this model, the ability to retain and retrieve arithmetic facts is highly dependent on the language system, as indicated by brain injury case studies (Delazer & Benke, 1997). Evidence in support of the model (specifically, the proposal that simple arithmetic facts can simply be retrieved rather than calculated through manipulations of quantity) comes from the finding that lesions in the inferior intraparietal region impair knowledge of number quantities but do not affect the retrieval of multiplication facts (for review, see Dehaene, Dehaene-Lambertz, & Cohen, 1998; Dehaene et al., 2003). On this view, all well-rehearsed simple math facts are verbally encoded, and all exact arithmetic problems, including those presented visually in the Arabic digit format, rely on transformation into this verbal code before retrieval is possible. Most importantly for bilingualism research, the verbal code is specific to the language in which the information was learned (Dehaene, Spelke, Pinel, Stanescu, & Tsivkin, 1999).

In contrast, the encoding-complex model (ECM) proposes that simple multiplication facts are encoded in format-specific representations, including one for the digit form (or forms, as is the case for Mandarin speakers) and one for each written number-word form—that is, each language has its own representation of arithmetic facts (Campbell & Clark, 1988). The facility with which facts are successfully retrieved is modulated by the strength of access to each format-specific representation, through a mechanism similar to how individual problems within the network of arithmetic might vary in retrieval strength as a function of experience (Campbell & Clark, 1988, 1992).

Since the digit format answers can be accessed independently of their verbal counterparts and since people have more experience using the digit than word format, the ECM makes the prediction that answers to arithmetic problems presented as digits should be more easily accessed and less susceptible to retrieval interference than answers to the same problems presented as number words (Campbell & Clark, 1992). This would seem to be in direct contrast to the TCM, in which processing of digits requires a kind of translation into the verbal code for successful answer retrieval.

Indeed, the TCM proposes that the disadvantages for the less frequently used language are the result of additional translation processes from the inexperienced language to the experienced language, from which the rehearsed verbal code is then accessed (Dehaene et al., 1999, p. 971). In contrast, the ECM predicts that retrieval can happen in each language without the need for translation across languages, although access to each language should vary based on the strength of the connections within and between modules (Campbell & Epp, 2004). That is, if a problem is presented using number words in the language with which the solver has more experience, retrieval for that language will be superior (at least on some measures) to retrieval for problems presented in the alternate language, due to frequency of exposure.

Owing to the similarities in predicted behavioral outcomes (e.g., both theories suggest slower responses for the inexperienced language), adjudicating between these views can be challenging with behavioral evidence alone. Next, we review behavioral results and note the level of support provided for each theory when possible.

Behavioral Evidence From Bilinguals as Tests of the TMC and ECM Models

In general, bilinguals are more efficient at processing arithmetic in the language of learning or acquisition (herein LA +), which is often, but not always, their first language (L1) (c.f., Bernardo, 2001).¹ Perhaps, the most influential study in this area was conducted by Spelke and Tsivkin (2001). Bilingual college students were taught a set of new math facts (and other information) in one of their languages (English or Russian) and then later attempted to recall the facts in the same language or the untrained language. No difference was observed across languages for math that required calculation, implying that this new knowledge was not encoded in a language-specific manner. However, math that would be more likely to rely on verbal retrieval strategies (i.e., exact arithmetic) was solved faster and more accurately in the language of learning than the other language (herein LA −). This advantage was interpreted as evidence that the math facts themselves were stored and processed in the language of learning, in line with the TCM. However, the emergence of this kind of processing advantage does not necessarily have to be the result of selective processing in LA + and an absence of processing in LA −. The delays for LA − relative to LA + could instead have been caused by comparatively weaker access to these math fact representations in LA −, consistent with the ECM. Thus, simple comparisons of behavioral performance under LA + and LA − formats can lead to interpretations in support of either theory.

Importantly, a third condition, wherein researchers additionally examine behavioral performance when problems are presented in the Arabic digit format, has produced results that are predominantly consistent with the predictions made by only the ECM theory. For example, Bernardo (2002) observed that Filipino high school students, who received their education in English but spoke Tagalog as a first language, performed better in arithmetic when it was presented in English (LA +) than when it was presented in Tagalog (LA −). As before, this result taken in isolation could still be interpreted as support for either view of bilingual arithmetic processing. However, the Arabic digit format also resulted in superior performance with number words (regardless of language), which is more consistent with the ECM, given that the TCM predicts that LA + should be the format with most direct access for fact retrieval.

In other research with bilinguals, Frenck-Mestre and Vaid (1993) similarly observed the fastest RTs for the verification of problems presented as Arabic digits, the slower RTs for problems presented as words in LA +, and the slowest RTs for problems presented as words in LA −. Our lab has also shown that the brain responses for multiplication verification occur earlier for problems presented as Arabic digits than number words (Salillas & Wicha, 2012); this is true for both bilinguals and monolinguals (Martinez-Lincoln, Giattino, Chapman, & Wicha, 2014). In brief, although it is true that the LA + word format is processed more efficiently than the LA − word format, the consistent advantages for the digit format over the number words present a challenge to the view that arithmetic problems are always solved by verbal translation into some kind of LA + code. On that view, it might be expected that number words themselves would be most efficient, given that they would provide access to the memorized arithmetic facts without the need for transcoding prior to retrieval (c.f., Siegler, 1988). This digit advantage over number words was a critical motivating factor for the development of the original ECM as a model of monolingual arithmetic processing (Campbell & Clark, 1992).

Thus, the ECM appears to do a more complete job than the TCM of explaining format-related effects in bilingualism and arithmetic in general. However, it is still an imperfect guide to explaining the mechanisms that bilinguals engage for exact arithmetic. There is accumulating evidence that the bilingual lexica are interconnected and that bilinguals are unable to completely suppress one language when processing the other (e.g., Dijkstra & Van Heuven, 2002; Hernandez, Woods, & Bradley, 2015; Hoshino & Kroll, 2008; Kroll, Sumutka, & Schwartz, 2005; Marian & Spivey, 2003; Perani & Abutalebi, 2005; Sunderman & Kroll, 2006). It is problematic, then, that the ECM proposes separate verbal memory stores for each language that are only connected to each other indirectly through the digit format or through the magnitude code (Campbell & Epp, 2004). The model's assumption that math facts are accessible only through one independently operating language at a time, with no direct associations across languages, is thus broadly inconsistent with what is otherwise known about the dynamics of bilingual verbal memory.

In general, then, no existing theory of bilingual arithmetic processing is perfect or complete. What seems likely to lead to improvements is to incorporate more knowledge from existing fields (i.e., the bilingualism literature more broadly) and to acquire more evidence from other methods for studying cognition (i.e., brain imaging). Neuroimaging research is important, not only because it provides unique insights into the brain mechanisms underlying behavioral performances (see also Berch, Geary, & Mann Koepke, 2016), but also because it can provide measures of distinct cognitive processes indiscernible by behavior methods alone. Namely, the multidimensional nature of the results can help determine if differences in performance across languages are caused by quantitative differences in the same underlying cognitive and neural processing mechanism, such as more efficient use of the same mechanisms, or qualitative differences in processing, such as different neural subsystems for each language.

Although both the TCM and the ECM predict a quantitative processing disadvantage for the LA −, the proposed source of the disadvantage is different. The TCM predicts qualitative differences in processing, with LA − relying on translation rather than direct retrieval, which would imply substantial differences in the cortical substrates as well. In contrast, the ECM predicts similar verbal memory retrieval mechanisms for both languages, perhaps using neighboring regions of cortex. We will now review brain evidence in bilingual processing of arithmetic, some of which can be taken as support of a single, language-specific system of arithmetic processing and some of which is more consistent with a system with multiple formats whose access is weighted by experience.

Review of Brain Evidence in Bilingual Arithmetic Processing

Neuroimaging Techniques

For the most part, neuroimaging techniques, such as functional magnetic resonance imaging (fMRI) (and other spatial localization techniques), have focused on locating activations in different regions of the brain during arithmetic processing by monolinguals. In general, verification tasks that are thought to rely on retrieval strategies (i.e., judging simple arithmetic problems for correctness) have been associated with a large frontoparietal network (e.g., Chochon, Cohen, Van De Moortele, & Dehaene, 1999; Fehr, Code, & Herrmann, 2007). Interestingly, particular subregions of the parietal lobe seem critical for successful arithmetic processing, especially when relatively more calculation and estimation is involved, as is the case when problems are less rehearsed (e.g., division problems) or when answer retrieval is otherwise not yet fully mastered (Delazer et al., 2003; Grabner, Saalbach, & Eckstein, 2012; Prado, Mutreja, & Booth, 2014; Qin et al., 2014; Rosenberg-Lee, Chang, Young, Wu, & Menon, 2011; Tschentscher & Hauk, 2014). The role of frontal regions during arithmetic processing has not been as well characterized but is often attributed to general cognitive demands on attention and executive function (Fehr et al., 2007; Prado et al., 2011; see Arsalidou & Taylor, 2011, for review).

As children learn arithmetic facts such as multiplication tables, they become increasingly reliant on direct retrieval strategies. As retrieval success increases, there is in turn a decrease in the engagement of parietal regions (Prado et al., 2014). At the same time, a compelling finding has been that increased use of verbal retrieval strategies can be associated with increased engagement of left temporal areas that are also associated with language processing (Berteletti, Prado, & Booth, 2014; Prado et al., 2014, 2011; also see Price, 2012, for a review of the rich and complicated neuroimaging findings in the language literature). These results largely confirm what the behavioral literature already suggested: some subset of cognitive resources associated with verbal language are used when retrieving arithmetic facts from memory, and these language resources are increasingly tapped as arithmetic facts are rehearsed and successfully memorized.

Not surprisingly, then, several studies have also provided evidence for a language of learning advantage when bilinguals are retrieving these facts, in the form of increased activity in frontal and parietal areas when processing in LA −. This pattern of activity is often attributed to an increase in difficulty for LA − fact retrieval, and it is typically interpreted as a sign that participants might be engaging in more complex strategies to solve these problems (i.e., calculation rather than fact retrieval) (e.g., Grabner et al., 2012; Lin, Imada, & Kuhl, 2012; Mondt et al., 2011; Venkatraman, Siong, Chee, & Ansari, 2006). Importantly, however, the network involved in processing math facts in either language is similar—what appears to change is how much relative demand there is on different parts of the same anatomical network. It seems fair to take these results as an indication that each language's retrieval mechanism is not completely autonomous and might share important commonalities. In general, the results from this initial work do not provide support for substantially different networks when processing arithmetic facts across languages. It is still possible, though, that additional research could reveal distinctions.

Temporal Precision in Neuroimaging Techniques

A limitation of localization techniques such as fMRI is that they trade off higher precision in space for lower precision in time, providing at best a 1 s resolution (precision in time) of real-time brain activity. However, changes in neural activity occur on the order of milliseconds (ms), with several processing stages taking place within the first second of experiencing a stimulus. In other words, the timing of these rapid changes in brain activity is lost to most neuroimaging techniques that prioritize anatomical localization. Time-sensitive brain imaging measures, such as neuronal electrophysiology, can then provide the complementary temporal precision that is needed to separately measure how processing unfolds prior to, during, and after a stimulus is perceived or a response is executed.

Electrophysiology and Event-Related Potentials

For this reason, electroencephalography (EEG) has been successfully used to study multiple stages of arithmetic processing in both monolinguals and bilinguals. EEG is a direct measure of neural activity, typically recorded from the scalp, which reflects the electric changes over time of large populations of neurons with millisecond precision. From intracranial recordings, we know that EEG generally captures postsynaptic potentials from cortical pyramidal neurons (for review, see Luck & Kappenman, 2011). The ongoing EEG can be time-locked to particular events of interest and then averaged across trials and subjects to generate the average brain response or event-related potential (ERP) to an experimental condition of interest. These derived ERPs are a time-sensitive multidimensional measure of brain electric activity, with functionally specific effects (i.e., changes in the recorded activity compared to a baseline). The amplitude of the change in voltage, polarity (whether the difference in voltage between conditions is negative or positive), latency (the timing of the effect), and scalp distribution (which electrodes show the effect) of the waveform are all independently informative about the nature of ongoing cognitive activities. Importantly, ERPs can provide information about processing without dependence on explicit responses or self-reported of strategies, which have been a particular issue of some contention in the domain of arithmetic problem solving (e.g., Fayol & Thevenot, 2012).

In the first 300 ms after the onset of a stimulus, ERP components (the typically observed modulations in amplitude) generally reflect modality-specific sensory processing. For example, with visual stimuli, early components are modulated by contrast, size, brightness, and attention (Pratt, 2011). After about 300 ms, more modality-independent effects emerge, including well-characterized cognitive components that have been the most relevant to studies of arithmetic processing—the N400 (see Fig. 1) and a subsequent positive shift or late positive component.

f07-01-9780128125748 — Fig. 1 Grand average ERPs from monolingual adults recorded in response to expected and unexpected sentence-final words (top, from Wicha, Moreno, & Kutas, 2004) and in response to correct and incorrect multiplication solutions presented as number words (middle) or Arabic digits (bottom) (unpublished data from Wicha lab). The data come from a representative electrode, with 1 s of time in milliseconds along the X-axis and voltage (in microvolts, with negative plotted up) on the Y-axis.

ERP studies of arithmetic have typically measured two effects: the congruency effect, primarily observed in verification tasks during recordings of the brain response to a provided answer (e.g., Niedeggen, Rosler, & Jost, 1999), and the problem-size effect, observed during recordings of the brain response either to provided answers or after presentation of the operands (e.g., 7 × 6) but prior to the answer itself (Jost, Hennighausen, & Rösler, 2004; Zhou et al., 2006). We focus here on the congruency effect, found in comparisons of brain responses for congruent solutions (correct answers) and incongruent solutions (incorrect answers), as this has been the primary effect measured in bilingual populations. Importantly, typical studies of the congruency effect report the time-locked brain response to the answer presented in isolation, after retrieval/calculation processes have already been initiated by prior presentation of the operands.

The congruency effect is characterized by a more negative response to incorrect solutions (e.g., 7 × 6 = 36) compared with correct solutions (e.g., 7 × 6 = 42), with a maximum peak amplitude difference occurring about 350–400 ms after the presentation of the answer (Niedeggen et al., 1999). This arithmetic congruency effect has drawn comparisons to similar effects in other domains that study the processing of potentially meaningful items (e.g., language and object processing). Notably, words that are unexpected/incongruent with a prior sentential context (e.g., the brain response to “dog” given the prior context, “he takes his coffee with cream and ___”) also elicit a more negative brain response than words that are expected/congruent with a prior context (e.g., the brain response to “sugar” given the same context), around 400 ms after the congruent/incongruent word is presented. Thus, the effect of answer congruency has been traditionally labeled an N400 effect (a negative-going wave sensitive to semantics/meaningfulness, peaking at 400 ms; see Kutas & Federmeier, 2011, for a review of N400 response properties).

Both the timing and the size of this congruency effect (i.e., the average difference in voltage across a time window surrounding the ERP effect) can be taken as independent measures of the brain's readiness to categorize an arithmetic problem as correct or not. If the maximal difference occurs later in time (i.e., at 400 ms instead of 350 ms), then it is reasonable to infer that the brain had to do additional processing before the correctness judgment could be fully rendered (much like an RT delay). Moreover, if the average voltage difference is a smaller or larger size in response to different contrasts of answer subtypes (e.g., table-related or table-unrelated incorrect solutions) or to the same answer subtypes under different contexts (e.g., digits/words and fast/slow presentation latencies), then the size of the difference itself can be taken as an indication of the brain's ability to distinguish between the correct and incorrect solutions. This initial and short-lived (~ 200 ms) effect is generally followed by a slow positive ERP selective to unambiguously incorrect answers. Notably, this later effect is not as consistent or as well characterized and also seems to reflect sensitivity to more subtle differences in answer types (as in traditional language studies, Van Petten & Luka, 2012, for review).

Using ERPs to Investigate Bilingual Processing of Multiplication Problems

We were the first to measure these ERP effects in bilinguals (Salillas & Wicha, 2012). Our samples of adult bilinguals were taught multiplication during childhood in only one of their languages. At the time of testing, they were fluent in both languages and used both languages frequently. Simple multiplication problems were visually presented to them, with one operand presented at a time, followed by the critical stimulus that was either a correct or incorrect solution (e.g., “3…2…6” and “3…2…7,” respectively). The multiplication problems were presented in three different formats: Arabic digit format (above) and written number-word formats in English (e.g., “three…two…six”) and in Spanish (e.g., “tres…dos…seis”).

As in other typical verification paradigms, the participants' task was to determine if the last number was the correct product of the first two numbers or not. We then measured the effect of congruency from the onset of the solution (the third and last number in the sequence) and compared this effect across three critical format conditions: digits, language of learning (LA +), and other languages (LA −). Notably, then, for some participants, LA + was Spanish, whereas, for other participants, the LA + was English. This was therefore not a study comparing the response to specific languages per se, but was instead a study of the effect of arithmetic-specific language experience on arithmetic performance under different contexts.

With respect to the primary effect of interest, the N400-like congruency effect, all three formats succeeded in eliciting robust modulations.² Critically, this effect was more similar than different across formats, indicating that the neural responses giving rise to the effects were at least partially overlapping. However, the ERP responses did differ across formats in two key ways.

First, the timescale for processing digits was different than the timescale for processing number words. Problems presented in the digit format elicited a congruency effect that started and peaked earlier (approximately 55 ms) than the effect for number words. This indicates that problems presented in the digit format were more efficiently processed than problems presented as number words were and is consistent with RT findings. Moreover, these ERP findings are able to pinpoint when congruency is processed in each format—prior to when any response is executed. Recall that ERP congruency effects reflect at least two stages of processing in adults, the N400 and the late positivity. The effect of format was observable on this initial congruency effect. Thus, any theory that attempts to account for format-related processing differences needs to incorporate the impact of the effects at the earliest stages of sensitivity to congruency (see also Megías & Macizo, 2016; Megías, Macizo, & Herrera, 2015).

When initially interpreting this latency difference, there was concern that the delayed processing of number words relative to digits might be related to the generally slower processing of words observed in bilinguals (as compared with monolinguals, e.g., Martin et al., 2012). However, we have since observed an equivalent delay in the latency of the effect for number words compared with Arabic digits in monolinguals (Martinez-Lincoln et al., 2014), suggesting that, indeed, the faster processing of digits was not simply an artifact of slower bilingual word reading. The effect for digits was also distributed more focally over the medial posterior sites on the scalp, whereas the effect for number words was more broadly distributed (spreading more to frontal electrode sites than did the effect in response to digits), suggesting at least partially different neural resources were recruited for arithmetic processing in digit and word formats.

The second key finding was that the number words in LA + elicited a larger-sized effect than did number words in LA −. This suggested that there were weaker activations in the memory network for multiplication facts in LA − although, mechanistically, the lower sensitivity to answer correctness in LA − could result from many possible sources (e.g., less stable representations and weaker access to those representations). Whatever the precise reason, it was apparent that when readers encountered problems in LA −, the assessment of the correctness of the answer was not as robust.

Critically, the effect was no different between LA + and LA − with respect to its distribution across the scalp or its timing. That the scalp distributions were similar suggests that accessing answers to arithmetic problems in LA − relied on similar processing mechanisms (or at least similar neural sources) as accessing answers in LA + did. Although this finding is in line with the ECM in the sense that problems were processed in a similar manner in each language, it is also possibly inconsistent with the notion that separate cortical regions are involved when accessing arithmetic representations in each language (but see Luck & Kappenman, 2011, about the ill-posed nature of source localization of ERPs). Moreover, the similar time course across languages does not support the suggestion that translation from LA − into LA + is necessary for fact retrieval, as suggested by the TCM, which would call for additional processing time when using LA − to accommodate that additional processing stage. In brief, the language of learning advantage in processing simple multiplications appeared to be quantitative in nature (i.e., reflected in the amplitude of the effect), with differences in processing efficiency, as suggested by the ECM, rather than qualitative, such as selective encoding of the facts into only one language, as suggested by the TCM.

An additional observation from Salillas and Wicha (2012) was that the correctness effect in LA −, but not LA +, was positively correlated with the frequency of the daily use of that language. That is, the more a bilingual adult used the LA − in daily life, the better the access to these facts (or stronger representations of these facts) in verbal memory and, consequently, the larger the magnitude of the N400-like congruency effect. This suggested that the processing advantage in LA + (or disadvantage in LA −) might be mitigated by experience using multiplication facts in the other language.

In a subsequent study, Martinez-Lincoln, Cortinas, and Wicha (2015) tested this possibility using the same multiplication verification task, this time with a population of elementary school teachers who were fluent Spanish-English bilinguals. The key element was that half of the instructors taught arithmetic in their LA − (e.g., having learned arithmetic in Spanish, they now taught in English), while the other half of the instructors still taught in their language of learning (LA +). The hypothesis, then, was that teaching in LA − could strengthen access to (and/or representations of) arithmetic facts in that language, perhaps overriding the advantage usually observed for LA +.

As expected, we observed the LA + advantage in teachers whose LA + was also their teaching language (i.e., they retained a larger magnitude congruency effect for LA + compared with LA −) (as per Salillas & Wicha, 2012). In contrast, there was no difference in magnitude for the congruency effect between LA + and LA − for the instructors teaching in their LA − (see Fig. 2). That is, either their LA − congruency effect became larger to match the LA + congruency effect (as a result of their experience with LA −) or the advantage they otherwise would have for LA + was lost from disuse (due to inexperience with LA + over time). Given that noninstructors possess the LA + advantage despite similar amounts of the lack of use, it seems more probable that the LA − congruency effect grew more robust in this population.³

f07-02-9780128125748 — Fig. 2 The effect of congruency is plotted in two groups of bilingual teachers performing a simple multiplication verification task in their LA + (left) and LA − (right). Although a robust congruency effect is present in all cases, a difference in the size of the congruency effect for performing the task in LA + compared with LA − was only observed in the teaching group who continued to use their LA + in the classroom (top chart). In the LA − teaching group (bottom chart), there was no difference in the size of the congruency effect as a function of language used in the task (i.e., no LA + advantage).

Another finding from this work was that the timing of the effect itself differed across languages based on experience, such that the effect peaked earlier in the teaching language than the nonteaching language regardless of what their language of learning had been. Thus, long-term experience instructing in a language not only benefited the strength of arithmetic fact representations in memory for that language (leading to a larger congruency effect) but also sped (facilitated) access to math facts in that language. That is, when participants had experience teaching in their LA −, the LA − became at least as efficient as the LA + for processing simple multiplication problems. This experience-driven explanation of arithmetic processing is more consistent with the ECM than the TCM, which (in its various iterations) has tended to be a more static model of arithmetic fact representations in the mind and brain (i.e., language of learning and experience seem not to be treated as independent properties of the system).

As alluded to in the introduction of ERP work with arithmetic, sometimes there are additional effects following this initial effect of congruency. Indeed, this was the case in the teacher study: there was a later positivity for incorrect solutions, selective to people who were instructing arithmetic in their LA −. Later positivities of this kind are often thought to reflect more explicit and controlled analyses of the presented item—which might be expected for a population who has needed to rely on self-monitoring strategies for success in teaching in a less dominant language. We include this result here, in part, as a demonstration of the value and power of ERPs for exploring the multidimensional nature of assessing the solution to a problem. Whereas behavioral results can tell us critical end-state outcomes, this dynamic view into the cognitive system allows us further insight into the complex processes unfolding as the answers are evaluated prior to a response being executed.

In general, our study of bilingual teachers provided the first evidence that language experience can affect arithmetic fact processing in the adult bilingual, suggesting that an adult bilingual's ability to perform arithmetic not only is influenced by early language experience (i.e., effects of LA + are long-lasting) but also remains sensitive to language use through adulthood (i.e., the effect of congruency was earlier in the teaching language regardless of the language in which arithmetic had originally been learned). These results support a view wherein bilingual arithmetic facts can be reorganized and strengthened in either language depending on how individuals use their languages at any point in time.

Using ERPs to Investigate Monolingual and Bilingual Children's Processing of Multiplication Problems

These findings in young adult bilinguals, when the encoding of multiplication facts is well established, have shifted our attention to the early learning of math facts in young children. Even with monolingual children, there has been relatively little brain imaging research conducted on this topic in the most relevant age group—namely, from third- to fifth-grade elementary school children. The few extant studies are not longitudinal and tend to be in older samples (Moore, Drollette, Scudder, Bharij, & Hillman, 2014; Prieto-Corona et al., 2010), leaving basic questions—even in monolingual samples—open to future study. Fortunately, we know that N400 effects emerge and stabilize even in quite young children (although Friedrich & Friederici, 2005, 2006 directly compare adults and children and suggest there are subtle but important underlying differences; for review, see Kutas & Federmeier, 2011). Thus, to the extent that arithmetic congruency effects are indexed by the N400 component, it should be possible to conduct these types of studies in children and obtain reasonably interpretable effects.

To address the many gaps in this literature, our lab has recently started an ERP study of multiplication verification in third- through fifth-grade children. Our goal is to establish the electrophysiological signatures of the developmental transition as children grow from novice to expert multipliers and, in turn, move from calculation to verbal memory retrieval. Our study uses a modified version of the multiplication verification task from Salillas and Wicha (2012).

Prior to ERP recording, the children begin by performing a series of cognitive tasks that measure their language fluency, working memory, math fluency, and other relevant skills. The children then perform an Arabic digit version of the verification task (e.g., 3…2…6 vs 3…2…7) while we record their ongoing brain waves. The off-line measures can then be used to estimate the impact of various factors on task performance and, most critically for us, brain activity. Fig. 3 shows preliminary ERP data (n = 49, collapsed across age and language groups;⁴ see below) measured from the onset of the Arabic digit solution. A robust congruency effect is clearly visible, with a more negative-going response for incorrect compared with correct solutions, which is consistent with the typical pattern for an N400 effect. However, although the direction of this effect is the same across adults and children (i.e., incorrect solutions elicit a more negative-going ERP than correct solutions), the precise morphology differs between populations (as in Prieto-Corona et al., 2010).

f07-03-9780128125748 — Fig. 3 Grand average ERPs for 49 children (ranging from third to fifth grades) time-locked to the onset of correct (solid black) and incorrect (dashed black) solutions during the multiplication verification task. The ERPs measured at the vertex electrode are shown. A robust N400 congruency effect is observed between 300 and 600 ms—this effect occurs later than the same effect in young adults (e.g., Salillas & Wicha, 2012). In addition, there is no noticeable target P300 response for the correct trials, as is observed in young adults (see Fig. 1).

There are two noticeable differences in the brain response in children (Fig. 3) compared with young adults (Fig. 1), both of which seem likely to reflect key developmental changes. First, the N400-like effect occurs later in time for the children than for adults, starting about 100 ms later (Fig. 3) and lasting longer than the typical 200 ms effect in adults (Fig. 1). It is known that age and language proficiency are two factors that can affect the timing of the N400 (Kutas & Federmeier, 2011; Moreno & Kutas, 2005), so this developmental shift is consistent with normal and expected age-related processing differences. Moreover, this result can be taken as an indication that access to this particular subset of semantic memory also becomes more efficient with age and experience, as indexed by the speed with which the brain attempts to extract meaning for the solution to the problems.

A second difference between children and young adults seems to be selective to the processing and recognition of the congruent trials (i.e., the brain response to correct solutions). In the adult data (Fig. 1), the effect of congruency appears to be driven not only by a larger negativity for the incorrect trials but also by another effect overlapping in the same time window (see Jasinski & Coch, 2012, for an expanded discussion of component identification issues in this literature). Most notably, in addition to the co-occurring N400-like effect, there appears to be a classic target P300 response to the correct solutions, specifically when they are presented in the Arabic digit format (bottom plot, Fig. 1).

The target P300 is a well-known ERP component that is most often reported after detection of a target stimulus (i.e., the word “apple” in a word list if the goal is to press a button in response to food items). In this case, the P300 is largest to the correct solution to the provided multiplication problem, which would be a target item based on the task (i.e., finding the correct answers to math problems). This large P300 is clearly visible in the ERP for correct solutions presented in the Arabic digit format (the visually/perceptually expected item). In contrast, the response to number words (middle plot, Fig. 1) is more unambiguously a traditional N400 effect with little evidence for a target P300 response for correct number words. These effects reflect different cognitive processes—whereas the N400 is linked to the obligatory processing of modality-independent meaning (and is largest/most negative for contextually unexpected items), the P300 is linked to stimulus classification and updating of context (and is largest/most positive for task-relevant target items) (Donchin & Coles, 1988; see Polich, 2011, for review).

Intriguingly, in our preliminary child data displayed in Fig. 3, there is no noticeable P300 in response to the correct solutions. Instead, children's responses to congruent digits solutions look more like the adult response to written number words (i.e., the congruency effect itself is primarily an N400 effect with no additional contribution from a P300; see middle plot of Fig. 1). We also see morphological differences in the children across grades (i.e., children with different levels of experience with exact arithmetic)—although, interestingly, even our fifth-grade sample, which we consider successful learners of multiplication, does not show this adult-like response. This suggests that the P300 pattern only emerges after many years of experience with arithmetic facts.

It is possible that children, who have less experience with detecting correct solutions, read multiplication problems for comprehension much like adults read sentences. That is, they process the correct digit like it was an expected word or conceptually congruent picture rather than an item to be detected and categorized. Thus, when encountering either correct or incorrect solutions, they might be attempting to extract the meaning in both cases, resulting in a solitary N400 effect. In contrast, by the time a person reaches adulthood, the solution to a simple multiplication problem is so easily retrievable that digit answers are effectively processed as target items, eliciting a robust P300 response once the end goal (the correct solution) is recognized and categorized. Incorrect solutions do not elicit this target detection response in adults and are instead processed more like unexpected words in a sentence or nontarget items in a word list (which also only produce an N400 component, c.f., Jasinski & Coch, 2012). In other words, adults may come to see arithmetic problems as a prompt to search for the correct answer at the end, whereas children are just trying to build a coherent meaningful representation as the problem unfolds.

Tentatively, then, children's ERPs to the solution appear to differ from adult's both in efficiency (slower latency of the congruency effect in children) and mechanistically (the lack of a P300 in children likely reflects an inability to process correct answers as target items). In addition, like other researchers, in our preliminary overall group average, we do not find the later positivity for incorrect answers that is often present in adults (see Fig. 1, bottom panel, and Moore et al., 2014; Prieto-Corona et al., 2010). This later positivity tends to be smaller than the earlier congruency effect and has more sensitivity to the particulars of task demands, so the possible reasons for its absence in children are likely to be even more complex to account for than P300/N400 differences between children and adults.

What we believe that can be largely taken from the ERP research to date is that the processes that children undergo when evaluating the solutions for multiplication problems are substantially different from the processes adults undergo. Notably, the nuances of these differences are not clearly predicted by any of the existing models of arithmetic, which were based primarily on behavioral responses that reflect the summation of multiple underlying cognitive processes. In contrast, ERPs are a temporally sensitive measure able to expose differences in these underlying cognitive processes with more specificity. Future models of arithmetic will need to take into account these findings to better explain not only the adult state of the system but also how the system arrives at that end state.

Conclusions and Future Directions

In addition to better characterizing the developmental trajectory that leads to adult-like responses, we also hope to construct a better understanding of the critical link between language experience, arithmetic ability itself, and the congruency effect(s). Thus, in addition to elucidating the developmental trajectory for multiplication fact retrieval when using the Arabic digit format, we are also simultaneously investigating the effect of language use and experience on the morphology of the congruency effect (i.e., its size, timing, distributional properties, and shape). To this end, the population we study includes children who are early Spanish-English bilinguals in addition to monolingual English-speaking children. Given the evidence for effects of bilingualism on other aspects of cognition, discussed above, our ultimate goal is to determine if being bilingual can impact multiplication fact retrieval, even in the digit format. Critical for this comparison, both linguistic populations will have learned—or are still learning—multiplication in English (i.e., all have English as their LA +).5

We have already described the ERP responses to problems presented in the Arabic digit format for our entire child sample to date (i.e., Fig. 3). What we have not revealed is that the children's brain responses are also being recorded when they perform the same verification task with the operands presented as number words instead of digits. For this task to better accommodate children, we adapted the number-word version of the multiplication verification task from Salillas and Wicha (2012) in important ways. Critically, when number words are used in the paradigm, they are presented as spoken (auditory) words, rather than written number words as was done with the adult bilinguals. Spoken number words are more naturalistic, given that multiplication problems are rarely presented as fully spelled out written words in an educational setting. And, relating back to the models of arithmetic, the verbal code in the TCM is discussed as a spoken-language memory trace, making spoken number words a more appropriate test for this model.

Additionally, using spoken number words is critical for this child population given that children can vary widely in their reading skills, especially at the younger end of our sample. Moreover, bilinguals tend not to have equivalent reading experience or ability in their two languages. Therefore, using spoken number words for the operands in each problem allows us to force the child to process the math problem in that language and at the same time eliminates a potential confound of processing differences based on reading ability (or at least reduces its influence), both across individual children and across languages within a bilingual child. We note that this is a more direct approach to studying the effect of language on processing arithmetic than is often used in the literature, where the language manipulation is done by providing the instructions in a particular language or by teaching a novel arithmetic skill in only one language (e.g., Spelke & Tsivkin, 2001; Venkatraman et al., 2006; c.f., Frenck-Mestre & Vaid, 1993, whose work does use number words but still has the potential confound of reading ability differences). It seems to us that our manipulation is closer to what is truly intended when we think about participants “performing math” in a specific language—each problem is deliberately presented in a particular language of interest, hopefully reducing and/or eliminating the potential for use of the alternative language.

To our knowledge, there have been no prior attempts to use auditory number words as operand stimuli in such a paradigm with ERPs, but auditory presentation of sentence stimuli tends to result in similar (if not identical) findings in the broader language literature (e.g., Federmeier, McLennan, Ochoa, & Kutas, 2002; Kutas, Neville, & Holcomb, 1987), so it seemed plausible that the same would be true for our arithmetic “sentences.”

The second modification from the adult study is that regardless of the format of the operands, which appear in the Arabic digit, LA + or LA − formats across alternate blocks, the solution is always presented as an Arabic digit. This is critical because the brain response to the solution will reflect processing specific to that format, as previously described and as depicted above in Fig. 1. Using Arabic digits for the solution across all conditions thus allows us to compare the brain response with physically identical stimuli (i.e., the solution) while still measuring the effect of retrieving (or otherwise processing) the problems in the specific format and/or language of the preceding operands. Thus far, we have observed a robust effect of congruency in both bilinguals and monolinguals in this novel cross modal arithmetic verification task that will lend itself to investigations of the influences of both math and linguistic experience on the developmental trajectory discussed above.

Due to the number of conditions, the number of potential subgroups of populations, and the developmental approach we are taking, this research opens numerous questions, the first of which is when and how being bilingual might affect how children process the provided solutions. To address this matter, our design allows us to compare effects of congruency in the tasks that both bilinguals and monolinguals perform (i.e., using data from both Arabic numeral and English, LA +, and number-word tasks). Given its emphasis on the role of language in math fact processing, the TCM would predict that there should be no differences between monolinguals and bilinguals when performing in English, the LA + for both groups. In contrast, a bilingual model that takes into account that the amount of exposure in a given format will determine the strength of access to that format, as in the ECM, might predict differences between monolinguals and bilinguals because bilinguals may have less exposure to arithmetic in any one language compared with monolinguals. That is, of their total time spent gaining experience with arithmetic, monolinguals typically spend all of it in English-only contexts, whereas there is a possibility that bilinguals could spend some of that time using arithmetic in their other language, by default having relatively less English exposure than monolinguals (see Gollan, Montoya, Cera, & Sandoval, 2008, for a similar argument regarding bilingual lexical access). Or, perhaps, given the evidence for differences between monolinguals and bilinguals in other domains (e.g., vocabulary acquisition, discussed at the beginning of this chapter), it is possible that bilinguals might outperform or underperform monolinguals in different ways and at different points in development that are not predicted by either numerical processing model.

We are also interested in delineating the developmental trajectory for arithmetic processing in both languages used by bilinguals. We thus plan to compare the effect of congruency when the operands are presented in the LA + (English) and the LA − (Spanish) as bilingual children gain more expertise in multiplication (from third to fifth grades). The primary evidence of interest will come from tracking the relative size of the congruency effect in the LA − as they gain experience multiplying and retrieving answers in the context of LA +. According to the TCM, the LA + might establish the sole representation for arithmetic facts during learning, leaving the LA − representations to emerge only via translation, perhaps later in development (thus predicting a larger congruency effect in LA + than LA −, especially in the least experienced cohort). However, if bilingual children process math facts the same way other information is processed in verbal memory, then it is possible that math fact representations are established in both languages during learning, even when only the LA + is explicitly used for learning the facts (predicting fairly similar congruency effects in both languages). Given that this work is in its nascent state, there are many other questions that we expect will emerge as we begin to understand the basic mechanisms of early learning and use of multiplication facts in the bilingual brain.

It seems to us that most theories of numerical cognition more carefully define how number-related information is structured in memory than the temporal structure through which this information is accessed (e.g., Campbell & Clark, 1992; Dehaene et al., 2003). By using ERP measures in combination with RT data, we are able to gain insight into the dynamics that lead up to (and follow) the response itself, which we think generates many new constraints to be considered within any model of math cognition. How do RTs and other behavioral outcomes relate to the latency and magnitude of the congruency effect, and does this differ for adults and children? Do children with superior behavioral performance show a target P300 response to correct answers, similar to adults, or does the P300 come later as part of some other developmental change? Why do adults have a secondary stage of processing after the congruency effect, and under what conditions does it emerge in children? And how do these various effects interact, or not, with language experience and bilingualism? At present, there seem to be more questions than answers in this domain, especially compared with the traditional language literature with ERPs. Although this state of affairs can make it challenging for interpreting the various effects, on the whole, we prefer to think that this gap provides current and future researchers a unique opportunity to make substantial contributions to this area in a comparatively brief amount of time.