Embodiment and second-language: Automatic activation of motor responses during processing spatially associated L2 words and emotion L2 words in a vertical Stroop paradigm

Brain & Language 132 (2014) 14–21

Contents lists available at ScienceDirect

Brain & Language journal homepage: www.elsevier.com/locate/b&l

Embodiment and second-language: Automatic activation of motor responses during processing spatially associated L2 words and emotion L2 words in a vertical Stroop paradigm Carolin Dudschig ⇑, Irmgard de la Vega, Barbara Kaup University of Tübingen, Germany

a r t i c l e

i n f o

Article history: Accepted 9 February 2014

Keywords: Embodiment Bilingualism Second-language Motor responses Spatial processing Emotion

a b s t r a c t Converging evidence suggests that understanding our first-language (L1) results in reactivation of experiential sensorimotor traces in the brain. Surprisingly, little is known regarding the involvement of these processes during second-language (L2) processing. Participants saw L1 or L2 words referring to entities with a typical location (e.g., star, mole) (Experiment 1 & 2) or to an emotion (e.g., happy, sad) (Experiment 3). Participants responded to the words’ ink color with an upward or downward arm movement. Despite word meaning being fully task-irrelevant, L2 automatically activated motor responses similar to L1 even when L2 was acquired rather late in life (age >11). Specifically, words such as star facilitated upward, and words such as root facilitated downward responses. Additionally, words referring to positive emotions facilitated upward, and words referring to negative emotions facilitated downward responses. In summary our study suggests that reactivation of experiential traces is not limited to L1 processing. Ó 2014 Elsevier Inc. All rights reserved.

1. Introduction ‘‘The limits of my language mean the limits of my thoughts’’ (Wittgenstein, 1922). Learning a second-language (L2) in school demands time and dedication but also opens the doors to new cultures and experiences. To what extent, however, do we become familiar with our second language? Recently, increasing evidence suggested that first-language (L1) processing is closely linked to spatial cognition, motor- and perceptual processing. For example, L1 can automatically activate motor responses compatible to the linguistically described event (e.g., Glenberg & Kaschak, 2002). Also, when reading words such as kick, specific motor cortex areas become activated that are also involved in performing the according action (e.g., Hauk, Johnsrude, & Pulvermüller, 2004). Additionally, even single words can trigger action-affordances (Bub, Masson, & Cree, 2008). These findings regarding a relationship between language, action and perception are typically explained within the grounded models of language understanding, suggesting that language comprehension relies on reactivation of sensorimotor experiences (Barsalou, 1999; Glenberg & Gallese, 2012; Glenberg & Kaschak, 2002; Zwaan & Madden, 2005). However,

⇑ Corresponding author. Address: University of Tübingen, Fachbereich Psychologie, Schleichstr. 4, 72076 Tübingen, Germany. Fax: +49 (0)7071 29 3363. E-mail address: [email protected] (C. Dudschig). http://dx.doi.org/10.1016/j.bandl.2014.02.002 0093-934X/Ó 2014 Elsevier Inc. All rights reserved.

what role does sensorimotor information play during L2 comprehension? When learning L1 we often hear a word in situations where we also experience its referent in the real world (Zwaan & Madden, 2005). For example, when encountering the word airplane as a child, this typically occurs in situations where someone points upward to the sky, with the child looking upward to see an airplane. According to the grounded language processing models, these manifold sensory experiences become reactivated when processing the word airplane and build the basis of understanding (Barsalou, 1999; Glenberg & Kaschak, 2002; Richter, Zwaan, & Hoever, 2009; Zwaan & Madden, 2005). Specifically, it is suggested that the neural sensorimotor activation during language understanding is similar to the neural activation when actually seeing the described entity or performing the described action (e.g., Lyons, Mattarella-Micke, Cieslak, Nusbaum, Small, & Beilock, 2010; Scorolli & Borghi, 2007, Pulvermüller, Shtyrov, & Ilmoniemi, 2005). Evidence for the involvement of sensorimotor processes during language comprehension is typically drawn from studies investigating the effect of language on subsequent perceptual or motor processes. For example, Estes, Verges, & Barsalou (2008) showed that centrally presented words referring to entities with a typical location in the world (e.g., sun, shoe) influence subsequent visual target processing in upper or lower screen locations. Similar results have been reported for studies implementing verbs (Verges & Duffy, 2009) and sentences (Bergen, Lindsay, Matlock, & Narayanan,

C. Dudschig et al. / Brain & Language 132 (2014) 14–21

2007). In addition to the influence of direction-associated language on visual processing (Bergen et al., 2007; Dudschig, Lachmair, de la Vega, De Filippis, & Kaup, 2012b; Gozli, Chasteen, & Pratt, 2013), language also interacts with motor responses (Dudschig, de la Vega, De Filippis, & Kaup, submitted for publication; Lachmair, Dudschig, De Filippis, de la Vega, & Kaup, 2011; Thornton, Loetscher, Yates, & Nicholls, 2012). For example, motor responses are faster if the response direction matches the typical location of the word’s referent in the real world (e.g., upward arm movements are faster following words such as sun). Similar language-action compatibility effects have been reported for verbs (e.g., fall, rise) (Dudschig, Lachmair, de la Vega, De Filippis, & Kaup, 2012a) and when measuring eye-movements (e.g., Dudschig, Souman, Lachmair, de la Vega, & Kaup, 2013). Beyond the influence of directionassociated words on motor or perceptual processing, other word categories, such as action words, are also directly linked to motor processes (e.g., Boulenger, Hauk, & Pulvermüller, 2009; Marino, Gough, Gallese, Riggio, & Buccino, 2013; Zwaan & Taylor, 2006). According to the embodied cognition framework of language comprehension, abstract language referring to things or situations we cannot directly experience also becomes related to sensory experiences (Glenberg, Sato, Cattaneo, Riggio, Palumbo, & Buccino, 2008; Lakoff & Johnson, 1980; Meier & Robinson, 2004; Santiago, Ouellet, Román, & Valenzuela, 2012). For example, language referring to something positive (negative) has been suggested to activate upper (lower) visual space. Taken together, according to the grounded models of language comprehension, language understanding is based upon modal experiences, and is not separate from our sensory system (e.g., Barsalou, 1999). Previous research investigating L2 understanding has primarily focused on the degree of automaticity to which L2 is accessed. These studies have focused on two aspects of language processing. First, it was investigated how emotional content becomes activated during L2 processing. Some findings suggest that emotional content (negative and taboo words) similarly recruits selective attention for L1 and L2 and as a result, slows subsequent behavioral responses (e.g., Eilola, Havelka, & Sharma, 2007; Sutton, Altarriba, Gianico, & Basnight-Brown, 2007). However, other studies show that negative words elicit greater autonomic arousal, as measured by skin conductance, in L1 compared to L2 (Harris, Aycicegi, & Gleason, 2003), especially when L2 was acquired rather late in life (after the age of 12) (see Pavlenko, 2012). Another paradigm used to investigate the automaticity of meaning access during L2 processing is the Stroop paradigm (Stroop, 1935). In the Stroop color-naming paradigm, task-irrelevant words are presented in a color whereby the color determines the response (for reviews see, Lu & Proctor, 1995; MacLeod, 1991). For example, participants see the word red that is printed in blue, and have to respond by saying ‘‘blue’’ and ignore the word meaning. Responses are faster if word meaning and response color match (e.g., the word red printed in red color). This finding is typically attributed to the automatic access of word meaning that interferes with the less dominant color naming process, whereby opposing color information results in conflicts and selective attention is required in order to overcome these conflicts (e.g., Botvinick, Braver, Barch, Carter, & Cohen, 2001; MacLeod, 1991). In L2 studies, a within-language or between-language version of the Stroop paradigm is typically implemented (e.g., Naylor, Stanley, & Wicha, 2012). In the between-language version, participants are presented with words in one language, but have to respond in their other language (e.g., by speaking out loud the words’ ink color). In the within-language version, the language of the task-irrelevant word matches the response language. Both the within and the between-language paradigms show Stroop interference effects with the within-language paradigm showing significantly larger Stroop interferences than the between-language paradigm (Dyer, 1971;

15

Francis, 1999), whereby the size of the interference effects depends on language proficiency (e.g., Mägiste, 1984). The dominance of within-language Stroop interference is attributed to response-set competition on the conceptual and lexical level being present in the within-language Stroop paradigm, whereas in the betweenlanguage Stroop paradigm, only conceptual overlap can cause response conflict (Goldfarb & Tzelgov, 2007; Roelofs, 2003). Taken together, these Stroop interference effects suggest that we automatically access word meaning when seeing L2 words to a level that subsequent color-naming responses can be influenced. However, this leaves open whether L2 processing is related to sensorimotor processes or not. Specifically, does automatic access to L2 word meaning result in action-compatibility effects to a similar degree than words from L1 (e.g., Glenberg & Kaschak, 2002; Lachmair et al., 2011; Thornton et al., 2012)? Typically we learn L2 in school and school-based language learning is rather different from L1 learning (e.g., Lave, 1996). L1 learning evolves over many years, on an everyday basis and in a very interactive manner with many people, and in various settings. Especially during childhood we often encounter language in situations where we also perceive the events, entities or feeling described. Moreover, these language percepts are typically combined with specific gestures, eye-movements and physical orientations towards the described entity (Engelen, Bouwmeester, de Bruin, & Zwaan, 2011; Glenberg & Gallese, 2012). In contrast, L2 learning in school typically takes place in a very specific and limited setting, whereby interactions with other people and physical experiences are less dominant during the learning phase. Indeed, many L2-acquisition researchers ‘‘view the object of inquiry as in large part an internal, mental process: the acquisition of new (linguistic) knowledge’’ (Long, 1997, p. 319). In such a view of L2 learning there is a ‘‘basic division between mind and world’’ (Atkinson, Nishino, Churchill & Okada, 2007, p. 170). Thus, if not only L2 acquisition but also L2 understanding is functionally different from L1, sensorimotor information might not be activated during L2 comprehension. Comparing L1 and L2 according to their association with sensorimotor processes is particularly interesting, as currently, it is controversally discussed whether L2 processing only semantically compares to L1 processing or also regarding its grounding in emotion and experience (Keysar, Hayakawa, & An, 2012; Pavlenko, 2012). The current study investigates basic associations between L2 and the sensorimotor system. We implement a paradigm that has previously been used to investigate whether L1 automatically activates motor responses (Lachmair et al., 2011; Thornton et al., 2012). In this paradigm, participants are presented with colored words and are required to respond to the color with an upward or downward arm movement while ignoring the meaning of the word. In Experiment 1 and 2 of the current study the L2 words referred to entities in the world with a typical location (e.g., bird vs. shoe). In Experiment 3 of the current study the association between ‘‘positive is up’’ and ‘‘negative is down’’ is investigated for L2 processing (Brookshire, Casasanto, & Ivry, 2010; Dudschig, de la Vega, & Kaup, submitted for publication; Meier & Robinson, 2004; Santiago et al., 2012). If the grounded models of language understanding are a general approach towards all types of language understanding, we predict that during L2 processing, sensorimotor information becomes similarly activated as during L1 processing. Evidence towards the involvement of sensorimotor processes during L2 understanding would largely increase the impact of the embodied models of language understanding by suggesting that interconnections to sensorimotor processing are not limited to L1 comprehension. In contrast, if L2 processing is based on non-modal representations of meaning, then L2 should not automatically activate sensorimotor processes in the same way as L1, and no sensorimotor associations should be observed during L2 comprehension. Indeed, it is possible that the interconnections between language

16

C. Dudschig et al. / Brain & Language 132 (2014) 14–21

and sensorimotor processing in L1 comprehension are due to associations we build during language acquisition in childhood, but that these do not play a central role for language comprehension in general, but are rather a by-product of L1 comprehension.

2. Experiments 2.1. Experiment 1 2.1.1. Method 2.1.1.1. Participants. Twenty native German speakers took part in this experiment (8 male, Mage = 24.70, SDage = 4.46). Participants started L2 learning in the 5th or 7th grade of high school (age between eleven and thirteen), and had never lived in an English speaking country at the time of testing.

2.1.1.2. Stimuli and apparatus. Words consisted of 40 German words and 40 English words with the same meaning. The words were rated according to their typical location in the world by two independent samples of German native speakers that did not take part in the actual experiment. English and German words did not differ according to word length, t(78) = 0.91, p = .37. The response apparatus is displayed in Fig. 1. A keyboard with a selfconstructed overlay was mounted in the vertical plane in front of the participants.

2.1.1.3. Procedure and design. Experimental procedure was controlled using MATLAB, Psychtoolbox 3.0 (Kleiner, Brainard, Pelli, Ingling, Murray, & Broussard, 2007). Stimuli were displayed on a 1700 CRT monitor screen (resolution 1280  960). Each trial began with a centrally presented fixation cross for 750 ms. Participants had to hold down both central keys of the keyboard with their left and right hand respectively (see Fig. 1). The fixation cross was replaced by a centrally presented word in one of four colors (blue RGB [0 0 255], red RGB [255 0 0], green RGB [0 192 0], orange RGB [255 128 0]); L1 and L2 words were randomly intermixed. Response direction was determined by color. Across participants the assignment of color to response direction was counterbalanced, as was the assignment of response hand to response direction. For upward responses the upper middle button had to be released and the top button had to be pressed. For downward responses the lower middle key had to be released and the very low key had to be pressed. The non-responding hand was kept stationary on the keyboard. If no response occurred within 1500 ms the participants received feedback that their response was ‘‘Too slow’’ and the next trial started automatically after 1500 ms. 16 practice trials were conducted before the experiment started. The experiment was divided into 8 blocks with 80 trials in each block.

2.1.2. Results and discussion The results are displayed in Fig. 2. Erroneous trials and reaction times (RTs) faster than 100 ms were excluded from analysis, reducing the data set by 4.6%. RTs were analysed with an ANOVA with the factors language, word-direction and response-direction. There was no main effect of response-direction, F(1, 19) = 0, MSE = 2935.6, p = .99, word-direction, F(1, 19) = 1.41, MSE = 126.2, p = .30, nor of language, F(1, 19) = 1.20, MSE = 160.2, p = .29. Most importantly there was an interaction between word-direction and response-direction, F(1, 19) = 14.62, MSE = 471, p < .01. There was no interaction between response-direction and language, F(1, 19) = 1.44, MSE = 150.3, p = .24, however word-direction interacted with language, F(1, 19) = 4.39, MSE = 223.5, p < .05. Additionally, there was a trend for a three way interaction between language, word-direction and response-direction, F(1, 19) = 3.18, MSE = 198.9, p = .09. Separate analysis of the German and English words showed that this was due to the interaction between word-direction and response-direction being slightly stronger for German, F(1, 19) = 14.10, MSE = 415, p < .001, than for English words, F(1, 19) = 6.55, MSE = 255.4, p < .05. Most interestingly, the interaction between word-direction and response-direction also showed when analyzing English words separately (see above), suggesting that these words activate spatially directed motor responses, similarly to L1 German words. In order to investigate whether the compatibility effect in L2 is of a different nature than the compatibility effect observed in L1, an analysis of the temporal characteristics of these compatibility effects was conducted. Temporal characteristics can provide important insights into the mechanisms underlying the compatibility effect in L1 and L2. If the temporal characteristics of the effects in L1 and L2 differ significantly, this would suggest that different mechanisms underlie these effects. For example, if the effect in L1 originates from very fast RTs, and in contrast the effect in L2 originates in rather long RTs, this might suggest that additional mechanisms are involved during L2 processing. Most likely such a pattern would suggest that the L2 words first need to be translated into L1 and only after this translation phase the described compatibility effect can be observed. In order to analyze the temporal characteristics of the compatibility effect in L1 and L2 we first split the RT distributions for compatible and incompatible trials for each participant separately into deciles (10 bins of RTs increasing from the fastest RT bin (1st decile) to the slowest RT bin (10th decile)). Subsequently we performed an ANOVA with the factors decile, compatibility and language. The ANOVA showed the trivial main effect of decile, F(9, 171) = 294.7, MSE = 4213, p < .001, reflecting the way the deciles are constructed (RTs increase with increasing decile). The ANOVA confirmed the compatibility effect, F(1, 19) = 15.13, MSE = 2282, p < .001. Additionally, there was an interaction between decile and compatibility, F(9, 171) = 9.35, MSE = 250.2, p < .001. Importantly, the three-way interaction between language, compatibility and decile was not significant, F < 1, suggesting that the language (L1 vs. L2) did not modify the timing when the compatibility effects occurred. As Fig. 2 shows, the compatibility effects emerged over time and increased with the increase of RT, and importantly this was the case for both L1 and L2 words.

2.2. Experiment 2

Fig. 1. Illustration of the experimental setup.

In Experiment 1 the presentation of L1 and L2 words was randomly intermixed. Thus, the pure presentation of the L1 words (e.g., Vogel) might have facilitated access to the meaning of the L2 words (e.g., bird). In Experiment 2 only L2 words were presented. If indeed L2 words automatically activate motor associations, we expect that L2 words results in motor associations

17

C. Dudschig et al. / Brain & Language 132 (2014) 14–21

First−Language

Second−Language

600

Down−Response 580

Up−Response

RT (ms)

560 540 520 500

Down−Words

Up−Words

Down−Words

Up−Words

850 800

Compatible

750

Incompatible

RT (ms)

700 650 600 550 500 450 400 1

2

3

4

5

6

7

8

9

10

1

2

3

4

5

6

7

8

9

10

RT Bin Fig. 2. Top row: Mean RTs from Experiment 1 for L1 (left panel) and L2 (right panel) separately for each response-direction and word-direction. Error bars represent confidence intervals for within-subject designs according to Loftus and Masson (1994). Bottom row: Mean RTs according to decile (increasing from 1st to 10th decile) for the compatible and incompatible condition for L1 (left panel) and L2 (right panel).

similar as in Experiment 1 (even if not intermixed with their L1 counterparts).

2.2.1. Method 2.2.1.1. Participants. Twenty native German speakers who were right-handed took part in this experiment (7 male, Mage = 24.3, SDage = 3.60). Participants started L2 learning in the 5th or 7th grade of high school (age between eleven and thirteen), and had never lived in an English speaking country at the time of testing.

2.2.1.2. Stimuli. The identical L2 words as in Experiment 1 were used. 2.2.1.3. Procedure and design. The experiment was identical to Experiment 1 with the difference that only the L2 words (English) were presented, resulting in 8 blocks with 40 trials in each block. 2.2.2. Results and discussion RTs were analyzed as in Experiment 1. The results are displayed in Fig. 3. Again erroneous trials and RTs faster than 100 ms were excluded from data analysis, reducing the data set by 1.6%. An

600

850 Down−Response

580

800

Up−Response

Incompatible

700

RT (ms)

560

RT (ms)

Compatible

750

540 520

650 600 550 500 450

500

400 Down−Words

Up−Words

1

2

3

4

5

6

7

8

9

10

RT Bin Fig. 3. Left panel: Mean RTs of Experiment 2 separately for each word-direction and response-direction. Error bars represent confidence intervals for within-subject designs according to Loftus and Masson (1994). Right panel: Mean RTs according to decile (increasing from 1st to 10th decile) for the compatible and incompatible condition.

18

C. Dudschig et al. / Brain & Language 132 (2014) 14–21

ANOVA with the factors word-direction and response-direction was performed. There were no main effects of response-direction, F(1, 19) = 1.49, MSE = 683.5, p = .24, nor of word-direction, F(1, 19) = 0.55, MSE = 167.78, p = .47. Importantly, there was a significant interaction between word-direction and response-direction, F(1, 19) = 5.60, MSE = 250.6, p < .05. This interaction was due to upwards responses being faster following up-words (511 ms) than down-words (522 ms), and downward responses being faster following down-words (521 ms) than up-words (527 ms). Again we performed an analysis of the temporal characteristics of the described compatibility effect in an additional ANOVA with the factor decile. First, this ANOVA showed the trivial main effect of decile, F(9, 171) = 313.3, MSE = 1454, p < .001. Again, the effect of compatibility was significant, F(1, 19) = 5.89, MSE = 1197, p < .05. Additionally, the compatibility effect interacted with the factor decile, F(9, 171) = 6.84, MSE = 193, p < .001. In line with the compatibility effect in L1 (see Experiment 1 and Lachmair et al., 2011) this interaction was due to the fact that the compatibility effect in L2 increases with increasing RTs (Fig. 3). 2.3. Experiment 3 Experiment 1 and 2 investigated the association between implicit location words (e.g., bird, shoe) and motor responses in the vertical space. In the grounded cognition framework of language understanding, it was suggested that also processing more abstract words referring to positive or negative emotional states become associated with sensorimotor processing (e.g., Meier & Robinson, 2004). Specifically, positive words become associated with upper and negative words with lower space. These findings have been attributed to the association between positive feelings (e.g., pride) and upright bodily postures, and negative feelings (e.g., depression) and slouched bodily postures (Stepper & Strack, 1993). If L2 valence word processing is similarly connected to embodied representations as L1, these bodily associations should also become activated during L2 processing. Specifically, words referring to positive states (e.g., happy) should facilitate upward movements and words referring to negative states (e.g., sad) should facilitate downward arm movements. 2.3.1. Method 2.3.1.1. Participants. Twenty native German speakers took part in this experiment (2 male, Mage = 22.75, SDage = 2.93). Participants started second-language learning in 5th or 7th grade of high school

(age between eleven and thirteen), and had never lived in an English speaking country at the time of testing. 2.3.1.2. Stimuli. Twenty positive and negative L2 valence words were used (e.g., happy, joyful, sad, depressed). The emotion words were selected according to a recent study suggesting that only a very specific subset of emotion words results in automatic associations between emotional language and vertical space (Dudschig et al., submitted for publication). 2.3.1.3. Procedure and design. Identical to the previous experiments. Again 8 blocks with 40 trials in each block were conducted. 2.3.2. Results and discussion RTs were analyzed as in the previous experiments. The results are displayed in Fig. 4. Again erroneous trials and RTs faster than 100 ms were excluded from data analysis, reducing the data set by 3.1%. There was no main effect of response direction, F(1, 19) = 0.03, MSE = 1610.0, p = .87, nor word valence, F(1, 19) = 0.39, MSE = 228.2, p = .54. Importantly, there was a significant interaction between word direction and word valence, F(1, 19) = 5.74, MSE = 389.3, p < .05. This interaction was due to up-responses being faster for positive than for negative words, and down-responses being faster for negative than for positive words. Again we performed an additional ANOVA including the factor decile. This ANOVA showed the main effect of decile, F(9, 171) = 260.3, MSE = 2613, p < .001. The effect of compatibility was also significant, F(1, 19) = 5.55, MSE = 2056, p < .05. Interestingly, the compatibility effect did not interact with decile, F(9, 171) = 1.48, MSE = 421, p = .16, suggesting that the compatibility effect triggered by emotional words in L2 did not vary as much over time as the compatibility effect triggered by locational words (Experiment 1 and 2). To our knowledge, there are no studies investigating the temporal characteristics of the compatibility effect between emotional words and spatial processing in L1, leaving open whether this rather early onset of the compatibility effect generally occurs with emotional words. Indeed, it has been shown that emotional words independent of their polarity have a processing advantage over non-emotional words (Kousta, Vinson, & Vigliocco, 2009). Such a processing advantage might also result in the rather early onset of the interactions between the sensorimotor system and language processing. Most importantly, this early onset of the compatibility effect does speak against an additional timeconsuming translation phase (of L2 words to their L1 counterparts)

600

850 Down−Response

580

800 750

Incompatible

700

RT (ms)

560

RT (ms)

Compatible

Up−Response

540

520

650 600 550 500 450

500

400 Negative−Words

Positive−Words

1

2

3

4

5

6

7

8

9

10

RT Bin Fig. 4. Left panel: Mean RTs of Experiment 3 separately for each word-direction and response-direction. Error bars represent confidence intervals for within-subject designs according to Loftus and Masson (1994). Right panel: Mean RTs according to decile (increasing from 1st to 10th decile) for the compatible and incompatible condition.

C. Dudschig et al. / Brain & Language 132 (2014) 14–21

occurring before the compatibility effect can be observed in L2, and suggests that L2 words become rather directly connected with experiential traces.

3. General discussion Recently, increasing evidence suggested that L1 processing relies on embodied representations of meaning and is closely connected to motor and perceptual processing (e.g., Barsalou, 1999; Estes et al., 2008; Glenberg & Kaschak, 2002; Hauk et al., 2004; Lachmair et al., 2011; Thornton et al., 2012). However, it remained open whether L2 results in different meaning representations, or whether it is similarly related to sensorimotor representations as L1 comprehension. In the current study we replicated the automatic activation of spatially directed motor responses during L1 comprehension (Lachmair et al., 2011, Thornston et al., 2012) and extend these findings to L2 processing. Specifically, in Experiment 1 and 2, implicit location words (e.g., bird, shoe) interacted with motor responses in the vertical space even when presented in L2. Experiment 3 showed that L2 words referring to positive or negative emotional states (e.g., happy, sad) also automatically activate motor responses analogously to the compatibility effects reported for L1 comprehension (e.g., Meier & Robinson, 2004). None of the experiments required participants to actively read or evaluate word meaning, suggesting that L2-sensorimotor associations are automatic in nature and do not depend on deeper semantic task demands (cf. Bub et al., 2008). In our view the automatic association between L2 and sensorimotor processes is particularly interesting for several reasons. First, as described above, our study minimized the linguistic task demands, so that participants were neither forced to actively read or understand nor to translate the words. In contrast, participants simply responded to the words’ color. Nevertheless, we find interactions between L2 and the sensorimotor system. This experimental setup (if used in L1 experiments) typically is interpreted in favor of a strong and automated connection between the words and the sensorimotor system, thus our results suggest that L2 words are similarly interconnected with the sensorimotor system as are L1 words. Second, the words used in our study did not directly transfer location information, but location was rather an attribute of the situation, in which we typically experience the described entities and of bodily experiences, such as pointing upward or standing upright or slouched (e.g., Dudschig et al., submitted for publication; Lachmair et al., 2011). Third, learning and speaking L2 differs in many ways from L1. We learn L1 to perfection seemingly without any effort, whereas, even after years of studying L2 we often fail to reach the proficiency of L1, especially when we learn L2 after early childhood (e.g., Johnson & Newport, 1989). One major distinction between L1 and L2 is that we typically learn L1 in a very interactive manner (e.g., Lave, 1996) and that we often hear the words in situations where we also experience their referents in the real world. Thus, the association between a specific word and a motor response might only occur due to manifold co-occurrences between this specific word and a specific sensorimotor experience. Indeed, a study by Jared, Poh, and Paivio (2013) emphasized the importance of the co-occurrence of a word (e.g. postbox) and a specific sensory experience during the time a language is learned and spoken (e.g. seeing a postbox). The study investigated whether words learned in different countries and in different languages (e.g. the Mandarin and English words for postbox) are associated with different referential images of the same entity, for example, a postbox in China looks different from a postbox in Canada. In this study rather high proficient Mandarin–English bilingual students were tested, all of them were born in China (living there for at

19

least 9 years) and then studied in Canada. The participants saw pictures of entities (e.g., a Canadian postbox or a Chinese postbox) and had to name the entity in their L1 (Mandarin) or L2 (English). The results showed that pictures of entities are named faster if the picture matches our experiences with that particular language. For example, participants were faster naming a Chinese postbox in Mandarin and a Canadian postbox in English. The authors concluded that words become associated with specific referential images, indicating the importance of our experiences during language learning for the subsequent associations between the language and visual representation of the entity. This study suggests that for high proficient L1 and L2 speakers, the association between language and visual imagery is not taking place on a generalized conceptual level, but is coded dually with specific images being related to words in a specific language. Bergen, Lau, Narayan and Wheeler (2010) showed that visual pictures of arm and foot movements influence subsequent word processing (e.g. scratch) for L2 English speakers, also suggesting that L2 processing can become related to perceptual processing. However, these studies leave open how automated these associations between language and sensory processes are, and what role these sensorimotor representations play in the case of less proficient L2 processing. For example, if a Chinese person learns the English word for postbox without ever having lived outside China, does the L2 word still have any sensorimotor representations by a transfer from the L1 representation to the L2 representation? Or is the L2 word represented in a fully amodal manner? Our sample of participants learned L2 in a stereotypical schoolbased setup, where vocabularies are studied in a much less interactive and stimulating environment. Nevertheless, our results suggest that these L2 words also become automatically interconnected with sensorimotor processes. Importantly, there are two explanations for the interactions between L2 and sensorimotor processes in our study: First, when learning L2, the words might be directly connected to the experiences we have made with the according referents during L1 learning. Second, L2 processing might automatically activate the L1 words, and these are subsequently triggering the experiential motor associations. Importantly, the aim of the current study was to investigate whether L2 does automatically activate motor responses even if word meaning is fully task-irrelevant. Both explanations support this hypothesis, suggesting that during L2 comprehension experiential associations are also automatically activated. Using a color-response task ensured that participants did not have to translate the words in order to perform the task, suggesting that L2 is automatically processed to a level at which sensorimotor associations become activated. However, the analysis of the temporal characteristics of the reported compatibility effects can help to answer the question whether additional translational processes might be involved before the compatibility effect emerges for L2 words. If the compatibility effect had arised in slow RTs in L2 but over the whole RT distribution in L1, this would have suggested that indeed an additional translational phase took place before the compatibility effect emerged in L2. However, our results show that for spatial words in both L1 and L2 the compatibility effect takes some time to develop, speaking against an additional time-consuming translation phase to take place in case of L2 words. The analysis of the temporal characteristics of the compatibility effect also showed that in case of emotional words (Experiment 3), the compatibility effect emerges already in rather fast RTs. Further studies are needed to clarify whether this early onset of the compatibility effect between emotional words and vertically directed spatial responses is due to a general processing benefit for emotional words, or whether other mechanisms are underlying this early onset of the compatibility effect. Importantly, a rather early onset of the compatibility effect again speaks against a

20

C. Dudschig et al. / Brain & Language 132 (2014) 14–21

time-consuming translational process into L1 to occur before the compatibility effect can emerge for L2 words. Previous studies investigating the automatic access of word meaning during L2 processing typically implemented a betweenor within-language version of the Stroop paradigm (Stroop, 1935). Thereby, the task-irrelevant word meaning interfered with naming the ink color of the words (e.g., Dyer, 1971; Preston & Lambert, 1969). In contrast to previous studies investigating the automatic influence of color-related words on color naming responses, the words in our study referred to various entities in the real world with a stereotypical location (e.g., bird = up; worm = down) or to emotions (e.g., happy, sad) and responses were arm movements in the vertical direction. Thus, our results demonstrate that not only color-naming responses are influenced by task-irrelevant L2 color-related words, a setting where the stimulus-set (color words) and the response-set (naming colors) overlap directly (Kornblum, Hasbroucq, & Osman, 1990). In contrast, our findings show that a wide set of L2 words can affect subsequent motor responses. As participants did not have to respond verbally, the stimulus format of the task-irrelevant words (linguistic) did not overlap with the response format (motor responses). Such paradigms are generally known to decrease the amount of interferences from task-irrelevant stimuli (Kornblum et al., 1990). In other words, even in within-language versions of the Stroop paradigm interference effects between an irrelevant stimulus dimension (e.g., words) and a response are largest if the stimulus format of the task-irrelevant stimulus (e.g., word = linguistic) matches the response format (e.g. naming = linguistic). In contrast in paradigms where the response format varies from the stimulus format (e.g. button press = motoric), the impact of the task-irrelevant stimulus to influence the response is typically decreased (e.g., Lu & Proctor, 2001). Thus, given that in the current series of experiments task-irrelevant words influence subsequent motor responses, this is an experimental setup that minimizes the influence of the task-irrelevant verbal information. If even under these conditions a compatibility effect can be observed, this can be interpreted as a strong and automatic interconnection between the task-irrelevant stimuli (e.g., words) and the response (e.g., motor responses). Thus, given that we used a paradigm that generally minimizes the influence of task-irrelevant verbal information, these results suggest a highly automatic association between L2 processing and motor responses. In summary, our results suggest that very basic sensorimotor associations during processing task-irrelevant L2 words resemble motor associations triggered by L1 words. These findings put a challenge on the general claim that L2 processing does not automatically activate experiential traces and takes place in a fully amodal manner. In contrast, our results suggest that regarding very basic, word-based simulation effects, L2 processing does directly compare to L1 processing. However, further studies will be required in order to investigate whether simulation effects extend to sentence processing, how language proficiency and sensorimotor association are interrelated, how exactly the transfer of sensorimotor information associated with L1 and L2 takes place, and to investigate whether neural substrates of sensorimotor associations directly compare between L1 and L2. Acknowledgements This project was supported by a Margarete-von-Wrangell Fellowship appointed to Carolin Dudschig (European Social Fund and the Ministry of Science, Research and the Arts Baden-Württemberg) and by the SFB833/B4 project of Barbara Kaup (German Research Foundation).

References Atkinson, D., Churchill, E., Nishino, T., & Okada, H. (2007). Alignment and interaction in a sociocognitive approach to second language acquisition. The Modern Language Journal, 91(2), 169–188. Barsalou, L. W. (1999). Perceptual symbol systems. Behavioral and Brain Sciences, 22(04), 577–660. Bergen, B., Lau, T. T. C., Narayan, S., Stojanovic, D., & Wheeler, K. (2010). Body part representations in verbal semantics. Memory & Cognition, 38(7), 969–981. Bergen, B. K., Lindsay, S., Matlock, T., & Narayanan, S. (2007). Spatial and linguistic aspects of visual imagery in sentence comprehension. Cognitive Science, 31, 733–764. Botvinick, M. M., Braver, T. S., Barch, D. M., Carter, C. S., & Cohen, J. D. (2001). Conflict monitoring and cognitive control. Psychological Review, 108(3), 624–652. Boulenger, V., Hauk, O., & Pulvermüller, F. (2009). Grasping ideas with the motor system: Semantic somatotopy in idiom comprehension. Cerebral Cortex, 19, 1905–1914. Brookshire, G., Casasanto, D., & Ivry, R. (2010). Modulation of motor-meaning congruity effects for valenced words. In Proceedings of the 32nd annual conference of the cognitive science society (pp. 1940–1945). Bub, D. N., Masson, M. E. J., & Cree, G. S. (2008). Evocation of functional and volumetric gestural knowledge by objects and words. Cognition, 106, 27–58. Dudschig, C., de la Vega, I., De Filippis, M., & Kaup, B. (submitted for publication). Language and Vertical Space: On the Automaticity of Language Action Interconnections. Cortex. Dudschig, C., de la Vega, I., & Kaup, B. (submitted for publication). What’s up? Emotion-specific activation of vertical space during language processing. Acta Psychologica. Dudschig, C., Lachmair, M., de la Vega, I., De Filippis, M., & Kaup, B. (2012a). Do taskirrelevant direction-associated motion verbs affect action planning? Evidence from a Stroop paradigm. Memory & Cognition, 40, 1081–1094. Dudschig, C., Lachmair, M., de la Vega, I., De Filippis, M., & Kaup, B. (2012b). From top to bottom: Spatial shifts of attention caused by linguistic stimuli. Cognitive Processing, 13(1), 151–154. Dudschig, C., Souman, J., Lachmair, M., de la Vega, I., & Kaup, B. (2013). Reading ‘‘sun’’ and looking up: The influence of language on saccadic eye movements in the vertical dimension. PLoS ONE, 8(2), e56872. Dyer, F. N. (1971). Color-naming interference in monolinguals and bilinguals. Journal of Verbal Learning and Verbal Behavior, 10(3), 297–302. Eilola, T. M., Havelka, J., & Sharma, D. (2007). Emotional activation in the first and second language. Cognition and Emotion, 21, 1064–1076. Engelen, J. A., Bouwmeester, S., de Bruin, A. B., & Zwaan, R. A. (2011). Perceptual simulation in developing language comprehension. Journal of Experimental Child Psychology, 110(4), 659–675. Estes, Z., Verges, M., & Barsalou, L. W. (2008). Head Up, Foot Down Object Words Orient Attention to the Objects’ Typical Location. Psychological Science, 19(2), 93–97. Francis, W. S. (1999). Cognitive integration of language and memory in bilinguals: Semantic representation. Psychological Bulletin, 125(2), 193–222. Glenberg, A. M., & Gallese, V. (2012). Action-based language: A theory of language acquisition, comprehension, and production. Cortex, 48, 905–922. Glenberg, A. M., & Kaschak, M. P. (2002). Grounding language in action. Psychonomic Bulletin & Review, 9(3), 558–565. Glenberg, A. M., Sato, M., Cattaneo, L., Riggio, L., Palumbo, D., & Buccino, G. (2008). Processing abstract language modulates motor system activity. The Quarterly Journal of Experimental Psychology, 61(6), 905–919. Goldfarb, L., & Tzelgov, J. (2007). The cause of the within-language Stroop superiority effect and its implications. The Quarterly Journal of Experimental Psychology, 60(2), 179–185. Gozli, D. G., Chasteen, A. L., & Pratt, J. (2013). The cost and benefit of implicit spatial cues for visual attention. Journal of Experimental Psychology: General, 142(4), 1028–1046. Harris, C. L., Aycicegi, A., & Gleason, J. B. (2003). Taboo words and reprimands elicit greater autonomic reactivity in a first language than in a second language. Applied Psycholinguistics, 24(4), 561–579. Hauk, O., Johnsrude, I., & Pulvermüller, F. (2004). Somatotopic representation of action words in human motor and premotor cortex. Neuron, 41(2), 301–307. Jared, D., Poh, R., & Paivio, A. (2013). L1 and L2 picture naming in Mandarin–English bilinguals: A test of bilingual dual coding theory. Bilingualism: Language and Cognition, 16(2), 383–396. Johnson, J. S., & Newport, E. L. (1989). Critical period effects in second language learning: The influence of maturational state on the acquisition of English as a second language. Cognitive Psychology, 21(1), 60–99. Keysar, B., Hayakawa, S. L., & An, S. G. (2012). The foreign-language effect thinking in a foreign tongue reduces decision biases. Psychological Science, 23(6), 661–668. Kleiner, M., Brainard, D., Pelli, D., Ingling, A., Murray, R., & Broussard, C. (2007). What’s new in psychtoolbox-3. Perception, 36(14), 1. Kornblum, S., Hasbroucq, T., & Osman, A. (1990). Dimensional overlap: Cognitive basis for stimulus-response compatibility – A model and taxonomy. Psychological Review, 97(2), 253–270. Kousta, S. T., Vinson, D. P., & Vigliocco, G. (2009). Emotion words, regardless of polarity, have a processing advantage over neutral words. Cognition, 112(3), 473–481.

C. Dudschig et al. / Brain & Language 132 (2014) 14–21 Lachmair, M., Dudschig, C., De Filippis, M., de la Vega, I., & Kaup, B. (2011). Root versus roof: Automatic activation of location information during word processing. Psychonomic Bulletin & Review, 18(6), 1180–1188. Lakoff, G., & Johnson, M. (1980). Metaphors we live by. Chicago: University of Chicago Press. Lave, J. (1996). Teaching, as learning, in practice. Mind, Culture, and Activity, 3(3), 149–164. Loftus, G. R., & Masson, M. E. (1994). Using confidence intervals in within-subject designs. Psychonomic Bulletin & Review, 1(4), 476–490. Lu, C. H., & Proctor, R. W. (2001). Influence of irrelevant information on human performance: Effects of SR association strength and relative timing. The Quarterly Journal of Experimental Psychology: Section A, 54(1), 95–136. Lu, C. H., & Proctor, R. W. (1995). The influence of irrelevant location information on performance: A review of the Simon and spatial Stroop effects. Psychonomic Bulletin & Review, 2(2), 174–207. Lyons, I. M., Mattarella-Micke, A., Cieslak, M., Nusbaum, H. C., Small, S. L., & Beilock, S. L. (2010). The role of personal experience in the neural processing of actionrelated language. Brain and language, 112(3), 214–222. MacLeod, C. M. (1991). Half a century of research on the Stroop effect: an integrative review. Psychological Bulletin, 109(2), 163–203. Mägiste, E. (1984). Stroop tasks and dichotic translation: The development of interference patterns in bilinguals. Journal of Experimental Psychology: Learning, Memory, and Cognition, 10(2), 304–315. Marino, B. F., Gough, P. M., Gallese, V., Riggio, L., & Buccino, G. (2013). How the motor system handles nouns: A behavioral study. Psychological Research Psychologische Forschung, 77, 64–73. Meier, B. P., & Robinson, M. D. (2004). Why the sunny side is up: association between affect and vertical position. Psychological science, 15(4), 243–247. Naylor, L. J., Stanley, E. M., & Wicha, N. Y. Y. (2012). Cognitive and electrophysiological correlates of the bilingual Stroop effect. Frontiers in Psychology, 3(81), 1–18. Pavlenko, A. (2012). Affective processing in bilingual speakers: Disembodied cognition? International Journal of Psychology, 47(6), 405–428.

21

Preston, M. S., & Lambert, W. E. (1969). Interlingual interference in a bilingual version of the Stroop color-word task. Journal of Verbal Learning and Verbal Behavior, 8(2), 295–301. Pulvermüller, F., Shtyrov, Y., & Ilmoniemi, R. (2005). Brain signatures of meaning access in action word recognition. Journal of Cognitive Neuroscience, 17(6), 884–892. Roelofs, A. (2003). Goal-referenced selection of verbal action: modeling attentional control in the Stroop task. Psychological Review, 110(1), 88–125. Richter, T., Zwaan, R. A., & Hoever, I. (2009). Acquiring experiential traces in wordreferent learning. Memory & Cognition, 37, 1187–1196. Santiago, J., Ouellet, M., Román, A., & Valenzuela, J. (2012). Attentional factors in conceptual congruency. Cognitive Science, 36, 1051–1077. Scorolli, C., & Borghi, A. M. (2007). Sentence comprehension and action: Effector specific modulation of the motor system. Brain Research, 1130, 119–124. Stepper, S., & Strack, F. (1993). Proprioceptive determinants of emotional and nonemotional feelings. Journal of Personality and Social Psychology, 64(2), 211. Stroop, J. R. (1935). Studies of interference in serial verbal reactions. Journal of Experimental Psychology, 18, 643–662. Sutton, T. M., Altarriba, J., Gianico, J. L., & Basnight-Brown, D. M. (2007). The automatic access of emotion: Emotional Stroop effects in Spanish–English bilingual speakers. Cognition and Emotion, 21, 1077–1090. Thornton, T., Loetscher, T., Yates, M. J., & Nicholls, M. E. R. (2012). The highs and lows of the interaction between word meaning and space. Journal of Experimental Psychology: Human Perception and Performance, 39(4), 964–973. Verges, M., & Duffy, S. (2009). Spatial representations elicit dual-coding effects in mental imagery. Cognitive Science, 33(6), 1157–1172. Zwaan, R. A., & Madden, C. J. (2005). Embodied sentence comprehension. In D. Pecher & R. A. Zwaan (Eds.), Grounding cognition: The role of perception and action in memory, language, and thinking (pp. 224–245). Cambridge, UK: Cambridge University Press. Zwaan, R. A., & Taylor, L. J. (2006). Seeing, acting, understanding: Motor resonance in language comprehension. Journal of Experimental Psychology: General, 135, 1–11.