How and when accentuation influences temporally selective attention and subsequent semantic processing during on-line spoken language comprehension: An ERP study

Neuropsychologia ] (]]]]) ]]]–]]]

Contents lists available at SciVerse ScienceDirect

Neuropsychologia journal homepage: www.elsevier.com/locate/neuropsychologia

How and when accentuation influences temporally selective attention and subsequent semantic processing during on-line spoken language comprehension: An ERP study Xiao-qing Li a,n, Gui-qin Ren b a b

State Key Laboratory of Brain and Cognitive Science, Institute of Psychology, Chinese Academy of Sciences, Beijing, China Research Center of Psychological Development and Education, Liaoning Normal University, Dalian, China

a r t i c l e i n f o

abstract

Article history: Received 29 July 2011 Received in revised form 11 April 2012 Accepted 14 April 2012

An event-related brain potentials (ERP) experiment was carried out to investigate how and when accentuation influences temporally selective attention and subsequent semantic processing during online spoken language comprehension, and how the effect of accentuation on attention allocation and semantic processing changed with the degree of accentuation. Chinese spoken sentences were used as stimuli. The critical word in the carrier sentence was either semantically congruent or incongruent to the preceding sentence context. Meanwhile, the critical word was de-accented (DeAccent), generally accented (Accent), or greatly accented (GreatAccent). Results showed that, relative to semantically congruent words, the semantically incongruent word elicited a parietal–occipital N400 effect in the Accent condition and a broadly distributed N400 effect in the GreatAccent condition; however, no significant N400 effect was found in the DeAccent condition. Further onset analysis found that the N400 effect in the GreatAccent condition started around 50 ms earlier than that in the Accent conditions. In addition, in the GreatAccent condition, the incongruent words also elicited an early negative effect in the window latency of 110–190 ms after the acoustic onset of the critical word. The results indicated that, during on-line speech processing, accentuation can rapidly modulate temporally selective attention and consequently influence the depth or the speed of subsequent semantic processing; the effect of accentuation on attention allocation and semantic processing can change with the degree of accentuation gradually. & 2012 Elsevier Ltd. All rights reserved.

Keywords: Event-related potentials Accentuation Temporally selective attention Semantic processing Spoken language comprehension

1. Introduction Spoken language is the archetypical form of language. Its comprehension requires the timely coordination of a number of different information types: prosody, semantic, syntax, and pragmatics. An important difference with written language is that spoken language carries prosodic information. However, in the field of language comprehension, the majority of research has focused on semantic or syntactic information during comprehension. How and when prosody influences spoken language comprehension is still a relatively underdeveloped area. Thus, the present study aimed to examine the mechanisms by which one aspect of prosody, accentuation, influences spoken language comprehension. We mainly investigated, during on-line speech processing, how accentuation guides temporally selective attention, and how the attention captured by accentuation influences the speed and depth of subsequent semantic processing.

n

Corresponding author. Tel.: þ86 10 64864012; fax: þ 86 10 64872070. E-mail address: [email protected] (X.-q. Li).

Accentuation is one type of prosodic information in the speech signal, which reflects the relative prominence of a particular syllable, word, or phrase in a certain prosodic structure realized by modulation of pitch or syllable duration (see Shattuck-Hufnagel & Turk, 1996 for a review). Psycholinguistic studies on accentuation mainly focused on the correspondence between accentuation and information structure. Behavioral studies found that speech processing was facilitated when new information (or focused information) is accented and given information de-accented (e.g. Bock & Mazzella, 1983; Dahan, Tanenhaus, & Chambers, 2002; Li & Yang, 2004; Terken & Noteboom, 1987). Recently, studies using the electroencephalogram (EEG) technique also reported immediate ERP effects (broadly-distributed negativity, N400, or P300) for missing pitch accent on new information or superfluous pitch accent on given information (Hruska, Alter, Steinhauer, & Steube, 2000; Johnson, Clifton, Breen, Martin, & Florak, 2003; Li, Hagoort, & Yang, 2008a; Magne et al., 2005). An ERP study conducted by Wang et al. further found that accented focused information was processed more deeply compared to conditions where focus and accentuation mismatched (Wang, Bastiaansen, Yang, & Hagoort, 2011). In short, all of the existing behavioral and ERP studies provide solid evidence

0028-3932/$ - see front matter & 2012 Elsevier Ltd. All rights reserved. http://dx.doi.org/10.1016/j.neuropsychologia.2012.04.013

Please cite this article as: Li, Xiao-qing, & Ren, Gui-qin How and when accentuation influences temporally selective attention and subsequent semantic processing during on-line.... Neuropsychologia (2012), http://dx.doi.org/10.1016/j.neuropsychologia.2012.04.013

2

Xiao-qing Li, Gui-qin Ren / Neuropsychologia ] (]]]]) ]]]–]]]

for the fact that accentuation plays a fundamental role in spoken language comprehension, and can influence the ease by which the current discourse information is processed. However, the specific mechanisms by which accentuation affects spoken language processing remain matters of debate. One account is that accentuation conveys sentence-level meaning directly, related to the focus distribution of the sentence (Dahan et al., 2002; Gussenhoven, 1983; Selkirk, 1995; Terken & Noteboom, 1987). Changing the accent pattern of an utterance, by accenting some words and failing to accent others, can change the meaning of an utterance dramatically. For example, reference resolution studies showed that listeners interpret an accented word as introducing a new discourse entity and a de-accented word as anaphoric (e.g., Dahan et al., 2002). Li et al. further found that, during spoken discourse comprehension, both accentuation violation and lexical tone violation elicited an immediate N400 effect and there was no significant interaction between them, which indicated that there might be a correspondence between the neural mechanisms underlying the processing of sentence-level meaning indicated by accentuation and lexical meaning indicated by lexical tone in context (Li, Yang, & Hagoort, 2008b). All of those evidences showed that the presence or absence of pitch accent was related to the focus distribution of the sentence, and the sentence-level meaning indicated by accentuation was processed semantically as lexical meaning. Accentuation can induce listener to take qualitatively different ways to interpret the expression it marks (Terken & Noteboom, 1987). A de-accented expression is interpreted within the restricted set of already activated discourse entities, which is appropriate for given information. However, the interpretation of an accented entity is built mainly on the basis of the information contained in the speech signal, which is appropriate for new information. Therefore, appropriate accentuation facilitates spoken language comprehension (Terken & Noteboom, 1987). Another account for the accentuation effect is that accentuation can influence speech processing by modulating general cognitive processes. Some researchers found that accentuation can modulate listeners’ selective attention during speech processing (Cutler, 1976; Sanford, Sanford, Molle, & Emmott, 2006). For example, using phoneme monitoring task, Cutler (1976) showed heightened attention (as indicated by faster phoneme monitoring responses) to a word that answered a preceding question (and hence was focused information) and received a pitch accent. According to her, the prosodic information carried by the words preceding the accented words already indicated that an accent was to follow, so that listeners could focus their attention on the accented words. Recently, using change detection task, Sanford et al. (2006) also found that accentuation is a kind of attentioncapture devices during discourse processing. In the task used by Sanford et al., participants were twice presented with short, recorded discourses and asked to detect whether any of the words changed between the first and second presentations. The target word was spoken with a non-contrastive accent or with a contrastive accent. The results revealed that the ability of participants to detect a one-word alteration was superior in the contrastive accent condition than that in the non-contrastive accent condition. According to Sanford et al., the results indicated that listeners pay more attention to words with contrastive accent than to words with non-contrastive accent (Sanford et al., 2006). That is, accentuation can modulate listener’s selective attention during language processing. Some researchers resorted (e.g., Cutler, 1976) to attention allocation to explain why correspondence between accent and information structure influences discourse processing. According to the attention account, when new information was accented and given information de-accented, more attention is allocated to more important new information, thereby speeding up language comprehension. In contrast, in inappropriate accenting, more attention is

directed to less important given information with less attention left for important information, which, as a whole, might impede language processing. Although there was a relatively clear picture of how and when the sentence-level meaning conveyed by accentuation influences spoken language comprehension, the role of accentuation in modulating temporally selective attention still needs to be explored further. First, there is no direct evidence that accentuation can modulate selective attention during the natural process of on-line spoken language comprehension. Although previous studies (Cutler, 1976; Sanford et al., 2006) showed heightened attention to a word receiving a pitch accent or contrastive accent, those studies used additional tasks, such as phoneme monitoring and change detection, hence not reflecting the natural process of spoken language comprehension. Although Cutler’s (1976) study investigated the effect of pitch accent on selective attention during on-line processing, it directly compared accented words with de-accented words, hence being not able to clarify whether the accentuation effect came from the difference in attention allocating or the difference in the acoustic cues of accented and de-accented words. Moreover, the previous studies all used discourses or question–answer pairs as materials. Thus, the accented words and de-accented words differed from each other not only in the presence/absence of pitch accent but also in their information state as indicated by the preceding discourse context. Later works showed that focused words received heightened attention even when cross-splicing made them identical prosodically to non-focused words (Cutler & Fodor, 1979) or when they were presented visually without prosodic information (Birch & Rayner, 1997; Wang, Hagoort, & Yang, 2009; Ward & Sturt, 2007). Therefore, in the previous studies (Cutler, 1976; Sanford et al., 2006), the mechanisms underlying attention allocation might directly involve focus rather than accentuation. Given the above reasons, it still needs to further examine whether accentuation can modulate selective attention during the natural process of online spoken language comprehension. Second, previous studies (Cutler, 1976; Sanford et al., 2006) just investigated how the captured attention influences the perceptual level of processing (phoneme monitoring or word change). It is unknown how, or even if, the attention captured by accentuation influences the depth or the speed of the semantic processing. Third, in everyday communication, the speakers not only distinguish presence and absence of pitch accent (no emphasis and emphasis) but also distinguish different degree of accentuation, such as generally accented information (emphasis) and greatly accented information (more emphasis). For example, Chen and Gussenhoven (2008) constructed single focus words (corrective focus) with three degrees of emphasis (no emphasis, emphasis, and more emphasis). They found comparable increases in syllable duration from the NoEmphasis condition to the Emphasis condition and from the latter to the MoreEmphasis condition; Pitch range expansion, however, was non-gradual: while there was a substantial increase in the pitch range from the NoEmphasis to the Emphasis condition, the expansion from the Emphasis to the More Emphasis condition was marginal (Chen & Gussenhoven, 2008). Recently, Zhao et al. designed three emphasis degrees (non-focus, focus imbedded in lower discourse hierarchy, and focus embedded in higher discourse hierarchy) with two-character focused words (informational focus). They found that there was significant increase in syllable duration and pitch range expansion from non-focus to focus words imbedded in lower discourse hierarchy, and from the latter to the focus words embedded in higher discourse hierarchy (Zhao, Yang, Yang, ¨ 2011). These researches have provided clear evidences that & Lu, the changes in the degree of emphasis can induce corresponding variations in the pattern of accentuation. It is still unclear

Please cite this article as: Li, Xiao-qing, & Ren, Gui-qin How and when accentuation influences temporally selective attention and subsequent semantic processing during on-line.... Neuropsychologia (2012), http://dx.doi.org/10.1016/j.neuropsychologia.2012.04.013

Xiao-qing Li, Gui-qin Ren / Neuropsychologia ] (]]]]) ]]]–]]]

whether, during spoken language comprehension, the different degree of accentuation had the same effect on attention allocation and subsequent semantic analysis. If not, what is the potential difference between them? Therefore, the aim of the present study was to further investigate the role of accentuation in modulating selective attention during online speech processing. First, we examined how and when accentuation guides selective attention during on-line speech processing. Second, we aimed to investigate how, or even if, the attention captured by accentuation influences the depth of or the speed of the subsequent semantic processing. Finally, we explored how the degree of accentuation influences the role of accentuation in selective attention and subsequent semantic processing. To address these questions, the event-related brain potentials (ERP) technique was used due to its high temporal resolution and due to the fact that ERP effects can be obtained in the absence of a potentially intrusive secondary task. The materials used in the present study were isolated Chinese spoken sentences, i.e. sentences presented without explicitly provided contextual information. In such isolated sentences, accentuation does not refer to some information give earlier. Therefore, we could investigate how accentuation influences selective attention without the interference of the information structure induced by contextual information or the correspondence between accentuation and the contextual information. In the present study, native Chinese speakers were asked to listen to the spoken sentences for comprehension. On the one hand, the critical words (CW) in the sentence were semantically congruent or incongruent to the preceding sentence context. On the other hand, the CWs were not accented (DeAccent, no emphasis), generally accented (Accent, emphasis), or greatly accented (GreatAccent, more emphasis). We expected that, relative to the semantically congruent CWs, the semantically incongruent CWs would elicited a larger N400, since N400 has been found to reflect the ease with which a word is integrated into the current context (Chwilla, Brown, & Hagoort, 1995; Hagoort & Brown, 2000; Van Berkum, Brown, & Hagoort, 1999; Van Berkum, Zwitserlood, Hagoort, & Brown, 2003). Moreover, except for the N400 effect, an early negative effect might be also elicited by the semantically incongruent words. The auditory perception is based on predictive representation of temporal regularities (Winkler, 2007; Winkler, Denham, & Nelken, 2009), and previous studies have suggested that the expectancy violations in auditory signals can be detected rapidly, and manifested by an early negativity in brain potentials. For example, pitch deviation in melody and spoken language elicited an early negativity followed by a prominent P300 ¨ anen, ¨ ¨ (Brattico, Tervaniemi, Na¨ at & Peretz, 2006; Magne, Schon, ¨ & Besson, 2006; Schon, Magne, & Besson, 2004); syntactic deviants in auditory language elicited an Early Left-lateralized Anterior Negativity (ELAN) (Hahne and Friederici, 1999); mismatch between the actually acoustic input and the expectations based on the sentence context evoked an early Phonological Mismatch Negativity (PMN) effect (Connolly, Phillips, & Forbes, 1995; Connolly & Phillips, 1994; Newman & Connollt, 2009). Therefore, in the current study, an early negative effect might be evoked by the semantically incongruent words given the mismatch between initial phoneme expectations and the actual spoken input of the incongruent words. Most importantly, if listeners would allocate more attention to accented words, and process them more deeply than de-accented words, there would be an immediate interaction between accentuation and semantic congruence. That is, the semantically incongruent words would elicit a N400 effect in the accented condition, but reduced or no N400 effect in the de-accented condition. Furthermore, by examining when accentuation interacts with the negative effects elicited by semantically incongruent words, we would know the

3

time point at which accentuation begins to guide selective attention. In addition, by examining the semantic congruence effect (amplitude and time course of the negative effects) in the different conditions of accentuation (Accent vs. GreatAccent), we could clarify how the degree of accentuation influences the effect of accentuation on attention allocation and subsequent semantic processing. 2. Methods 2.1. Participants Twenty right-handed university students (10 males and 10 females), all of whom were native Chinese speakers, participated in the experiment. The mean age was 23 years (range 19–26). None of the participants had any neurological impairment, had experienced any neurological trauma, or used neuroleptics. All participants gave informed consent. This study was approved by the ethics committee in the Institute of Psychology, Chinese Academy of Sciences. The data of five participants (three males and two females) were excluded because of excessive artifacts. 2.2. Stimuli The stimuli consisted of 240 sets of experimental stimuli and 120 filler in Chinese. These sentences were spoken by a female speaker and recorded at a sampling rate of 22,050 Hz. All experimental sentences took the form of Adverbial phraseþ Subject [(‘‘be i’’)þ Noun1þ Verbþ ‘‘de’’ þNoun2] þ verb phrase (see Table 1). In Chinese, Noun1 þVerb þ‘‘de’’ þ Noun2 is a complex noun phrase consisting of the object-extracted relative clause and the head noun Noun2. That is, Noun2 is assigned an undergoer role and Noun1 an actor role; Noun2 is the head noun; both Noun1 and the verb are used to modify Noun2. In the present study, Noun2 is the critical word. Both Noun1 and Noun2 are two-character words, with the former always being an animate noun and the later always being an inanimate noun, which was consistent with their argument roles. On the one hand, the semantic congruence of the critical word Noun2 was manipulated by the presence/absence of be i preceding Noun1þ Verbþ ‘‘de’’þ Noun2. In modern Chinese, the word be i has little lexical meaning and is understood as the indicator of a passive relation. For example, in the complex noun phrase [(‘‘be i’’)þ Noun1þ Verbþ ‘‘de’’þ Noun2], be i indicated that the thematic role of the head noun Noun2 is that of an undergoer. Although be i has little lexical meaning, it usually indicates something unexpected happened, and the undergoer in the be i construction is usually animate (Fan, 2006). It was pointed out that in modern Chinese, the word be i leads to an adversative reading to the undergoer argument. Crucially, this reading is associated with semantic consequences, i.e. an adversely affected argument must be able to experience a psychological state and should therefore be animate. When this construction-based semantic constraint was violated, a neurophysiological response indexing a semantic mismatch (N400; see Frisch & Schlesewsky, 2001; Philipp, Bornkessel-Schlesewsky, Bisang, & Schlesewsky, 2008) occurs. However, some linguistics assumed that the undergoer in the be i construction can sometimes be inanimate, which happened only when the verb has the meaning of ‘to destroy, to damage, or to break’ (Fan, 2006). In the present study, to ensure the semantic violation of the critical word Noun2, the verb in the complex noun phrase did not have the meaning of ‘to destroy, to damage, or to break’ (see the appendix). The critical word was semantically congruent to the preceding sentence context when the complex noun phrase was not preceded by the word be i. Otherwise, the critical word Noun2 was semantically incongruent. That is, we manipulated the semantic congruence of the critical word without changing the meaning of the sentence and the critical word itself. On the other hand, the critical words were not accented (DeAccent, no emphasis), generally accented (Accent, emphasis), or greatly accented (GreatAccent, more emphasis). During recording, the experimental sentences were presented in certain context in order to guarantee that the sentences were spoken with the intended accentuation pattern. For example, for the experimental sentence The key Wang Yan found just now is of great help, in the DeAccent condition, it was recorded in the context of A; in the Accent condition, it was recorded in the context of B; in the GreatAccent condition, it was recorded in the context of C. That is, the difference between the two pronunciations of the critical word in Accent and DeAccent conditions is discrete, in the sense that the critical word was accented or not accented. The difference between the two pronunciations of the critical word in Accent and GreatAccent conditions is gradient, and the greatly accented words were more prominent than the generally accented words. In Chinese, expect for accentuation, there were also some lexical markers for focus constituents, such as shi. Therefore, when constructing the materials, it was guaranteed that there was no lexical marker of focus in the sentences. A: How about the key Wang Yan found just now? B: What Wang Yan found just now is of great help? C: What Wang Yan found just now is of great help?

Please cite this article as: Li, Xiao-qing, & Ren, Gui-qin How and when accentuation influences temporally selective attention and subsequent semantic processing during on-line.... Neuropsychologia (2012), http://dx.doi.org/10.1016/j.neuropsychologia.2012.04.013

4

Xiao-qing Li, Gui-qin Ren / Neuropsychologia ] (]]]]) ]]]–]]]

Table 1 Illustrations for the experimental materials. For example, Just now [(Wang Yan)Noun1 (found)verb ‘‘de’’ (key)Noun2]Subject [is of great help]Verb The key Wang Yan found just now is of great help

phrase

Just now [‘‘be i’’ (Wang Yan)Noun1 (found)verb ‘‘de’’ (key)Noun2]Subject [is of great help]Verb The key Wang Yan found just now is of great help.

phrase

(1) Noun2 was de-accented (DeAccent) and semantically congruent

The key Wang Yan found just now is of great help. (2) Noun2 was de-accented (DeAccent) and semantically incongruent

The key Wang Yan found just now is of great help. (3) Noun2 was accented (Accent) and semantically congruent

The key Wang Yan found just now is of great help. (4) Noun2 was accented (Accent) and semantically incongruent

The key Wang Yan found just now is of great help. (5) Noun2 was greatly accented (GreatAccent) and semantically congruent

The key Wang Yan found just now is of great help. (6) Noun2 was greatly accented (GreatAccent) and semantically incongruent

The key Wang Yan found just now is of great help. Note: the underlined words are the critical words (Noun2); ERPs are aligned to the critical words. Bold indicated the critical words were generally accented; bold and italic indicated the critical words were greatly accented.

The ‘‘key’’ Wang Yan found just now is of great help. ‘‘What’’ Wang Yan found just now is of great help? (The listener didn’t hear the answer clearly, hence asking again.)

To confirm the semantic congruence/incongruence of the critical word, 48 subjects who did not attend the ERP experiment were asked to mark the congruence of Noun2 in the complex noun phrases [(‘‘be i’’)þ Noun1þVerbþ ‘‘de’’þNoun2] on a 5-point scale (from  2 to 2). -2 indicated that Noun2 was highly incongruent in the phrase; 2 indicated that Noun2 was highly congruent in the phrase. The 240 pairs of complex noun phrases were divided into two groups, with each group including 120 pairs. Then, each group of the 120 pairs of phrases was grouped into 2 lists of 120 sentences according to the Latin square procedure based on the two experimental conditions (with be i, without be i). In each list, there were an equal number of phrases (60 phrases with be i and 60 phrases without be i) for the two experimental conditions, and there was no repetition of the phrases across the two lists. Totally, we had 4 lists of materials with each including 120 experimental complex noun phrases. In very list, there were also 80 filler phrases. Subjects were divided into four groups, with each group marking only one list of materials. The mean score for the complex noun phrases without be i was 1.25 (STDEV¼ .58), and the mean score for the complex noun phrases with be i was  .33 (STDEV¼.56). The statistic analysis revealed that there was significant difference between the two kinds of complex noun phrases (t1(47) ¼13.17, po.0001; t2(239) ¼ 27.89, po.0001). The rating results indicated that the congruence of the critical word in the be i construction was indeed different from that without be i. To establish that our speaker had succeeded in correctly accenting the relevant Noun2, ANOVAs were performed on the corresponding acoustic measurements, with Accentuation (DeAccent, Accent, vs. GreatAccent) and Semantic Congruence (congruent vs. incongruent) as independent factors. Chinese is a tone language, in which there are four kinds of lexical tone: HH (for High tone), LH (for Rising tone), LL (for Low tone), and HL (for Falling tone) (Shih, 1988). The pitch correlate of the lexical tone in Chinese is not a single point but a pitch contour, which is called pitch register. Previous studies found that, in Chinese, focus accent was realized by the expansion of the pitch range of the lexical tone (pitch register) and by the lengthening of the syllable duration; (Chen, 2003, 2006; Chen & Gussenhoven, ¨ & Yang, 2008; Jia, Xiong, & Li, 2006; Jia, Li, & Chen, 2008; Liu & Xu, 2005; Wang, Lu,

2002; Xu, 1999). As to what gave rise to the pitch range expansion, Xu (1999) found that focus accent raises the pitch maximum of H tones (HH, LH, and HL), and lowers the pitch minimum of L tones (LH, LL, and HL). However, other studies found that, for two-character words, focus accent mainly raises the pitch maximum of H tones and had no significant effect on L tones (Jia et al., 2006, 2008). In short, the more reliable and consistent acoustic realization of focus accent in Chinese is syllable duration lengthening and pitch range expansion, and the latter is mainly due to the raising of pitch maximum. The critical words in the present study were all two-character words. Even when the two characters were both Low tones (LL), Third-tone sandhi changed the first of a sequence of two Low tones into LH. Namely, all of the critical words in the present study included at least one H tone. Moreover, the critical words were exactly the same in the six experimental conditions. Given the acoustic realization of focus accent in Chinese and the materials used in the present study, the dependent factors for the ANOVAs were pitch maximum, pitch range, and syllable duration. Pitch maximum ¼ 12 log2 ðMaximum Pitch=100Þ: Pitch range ¼ 12 log2 ðMaximum Pitch=Minimum PitchÞ: The results of the ANOVAs (with duration, pitch maximum, or pitch range of the critical words as dependent factors) revealed a significant main effect of Accentuation (F(2,478) ¼ 1345.41, p o.0001; F(2,478) ¼ 662.57, p o.0001; F(2,478) ¼ 305.44, p o .0001 for duration, pitch maximum, and pitch range respectively). Both the main effect of Semantic Congruence and the two-way interaction between Accentuation and Semantic Congruence failed to reach significance (all ps 4.1). Subsequent Pair-wise comparisons (Bonferroni adjustment) found that there was significant increase in syllable duration, pitch maximum, and pitch range expansion from de-accented words to accented words (107 ms, 2.9 semitones, and 3.9 semitones respectively) and from the latter to the greatly accented words (228 ms, 1.4 semitones, and .9 semitones respectively) (all ps o.0001) (see Table 2). The pitch range expansion and syllable duration lengthening from deaccent to accented words was consistent with previous studies (Chen, 2003, 2006; Jia et al., 2008; Liu & Xu, 2005; Wang et al., 2002; Xu, 1999). The pitch range expansion from accented (emphasis) to greatly accented words (more emphasis) was consistent with Zhao and et al.’s (2011) study but inconsistent with Chen and Gussenhoven’s (2008) study, which may be due to the fact that the focus used to

Please cite this article as: Li, Xiao-qing, & Ren, Gui-qin How and when accentuation influences temporally selective attention and subsequent semantic processing during on-line.... Neuropsychologia (2012), http://dx.doi.org/10.1016/j.neuropsychologia.2012.04.013

Xiao-qing Li, Gui-qin Ren / Neuropsychologia ] (]]]]) ]]]–]]]

Table 2 Acoustic parameters of the critical words in different experimental conditions. DeAccent M

Accent SD

M

Great Accent SD

M

21.6 21.5

1.8 1.8

23.0 23.0

1.6 1.4

Pitch range (semitones) Congruent 8.1 Incongruent 8.3

3.5 3.0

12.0 12.2

4.0 4.4

14.1 13.7

5.1 4.1

61.2 62.2

699.9 695.0

75.9 86.7

925.4 925.3

140.5 143.6

Duration (ms) Congruent Incongruent Intensity (dB) Congruent Incongruent

591.4 590.2 61.6 61.2

2.3 2.3

Table 3 The acoustic parameters of the words preceding the CWs (from the onset of Verb in the complex noun phrase to the onset of the CWs) in different experimental conditions.

SD

Pitch maximum (semitones) Congruent 18.7 2.4 Incongruent 18.7 2.5

DeAccent M

62.0 61.8

2.1 2.1

62.3 62.2

1.9 1.9

Note: DeAccent indicated that the critical words were not accented; Accent indicated that the critical words were generally accented; GreatAccent indicated that the critical words were greatly accented. Pitch maximum ¼ 12 log2(Maximum Pitch/100). Pitch range¼ 12 log2(Maximum Pitch/Minimum Pitch). elicit accentuation was informational focus in the present study and Zhao et al.’s study, but was corrective focus in Chen and Gussenhoven’s study. Anyway, the acoustic measurements in the present study confirmed that the sentences were spoken with the intended accentuation pattern. Moreover, in the present study, we used the semantic congruence effect to investigate how accentuation influences selective attention and the subsequent semantic processing. Namely, we examined how the semantic congruence effect interacts with the manipulations of accentuation, but not compared different kinds of accentuation directly. We need to confirm that the acoustic parameters of the speech signal in the semantically congruent condition were not different from those in the semantically incongruent condition. The above ANOVAs with duration, pitch maximum, and pitch range of the CWs as dependent factors indeed found that there was no significance difference between the semantic congruence and incongruence conditions. To further confirm that the confounding factor mentioned above did not exist, we also measured syllable duration, pitch maximum, and pitch range preceding the critical words (namely, from the onset of Verb in the complex noun phrase to the onset of the CW), and performed the same ANOVAs. In addition, even though intensity is not a reliable cue to focused accent, we also conducted ANOVAs with intensity both on and preceding the CWs as dependent factors. The results found that (see Tables 3 and 2): for preceding pitch maximum, we only found a significant decrease from Accent condition to GreatAccent condition ( .4 semitones) (po.0001); for preceding pitch range, the main effect of Accentuation did not reached significance (F(2,478) ¼2.69, p¼.071); for preceding syllable duration, there was a gradual increase from DeAccent condition to Accent condition (58 ms), and from the latter to GreatAccent condition (73 ms) (all pso.0001). The syllable lengthening on the words preceding the focused words was consistent with previous studies (Chen, 2006) which revealed that when a focused domain is multisyllabic, there is spill-over lengthening on the neighboring syllables outside the focused constituent. In addition, in the current study, for intensity on the CWs, there was significant increase from deaccented words to accented words (.51 dB) and from the latter to the greatly accented words (.37 dB) (all pso.0001, see Table 2); for intensity preceding the CWs, there was only a significant increase from DeAccent condition to Accent condition (.54 dB) (po.0001). Most importantly, for preceding syllable duration, preceding pitch maximum, and preceding pitch range, neither the main effect of Semantic Congruence nor the interaction between Accentuation and Semantic Congruence reached significance. For intensity, even though Congruence condition had significant higher intensity than Incongruence conditions (F(1,239) ¼30.36, po.0001, Intensity difference¼ .22 dB; F(1,239) ¼ 44.55, po.0001, Intensity difference¼ .26 dB for intensity on and preceding the CWs respectively), we did not expect the small difference in intensity (.22 dB or .26 dB) would produce an N1 response in the subjects. Previous ERP results revealed that the auditory N1 response is sensitive to changes in intensity of the sound signals, and the N1 intensity discrimination thresholds ranged from 2 to 3 dB at 500 Hz or 300 Hz (Harris, Mills, & Dubno, 2007; McCandless & Rose, 1970; Martin & Boothroyd, 2000). Therefore, in the current study, although there was difference in the intensity between congruent and incongruent conditions, we did not expect the small difference in intensity (.22 dB or .26 dB) would produce an N1 response in the subjects. Above all, the CWs and the words preceding CWs in the semantically congruent and incongruent conditions were not only the same written words but also matched on the acoustic parameters, such as duration, pitch maximum, pitch range, and intensity. The experiment design was fully factorial, combining all conditions of Accentuation (DeAccent, Accent, vs. GreatAccent) and Semantic Congruence (congruent vs. incongruent). Experimental materials were grouped into 6 lists of

5

Accent SD

M

GreatAccent SD

M

SD

Pitch maximum (semitones) Congruent 19.1 Incongruent 19.0

2.0 2.0

19.0 19.0

1.8 2.0

18.5 18.5

2.0 1.9

Pitch range (semitones) Congruent 7.2 Incongruent 7.3

2.6 2.2

7.1 7.2

2.5 2.0

6.9 7.1

2.1 2.0

Duration (ms) Congruent Incongruent

479.4 480.5

54.5 55.8

535.2 539.6

65.0 69.0

609.0 612.1

85.0 88.6

Intensity (dB) Congruent Incongruent

62.8 62.5

2.2 2.3

63.3 63.0

2.0 2.0

63.4 63.1

1.8 1.9

Note: CW indicated critical words; DeAccent indicated that the critical words were not accented; Accent indicated that the critical words were generally accented; GreatAccent indicated that the critical words were greatly accented. Pitch maximum¼ 12 log2(Maximum Pitch/100). Pitch range¼ 12 log2(Maximum Pitch/ Minimum Pitch).

240 sentences according to the Latin square procedure based on the six experimental conditions. In each list, there were an equal number of sentences (40 sentences) for every of the six conditions, and there was no repetition of the critical words across the six conditions. In addition, there were also 120 filler sentences in every list. Subjects were divided into six groups, with each group listening to only one list of materials. That is, each subject was presented with 40 sentences for every of the six experimental conditions and an additional 120 filler sentences. 2.3. Procedure After the electrodes were positioned, subjects were asked to listen to each sentence for comprehension. Meanwhile, their EEG signals were recorded. To ensure that the subjects indeed listened to the sentences for comprehension, at the end of each of the 80 sentences (40 experimental sentences and 40 filler sentences) in all of the materials, they were asked to judge the correctness of a question sentence regarding the meaning of the sentence just heard. For example, the question sentence followed the spoken sentence ‘The singer who was criticized by the audience yesterday doesn’t want to come on the stage’ was ‘The singer was praised’. Each trial consisted of a 300 ms auditory warning tone, followed by 700 ms of silence and the target sentence. To inform subjects of when to fixate and sit still for EEG recording, an asterisk was displayed from 500 ms before onset of the sentence to 1000 ms after its offset. After a short practice session that consisted of 10 sentences, the trials were presented in four blocks of approximately 10 min each, separated by brief resting periods. 2.4. EEG acquisition EEG was recorded (.05–100 Hz, sampling rate 500 Hz) from 64 Ag/AgCl electrodes mounted in an elastic cap (Neuroscan Inc.), with an on-line reference linked to the left mastoid and off-line algebraic re-reference linked to the left and right mastoids. EEG and EOG data were amplified with AC amplifiers (Synamps, Neuroscan Inc.). Vertical eye movements were monitored via a supra- to suborbital bipolar montage. A right-to-left canthal bipolar montage was used to monitor horizontal eye movements. All electrode impedance levels (EEG and EOG) were kept below 5 kO. 2.5. Data preprocessing The raw EEG data were first corrected for eye-blink artifacts and filtered with a band-pass filter .1–40 Hz. Subsequently, the EEG data were divided into epochs ranging from 100 ms before the acoustic onset of the CW to 1000 ms after the acoustic onset of the CW. A time window of 100 ms preceding the onset of the CW was used for baseline correction. Trials contaminated by eye movements, muscle artifacts, electrode drifting, amplifier saturation, or other artifacts were identified with a semiautomatic artifact rejection (automatic criterion: signal amplitude exceeding 775 mV, followed by a manual check). Trials containing the abovementioned artifacts were rejected

Please cite this article as: Li, Xiao-qing, & Ren, Gui-qin How and when accentuation influences temporally selective attention and subsequent semantic processing during on-line.... Neuropsychologia (2012), http://dx.doi.org/10.1016/j.neuropsychologia.2012.04.013

6

Xiao-qing Li, Gui-qin Ren / Neuropsychologia ] (]]]]) ]]]–]]]

(5% overall). Rejected trials were evenly distributed among conditions. Then, with the help of EEGLAB (http://sccn.ucsd.edu/), we used Independent Component Analysis to remove the residual eye movement artifacts from the EEG data. Finally, averages were computed for each participant, each condition, and at each electrode site before grand averages were calculated across all participants. Grand averages were filtered with a 12-Hz low-pass filter for presentation purposes only, and statistical analyses were conducted with the .1–40 Hz band-pass filtered data (Fig. 1) Figs. 2–4 show overlays of the ERP waveforms (time-locked to the onset of Noun2) in the six conditions. Visual inspection of the grand average waveforms reveals that, in the Accent and GreatAccent conditions, relative to semantically congruent words, semantically incongruent words elicited a larger negative deflection both in the relatively earlier and later time windows (see Figs. 3–5). However, in the DeAccent condition, that semantic congruence effect disappeared (see Fig. 2).

identify epochs over which the extracted components were best represented. The ERP data were examined using PCA defined epochs. Separate ANOVAs were conducted on each of these PCA-defined epochs with mean amplitudes as the dependent factor. Those ANOVAs were performed on a selection of midline electrodes and lateral electrodes respectively. For the midline electrode sites, the mean amplitude values were entered into repeated measures ANOVAs with Accentuation (DeAccent, Accent, vs. GreatAccent), Semantic Congruence (congruent vs. incongruent), and Electrodes (Fz, Cz, Pz, Oz) as independent factors. For lateral electrodes, the mean amplitude values were entered into repeated measures ANOVAs with Hemisphere (left, right) as an additional factor and lateral electrodes (F5/F6; F3/F4; C5/C6; C3/C4; P5/P6; P3/P4; PO5/PO6; PO3/PO4 ) nested under Hemisphere. When the degree of freedom in the numerator was larger than one, the Greenhouse–Geisser correction was applied.

2.6. PCA and statistical analysis

3. Results

Analysis of the ERP data was carried out using principle-component analysis (PCA) to define the negative components in the different time windows. This was performed from the onset of the critical word to 1000 ms after it. The 500 sample points were down sampled to 250 4-ms time points to reduce the computational burden. The combination of 60 electrodes (PO7 and PO8 were deleted, because PO8 was connected to left mastoid during EEG recording), 15 subjects, and six stimuli types totaled 5400 observations at each time point for the PCA. Correlation matrix and Varimax rotation (Chapman & McCrary, 1995; Picton et al., 2000; van Boxtel, 1998) were used to do the analysis. The factor loadings were used to

3.1. Results of the PCA analysis

Onset

Offset

500

F0 (Hz)

75

3.378

0

500

75

3.2. Results in the 400–610 ms latency range (component 2)

0

3.557

500

75

The PCA reduced temporal dimensions of the data set from 250 time-points to 4 components (see Fig. 6) which explained 71.32% of the variance in the data. Component 1 (48.99% explained variance) is a slow wave component which has high loadings at the end of the epoch. Although such a component may reflect cognitive activity, more often it is the result of the auto-correlated nature of the data and reflects direct current drift over the trial (Wastell, 1981). Component 1 was excluded from further analysis. Component 2 (12.53% explained variance) had high loadings in the window latency of 400–610 ms. The negative effect that Component 2 matched in our grand averaged ERPs had parietal–occipital distribution in the Accent condition, and widely distributed across the scalp and reached maximum over the central–parietal lobe in the GreatAccent condition (see Figs. 3–5). Given its latency and topography, we classified that negative effect as N400 effect. In addition, Component 3 (5.81% explained variance) and Component 4 (4.09% explained variance) had high loadings in the window latency of 240–340 ms and 110–190 ms respectively. Those two components matched the earlier negative effects preceding the N400 effect in our grand averaged ERP. The mean amplitudes in the window latency of 400–610 ms, 110–190 ms, and 240–340 ms were calculated for further statistical analyses.

0

4.216 Time (s)

Fig. 1. The voice spectrographs and uncorrected fundamental frequency contours (white line) for the sentence ‘‘The key Wang Yan found just now is of great help’’. The black vertical line indicated the onset and the offset of the Noun2 (key). (A) Noun2 was not accented; (B) Noun2 was generally accented; (C) Noun2 was greatly accented (figure created using PRAAT software: PRAAT Ver. 4.6.01, University of Amsterdam, Netherland).

In the 400–610 ms window latency, the ANOVAs revealed a significant main effect of Semantic Congruence (Fmidline(1,14)¼ 7.11, p o.05; Flateral(1,14)¼6.93, po.05), suggesting that the semantically incongruent words elicited a larger N400 than the semantically congruent words (effect magnitude:  .51 mV and  .52 mV for midline and lateral analysis respectively). Importantly, there were a significant three-way Accentuation  Semantic Congruence  Electrodes interaction (Fmidline(6,84)¼3.77, po.005; Flateral(6,84)¼4.32, p o.05). Resolving the three-way interaction by Accentuation showed interactions of Semantic Congruence  Electrodes for Accent condition (Fmidline(3,42)¼ 6.59, p o.01; Flateral(3,42)¼ 7.73, p o.01), but not for DeAccent and GreatAccent conditions (Fmidline(3,42)¼.29, p ¼.743 and Flateral(3,42)¼.13, p¼.830; Fmidline(3,42)¼ 1.94, p¼.169 and Flateral(3,42)¼.86, p ¼.407 respectively). Further simple–simple analysis found that, for the Accent condition, the Semantic Congruence effect reached significance over the parietal (PZ, P3/P4, P5/P6) and occipital electrodes (OZ, PO3/PO4, PO5/PO6) (Fmidline(1,14)¼5.56, p o.05 and Flateral(1,14)¼3.99, p ¼.066; Fmidline(1,14)¼5.00, p o.05 and Flateral(1,14)¼3.12, p o.05 for the parietal and occipital areas respectively); for the GreatAccent condition, the Semantic Congruence effect widely distributed across the scalp and reached maximum over the central or parietal

Please cite this article as: Li, Xiao-qing, & Ren, Gui-qin How and when accentuation influences temporally selective attention and subsequent semantic processing during on-line.... Neuropsychologia (2012), http://dx.doi.org/10.1016/j.neuropsychologia.2012.04.013

Xiao-qing Li, Gui-qin Ren / Neuropsychologia ] (]]]]) ]]]–]]]

7

Fig. 2. Grand-average ERPs time-locked to the Noun2 for the two conditions of de-accented information (DeAccent). There was no significant difference between the negative deflections elicited by semantically congruent words and semantically incongruent words.

Fig. 3. Grand-average ERPs time-locked to the Noun2 for the two conditions of generally accented information (Accent). Relative to semantically congruent words, semantically incongruent words elicited a larger parietal–occipital N400.

lobe (Fmidline(1,14)¼5.36, po.05 and Flateral(1,14)¼4.48, p¼.053; Fmidline(1,14)¼8.14, po.05 and Flateral(1,14)¼9.44, po.01; Fmidline(1,14)¼11.87, po.005 and Flateral(1,14)¼6.88, po.05; Fmidline(1,14)¼3.82, p¼.071 and Flateral(1,14)¼6.19, po.05 for the frontal, central, parietal, and occipital areas respectively); for the DeAccent condition, no significant Semantic Congruence effect was found over any of the electrodes (all ps4.1). The above results revealed that, for Accent condition, semantically incongruent words elicited a parietal–occipital N400 effect; however, for GreatAccent, semantically incongruent words evoked a broadly-distributed N400 effect. Then, we compared

the N400 effect in the Accent condition with that in the GreatAccent condition. The ANOVA with Accentuation type, Electrodes (and Hemisphere), as independent factors resulted in a significant interaction between Accentuation type and Electrodes (Fmidline (3,42)¼6.60, p o.01; Flateral(3,42)¼6.99, p o.01). Further simpleanalysis found that the N400 effect in the GreatAccent condition was significant larger than that in the Accent condition over the central electrodes for the lateral analysis (Flateral(1,14)¼4.61, po.05; effect magnitude: 1.58 mV and.38 mV for the GreatAccent and Accent conditions respectively) and over frontal electrodes for the midline analysis (Flateral(1,14)¼5.92, p o.05;

Please cite this article as: Li, Xiao-qing, & Ren, Gui-qin How and when accentuation influences temporally selective attention and subsequent semantic processing during on-line.... Neuropsychologia (2012), http://dx.doi.org/10.1016/j.neuropsychologia.2012.04.013

8

Xiao-qing Li, Gui-qin Ren / Neuropsychologia ] (]]]]) ]]]–]]]

Fig. 4. Grand-average ERPs time-locked to the Noun2 for the two conditions of greatly accented information (GreatAccent). Relative to semantically congruent words, semantically incongruent words elicited a larger widely-distributed N400 and a larger early negative deflection in the window latency of 110–190 ms that had a frontal– central distribution.

Fig. 6. Principle component analysis (PCA) factor loadings of four major components.

Fig. 5. Topography of the ERP effects for the semantic congruence effects (semantic incongruence vs. semantic congruence) under different conditions of accentuation.

effect magnitude: 1.20 mV and.79 mV for the GreatAccent and Accent conditions respectively). Next, we analyzed the onset of the Semantic Congruence effects (N400 effect). To establish the onsets of this effect, we conducted a series of analyses in consecutive mean amplitude latency bins of 10 ms wide from 300 ms after the acoustic onset of CW (e.g. 300–310 ms, 310–320 ms, etc.). Significance (P o.05) or marginal significance (Po.07) on 4 consecutive bins on the same electrode was taken as evidence for the onset of a certain effect. According to this criterion, in the Accent condition, the Semantic

Congruence effect started in 450–460 ms latency bin for both the midline electrode (PZ, Fmidline(1,14)¼4.40, p ¼.055) and lateral electrodes (PO3/PO4 and PO5/PO6, Flateral(1,14)¼4.37, p¼.055); in the GreatAccent condition, the Semantic Congruence effect started in 400–410 ms latency bin for the midline electrode (PZ, Fmidline(1,14)¼5.60, p o.05) and in 420–430 ms latency bin for the lateral electrodes (P3/P4 and P5/P6, Flateral(1,14)¼10.13, po.01; PO3/PO4, PO5/PO6, Flateral(1,14)¼9.55, p o.01). Thus, the Semantic Congruence effect in the GreatAccent condition occurred approximately 50 ms earlier than that in the Accent condition. To further confirm that the onset of the N400 effect in the GreatAccent condition was earlier than that in the Accent condition, the PCA was done separately on the Accented and GreatAccent conditions. As seen in Fig. 7, for the same loading score, the N400 component in the GreatAccent conditions started earlier than that in the Accent condition. In short, both the onset analysis of the ERPs and the PCA analysis revealed that the semantic congruence effect (N400) in the GreatAccent condition occurred earlier than that in the Accent condition.

Please cite this article as: Li, Xiao-qing, & Ren, Gui-qin How and when accentuation influences temporally selective attention and subsequent semantic processing during on-line.... Neuropsychologia (2012), http://dx.doi.org/10.1016/j.neuropsychologia.2012.04.013

Xiao-qing Li, Gui-qin Ren / Neuropsychologia ] (]]]]) ]]]–]]]

9

between DeAccent and GreatAccent in the semantically incongruent condition. In the window latency of 240–340 ms, there were only a significant or a marginally significant two-way Semantic Congruence  Electrodes interaction (Fmidline(3,42)¼3.13, p¼.066; Flateral(3,42)¼6.77, p o.05). However, further simple analysis found that the effect of Semantic Congruence reached significance over none of the electrodes.

4. Discussion

Fig. 7. Factor loadings of four major components in the principle component analysis (PCA) of the accented words (Accent) and greatly accented words (GreatAccent) respectively. GAcom1, GAcom2, GAcom3, and GAcom4 indicated component 1, component 2, component 3, and component 4 in the GreatAccent condition respectively; Acom1, Acom2, Acom3, and Acom4 indicated component 1, component 2, component 3, and component 4 in the Accent condition respectively. Component 2 in the GreatAccent condition and component 3 in the Accent condition matched the N400 deflection in the ERPs.

3.3. Results in the 110–190 ms and 240–340 ms latency ranges (component 4 and component 3) The ANOVA in the window latency of 110–190 ms resulted in a significant two-way Accentuation  Semantic Congruence interaction (Fmidline(2,28)¼5.26, p o.05; Flateral(2,28)¼5.55, p o.05). Simple analysis found that the effect of Semantic Congruence reached significance only in the GreatAccent condition (Fmidline (1,14)¼7.86, po.05; Flateral(1,14)¼ 5.71, p o.05), suggesting that the negative deflection in the incongruence condition was significantly larger than that in the congruence condition (effect magnitude: 1.01 mV and  .77 mV for midline and lateral analysis respectively). In addition, there was a three-way Accentuation  Semantic Congruence  Electrodes interaction (Fmidline (6,84)¼ 3.28, po.05; Flateral(6,84)¼2.56, p¼.088). Further simple– simple analysis revealed that the Semantic Congruence effect (early negative effect) reached significance only in the GreatAccent condition, which was significant over the frontal, central, and parietal electrodes for the midline analysis (Fmidline(1,14)¼6.81, po.05; Fmidline(1,14)¼9.06, po.01; Fmidline(1,14)¼5.76, po.05 for the frontal, central, parietal areas respectively), and over the frontal and central electrodes for the lateral analysis (Flateral(1,14)¼6.16, po.05; Flateral(1,14)¼6.32, po.05; for the frontal and central areas respectively). In short, in the window latency of 110–190 ms, the Semantic Congruence effect (early negative effect) reached significance only in the GreatAccent condition, which mainly had a frontal–central scalp distribution. In addition, in order to examine how the negative deflection in the 110–190 ms time window was modulated by the acoustic changes in the different kinds of accentuation, we also analyzed the simple main effect of Accentuation over levels of Semantic Congruence. This simple analysis revealed that there was a marginally significant simple main effect of Accentuation in the incongruence condition (Fmidline(2,28)¼3.28, p¼.052; Flateral(2,28)¼ 2.81, p¼.077), but not in the congruent condition (Fmidline(2,28)¼1.29, p¼.292; Flateral(2,28)¼1.84, p¼.178). Pairwise comparisons in the incongruent condition revealed a trend towards significance for the difference between DeAccent and GreatAccent, suggesting that the later elicited a relatively larger negative deflection than the former (p¼.052 and p¼.077 for the midline and lateral analysis respectively). Therefore, the early negative deflections elicited by the different kinds of accentuation did not differ from each other when the CWs were semantically congruent. The negative effect in the 110–190 ms time window mainly came from the difference

In this experiment, we investigated how accentuation influences temporally selective attention and subsequent semantic processing during on-line speech comprehension, and how the effect of accentuation on attention allocation and semantic processing changes with the degree of accentuation. Selective attention and semantic processing was evaluated by measuring ERP effects elicited by semantically incongruent words relative to congruent words in the sentence context. The major results of the current study were that, we found a parietal–occipital N400 effect in the Accent condition and a broadly distributed N400 effect in the GreatAccent condition. However, no significant N400 effect was found in the DeAccent condition. The N400 effects in the Accent and GreatAccent conditions had different onset latency and different scalp distribution. In addition, in the GreatAccent condition, the incongruent words also elicited an early negative effect in the 110–190 ms window latency. These results are discussed in more detail below. 4.1. The effect of accentuation on attention allocation and semantic processing A very important motivation for the current study was to investigate how, or even if, accentuation influences temporally selective attention and subsequent semantic processing during online speech comprehension. The current results revealed, with the presence of pitch accent (Accent and GreatAccent), semantically incongruent words elicited a significant N400 effect. However, the semantic congruence effect (N400 effect) disappeared when the critical words were not accented. There were three interpretations of the N400 effects. The first possibility is that the N400 effect was the residual effect of the early negative effect (110–190 ms) induced by acoustic abnormalities. This was disproved by the fact that the N400 effect has a parietal–occipital or wide distribution and the early negative effect (110–190) had a frontal–central distribution. Moreover, if the N400 effect was the residual effect of the early negative effect (110–190 ms), the negative effect (240–340 ms) between them would be significant too. However, the negative effect in the window latency of 240–340 ms did not reach significance at all, which was inconsistent with the residual-effect explanation of the N400 effect. The second possibility is that the N400 effect might be related to a phonological expectancy violation due to the pitch excursion in the Accent and GreatAccent conditions. It is indeed that there was gradual increase in syllable duration and pitch range expansion from DeAccent to Accent, and from the latter to GreatAccent. However, in the current study, we did not compare the different kinds of accentuation directly. In contrast, we compared the semantically incongruent words with congruent words in the same condition of accentuation, and then examine how the semantic congruence effect was modulated by accentuation. For the acoustic parameters (duration, pitch maximum, pitch range, and intensity) of both the critical words and the preceding words, the ANOVAs did not found reliable or significant difference between the semantic congruence and incongruence conditions. Moreover, even if Accent or GreatAccent condition induced other subtle phonetic variations, it would have the

Please cite this article as: Li, Xiao-qing, & Ren, Gui-qin How and when accentuation influences temporally selective attention and subsequent semantic processing during on-line.... Neuropsychologia (2012), http://dx.doi.org/10.1016/j.neuropsychologia.2012.04.013

10

Xiao-qing Li, Gui-qin Ren / Neuropsychologia ] (]]]]) ]]]–]]]

same effect on semantic congruence and incongruence conditions, since in those two conditions, the critical words and the words processing and following the critical words were exactly the same. Therefore, the semantic congruent words and incongruent words (in the same condition of accentuation) had the same acoustic properties, and it was not likely that the N400 effect was caused by the phonological expectancy violation due to the acoustic variations. The third and more reasonable explanation was that the N400 effect as evoked by the be i-construction based semantic violation. As mentioned in Section 2, the rating test revealed that when the head noun was inanimate, the complex noun phrases with be i were significantly less congruent than those without be i. Previous results also showed that, in the be i sentence, when the undergoer argument was inanimate, a neurophysiological response indexing a semantic mismatch (N400; see Frisch & Schlesewsky, 2001; Philipp et al., 2008) occurred. Consistent with previous results, the N400 effect in the GreatAccent and Accent conditions was also elicited by the be i-construction based semantic violation. In the Accent and GreatAccent conditions, the listeners might allocate more attention to the accented information and process them more deeply, hence the semantic inappropriateness being successfully detected. On the contrary, when the corresponding information was not accented, the listeners might allocate relatively less attention to it and take a shallow processing. Moreover, although the rating test revealed a highly significant difference between semantic congruence (1.25) and incongruence (.33) conditions, the semantic incongruence is quite subtle since the value of .33 was just below zero. The shallow processing induced by de-accentuation and the subtleness of the semantic incongruence might lead to a failure in detecting the semantic inappropriateness. The dissociation of the N400 effects for the information with and without pitch accent demonstrated that, during on-line speech processing, accentuation can modulate listener’s temporally selective attention and influence the depth of subsequent semantic processing. The shallow processing of the de-accented words in the current study was consistent with previous studies. A variety of results in psycholinguistics have shown that language processing can be shallower than is commonly assumed. Sometimes, the full meaning of a word is not incorporated into the interpretation of a sentence (Ferreira, Bailey, & Ferraro, 2002; Sanford & Sturt, 2002). A well known example is the ‘‘Moses illusion’’ (Erickson & Mattson, 1981). When people were asked How may animals of each kind did Moses take on the ark?, quite a few people gave a answer ‘two’, failing to notice that it was Noah, not Moses who put animals on the park. Recently some ERP studies also found that, sometimes, the reader or listener could not detect the semantic anomaly during language comprehension (Nieuwland & Van Berkum, 2005; Wang et al., 2009). For example, in Nieuwland and Van Berkum’s study, a human character central to the story at hand (e.g., ‘‘a tourist’’) was suddenly replaced by an inanimate object (e.g., ‘‘a suitcase’’). Instead of the standard N400 effect, anomalous words elicited a positive ERP effect from about 500–600 ms onwards. The absence of an N400 effect suggests that subjects did not immediately notice the semantic anomaly (Nieuwland & Van Berkum, 2005). Recently, Wang et al. investigated how information structure influences the depth of semantic processing. In their study, following different questions in whquestion–answer pairs (e.g. What kind of vegetable did Ming buy for cooking today?/Who bought the vegetables for cooking today?), the answer sentences (e.g., Ming bought eggplant/beef to cook today.) contained a critical word, which was either semantically congruent (eggplant) or incongruent (beef), and either focus or non-focus. The results revealed a significant N400 effect for focus; however, for non-focus, there was no significant difference between the semantic congruent and semantic incongruent conditions, suggesting that the reader did not detect the semantic

incongruence on non-focus words (Wang et al., 2009). That is, all of the above studies showed that, during language comprehension, the processers sometimes take a shallower processing without maintaining a full analysis. The current results were consistent with previous results by showing that the listeners could not detect the subtle semantic mismatch induced by ‘‘be i’’ construction. More importantly, the current results also revealed that accentuation could modulate the depth of semantic processing, and the semantic mismatch effect (N400 effect) can be detected successfully in the Accent and GreatAccent conditions. Another important question addressed in the current study is the time point at which accentuation begins to influence temporally selective attention during on-line speech processing. The present results showed that, for greatly accented information (GreatAccent), semantically incongruent words evoked a larger negative deflection than the semantically congruent words in the 110–190 ms window latency after the acoustic onset of the CW; however, for the de-accented (DeAccent) and generally accented (Accent) information, this early negative effect disappeared. One possible explanation for this early negative effect is an exogenous N1 effect due to differences in the acoustic properties of congruent and incongruent words. However, our finding of enhanced early negativity in the semantically incongruent condition should not be simply attributed to physical feature changes. First, the peak latency (around 170 ms) of the early negative effect was later than that of classic N1, which usually peaks between 80 and 120 ms and returns to baseline at 160–180 ms post stimulus onset (Woods, 1995). Second, for the GreatAccent condition, the CWs in the congruent and incongruent conditions were the same written words and matched on acoustic characteristics, such as intensity, length, and pith variation. Third, as reported in the results, although the acoustic cues (duration, pitch maximum, and pitch range) of the different kinds of accentuation differed from each other, there was no significant difference in the early negative deflections elicited by them when the CWs were semantically congruent. An extraneous N1 account of the early negativity is farfetched. An alternative explanation is that it reflected the expectancy violation of the speech signal. As mentioned in the introduction, previous studies have suggested that the expectancy violations in auditory signals can be detected rapidly, and manifested by an early negativity in brain potentials (Brattico et al., 2006; Connolly et al., 1995; Connolly & Phillips, 1994; Hagoort & Brown, 2000; Hahne & Friederici, 1999; Magne et al., 2006; Newman & ¨ et al., 2004). In the present study, the early Connollt, 2009; Schon negative effect in the GreatAccent condition indicated that, at the early perceptual stage, the greatly accented information had captured relatively more attention, thus the listeners could detect the mismatch between the actual spoken input and the initial phoneme expectations based on the sentence context. The absence of the early negative effect in the DeAccent and Accent conditions implied that, during the initial perceptual processing, the listener did not allocate enough attention to the de-accented or generally accented information, hence not detecting the initial phoneme mismatch. The dissociation of the early negativity effects for those differently accented information suggested that accentuation might begin to modulate attention allocation at the early perceptual stage, around 110–190 ms after the acoustic onset of the CW. For example, Astheimer and Sanders (2009) found that speech-like probes presented within the first 150 ms of a word elicited a larger N1 than probes presented before a word onset or at random control times, demonstrating that attention was directed to times that contain word-initial segments during speech processing. The present study demonstrated that, as did the word onset, accentuation can also rapidly modulate temporally selective attention during on-line speech processing. The current results also provided information about how the degree of accentuation influences the effect of accentuation on

Please cite this article as: Li, Xiao-qing, & Ren, Gui-qin How and when accentuation influences temporally selective attention and subsequent semantic processing during on-line.... Neuropsychologia (2012), http://dx.doi.org/10.1016/j.neuropsychologia.2012.04.013

Xiao-qing Li, Gui-qin Ren / Neuropsychologia ] (]]]]) ]]]–]]]

attention allocation and subsequent semantic processing. The current results revealed that the scalp distribution of the N400 effects differed between the Accented and GreatAccent conditions, with the former focusing on the parietal–occipital areas and the later widely distributing across the scalp. When comparing the N400 effects in the two conditions, it was also found that the N400 effects in the GreatAccent conditions were larger than that in the Accent conditions over the frontal–central electrodes. The different topography of the N400 effects in the Accent and GreatAccent conditions suggested that, relative to the accented information (Accent), the listeners might allocate more attention to the greatly accented information (GreatAccent), hence its information being more deeply processed. In addition, the current results also showed that the N400 effect (semantic congruence effect) in the GreatAccent condition started around 50 ms earlier that that in the Accent condition, which indicated that the semantic information in the GreatAccent condition was not only processed in a more elaborative way but also faster than that in the Accent condition. Therefore, the effect of accentuation on temporally selective attention was not all-or-none. The amount of attention allocated and the depth (or speed) of subsequent semantic processing could change with the degree of accentuation gradually. In summary, the results of previous behavioral studies have already shown that accentuation can modulate listener’s selective attention during language processing (Cutler, 1976; Sanford et al., 2006). However, as mentioned in the introduction, the previous studies could not provide solid evidence for the role of accentuation in modulating attention allocation due to the confounding factors, such as acoustic cues and information induced by the discourse context. Moreover, previous studies usually used delayed second task, hence not reflecting the natural process of on-line language comprehension. The current study, with the help of EEG and by controlling the interference effect of acoustic cues and the contextual information, proved that accentuation indeed can guide temporally selective attention during on-line speech processing. More importantly, the current study also furthers our understanding of the effect of accentuation on attention allocation by showing that: during online speech comprehension, accentuation can rapidly modulate temporally selective attention at the early perceptual stage; the attention captured by accentuation can influence the depth or the speed of subsequent semantic processing; the effect of accentuation on attention allocation and subsequent semantic processing can change with the degree of accentuation gradually.

11

such as accentuation and word onset, but also some top-down factors, such as syntactic or semantic predictability, might be able to modulate the temporally selective attention. In future studies, it will be important to explore, during on-line speech processing, how the different kinds of factors interact to guide the listeners’ temporally selective attention, and how the captured attention influences further language comprehension processes. 4.3. The interaction between the processing of prosodic information (accentuation) and the processing of other linguistic information During on-line spoken language comprehension, how and when the prosodic information interacts with other linguistic information types, such as semantic information? With respect to the stage of information processing at which different kinds of linguistic information types are integrated, two main classes of psycholinguistic models have been proposed. The two-stage models (e.g., Clifton et al., 2003; Frazier & Fodor, 1978; Friederici, 2002) assume a later interaction between syntactic and semantic processes (reflected by the P600 response) but explicitly remain open with respect to the temporal structure of the processing of prosodic information. The interactive models (Hagoort, 2003; Trueswell, Tanenhaus, & Kello, 1993), however, predict that different information types can be used immediately to co-determine the interpretation of linguistic expressions when they become available. The current results demonstrated clearly that accentuation can immediately influence the depth or the speed of semantic processing as indicated by the N400 effects in the different conditions of accentuation, lending further support to the interactive model. As to the time characteristics of the interaction between accentuation and other information types, our results are consistent with previous studies which found immediate ERP effect (N400 or P300) when there was mismatch between accentuation and information structure induced by preceding context. Furthermore, our data also extended previous results by showing that accentuation can interact with other linguistic information at the early perceptual stage as indicated by the early negative effect in the window latency of 110–190 ms. That is, accentuation, as a kind of supra-segmental information, has an immediate effect on the early perceptual processing of the segmental information in the speech signal. In short, as seen from the current and previous results, during on-line spoken language comprehension, accentuation can immediately interact with the processing of other linguistic information, even at the early perceptual stage.

4.2. The role of accentuation in conveying semantic meaning and in modulating temporally selective attention Acknowledgments What is the specific mechanisms by which accentuation affects spoken language comprehension? One account is that accentuation conveys semantic meaning directly, related to the focus distribution of the sentence (Dahan et al., 2002; Gussenhoven, 1983; Selkirk, 1995; Terken & Noteboom, 1987; Li, Yang, & Hagoort, 2008b). When the sentence-level meaning indicated by accentuation is consistent with the preceding discourse context, speech processing is facilitated; otherwise, there is difficulty in integrating the current information into the discourse context (e.g. Bock & Mazzella, 1983; Dahan et al., 2002; Hruska et al., 2000; Johnson et al., 2003; Li et al., 2008a; Magne et al., 2005; Terken & Noteboom, 1987). However, in the current study, when the effect of accentuation violation was controlled by using isolated sentences, we still found that accentuation can influence the depth and the speed of semantic processing. The current results suggested that accentuation influences spoken language comprehension not only by conveying semantic meaning directly but also by modulating general cognitive processes, such as temporally selective attention. However, during spoken language comprehension, not only the bottom-up factors,

This research was supported by Grants from the National Natural Science Foundation of China (30800296 and 31100732).

Appendix. study

Sample complex noun phrases used in the current

(the signpost raised by XiaoSu) (the list of names confessed by XiaoKong) (the paper praised by LaoZhao) (the seal gotten by cheating by XiaoGang ) (the movie extolled by LaoLiang) (the perfume bragged about by XiaoJiang) (the watermelon selected by LaoWang) (the business card picked up by LaoDing) (the sweets brought here by aunt) (the works on which the boss has a good prospect)

Please cite this article as: Li, Xiao-qing, & Ren, Gui-qin How and when accentuation influences temporally selective attention and subsequent semantic processing during on-line.... Neuropsychologia (2012), http://dx.doi.org/10.1016/j.neuropsychologia.2012.04.013

12

Xiao-qing Li, Gui-qin Ren / Neuropsychologia ] (]]]]) ]]]–]]]

(the cement transported here by LaoZhao) (the stretcher carried here by them) (the eggs delivered here by CaoMa) (the machine attended to by LaoShen) (the files transferred here by LaoZhou) (the beef bought by LaoSong) (the medicine left by LaoRen) (the goods train robbed by LiuFeng) (the bridal chamber dressed up by them) (the key found by WangYan) (the equipment recommended by LoaFan) (the special train dispatched here by the minister) (the watch redeemed by the class monitor) (the project paid attention to by LaoXia) (the bullet hided by XiaoHu) (the balloon hit by XiaoCang) (the law case exposed by XiaoSui) (the jewels wrapped up by sister) (the cloth-wrappers carried on the back by ZhaoJie) (the shoes taken care of by LaoLiang)

References Astheimer, L. B., & Sanders, L. D. (2009). Listeners modulate temporally selective attention during natural speech processing. Biological Psychology, 80, 23–34. Birch, S., & Rayner, K. (1997). Linguistic focus affects eye movements during reading. Memory and Cognition, 25(5), 653–660. Bock, J. K., & Mazzella, J. R. (1983). Intonational marking of given and new information: Some consequences for comprehension. Memory and Cognition, 11, 64–76. ¨ anen, ¨ Brattico, E., Tervaniemi, M., Na¨ at R., & Peretz, I. (2006). Musical scale properties are automatically processed in the human auditory cortex. Brain Research, 1117, 162–174. Chapman, R. M., & McCrary, J. W. (1995). ERP component identification and measurement by principal components analysis. Brain and Cognition, 27, 288–310. Chen, Y. (2003). The phonetics and phonology of contrastive focus in standard Chinese. Ph.D. Dissertation. Stony Brook: Stony Brook University. Chen, Y. (2006). Durational adjustment under corrective focus in Standard Chinese. Journal of Phonetics, 34, 176–201. Chen, Y. Y., & Gussenhoven, C. (2008). Emphasis and tonal implementation in standard Chinese. Journal of Phonetics, 36, 724–746. Chwilla, D. J., Brown, C. M., & Hagoort, P. (1995). The N400 as a function of the level of processing. Psychophysiology, 32, 274–285. Clifton, C. J., Traxler, M. J., Taha Mohamed, T., Williams, R. S., Morris, R. K., & Rayner, K. (2003). The use of thematic role information in parsing: Syntactic processing autonomy revised. Journal of Memory and Language, 49, 317–334. Connolly, J. F., & Phillips, N. A. (1994). Event-related potential components reflect phonological and semantic processing of the terminal word of spoken sentences. Journal of Cognitive Neuroscience, 6, 256–266. Connolly, J. F., Phillips, N. A., & Forbes, K. A. K. (1995). The effects of phonological and semantic features of sentence-ending words on visual event-related brain potentials. Electroencephalography and Clinical Neurophysiology, 94, 276–287. Cutler, A. (1976). Phonome-monitoring reaction time as a function of preceding intonation contour. Perception and Psychophysics, 20, 55–60. Cutler, A., & Fodor, J. A. (1979). Semantic focus and sentence comprehension. Cognition, 7, 49–59. Dahan, D., Tanenhaus, M. K., & Chambers, C. G. (2002). Accent and reference resolution in spoken-language comprehension. Journal of Memory and Language, 47, 292–314. Erickson, T. D., & Mattson, M. E. (1981). From words to meaning: A semantic illusion. Journal of verbal learning and verbal behavior, 20(5), 540–551. Fan, X. (2006). The semantic characteristics of the verb in Bei-sentence. Yangtze River Academic, 02, 79–89. Ferreira, F., Bailey, K. G. D., & Ferraro, V. (2002). Good-Enough representation in language comprehension. Current Direction in Psychological Science, 11(1), 11–15. Frazier, L., & Fodor, J. (1978). The sausage machine: A new two-stage parsing model. Cognition, 6, 291–325. Friederici, A. D. (2002). Towards a neural basis of auditory sentence processing. Trends in Cognitive Neuroscience, 6, 78–84. Frisch, S., & Schlesewsky, M. (2001). The N400 indicates problems of thematic hierarchizing. Neuroreport, 12, 3391–3394. Gussenhoven, C. (1983). Focus, mode and the nucleus. Journal of linguistics, 19, 377–417.

Hagoort, P. (2003). Interplay between syntax and semantics during sentence comprehension: ERP effects of combining syntactic and semantic violations. Journal of Cognitive Neuroscience, 15, 883–899. Hagoort, P., & Brown, C. M. (2000). ERP effects of listening to speech compared to reading: The P600/SPS to syntactic violations in spoken sentences and rapid serial visual presentation. Neuropsychologia, 38, 1531–1549. Hahne, A., & Friederici, A. (1999). Electrophysiological evidence for two steps in syntactic analysis: Early automatic and late controlled processes. Journal of Cognitive Neuroscience, 11, 194–205. Harris, K. C., Mills, J. H., & Dubno, J. R. (2007). Electrophysiologic correlates of intensity discrimination in cortical evoked potentials of younger and older adults. Hearing Research, 228, 58–69. Hruska, C., Alter, K., Steinhauer, K., & Steube, A. (2000). Can wrong prosodic information be mistaken by the brain?. Journal of Cognitive Neuroscience [Supplement: 122] Jia, Y., Xiong, Z. Y., & Li, A. J. (2006). Phonetic and phonological analysis of focal accents of disyllabic words in standard Chinese. In Proceedings of the 5th international symposium (ISCSLP) (pp. 55–66). Singapore: Springer Press. Jia, Y., Li, A. J., & Chen, Y. (2008). Pitch and durational patterns of five-syllable constituents in standard Chinese. Applied Linguistics (in Chinese), 4, 53–61. Johnson, S. M., Clifton, C. E., Breen, M. E., Martin, A. E., & Florak, J. M. (2003). ERP investigation of prosodic and semantic focus. New York City: Poster presented at Cognitive Neuroscience. Li, X. Q., & Yang, Y. F. (2004). The role of accentuation in spoken discourse comprehension. Acta Psychologica Sinica, 36, 393–399. Li, X. Q., Hagoort, P., & Yang, Y. F. (2008a). Event-related potential evidence on the influence of accentuation in spoken discourse comprehension in Chinese. Journal of Cognitive Neuroscience, 20, 1–10. Li, X. Q., Yang, Y. F., & Hagoort, P. (2008b). Pitch accent and lexical tone processing in Chinese discourse comprehension: An ERP study. Brain Research, 1222, 192–200. Liu, F., & Xu, Y. (2005). Parallel encoding of focus and interrogative meaning in Mandarin intonation. Phonetica, 62(2–4), 70–87. Magne, C., Aste´sano, C., Lacheret-Dujour, A., Morel, M., Alter, K., & Besson, M. (2005). On-line processing of ‘‘pop-out’’ words in spoken French dialogues. Journal of Cognitive Neuroscience, 17, 740–756. ¨ D., & Besson, M. (2006). Musician children detect pitch violations Magne, C., Schon, in both music and language better than nonmusician children: Behavioral and electrophysiological approaches. Journal of Cognitive Neuroscience, 18, 199–211. Martin, B. A., & Boothroyd, A. (2000). Cortical, auditory, evoked potentials in response to changes of spectrum and amplitude. Journal of the Acoustical Society of America, 107, 2155–2161. McCandless, G. A., & Rose, D. E. (1970). Evoked cortical responses to stimulus change. Journal of speech and hearing research, 13, 624–634. Newman, R. L., & Connollt, J. F. (2009). Electrophysiological markers of pre-lexical speech processing: Evidence for bottom–up and top–down effects on spoken word processing. Biological Psychology, 80, 114–121. Nieuwland, M. S., & Van Berkum, J. J. A. (2005). Testing the limits of the semantic illusion phenomenon: ERPs reveal temporary semantic change deafness in discourse comprehension. Cognitive Brain Research, 24, 691–701. Philipp, M., Bornkessel-Schlesewsky, I., Bisang, W., & Schlesewsky, M. (2008). The role of animacy in the real time comprehension of Mandarin Chinese: Evidence from auditory event-related brain potentials. Brain and Language, 105, 112–133. Picton, T. W., Bentin, S., Berg, P., Donchin, E., Hillyard, S. A., Johnson, R., Jr., et al. (2000). Guidelines for using human event-related potentials to study cognition: Recording standards and publication criteria. Psychophysiology, 37, 127–152. Sanford, A. J. S., Sanford, A. J., Molle, J., & Emmott, C. (2006). Shallow processing and attention capture in written and spoken discourse. Discourse Process, 42, 109–130. Sanford, A. J., & Sturt, P. (2002). Depth of processing in language comprehension: Not noticing the evidence. Trends in Cognitive Sciences, 6(9), 382–386. ¨ D., Magne, C., & Besson, M. (2004). The music of speech: Music training Schon, facilitates pitch processing in both music and language. Psychophysiology, 41, 341–349. Selkirk, E. O. (1995). Sentence prosody: Intonation, stress, and phasing. In: J. Goldsmith (Ed.), Handbook of phonological theory (pp. 550–569). Oxford: Blackwell. Shattuck-Hufnagel, S., & Turk, A. (1996). A prosody tutorial for investigators of auditory sentence processing. Journal of Psycholinguistic Research, 25(2), 193–247. Shih, C. (1988). Tone and intonation in Mandarin (pp. 83–109), 3. Working Papers of the Cornell Phonetics Laboratory. Terken, J., & Noteboom, S. D. (1987). Opposite effects of accentuation and deaccentuation on verification. Latencies for given and new information. Language and Cognitive Process, 2, 145–163. Trueswell, J. C., Tanenhaus, M. K., & Kello, C. (1993). Verb-specific constraints in sentence processing: Separating effects of lexical preference from garden paths. Journal of Experimental Psychology: Learning, Memory, and Cognition, 19, 529–553. Van Berkum, J. J. A., Brown, C. M., & Hagoort, P. (1999). Early referential context effects in sentence processing:Evidence from event-related brain potentials. Journal of Memory and Language, 41, 147–182.

Please cite this article as: Li, Xiao-qing, & Ren, Gui-qin How and when accentuation influences temporally selective attention and subsequent semantic processing during on-line.... Neuropsychologia (2012), http://dx.doi.org/10.1016/j.neuropsychologia.2012.04.013

Xiao-qing Li, Gui-qin Ren / Neuropsychologia ] (]]]]) ]]]–]]]

Van Berkum, J. J. A., Zwitserlood, P., Hagoort, P., & Brown, C. M. (2003). When and how do listeners relate a sentence to the wider discourse? Cognitive Brain Research, 17, 701–718. van Boxtel, G. J. M. (1998). Computational and statistical methods for analyzing event-related potential data. Behavior Research Methods, Instruments and Computers, 30, 87–102. ¨ S. N., & Yang, Y. F. (2002). The pitch movement of stressed syllable in Wang, B., Lu, Chinese sentences. Acta Acustica, 27(3), 234–240. Wang, L., Hagoort, P., & Yang, Y. F. (2009). Semantic illusion depends on information structure: ERP evidence. Brain Research, 1282, 50–56. Wang, L., Bastiaansen, M., Yang, Y. F., & Hagoort, P. (2011). The influence of information structure on the depth of semantic processing: How focus and pitch accent determine the size of the N400 effect. Neuropsychologia, 49, 813–820. Ward, P., & Sturt, P. (2007). Linguistic focus and memory: An eye movement study. Memory and Cognition, 35, 73–86.

13

Wastell, D. G. (1981). On the correlated nature of evoked brain activity: Biophysical and statistical considerations. Biological Psychology, 13, 51–69. Winkler, I. (2007). Interpreting the mismatch negativity. Journal of Psychophysiology, 21, 147. Winkler, I., Denham, S., & Nelken, I. (2009). Modeling the auditory scene: Predictive regularity representations and perceptual objects. Trends in ognitive Sciences, 13, 532–540. Woods, D. (1995). The component structure of the N 1 wave of the human auditory evoked potential. Electroencephalography and Clinical Neurophysiology, 102–109 [Supplements only]. Xu, Y. (1999). Effects of tone and focus on the formation and alignment of F0 contours. Journal of Phonetics, 27(1), 55–105. ¨ S. N. (2011). The influence of discourse Zhao, J. J., Yang, X. H., Yang, Y. F., & Lu, hierarchy on the acoustic manifestation of focus in standard Chinese. Chinese Journal of Acoustics, 30(4), 437–452.

Please cite this article as: Li, Xiao-qing, & Ren, Gui-qin How and when accentuation influences temporally selective attention and subsequent semantic processing during on-line.... Neuropsychologia (2012), http://dx.doi.org/10.1016/j.neuropsychologia.2012.04.013