Event-related potentials in response to violations of content and temporal event knowledge

Neuropsychologia 80 (2016) 47–55

Contents lists available at ScienceDirect

Neuropsychologia journal homepage: www.elsevier.com/locate/neuropsychologia

Event-related potentials in response to violations of content and temporal event knowledge Janna Drummer a, Elke van der Meer b,d, Gesa Schaadt b,c,n a

Potsdam Research Institute for Multilingualism, Universität Potsdam, Karl-Liebknecht-Str. 24–25, 14476 Potsdam, Germany Department of Psychology, Humboldt-Universität zu Berlin, Rudower Chaussee 18, 12489 Berlin, Germany c Department of Neuropsychology, Max Planck Institute for Human Cognitive and Brain Sciences, Stephanstr. 1 a, 04103 Leipzig, Germany d Graduate School of Mind and Brain, Humboldt-Universität zu Berlin, Luisenstraße 56, 10117 Berlin, Germany b

art ic l e i nf o

a b s t r a c t

Article history: Received 18 December 2014 Received in revised form 16 October 2015 Accepted 8 November 2015 Available online 10 November 2015

Scripts that store knowledge of everyday events are fundamentally important for managing daily routines. Content event knowledge (i.e., knowledge about which events belong to a script) and temporal event knowledge (i.e., knowledge about the chronological order of events in a script) constitute qualitatively different forms of knowledge. However, there is limited information about each distinct process and the time course involved in accessing content and temporal event knowledge. Therefore, we analyzed event-related potentials (ERPs) in response to either correctly presented event sequences or event sequences that contained a content or temporal error. We found an N400, which was followed by a posteriorly distributed P600 in response to content errors in event sequences. By contrast, we did not find an N400 but an anteriorly distributed P600 in response to temporal errors in event sequences. Thus, the N400 seems to be elicited as a response to a general mismatch between an event and the established event model. We assume that the expectancy violation of content event knowledge, as indicated by the N400, induces the collapse of the established event model, a process indicated by the posterior P600. The expectancy violation of temporal event knowledge is assumed to induce an attempt to reorganize the event model in working memory, a process indicated by the frontal P600. & 2015 Elsevier Ltd. All rights reserved.

Keywords: Event model Content event knowledge Temporal event knowledge N400 P600

1. Introduction Everyday activities rely heavily on the use of pre-established knowledge structures (e.g., Abelson, 1981; Schank and Abelson, 1977). Schank and Abelson (1977) introduced the term scripts. Scripts store our knowledge of goal-directed sequences from everyday events, such as Dining at a restaurant, Grocery shopping, or Washing the dishes (cf. Abelson, 1981; Nottenburg and Shoben, 1980). Scripts are stored in long-term memory and contain information about the events that typically belong to a specific everyday activity (Anderson, 1980; Bower et al., 1979; Den Uyl and van Oostendorp, 1980). However, it is not only knowledge about which events belong to a script (i.e., content event knowledge), but also knowledge about their stereotypical temporal order (i.e., temporal event knowledge) that is stored in long-term memory, at least for strong scripts. As stated by Abelson (1981), in strong scripts, events are linked according to the chronological order of their occurrence in real life. Correspondingly, Schank and Abelson n Corresponding author at: Department of Psychology, Humboldt-Universität zu Berlin, Rudower Chaussee 18, 12489 Berlin, Germany. Fax: þ 4930/20939361. E-mail address: [email protected] (G. Schaadt).

http://dx.doi.org/10.1016/j.neuropsychologia.2015.11.007 0028-3932/& 2015 Elsevier Ltd. All rights reserved.

(1977) as well as Barsalou and Sewell (1985) proposed that scripts are organized according to their temporal dimension (referred to as dimensional organization). An example of a strong script is Dining at a restaurant, which consists of events such as entering the restaurant, taking off one’s coat, taking a seat at the table, and so forth. Consequently, for strong scripts, an event triggers expectations about which events will come next and thus expectations concerning the temporal order of these events (Abelson, 1981; Bower et al., 1979). By contrast, in weak scripts, the appearance of events does not follow a stereotypical order. For example, the script for Visiting a circus contains events such as performance by a clown, performance by a trapeze artist, and performance by a lion tamer, but no information about a stereotypical temporal order (Abelson, 1981). Supporting the concept of temporal event knowledge, for strong scripts, it has been shown that event sequences presented in the correct chronological order are processed faster and with less effort than event sequences presented in an incorrect chronological order (Raisig et al., 2012, 2007, 2010). In order for an individual to adequately perform everyday activities, the individual must plan appropriate actions and anticipate future events rather than merely reacting to current sensory information. The event segmentation theory by Zacks et al. (2007)

48

J. Drummer et al. / Neuropsychologia 80 (2016) 47–55

provides insights into how predictions about future events are generated. According to this approach, predictions are derived from an event model, which is working memory's representation of the currently ongoing event, based on event knowledge stored in long-term memory (i.e., scripts). This representation is relatively stable, as long as the predictions generated by the event model match the information that is provided by the sensory input. Sensory input that is not predicted (i.e., unexpected) by the event model leads to the activation of an error detection mechanism, which identifies the incorrect predictions and induces a resetting of the current event model in working memory (Zacks et al., 2007), such as model transformation (Radvansky and Zacks, 2011). Although there is evidence for content and temporal event knowledge in strong scripts (Abelson, 1981; Bower et al., 1979), until now, there has not been much information about the specific and potentially different mechanisms involved in the accessing and processing of prediction errors. The event-related potential (ERP) method is thought to be a suitable approach for addressing these issues as ERPs can be used to distinguish between different cognitive processes, while at the same time providing information about their time course (Kutas and Federmeier, 2000). A large body of research using ERPs has shown that the processing of an unexpected event is accompanied by a negative deflection with a centro-parietal distribution at approximately 400 ms following the onset of the critical stimulus (Kutas and Hillyard, 1984). This ERP is referred to as the N400 effect and has been interpreted to index access to lexical and conceptual longterm memory representations (for a review, see Kutas and Federmeier, 2000; Lau et al., 2008). The N400 component is sensitive to lexical-semantic processing on the levels of words, sentences, and discourse (Kutas and Hillyard, 1984; Van Berkum et al., 1999). Moreover, the N400 has been shown to be sensitive to the semantic processing of everyday activities. Metusalem et al. (2012) demonstrated that, in response to a semantically incorrect word, the N400 was less pronounced if this word still referred to the context of an everyday activity that was introduced before, compared with a completely unrelated word. The authors presented participants with three sentences, describing an everyday activity such as Going to the dentist. The last sentence contained a critical word that was (a) semantically correct and part of the everyday activity (PAIN), (b) semantically incorrect but still part of the everyday activity (DENTIST), or (c) not part of the everyday activity (DRIVER). In the last two conditions, the processing of the critical word was accompanied by the elicitation of an N400; however, this effect was reduced for words that belonged to the everyday activity (DENTIST). This indicates that the N400 response is sensitive to the match between a (semantically incorrect) word and the context of an everyday activity. In addition, a second ERP (i.e., the P600 effect) has been shown to be relevant for processing unexpected events. It is a positive ERP deflection that occurs at approximately 600 ms after the onset of the critical stimulus. The P600 was originally interpreted as an index of syntactic processing difficulties in sentence processing (Friederici et al., 1996) and as an index for the syntactic reanalysis of the structure of a sentence (Friederici et al., 1993). More recent studies, however, have described the P600 as an indicator of broader types of reanalysis processes, such as semantic reanalysis (Nieuwland and Van Berkum, 2005). Thus, the N400 and the P600 seem to be suitable for investigating distinct mechanisms involved in the processing of content and temporal event knowledge. The present study was designed to investigate content and temporal event knowledge by analyzing the N400 and P600 ERPs in response to content violations and temporal order violations. Following Raisig et al. (2007, 2010, 2012)'s approach, we presented participants with the header of a script (e.g., Dining at a restaurant) followed by an event triplet (e.g., entering the restaurant, taking off

one's coat, taking a seat at the table). The event triplet (a) was correct (i.e., events belonged to the script and were presented in the correct temporal order), (b) contained a content error (i.e., one event did not belong to the script), or (c) contained a temporal error (i.e., events were presented in the wrong temporal order). After the last event of the triplet was presented, participants had to decide whether the event triplet was correct or not. If different processes underlie the detection of content errors versus temporal errors in event sequences, different ERPs should accompany the processing of these different kinds of violations of event sequences. On the basis of the literature cited above, we predicted that content errors would induce an N400 effect, indexing the detection of an unexpected event concerning the specific event model. By contrast, we did not expect an N400 in response to temporal errors in event sequences because these events still formed a part of the specific event model. However, we hypothesized that temporal errors would induce a P600 effect because a transformation and a reanalysis of the event model in terms of its temporal order might be induced. As the P600 occurs later in processing than the N400, this later activation of temporal event knowledge might also be reflected in longer response times for detecting a temporal error in event sequences compared with shorter response times for detecting a content error in event sequences. A further aim of our study was to shed light on a controversial debate concerning the dimensional organization of scripts (Barsalou and Sewell, 1985; Bower et al., 1979; Galambos and Rips, 1982; Nottenburg and Shoben, 1980). As already mentioned, scripts were proposed to be organized chronologically in a “giant causal chain” (Schank and Abelson, 1977, p. 41) that mirrors the temporal directionality of real-life events (i.e., dimensional organization) (cf. Freyd, 1987; Friedman, 2002). During the retrieval of strong scripts, events that belong to the script are automatically activated according to their correct chronological order (cf. van der Meer et al., 2002). While events that belong to a script (i.e., by content) should be verified easily by evaluating the general match between the event and the activated event model, the verification of the temporal order should require a more complex evaluation process. Here, not only the match between the event and event model but also the global sequence of events needs to be evaluated in terms of its chronological order. As a consequence, the process of temporal error detection requires more information processing and should therefore be more complex than the detection of content errors. While some studies have supported this notion of a dimensional organization of scripts (cf. Barsalou and Sewell, 1985; Raisig et al., 2007), other studies have not found supporting evidence (Bower et al., 1979; Nottenburg and Shoben, 1980). Raisig et al. (2007) argued that these contradictory studies have used stimulus material that consisted of either only two events or a script header (e.g., Dining at a restaurant) and one event. Thus, they mainly stressed the retrieval of content event knowledge but did not focus on temporal aspects of event knowledge. In order to analyze the dimensional organization of scripts, we used event triplets and varied the point of violation: Incorrect events could appear at either the second (early violation) or the third (late violation) triplet position. According to a dimensional organization of scripts, the point of violation should have an effect on the processing of temporal errors in event sequences. The temporal error should be detected and processed more easily for late compared with early violation points because more events have been retrieved and, thus, more information about the global sequence of events is available when arriving at the third triplet position compared with arriving at the second triplet position. As the decision about a content error does not demand this global sequence information, the point of violation should not have an impact on the processing

J. Drummer et al. / Neuropsychologia 80 (2016) 47–55

49

Table 1 The resulting six experimental conditions (three for critical events presented at the early point of violation, three for critical events presented at the late point of violation) and two filler conditions for the script “Dining at a restaurant.” Events

1

2

3

Early point of violation Correct das Restaurant betreten (entering the die Garderobe ablegen (taking off one's coat) restaurant) Content error das Restaurant betreten (entering the einen Brieftext schreiben (to write a text for restaurant) a letter) Temporal error die Speisekarte geben lassen (being given the die Garderobe ablegen (taking off one's coat) menu) Filler die Speisekarte geben lassen (being given the die Speisen bestellen (ordering the food) menu) Late point of violation Correct das Restaurant betreten (entering the die Garderobe ablegen (taking off one's coat) restaurant) Content error das Restaurant betreten (entering the die Garderobe ablegen (taking off one's coat) restaurant) Temporal error das Restaurant betreten (entering the die Speisen bestellen (ordering the food) restaurant) Filler das Restaurant betreten (entering the die Speisen bestellen (ordering the food) restaurant)

die Speisen verzehren (eating the food) die Speisen verzehren (eating the food) die Speisen verzehren (eating the food) die Speisen verzehren (eating the food)

am Tisch Platz nehmen (taking a seat at the table) den Brief einwerfen (to post the letter) am Tisch Platz nehmen (taking a seat at the table) die Speisen verzehren (eating the food)

note: Critical events are presented in bold. Events were presented in the German language. Translations are provided in parentheses.

of content errors. Concerning ERPs, a P600, indicating temporal error processing, was expected to be stronger for late compared with early violations. For content errors, the point of violation was not expected to have an effect, such that the N400 and P600 were not expected to differ between early and late violation points.

2. Method 2.1. Participants Twenty-five native German speakers (18 female) participated in the present experiment. Participants came from various professions and had different educational levels. Their ages ranged from 20–36 years (mean age¼26.36, SD ¼ 4.01). All participants had normal or corrected-to-normal vision and were right-handed. Participants were reimbursed for their participation (8.00 € per hour). The study followed American Psychological Association (APA) standards in accordance with the Declaration of Helsinki from 1964 (World Medical Association, 2013). 2.2. Stimulus material In order to ensure that the scripts retrieved by each participant during our experimental task were similar, it was fundamentally important to resort to previously rated and standardized event sequences. Thirty such standardized scripts of everyday activities were provided by Raisig et al. (2009) and were used for stimulus generation. These scripts each consisted of sequences of about 15 events, which were validated concerning the characteristic of strong scripts, namely, a stereotypical temporal order. Event triplets (three events) were selected according to Raisig et al. (2007, 2010)'s procedure. There were a total of six experimental conditions for each script. First, two correct event triplets for each of the 30 scripts were generated, such that both triplets consisted of events in the correct temporal order. This resulted in 60 correct triplets. Second, based on these correct triplets, 60 triplets for the content error condition and 60 triplets for the temporal error condition were generated. In 30 out of these 60 triplets, the violation occurred at the second event (i.e., early violation point), and in the other 30 triplets, the violation occurred at the third event (i.e., late violation point).

As events differ in the number of words required to describe them properly, it was crucial to ensure that this difference would not cause the ERPs to differ between conditions. The following procedure ensured that the exact same events were presented in each condition, appearing as a correct event, a content error event, or a temporal error event depending on the surrounding event context. For the content error condition, the second or third event (i.e., critical event for ERP analysis) of the correct triplet was replaced by an event that did not belong to the script at hand but belonged to one of the other 29 scripts. For the temporal error condition, not the critical but the event before the critical event was exchanged for an event belonging to the script at hand but temporally succeeding the critical event in reality and resulting in a temporal error. It is important to note that this event did not form part of the correct condition. Further, to balance the number of correct and incorrect items, an additional correct filler condition consisting of 60 event triplets was generated. For the correct filler condition, the temporal-error condition served as a basis. The second event for the early violation point condition and the third for the late violation point condition were replaced by an event from the same script that was appropriate in terms of its temporal order but did not belong to the correct condition mentioned above. Thus, 30 correct triplets were created for the early and 30 correct triplets were created for the late experimental variation as a filler condition, resulting in a total of 60 correct filler triplets. This procedure led to a total of 240 event triplets. For an example of all four conditions for the early and late violation points, see Table 1. It is important to note that only the correct, content error, and temporal error conditions were used for data analysis, and ERPs were analyzed only in response to the critical event, which was at either the second or the third position of the event triplet. Before the experiment was conducted, we wanted to ensure that the 240 triplets that we generated were judged accordingly as (a) correct, (b) containing a content error, or (c) containing a temporal error in a large sample. An online study was conducted to validate the triplets. We created eight lists, each consisting of the 30 scripts in one of the eight triplet versions (cf. Table 1). A total of 320 native German speakers (mean age¼25.10 years; SD ¼4.45; 69.7% female) participated (40 per list). Participants were presented with a script header (e.g., Dining at a restaurant) and one of the event triplet versions (i.e., correct, content error, temporal error, correct filler). They were asked to place the triplets into one of the following categories: correct, wrong event (content

50

J. Drummer et al. / Neuropsychologia 80 (2016) 47–55

error condition), or wrong temporal order (temporal error condition). Overall, almost 93% of the triplets were correctly assigned to their corresponding category. This high percentage of correct answers indicates that the generated triplet versions could be identified as correct, containing a content error, or containing a temporal error. However, it has to be mentioned that the classification of triplets did differ significantly (F(3,236) ¼4.13; p o0.05). Post hoc pairwise comparisons revealed that content error triplets (M ¼97.33%) were more often classified correctly than the other three triplet versions (correct: M¼91.21%; temporal error: M¼ 93.58%; correct filler: M¼91.88%). The other three triplet versions did not differ significantly from each other. 2.3. Procedure Participants were asked to fill out a short questionnaire on demographic information (gender, age, and handedness), were informed about the procedure, and provided written consent for their participation. Following electrode preparation, participants were seated in front of a 15” monitor screen (display resolution: 1024  768). The distance between the participant and the monitor screen was 75 cm. Participants received standardized instructions before the experiment started. As ERPs are influenced by the repetition of stimuli, especially when presented close to each other (Besson et al., 1992; Neville et al., 1986), we tried to ensure a maximal distance between the presentation of the same script in the same function (i.e., correct vs. error) by dividing the experiment into two blocks. In block 1, the critical event and thus the point of violation occurred at the second triplet position (i.e., early violation point), and in block 2, the critical event and thus the violation point occurred at the third triplet position (i.e., late violation point). Questioning the participants after testing verified that they had no awareness of the difference between blocks 1 and 2. Across participants, the order of blocks was counterbalanced. Each block consisted of 120 trials. The experiment began with eight practice trials with feedback. In order to ensure that the task was understood, participants were required to answer at least seven of the eight practice trials correctly before the actual experiment began. Only one participant had to repeat the practice trials once. All other participants met this criterion on their first attempt. One block contained four subblocks, each consisting of 30 trials. To further minimize effects of repetition, the following pseudo-randomization procedure ensured that the same scripts, in different conditions, were presented with the maximum possible distance between them: One of the four versions of each script was presented in the first subblock (trials 1–30), one version in the second sub-block (trails 30– 60), one version in the third sub-block (trials 60–90), and one version in the fourth sub-block (trials 90–120). The four experimental conditions (correct, correct filler, content error, temporal error) were equally distributed across the sub-blocks. Stimulus presentation was randomized within each sub-block. The order of presentation of sub-blocks was counterbalanced across participants. Fig. 1 illustrates the presentation of one trial. Each trial began with an inter-stimulus interval (ISI) of 1000 ms, followed by a fixation cross, which was presented for 500 ms. After the fixation cross, there was an ISI of 300 ms, followed by the script header (e.g., Dining at a restaurant), which was presented for 2000 ms in a 30 pt. Arial font type. After an ISI of 300 ms, the three events in an event triplet were successively presented (30 pt. Arial font type) for 1500 ms each. There was an ISI of 300 ms between the presentation of the events and an ISI of 100 ms after the last event. After each trial, participants had to decide whether the event triplet was correct or not by pressing either the left or the right answer button. Participants were instructed to respond as quickly and correctly as possible. The

Fig. 1. Schematic illustration of trial procedure. The example is provided for the event “Dining at a restaurant” in the correct condition. The sequence of experimental slides is shown. Information about the inter-stimulus interval (ISI) is provided on the left side of the slides. Information about presentation time is provided on the right side of the slides.

assignment of the answer buttons (right/left) to the type of answer (correct/wrong) was counterbalanced across participants. There was a break between the blocks and within each block after 60 trials. Participants could decide when to continue with the experiment at their own pace. In addition, participants could ask for additional breaks if necessary, but this never happened. The experiment was programmed and presented using Presentations software (Version 14.9, www.neurobs.com). 2.4. Data recording Forty-four Ag/AgCl cap-mounted electrodes (Easy Cap GmbH, Germany) were positioned according to the 10–20 International System of Electrode Placement (Jasper, 1958). The electrode sites were: FP1, FP2, AF3, AF4, F9, F7, F3, FZ, F4, F8, F10, FT7, FC5, FC1, FC2, FC6, FT8, T7, C3, CZ, C4, T8, TP7, CP5, CP1, CP2, CP6, TP8, P7, P3, PZ, P4, P8, PO9, PO7, PO3, PO4, PO8, PO10, O1, OZ, O2, M1, and M2. To measure horizontal and vertical eye-movements, the EOG (Vþ , V  , Hþ, H  ) was registered by placing additional electrodes above and below the right eye and on the outer canthus of each eye. In addition, an electrode positioned between the eyebrows served as a ground electrode. The EEG and EOG were amplified with the Refa system (Twente Medical Systems International B.V.) and digitized online at a rate of 500 Hz (anti-aliasing low pass filter of 135 Hz). QRefa Acquisition Software, Version 1.0 beta (Max Planck Institute for Human Cognitive and Brain Science, Leipzig, Germany) was used for data recording. Each electrode was referenced online to the CZ. Impedances were kept below 5 kΩ. 2.5. Data processing and analyses Data analyses were carried out using the EEP software package (Max-Planck Institute for Human Cognitive and Brain Sciences, Leipzig, Germany). Electrode signals were re-referenced off-line to the average of the two mastoids (M1, M2). In order to remove most slow drifts and muscle artifacts, a 0.1–30 Hz bandpass filter was applied to the data. Erroneous trials (i.e., wrong answers) were identified and excluded from further analyses. In a next step, trials with an amplitude that fell outside the range of þ/  80 μV within a sliding window of 200 ms were considered to contain artifacts and were excluded. In addition, prototypical blinks for each subject were identified and used as an individual template to detect and exclude further blinks. After the blink correction, the mean of

J. Drummer et al. / Neuropsychologia 80 (2016) 47–55

the trials that were included across all experimental conditions ranged from 91.95–94.32%. Experimental conditions did not differ significantly in the number of trials that were included (Fo 1). The EEG data were averaged per person and per condition between 200 and 1000 ms relative to the stimulus onset (i.e., critical event either at the second or third triplet position). Baseline correction was applied to a period from  200 to 0 ms. Finally, grand averages were calculated across subjects for each critical condition (i.e., correct, content-error, temporal-error) but not for the correct filler condition. Due to severe artifacts, four participants had to be excluded from further analyses. 2.6. Statistical analyses The Statistical Package for the Social Sciences (SPSS) Software Version 19 (IBM, Walldorf, Germany) was used to analyze the behavioral and EEG data. For behavioral data analyses, statistical means for both response times (RTs) and error rates (ERs) in percent were computed for all experimental conditions. RTs that exceeded 2 standard deviations (SD) above or below the mean were excluded from the analyses. Erroneous trials were excluded from the analyses of the RTs. To analyze the effect of condition and point of violation, a repeated-measures analysis of variance (ANOVA) containing the factors condition (correct, content error, temporal error) and point of violation (early, late) was computed for both the RTs and ERs. If the interaction between condition and point of violation was significant (pr 0.05), post hoc pairwise comparisons were applied. Greenhouse-Geisser corrections were utilized when needed, and effect sizes ηp² are provided. To analyze the EEG data, eight Regions of Interest (ROIs) at lateral electrode sites, four on each hemisphere, were defined. Every ROI contained 4 electrodes. The regions were: anterior-lateral (left: F7, FT7, FC5,T7; right: F8, FT8, FC6, T8), anterior-central (left: AF3, F3, FC1, C3; right: AF4, F4, FC2, C4), posterior-lateral (left: TP7, CP5, P7, PO7; right: CP6, TP8, P8, PO8), and posteriorcentral (left: CP1, P3, PO3, O1; right: CP2, P4, PO4, O2). In addition, a separate analysis was performed for central electrodes (FZ, CZ, PZ, OZ). Here, the electrode was entered as a factor (see Vos and Friederici, 2003, for a similar methodological approach). All analyses were applied to two time windows (TWs). For the N400, we chose a TW ranging from 300–500 ms; and for the P600, we chose a TW ranging from 500–900 ms. These are the TWs in which the N400 effects or P600 effects, respectively, were to be expected (Federmeier et al., 2007; Kaan et al., 2000). These TWs were defined relative to the onset of the critical event (i.e., the second or third event) of the event triplet. For each TW, a four-factor ANOVA for lateral electrode sites containing the factors condition (correct, content error, temporal error), hemisphere (left, right), region (anterior-lateral, anterior-central, posterior-lateral, posterior-central), and point of violation (early, late) was computed. Further, a three-factor ANOVA for central electrodes containing the factors condition (correct, content error, temporal error), electrode (FZ, CZ, PZ, OZ), and point of violation (early, late) was computed. Whenever interactions relevant to the research questions were significant (pr 0.05), post hoc pairwise comparisons were applied to the corresponding factor levels. Greenhouse-Geisser corrections were utilized when needed, and effect sizes ηp² are provided. To control for a potential effect of gaze behavior on the patterns of results, we computed a repeated-measures ANOVA on the electrode site, reflecting horizontal eye movements (i.e., EOGH) with the factors condition (correct, content error, temporal error) and point of violation (early, late) for each TW. We corrected the pValues with the Bonferroni correction for multiple comparisons.

51

Table 2 Mean Response Times (RTs in Milliseconds) and Mean Error Rates (ERs in %) for experimental conditions (correct, content error, temporal error) for critical events presented at the early point of violation and critical events presented at the late point of violation.

Early point of violation Late point of violation

Early point of violation Late point of violation

Correct

Content error

Temporal error

Mean RT (SD) 559 (192) 611 (223) Correct Mean ER (SD) 5.08 (0.05) 6.35 (0.04)

Mean RT (SD) 430 (142) 446 (238) Content error Mean ER (SD) 0.95 (0.06) 0.95 (0.02)

Mean RT (SD) 552 (201) 554 (201) Temporal error Mean ER (SD) 15.87 (0.13) 8.73 (0.08)

note: SD ¼Standard Deviation; SDs are presented in parentheses.

3. Results 3.1. Behavioral data Response Times (RTs): Descriptive statistics (mean, SD) for each experimental condition are presented in Table 2. A repeatedmeasures ANOVA revealed a significant main effect of condition (F (2,40) ¼16.04; p o0.001; ηp² ¼0.45). Post hoc pairwise comparisons revealed that this was due to significantly shorter RTs in the content error condition in comparison with the correct condition (p o0.001) and the temporal error condition (p o0.01). The interaction between condition and point of violation was not significant. Error Rates (ERs): Descriptive statistics (mean, SD) for each experimental condition are presented in Table 2. A repeatedmeasures ANOVA revealed a significant main effect of condition (F(2,40) ¼21.59; p o0.001; ηp² ¼0.51). Post hoc pairwise comparisons revealed that this was due to significantly lower ERs in the content error condition in comparison with the correct condition (p o0.001) and the temporal error condition (p o0.05) and significantly lower ERs in the correct condition in comparison with the temporal error condition (p o0.001). In addition, the interaction between point of violation and condition was significant (F(2,40) ¼9.31; po 001; ηp² ¼0.32). Post hoc pairwise comparisons revealed that this was due to lower ERs for late violations in comparison with early violations in the temporal error condition (p o0.01). In summary, we found shorter RTs for the content error condition compared with the other two conditions. Further, we found significantly lower ERs in the content error condition compared with the other two conditions and for the correct condition compared with the temporal error condition. Concerning the point of violation, we found lower ERs only for late violations compared with early violations in the temporal error condition. 3.2. Event-related potential data Repeated-measures ANOVAs on horizontal eye movements did not reveal any significant differences between experimental conditions for TW300–500 ms (Fo 1) or for TW500–900 ms (Fo 1). As the eye movements did not differ between experimental conditions, it is most likely the case that gaze behavior did not contribute to the following results. Fig. 2 illustrates the ERP responses to the correct and content error conditions, and Fig. 3 illustrates the ERP responses to the correct and temporal error conditions. Negative and positive deflections in the content error and temporal error conditions are always to be understood as contrasted with the correct condition. For TW300–500 ms, we found a significant main effect of condition (F(2,40) ¼7.58; p r0.01; ηp² ¼0.28) and significant

52

J. Drummer et al. / Neuropsychologia 80 (2016) 47–55

Fig. 2. (a) Event-related potentials (ERPs) across all electrodes for the correct condition (blue line) and the content error condition (red line), collapsed for early and late point of violation. (b) Zoom in on PZ electrode indicating a more posteriorly distributed N400 and P600 in response to the content error condition. (c) Topographic plots for the contrast content error minus correct condition for 300–500 ms and 500–900 ms after stimulus onset. (For interpretation of the references to color in this figure legend, the reader is referred to the web version of this article.)

interactions between condition and hemisphere (F(2,40) ¼7.14; p r0.01; ηp²¼ 0.26) and between condition, hemisphere, and region (F(6,120) ¼2.10; p r0.05; ηp²¼ 0.10) at lateral electrode sites. Post hoc pairwise comparisons revealed that these effects were due to a negative deflection that was posteriorly distributed in response to the content error condition (for detailed statistics, see Table 3). Central electrodes did not reveal a main effect or an interaction in this TW1. In summary, a more posteriorly distributed N400 was found between 300 and 500 ms for the content error condition. There was no N400 effect for the temporal error condition and no effects for point of violation. For TW500–900 ms, we found a significant interaction between condition and region (F(6,120) ¼5.92; p r0.01; ηp² ¼0.23) at lateral electrode sites. Post hoc pairwise comparisons revealed that these effects were due to a posteriorly distributed positive deflection in response to the content error condition and a more anteriorly distributed positive deflection in response to the temporal error condition (for detailed statistics, see Table 3). Central electrodes revealed a significant interaction between condition and electrode (F(6,120) ¼3.67; p r0.03; ηp²¼ 0.14). Post hoc pairwise comparisons revealed that the effect was due to a positive deflection in response to the content error condition at PZ (F 1 Please note that the results did not change when gaze behavior (i.e., EOGH) for each experimental condition was added as a covariate.

(2,19) ¼ 6.63; pr 0.01; ηp² ¼0.41) and a positive deflection in response to the temporal error condition at FZ (F(2,19) ¼5.72; pr 0.02; ηp² ¼0.38)1. In summary, we found a P600 in posterior regions for the content error condition and a more anteriorly distributed P600 for the temporal error condition. We did not find any effects of point of violation. As ERPs have been shown to be sensitive to repetition (Besson et al., 1992; Neville et al., 1986), we wanted to ensure that the results were not driven by repeatedly presenting the events in all conditions. Thus, we computed post hoc repeated-measures ANOVAs to compare the first and last blocks of the experiment. As we did not find any effect of the point of violation, we computed ANOVAs across both the early and late points of violation. We did not find significant interactions involving the experimental block factor (i.e., first block, last block) for TW300–500 or for TW500– 900. However, we still found a significant N400 (F(2,13) ¼10.50; pr 0.01; ηp²¼ 0.53) and a close-to-significant P600 (F(2,13) ¼2.22; pr 0.07; ηp²¼ .19) with posterior scalp distribution in response to the content error condition. In response to the temporal error condition, we no longer found a significant frontally distributed P600. However, the descriptive statistics showed a stronger positivity for frontal regions (mean difference wave: temporal error condition – correct condition ¼4.04) compared with posterior regions (mean difference wave: temporal error condition – correct condition ¼ 0.41).

J. Drummer et al. / Neuropsychologia 80 (2016) 47–55

53

Fig. 3. (a) Event-related potentials (ERPs) across all electrodes for the correct condition (blue line) and the temporal error condition (green line), collapsed for the early and late points of violation. (b) Zoom in on AF4 electrode indicating a more anteriorly distributed P600 in response to the temporal error condition. (c) Topographic plots for the contrast temporal error minus correct condition for 300–500 ms and 500–900 ms after stimulus onset. (For interpretation of the references to color in this figure legend, the reader is referred to the web version of this article.)

4. Discussion The main goal of the present study was to shed light on the question of which processes are involved in the retrieval of content and temporal event knowledge. More precisely, we aimed to investigate the detection of content and temporal errors in event sequences and the subsequent attempt to update the event model representation. To address this issue, we compared ERPs in response to correct, content error, and temporal error conditions. The following main findings were obtained. First, we found an N400, followed by a posteriorly distributed P600 in response to the content error condition. Second, we did not find an N400 in response to the temporal error condition, but we found a P600like pattern that was more anteriorly pronounced. Third, for the behavioral data, we found shorter RTs and lower ERs for the content error condition in comparison with the correct condition and the temporal error condition. In line with our hypothesis, the processing of the content error condition induced an N400, indicating the violation of expectations due to the mismatch between the established event model and the event containing a content error (Metusalem et al., 2012; Van Berkum et al., 1999). By contrast, the processing of a temporal error did not induce an N400; that is, it did not constitute a mismatch during this stage of processing. In line with this finding, but for word processing, Federmeier et al. (2007) argued that “The first stage, indexed by the N400, seems to be sensitive only to the

match between information contained in or implied by a sentence context and that associated with the word currently being processed” (Federmeier et al., 2007, p. 81). Accordingly, we argue that the first stage of event processing, indicated by the N400, is sensitive to the match between the event model and the currently processed event but not sensitive to temporal errors because these errors do not hamper the global match between the event and the event model. In line with our expectations, we found a P600 effect in the temporal error condition. This P600 was distributed across the frontal electrodes. The content error condition also showed a P600 but distributed across the posterior electrode sites. We assume that this different distribution of the P600 indicates the different processes involved in the reanalysis of the event model representation. In sentence processing, a posterior P600 was associated with the collapse of a mental structure (Hagoort et al., 1999) and an attempt to revise and repair (Kaan and Swaab, 2003). Therefore, it can be assumed that content error events cause a strong mismatch with the event model representation, indicated by the N400, leading to a repair attempt, that is, an attempt to integrate the new information provided by the content error event into the event model. However, this repair fails, and the event model representation collapses. For temporal errors, we found a frontal P600. Similarly, Federmeier et al. (2007) described a frontal positivity for words that are unexpected but still plausible in a certain context. Accordingly,

54

J. Drummer et al. / Neuropsychologia 80 (2016) 47–55

Table 3 Post hoc pairwise comparisons at lateral electrode sites for the N400 and P600 time windows (TWs). N400 TW

Effect

300–500 ms condition x hemisphere x region Content error left posterior-central Content error right anterior-central Content error right posterior-lateral Content error right posterior-central

Statistics

p-Value

ηp²

F (2,19) ¼ 11.10 F (2,19) ¼ 5.67 F (2,19) ¼ 21.90 F (2,19) ¼ 20.75

0.001

0.54 Negativity

0.012

0.37 Negativity

Statistics

p-Value

ηp²

F (2,18) ¼ 3.68 F (2,13) ¼ 4.61 F (2,19) ¼ 5.06

0.045

0.28 Positivity

0.023

0.33 Positivity

0.017

0.35 Positivity

Polarity of Effect

o 0.001 0.68 Negativity o 0.001 0.69 Negativity

P600 TW

Effect

Polarity of Effect

500–900 ms condition x region Content error posterior lateral Content error posterior central Temporal error anterior central

note. p-Values are Bonferroni-corrected.

events containing a temporal error are unexpected and incorrect but still correspond to the event model in terms of their content. Furthermore, findings from sentence processing associate a frontal P600 in response to syntactical errors with the overwriting of the most strongly activated mental structure (Hagoort et al., 1999). Radvansky and Zacks (2011) argue that model transformation, the most sophisticated model-updating process, “occurs when there has been a major change in an event, but it is still represented as being part of the same course of events” (Radvansky and Zacks, 2011, p. 619). Thus, the temporal error event might induce an attempt to mentally reorganize the sensory information (i.e., the event triplet) into the correct temporal order as indicated by the frontal P600. This mental reorganization succeeds, such that the reorganized “correct” event model can be compared with the order of the formerly presented event triplet, thus triggering the decision that there is an actual temporal error. As stated above, the process of model transformation is assumed to be the most sophisticated form of model updating. In line with this idea, response times were longer for the temporal error condition compared with the content error condition. It is interesting that when analyzing smaller time windows of the ERPs, we found that a significant posterior P600 in response to the content error condition already began between 600 and 700 ms after stimulus onset, whereas the frontal P600 in response to the temporal error condition became significant only after 700 ms. These findings support the assumption that temporal errors require more sophisticated processing compared with content errors. Furthermore, the frontally distributed P600 is also consistent with the notion that the established event model is represented in the frontal lobe (Zalla et al., 2000). Moreover, it has been proposed that cognitive frontal lobe networks contain story grammar, which is involved in event sequencing (Sirigu et al., 1998), an important prerequisite for the mental reorganization of the temporal order of events. The involvement of frontal areas in the processing of sequences in general has been proposed by a large body of literature (for a review, see Fiebach and Schubotz, 2006). In line with this

idea, an fMRI study by Knutson, Wood, and Grafman (2004) provided evidence that frontal brain regions are involved in the processing of event sequences. Thus, the frontally distributed P600 in the temporal error condition seems to reflect the idea that a qualitatively different process is associated with temporal order event knowledge compared with the process associated with content event knowledge. Further, this study was motivated by the aim to analyze the previously mentioned notion about the dimensional organization of scripts. As described above, scripts were assumed to be organized chronologically in a “giant causal chain” (Schank and Abelson, 1977, p. 41). During retrieval, strong events should be automatically activated according to their correct chronological order. To address this issue, erroneous events could appear at either the second or the third triplet position of the event triplets (i.e., point of violation). If scripts are dimensionally organized, temporal errors should be processed and detected more easily (indicated by an enhanced P600 amplitude, faster RT, and reduced ER) at the late point of violation compared with the early point of violation because at the late point of violation, more events have been retrieved and thus more information about the temporal order of events is available. Our ER data provided evidence for a dimensional organization, such that the error rate was reduced in the temporal error condition for the late compared with the early point of violation, a finding that is in line with other studies supporting the dimensional organization of scripts (e.g., Raisig et al., 2007). However, our ERP and RT data did not provide evidence for a dimensional organization of scripts because we did not find a significantly enhanced P600 amplitude, nor did we find shorter RTs for temporal errors that occurred at the late point of violation compared with those occurring at the early point of violation. This finding might be due to the relatively long presentation of the script header (i.e., 2000 ms). Using a sentence-probe-recognition paradigm, van der Meer et al. (2002) analyzed the processing of strong scripts by manipulating the stimulus onset asynchrony (SOA) between sentence event and probe event (900 ms, 1200 ms, 3000 ms). They showed that after 3000 ms, all events belonging to a script and their temporal order were activated (van der Meer et al., 2002). In the present study, the script header was displayed for 2000 ms, followed by an ISI of 300 ms and the presentation of the first event for 1500 ms. This presentation procedure allowed for the activation of all events in a script and their temporal order. Consequently, there were no differences in P600 and RTs for the early compared with the late point of violation. As this study is the first to investigate the processing of content and temporal order event knowledge by using event triplets as material and ERPs as a method, we wanted to ensure that participants had enough time to activate the script in order to ensure elicitation of ERPs as a first step. Thus, we decided to use a relatively long presentation of the script header (i.e., 2000 ms). In future research, ERPs in response to content errors and temporal errors should be analyzed by varying the display time of stimuli and ISIs to further address the question of when the whole script is represented in the event model in more detail. A possible limitation of the present study comes from a methodological concern. Due to a limited number of available standardized event sequences, participants were presented these events in all conditions. This repetition might be problematic because ERPs have been shown to be sensitive to repetition (Besson et al., 1992; Neville et al., 1986). We tried to minimize the effect of repetition by using a randomization procedure that ensured a delayed repetition of events, as the N400 is only affected after an immediate but not after a delayed repetition (i.e., repetition after 5 intervening words; Kim et al., 2001). It is important to note that we demonstrated that there were no significant differences

J. Drummer et al. / Neuropsychologia 80 (2016) 47–55

between the first and last experimental blocks concerning the N400 and P600 in response to content errors. For temporal errors, we no longer found a significant frontally distributed P600, which might be attributed to a reduced signal-to-noise ratio because of a reduction in the number of trials that were included. However, the descriptive statistics showed a stronger positivity for frontal regions compared with posterior regions. Thus, it seems highly unlikely that the repetition of events had an influence on our results.

5. Conclusion The present study provides valuable insight into processes that are involved in accessing content and temporal event knowledge. Our findings indicate that when a person is processing event knowledge, the N400 is sensitive to whether an event belongs to an event model. Thus, we found an N400 in response to the content error condition but not to the temporal error condition where the event still belongs to the event model with regard to its content. However, in both the temporal and content error conditions, an updating of the event model needs to be considered. We propose that for content errors, this process consists of verifying whether the new information can be integrated into the present event model. This attempt finally results in rejecting the new information as part of the event model and leads to a collapse of the event model representation, as indicated by a posterior P600. By contrast, for temporal errors, an attempt to transform the model takes place to check whether the event model can be updated in terms of its temporal order. We propose that the event triplet is reorganized mentally, as indicated by the frontal P600. This reorganized event model can then be compared with the formerly presented event triplet, which then triggers the decision that it contained a temporal error. Thus, the present study demonstrates that accessing and processing content event knowledge is qualitatively different from accessing and processing temporal event knowledge.

Acknowledgments We are grateful to Christina Rügen for her help with the data acquisition, to Dr. Susanne Raisig for supplying the stimulus material, to Dr. Stefanie Regel for helpful comments on the data analyses, to Paul Schütte for programming the experiment, and to Dorothea Reiter and Dr. Tinka Welke for many fruitful discussions. In addition, we would like to thank the reviewers for their valuable comments and Jane Zagorski for proofreading the manuscript.

References Abelson, R.P., 1981. Psychological status of the script concept. Am. Psychol. 36 (7), 715–729. Anderson, J.R., 1980. Concepts, propositions, and schemata: what are the cognitive units? Nebr. Symp. Motiv. 28, 121–162. Barsalou, L.W., Sewell, D.R., 1985. Contrasting the representation of scripts and categories. J. Mem. Lang. 24, 646–665. Besson, M., Kutas, M., Petten, C.V., 1992. An Event-Related Potential (ERP) analysis of semantic congruity and repetition effects in sentences. J. Cognit. Neurosci. 4 (2), 132–149. http://dx.doi.org/10.1162/jocn.1992.4.2.132. Bower, G.H., Black, J.B., Turner, T.J., 1979. Scripts in memory for text. Cognit. Psychol. 220 (11), 177–220. Den Uyl, M., van Oostendorp, H., 1980. The use of scripts in text comprehension. Poetics 9, 275–294. Federmeier, K.D., Wlotko, E.W., De Ochoa-Dewald, E., Kutas, M., 2007. Multiple effects of sentential constraint on word processing. Brain Res. 1146, 75–84. http://dx.doi.org/10.1016/j.brainres.2006.06.101. Fiebach, C.J., Schubotz, R.I., 2006. Dynamic anticipatory processing of hierarchical sequential events: a common role for Broca's area and ventral premotor cortex across domains? Cortex 24, 499–502.

55

Friederici, A.D., Hahne, A., Mecklinger, A., 1996. Temporal structure of syntactic parsing: early and late event-related brain potential effects. J. Exp. Psychol.: Learn. Mem. Cogn. 22 (5), 1219–1248. Freyd, J.J., 1987. Dynamic mental representations. Psychol. Rev. 94 (4), 427–438. Friederici, A.D., Pfeifer, E., Hahne, A., 1993. Event-related brain potentials during natural speech processing: effects of semantic, morphological and syntactic violations. Cognit. Brain Res. 1 (3), 183–192. http://dx.doi.org/10.1016/ 0926-6410(93)90026-2. Friedman, W.J., 2002. Arrows of time in infancy: the representation of temporalcausal invariances. Cognit. Psychol. 44, 252–296. Galambos, J. a, Rips, L.J., 1982. Memory for routines. J. Verbal Learn. Verbal Behav. 21 (3), 260–281. http://dx.doi.org/10.1016/S0022-5371(82)90604-1. Hagoort, P., Brown, C.M., Osterhout, L., 1999. The neurocognition of syntactic processing. In: Brown, C.M., Hagoort, P. (Eds.), Neurocognition of Language. Universitiy Press, Oxford. Jasper, H.H., 1958. The ten twenty electrode system of the international federation. Electroencephalogr. Clin. Neurophysiol. 10, 371–375. Kaan, E., Harris, A., Gibson, E., Holcomb, P., 2000. The P600 as an index of syntactic integration difficulty. Lang. Cognit. Process. 15 (2), 159–201. Kaan, E., Swaab, T.Y., 2003. Repair, revision, and complexity in syntactic analysis: an electrophysiological differentiation. J. Cognit. Neurosci. 15 (1), 98–110. Kim, M.-S., Kim, J.-J., Kwon, J.S., 2001. The effect of immediate and delayed word repetition on event-related potential in a continuous recognition task. Cognit. Brain Res. 11 (3), 387–396. http://dx.doi.org/10.1016/S0926-6410(01)00011-8. Knutson, K.M., Wood, J.N., Grafman, J., 2004. Brain activation in processing temporal sequence: an fMRI study. NeuroImage 23 (4), 1299–1307. http://dx.doi. org/10.1016/j.neuroimage.2004.08.012. Kutas, M., Federmeier, K.D., 2000. Electrophysiology reveals semantic memory use in language comprehension. Trends Cognit. Sci. 4 (12), 463–470. http://dx.doi. org/10.1016/S1364-6613(00)01560-6. Kutas, M., Hillyard, S.A., 1984. Brain potentials during reading reflect word expectancy and semantic association. Nature 307, 161–163. Lau, E.F., Phillips, C., Poeppel, D., 2008. A cortical network for semantics: (de)constructing the N400. Nat. Rev. Neurosci. 9 (12), 920–933. http://dx.doi.org/ 10.1038/nrn2532. Metusalem, R., Kutas, M., Urbach, T.P., Hare, M., McRae, K., Elman, J.L., 2012. Generalized event knowledge activation during online sentence comprehension. J. Mem. Lang. 66 (4), 545–567. http://dx.doi.org/10.1016/j.jml.2012.01.001. Neville, H.J., Kutas, M., Chesney, G., Schmidt, A.L., 1986. Event-related brain potentials during initial encoding and recognition memory of congruous and incongruous words. J. Mem. Lang. 25 (1), 75–92. http://dx.doi.org/10.1016/ 0749-596X(86)90022-7. Nieuwland, M.S., Van Berkum, J.J.A., 2005. Testing the limits of the semantic illusion phenomenon: ERPs reveal temporary semantic change deafness in discourse comprehension. Cognit. Brain Res. 24 (3), 691–701. http://dx.doi.org/10.1016/j. cogbrainres.2005.04.003. Nottenburg, G., Shoben, E.J., 1980. Scripts as linear orders. J. Exp. Soc. Psychol. 16 (4), 329–347. http://dx.doi.org/10.1016/0022-1031(80)90026-8. Radvansky, G., Zacks, J.M., 2011. Event perception. Wiley Interdiscip. Rev. Cogn. Sci. 2 (6), 608–620. http://dx.doi.org/10.1002/wcs.133. Raisig, S., Hagendorf, H., van der Meer, E., 2012. The role of temporal properties on the detection of temporal violations: insights from pupillometry. Cognit. Process. 13 (1), 83–91. http://dx.doi.org/10.1007/s10339-011-0413-0. Raisig, S., Welke, T., Hagendorf, H., van der Meer, E., 2007. Investigating dimensional organization in scripts using the pupillary response. Psychophysiol. 44 (6), 864–873. http://dx.doi.org/10.1111/j.1469-8986.2007.00592.x. Raisig, S., Welke, T., Hagendorf, H., van der Meer, E., 2009. Insights into knowledge representation: the influence of amodal and perceptual variables on event knowledge retrieval from memory. Cognit. Sci. 33 (7), 1252–1266. http://dx.doi. org/10.1111/j.1551-6709.2009.01044.x. Raisig, S., Welke, T., Hagendorf, H., van der Meer, E., 2010. I spy with my little eye: detection of temporal violations in event sequences and the pupillary response. Int. J. Psychophysiol. 76 (1), 1–8. http://dx.doi.org/10.1016/j. ijpsycho.2010.01.006. Schank, R.C., Abelson, R.P., 1977. Scripts, Plans, Goals, and Understanding: Inquiry into Human Knowledge Structures. Lawrence Erlbaum Associates, Hillsdale, NJ. Sirigu, A., Cohen, L., Zalla, T., van Eeckhout, P., Grafman, J., Agid, Y., 1998. Distinct frontal regions for processing sentence syntax and story grammar. Cortex 34, 771–778. Van Berkum, J.J.A., Hagoort, P., Brown, C.M., 1999. Semantic integration in sentences and discourse: evidence from the N400. J. Chronic Dis. 11 (6), 657–671. van der Meer, E., Beyer, R., Heinze, B., Badel, I., 2002. Temporal order relations in language comprehension. J. Exp. Psychol.: Learn. Mem. Cogn. 28 (4), 770. Vos, S.H., Friederici, A.D., 2003. Intersentential syntactic context effects on comprehension: the role of working memory. Cognit. Brain Res. 16, 111–122. World Medical Association., 2013. World Medical Association Declaration of Helsinki: ethical principles for medical research involving human subjects. JAMA : J. Am. Med. Assoc. 310 (20), 2191–2194. http://dx.doi.org/10.1001/ jama.2013.281053. Zacks, J.M., Speer, N.K., Swallow, K.M., Braver, T.S., Reynolds, J.R., 2007. Event perception: a mind-brain perspective. Psychol. Bull. 133 (2), 273–293. Zalla, T., Sirigu, A., Pillon, B., Dubois, B., Agid, Y., Grafman, J., 2000. How patients with Parkinson's disease retrieve and manage cognitive event knowledge. Cortex 36, 163–179.