- Email: [email protected]
Semantic parafoveal-on-foveal effects and preview benefits in reading: Evidence from Fixation Related Potentials
Brain & Language 162 (2016) 29–34
Contents lists available at ScienceDirect
Brain & Language journal homepage: www.elsevier.com/locate/b&l
Short Communication
Semantic parafoveal-on-foveal effects and preview benefits in reading: Evidence from Fixation Related Potentials P.J. López-Peréz a, J. Dampuré a,b, J.A. Hernández-Cabrera a,c, H.A. Barber a,c,⇑ a
Institute for Biomedical Technologies (ITB), University of La Laguna, Spain Centre de Recherches sur la Cognition et l’Apprentissage, UMR 7295, University of Poitiers, France c Basque Center on Cognition, Brain and Language (BCBL), Spain b
a r t i c l e
i n f o
Article history: Received 28 November 2015 Revised 23 June 2016 Accepted 25 July 2016
Keywords: ERPs Fixation Related Potentials Reading Parafovea Parafoveal-on-foveal effect Preview benefit N400
a b s t r a c t During reading parafoveal information can affect the processing of the word currently fixated (parafoveaon-fovea effect) and words perceived parafoveally can facilitate their subsequent processing when they are fixated on (preview effect). We investigated parafoveal processing by simultaneously recording eye movements and EEG measures. Participants read word pairs that could be semantically associated or not. Additionally, the boundary paradigm allowed us to carry out the same manipulation on parafoveal previews that were displayed until reader’s gaze moved to the target words. Event Related Potentials time-locked to the prime-preview presentation showed a parafoveal-on-foveal N400 effect. Fixation Related Potentials time locked to the saccade offset showed an N400 effect related to the prime-target relationship. Furthermore, this later effect interacted with the semantic manipulation of the previews, supporting a semantic preview benefit. These results demonstrate that at least under optimal conditions foveal and parafoveal information can be simultaneously processed and integrated. Ó 2016 Elsevier Inc. All rights reserved.
1. Introduction In contrast to what happens in auditory language perception, during reading more than one word representation usually reaches the sensory receptors simultaneously. When one word (n) is fixated at the fovea, there is still additional information (e.g. the next word in the sentence; n + 1) that can be perceived in the parafoveal visual field. Therefore, a central issue to fully understand language comprehension in reading is the nature and amount of information that can be extracted from the parafovea during any given fixation and how and when the parafoveal information is integrated with the foveal information. Parafoveal perception could influence reading in at least two different ways: (a) it can modulate the processing of the currently fixated word (i.e. parafovea-on-fovea effect), and (b) it may facilitate the processing of an incoming word (n + 1), which has already been perceived parafoveally and thus partially processed (i.e. preview benefit). In recent decades, psycholinguistic research has considerably increased what is known about the role of parafoveal perception during reading by monitoring eye movements (EM; see review in Schotter, Angele, & Rayner, 2012), and most of this research has ⇑ Corresponding author at: University of La Laguna, Campus de Guajara s/n, S/C de Tenerife 38205, Spain. E-mail address: [email protected] (H.A. Barber). http://dx.doi.org/10.1016/j.bandl.2016.07.009 0093-934X/Ó 2016 Elsevier Inc. All rights reserved.
used gaze-contingency paradigms. In the boundary paradigm (Rayner, 1975), words n + 1 are manipulated while they are perceived in the parafovea, and during the subsequent saccade (when crossing an invisible boundary) the word is changed to the final form presented in the fovea. In this manner, the amount of overlap between the parafoveal and the foveal information will determine the size of the preview benefit, which is measured through the fixation times on n + 1. Empirical evidence has consistently shown preview benefits when parafoveal previews and target words share orthographic or phonological characteristics. However, evidence of preview semantic effects is more scarce and has been obtained only under certain conditions; for example in reading with nonalphabetic scripts (Yan, Richter, Shu, & Kliegl, 2009), or using a modification of the standard boundary paradigm with shorter presentation times of the parafoveal previews (Hohenstein, Laubrock, & Kliegl, 2010). The contradictory results with this technique keep the question open about whether words in the parafovea are fully processed and their meanings immediately integrated. Empirical support of parafovea-on-fovea effects is derived from the modulation of fixation duration of the word n as a function of the physical and lexical characteristics of n + 1 (Vitu, Brysbaert, & Lancelin, 2004). These fixation modulations could be explained as a priming effect or as an online integration of n and n + 1. The latter possibility also raises the more general question of whether the processing of two adjacent words takes place serially or in parallel during normal
30
P.J. López-Peréz et al. / Brain & Language 162 (2016) 29–34
reading. Although several eye-tracking experiments have addressed this question, they have led to conflicting serial (Drieghe, Rayner, & Pollatsek, 2008) and parallel (Kennedy & Pynte, 2005) views of processing foveal and parafoveal information. There is also a growing interest in obtaining electrophysiological measures of parafoveal processing. A few studies have obtained Event Related Potentials (ERPs) associated to parafoveal processing in reading by using a modification of the RSVP paradigm. In the flanker-word RSVP paradigm, sentences are presented word by word at the center of a computer monitor (to avoid eye movements that generate undesirable activity in the EEG), but each word is flanked by other two parafoveal words that can be conveniently manipulated. Using this paradigm, parafovea-on-fovea effects were found related to semantic congruency manipulations of the right flankers (Barber, Doñamayor, Kutas, & Münte, 2010), and this type of parafoveal effects have been modulated by variables such as reading direction (Barber, Ben-Zvi, Bentin, & Kutas, 2011), the stimulus presentation rate, or the predictability of the parafoveal words (Barber, Van der Meij, & Kutas, 2013). Interestingly, some of these effects impinge on the amplitudes of the N400, an ERP component that has been consistently linked to semantic processing (Kutas & Federmeier, 2011). These N400 parafoveal effects support the views that semantic information of parafoveal words can be fully activated and immediately integrated with the foveal information. A different approach is to simultaneously record EEG and EM and obtain Fixation Related Potentials (FRPs) time-locked to ocular movement events (Dimigen, Sommer, Hohlfeld, Jacobs, & Kliegl, 2011; Hutzler et al., 2007; Kretzschmar, Schlesewsky, & Staub, 2015). Baccino and Manunta (2005) asked the participants of their experiment to read word pairs that could be semantically associated or not. They used an eye-tracking device to select trials with specific fixation durations on the first word and obtained the ERPs from the reading of those words before saccades took place. They reported a semantic effect (i.e. parafovea-on-fovea effect) in the time range of the P2 component. However, this effect could not be replicated in another study using a similar paradigm, although a lexicality effect (word versus nonword) was found at comparable latencies (Simola, Holmqvist, & Lindgren, 2009). It is important to emphasize that these studies restricted their analyses to early components (<300 ms after stimulus onset presentation) excluding
the N400 and later components that can be more adequate indexes of conceptual unification processes. A similar situation is found when we compare studies that have applied co-registration methods to sentence reading. While one study reported a parafoveal N400 effect with highly constraining sentence contexts (Kretzschmar, Bornkessel-Schlesewsky, & Schlesewsky, 2009), two other experiments in which word predictability was manipulated found a N400 modulation when target (n + 1) words were fixated, but reported no parafovea-on-fovea effects on the pre-target (n) words (Dimigen et al., 2011; Kretzschmar et al., 2015). Another study obtained concurrent EEG and EM measures in combination with the boundary paradigm described above (Dimigen, Kliegl, & Sommer, 2012). In this study, participants read five nouns from left to right as in normal reading. Target nouns were preceded by parafoveal previews that were either (a) identical, (b) semantically related but orthographically unrelated, or (c) completely unrelated to the target nouns. FRPs were recorded to target and pretarget words. While no parafoveal-on-foveal effects were observed, a temporal-occipital positivity between 200 and 280 ms and a N400 modulation differentiated the identity preview from the two other conditions. Moreover, no differences were observed between the two orthographically unrelated conditions, suggesting that word meaning did not contribute to the preview effects. These orthographic preview effects have been replicated in a recent study using the flanker-word VRSP paradigm and reading in Chinese (Li, Niefind, Wang, Sommer, & Dimigen, 2015). To sum up, whereas with EM data parafovea-on-fovea effects and preview semantic effects have not been firmly established (Schotter et al., 2012), ERPs have shown these effects using a paradigm that prevents eye movements and displays the stimuli with constant presentation times (Barber et al., 2010, 2011, 2013; Li et al., 2015). Coregistration of EEG and EM has also led to mixed results, with some studies reporting parafovea-on-fovea effects (Baccino & Manunta, 2005; Kretzschmar et al., 2009) and others that do not (Dimigen et al., 2011, 2012; Simola et al., 2009). The present experiment aims to dissociate parafovea-on-fovea and preview semantic effects by using a combination of the FRP method and the boundary paradigm, in a situation of word pair reading. Participants in our experiment read word pairs from left to right and decided if the words of each pair (n and n + 1) were semantically associated or not (see Fig. 1). The boundary paradigm
Fig. 1. Presentation procedure.
P.J. López-Peréz et al. / Brain & Language 162 (2016) 29–34
allowed us to manipulate the semantic relationship between the prime word (n) and parafoveal previews, which were displayed only up to the onset of the saccade to the target word (n + 1). This way, the control and manipulation of the information displayed in the parafovea allowed us to establish the onset of the effects associated to the post-saccadic processing and differentiate them from those related to the parafoveal previews. Although this approach does not resolve the signal-overlapping problem that arises in any reading task in which participants freely move their eyes, it facilitates the dissociation of the different effects and the study of their time courses and interactions.
2. Results 2.1. Behavioral measures Participants correctly responded to 92% of the trials, with no statistical differences between experimental conditions. Analysis of the first fixations on the prime words revealed no significant differences between related previews (mean = 351.5 ms, SD = 85.5) and unrelated previews (mean = 349.5 ms, SD = 83.5) or any effect of the TARGET factor (Fs < 1). Gaze duration measures showed no significant effects, either. Related previews: mean = 426.5 ms, SD = 105; Unrelated previews: mean = 422.5 ms, SD = 99).
31
2.2. ERPs associated to prime (n) and preview word reading Fig. 2 (upper graph) depicts the ERPs time-locked to the onset presentation of the word pair and therefore the fixation onset of the prime word. The grand averages corresponding to the four experimental conditions are shown separately. The amplitude of the N400 component is modulated by the semantic relationship of the prime with the parafoveal preview. Between 400 and 550 ms after the stimulus onset presentation, semantically unrelated word pairs resulted in more negative amplitudes compared to the related pairs. As expected, there is no influence of the target words (displayed after the saccade) at these latencies. Statistical analyses of the mean amplitudes in the 400–550 ms time window supported this observation: The ANOVA (PREVIEW TARGET ELECTRODE) yielded a main effect of the PREVIEW factor, F(1, 44) = 13.45, p < 0.001, g2p = 0.23, with no interaction with the TARGET factor or the ELECTRODE factor (Fs < 1). Effects of prime-target semantic relationship start to be evident after 550 ms post stimulus presentation. However, since these are post-saccade effects they are obscured by the variability in the fixation durations and they are better analyzed with the FRPs time-locked to the saccade offset. 2.3. FRPs associated to the saccade offset: fixation onset on the target (n + 1) word As can be seen in Fig. 2 (lower graph), the amplitude of the N400 component associated to the target word was modulated by both the semantic manipulation on the targets and on the previews. Targets without semantic association with the prime produced more negative amplitudes than the related ones (Fig. 3a). In addition, this prime-target effect was modulated by the manipulation of the previews in the same direction; unrelated primepreview pairs increased target N400 amplitudes in comparison to the related prime-preview pairs (Fig. 3b). The preview effect is also observed at earlier latencies, even before the fixation onset. This effect clearly takes place in the latency range of the effect described above related to prime-preview processing. However, the preview effect on the target N400 was larger when target words were unrelated with the prime than when prime and target were related, suggesting an interaction between the preview and target manipulations and not just a linear addition. Finally, the differences due to the prime-target manipulation continue between 550 and 1300 ms, but between 500 and 800 ms, the two conditions in which there was a mismatch between preview and target (invalid previews) produced a late positivity (LPC) with respect to the two conditions in which preview and target were identical (Fig. 3c). Statistical results confirmed these observations. 2.3.1. 0–200 ms time window The ANOVA revealed a main effect of the PREVIEW factor, F (1, 44) = 15.81, p < 0.001, g2p = 0.26, and an interaction between the factors between the PREVIEW and ELECTRODE, F(8, 352) = 4.02, p < 0.001, g2p = 0.08. Post-hoc comparisons at each electrode area showed that differences were larger at central and posterior electrodes. Results showed no effect of TARGET or any interaction involving this factor (Fs < 1).
Fig. 2. Grand average ERPs and FRPs at the CP2 electrode. Four experimental conditions: (a) prime-preview unrelated + prime-target unrelated, (b) primepreview related + prime-target related, (c) prime-preview unrelated + prime-target related, (d) prime-preview related + prime-target unrelated.
2.3.2. 300–500 ms time window (target N400) The ANOVA yielded the main effects of the factors PREVIEW, F (1, 44) = 20.57, p < 0.001, g2p = 0.31, and TARGET, F(1, 44) = 140.07, p < 0.001, g2p = 0.76. There was a also a two-way interaction between PREVIEW and TARGET, F(1, 44) = 38.84, p < 0.001, g2p = 0.46, and a three-way interaction of PREVIEW, TARGET and ELECTRODE, F(8, 352) = 5.81, p < 0.001, g2p = 0.11. The preview effect was larger when the target was related to the prime ( 1.35 mV) than when prime and target were unrelated
32
P.J. López-Peréz et al. / Brain & Language 162 (2016) 29–34
Fig. 3. FRPs: (a) prime-target relatedness (collapsed across other conditions), (b) prime-preview relatedness (collapsed across other conditions), and (c) preview-target relatedness (collapsed across other conditions). On the right: Topographical maps of each effect in the N400 time window.
(0.3 mV). Post-hoc tests showed that while the preview effect on related targets was present at all electrode areas, the same effect on unrelated targets was only confirmed in three areas: medial anterior, right central, and right posterior. 2.3.3. 500–750 ms time window (LPC) The ANOVA in this time window showed a main effect of PREVIEW, F(1, 44) = 5.01, p < 0.03, g2p = 0.10, and TARGET, F(1, 44) = 56.15, p < 0.001, g2p = 0.56. There was also a two-way interaction between these two factors, F(1, 44) = 9.4, p < 0.003, g2p = 0.17, and the three-way interaction of PREVIEW, TARGET and ELECTRODE, F(8, 352) = 3.4, p < 0.002, g2p = 0.07. In contrast to the previous time window, unrelated previews resulted in more positive amplitudes than related previews. Post-hoc comparisons confirmed a broadly distributed effect of preview on the unrelated targets (Related-Pre view/Unrelated-Target condition produced more positive amplitudes than the Unrelated-Preview/Unrelated-Target condition; 0.19 lV and 0.89 lV respectively), but the same difference in the case of the related targets was statistically significant only in the left posterior area. 3. Discussion A parafovea-on-fovea effect was described in the ERPs timelocked to the onset presentation of the prime-preview word pair. While participants were fixating the prime, the processing of this word was influenced by the meaning of the word perceived in the parafovea. A N400 effect starts around the time the average saccade takes place and is not modulated by the prime-target relationship. The characteristics of this N400-parafoveal effect match with the standard N400 effect associated to semantic processing of words in the fovea. The N400 component has been proposed as an index of binding processes in which stimulus-driven activity
and the current state of a broad semantic neural network enters into temporal synchrony, resulting in dynamically created new conceptual representations (Kutas & Federmeier, 2011). The fact that our parafoveal manipulation was semantic in nature and that it affected the N400 component indicates that words in the parafovea were fully processed; their meanings were activated and quickly bounded with the meanings of the words simultaneously perceived in the fovea. Previous studies using a similar task resulted in mixed results (Baccino & Manunta, 2005; Simola et al., 2009), however they only analyzed the early latencies previous to the N400 component. Our results are consistent with data obtained in sentence reading with the flanker-word RSVP paradigm (Barber et al., 2010, 2011, 2013; Li et al., 2015) and with one FRP experiment that reported parafoveal N400 effects during sentence reading (Kretzschmar et al., 2009). Even when the ERPs showed a clear parafovea-on-fovea effect, EM measures in the present experiment were not sensitive to our experimental manipulation. This dissociation between both measures has been previously reported in sentence reading experiments (see Kretzschmar et al., 2015 for further discussion). However, it is important to note that the EM measures analyzed in the present study should be interpreted with caution, because they refer to fixations on the first word of a sequence, which involve different processes than those of words located in the middle of a sentence. Even so, the comparison between the time courses of both measures reveals that the saccades to the target words start before the offset of the N400 effect. Therefore a certain amount of lexical processing seems to be enough to trigger the saccadic movement. Interestingly, the lack of modulation of the EM by our experimental manipulation would indicate that although semantic parafoveal information is quickly activated and integrated, this information does not affect the saccade programming, which could be based only on the formal characteristics of the target at an earlier stage. In a more general
P.J. López-Peréz et al. / Brain & Language 162 (2016) 29–34
theoretical framework, our results support those views that propose parallel activations of foveal and parafoveal information during reading. The ERPs in our study do not show any evidence of serial processing of the two words presented at the same time. In contrast, the processing of both meanings seems to affect the same component, and therefore they are probably being activated and integrated as part of the same cognitive process. Consistent with the parafovea-on-fovea effect, FRPs showed a subsequent facilitation of the processing of the target word when the reader’s gaze landed on it. In other words, the preview manipulation clearly affected the processing of the target words. Fig. 3c shows the grand averages of the valid versus invalid previews, independently of the semantic manipulations. Preview-target mismatch (invalid previews) increased N400 amplitudes associated to target processing and produced a later positivity. These effects are not due to the detection of a perceptual change, because font case always changed between previews and targets. This N400 effect confirms the preview effect reported by Dimigen and colleagues in two different studies (Dimigen et al., 2012; Li et al., 2015). However, the analyses of our experimental conditions separately revealed an additional contribution of the preview meanings (or the previous prime-preview integration) on the target processing (see Fig. 3b). Moreover, we observed an interaction between preview and target semantic manipulations; the effect of preview semantic relationship was larger on targets semantically related with the primes. Later on, a late positivity was found with the invalid previews. Considering its latency (after the question mark presentation), this positivity seems to be related to the interference produced in these trials at the time of the word pair categorization, which was necessary to perform the task. Although EM research using the boundary paradigm has consistently demonstrated that orthographically and phonologically-related previews improve target processing, it has also repeatedly failed to find evidence of preview semantic effects (Schotter et al., 2012; but see Hohenstein et al., 2010). A more intriguing question is why Dimigen et al. (2012) did not find parafoveal semantic effects in their FRPs measures in spite of also using the boundary paradigm. There are important differences between that study and the one described here that should be considered. Firstly, their prime-target pairs were embedded in a longer string of five words. As a consequence, prime words received shorter fixations, which implies less time to process parafoveal information. In a previous study, we showed that the N400 parafoveal effects were constrained by the stimulus presentation time (Barber et al., 2013). Another important difference is that our task explicitly asked for the semantic integration of the words read, while Dimigen et al. used a conceptual search task, which guarantees meaning activation but could prevent integration processes (see also Dampuré, Ros, Rouet, & Vibert, 2014). Finally and related to the previous point, the Dimigen et al. experiment manipulated the semantic relationship between targets and previews, whereas in our study we additionally manipulated the semantic relationship between previews and primes. This means that the targets had to be integrated with the ongoing primepreview representation as in sentence reading. To sum up, we have been able to show semantic parafovea-onfovea and preview effects in a situation in which similar EM and FRP studies failed to find them. Our description of the reported semantic parafoveal effects (dissociation, time-course and interaction) is a valuable contribution to the topic of parafoveal processing in reading, even when such effects refer to a highly controlled experimental situation in which word pair reading is performed without a larger context. Reading texts involve more cognitive constraints and demands than reading word pairs, but there is no reason to think that the constraints that operate at the word pair level will not affect the more complex forms of reading. We can thus conclude that under optimal circumstances,
33
foveal and parafoveal information can be processed simultaneously and the resulting unified representation can affect subsequent reading. Likewise, it has been demonstrated that the combination of FRPs with the boundary technique is a suitable method to further determine the variables that can modulate this parallel processing during sentence reading. 4. Method 4.1. Participants Forty-five undergraduate students (35 female, 18–29 years of age, mean age 20). They were right-handed Spanish speakers with no history of neurological impairment. 4.2. Stimuli Three hundred sixty semantically related Spanish word pairs were obtained from the Rules of Free Association in Spanish of the University of Salamanca (www.usal.es/gimc/nalc). Semantic association between primes and parafoveal previews and between primes and targets were manipulated in a 2 2 factorial withinsubject design, resulting in four experimental conditions (examples for the word prime arena; sand): – Related Preview + Related Target (playa + playa; beach + beach) – Related Preview + Unrelated Target (playa + letra; beach + letter) – Unrelated Preview + Related Target (letra + playa; letter + beach) – Unrelated Preview + Unrelated Target (letra + letra; letter + letter) The related preview/target of each prime was used as an unrelated preview/target with a different prime. No participant saw any given word pair more than once but across participants each pair was presented in the four conditions. All words were between 4 and 7 letters long, (mean length: primes = 5.6, previews/targets = 5.3). The average lexical frequency (logFreq: logarithm in base 10 of the raw number of corpus occurrences + 1) for the prime words was 1.2 (SD = 0.77), and for preview/target words was 1.5 (SD = 0.59). The average number of orthographic neighbors for the prime and preview/target words was 6.8 (SD = 6.7) and 9.1 (SD = 8.2) respectively (EsPal database; Duchon, Perea, SebastianGalles, Marti, & Carreiras, 2013). 4.3. Apparatus and procedure The EEG was recorded via 27 Ag/AgCl electrodes referenced to the left mastoid. Four additional external electrodes, two at the outer canthus of each eye and two at the infraorbital and supraorbital regions of the right eye, provided bipolar recordings of the horizontal and vertical electro-oculogram. All electrical activity was recorded and amplified with a bandwidth of 0.01–100 Hz, and sampled at 500 Hz. Impedances were kept below 5 kX (electro-oculogram <10 kX). The recorded data were filtered offline with a band-pass 0.1–30 Hz, and re-referenced to the algebraic mean of the activity at the two mastoids. Eye movements were registered with an EyeLink 1000 eye-tracking system (SR Research Ltd., Ontario, Canada), with a sampling rate of 1000 Hz and an instrument spatial resolution of 0.01°. For each data sample, the EyelinkII saccade parser algorithm computes the instantaneous velocity and acceleration and compares these to threshold criteria for velocity and acceleration (30 deg/s and 8000 deg/s); a distance threshold of 0.1° was used to delay the onset of the saccade until
34
P.J. López-Peréz et al. / Brain & Language 162 (2016) 29–34
the eye had moved significantly. Calibration was performed on a standard nine-point grid. Both systems were synchronized sending TTL pulses form the eye tracker to the EEG recorder. Due to the different sampling rates used in both systems (500 Hz vs 1 kHz), synchronization was verified offline analyzing the correlation between the markers of both recordings (r = 0.97; p = 0.01), and confirming deviations shorter than 5 ms. Each subject was seated at a distance of 60 cm from the computer screen (23 in. Eizo Foris, resolution: 1024 768 pixels, vertical refresh rate: 120 Hz). Words were presented in Courier New black letters against a white background via Experiment Builder software (SR Research Ltd., Canada). Primes and previews were presented in lowercase letters, whereas targets were presented in uppercase. This procedure allow us to avoid physical overlap between words, so potential priming effects can be interpreted in relation their abstract linguistic representations and meanings. The distance between the center of the foveal word and the left edge of the parafoveal word was 2° of visual angle. Parafoveal previews were manipulated using a saccade-contingent display change in a boundary paradigm. The sequence of events in each trial was as follows (see Fig. 1): (1) fixation cross in the center of the screen, (2) once a stable fixation was detected, the fixation cross reduced its size as a preparation cue, (3) after 800 ms, prime-preview words were displayed simultaneously, (4) previews were substituted by the targets when participants shifted their gaze from left to right and crossed the invisible boundary (located between the n and n + 1 regions, at 1.8° from the center of the screen), and (5) the target word was replaced by a question mark at the same location, 400 ms after the target fixation onset, and participants had to decide whether primes and targets were semantically related or not (therefore targets were always displayed for a fixed time). For each participant, each stimulus list was presented in a different random presentation order. In total, each participant saw 360 trials and the experimental session lasted about 90 min. 4.4. Data analysis We report EM data of first-pass fixation relative to the reading of the prime words: first fixation duration and gaze duration. The prime word was presented at the same location as the fixation cross. Even though initial fixations included both the perception of the fixation cross (800 ms) and the prime, the EM measures reported here refer exclusively to the time in which primes were visible on the screen. These data were entered in two separate ANOVAs with the factors PREVIEW (related versus unrelated) and TARGET (related versus unrelated). Because targets were displayed for a fixed time and were replaced by a question mark at the same location, EM data on target words were not analyzed. On average, 11% of trials were excluded because of artifacts (drifts, regressions), anomalous fixations (<50 or >800 ms) or late display changes (>12 ms; 4%). Seventeen participants reported to have detected some kind of change in the parafovea but none of them could establish what kind of change, or reported any particular word. The average number of accepted trials was equally distributed across experimental conditions and participants. The EEG data were segmented in epochs of interest: 200 to 1500 ms time-locked to the prime-preview onset presentation, and 700 to 2000 ms time-locked to the saccade offset. Baseline correction was performed taking 200 ms previous to stimulus onset for the ERPs, and between 700 and 500 ms before saccade offset for the FRPs. Artifacts were removed semi-automatically with rejection values adjusted for each participant. ERP and FRP mean amplitudes of the selected time-windows were analyzed in separate ANOVAs with the factors PREVIEW (related versus unre-
lated) and TARGET (related versus unrelated) and the topographical factor ELECTRODE. For this factor, nine different electrode groups were computed, each compromising the mean of 3 electrodes: Left anterior (Fp1, F7, F3), left central (FC5, T7, C3), left posterior (CP5, P7, P3), right anterior (Fp2, F8, F4), right central (FC6, T8, C4), right posterior (CP6, P8, P4), medial anterior (Fz, FC1, FC2), medial central (Cz, CP1, CP2), and medial posterior (Pz, O1, O2). Statistical analyses were performed with R software (http:// www.rproject.org). Results reported include the original degrees of freedom and p values corrected for violation of sphericity using the Greenhouse-Geisser epsilon. Acknowledgments HAB was supported by the grand PSI2013-46334-P of the Spanish Ministry of Economy. References Baccino, T., & Manunta, Y. (2005). Eye-fixation-related potentials: Insight into parafoveal processing. Journal of Psychophysiology, 19(3), 204–215. Barber, H. A., Doñamayor, N., Kutas, M., & Münte, T. (2010). Parafoveal N400 effect during sentence reading. Neuroscience Letters, 479(2), 152–156. Barber, H. A., Ben-Zvi, S., Bentin, S., & Kutas, M. (2011). Parafoveal perception during sentence reading? An ERP paradigm using rapid serial visual presentation (RSVP) with flankers. Psychophysiology, 48(4), 523–531. Barber, H. A., Van der Meij, M., & Kutas, M. (2013). An electrophysiological analysis of contextual and temporal constraints on parafoveal word processing. Psychophysiology, 50(1), 48–59. Dampuré, J., Ros, C., Rouet, J.-F., & Vibert, N. (2014). Task-dependent sensitization of perceptual and semantic processing during visual search for words. Journal of Cognitive Psychology, 26, 530–549. Dimigen, O., Kliegl, R., & Sommer, W. (2012). Trans-saccadic parafoveal preview benefits in fluent reading: A study with fixation-related brain potentials. NeuroImage, 62(1), 381–393. Dimigen, O., Sommer, W., Hohlfeld, A., Jacobs, A. M., & Kliegl, R. (2011). Coregistration of eye movements and EEG in natural reading: Analyses and review. Journal of Experimental Psychology: General, 140(4), 552–572. Drieghe, D., Rayner, K., & Pollatsek, A. (2008). Mislocated fixations can account for parafoveal-on-foveal effects in eye movements during reading. Quarterly Journal of Experimental Psychology, 61(8), 1239–1249. Duchon, A., Perea, M., Sebastian-Galles, N., Marti, A., & Carreiras, M. (2013). EsPal: One-stop shopping for Spanish word properties. Behavior Research Methods, 45, 1246–1258. Hohenstein, S., Laubrock, J., & Kliegl, R. (2010). Semantic preview benefit in eye movements during reading: A parafoveal fastpriming study. Journal of Experimental Psychology: Learning, Memory, and Cognition, 36, 1150–1170. Hutzler, F., Braun, M., Vö, M. L. H., Engl, V., Hofmann, M., Dambacher, M., & Jacobs, A. M. (2007). Welcome to the real world: Validating fixation-related brain potentials for ecologically valid settings. Brain Research, 1172, 124–129. Kennedy, A., & Pynte, J. (2005). Parafoveal-on-foveal effects in normal reading. Vision Research, 45(2), 153–168. Kretzschmar, F., Bornkessel-Schlesewsky, I., & Schlesewsky, M. (2009). Parafoveal versus foveal N400s dissociate spreading activation from contextual fit. NeuroReport, 20(18), 1613–1618. Kretzschmar, F., Schlesewsky, M., & Staub, A. (2015). Dissociating word frequency and predictability effects in reading: Evidence from co-registration of eye movements and EEG. Journal of Experimental Psychology: Learning, Memory, and Cognition, 41(6), 1648–1662. Kutas, M., & Federmeier, K. D. (2011). Thirty years and counting: Finding meaning in the N400 component of the event-related brain potential (ERP). Annual Review Psychology, 62, 621–647. Li, N., Niefind, F., Wang, S., Sommer, W., & Dimigen, O. (2015). Parafoveal processing in reading Chinese sentences: Evidence from event-related brain potentials. Psychophysiology, 52(10), 1361–1374. Rayner, K. (1975). The perceptual span and peripheral cues in reading. Cognitive Psychology, 7, 65–81. Schotter, E. R., Angele, B., & Rayner, K. (2012). Parafoveal processing in reading. Attention, Perception, & Psychophysics, 74(1), 5–35. Simola, J., Holmqvist, K., & Lindgren, M. (2009). Right visual field advantage in parafoveal processing: Evidence from eye-fixation-related potentials. Brain and Language, 111(2), 101–113. Vitu, F., Brysbaert, M., & Lancelin, D. (2004). A test of parafovealon-foveal effects with pairs of orthographically related words. European Journal of Cognitive Psychology, 16, 154–177. Yan, M., Richter, E. M., Shu, H., & Kliegl, R. (2009). Readers of Chinese extract semantic information from parafoveal words. Psychonomic Bulletin & Review, 16 (3), 561–566.
Contents lists available at ScienceDirect
Brain & Language journal homepage: www.elsevier.com/locate/b&l
Short Communication
Semantic parafoveal-on-foveal effects and preview benefits in reading: Evidence from Fixation Related Potentials P.J. López-Peréz a, J. Dampuré a,b, J.A. Hernández-Cabrera a,c, H.A. Barber a,c,⇑ a
Institute for Biomedical Technologies (ITB), University of La Laguna, Spain Centre de Recherches sur la Cognition et l’Apprentissage, UMR 7295, University of Poitiers, France c Basque Center on Cognition, Brain and Language (BCBL), Spain b
a r t i c l e
i n f o
Article history: Received 28 November 2015 Revised 23 June 2016 Accepted 25 July 2016
Keywords: ERPs Fixation Related Potentials Reading Parafovea Parafoveal-on-foveal effect Preview benefit N400
a b s t r a c t During reading parafoveal information can affect the processing of the word currently fixated (parafoveaon-fovea effect) and words perceived parafoveally can facilitate their subsequent processing when they are fixated on (preview effect). We investigated parafoveal processing by simultaneously recording eye movements and EEG measures. Participants read word pairs that could be semantically associated or not. Additionally, the boundary paradigm allowed us to carry out the same manipulation on parafoveal previews that were displayed until reader’s gaze moved to the target words. Event Related Potentials time-locked to the prime-preview presentation showed a parafoveal-on-foveal N400 effect. Fixation Related Potentials time locked to the saccade offset showed an N400 effect related to the prime-target relationship. Furthermore, this later effect interacted with the semantic manipulation of the previews, supporting a semantic preview benefit. These results demonstrate that at least under optimal conditions foveal and parafoveal information can be simultaneously processed and integrated. Ó 2016 Elsevier Inc. All rights reserved.
1. Introduction In contrast to what happens in auditory language perception, during reading more than one word representation usually reaches the sensory receptors simultaneously. When one word (n) is fixated at the fovea, there is still additional information (e.g. the next word in the sentence; n + 1) that can be perceived in the parafoveal visual field. Therefore, a central issue to fully understand language comprehension in reading is the nature and amount of information that can be extracted from the parafovea during any given fixation and how and when the parafoveal information is integrated with the foveal information. Parafoveal perception could influence reading in at least two different ways: (a) it can modulate the processing of the currently fixated word (i.e. parafovea-on-fovea effect), and (b) it may facilitate the processing of an incoming word (n + 1), which has already been perceived parafoveally and thus partially processed (i.e. preview benefit). In recent decades, psycholinguistic research has considerably increased what is known about the role of parafoveal perception during reading by monitoring eye movements (EM; see review in Schotter, Angele, & Rayner, 2012), and most of this research has ⇑ Corresponding author at: University of La Laguna, Campus de Guajara s/n, S/C de Tenerife 38205, Spain. E-mail address: [email protected] (H.A. Barber). http://dx.doi.org/10.1016/j.bandl.2016.07.009 0093-934X/Ó 2016 Elsevier Inc. All rights reserved.
used gaze-contingency paradigms. In the boundary paradigm (Rayner, 1975), words n + 1 are manipulated while they are perceived in the parafovea, and during the subsequent saccade (when crossing an invisible boundary) the word is changed to the final form presented in the fovea. In this manner, the amount of overlap between the parafoveal and the foveal information will determine the size of the preview benefit, which is measured through the fixation times on n + 1. Empirical evidence has consistently shown preview benefits when parafoveal previews and target words share orthographic or phonological characteristics. However, evidence of preview semantic effects is more scarce and has been obtained only under certain conditions; for example in reading with nonalphabetic scripts (Yan, Richter, Shu, & Kliegl, 2009), or using a modification of the standard boundary paradigm with shorter presentation times of the parafoveal previews (Hohenstein, Laubrock, & Kliegl, 2010). The contradictory results with this technique keep the question open about whether words in the parafovea are fully processed and their meanings immediately integrated. Empirical support of parafovea-on-fovea effects is derived from the modulation of fixation duration of the word n as a function of the physical and lexical characteristics of n + 1 (Vitu, Brysbaert, & Lancelin, 2004). These fixation modulations could be explained as a priming effect or as an online integration of n and n + 1. The latter possibility also raises the more general question of whether the processing of two adjacent words takes place serially or in parallel during normal
30
P.J. López-Peréz et al. / Brain & Language 162 (2016) 29–34
reading. Although several eye-tracking experiments have addressed this question, they have led to conflicting serial (Drieghe, Rayner, & Pollatsek, 2008) and parallel (Kennedy & Pynte, 2005) views of processing foveal and parafoveal information. There is also a growing interest in obtaining electrophysiological measures of parafoveal processing. A few studies have obtained Event Related Potentials (ERPs) associated to parafoveal processing in reading by using a modification of the RSVP paradigm. In the flanker-word RSVP paradigm, sentences are presented word by word at the center of a computer monitor (to avoid eye movements that generate undesirable activity in the EEG), but each word is flanked by other two parafoveal words that can be conveniently manipulated. Using this paradigm, parafovea-on-fovea effects were found related to semantic congruency manipulations of the right flankers (Barber, Doñamayor, Kutas, & Münte, 2010), and this type of parafoveal effects have been modulated by variables such as reading direction (Barber, Ben-Zvi, Bentin, & Kutas, 2011), the stimulus presentation rate, or the predictability of the parafoveal words (Barber, Van der Meij, & Kutas, 2013). Interestingly, some of these effects impinge on the amplitudes of the N400, an ERP component that has been consistently linked to semantic processing (Kutas & Federmeier, 2011). These N400 parafoveal effects support the views that semantic information of parafoveal words can be fully activated and immediately integrated with the foveal information. A different approach is to simultaneously record EEG and EM and obtain Fixation Related Potentials (FRPs) time-locked to ocular movement events (Dimigen, Sommer, Hohlfeld, Jacobs, & Kliegl, 2011; Hutzler et al., 2007; Kretzschmar, Schlesewsky, & Staub, 2015). Baccino and Manunta (2005) asked the participants of their experiment to read word pairs that could be semantically associated or not. They used an eye-tracking device to select trials with specific fixation durations on the first word and obtained the ERPs from the reading of those words before saccades took place. They reported a semantic effect (i.e. parafovea-on-fovea effect) in the time range of the P2 component. However, this effect could not be replicated in another study using a similar paradigm, although a lexicality effect (word versus nonword) was found at comparable latencies (Simola, Holmqvist, & Lindgren, 2009). It is important to emphasize that these studies restricted their analyses to early components (<300 ms after stimulus onset presentation) excluding
the N400 and later components that can be more adequate indexes of conceptual unification processes. A similar situation is found when we compare studies that have applied co-registration methods to sentence reading. While one study reported a parafoveal N400 effect with highly constraining sentence contexts (Kretzschmar, Bornkessel-Schlesewsky, & Schlesewsky, 2009), two other experiments in which word predictability was manipulated found a N400 modulation when target (n + 1) words were fixated, but reported no parafovea-on-fovea effects on the pre-target (n) words (Dimigen et al., 2011; Kretzschmar et al., 2015). Another study obtained concurrent EEG and EM measures in combination with the boundary paradigm described above (Dimigen, Kliegl, & Sommer, 2012). In this study, participants read five nouns from left to right as in normal reading. Target nouns were preceded by parafoveal previews that were either (a) identical, (b) semantically related but orthographically unrelated, or (c) completely unrelated to the target nouns. FRPs were recorded to target and pretarget words. While no parafoveal-on-foveal effects were observed, a temporal-occipital positivity between 200 and 280 ms and a N400 modulation differentiated the identity preview from the two other conditions. Moreover, no differences were observed between the two orthographically unrelated conditions, suggesting that word meaning did not contribute to the preview effects. These orthographic preview effects have been replicated in a recent study using the flanker-word VRSP paradigm and reading in Chinese (Li, Niefind, Wang, Sommer, & Dimigen, 2015). To sum up, whereas with EM data parafovea-on-fovea effects and preview semantic effects have not been firmly established (Schotter et al., 2012), ERPs have shown these effects using a paradigm that prevents eye movements and displays the stimuli with constant presentation times (Barber et al., 2010, 2011, 2013; Li et al., 2015). Coregistration of EEG and EM has also led to mixed results, with some studies reporting parafovea-on-fovea effects (Baccino & Manunta, 2005; Kretzschmar et al., 2009) and others that do not (Dimigen et al., 2011, 2012; Simola et al., 2009). The present experiment aims to dissociate parafovea-on-fovea and preview semantic effects by using a combination of the FRP method and the boundary paradigm, in a situation of word pair reading. Participants in our experiment read word pairs from left to right and decided if the words of each pair (n and n + 1) were semantically associated or not (see Fig. 1). The boundary paradigm
Fig. 1. Presentation procedure.
P.J. López-Peréz et al. / Brain & Language 162 (2016) 29–34
allowed us to manipulate the semantic relationship between the prime word (n) and parafoveal previews, which were displayed only up to the onset of the saccade to the target word (n + 1). This way, the control and manipulation of the information displayed in the parafovea allowed us to establish the onset of the effects associated to the post-saccadic processing and differentiate them from those related to the parafoveal previews. Although this approach does not resolve the signal-overlapping problem that arises in any reading task in which participants freely move their eyes, it facilitates the dissociation of the different effects and the study of their time courses and interactions.
2. Results 2.1. Behavioral measures Participants correctly responded to 92% of the trials, with no statistical differences between experimental conditions. Analysis of the first fixations on the prime words revealed no significant differences between related previews (mean = 351.5 ms, SD = 85.5) and unrelated previews (mean = 349.5 ms, SD = 83.5) or any effect of the TARGET factor (Fs < 1). Gaze duration measures showed no significant effects, either. Related previews: mean = 426.5 ms, SD = 105; Unrelated previews: mean = 422.5 ms, SD = 99).
31
2.2. ERPs associated to prime (n) and preview word reading Fig. 2 (upper graph) depicts the ERPs time-locked to the onset presentation of the word pair and therefore the fixation onset of the prime word. The grand averages corresponding to the four experimental conditions are shown separately. The amplitude of the N400 component is modulated by the semantic relationship of the prime with the parafoveal preview. Between 400 and 550 ms after the stimulus onset presentation, semantically unrelated word pairs resulted in more negative amplitudes compared to the related pairs. As expected, there is no influence of the target words (displayed after the saccade) at these latencies. Statistical analyses of the mean amplitudes in the 400–550 ms time window supported this observation: The ANOVA (PREVIEW TARGET ELECTRODE) yielded a main effect of the PREVIEW factor, F(1, 44) = 13.45, p < 0.001, g2p = 0.23, with no interaction with the TARGET factor or the ELECTRODE factor (Fs < 1). Effects of prime-target semantic relationship start to be evident after 550 ms post stimulus presentation. However, since these are post-saccade effects they are obscured by the variability in the fixation durations and they are better analyzed with the FRPs time-locked to the saccade offset. 2.3. FRPs associated to the saccade offset: fixation onset on the target (n + 1) word As can be seen in Fig. 2 (lower graph), the amplitude of the N400 component associated to the target word was modulated by both the semantic manipulation on the targets and on the previews. Targets without semantic association with the prime produced more negative amplitudes than the related ones (Fig. 3a). In addition, this prime-target effect was modulated by the manipulation of the previews in the same direction; unrelated primepreview pairs increased target N400 amplitudes in comparison to the related prime-preview pairs (Fig. 3b). The preview effect is also observed at earlier latencies, even before the fixation onset. This effect clearly takes place in the latency range of the effect described above related to prime-preview processing. However, the preview effect on the target N400 was larger when target words were unrelated with the prime than when prime and target were related, suggesting an interaction between the preview and target manipulations and not just a linear addition. Finally, the differences due to the prime-target manipulation continue between 550 and 1300 ms, but between 500 and 800 ms, the two conditions in which there was a mismatch between preview and target (invalid previews) produced a late positivity (LPC) with respect to the two conditions in which preview and target were identical (Fig. 3c). Statistical results confirmed these observations. 2.3.1. 0–200 ms time window The ANOVA revealed a main effect of the PREVIEW factor, F (1, 44) = 15.81, p < 0.001, g2p = 0.26, and an interaction between the factors between the PREVIEW and ELECTRODE, F(8, 352) = 4.02, p < 0.001, g2p = 0.08. Post-hoc comparisons at each electrode area showed that differences were larger at central and posterior electrodes. Results showed no effect of TARGET or any interaction involving this factor (Fs < 1).
Fig. 2. Grand average ERPs and FRPs at the CP2 electrode. Four experimental conditions: (a) prime-preview unrelated + prime-target unrelated, (b) primepreview related + prime-target related, (c) prime-preview unrelated + prime-target related, (d) prime-preview related + prime-target unrelated.
2.3.2. 300–500 ms time window (target N400) The ANOVA yielded the main effects of the factors PREVIEW, F (1, 44) = 20.57, p < 0.001, g2p = 0.31, and TARGET, F(1, 44) = 140.07, p < 0.001, g2p = 0.76. There was a also a two-way interaction between PREVIEW and TARGET, F(1, 44) = 38.84, p < 0.001, g2p = 0.46, and a three-way interaction of PREVIEW, TARGET and ELECTRODE, F(8, 352) = 5.81, p < 0.001, g2p = 0.11. The preview effect was larger when the target was related to the prime ( 1.35 mV) than when prime and target were unrelated
32
P.J. López-Peréz et al. / Brain & Language 162 (2016) 29–34
Fig. 3. FRPs: (a) prime-target relatedness (collapsed across other conditions), (b) prime-preview relatedness (collapsed across other conditions), and (c) preview-target relatedness (collapsed across other conditions). On the right: Topographical maps of each effect in the N400 time window.
(0.3 mV). Post-hoc tests showed that while the preview effect on related targets was present at all electrode areas, the same effect on unrelated targets was only confirmed in three areas: medial anterior, right central, and right posterior. 2.3.3. 500–750 ms time window (LPC) The ANOVA in this time window showed a main effect of PREVIEW, F(1, 44) = 5.01, p < 0.03, g2p = 0.10, and TARGET, F(1, 44) = 56.15, p < 0.001, g2p = 0.56. There was also a two-way interaction between these two factors, F(1, 44) = 9.4, p < 0.003, g2p = 0.17, and the three-way interaction of PREVIEW, TARGET and ELECTRODE, F(8, 352) = 3.4, p < 0.002, g2p = 0.07. In contrast to the previous time window, unrelated previews resulted in more positive amplitudes than related previews. Post-hoc comparisons confirmed a broadly distributed effect of preview on the unrelated targets (Related-Pre view/Unrelated-Target condition produced more positive amplitudes than the Unrelated-Preview/Unrelated-Target condition; 0.19 lV and 0.89 lV respectively), but the same difference in the case of the related targets was statistically significant only in the left posterior area. 3. Discussion A parafovea-on-fovea effect was described in the ERPs timelocked to the onset presentation of the prime-preview word pair. While participants were fixating the prime, the processing of this word was influenced by the meaning of the word perceived in the parafovea. A N400 effect starts around the time the average saccade takes place and is not modulated by the prime-target relationship. The characteristics of this N400-parafoveal effect match with the standard N400 effect associated to semantic processing of words in the fovea. The N400 component has been proposed as an index of binding processes in which stimulus-driven activity
and the current state of a broad semantic neural network enters into temporal synchrony, resulting in dynamically created new conceptual representations (Kutas & Federmeier, 2011). The fact that our parafoveal manipulation was semantic in nature and that it affected the N400 component indicates that words in the parafovea were fully processed; their meanings were activated and quickly bounded with the meanings of the words simultaneously perceived in the fovea. Previous studies using a similar task resulted in mixed results (Baccino & Manunta, 2005; Simola et al., 2009), however they only analyzed the early latencies previous to the N400 component. Our results are consistent with data obtained in sentence reading with the flanker-word RSVP paradigm (Barber et al., 2010, 2011, 2013; Li et al., 2015) and with one FRP experiment that reported parafoveal N400 effects during sentence reading (Kretzschmar et al., 2009). Even when the ERPs showed a clear parafovea-on-fovea effect, EM measures in the present experiment were not sensitive to our experimental manipulation. This dissociation between both measures has been previously reported in sentence reading experiments (see Kretzschmar et al., 2015 for further discussion). However, it is important to note that the EM measures analyzed in the present study should be interpreted with caution, because they refer to fixations on the first word of a sequence, which involve different processes than those of words located in the middle of a sentence. Even so, the comparison between the time courses of both measures reveals that the saccades to the target words start before the offset of the N400 effect. Therefore a certain amount of lexical processing seems to be enough to trigger the saccadic movement. Interestingly, the lack of modulation of the EM by our experimental manipulation would indicate that although semantic parafoveal information is quickly activated and integrated, this information does not affect the saccade programming, which could be based only on the formal characteristics of the target at an earlier stage. In a more general
P.J. López-Peréz et al. / Brain & Language 162 (2016) 29–34
theoretical framework, our results support those views that propose parallel activations of foveal and parafoveal information during reading. The ERPs in our study do not show any evidence of serial processing of the two words presented at the same time. In contrast, the processing of both meanings seems to affect the same component, and therefore they are probably being activated and integrated as part of the same cognitive process. Consistent with the parafovea-on-fovea effect, FRPs showed a subsequent facilitation of the processing of the target word when the reader’s gaze landed on it. In other words, the preview manipulation clearly affected the processing of the target words. Fig. 3c shows the grand averages of the valid versus invalid previews, independently of the semantic manipulations. Preview-target mismatch (invalid previews) increased N400 amplitudes associated to target processing and produced a later positivity. These effects are not due to the detection of a perceptual change, because font case always changed between previews and targets. This N400 effect confirms the preview effect reported by Dimigen and colleagues in two different studies (Dimigen et al., 2012; Li et al., 2015). However, the analyses of our experimental conditions separately revealed an additional contribution of the preview meanings (or the previous prime-preview integration) on the target processing (see Fig. 3b). Moreover, we observed an interaction between preview and target semantic manipulations; the effect of preview semantic relationship was larger on targets semantically related with the primes. Later on, a late positivity was found with the invalid previews. Considering its latency (after the question mark presentation), this positivity seems to be related to the interference produced in these trials at the time of the word pair categorization, which was necessary to perform the task. Although EM research using the boundary paradigm has consistently demonstrated that orthographically and phonologically-related previews improve target processing, it has also repeatedly failed to find evidence of preview semantic effects (Schotter et al., 2012; but see Hohenstein et al., 2010). A more intriguing question is why Dimigen et al. (2012) did not find parafoveal semantic effects in their FRPs measures in spite of also using the boundary paradigm. There are important differences between that study and the one described here that should be considered. Firstly, their prime-target pairs were embedded in a longer string of five words. As a consequence, prime words received shorter fixations, which implies less time to process parafoveal information. In a previous study, we showed that the N400 parafoveal effects were constrained by the stimulus presentation time (Barber et al., 2013). Another important difference is that our task explicitly asked for the semantic integration of the words read, while Dimigen et al. used a conceptual search task, which guarantees meaning activation but could prevent integration processes (see also Dampuré, Ros, Rouet, & Vibert, 2014). Finally and related to the previous point, the Dimigen et al. experiment manipulated the semantic relationship between targets and previews, whereas in our study we additionally manipulated the semantic relationship between previews and primes. This means that the targets had to be integrated with the ongoing primepreview representation as in sentence reading. To sum up, we have been able to show semantic parafovea-onfovea and preview effects in a situation in which similar EM and FRP studies failed to find them. Our description of the reported semantic parafoveal effects (dissociation, time-course and interaction) is a valuable contribution to the topic of parafoveal processing in reading, even when such effects refer to a highly controlled experimental situation in which word pair reading is performed without a larger context. Reading texts involve more cognitive constraints and demands than reading word pairs, but there is no reason to think that the constraints that operate at the word pair level will not affect the more complex forms of reading. We can thus conclude that under optimal circumstances,
33
foveal and parafoveal information can be processed simultaneously and the resulting unified representation can affect subsequent reading. Likewise, it has been demonstrated that the combination of FRPs with the boundary technique is a suitable method to further determine the variables that can modulate this parallel processing during sentence reading. 4. Method 4.1. Participants Forty-five undergraduate students (35 female, 18–29 years of age, mean age 20). They were right-handed Spanish speakers with no history of neurological impairment. 4.2. Stimuli Three hundred sixty semantically related Spanish word pairs were obtained from the Rules of Free Association in Spanish of the University of Salamanca (www.usal.es/gimc/nalc). Semantic association between primes and parafoveal previews and between primes and targets were manipulated in a 2 2 factorial withinsubject design, resulting in four experimental conditions (examples for the word prime arena; sand): – Related Preview + Related Target (playa + playa; beach + beach) – Related Preview + Unrelated Target (playa + letra; beach + letter) – Unrelated Preview + Related Target (letra + playa; letter + beach) – Unrelated Preview + Unrelated Target (letra + letra; letter + letter) The related preview/target of each prime was used as an unrelated preview/target with a different prime. No participant saw any given word pair more than once but across participants each pair was presented in the four conditions. All words were between 4 and 7 letters long, (mean length: primes = 5.6, previews/targets = 5.3). The average lexical frequency (logFreq: logarithm in base 10 of the raw number of corpus occurrences + 1) for the prime words was 1.2 (SD = 0.77), and for preview/target words was 1.5 (SD = 0.59). The average number of orthographic neighbors for the prime and preview/target words was 6.8 (SD = 6.7) and 9.1 (SD = 8.2) respectively (EsPal database; Duchon, Perea, SebastianGalles, Marti, & Carreiras, 2013). 4.3. Apparatus and procedure The EEG was recorded via 27 Ag/AgCl electrodes referenced to the left mastoid. Four additional external electrodes, two at the outer canthus of each eye and two at the infraorbital and supraorbital regions of the right eye, provided bipolar recordings of the horizontal and vertical electro-oculogram. All electrical activity was recorded and amplified with a bandwidth of 0.01–100 Hz, and sampled at 500 Hz. Impedances were kept below 5 kX (electro-oculogram <10 kX). The recorded data were filtered offline with a band-pass 0.1–30 Hz, and re-referenced to the algebraic mean of the activity at the two mastoids. Eye movements were registered with an EyeLink 1000 eye-tracking system (SR Research Ltd., Ontario, Canada), with a sampling rate of 1000 Hz and an instrument spatial resolution of 0.01°. For each data sample, the EyelinkII saccade parser algorithm computes the instantaneous velocity and acceleration and compares these to threshold criteria for velocity and acceleration (30 deg/s and 8000 deg/s); a distance threshold of 0.1° was used to delay the onset of the saccade until
34
P.J. López-Peréz et al. / Brain & Language 162 (2016) 29–34
the eye had moved significantly. Calibration was performed on a standard nine-point grid. Both systems were synchronized sending TTL pulses form the eye tracker to the EEG recorder. Due to the different sampling rates used in both systems (500 Hz vs 1 kHz), synchronization was verified offline analyzing the correlation between the markers of both recordings (r = 0.97; p = 0.01), and confirming deviations shorter than 5 ms. Each subject was seated at a distance of 60 cm from the computer screen (23 in. Eizo Foris, resolution: 1024 768 pixels, vertical refresh rate: 120 Hz). Words were presented in Courier New black letters against a white background via Experiment Builder software (SR Research Ltd., Canada). Primes and previews were presented in lowercase letters, whereas targets were presented in uppercase. This procedure allow us to avoid physical overlap between words, so potential priming effects can be interpreted in relation their abstract linguistic representations and meanings. The distance between the center of the foveal word and the left edge of the parafoveal word was 2° of visual angle. Parafoveal previews were manipulated using a saccade-contingent display change in a boundary paradigm. The sequence of events in each trial was as follows (see Fig. 1): (1) fixation cross in the center of the screen, (2) once a stable fixation was detected, the fixation cross reduced its size as a preparation cue, (3) after 800 ms, prime-preview words were displayed simultaneously, (4) previews were substituted by the targets when participants shifted their gaze from left to right and crossed the invisible boundary (located between the n and n + 1 regions, at 1.8° from the center of the screen), and (5) the target word was replaced by a question mark at the same location, 400 ms after the target fixation onset, and participants had to decide whether primes and targets were semantically related or not (therefore targets were always displayed for a fixed time). For each participant, each stimulus list was presented in a different random presentation order. In total, each participant saw 360 trials and the experimental session lasted about 90 min. 4.4. Data analysis We report EM data of first-pass fixation relative to the reading of the prime words: first fixation duration and gaze duration. The prime word was presented at the same location as the fixation cross. Even though initial fixations included both the perception of the fixation cross (800 ms) and the prime, the EM measures reported here refer exclusively to the time in which primes were visible on the screen. These data were entered in two separate ANOVAs with the factors PREVIEW (related versus unrelated) and TARGET (related versus unrelated). Because targets were displayed for a fixed time and were replaced by a question mark at the same location, EM data on target words were not analyzed. On average, 11% of trials were excluded because of artifacts (drifts, regressions), anomalous fixations (<50 or >800 ms) or late display changes (>12 ms; 4%). Seventeen participants reported to have detected some kind of change in the parafovea but none of them could establish what kind of change, or reported any particular word. The average number of accepted trials was equally distributed across experimental conditions and participants. The EEG data were segmented in epochs of interest: 200 to 1500 ms time-locked to the prime-preview onset presentation, and 700 to 2000 ms time-locked to the saccade offset. Baseline correction was performed taking 200 ms previous to stimulus onset for the ERPs, and between 700 and 500 ms before saccade offset for the FRPs. Artifacts were removed semi-automatically with rejection values adjusted for each participant. ERP and FRP mean amplitudes of the selected time-windows were analyzed in separate ANOVAs with the factors PREVIEW (related versus unre-
lated) and TARGET (related versus unrelated) and the topographical factor ELECTRODE. For this factor, nine different electrode groups were computed, each compromising the mean of 3 electrodes: Left anterior (Fp1, F7, F3), left central (FC5, T7, C3), left posterior (CP5, P7, P3), right anterior (Fp2, F8, F4), right central (FC6, T8, C4), right posterior (CP6, P8, P4), medial anterior (Fz, FC1, FC2), medial central (Cz, CP1, CP2), and medial posterior (Pz, O1, O2). Statistical analyses were performed with R software (http:// www.rproject.org). Results reported include the original degrees of freedom and p values corrected for violation of sphericity using the Greenhouse-Geisser epsilon. Acknowledgments HAB was supported by the grand PSI2013-46334-P of the Spanish Ministry of Economy. References Baccino, T., & Manunta, Y. (2005). Eye-fixation-related potentials: Insight into parafoveal processing. Journal of Psychophysiology, 19(3), 204–215. Barber, H. A., Doñamayor, N., Kutas, M., & Münte, T. (2010). Parafoveal N400 effect during sentence reading. Neuroscience Letters, 479(2), 152–156. Barber, H. A., Ben-Zvi, S., Bentin, S., & Kutas, M. (2011). Parafoveal perception during sentence reading? An ERP paradigm using rapid serial visual presentation (RSVP) with flankers. Psychophysiology, 48(4), 523–531. Barber, H. A., Van der Meij, M., & Kutas, M. (2013). An electrophysiological analysis of contextual and temporal constraints on parafoveal word processing. Psychophysiology, 50(1), 48–59. Dampuré, J., Ros, C., Rouet, J.-F., & Vibert, N. (2014). Task-dependent sensitization of perceptual and semantic processing during visual search for words. Journal of Cognitive Psychology, 26, 530–549. Dimigen, O., Kliegl, R., & Sommer, W. (2012). Trans-saccadic parafoveal preview benefits in fluent reading: A study with fixation-related brain potentials. NeuroImage, 62(1), 381–393. Dimigen, O., Sommer, W., Hohlfeld, A., Jacobs, A. M., & Kliegl, R. (2011). Coregistration of eye movements and EEG in natural reading: Analyses and review. Journal of Experimental Psychology: General, 140(4), 552–572. Drieghe, D., Rayner, K., & Pollatsek, A. (2008). Mislocated fixations can account for parafoveal-on-foveal effects in eye movements during reading. Quarterly Journal of Experimental Psychology, 61(8), 1239–1249. Duchon, A., Perea, M., Sebastian-Galles, N., Marti, A., & Carreiras, M. (2013). EsPal: One-stop shopping for Spanish word properties. Behavior Research Methods, 45, 1246–1258. Hohenstein, S., Laubrock, J., & Kliegl, R. (2010). Semantic preview benefit in eye movements during reading: A parafoveal fastpriming study. Journal of Experimental Psychology: Learning, Memory, and Cognition, 36, 1150–1170. Hutzler, F., Braun, M., Vö, M. L. H., Engl, V., Hofmann, M., Dambacher, M., & Jacobs, A. M. (2007). Welcome to the real world: Validating fixation-related brain potentials for ecologically valid settings. Brain Research, 1172, 124–129. Kennedy, A., & Pynte, J. (2005). Parafoveal-on-foveal effects in normal reading. Vision Research, 45(2), 153–168. Kretzschmar, F., Bornkessel-Schlesewsky, I., & Schlesewsky, M. (2009). Parafoveal versus foveal N400s dissociate spreading activation from contextual fit. NeuroReport, 20(18), 1613–1618. Kretzschmar, F., Schlesewsky, M., & Staub, A. (2015). Dissociating word frequency and predictability effects in reading: Evidence from co-registration of eye movements and EEG. Journal of Experimental Psychology: Learning, Memory, and Cognition, 41(6), 1648–1662. Kutas, M., & Federmeier, K. D. (2011). Thirty years and counting: Finding meaning in the N400 component of the event-related brain potential (ERP). Annual Review Psychology, 62, 621–647. Li, N., Niefind, F., Wang, S., Sommer, W., & Dimigen, O. (2015). Parafoveal processing in reading Chinese sentences: Evidence from event-related brain potentials. Psychophysiology, 52(10), 1361–1374. Rayner, K. (1975). The perceptual span and peripheral cues in reading. Cognitive Psychology, 7, 65–81. Schotter, E. R., Angele, B., & Rayner, K. (2012). Parafoveal processing in reading. Attention, Perception, & Psychophysics, 74(1), 5–35. Simola, J., Holmqvist, K., & Lindgren, M. (2009). Right visual field advantage in parafoveal processing: Evidence from eye-fixation-related potentials. Brain and Language, 111(2), 101–113. Vitu, F., Brysbaert, M., & Lancelin, D. (2004). A test of parafovealon-foveal effects with pairs of orthographically related words. European Journal of Cognitive Psychology, 16, 154–177. Yan, M., Richter, E. M., Shu, H., & Kliegl, R. (2009). Readers of Chinese extract semantic information from parafoveal words. Psychonomic Bulletin & Review, 16 (3), 561–566.