- Email: [email protected]
Attenuation of visual evoked responses to hand and saccade-initiated flashes
Cognition 179 (2018) 14–22
Contents lists available at ScienceDirect
Cognition journal homepage: www.elsevier.com/locate/cognit
Original Articles
Attenuation of visual evoked responses to hand and saccade-initiated flashes a,⁎
a
b
a
T
a
Nathan G. Mifsud , Tom Beesley , Tamara L. Watson , Ruth B. Elijah , Tegan S. Sharp , Thomas J. Whitforda a b
School of Psychology, UNSW Sydney, Sydney, New South Wales, Australia School of Social Sciences and Psychology, Western Sydney University, Bankstown, New South Wales, Australia
A R T I C LE I N FO
A B S T R A C T
Keywords: Event-related potentials Predictive processing Sensory attenuation Saccadic movements Efference copy Corollary discharge
Sensory attenuation refers to reduced brain responses to self-initiated sensations relative to those produced by the external world. It is a low-level process that may be linked to higher-level cognitive tasks such as reality monitoring. The phenomenon is often explained by prediction error mechanisms of universal applicability to sensory modality; however, it is most widely reported for auditory stimuli resulting from self-initiated hand movements. The present series of event-related potential (ERP) experiments explored the generalizability of sensory attenuation to the visual domain by exposing participants to flashes initiated by either their own button press or volitional saccade and comparing these conditions to identical, computer-initiated stimuli. The key results showed that the largest reduction of anterior visual N1 amplitude occurred for saccade-initiated flashes, while button press-initiated flashes evoked an intermediary response between the saccade-initiated and externally initiated conditions. This indicates that sensory attenuation occurs for visual stimuli and suggests that the degree of electrophysiological attenuation may relate to the causal likelihood of pairings between the type of motor action and the modality of its sensory response.
1. Introduction Sensory attenuation refers to self-initiated stimuli evoking reduced neurophysiological (e.g., Baess, Jacobsen, & Schröger, 2008; Houde, Nagarajan, Sekihara, & Merzenich, 2002; Schafer & Marcus, 1973) and phenomenological (e.g., Blakemore, Frith, & Wolpert, 1999; CardosoLeite, Mamassian, Schütz-Bosbach, & Waszak, 2010; Sato, 2008) sensory representations, compared to the sensory representations evoked by physically identical, externally initiated stimuli. The attenuation taking place here is thus related to a new external stimulus that is a consequence of enacting a motor action; a phenomenon considered here as separate to the suppression of the sensory consequences of enacting a movement within a constant sensory environment (e.g., motion across the retina produced by an eye movement). The phenomenon is typically explained using a forward model that predicts the sensory consequences of intended actions based on internal motor commands, where these predictions are subtracted from actual sensory input (Bays & Wolpert, 2007; Wolpert, Ghahramani, & Jordan, 1995). Conversely, externally initiated stimuli lack accompanying motor information, and are thus marked by a larger disparity between predicted and actual sensory inputs—a distinction that may play a central role in cognition; specifically, our sense of agency (Engbert, Wohlschlager, & Haggard, 2008; Subramaniam, Kothare, Mizuiri, ⁎
Nagarajan, & Houde, 2018). Notably, Feinberg (1978) first suggested that disruption of this distinction between self and the external world could account for some of the characteristic symptoms of schizophrenia (e.g., delusions of control), and evidence has emerged to support this theory (Ford et al., 2001; Pinheiro, Rezaii, Rauber, & Niznikiewicz, 2016; Whitford et al., 2011). Studies of sensory attenuation have thus far largely been limited to the auditory domain (e.g., see Table 3 in Hughes, Desantis, & Waszak, 2013), centerd on a reliable event-related potential (ERP) component that is used by multiple research groups as an index of sensory attenuation—that is, the N1 or N1m component, an evoked potential or magnetic field that is consistently reduced for self-initiated vocalizations and tones (e.g., Baess et al., 2008; Curio, Neuloh, Numminen, Jousmaki, & Hari, 2000; Houde et al., 2002; Mifsud & Whitford, 2017; Sowman, Kuusik, & Johnson, 2012). Given the well-established positive relationship between the auditory N1 component and stimulus intensity (Näätänen & Picton, 1987)—i.e., loud sounds evoke larger auditory N1 amplitudes than do soft sounds—the finding that self-initiated sounds have a reduced auditory N1 response suggests that the brain processes them as being “softer”. This reduced perceived loudness of self-initiated sounds may reflect an ecological adaptation, in the sense that the strong auditory feedback associated with our own speech may require attenuation to preserve the sensitivity of receptors to incoming sounds
Corresponding author. School of Psychology, Mathews Building, UNSW Sydney, Sydney, NSW 2052, Australia. E-mail address: [email protected] (N.G. Mifsud).
https://doi.org/10.1016/j.cognition.2018.06.005 Received 18 June 2017; Received in revised form 4 June 2018; Accepted 5 June 2018 0010-0277/ Crown Copyright © 2018 Published by Elsevier B.V. All rights reserved.
Cognition 179 (2018) 14–22
N.G. Mifsud et al.
sensitive to loudness. The visual N1 component is a likely candidate, as an early, sensory-evoked component that appears to be partially generated from occipital cortex (Clark, Fan, & Hillyard, 1995), but (to our knowledge) no previous studies explicitly demonstrate that the visual N1 (i.e., frontocentral maxima, mastoid-referenced) is sensitive to changes in luminance. Hence, the primary aim of Experiment 1—which only measured ERP responses to passively viewed stimuli—was to demonstrate the luminance-dependence of the visual N1.
(Bendixen, SanMiguel, & Schröger, 2012). In contrast to the auditory domain, studies of sensory attenuation in the visual domain are relatively scarce, and results are less easily reconciled. Reported differences in visual-evoked potentials (VEPs) between self- and externally initiated visual stimuli are inconsistent in terms of both their direction and spatial location. This may be due to a diverse range of stimuli, and, relatedly, the chosen event-related components and reference electrode sites. Self-initiation has been shown to result in anterior (but not occipital) reduction of N1 for flashes (Schafer & Marcus, 1973, mastoid-referenced data) and arrow shapes (Gentsch & Schütz-Bosbach, 2011, average-referenced data), and occipital reduction of P2 for faces and houses (Hughes & Waszak, 2014, FCz-referenced data). Conversely, occipital amplification of P1 has been shown for pattern-onset stimuli (Hughes & Waszak, 2011, vertex-referenced data) and occipital amplification of N145 for pattern-reversal stimuli (Mifsud, Oestreich, et al., 2016, Fz-referenced data). However, sensory attenuation has also been observed in behavioural tasks using Gabor patches (Cardoso-Leite et al., 2010; Stenner, Bauer, Haggard, Heinze, & Dolan, 2014). The clear differences in the reported results means that further experimental work is required in the visual domain that builds on existing self-initiation paradigms. A further limiting factor of previous studies of sensory attenuation in the visual domain is that nearly all self-initiation conditions have involved button pressing to receive a visual stimulus. This highly specific experimental condition, while relevant, must be considered in conjunction with other action–sensation contingencies to account for the wide range of circumstances that may involve sensory attenuation. In other words, there is insufficient support for assuming that findings from button-press studies can be generalized to other action–sensation contingencies. In the auditory domain, the limits of this assumption have been tested by van Elk, Salomon, Kannape, and Blanke (2014) and Mifsud, Beesley, Watson, and Whitford (2016), who employed paradigms using foot and saccade initiation respectively to demonstrate that differences in auditory-evoked potentials were dependent on the region of motor output used to produce the incoming stimulus. In the study by Mifsud, Beesley, et al. (2016), for example, a greater degree of auditory N1 attenuation was observed for button press-initiated tones than for saccade-initiated tones, consistent with the fact that while hand movements are strongly associated with auditory sensations (e.g., the sound of one’s fingers typing on a keyboard), eye movements are rarely, if ever, accompanied by auditory feedback. The present series of experiments explored whether a similar pattern of ERP effects would be observed for self-initiated visual stimuli. In Experiment 1, we tested the luminance-dependence of the frontocentral visual N1 component, and thus its conceptual similarity to the loudnessdependent auditory N1 measure used in previous sensory attenuation studies. In Experiment 2, we tested the influence of self-initiation of visual stimuli on visual N1 amplitude, extending the saccade initiation paradigm (Mifsud, Beesley, et al., 2016) to the visual domain. Lastly, in Experiment 3, we replicated the self-initiation manipulation of Experiment 2 with an added condition designed to probe the effect of temporal predictability.
2.1. Method 2.1.1. Participants Eleven participants were recruited at UNSW Sydney. Six were female, 8 were right-handed, and mean age was 19 years (SD = 1). Participants gave written, informed consent, and received course credit in exchange for their time. This experiment, and the two that follow, were approved by the UNSW Human Research Ethics Advisory Panel (Psychology). 2.1.2. Procedure Following provision of their demographical information, participants were fitted with an EEG cap and electrodes. EEG was continuously recorded while participants completed the experiment, seated 60 cm from a computer monitor with an integrated eye tracking system (Tobii TX300: 300 Hz gaze sampling rate; 23″, 60 Hz, 1920 × 1080 resolution TFT screen; accuracy of 0.4° visual angle; system latency under 10 ms). The eye tracking function was not used in Experiment 1, but was required for saccade detection in Experiment 2. The experiment comprised of a series of stimulus presentations of four different types: an unstructured full-field white flash of 33.33 ms duration (i.e., two frames, verified with a photometer), that was one of two mean luminance levels, dim (10 cd/m2) or bright (100 cd/m2); and two types of pure tones (the data for which are not presented). The mean luminance levels were approximated based on measurements with a handheld instrument (Minolta Chroma Meter CS-100A). Participant input was not required at any time. Each trial type was shown for 120 trials in total, intermixed in a total 480-trial sequence whose order was randomized between participants and split into 10 equal blocks separated by 30-s rest periods. Individual trials were separated by a uniformly distributed random interval (1–4 s). The EEG recording lasted approximately 30 min. Stimulus presentation was controlled by MATLAB (MathWorks, Natick, US) using the Psychophysics Toolbox extensions (Brainard, 1997; Kleiner, Brainard, & Pelli, 2007; Pelli, 1997). EEG was recorded with a BioSemi ActiveTwo system using 64 AgAgCl active electrodes placed according to the extended 10–20 system. Analog signals were anti-aliased with a fixed first-order filter (−3 dB at 3600 Hz) and continuously digitized at a sampling frequency of 2048 Hz, with common mode sense (CMS) and driven right leg (DRL) used as reference and ground electrodes. During offline preprocessing, data were re-referenced to the averaged mastoid electrodes as is typical for the visual N1 (Clark et al., 1995; Vogel & Luck, 2000), band-pass filtered from 0.01 to 30 Hz (8th order zero-phase Butterworth IIR), and separated into 600-ms epochs (100 ms pre-onset and 500 ms postonset). Data were baseline corrected with the average voltage between −100 and 0 ms. To address eye blinks and movement artefacts, we rejected individual epochs at any electrode site that contained EEG activity exceeding ± 75 µV or min-max changes in excess of 75 µV between adjacent 100-ms intervals. Individual trials were then averaged for each condition to produce ERPs for each participant. Data preprocessing was done in BrainVision Analyzer 2 (Brain Products GmbH, Munich, Germany), and statistical analyses were performed in SPSS version 23 (IBM Corp, Armonk, US). As the latency and amplitude of flash ERPs are sensitive to stimulus parameters, a collapsed localizer approach was used to guide our analysis (Luck & Kappenman, 2012). The waveforms of the dim and bright
2. Experiment 1 An underlying premise of neurophysiological sensory attenuation is that reductions in the ERP reflected alterations in the perceived intensity of a stimulus. For example, decreased auditory N1 amplitude has been consistently observed for self-initiated auditory tones (e.g., Sowman et al., 2012) and auditory N1 amplitude is known to decrease with decreasing stimulus intensity (Näätänen & Picton, 1987). As mentioned earlier, this suggests that self-initiated sounds are processed as “softer” (Weiss, Herwig, & Schütz-Bosbach, 2011). If we are to measure sensory attenuation in the visual domain, we must use a VEP component that reflects the intensity of a visual stimulus, in the same manner that the N1 component of the auditory-evoked potential is 15
Cognition 179 (2018) 14–22
N.G. Mifsud et al.
Fig. 1. Grand-averaged ERPs for the bright and dim flash conditions at electrodes Fz, FC1, FCz, FC2, and Cz. The x-axes represent time in milliseconds (ms) where 0 is flash onset, and the y-axes represent amplitude in microvolts (µV). Grey areas indicate measurement windows.
luminance. Specifically, a passively viewed bright flash produced larger mean visual N1 amplitude than a passively viewed dim flash at FCz and its neighbours. This demonstration of luminance-dependence suggests that the amplitude of the flash-evoked visual N1 could, in subsequent studies that incorporate self-initiation conditions, serve as a proxy measurement for sensory attenuation in the visual domain (an analogue to the well-established use of the auditory N1 in sensory attenuation studies in the auditory domain). Accordingly, in Experiment 2, we again measured the flash-evoked visual N1 to test the hypothesis that sensory attenuation may be influenced by the causal likelihood of pairings between action and visual sensation. Crucially, in Experiment 2, stimulus intensity was held constant (i.e., all flashes had the same luminance as the bright flash in Experiment 1) and the experimental manipulation was the type of initiating motor action.
conditions were averaged, and this collapsed waveform was used to identify a measurement window centerd on the peak and electrode site at which the visual N1 was maximal. A two-way analysis of variance (ANOVA) was then conducted to examine the effects of flash intensity on the mean amplitude of the visual N1 (within the time window of 149–159 ms). The Condition factor consisted of two levels (dim, bright) and the Site factor consisted of five levels: of primary interest, the maximal N1 site (FCz), and its adjacent electrodes (i.e., Fz, FC1, FC2, and Cz). 2.2. Results and discussion Fig. 1 presents the grand-averaged ERPs for the 5-site cluster (Fz, FC1, FCz, FC2, and Cz). Fig. 2 presents the scalp distributions for each stimulus condition. A main effect of Condition, F(1,10) = 30.03, p < .001, ηp2 = .75, indicated that mean visual N1 amplitude across electrodes Fz, FC1, FCz, FC2, and Cz differed between the dim (M = −2.60, SD = 0.51) and bright (M = −5.27, SD = 0.51) conditions. There was neither a main effect of Site, F(4,40) = 0.18, p = .728, ηp2 = .02, nor an interaction between Condition and Site, F(4,40) = 1.24, p = .312, ηp2 = .11. A follow-up pairwise comparison confirmed that mean visual N1 amplitude between the dim (M = −2.70, SD = 1.79) and bright (M = −5.36, SD = 1.75) conditions at the maximal site (FCz) differed significantly, F(1,10) = 13.54, p < 001. This indicated that the mastoid-referenced visual N1 at frontocentral sites was sensitive to
3. Experiment 2 We aimed to determine whether VEP attenuation would occur for button-press and saccade-initiated flashes compared to externally initiated flashes. We expected that button press-initiated flashes would evoke attenuated visual N1 based on a similar condition reported by Schafer and Marcus (1973). Notably, however, we hypothesized that there would be increased visual N1 attenuation for saccade-initiated flashes compared to button press-initiated flashes, given that eye movements, rather than hand movements, are more tightly coupled to 16
Cognition 179 (2018) 14–22
N.G. Mifsud et al.
Fig. 2. Topographic maps of the visual N1 (149–159 ms) for each stimulus condition.
visual sensations. 3.1. Method 3.1.1. Participants Forty participants were recruited at UNSW Sydney. Seven participants were excluded due to their dataset lacking discernible flash ERP waveforms in one or more stimulus blocks. Of the remaining 33 participants, 15 were female, 29 were right-handed, and mean age was 22 years (SD = 6). Participants gave written, informed consent, and received either course credit (n = 21) or financial reimbursement (n = 12, A$30) in exchange for their time. 3.1.2. Procedure The prerecording routine, spatial arrangements, and hardware specifications were identical to Experiment 1, with an additional 5point procedure to calibrate the eye tracking system for saccade detection. Experiment 2 comprised five conditions: two types of self-initiation conditions (i.e., button-press and saccade-initiated flashes) and their corresponding motor control conditions (i.e., button presses and saccades without consequent flashes), and an externally initiated condition (i.e., flashes initiated without participant input). Each condition was presented in a homogenous 80-trial block, and block order was randomized between participants. Three practice trials preceded each block to ensure that participants understood the instructions displayed on screen, and, where appropriate, allowed the experimenter to verbally encourage self-paced rather than speeded responses. Individual trials in all conditions were separated by a blank, black screen for a uniformly distributed random interval (2–4 s), which began following the end of the flash (or, in the motor conditions, after the disappearance of the fixation dot). The EEG recording lasted approximately 50 min. Fig. 3 depicts the experimental protocol.
Fig. 3. Experimental protocol. In the press-initiated condition, participants depressed the space bar at will any time after a fixation dot appeared, which immediately delivered a flash (two frames of white fill). In the saccade-initiated condition, participants focused on a distal dot and then shifted at will to the central fixation dot, which immediately delivered a flash. In the externally initiated condition, flash delivery followed a variable delay without motor input. Control conditions were identical to their respective stimulus conditions, except that motor input did not result in a stimulus.
recordings. If detection took longer than 5 s, trials were skipped with replacement (M = 2.7 skipped trials per participant across both stimulus and motor saccade-initiated blocks). Following fixation on the white circle, participants shifted their gaze at will to the red circle, which immediately delivered a full-field flash identical to the pressinitiated stimulus. More precisely, flashes followed detection of the gaze within the 200-px (5°) square area of interest surrounding the central red circle. We confirmed that system latency was identical in both self-initiation conditions using a photometer to detect actual delivery of the flash. Specifically, mean latency between the end of the action (button press or the eye attaining fixation in the center) and stimulus delivery was 37 ms ± 7 ms (SD) trials in both conditions.
3.1.2.1. Press condition. In this condition, a visual stimulus was selfinitiated by a button press (i.e., hand motor output). Participants were instructed to respond at will any time after the appearance of a red fixation dot (0.7° diameter) presented in the center of a black screen, and did so by pressing the space bar on a low-latency keyboard (Ducky Shine 4: 1000 Hz report rate) with their dominant hand. Responses immediately replaced the fixation screen with a full-field flash identical to the bright flash used in Experiment 1 (i.e., 100 cd/m2 mean luminance, two-frame duration).
3.1.2.3. Motor conditions. The motor control conditions were identical to their respective self-initiated conditions, except that pressing the space bar or shifting gaze between circles did not result in the delivery of a stimulus. The ensuing EEG activity was subsequently subtracted from the appropriate self-initiated conditions to remove EEG activity associated with button pressing (for the press condition) or a singular, volitional eye movement (for the saccade condition), as is standard practice in button-press studies of this nature (Baess et al., 2008; Martikainen, Kaneko, & Hari, 2005; Whitford et al., 2011), and as was previously used for the saccade initiation condition in the study of
3.1.2.2. Saccade condition. In this condition, the visual stimulus was self-initiated by a volitional saccade (i.e., eye motor output). Each trial began with two dots appearing on screen: a solid red circle in the center of screen (identical to the fixation in the press-initiated condition) and a distal (17° left) hollow white circle. Participants were instructed to initially fixate on the white circle, which would turn solid once the script detected their gaze, based on a 20 ms sample of location 17
Cognition 179 (2018) 14–22
N.G. Mifsud et al.
used Fisher’s protected least significant difference (LSD) procedure to examine pairwise comparisons, which controls the family-wise type I error rate when there are exactly 3 groups and a significant omnibus test (Meier, 2006). The follow-up pairwise comparisons indicated that external differed significantly from press, F(1,32) = 4.89, p = .034, as well as saccade, F(1,32) = 16.54, p < .001. Likewise, press and saccade differed significantly from each other, F(1,32) = 4.92, p = .034. Crucially, this demonstrated that saccade-initiated flashes evoked a significant reduction in mean visual N1 amplitude at FCz compared to both button press- and externally initiated flashes. Similarly, for the 5-site cluster, a main effect of Condition, F(2,64) = 10.41, p < .001, ηp2 = .25, indicated that mean visual N1 amplitude across electrodes Fz, FC1, FCz, FC2, and Cz differed between the press (M = −4.04, SD = 0.67), saccade (M = −1.54, SD = 0.83), and external (M = −5.37, SD = 0.59) conditions. Follow-up pairwise comparisons indicated that external differed significantly from saccade, F(1,32) = 17.31, p < .001, but not press, F(1,32) = 3.52, p = .070. However, press and saccade differed significantly from each other, F (1,32) = 7.52, p = .010. There was neither a main effect of Site, F (4,128) = 1.51, p = .223, ηp2 = .05, nor an interaction between Condition and Site, F(8,256) = 1.68, p = .104, ηp2 = .05. Like the FCz analysis, the 5-site cluster analysis therefore demonstrated that saccade initiation led to significantly reduced mean visual N1 amplitude compared to external initiation. As predicted, button-press initiation led to a smaller, but still significant, reduction in mean visual N1 amplitude, representing an intermediary point between the saccade and external conditions.
Mifsud, Beesley, et al. (2016). 3.1.2.4. External condition. In this condition, stimuli were delivered automatically (i.e., without participant input) to assess electrophysiological response to externally initiated, temporally unpredictable stimuli. Trials began with a red fixation dot followed by a uniformly distributed random interval (0.5–2.5 s) before a flash was presented (identical to that in the self-initiated conditions). Participants were instructed to keep their eyes open and maintain their gaze on the screen at all times. 3.1.2.5. EEG data acquisition. EEG was recorded and preprocessed exactly as described in Experiment 1, including data re-referencing to the averaged mastoid electrodes, and a step function artefact rejection procedure that resulted in a mean rejection rate at electrode FCz of 4.5% ± 5.5% (SD) trials (press: 4.7% ± 6.0%, saccade: 3.9% ± 6.4%, external: 5.0% ± 5.5%), with no significant differences between stimulus blocks, F(2,64) = 0.99, p = .378, ηp2 = .03. Once individual trials for each condition were averaged to produce ERPs for each participant, motor waveforms were subtracted from the appropriate self-initiated waveforms to produce difference waveforms. Fig. 4 demonstrates the effect of the motor subtraction procedure by presenting grand-averaged ERPs at electrode FCz for the uncorrected waveforms in the self-initiated conditions and their corresponding motor waveforms. Hereafter, unless explicitly noted otherwise, mentions of self-initiated waveforms refer to motorcorrected waveforms. To investigate the effect of self-initiation on the visual ERP, separate ANOVAs were conducted on the Condition factor (press, saccade, external) on the mean amplitudes of the visual N1 (150–160 ms) at electrode FCz and for a 5-site cluster that included adjacent electrodes (i.e., Fz, FC1, FC2, and Cz). In cases where the assumption of sphericity was violated, a Greenhouse-Geisser correction was applied. This analysis mirrored Experiment 1, with a trivially different measurement window (i.e., 149–159 ms vs. 150–160 ms) due to the latency of the peak identified in the collapsed waveform formed from the average of the press, saccade, and external conditions.
4. Experiment 3 An issue in contingency paradigms exploring sensory attenuation is the inherent difference in the temporal predictability of stimuli across self- and externally initiated conditions. As participants choose when to initiate stimuli in self-initiated conditions, the stimulus onset can be accurately anticipated, unlike externally initiated conditions in which stimuli are initiated without warning (see Hughes et al., 2013). Various attempts have been made to control for this by “cueing” externally initiated stimuli (Ford, Gray, Faustman, Roach, & Mathalon, 2007; Lange, 2009; Mifsud, Oestreich, et al., 2016; Oestreich et al., 2015; Schafer, Amochaev, & Russell, 1981), and, though observations are varied (see Lange, 2013), temporal predictability appears to partly contribute to observed ERP effects in the auditory domain. Hence, because the present study is motivated by putative cross-modal effects, Experiment 3 introduced a cued condition to determine if temporal predictability influenced the attenuation of visual N1 amplitude observed to saccade-initiated flashes.
3.2. Results and discussion Fig. 5 presents the grand-averaged ERPs at electrode FCz and adjacent sites. Fig. 6 presents the scalp distributions for each stimulus condition. At electrode FCz, a main effect of Condition, F(2,64) = 9.19, p < .001, ηp2 = .22, indicated that mean visual N1 amplitude differed between the press (M = −3.88, SD = 4.34), saccade (M = −1.61, SD = 5.10), and external (M = −5.59, SD = 3.45) conditions. We then
Fig. 4. Grand-averaged ERPs at electrode FCz for uncorrected self-initiated conditions (solid traces, left), their corresponding motor conditions (dotted traces, left), and motor-corrected self-initiation conditions (right). The externally initiated condition (black line, both panels) is included for comparison. The x-axes represent time in milliseconds (ms) where 0 is flash onset, and the y-axes represent amplitude in microvolts (µV). 18
Cognition 179 (2018) 14–22
N.G. Mifsud et al.
Fig. 5. Grand-averaged ERPs for press, saccade, and external conditions at electrodes Fz, FC1, FCz, FC2, and Cz. Self-initiated conditions (i.e., press and saccade) are motor-corrected. The x-axes represent time in milliseconds (ms) where 0 is flash onset, and the y-axes represent amplitude in microvolts (µV). Grey areas indicate measurement windows.
4.1.2. Procedure Experiment 3 comprised six conditions: the same five conditions in Experiment 2 (button-press and saccade initiation, their motor-only controls, and an externally initiated, temporally unpredictable condition; refer to Fig. 3) plus a cued condition (externally initiated, temporally predictable stimuli). In this condition, flashes were delivered without participant input following a countdown composed of fixation targets (i.e., ‘• • •’, then ‘• •’, then ‘•’), each of which were separated by a 500 ms interval. All other characteristics of the experiment, including EEG data acquisition, were identical to Experiment 2. A one-way
4.1. Method 4.1.1. Participants Thirty-two participants were recruited at UNSW Sydney. Seven participants were excluded due to their dataset lacking discernible flash ERP waveforms in one or more stimulus blocks. Of the remaining 25 participants, 13 were female, 23 were right-handed, and mean age was 23 years (SD = 6). Participants gave written, informed consent, and received course credit in exchange for their time.
Fig. 6. Topographic maps of the visual N1 (150–160 ms) for each stimulus condition. Self-initiated conditions (i.e., press and saccade) are motor-corrected. 19
Cognition 179 (2018) 14–22
N.G. Mifsud et al.
Fig. 7. Grand-averaged ERPs for press, saccade, external, and cued conditions at electrode Fz. Self-initiated conditions (i.e., press and saccade) are motor-corrected. The x-axes represent time in milliseconds (ms) where 0 is flash onset, and the y-axes represent amplitude in microvolts (µV).
saccadic production and control around the immediate time of saccadic onset (e.g., Reingold & Stampe, 2002) and physiological effects during reading (Ibbotson & Krekelberg, 2011; Yang & McConkie, 2001), the present study focused on the sensory consequences cued by the eye movement, rather than the movement itself. The key novel finding, demonstrated in Experiment 2 and replicated in Experiment 3, was that saccade-initiated flashes evoked attenuated visual N1 amplitude compared to both button press- and externally initiated flashes. It is unlikely that the observed attenuation is attributable to saccadic suppression; given the size of the initiating eye movements, the flashes were presented outside the time course of such effects (see Diamond, Ross, & Morrone, 2000). Importantly, as Experiment 1 showed that visual N1 amplitude was luminance-dependent, and given the similarities between both the procedures and results presented here and those in the auditory domain (e.g., Baess et al., 2008; Sowman et al., 2012; Whitford et al., 2011), we suggest that sensory attenuation due to stimulus selfinitiation takes place in the visual domain as well, and that flash-evoked visual N1 amplitude may be a measure well-suited for further investigation. The attenuation of visual N1 amplitude that followed button-press initiation was less marked. At the primary site of analysis (FCz) in Experiment 2, significant reductions were observed for button-press compared to externally initiated flashes, but this effect disappeared when using an arguably better externally initiated condition (i.e., temporally cued stimuli) in Experiment 3. There are previous reports of attenuated anterior visual N1 amplitude for button press-initiated visual stimuli (Gentsch & Schütz-Bosbach, 2011; Schafer & Marcus, 1973). Indeed, Gentsch and Schütz-Bosbach (2011) observed that the quantitative effect of button-press initiation on visual N1 amplitude was less than effects reported in the literature measuring the auditory N1 (e.g., Heinks-Maldonado, Mathalon, Gray, & Ford, 2005). Hence, the relative weakness of button press-produced sensory attenuation in the
ANOVA was conducted on the Condition factor (press, saccade, external, cued) on the mean amplitudes of the visual N1 (125–135 ms) at electrode FCz, with peak latency determined by the collapsed waveform of all stimulus conditions. Given that Experiment 2 involved four groups, a Bonferroni adjustment for multiple comparisons was used for the follow-up pairwise comparisons, providing an alpha value of .008 (.05 divided by k = 6 comparisons). 4.2. Results and discussion Fig. 7 presents the grand-averaged ERPs at electrode FCz. Fig. 8 presents the scalp distributions for each stimulus condition. A main effect of Condition, F(2,72) = 11.94, p < .001, ηp2 = .33, indicated that there was a difference in mean visual N1 amplitude at electrode FCz among the press (M = −6.20, SD = 3.91), saccade (M = −3.35, SD = 3.73), external (M = −7.41, SD = 2.92), and cued (M = −6.65, SD = 3.03) conditions. Follow-up pairwise comparisons indicated that saccade differed significantly from press, F(1,24) = 8.62, p = .006, external, F(1,24) = 22.96, p < .001, and cued, F(1,24) = 13.96, p < .001. Press did not significantly differ from external, F(1,24) = 2.38, p = .062, or cued, F(1,24) = 0.31, p = .498, nor did external significantly differ from cued, F(1,24) = 1.00, p = .199. The results indicated that saccade-initiated flashes evoked a significant reduction in mean visual N1 amplitude at FCz compared to button press- and externally initiated conditions, regardless of temporal predictability. 5. General discussion The present study used a saccadic paradigm to investigate the effect of self-initiation on subsequent visual response for the first time, extending similar work conducted in the auditory domain (Mifsud, Beesley, et al., 2016). While a corpus of research has focused on
Fig. 8. Topographic maps of the visual N1 (125–135 ms) for each stimulus condition. Self-initiated conditions (i.e., press and saccade) are motor-corrected. 20
Cognition 179 (2018) 14–22
N.G. Mifsud et al.
Appendix A. Supplementary material
visual domain may help to explain the heterogeneity of our buttonpress initiation findings. Note also that temporal cuing of externally initiated stimuli in Experiment 3 did not evoke significantly reduced visual N1 amplitude compared to unpredictable, externally initiated stimuli. This may have been due to nature of the full-screen flash: comparable studies that have involved cueing of bright, foveal stimuli, such as crosses (Miniussi, Wilding, Coull, & Nobre, 1999) and patternreversals (Mifsud, Oestreich, et al., 2016), were associated with an absence of VEP modulation. Importantly, we can still compare relative degrees of visual N1 attenuation across different types of initiating motor output, as saccade initiation was significantly stronger than button-press initiation in both Experiments 2 and 3. Visual N1 attenuation associated with eye movements was greater than the N1 attenuation associated with finger movements (i.e., button-press initiation), even when controlling for between-condition differences in motor-evoked potentials. It is conceivable that the size of each effect relates to the strength of the habitual association between the type of motor action (eye or hand movement) and the resultant perceptual sensations (flashes). That is, the strength of pre-existing associations gained throughout a lifetime of experience appears to positively correlate with the degree of sensory attenuation; eye movements are strongly associated with changes in visual sensation, whereas hand movements are only weakly related to changes in visual sensation. This hypothesis accords with saccade-initiated tones producing less auditory N1 attenuation than button pressinitiated tones (Mifsud, Beesley, et al., 2016), perhaps because hand movements produce stronger auditory feedback than do saccades (which may indeed have some influence on hearing; Gruters et al., 2018). The overall correspondence between these studies is that electrophysiological sensory attenuation for an initiating motor action is stronger when there is a high causal likelihood that it is typically paired with the sensory modality under investigation (i.e., saccade–visual and press–auditory). This can be aligned with the popular forward model account of sensory attenuation, in which the predicted sensory consequences of an action in the form of physical, corollary signals modulate actual sensory feedback. Specifically, corollary signals seem most likely to exist where there are direct neural connections between the relevant areas of the brain. Such sensorimotor connections conceivably exist between the motor area of the brain involved in eye movements and the visual cortex (e.g., the frontal eye field in prefrontal cortex, see Schall, 2002), whereas it may be less likely that such established connections exist between the visual cortex and the parts of the motor cortex involved in hand movements. After demonstrating the luminance-dependence of the visual N1, we employed a new saccade initiation paradigm to investigate the sensory attenuation of visual stimuli that were initiated by either a saccade or button press, introducing a novel procedure to complement the large body of evidence showing sensory attenuation for auditory stimuli that are initiated by button press. We observed attenuation of visual N1 amplitude following saccade initiation, over and above that which occurred following button press-initiation, and suggested that this may relate to the strength of the association between eye movements and visual events. The experiments in the present study indicate that expanding the repertoire of action–sensation contingencies studied in the literature may shed light on the mechanisms underlying of the ubiquitous sensory attenuation phenomenon.
The electronic version of this article is accompanied by a spreadsheet file in .xls format that contains the individual data used to conduct each of the experimental analyses. Experiments 1, 2, and 3 appear on their own sheets. For all Sex and Handedness columns, female and right-handed are coded as ‘0’, while male and left-handed are coded as ‘1’. Age is in years. All other numbers represent amplitude in microvolts (µV). Supplementary data associated with this article can be found, in the online version, at https://doi.org/10.1016/j.cognition.2018.06. 005. References Baess, P., Jacobsen, T., & Schröger, E. (2008). Suppression of the auditory N1 eventrelated potential component with unpredictable self-initiated tones: Evidence for internal forward models with dynamic stimulation. International Journal of Psychophysiology, 70(2), 137–143. http://dx.doi.org/10.1016/j.ijpsycho.2008.06. 005. Bays, P. M., & Wolpert, D. M. (2007). Computational principles of sensorimotor control that minimize uncertainty and variability. Journal of Physiology, 578, 387–396. http://dx.doi.org/10.1113/jphysiol.2006.120121. Bendixen, A., SanMiguel, I., & Schröger, E. (2012). Early electrophysiological indicators for predictive processing in audition: A review. International Journal of Psychophysiology, 83(2), 120–131. http://dx.doi.org/10.1016/j.ijpsycho.2011.08. 003. Blakemore, S. J., Frith, C. D., & Wolpert, D. M. (1999). Spatio-temporal prediction modulates the perception of self-produced stimuli. Journal of Cognitive Neuroscience, 11(5), 551–559. http://dx.doi.org/10.1162/089892999563607. Brainard, D. H. (1997). The psychophysics toolbox. Spatial Vision, 10(4), 433–436. http:// dx.doi.org/10.1163/156856897X00357. Cardoso-Leite, P., Mamassian, P., Schütz-Bosbach, S., & Waszak, F. (2010). A new look at sensory attenuation: Action-effect anticipation affects sensitivity, not response bias. Psychological Science, 21(12), 1740–1745. http://dx.doi.org/10.1177/ 0956797610389187. Clark, V. P., Fan, S., & Hillyard, S. A. (1995). Identification of early visual evoked potential generators by retinotopic and topographic analyses. Human Brain Mapping, 2, 170–187. http://dx.doi.org/10.1002/hbm.460020306. Curio, G., Neuloh, G., Numminen, J., Jousmaki, V., & Hari, R. (2000). Speaking modifies voice-evoked activity in the human auditory cortex. Human Brain Mapping, 9(4), 183–191. http://dx.doi.org/10.1002/(SICI)1097-0193(200004)9:4<183::AIDHBM1>3.0.CO;2-Z. Diamond, M. R., Ross, J., & Morrone, M. C. (2000). Extraretinal control of saccadic suppression. Journal of Neuroscience, 20(9), 3449–3455. Engbert, K., Wohlschlager, A., & Haggard, P. (2008). Who is causing what? The sense of agency is relational and efferent-triggered. Cognition, 107(2), 693–704. http://dx.doi. org/10.1016/j.cognition.2007.07.021. Feinberg, I. (1978). Efference copy and corollary discharge: Implications for thinking and its disorders. Schizophrenia Bulletin, 4, 636–640. http://dx.doi.org/10.1093/schbul/4. 4.636. Ford, J. M., Gray, M., Faustman, W. O., Roach, B. J., & Mathalon, D. H. (2007). Dissecting corollary discharge dysfunction in schizophrenia. Psychophysiology, 44(4), 522–529. http://dx.doi.org/10.1111/j.1469-8986.2007.00533.x. Ford, J. M., Mathalon, D. H., Heinks, T., Kalba, S., Faustman, W. O., & Roth, W. T. (2001). Neurophysiological evidence of corollary discharge dysfunction in schizophrenia. American Journal of Psychiatry, 158(12), 2069–2071. http://dx.doi.org/10.1176/ appi.ajp.158.12.2069. Gentsch, A., & Schütz-Bosbach, S. (2011). I did it: Unconscious expectation of sensory consequences modulates the experience of self-agency and its functional signature. Journal of Cognitive Neuroscience, 23(12), 3817–3828. http://dx.doi.org/10.1162/ jocn_a_00012. Gruters, K. G., Murphy, D. L. K., Jenson, C. D., Smith, D. W., Shera, C. A., & Groh, J. M. (2018). The eardrums move when the eyes move: A multisensory effect on the mechanics of hearing. Proceedings of the National Academy of Sciences. http://dx.doi.org/ 10.1073/pnas.1717948115. Heinks-Maldonado, T. H., Mathalon, D. H., Gray, M., & Ford, J. M. (2005). Fine-tuning of auditory cortex during speech production. Psychophysiology, 42(2), 180–190. http:// dx.doi.org/10.1111/j.1469-8986.2005.00272.x. Houde, J. F., Nagarajan, S. S., Sekihara, K., & Merzenich, M. M. (2002). Modulation of the auditory cortex during speech: An MEG study. Journal of Cognitive Neuroscience, 14(8), 1125–1138. http://dx.doi.org/10.1162/089892902760807140. Hughes, G., Desantis, A., & Waszak, F. (2013). Mechanisms of intentional binding and sensory attenuation: The role of temporal prediction, temporal control, identity prediction, and motor prediction. Psychological Bulletin, 139(1), 133–151. http://dx. doi.org/10.1037/a0028566. Hughes, G., & Waszak, F. (2011). ERP correlates of action effect prediction and visual sensory attenuation in voluntary action. Neuroimage, 56(3), 1632–1640. http://dx. doi.org/10.1016/j.neuroimage.2011.02.057. Hughes, G., & Waszak, F. (2014). Predicting faces and houses: Category-specific visual action-effect prediction modulates late stages of sensory processing. Neuropsychologia, 61, 11–18. http://dx.doi.org/10.1016/j.neuropsychologia.2014.06.002.
Acknowledgements This work was supported by a Career Development Fellowship from the National Health and Medical Research Council of Australia (APP1090507) and Discovery Projects from the Australian Research Council (DP140104394; DP170103094) to TJW, and an Australian Postgraduate Award (APA) to NGM. The authors thank Miranda Chilver for her assistance with data collection in Experiment 1. 21
Cognition 179 (2018) 14–22
N.G. Mifsud et al.
1080/13546805.2016.1208611. Reingold, E. M., & Stampe, D. M. (2002). Saccadic inhibition in voluntary and reflexive saccades. Journal of Cognitive Neuroscience, 14(3), 371–388. http://dx.doi.org/10. 1162/089892902317361903. Sato, A. (2008). Action observation modulates auditory perception of the consequence of others’ actions. Consciousness and Cognition, 17(4), 1219–1227. http://dx.doi.org/10. 1016/j.concog.2008.01.003. Schafer, E. W., Amochaev, A., & Russell, M. J. (1981). Knowledge of stimulus timing attenuates human evoked cortical potentials. Electroencephalography and Clinical Neurophysiology, 52(1), 9–17. http://dx.doi.org/10.1016/0013-4694(81)90183-8. Schafer, E. W., & Marcus, M. M. (1973). Self-stimulation alters human sensory brain responses. Science, 181(4095), 175–177. http://dx.doi.org/10.1126/science.181.4095. 175. Schall, J. D. (2002). The neural selection and control of saccades by the frontal eye field. Philosophical Transactions of the Royal Society B: Biological Sciences, 357(1424), 1073–1082. http://dx.doi.org/10.1098/rstb.2002.1098. Sowman, P. F., Kuusik, A., & Johnson, B. W. (2012). Self-initiation and temporal cueing of monaural tones reduce the auditory N1 and P2. Experimental Brain Research, 222, 149–157. http://dx.doi.org/10.1007/s00221-012-3204-7. Stenner, M. P., Bauer, M., Haggard, P., Heinze, H. J., & Dolan, R. (2014). Enhanced alphaoscillations in visual cortex during anticipation of self-generated visual stimulation. Journal of Cognitive Neuroscience, 26(11), 2540–2551. http://dx.doi.org/10.1162/ jocn_a_00658. Subramaniam, K., Kothare, H., Mizuiri, D., Nagarajan, S. S., & Houde, J. F. (2018). Reality monitoring and feedback control of speech production are related through selfagency. Frontiers in Human Neuroscience, 12(82), http://dx.doi.org/10.3389/fnhum. 2018.00082. van Elk, M., Salomon, R., Kannape, O., & Blanke, O. (2014). Suppression of the N1 auditory evoked potential for sounds generated by the upper and lower limbs. Biological Psychology, 102, 108–117. http://dx.doi.org/10.1016/j.biopsycho.2014.06.007. Vogel, E. K., & Luck, S. J. (2000). The visual N1 component as an index of a discrimination process. Psychophysiology, 37(2), 190–203. http://dx.doi.org/10.1111/ 1469-8986.3720190. Weiss, C., Herwig, A., & Schütz-Bosbach, S. (2011). The self in action effects: Selective attenuation of self-generated sounds. Cognition, 121(2), 207–218. http://dx.doi.org/ 10.1016/j.cognition.2011.06.011. Whitford, T. J., Mathalon, D. H., Shenton, M. E., Roach, B. J., Bammer, R., Adcock, R. A., ... Ford, J. M. (2011). Electrophysiological and diffusion tensor imaging evidence of delayed corollary discharges in patients with schizophrenia. Psychological Medicine, 41(5), 959–969. http://dx.doi.org/10.1017/S0033291710001376. Wolpert, D. M., Ghahramani, Z., & Jordan, M. I. (1995). An internal model for sensorimotor integration. Science, 269, 1880–1882. http://dx.doi.org/10.1126/science. 7569931. Yang, S. N., & McConkie, G. W. (2001). Eye movements during reading: A theory of saccade initiation times. Vision Research, 41, 3567–3585. http://dx.doi.org/10.1016/ S0042-6989(01)00025-6.
Ibbotson, M., & Krekelberg, B. (2011). Visual perception and saccadic eye movements. Current Opinion in Neurobiology, 21(4), 553–558. http://dx.doi.org/10.1016/j.conb. 2011.05.012. Kleiner, M., Brainard, D. H., & Pelli, D. G. (2007). What’s new in Psychtoolbox-3? Perception, 36, ECVP Abstract Supplement. Lange, K. (2009). Brain correlates of early auditory processing are attenuated by expectations for time and pitch. Brain and Cognition, 69(1), 127–137. http://dx.doi.org/ 10.1016/j.bandc.2008.06.004. Lange, K. (2013). The ups and downs of temporal orienting: A review of auditory temporal orienting studies and a model associating the heterogeneous findings on the auditory N1 with opposite effects of attention and prediction. Frontiers in Human Neuroscience, 7, 263. http://dx.doi.org/10.3389/fnhum.2013.00263. Luck, S. J., & Kappenman, E. S. (2012). The Oxford handbook of event-related potential components. USA: Oxford University Press. Martikainen, M. H., Kaneko, K., & Hari, R. (2005). Suppressed responses to self-triggered sounds in the human auditory cortex. Cerebral Cortex, 15(3), 299–302. http://dx.doi. org/10.1093/cercor/bhh131. Meier, U. (2006). A note on the power of Fisher's least significant difference procedure. Pharmaceutical Statistics, 5(4), 253–263. http://dx.doi.org/10.1002/pst.210. Mifsud, N. G., Beesley, T., Watson, T. L., & Whitford, T. J. (2016). Attenuation of auditory evoked potentials for hand and eye-initiated sounds. Biological Psychology, 120, 61–68. http://dx.doi.org/10.1016/j.biopsycho.2016.08.011. Mifsud, N. G., Oestreich, L. K., Jack, B. N., Ford, J. M., Roach, B. J., Mathalon, D. H., & Whitford, T. J. (2016). Self-initiated actions result in suppressed auditory but amplified visual evoked components in healthy participants. Psychophysiology, 53(5), 723–732. http://dx.doi.org/10.1111/psyp.12605. Mifsud, N. G., & Whitford, T. J. (2017). Sensory attenuation of self-initiated sounds maps onto habitual associations between motor action and sound. Neuropsychologia, 103, 38–43. http://dx.doi.org/10.1016/j.neuropsychologia.2017.07.019. Miniussi, C., Wilding, E. L., Coull, J., & Nobre, A. C. (1999). Orienting attention in time: Modulation of brain potentials. Brain, 122(8), 1507–1518. Näätänen, R., & Picton, T. (1987). The N1 wave of the human electric and magnetic response to sound: A review and an analysis of the component structure. Psychophysiology, 24(4), 375–425. http://dx.doi.org/10.1111/j.1469-8986.1987. tb00311.x. Oestreich, L. K., Mifsud, N. G., Ford, J. M., Roach, B. J., Mathalon, D. H., & Whitford, T. J. (2015). Subnormal sensory attenuation to self-generated speech in schizotypy: Electrophysiological evidence for a “continuum of psychosis”. International Journal of Psychophysiology, 97(2), 131–138. http://dx.doi.org/10.1016/j.ijpsycho.2015.05. 014. Pelli, D. G. (1997). The VideoToolbox software for visual psychophysics: Transforming numbers into movies. Spatial Vision, 10, 437–442. http://dx.doi.org/10.1163/ 156856897X00366. Pinheiro, A. P., Rezaii, N., Rauber, A., & Niznikiewicz, M. (2016). Is this my voice or yours? The role of emotion and acoustic quality in self-other voice discrimination in schizophrenia. Cognitive Neuropsychiatry, 21(4), 335–353. http://dx.doi.org/10.
22
Contents lists available at ScienceDirect
Cognition journal homepage: www.elsevier.com/locate/cognit
Original Articles
Attenuation of visual evoked responses to hand and saccade-initiated flashes a,⁎
a
b
a
T
a
Nathan G. Mifsud , Tom Beesley , Tamara L. Watson , Ruth B. Elijah , Tegan S. Sharp , Thomas J. Whitforda a b
School of Psychology, UNSW Sydney, Sydney, New South Wales, Australia School of Social Sciences and Psychology, Western Sydney University, Bankstown, New South Wales, Australia
A R T I C LE I N FO
A B S T R A C T
Keywords: Event-related potentials Predictive processing Sensory attenuation Saccadic movements Efference copy Corollary discharge
Sensory attenuation refers to reduced brain responses to self-initiated sensations relative to those produced by the external world. It is a low-level process that may be linked to higher-level cognitive tasks such as reality monitoring. The phenomenon is often explained by prediction error mechanisms of universal applicability to sensory modality; however, it is most widely reported for auditory stimuli resulting from self-initiated hand movements. The present series of event-related potential (ERP) experiments explored the generalizability of sensory attenuation to the visual domain by exposing participants to flashes initiated by either their own button press or volitional saccade and comparing these conditions to identical, computer-initiated stimuli. The key results showed that the largest reduction of anterior visual N1 amplitude occurred for saccade-initiated flashes, while button press-initiated flashes evoked an intermediary response between the saccade-initiated and externally initiated conditions. This indicates that sensory attenuation occurs for visual stimuli and suggests that the degree of electrophysiological attenuation may relate to the causal likelihood of pairings between the type of motor action and the modality of its sensory response.
1. Introduction Sensory attenuation refers to self-initiated stimuli evoking reduced neurophysiological (e.g., Baess, Jacobsen, & Schröger, 2008; Houde, Nagarajan, Sekihara, & Merzenich, 2002; Schafer & Marcus, 1973) and phenomenological (e.g., Blakemore, Frith, & Wolpert, 1999; CardosoLeite, Mamassian, Schütz-Bosbach, & Waszak, 2010; Sato, 2008) sensory representations, compared to the sensory representations evoked by physically identical, externally initiated stimuli. The attenuation taking place here is thus related to a new external stimulus that is a consequence of enacting a motor action; a phenomenon considered here as separate to the suppression of the sensory consequences of enacting a movement within a constant sensory environment (e.g., motion across the retina produced by an eye movement). The phenomenon is typically explained using a forward model that predicts the sensory consequences of intended actions based on internal motor commands, where these predictions are subtracted from actual sensory input (Bays & Wolpert, 2007; Wolpert, Ghahramani, & Jordan, 1995). Conversely, externally initiated stimuli lack accompanying motor information, and are thus marked by a larger disparity between predicted and actual sensory inputs—a distinction that may play a central role in cognition; specifically, our sense of agency (Engbert, Wohlschlager, & Haggard, 2008; Subramaniam, Kothare, Mizuiri, ⁎
Nagarajan, & Houde, 2018). Notably, Feinberg (1978) first suggested that disruption of this distinction between self and the external world could account for some of the characteristic symptoms of schizophrenia (e.g., delusions of control), and evidence has emerged to support this theory (Ford et al., 2001; Pinheiro, Rezaii, Rauber, & Niznikiewicz, 2016; Whitford et al., 2011). Studies of sensory attenuation have thus far largely been limited to the auditory domain (e.g., see Table 3 in Hughes, Desantis, & Waszak, 2013), centerd on a reliable event-related potential (ERP) component that is used by multiple research groups as an index of sensory attenuation—that is, the N1 or N1m component, an evoked potential or magnetic field that is consistently reduced for self-initiated vocalizations and tones (e.g., Baess et al., 2008; Curio, Neuloh, Numminen, Jousmaki, & Hari, 2000; Houde et al., 2002; Mifsud & Whitford, 2017; Sowman, Kuusik, & Johnson, 2012). Given the well-established positive relationship between the auditory N1 component and stimulus intensity (Näätänen & Picton, 1987)—i.e., loud sounds evoke larger auditory N1 amplitudes than do soft sounds—the finding that self-initiated sounds have a reduced auditory N1 response suggests that the brain processes them as being “softer”. This reduced perceived loudness of self-initiated sounds may reflect an ecological adaptation, in the sense that the strong auditory feedback associated with our own speech may require attenuation to preserve the sensitivity of receptors to incoming sounds
Corresponding author. School of Psychology, Mathews Building, UNSW Sydney, Sydney, NSW 2052, Australia. E-mail address: [email protected] (N.G. Mifsud).
https://doi.org/10.1016/j.cognition.2018.06.005 Received 18 June 2017; Received in revised form 4 June 2018; Accepted 5 June 2018 0010-0277/ Crown Copyright © 2018 Published by Elsevier B.V. All rights reserved.
Cognition 179 (2018) 14–22
N.G. Mifsud et al.
sensitive to loudness. The visual N1 component is a likely candidate, as an early, sensory-evoked component that appears to be partially generated from occipital cortex (Clark, Fan, & Hillyard, 1995), but (to our knowledge) no previous studies explicitly demonstrate that the visual N1 (i.e., frontocentral maxima, mastoid-referenced) is sensitive to changes in luminance. Hence, the primary aim of Experiment 1—which only measured ERP responses to passively viewed stimuli—was to demonstrate the luminance-dependence of the visual N1.
(Bendixen, SanMiguel, & Schröger, 2012). In contrast to the auditory domain, studies of sensory attenuation in the visual domain are relatively scarce, and results are less easily reconciled. Reported differences in visual-evoked potentials (VEPs) between self- and externally initiated visual stimuli are inconsistent in terms of both their direction and spatial location. This may be due to a diverse range of stimuli, and, relatedly, the chosen event-related components and reference electrode sites. Self-initiation has been shown to result in anterior (but not occipital) reduction of N1 for flashes (Schafer & Marcus, 1973, mastoid-referenced data) and arrow shapes (Gentsch & Schütz-Bosbach, 2011, average-referenced data), and occipital reduction of P2 for faces and houses (Hughes & Waszak, 2014, FCz-referenced data). Conversely, occipital amplification of P1 has been shown for pattern-onset stimuli (Hughes & Waszak, 2011, vertex-referenced data) and occipital amplification of N145 for pattern-reversal stimuli (Mifsud, Oestreich, et al., 2016, Fz-referenced data). However, sensory attenuation has also been observed in behavioural tasks using Gabor patches (Cardoso-Leite et al., 2010; Stenner, Bauer, Haggard, Heinze, & Dolan, 2014). The clear differences in the reported results means that further experimental work is required in the visual domain that builds on existing self-initiation paradigms. A further limiting factor of previous studies of sensory attenuation in the visual domain is that nearly all self-initiation conditions have involved button pressing to receive a visual stimulus. This highly specific experimental condition, while relevant, must be considered in conjunction with other action–sensation contingencies to account for the wide range of circumstances that may involve sensory attenuation. In other words, there is insufficient support for assuming that findings from button-press studies can be generalized to other action–sensation contingencies. In the auditory domain, the limits of this assumption have been tested by van Elk, Salomon, Kannape, and Blanke (2014) and Mifsud, Beesley, Watson, and Whitford (2016), who employed paradigms using foot and saccade initiation respectively to demonstrate that differences in auditory-evoked potentials were dependent on the region of motor output used to produce the incoming stimulus. In the study by Mifsud, Beesley, et al. (2016), for example, a greater degree of auditory N1 attenuation was observed for button press-initiated tones than for saccade-initiated tones, consistent with the fact that while hand movements are strongly associated with auditory sensations (e.g., the sound of one’s fingers typing on a keyboard), eye movements are rarely, if ever, accompanied by auditory feedback. The present series of experiments explored whether a similar pattern of ERP effects would be observed for self-initiated visual stimuli. In Experiment 1, we tested the luminance-dependence of the frontocentral visual N1 component, and thus its conceptual similarity to the loudnessdependent auditory N1 measure used in previous sensory attenuation studies. In Experiment 2, we tested the influence of self-initiation of visual stimuli on visual N1 amplitude, extending the saccade initiation paradigm (Mifsud, Beesley, et al., 2016) to the visual domain. Lastly, in Experiment 3, we replicated the self-initiation manipulation of Experiment 2 with an added condition designed to probe the effect of temporal predictability.
2.1. Method 2.1.1. Participants Eleven participants were recruited at UNSW Sydney. Six were female, 8 were right-handed, and mean age was 19 years (SD = 1). Participants gave written, informed consent, and received course credit in exchange for their time. This experiment, and the two that follow, were approved by the UNSW Human Research Ethics Advisory Panel (Psychology). 2.1.2. Procedure Following provision of their demographical information, participants were fitted with an EEG cap and electrodes. EEG was continuously recorded while participants completed the experiment, seated 60 cm from a computer monitor with an integrated eye tracking system (Tobii TX300: 300 Hz gaze sampling rate; 23″, 60 Hz, 1920 × 1080 resolution TFT screen; accuracy of 0.4° visual angle; system latency under 10 ms). The eye tracking function was not used in Experiment 1, but was required for saccade detection in Experiment 2. The experiment comprised of a series of stimulus presentations of four different types: an unstructured full-field white flash of 33.33 ms duration (i.e., two frames, verified with a photometer), that was one of two mean luminance levels, dim (10 cd/m2) or bright (100 cd/m2); and two types of pure tones (the data for which are not presented). The mean luminance levels were approximated based on measurements with a handheld instrument (Minolta Chroma Meter CS-100A). Participant input was not required at any time. Each trial type was shown for 120 trials in total, intermixed in a total 480-trial sequence whose order was randomized between participants and split into 10 equal blocks separated by 30-s rest periods. Individual trials were separated by a uniformly distributed random interval (1–4 s). The EEG recording lasted approximately 30 min. Stimulus presentation was controlled by MATLAB (MathWorks, Natick, US) using the Psychophysics Toolbox extensions (Brainard, 1997; Kleiner, Brainard, & Pelli, 2007; Pelli, 1997). EEG was recorded with a BioSemi ActiveTwo system using 64 AgAgCl active electrodes placed according to the extended 10–20 system. Analog signals were anti-aliased with a fixed first-order filter (−3 dB at 3600 Hz) and continuously digitized at a sampling frequency of 2048 Hz, with common mode sense (CMS) and driven right leg (DRL) used as reference and ground electrodes. During offline preprocessing, data were re-referenced to the averaged mastoid electrodes as is typical for the visual N1 (Clark et al., 1995; Vogel & Luck, 2000), band-pass filtered from 0.01 to 30 Hz (8th order zero-phase Butterworth IIR), and separated into 600-ms epochs (100 ms pre-onset and 500 ms postonset). Data were baseline corrected with the average voltage between −100 and 0 ms. To address eye blinks and movement artefacts, we rejected individual epochs at any electrode site that contained EEG activity exceeding ± 75 µV or min-max changes in excess of 75 µV between adjacent 100-ms intervals. Individual trials were then averaged for each condition to produce ERPs for each participant. Data preprocessing was done in BrainVision Analyzer 2 (Brain Products GmbH, Munich, Germany), and statistical analyses were performed in SPSS version 23 (IBM Corp, Armonk, US). As the latency and amplitude of flash ERPs are sensitive to stimulus parameters, a collapsed localizer approach was used to guide our analysis (Luck & Kappenman, 2012). The waveforms of the dim and bright
2. Experiment 1 An underlying premise of neurophysiological sensory attenuation is that reductions in the ERP reflected alterations in the perceived intensity of a stimulus. For example, decreased auditory N1 amplitude has been consistently observed for self-initiated auditory tones (e.g., Sowman et al., 2012) and auditory N1 amplitude is known to decrease with decreasing stimulus intensity (Näätänen & Picton, 1987). As mentioned earlier, this suggests that self-initiated sounds are processed as “softer” (Weiss, Herwig, & Schütz-Bosbach, 2011). If we are to measure sensory attenuation in the visual domain, we must use a VEP component that reflects the intensity of a visual stimulus, in the same manner that the N1 component of the auditory-evoked potential is 15
Cognition 179 (2018) 14–22
N.G. Mifsud et al.
Fig. 1. Grand-averaged ERPs for the bright and dim flash conditions at electrodes Fz, FC1, FCz, FC2, and Cz. The x-axes represent time in milliseconds (ms) where 0 is flash onset, and the y-axes represent amplitude in microvolts (µV). Grey areas indicate measurement windows.
luminance. Specifically, a passively viewed bright flash produced larger mean visual N1 amplitude than a passively viewed dim flash at FCz and its neighbours. This demonstration of luminance-dependence suggests that the amplitude of the flash-evoked visual N1 could, in subsequent studies that incorporate self-initiation conditions, serve as a proxy measurement for sensory attenuation in the visual domain (an analogue to the well-established use of the auditory N1 in sensory attenuation studies in the auditory domain). Accordingly, in Experiment 2, we again measured the flash-evoked visual N1 to test the hypothesis that sensory attenuation may be influenced by the causal likelihood of pairings between action and visual sensation. Crucially, in Experiment 2, stimulus intensity was held constant (i.e., all flashes had the same luminance as the bright flash in Experiment 1) and the experimental manipulation was the type of initiating motor action.
conditions were averaged, and this collapsed waveform was used to identify a measurement window centerd on the peak and electrode site at which the visual N1 was maximal. A two-way analysis of variance (ANOVA) was then conducted to examine the effects of flash intensity on the mean amplitude of the visual N1 (within the time window of 149–159 ms). The Condition factor consisted of two levels (dim, bright) and the Site factor consisted of five levels: of primary interest, the maximal N1 site (FCz), and its adjacent electrodes (i.e., Fz, FC1, FC2, and Cz). 2.2. Results and discussion Fig. 1 presents the grand-averaged ERPs for the 5-site cluster (Fz, FC1, FCz, FC2, and Cz). Fig. 2 presents the scalp distributions for each stimulus condition. A main effect of Condition, F(1,10) = 30.03, p < .001, ηp2 = .75, indicated that mean visual N1 amplitude across electrodes Fz, FC1, FCz, FC2, and Cz differed between the dim (M = −2.60, SD = 0.51) and bright (M = −5.27, SD = 0.51) conditions. There was neither a main effect of Site, F(4,40) = 0.18, p = .728, ηp2 = .02, nor an interaction between Condition and Site, F(4,40) = 1.24, p = .312, ηp2 = .11. A follow-up pairwise comparison confirmed that mean visual N1 amplitude between the dim (M = −2.70, SD = 1.79) and bright (M = −5.36, SD = 1.75) conditions at the maximal site (FCz) differed significantly, F(1,10) = 13.54, p < 001. This indicated that the mastoid-referenced visual N1 at frontocentral sites was sensitive to
3. Experiment 2 We aimed to determine whether VEP attenuation would occur for button-press and saccade-initiated flashes compared to externally initiated flashes. We expected that button press-initiated flashes would evoke attenuated visual N1 based on a similar condition reported by Schafer and Marcus (1973). Notably, however, we hypothesized that there would be increased visual N1 attenuation for saccade-initiated flashes compared to button press-initiated flashes, given that eye movements, rather than hand movements, are more tightly coupled to 16
Cognition 179 (2018) 14–22
N.G. Mifsud et al.
Fig. 2. Topographic maps of the visual N1 (149–159 ms) for each stimulus condition.
visual sensations. 3.1. Method 3.1.1. Participants Forty participants were recruited at UNSW Sydney. Seven participants were excluded due to their dataset lacking discernible flash ERP waveforms in one or more stimulus blocks. Of the remaining 33 participants, 15 were female, 29 were right-handed, and mean age was 22 years (SD = 6). Participants gave written, informed consent, and received either course credit (n = 21) or financial reimbursement (n = 12, A$30) in exchange for their time. 3.1.2. Procedure The prerecording routine, spatial arrangements, and hardware specifications were identical to Experiment 1, with an additional 5point procedure to calibrate the eye tracking system for saccade detection. Experiment 2 comprised five conditions: two types of self-initiation conditions (i.e., button-press and saccade-initiated flashes) and their corresponding motor control conditions (i.e., button presses and saccades without consequent flashes), and an externally initiated condition (i.e., flashes initiated without participant input). Each condition was presented in a homogenous 80-trial block, and block order was randomized between participants. Three practice trials preceded each block to ensure that participants understood the instructions displayed on screen, and, where appropriate, allowed the experimenter to verbally encourage self-paced rather than speeded responses. Individual trials in all conditions were separated by a blank, black screen for a uniformly distributed random interval (2–4 s), which began following the end of the flash (or, in the motor conditions, after the disappearance of the fixation dot). The EEG recording lasted approximately 50 min. Fig. 3 depicts the experimental protocol.
Fig. 3. Experimental protocol. In the press-initiated condition, participants depressed the space bar at will any time after a fixation dot appeared, which immediately delivered a flash (two frames of white fill). In the saccade-initiated condition, participants focused on a distal dot and then shifted at will to the central fixation dot, which immediately delivered a flash. In the externally initiated condition, flash delivery followed a variable delay without motor input. Control conditions were identical to their respective stimulus conditions, except that motor input did not result in a stimulus.
recordings. If detection took longer than 5 s, trials were skipped with replacement (M = 2.7 skipped trials per participant across both stimulus and motor saccade-initiated blocks). Following fixation on the white circle, participants shifted their gaze at will to the red circle, which immediately delivered a full-field flash identical to the pressinitiated stimulus. More precisely, flashes followed detection of the gaze within the 200-px (5°) square area of interest surrounding the central red circle. We confirmed that system latency was identical in both self-initiation conditions using a photometer to detect actual delivery of the flash. Specifically, mean latency between the end of the action (button press or the eye attaining fixation in the center) and stimulus delivery was 37 ms ± 7 ms (SD) trials in both conditions.
3.1.2.1. Press condition. In this condition, a visual stimulus was selfinitiated by a button press (i.e., hand motor output). Participants were instructed to respond at will any time after the appearance of a red fixation dot (0.7° diameter) presented in the center of a black screen, and did so by pressing the space bar on a low-latency keyboard (Ducky Shine 4: 1000 Hz report rate) with their dominant hand. Responses immediately replaced the fixation screen with a full-field flash identical to the bright flash used in Experiment 1 (i.e., 100 cd/m2 mean luminance, two-frame duration).
3.1.2.3. Motor conditions. The motor control conditions were identical to their respective self-initiated conditions, except that pressing the space bar or shifting gaze between circles did not result in the delivery of a stimulus. The ensuing EEG activity was subsequently subtracted from the appropriate self-initiated conditions to remove EEG activity associated with button pressing (for the press condition) or a singular, volitional eye movement (for the saccade condition), as is standard practice in button-press studies of this nature (Baess et al., 2008; Martikainen, Kaneko, & Hari, 2005; Whitford et al., 2011), and as was previously used for the saccade initiation condition in the study of
3.1.2.2. Saccade condition. In this condition, the visual stimulus was self-initiated by a volitional saccade (i.e., eye motor output). Each trial began with two dots appearing on screen: a solid red circle in the center of screen (identical to the fixation in the press-initiated condition) and a distal (17° left) hollow white circle. Participants were instructed to initially fixate on the white circle, which would turn solid once the script detected their gaze, based on a 20 ms sample of location 17
Cognition 179 (2018) 14–22
N.G. Mifsud et al.
used Fisher’s protected least significant difference (LSD) procedure to examine pairwise comparisons, which controls the family-wise type I error rate when there are exactly 3 groups and a significant omnibus test (Meier, 2006). The follow-up pairwise comparisons indicated that external differed significantly from press, F(1,32) = 4.89, p = .034, as well as saccade, F(1,32) = 16.54, p < .001. Likewise, press and saccade differed significantly from each other, F(1,32) = 4.92, p = .034. Crucially, this demonstrated that saccade-initiated flashes evoked a significant reduction in mean visual N1 amplitude at FCz compared to both button press- and externally initiated flashes. Similarly, for the 5-site cluster, a main effect of Condition, F(2,64) = 10.41, p < .001, ηp2 = .25, indicated that mean visual N1 amplitude across electrodes Fz, FC1, FCz, FC2, and Cz differed between the press (M = −4.04, SD = 0.67), saccade (M = −1.54, SD = 0.83), and external (M = −5.37, SD = 0.59) conditions. Follow-up pairwise comparisons indicated that external differed significantly from saccade, F(1,32) = 17.31, p < .001, but not press, F(1,32) = 3.52, p = .070. However, press and saccade differed significantly from each other, F (1,32) = 7.52, p = .010. There was neither a main effect of Site, F (4,128) = 1.51, p = .223, ηp2 = .05, nor an interaction between Condition and Site, F(8,256) = 1.68, p = .104, ηp2 = .05. Like the FCz analysis, the 5-site cluster analysis therefore demonstrated that saccade initiation led to significantly reduced mean visual N1 amplitude compared to external initiation. As predicted, button-press initiation led to a smaller, but still significant, reduction in mean visual N1 amplitude, representing an intermediary point between the saccade and external conditions.
Mifsud, Beesley, et al. (2016). 3.1.2.4. External condition. In this condition, stimuli were delivered automatically (i.e., without participant input) to assess electrophysiological response to externally initiated, temporally unpredictable stimuli. Trials began with a red fixation dot followed by a uniformly distributed random interval (0.5–2.5 s) before a flash was presented (identical to that in the self-initiated conditions). Participants were instructed to keep their eyes open and maintain their gaze on the screen at all times. 3.1.2.5. EEG data acquisition. EEG was recorded and preprocessed exactly as described in Experiment 1, including data re-referencing to the averaged mastoid electrodes, and a step function artefact rejection procedure that resulted in a mean rejection rate at electrode FCz of 4.5% ± 5.5% (SD) trials (press: 4.7% ± 6.0%, saccade: 3.9% ± 6.4%, external: 5.0% ± 5.5%), with no significant differences between stimulus blocks, F(2,64) = 0.99, p = .378, ηp2 = .03. Once individual trials for each condition were averaged to produce ERPs for each participant, motor waveforms were subtracted from the appropriate self-initiated waveforms to produce difference waveforms. Fig. 4 demonstrates the effect of the motor subtraction procedure by presenting grand-averaged ERPs at electrode FCz for the uncorrected waveforms in the self-initiated conditions and their corresponding motor waveforms. Hereafter, unless explicitly noted otherwise, mentions of self-initiated waveforms refer to motorcorrected waveforms. To investigate the effect of self-initiation on the visual ERP, separate ANOVAs were conducted on the Condition factor (press, saccade, external) on the mean amplitudes of the visual N1 (150–160 ms) at electrode FCz and for a 5-site cluster that included adjacent electrodes (i.e., Fz, FC1, FC2, and Cz). In cases where the assumption of sphericity was violated, a Greenhouse-Geisser correction was applied. This analysis mirrored Experiment 1, with a trivially different measurement window (i.e., 149–159 ms vs. 150–160 ms) due to the latency of the peak identified in the collapsed waveform formed from the average of the press, saccade, and external conditions.
4. Experiment 3 An issue in contingency paradigms exploring sensory attenuation is the inherent difference in the temporal predictability of stimuli across self- and externally initiated conditions. As participants choose when to initiate stimuli in self-initiated conditions, the stimulus onset can be accurately anticipated, unlike externally initiated conditions in which stimuli are initiated without warning (see Hughes et al., 2013). Various attempts have been made to control for this by “cueing” externally initiated stimuli (Ford, Gray, Faustman, Roach, & Mathalon, 2007; Lange, 2009; Mifsud, Oestreich, et al., 2016; Oestreich et al., 2015; Schafer, Amochaev, & Russell, 1981), and, though observations are varied (see Lange, 2013), temporal predictability appears to partly contribute to observed ERP effects in the auditory domain. Hence, because the present study is motivated by putative cross-modal effects, Experiment 3 introduced a cued condition to determine if temporal predictability influenced the attenuation of visual N1 amplitude observed to saccade-initiated flashes.
3.2. Results and discussion Fig. 5 presents the grand-averaged ERPs at electrode FCz and adjacent sites. Fig. 6 presents the scalp distributions for each stimulus condition. At electrode FCz, a main effect of Condition, F(2,64) = 9.19, p < .001, ηp2 = .22, indicated that mean visual N1 amplitude differed between the press (M = −3.88, SD = 4.34), saccade (M = −1.61, SD = 5.10), and external (M = −5.59, SD = 3.45) conditions. We then
Fig. 4. Grand-averaged ERPs at electrode FCz for uncorrected self-initiated conditions (solid traces, left), their corresponding motor conditions (dotted traces, left), and motor-corrected self-initiation conditions (right). The externally initiated condition (black line, both panels) is included for comparison. The x-axes represent time in milliseconds (ms) where 0 is flash onset, and the y-axes represent amplitude in microvolts (µV). 18
Cognition 179 (2018) 14–22
N.G. Mifsud et al.
Fig. 5. Grand-averaged ERPs for press, saccade, and external conditions at electrodes Fz, FC1, FCz, FC2, and Cz. Self-initiated conditions (i.e., press and saccade) are motor-corrected. The x-axes represent time in milliseconds (ms) where 0 is flash onset, and the y-axes represent amplitude in microvolts (µV). Grey areas indicate measurement windows.
4.1.2. Procedure Experiment 3 comprised six conditions: the same five conditions in Experiment 2 (button-press and saccade initiation, their motor-only controls, and an externally initiated, temporally unpredictable condition; refer to Fig. 3) plus a cued condition (externally initiated, temporally predictable stimuli). In this condition, flashes were delivered without participant input following a countdown composed of fixation targets (i.e., ‘• • •’, then ‘• •’, then ‘•’), each of which were separated by a 500 ms interval. All other characteristics of the experiment, including EEG data acquisition, were identical to Experiment 2. A one-way
4.1. Method 4.1.1. Participants Thirty-two participants were recruited at UNSW Sydney. Seven participants were excluded due to their dataset lacking discernible flash ERP waveforms in one or more stimulus blocks. Of the remaining 25 participants, 13 were female, 23 were right-handed, and mean age was 23 years (SD = 6). Participants gave written, informed consent, and received course credit in exchange for their time.
Fig. 6. Topographic maps of the visual N1 (150–160 ms) for each stimulus condition. Self-initiated conditions (i.e., press and saccade) are motor-corrected. 19
Cognition 179 (2018) 14–22
N.G. Mifsud et al.
Fig. 7. Grand-averaged ERPs for press, saccade, external, and cued conditions at electrode Fz. Self-initiated conditions (i.e., press and saccade) are motor-corrected. The x-axes represent time in milliseconds (ms) where 0 is flash onset, and the y-axes represent amplitude in microvolts (µV).
saccadic production and control around the immediate time of saccadic onset (e.g., Reingold & Stampe, 2002) and physiological effects during reading (Ibbotson & Krekelberg, 2011; Yang & McConkie, 2001), the present study focused on the sensory consequences cued by the eye movement, rather than the movement itself. The key novel finding, demonstrated in Experiment 2 and replicated in Experiment 3, was that saccade-initiated flashes evoked attenuated visual N1 amplitude compared to both button press- and externally initiated flashes. It is unlikely that the observed attenuation is attributable to saccadic suppression; given the size of the initiating eye movements, the flashes were presented outside the time course of such effects (see Diamond, Ross, & Morrone, 2000). Importantly, as Experiment 1 showed that visual N1 amplitude was luminance-dependent, and given the similarities between both the procedures and results presented here and those in the auditory domain (e.g., Baess et al., 2008; Sowman et al., 2012; Whitford et al., 2011), we suggest that sensory attenuation due to stimulus selfinitiation takes place in the visual domain as well, and that flash-evoked visual N1 amplitude may be a measure well-suited for further investigation. The attenuation of visual N1 amplitude that followed button-press initiation was less marked. At the primary site of analysis (FCz) in Experiment 2, significant reductions were observed for button-press compared to externally initiated flashes, but this effect disappeared when using an arguably better externally initiated condition (i.e., temporally cued stimuli) in Experiment 3. There are previous reports of attenuated anterior visual N1 amplitude for button press-initiated visual stimuli (Gentsch & Schütz-Bosbach, 2011; Schafer & Marcus, 1973). Indeed, Gentsch and Schütz-Bosbach (2011) observed that the quantitative effect of button-press initiation on visual N1 amplitude was less than effects reported in the literature measuring the auditory N1 (e.g., Heinks-Maldonado, Mathalon, Gray, & Ford, 2005). Hence, the relative weakness of button press-produced sensory attenuation in the
ANOVA was conducted on the Condition factor (press, saccade, external, cued) on the mean amplitudes of the visual N1 (125–135 ms) at electrode FCz, with peak latency determined by the collapsed waveform of all stimulus conditions. Given that Experiment 2 involved four groups, a Bonferroni adjustment for multiple comparisons was used for the follow-up pairwise comparisons, providing an alpha value of .008 (.05 divided by k = 6 comparisons). 4.2. Results and discussion Fig. 7 presents the grand-averaged ERPs at electrode FCz. Fig. 8 presents the scalp distributions for each stimulus condition. A main effect of Condition, F(2,72) = 11.94, p < .001, ηp2 = .33, indicated that there was a difference in mean visual N1 amplitude at electrode FCz among the press (M = −6.20, SD = 3.91), saccade (M = −3.35, SD = 3.73), external (M = −7.41, SD = 2.92), and cued (M = −6.65, SD = 3.03) conditions. Follow-up pairwise comparisons indicated that saccade differed significantly from press, F(1,24) = 8.62, p = .006, external, F(1,24) = 22.96, p < .001, and cued, F(1,24) = 13.96, p < .001. Press did not significantly differ from external, F(1,24) = 2.38, p = .062, or cued, F(1,24) = 0.31, p = .498, nor did external significantly differ from cued, F(1,24) = 1.00, p = .199. The results indicated that saccade-initiated flashes evoked a significant reduction in mean visual N1 amplitude at FCz compared to button press- and externally initiated conditions, regardless of temporal predictability. 5. General discussion The present study used a saccadic paradigm to investigate the effect of self-initiation on subsequent visual response for the first time, extending similar work conducted in the auditory domain (Mifsud, Beesley, et al., 2016). While a corpus of research has focused on
Fig. 8. Topographic maps of the visual N1 (125–135 ms) for each stimulus condition. Self-initiated conditions (i.e., press and saccade) are motor-corrected. 20
Cognition 179 (2018) 14–22
N.G. Mifsud et al.
Appendix A. Supplementary material
visual domain may help to explain the heterogeneity of our buttonpress initiation findings. Note also that temporal cuing of externally initiated stimuli in Experiment 3 did not evoke significantly reduced visual N1 amplitude compared to unpredictable, externally initiated stimuli. This may have been due to nature of the full-screen flash: comparable studies that have involved cueing of bright, foveal stimuli, such as crosses (Miniussi, Wilding, Coull, & Nobre, 1999) and patternreversals (Mifsud, Oestreich, et al., 2016), were associated with an absence of VEP modulation. Importantly, we can still compare relative degrees of visual N1 attenuation across different types of initiating motor output, as saccade initiation was significantly stronger than button-press initiation in both Experiments 2 and 3. Visual N1 attenuation associated with eye movements was greater than the N1 attenuation associated with finger movements (i.e., button-press initiation), even when controlling for between-condition differences in motor-evoked potentials. It is conceivable that the size of each effect relates to the strength of the habitual association between the type of motor action (eye or hand movement) and the resultant perceptual sensations (flashes). That is, the strength of pre-existing associations gained throughout a lifetime of experience appears to positively correlate with the degree of sensory attenuation; eye movements are strongly associated with changes in visual sensation, whereas hand movements are only weakly related to changes in visual sensation. This hypothesis accords with saccade-initiated tones producing less auditory N1 attenuation than button pressinitiated tones (Mifsud, Beesley, et al., 2016), perhaps because hand movements produce stronger auditory feedback than do saccades (which may indeed have some influence on hearing; Gruters et al., 2018). The overall correspondence between these studies is that electrophysiological sensory attenuation for an initiating motor action is stronger when there is a high causal likelihood that it is typically paired with the sensory modality under investigation (i.e., saccade–visual and press–auditory). This can be aligned with the popular forward model account of sensory attenuation, in which the predicted sensory consequences of an action in the form of physical, corollary signals modulate actual sensory feedback. Specifically, corollary signals seem most likely to exist where there are direct neural connections between the relevant areas of the brain. Such sensorimotor connections conceivably exist between the motor area of the brain involved in eye movements and the visual cortex (e.g., the frontal eye field in prefrontal cortex, see Schall, 2002), whereas it may be less likely that such established connections exist between the visual cortex and the parts of the motor cortex involved in hand movements. After demonstrating the luminance-dependence of the visual N1, we employed a new saccade initiation paradigm to investigate the sensory attenuation of visual stimuli that were initiated by either a saccade or button press, introducing a novel procedure to complement the large body of evidence showing sensory attenuation for auditory stimuli that are initiated by button press. We observed attenuation of visual N1 amplitude following saccade initiation, over and above that which occurred following button press-initiation, and suggested that this may relate to the strength of the association between eye movements and visual events. The experiments in the present study indicate that expanding the repertoire of action–sensation contingencies studied in the literature may shed light on the mechanisms underlying of the ubiquitous sensory attenuation phenomenon.
The electronic version of this article is accompanied by a spreadsheet file in .xls format that contains the individual data used to conduct each of the experimental analyses. Experiments 1, 2, and 3 appear on their own sheets. For all Sex and Handedness columns, female and right-handed are coded as ‘0’, while male and left-handed are coded as ‘1’. Age is in years. All other numbers represent amplitude in microvolts (µV). Supplementary data associated with this article can be found, in the online version, at https://doi.org/10.1016/j.cognition.2018.06. 005. References Baess, P., Jacobsen, T., & Schröger, E. (2008). Suppression of the auditory N1 eventrelated potential component with unpredictable self-initiated tones: Evidence for internal forward models with dynamic stimulation. International Journal of Psychophysiology, 70(2), 137–143. http://dx.doi.org/10.1016/j.ijpsycho.2008.06. 005. Bays, P. M., & Wolpert, D. M. (2007). Computational principles of sensorimotor control that minimize uncertainty and variability. Journal of Physiology, 578, 387–396. http://dx.doi.org/10.1113/jphysiol.2006.120121. Bendixen, A., SanMiguel, I., & Schröger, E. (2012). Early electrophysiological indicators for predictive processing in audition: A review. International Journal of Psychophysiology, 83(2), 120–131. http://dx.doi.org/10.1016/j.ijpsycho.2011.08. 003. Blakemore, S. J., Frith, C. D., & Wolpert, D. M. (1999). Spatio-temporal prediction modulates the perception of self-produced stimuli. Journal of Cognitive Neuroscience, 11(5), 551–559. http://dx.doi.org/10.1162/089892999563607. Brainard, D. H. (1997). The psychophysics toolbox. Spatial Vision, 10(4), 433–436. http:// dx.doi.org/10.1163/156856897X00357. Cardoso-Leite, P., Mamassian, P., Schütz-Bosbach, S., & Waszak, F. (2010). A new look at sensory attenuation: Action-effect anticipation affects sensitivity, not response bias. Psychological Science, 21(12), 1740–1745. http://dx.doi.org/10.1177/ 0956797610389187. Clark, V. P., Fan, S., & Hillyard, S. A. (1995). Identification of early visual evoked potential generators by retinotopic and topographic analyses. Human Brain Mapping, 2, 170–187. http://dx.doi.org/10.1002/hbm.460020306. Curio, G., Neuloh, G., Numminen, J., Jousmaki, V., & Hari, R. (2000). Speaking modifies voice-evoked activity in the human auditory cortex. Human Brain Mapping, 9(4), 183–191. http://dx.doi.org/10.1002/(SICI)1097-0193(200004)9:4<183::AIDHBM1>3.0.CO;2-Z. Diamond, M. R., Ross, J., & Morrone, M. C. (2000). Extraretinal control of saccadic suppression. Journal of Neuroscience, 20(9), 3449–3455. Engbert, K., Wohlschlager, A., & Haggard, P. (2008). Who is causing what? The sense of agency is relational and efferent-triggered. Cognition, 107(2), 693–704. http://dx.doi. org/10.1016/j.cognition.2007.07.021. Feinberg, I. (1978). Efference copy and corollary discharge: Implications for thinking and its disorders. Schizophrenia Bulletin, 4, 636–640. http://dx.doi.org/10.1093/schbul/4. 4.636. Ford, J. M., Gray, M., Faustman, W. O., Roach, B. J., & Mathalon, D. H. (2007). Dissecting corollary discharge dysfunction in schizophrenia. Psychophysiology, 44(4), 522–529. http://dx.doi.org/10.1111/j.1469-8986.2007.00533.x. Ford, J. M., Mathalon, D. H., Heinks, T., Kalba, S., Faustman, W. O., & Roth, W. T. (2001). Neurophysiological evidence of corollary discharge dysfunction in schizophrenia. American Journal of Psychiatry, 158(12), 2069–2071. http://dx.doi.org/10.1176/ appi.ajp.158.12.2069. Gentsch, A., & Schütz-Bosbach, S. (2011). I did it: Unconscious expectation of sensory consequences modulates the experience of self-agency and its functional signature. Journal of Cognitive Neuroscience, 23(12), 3817–3828. http://dx.doi.org/10.1162/ jocn_a_00012. Gruters, K. G., Murphy, D. L. K., Jenson, C. D., Smith, D. W., Shera, C. A., & Groh, J. M. (2018). The eardrums move when the eyes move: A multisensory effect on the mechanics of hearing. Proceedings of the National Academy of Sciences. http://dx.doi.org/ 10.1073/pnas.1717948115. Heinks-Maldonado, T. H., Mathalon, D. H., Gray, M., & Ford, J. M. (2005). Fine-tuning of auditory cortex during speech production. Psychophysiology, 42(2), 180–190. http:// dx.doi.org/10.1111/j.1469-8986.2005.00272.x. Houde, J. F., Nagarajan, S. S., Sekihara, K., & Merzenich, M. M. (2002). Modulation of the auditory cortex during speech: An MEG study. Journal of Cognitive Neuroscience, 14(8), 1125–1138. http://dx.doi.org/10.1162/089892902760807140. Hughes, G., Desantis, A., & Waszak, F. (2013). Mechanisms of intentional binding and sensory attenuation: The role of temporal prediction, temporal control, identity prediction, and motor prediction. Psychological Bulletin, 139(1), 133–151. http://dx. doi.org/10.1037/a0028566. Hughes, G., & Waszak, F. (2011). ERP correlates of action effect prediction and visual sensory attenuation in voluntary action. Neuroimage, 56(3), 1632–1640. http://dx. doi.org/10.1016/j.neuroimage.2011.02.057. Hughes, G., & Waszak, F. (2014). Predicting faces and houses: Category-specific visual action-effect prediction modulates late stages of sensory processing. Neuropsychologia, 61, 11–18. http://dx.doi.org/10.1016/j.neuropsychologia.2014.06.002.
Acknowledgements This work was supported by a Career Development Fellowship from the National Health and Medical Research Council of Australia (APP1090507) and Discovery Projects from the Australian Research Council (DP140104394; DP170103094) to TJW, and an Australian Postgraduate Award (APA) to NGM. The authors thank Miranda Chilver for her assistance with data collection in Experiment 1. 21
Cognition 179 (2018) 14–22
N.G. Mifsud et al.
1080/13546805.2016.1208611. Reingold, E. M., & Stampe, D. M. (2002). Saccadic inhibition in voluntary and reflexive saccades. Journal of Cognitive Neuroscience, 14(3), 371–388. http://dx.doi.org/10. 1162/089892902317361903. Sato, A. (2008). Action observation modulates auditory perception of the consequence of others’ actions. Consciousness and Cognition, 17(4), 1219–1227. http://dx.doi.org/10. 1016/j.concog.2008.01.003. Schafer, E. W., Amochaev, A., & Russell, M. J. (1981). Knowledge of stimulus timing attenuates human evoked cortical potentials. Electroencephalography and Clinical Neurophysiology, 52(1), 9–17. http://dx.doi.org/10.1016/0013-4694(81)90183-8. Schafer, E. W., & Marcus, M. M. (1973). Self-stimulation alters human sensory brain responses. Science, 181(4095), 175–177. http://dx.doi.org/10.1126/science.181.4095. 175. Schall, J. D. (2002). The neural selection and control of saccades by the frontal eye field. Philosophical Transactions of the Royal Society B: Biological Sciences, 357(1424), 1073–1082. http://dx.doi.org/10.1098/rstb.2002.1098. Sowman, P. F., Kuusik, A., & Johnson, B. W. (2012). Self-initiation and temporal cueing of monaural tones reduce the auditory N1 and P2. Experimental Brain Research, 222, 149–157. http://dx.doi.org/10.1007/s00221-012-3204-7. Stenner, M. P., Bauer, M., Haggard, P., Heinze, H. J., & Dolan, R. (2014). Enhanced alphaoscillations in visual cortex during anticipation of self-generated visual stimulation. Journal of Cognitive Neuroscience, 26(11), 2540–2551. http://dx.doi.org/10.1162/ jocn_a_00658. Subramaniam, K., Kothare, H., Mizuiri, D., Nagarajan, S. S., & Houde, J. F. (2018). Reality monitoring and feedback control of speech production are related through selfagency. Frontiers in Human Neuroscience, 12(82), http://dx.doi.org/10.3389/fnhum. 2018.00082. van Elk, M., Salomon, R., Kannape, O., & Blanke, O. (2014). Suppression of the N1 auditory evoked potential for sounds generated by the upper and lower limbs. Biological Psychology, 102, 108–117. http://dx.doi.org/10.1016/j.biopsycho.2014.06.007. Vogel, E. K., & Luck, S. J. (2000). The visual N1 component as an index of a discrimination process. Psychophysiology, 37(2), 190–203. http://dx.doi.org/10.1111/ 1469-8986.3720190. Weiss, C., Herwig, A., & Schütz-Bosbach, S. (2011). The self in action effects: Selective attenuation of self-generated sounds. Cognition, 121(2), 207–218. http://dx.doi.org/ 10.1016/j.cognition.2011.06.011. Whitford, T. J., Mathalon, D. H., Shenton, M. E., Roach, B. J., Bammer, R., Adcock, R. A., ... Ford, J. M. (2011). Electrophysiological and diffusion tensor imaging evidence of delayed corollary discharges in patients with schizophrenia. Psychological Medicine, 41(5), 959–969. http://dx.doi.org/10.1017/S0033291710001376. Wolpert, D. M., Ghahramani, Z., & Jordan, M. I. (1995). An internal model for sensorimotor integration. Science, 269, 1880–1882. http://dx.doi.org/10.1126/science. 7569931. Yang, S. N., & McConkie, G. W. (2001). Eye movements during reading: A theory of saccade initiation times. Vision Research, 41, 3567–3585. http://dx.doi.org/10.1016/ S0042-6989(01)00025-6.
Ibbotson, M., & Krekelberg, B. (2011). Visual perception and saccadic eye movements. Current Opinion in Neurobiology, 21(4), 553–558. http://dx.doi.org/10.1016/j.conb. 2011.05.012. Kleiner, M., Brainard, D. H., & Pelli, D. G. (2007). What’s new in Psychtoolbox-3? Perception, 36, ECVP Abstract Supplement. Lange, K. (2009). Brain correlates of early auditory processing are attenuated by expectations for time and pitch. Brain and Cognition, 69(1), 127–137. http://dx.doi.org/ 10.1016/j.bandc.2008.06.004. Lange, K. (2013). The ups and downs of temporal orienting: A review of auditory temporal orienting studies and a model associating the heterogeneous findings on the auditory N1 with opposite effects of attention and prediction. Frontiers in Human Neuroscience, 7, 263. http://dx.doi.org/10.3389/fnhum.2013.00263. Luck, S. J., & Kappenman, E. S. (2012). The Oxford handbook of event-related potential components. USA: Oxford University Press. Martikainen, M. H., Kaneko, K., & Hari, R. (2005). Suppressed responses to self-triggered sounds in the human auditory cortex. Cerebral Cortex, 15(3), 299–302. http://dx.doi. org/10.1093/cercor/bhh131. Meier, U. (2006). A note on the power of Fisher's least significant difference procedure. Pharmaceutical Statistics, 5(4), 253–263. http://dx.doi.org/10.1002/pst.210. Mifsud, N. G., Beesley, T., Watson, T. L., & Whitford, T. J. (2016). Attenuation of auditory evoked potentials for hand and eye-initiated sounds. Biological Psychology, 120, 61–68. http://dx.doi.org/10.1016/j.biopsycho.2016.08.011. Mifsud, N. G., Oestreich, L. K., Jack, B. N., Ford, J. M., Roach, B. J., Mathalon, D. H., & Whitford, T. J. (2016). Self-initiated actions result in suppressed auditory but amplified visual evoked components in healthy participants. Psychophysiology, 53(5), 723–732. http://dx.doi.org/10.1111/psyp.12605. Mifsud, N. G., & Whitford, T. J. (2017). Sensory attenuation of self-initiated sounds maps onto habitual associations between motor action and sound. Neuropsychologia, 103, 38–43. http://dx.doi.org/10.1016/j.neuropsychologia.2017.07.019. Miniussi, C., Wilding, E. L., Coull, J., & Nobre, A. C. (1999). Orienting attention in time: Modulation of brain potentials. Brain, 122(8), 1507–1518. Näätänen, R., & Picton, T. (1987). The N1 wave of the human electric and magnetic response to sound: A review and an analysis of the component structure. Psychophysiology, 24(4), 375–425. http://dx.doi.org/10.1111/j.1469-8986.1987. tb00311.x. Oestreich, L. K., Mifsud, N. G., Ford, J. M., Roach, B. J., Mathalon, D. H., & Whitford, T. J. (2015). Subnormal sensory attenuation to self-generated speech in schizotypy: Electrophysiological evidence for a “continuum of psychosis”. International Journal of Psychophysiology, 97(2), 131–138. http://dx.doi.org/10.1016/j.ijpsycho.2015.05. 014. Pelli, D. G. (1997). The VideoToolbox software for visual psychophysics: Transforming numbers into movies. Spatial Vision, 10, 437–442. http://dx.doi.org/10.1163/ 156856897X00366. Pinheiro, A. P., Rezaii, N., Rauber, A., & Niznikiewicz, M. (2016). Is this my voice or yours? The role of emotion and acoustic quality in self-other voice discrimination in schizophrenia. Cognitive Neuropsychiatry, 21(4), 335–353. http://dx.doi.org/10.
22