- Email: [email protected]
Effect anticipation modulates deviance processing in the brain
BR A IN RE S EA RCH 1 1 83 ( 20 0 7 ) 7 4 –82
a v a i l a b l e a t w w w. s c i e n c e d i r e c t . c o m
w w w. e l s e v i e r. c o m / l o c a t e / b r a i n r e s
Research Report
Effect anticipation modulates deviance processing in the brain Florian Waszak a,⁎, Arvid Herwig b a b
Laboratoire Psychologie de la Perception, CNRS and Université Paris Descartes, Paris, France Max Planck Institute for Human Cognitive and Brain Sciences, Department of Psychology, Leipzig, Germany
A R T I C LE I N FO
AB S T R A C T
Article history:
Humans constantly perform actions to achieve desired goals in the environment. However,
Accepted 31 August 2007
only very little is known about how actions influence stimulus processing. The present
Available online 16 September 2007
study addresses the question as to how performing an action that is associated with a particular auditory effect influences deviance processing in the brain. In the first part of the
Keywords:
experiment, subjects performed left and right keypresses that were always followed by one
Event-related potential
of two tones, establishing an association between the particular action and the perceptual
Voluntary action
code of the effect tone. In the second part subjects were required to perform random series
Action anticipation
of left and right keypresses. The action triggered randomly one of the experimental stimuli
Orienting response
of a typical oddball task (i.e., most of the time a standard tone and, rarely, a perceptually
P300
deviant tone). Deviant and standard stimuli were the same tones used as effect tones in the
P3a
first phase of the experiment. Deviant stimuli elicited a larger P3a when the action that
Mismatch negativity
triggered stimulus presentation was associated with the standard tone than when it was
Oddball task
associated with the deviant tone. This indicates a larger orienting response in the former
Ideomotor action
case. The findings suggest that the context to which incoming sensory information is compared in order to detect deviant stimuli is codetermined by the sensory effects humans anticipate their actions to have. © 2007 Elsevier B.V. All rights reserved.
1.
Introduction
People are almost constantly exposed to a multitude of external stimuli. Since the brain cannot analyse all of the incoming information to the same extent, humans select the most important aspects of their environment for in-depth processing. Yet, humans are not in danger to miss important or potentially perilous events in the surroundings, because novel or salient stimuli elicit an orienting response, i.e., an involuntary shift of attention to the new or unexpected stimulus (e.g., Sokolov, 1963). For example, when driving a car people might well attend to the radio but a sudden change in the background noise nonetheless triggers an orienting response towards the potential danger.
Hence, humans distinguish between events that are novel or unexpected and events that have already been experienced or were to be expected. Evidently, whether or not a stimulus is considered novel or unexpected depends heavily upon the context in which it is embedded. In most experimental paradigms investigating the orienting response, the subject takes the role of a passive observer who is presented with a series of stimuli, some of which are deviant with respect to the stimulus context. However, humans are hardly ever mere passive observers. Rather, they act on their surroundings in order to achieve desired goals and the stimulus context is strongly modulated by their actions. The ideomotor theory addresses this kind of “intentionbased” action that is meant to produce some internally pre-
⁎ Corresponding author. Laboratoire Psychologie de la Perception, CNRS – Université Paris Descartes, Centre Biomédical des Saints Pères, 45 rue des Sts Pères, 75270 Paris cedex 06, France. E-mail address: [email protected] (F. Waszak). 0006-8993/$ – see front matter © 2007 Elsevier B.V. All rights reserved. doi:10.1016/j.brainres.2007.08.082
BR A IN RE S E A RCH 1 1 83 ( 20 0 7 ) 7 4 –8 2
specified effect (see for example Prinz, 1997). The ideomotor approach claims that performing an action leaves behind an association between the action's motor code and the sensory effects the action produces (“action-effect bindings”). These associations are bi-directional and can, therefore, be used to retrieve an action by “anticipating” its effects (e.g., Elsner and Hommel, 2001; Herwig et al., in press; Kunde, 2001). Actually, the common coding approach (e.g., Prinz, 1990, 1997) assumes that perceived and to-be-produced events share a common representational medium. In this medium, both actions and perceptual events are represented in an abstract format. As a consequence, perceived events and actions destined to produce a sensory effect can interact with each other directly (for a review see Hommel et al., 2001). From this point of view, humans need not only monitor the input for stimuli that are deviant with respect to the context of perceived stimuli, but also for unexpected action effects. If they do so, then the “context”, to which the effect stimulus is compared, is not only defined by the perceived stimuli of the stimulus context. It is also defined by the anticipated consequences of the agent's actions (which, according to the common coding principle, have the same representational format as perceived stimuli). In the present study, we adopt an ideomotor/commoncoding perspective to investigate the effect of intention-based action production (or rather effect production) on deviance processing in the brain. In one of the most prominent paradigms for the study of the orienting response, two classes of stimuli are presented, a frequently occurring stimulus that defines the context – the standard stimulus – and an infrequently occurring stimulus that is meant to elicit the orienting response – the deviant stimulus. There are several variants of this “oddball task”, differing primarily in the task the subjects are required to perform, e.g., whether they have to react to the oddballs, to count them silently, or to watch a silent movie and, thus, to ignore the stimuli (see, for example, Friedman et al., 2001). The majority of studies have used the auditory modality, such that we know a lot about electrophysiological correlates of orienting to deviant auditory events (see Näätänen, 1990; Friedman et al., 2001). The earliest event-related potential (ERP) indicating the detection of a change in an otherwise invariant context of stimulus events is the mismatch negativity (MMN; see Alho, 1995; Näätänen, 1992). The MMN has a fronto-central topography and a latency of 120–250 ms post-stimulus. It is probably generated in and around the primary auditory cortex (Alho, 1995) and, as a secondary source, in the frontal cortex (e.g., Giard et al., 1990). It has been proposed that the frontal cortex receives input from the sensory-specific MMN generator indicating a potentially relevant event and calling for an involuntary orienting response. If the stimulus is sufficiently deviant, the MMN gives rise to a P300. The P300 is considered to consist of two different components that are probably mutually related and may be elicited in tandem, the P3a and P3b (Courchesne et al., 1975; Friedman et al., 2001; Polich, 2003). The P3a has a fronto-central distribution and peaks at about 300 ms post-stimulus. It is elicited by contextually deviant or novel stimuli, and is considered to reflect the frontal lobe function related to orienting of attention (see Posner and Petersen, 1990). The P3b has a more posterior
75
distribution and peaks later. It is elicited by infrequently occurring stimuli that are task-relevant, or involve a decision. It has been proposed that the P3b results from memory updating operations involving the hippocampal formation and the parietal cortex (e.g., Knight, 1996). In the present study, we used the three-tone oddball paradigm, which elicits the P3a and P3b components relatively independently (Comerchero and Polich, 1999; Katayama and Polich, 1998). In this paradigm, subjects are presented with a series of stimuli of either of three classes, a high-probability stimulus – the standard – and two different low-probability stimuli – the target and the deviant. The subjects' task was to silently count the target stimuli. The perceptual difference between standard and target stimuli was small (1940 Hz vs. 2000 Hz), whereas the difference between standard and taskirrelevant deviant stimuli was large (1940 Hz vs. 500 Hz). The P300 to high-deviant (but task-irrelevant) stimuli consists mainly of the P3a component; whereas the P300 to task-relevant (but low-deviant) stimuli consists mainly of the P3b component (see Comerchero and Polich, 1999; Katayama and Polich, 1998). Importantly, Nittono (2006) made subjects perform a threetone oddball task in two conditions that already shed some light on the question as to whether action production affects deviance processing (see also Nittono and Ullsperger, 2000; Nittono et al., 2003). In the self condition, the stimuli were presented in response to subjects' voluntary key presses. In this condition, subjects were instructed to press a pre-specified key with the index finger not faster than once every 2 s. Each keypress triggered one of the three stimuli. In the auto condition, the stimuli were presented automatically at the same inter-stimulus intervals as those recorded in the preceding self condition. Nittono (2006) showed that the P3a elicited by highdeviant task-irrelevant stimuli were enhanced in the self condition compared to the auto condition. By contrast, the P3b to low-deviant task-relevant stimuli was the same in both conditions. This was also true for the MMN. As Nittono (2006) pointed out, the enhanced P3a he observed in the self condition is not necessarily due to an effect of voluntary effect production. Rather, it might be that voluntary effect production merely serves to increase the allocation of attention to the oddball task. However, Nittono also points out that the results may indicate (but not prove beyond doubt) that the neural context to which a stimulus is compared is partially defined by the anticipated effect of the action that triggers the stimulus. According to the ideomotor principle, the perceptual representation of a forthcoming action effect is activated when people intend to produce it by voluntary action. It has been shown that when people frequently experience a perceptual event after a particular action a bidirectional link between action and effect is formed through associative learning mechanisms (Elsner and Hommel, 2001, 2004; Herwig et al., in press). It is plausible to assume that an action is associated to the stimulus event that the subject most often experiences after the execution of the action and that the action, therefore, at some point activates the representation of that stimulus. If so, the action subjects performed in the self condition of Nittono's (2006) experiment would have activated the representation of the high-probability standard stimulus. This anticipatory activation of the standard stimulus, in turn, would have made a deviant
76
BR A IN RE S EA RCH 1 1 83 ( 20 0 7 ) 7 4 –82
stimulus more salient, resulting in a larger orienting response and, therefore, an enhanced P3a. The present study addresses the question as to how performing an action associated to a particular auditory effect affects deviance processing in the brain as reflected by P3a and MMN. Our experiment comprised two phases. First, subjects undergo an acquisition phase, in which subjects performed self-selected keypresses that were always followed by a certain tone (e.g. left keypress → high pitch tone [1940 Hz, later used as the standard stimulus]; right keypress → low pitch tone [500 Hz, later used as the deviant stimulus]). This acquisition phase was meant to establish an association between the particular action code and the perceptual code of the effect stimulus, as demonstrated for example by Elsner and Hommel (2001, 2004) and Herwig et al. (in press). In the second phase, subjects performed a three-tone oddball task, as described above. Importantly, in our paradigm, stimulus presentation was triggered by the subjects' voluntary keypresses. As in the first phase, subjects were required on each trial to choose between a left and a right keypress. The action triggered randomly one of the three types of stimulus. The standard stimulus (1940 Hz) was presented with a probability of .75, target (2000 Hz) and deviant (500 Hz) stimuli were presented with a probability of .125 each. Subjects were required to silently count the target stimuli. Fig. 1 illustrates the experiment.
Fig. 1 – Illustration of the paradigm. In the acquisition phase, left/right keypresses are continuously followed by certain tones, leaving behind a bi-directional association between the action's motor code and the sensory code of the effect. In the oddball phase, subjects' keypresses trigger the presentation of a standard, a target or a deviant tone. According to the ideomotor theory of action control, the voluntary selection of the keypresses involves the anticipation of the effect tone associated to the given action.
Importantly, depending on the action performed on the given trial, the subject anticipated either the standard or the deviant stimulus. Depending on the stimulus that was actually presented, the subject's anticipation could or could not be fulfilled. Our reasoning is the following. According to the common coding approach, effect anticipations are represented in the same domain as perceived stimuli. As a consequence, the anticipation of an action effect constitutes one component of the neural context incoming stimulation is compared to, just as the perceived stimuli that constitute the stimulus context. If the subject anticipates the standard stimulus, then the occurrence of a deviant stimulus should be all the more unexpected and, thus, elicit a larger orienting response. Likewise, if the subject anticipates the deviant stimulus, then the occurrence of a deviant stimulus should be less unexpected. As a consequence, ERPs indicating deviance processing in the brain should be larger in the former case than in the latter. To be precise, on the basis of the findings from Nittono (2006) we expected to find an increased P3a for deviant stimuli when the “anticipated” action outcome is the standard stimulus than when the anticipated action outcome is the deviant stimulus. By contrast, since Nittono failed to find a modulation of the MMN by voluntary action production in the experiment described above, we predicted that the MMN does not interact with the “anticipated” action outcome. Notice that our paradigm does not confound effect anticipation and allocation of attention, since all stimuli in the oddball phase are triggered by voluntary keypresses.
2.
Results
2.1.
Electrophysiological data analysis
Trials that were too close (b1 s) or too remote (N3.5 s) from the previous trials were excluded from the analysis (1.1%). Stimulus-related event-related potentials (ERPs) were calculated timelocked to the onset of the three types of stimuli presented in the oddball phase. The epoch between 200 ms before and 800 ms after stimulus onset was averaged separately for each subject, site, stimulus type (standard, target, deviant), and anticipated effect. The latter factor refers to the perceptual code activated by the actions performed in the oddball phase. If, during the acquisition phase, the given action was followed by the standard stimulus, performing this action during the oddball phase should activate the perceptual code of the standard stimulus (ST-anticipating). If the action has been associated to the deviant stimulus, the action should activate the perceptual code of the deviant stimulus (D-anticipating). (Notice that this factor is counterbalanced across the key position, since the action → effect mapping of the acquisition phase was counterbalanced across subjects.) Stimulus-related ERPs were computed relative to a 200 ms prestimulus baseline voltage level. To examine movement-related ERPs, the epoch between 1000 ms before and 300 ms after action onset (which coincides with the stimulus onset) was averaged collapsing data across the three stimulus types presented in the oddball phase. Movement-related ERPs were computed relative to the 200 ms baseline voltage level between 1000 ms and 800 ms before stimulus/action onset.
BR A IN RE S E A RCH 1 1 83 ( 20 0 7 ) 7 4 –8 2
Amplitudes of the P300s were measured at electrodes Fz, Cz, and Pz and averaged across the time windows 248–420 ms for deviant stimuli and 300–540 ms for target stimuli. The same procedure was done – for both time windows – for the ERPs to standard stimuli. For deviant and target stimuli, peak latencies within the same time windows were measured. Moreover, we measured mean amplitudes of the interval between 100 ms and 148 ms after stimulus-onset for deviant and standard stimuli at electrodes Fz, Cz, and Pz. Usually, the MMN is represented by the difference waveform between deviant and standard stimuli in this time window. Statistical analyses are often run on the difference waveforms. This typical representation of the MMN is shown in Fig. 2c. However, notice that in our paradigm the anticipated effect (D-antici-
77
pating vs. ST-anticipating) could, in principle, modulate both the waveforms elicited by deviant stimuli and also the waveforms elicited by the standard stimuli. By subtracting standard from deviant waveforms, the information which waveform is affected is lost. Therefore, we preferred to analyse the unsubtracted waveforms, including Stimulus Type as a factor in the analysis. These unsubtracted waveforms are shown in Fig. 2b. Notice also that the results do not differ in either way of analysis. For the P300 to deviant stimuli (P3a), the mean amplitudes of the corresponding time window were entered into an ANOVA including the factors Stimulus Type (standard, deviant), Effect Anticipation (ST-anticipating, D-anticipating) and Site (Fz, Cz, Pz). The same was done for the P300 to target
Fig. 2 – (a and b) Grand mean ERP waveforms for target (P3b) and deviant stimuli (P3a; both waveforms are shown together with ERPs for standard stimuli) separately for ST- and D-anticipating actions, and for electrodes Fz, Cz, and Pz. (c) Difference waveforms calculated by subtracting the ERPs to standard stimuli from the ERPs to deviant stimuli. The P3a is larger for deviant stimuli following ST-anticipating actions. The ordinates indicate the stimulus-onset (which coincides with movement-onset).
78
BR A IN RE S EA RCH 1 1 83 ( 20 0 7 ) 7 4 –82
stimuli (P3b) and the unsubtracted waveforms of the MMN. Peak P300 latencies for target and deviant stimuli were analysed by two separate ANOVAs including the factors Effect Anticipation (ST-anticipating, D-anticipating) and Site (Fz, Cz, Pz). Violations of sphericity were corrected using the Huynh– Feldt ε (to facilitate reading, the uncorrected degrees of freedom are provided).
2.2.
Behavioural data
Mean keypress intervals in the oddball phase ranged from 1267 to 2245 ms across participants (mean = 1738 ms, SD = 272 ms). Keypress intervals were not different for ST-anticipating and D-anticipating keypresses (1736 ms vs. 1740 ms, respectively). T-tests showed the mean response ratios for left and right keypresses not to deviate from .5. (.505 vs. .495 for ST-anticipating and D-anticipating trials, respectively; SE = 0.8; t(9) = 1.26). Only a few errors occurred in the counting task. The mean deviation from the correct number (the total of 90 targets for the six blocks) was 3.40.
2.3.
ERPs
Fig. 2a and b show grand mean ERP waveforms. Both target (panel a) and deviant stimuli (panel b) elicited large P300s. As also illustrated in Fig. 3 (panel b), the waveform elicited by target stimuli has the typical characteristics of the P3b: a peak latency of about 440 ms and a posterior topography. (In Fig. 3, the topographic maps are collapsed across Effect Anticipation.) The corresponding ANOVA on the amplitudes yielded a significant main effect of Stimulus Type [F(1,9) = 41.43, p b 0.001] and Site [F(2,18) = 51.27, p b 0.001] qualified by a significant interaction Stimulus Type × Site [F(2,18) = 11.14, p b 0.001]. Two-
tailed t-tests confirmed the posterior topography of the observed P300: they showed the mean P3b amplitudes collapsed across Effect Anticipation to increase from frontal to parietal sites (Fz vs. Cz: p b 0.001; Fz vs. Pz: p b 0.001; Cz vs. Pz: p b 0.01). The ANOVA run on the peak latencies yielded a significant effect of Site [F(2,18) = 8.20, p b 0.01], indicating that the P3b peaks earlier at frontal than at parietal sites. Notice that the P3b was not the focus of the present study, since the manipulation of Effect Anticipation did not concern target stimuli, but only deviant and standard stimuli. We will turn to deviant stimuli now. The waveform in response to deviant stimuli shows the typical characteristics of the P3a: a central topography and a peak latency of about 310 ms post-stimulus (see Fig. 3a). Importantly, the mean P3a amplitudes to deviant stimuli were larger when the stimulus followed an ST-anticipating action than when following a D-anticipating action. Standard stimuli were not affected by whether the subjects predicted the deviant or the standard stimuli. The effect of Effect Anticipation on the P3a is illustrated best in Fig. 2c, which shows the P3a as the difference waveform calculated by subtracting the ERPs to standard stimuli from the ERPs to deviant stimuli. The corresponding ANOVA run on the amplitudes corroborates the pattern of results. The ANOVA yielded significant main effects of Stimulus Type [F(1,9) = 59.63, p b 0.001], Site [F(2,18) = 18.13, p b 0.001] and Effect Anticipation [F(1,9) = 6.55, p b 0.05]. The main effects were qualified by a significant interaction Stimulus Type × Effect Anticipation [F(1,9) = 6.87, p b 0.05]. As concerns the topography of the P3a, two-tailed t-tests showed the mean amplitudes collapsed across Effect Anticipation to be larger at central and parietal sites than at frontal sites, but not to differ between central and parietal sites (Fz vs. Cz: p b 0.01; Fz vs. Pz: p b 0.02; Cz vs. Pz: p N 0.5). As concerns the interaction Stimulus Type × Effect Anticipation, separate ANOVAs for
Fig. 3 – Topographic maps of the P3a to deviant stimuli (a) and P3b to target stimuli (b) collapsed across ST- and D-anticipating actions.
BR A IN RE S E A RCH 1 1 83 ( 20 0 7 ) 7 4 –8 2
deviant and standard ERPs showed the main effect of Effect Anticipation to be significant only for deviant ERPs but not for standard ERPs [F(1,9) = 8.1, p b 0.02 and F(1,9) b 1, p = 0.92, respectively]. The ANOVA run on the peak latencies did not yield any significant effect. Fig. 2b shows an early negative potential peaking about 120 ms after stimulus onset that was larger for deviant than for standard stimuli. Fig. 2c shows the difference waveform between deviant and standard stimuli. The early peak at about 120 ms represents the MMN. The amplitude of this potential was not affected by Effect Anticipation and had a centroparietal topography.1 The only significant effect of the corresponding ANOVA was the interaction Stimulus Type × Site [F(2,18) = 5.66, p b 0.02]. One-tailed t-tests showed the difference between standard and deviant stimuli to be significant for electrodes Cz and Pz (Fz: t = 0.29, df = 9, p = 0.4; Cz: t = 1.97, df = 9, p b 0.05; Pz: t = 3.2, df = 9, p b 0.01). Movement-related ERPs show the typical slow negative shift preceding movement onset at central sites. Running t-tests revealed that this “readiness potential” (Kornhuber and Deecke, 1965) was the same for ST-anticipating and D-anticipating actions.
3.
Discussion
The main results of the present study can be summarized as follows. Deviant non-target stimuli elicited a P300 with a more anterior topography and shorter peak latency than the P300 elicited by low-deviant target stimuli. Thus, the P300s elicited by the two types of oddball probably consist primarily of the P3a and P3b components, respectively (Comerchero and Polich, 1999). The P3a in response to deviant stimuli was larger when the stimulus was triggered by an action that, in the acquisition phase, was continuously followed by the standard stimulus than when triggered by an action that, in the acquisition phase, was continuously followed by the deviant stimulus. Moreover, the MMN peaking about 120 ms poststimulus (and also the P3b) were the same for stimuli triggered by ST- and D-anticipating actions. These findings are in accordance with our hypotheses derived from the study of Nittono (2006) mentioned in the introduction.
3.1.
P3a and P3b
As outlined above, the P3a is considered to reflect a switch of attention towards a deviant stimulus that presumably serves to make the event available to consciousness and behavioural control (Knight and Scabini, 1998; Squires et al., 1975; Woods, 1990; Friedman et al., 2001). Evidently, for a stimulus to be considered as “deviant” it must be compared to the “context” in which it occurs. Most studies investigating deviance processing presume that the context upon which attentional processes operate is provided by the neural representation of 1
The MMN is typically larger at Fz than Cz (see Kujala et al., 2007). We are not aware of whether the centro-parietal distribution observed in the present study is due to the deviant stimuli being initiated by voluntary keypresses and do not want to engage in speculations.
79
the stimulus environment. A stimulus that is not predicted given the current stimulus context elicits an orienting response. The present study extends this notion to stimulus deviance that results from the mismatch between the anticipated and the actual effect of a voluntary action. In the introduction, we suggested that the context to which external stimulation is compared is partially provided by the neural representation of the perceptual effects that the agent predicts his actions to have in the environment. An action effect that does not correspond to the agent's anticipation should be considered as deviant and, thus, elicit an orienting response. The results presented above support this hypothesis. We explain the results along the lines of the ideomotor theory of action control. This theory proposes a model of how humans select a suitable action to achieve a particular goal. Ideomotor approaches assume that performing an action establishes a link between the action's motor code and the sensory effects the action produces (“action–effect bindings”; see Elsner and Hommel, 2001, 2004; Herwig et al., in press). Herwig et al. consider this integration of action and effect to take place whenever a particular intention-based, self-selected action and a particular sensory code are frequently co-activated. These associations are bi-directional and are used to retrieve an action by internally “evoking” its effects. In other words, intentionbased actions are selected with respect to their perceptual consequences and anticipating these consequences is an integral part of the control of voluntary action (e.g., Prinz, 1990, 1997; Hommel et al., 2001; Kunde, 2001; Herwig et al., in press). During the acquisition phase of the present study, the action codes of the left/right keypresses were associated with the representations of the ensuing effect tones (standard or deviant, respectively). As a consequence, voluntarily selecting and performing left/right keypresses during the oddball phase involved the anticipation of these effects. Hence, subjects performed keypresses that involve the anticipation of either the standard or the deviant tone. From this ideomotor perspective, the explanation of the modulation of the P3a by whether the subject performed an ST- or a D-anticipating action is straightforward. If the subject's action involved the anticipation of the standard tone and if this anticipation partially defines the neural context of the current processing episode, then the occurrence of the deviant tone is all the more unexpected and, as a consequence, elicits a larger orienting response (i.e., a larger P3a). If, by contrast, the subject's action involved the anticipation of the deviant tone, the occurrence of this tone is less unexpected, making the ensuing orienting response weaker. This mechanism helps people to focus attention to unintended action effects and to initiate an appropriate correction of the current stream of action. In the current context, a recent study from Ullsperger, Nittono, and von Cramon (2007) is of great interest. Data of Ullsperger et al. suggest that the posterior medial frontal cortex (pMFC) signals the need for adjustments when the consequences of an action are worse than intended (see also Walton et al., 2004 who suggest that the pMFC is involved in monitoring the outcomes of self-generated actions). Ullsperger et al. also suggest that the pMFC plays a role in the selection of appropriate compensatory actions. Since the malfunctions they simulated were rare and unexpected action outcomes,
80
BR A IN RE S EA RCH 1 1 83 ( 20 0 7 ) 7 4 –82
Ullsperger et al. argued that the activity in the pMFC they observed may reflect a general response to an oddball event; an assumption that is corroborated by the fact that infrequent trial types in a target detection task have been demonstrated to elicit pMFC activity (Braver et al., 2001). One may thus speculate that the increased P3a observed in the present study is due to increased activity in the pMFC elicited by unpredicted action effects. However, notice that, in contrast to action-relevant target events, passively perceived oddball events (that are not triggered by the subjects' actions, as in the present study) do not seem to engage the pMFC (Liebenthal et al., 2003; Opitz et al., 2002). Hence, if the increased P3a observed in the present study is due to activity in the pMFC, then the pMFC seems to be involved in monitoring the interaction between the agent and the environment, be it that the agent reacts to external stimuli (target detection task) or that the agent produces effects in the environment (present study). As mentioned before, in principle, the factor Effect Anticipation could influence both ERPs to deviant and to standard stimuli. However, the results reported above clearly show that only the P3a elicited by the former are affected by the subject's action. This shows that the context to which the incoming input is compared is merely codetermined by the anticipated effect, i.e., the anticipated effect only modulates the stimulus context, but cannot overrule it. Since the standard stimulus is in no way deviant compared to the stimulus context, it does not yield an orienting response, regardless of whether it was anticipated by the action or not. The P3b was not the focus of the present study and is not discussed further. Notice that the actions were associated to the standard and the deviant stimulus, but not to the target. Thus, it was not to be expected that the P3b to target stimuli would be modulated by the subjects' actions. We opted for the inclusion of target stimuli in order to show that the waveform elicited by the deviant stimulus unambiguously reflects a P3a. Moreover, we wanted to be able to directly compare our results to the study of Nittono (2006) who used a three-tone oddball paradigm.
3.2.
Mismatch negativity
The MMN is the first electrophysiological indicator of the detection of a change in an otherwise invariant context of stimuli (e.g., Näätänen, 1992). It is elicited even when the subject is not attending to the stimuli (Woldorff et al., 1998; Näätänen, 1991) and, in fact, is considered to trigger a switch of attention (reflected by the P3a) if the deviant stimulus event is intrusive (Donchin and Coles, 1988; Escera et al., 2000). The results presented above clearly show that the MMN was not affected by the effect subjects anticipate. This is in line with the findings of Nittono (2006) who showed that the MMN did not differ between tones initiated by the computer and tones initiated by a voluntary keypress. We believe this to be an interesting finding, since it has been shown that the MMN does not reflect the mere detection of deviation from static stimulus properties. Rather, the MMN reflects the detection of deviation from stimulus regularities, as indicated by the fact that it is elicited by violations of an abstract rule of auditory input (e.g. pitch repetition in a sequence of regularly descending sounds [Tervaniemi et al., 1994; see also Paavilai-
nen et al., 2001]). In this case, the context is represented by the abstract rule. Thus, a priori it would not be implausible to assume that the MMN is also sensitive to violations of effect anticipation, which, after all, must involve the application of action–effect rules. That this is not the case may be taken to demonstrate that the neural representation, on which the early mechanisms screening the input for deviant events work, is not yet enhanced with information about anticipated action effects. This conclusion is in line with an experiment reported by Rinne, Antila, and Winkler (2001). These authors investigated whether the MMN is affected by the subject's prior knowledge of the sound changes. They instructed subjects to produce a sequence of keypresses by pressing one key frequently and another infrequently. In one condition, the frequently performed keypress triggered the standard tone, the other keypress the deviant tone. In another condition, each keypress triggered the next tone of a prearranged standard/deviant sequence. Rinne et al. did not observe any difference in the MMN between the two conditions. This shows, in full accordance with the results of the present study, that top–down predictive information does not affect the MMN-generating processes.
3.3.
Conclusion
In order to interact effectively with the environment we have to set up perceptual expectations of our actions' effects on the environment. This anticipation optimises the accuracy and speed of complex action sequences, where actions often depend on the outcome of the preceding action. However, the effect of a goal-directed action does not always come out as expected. As a matter of course, unpredicted action effects indicate a major perturbance of the ongoing agent–environment interaction and indicate immediate need for action correction. Thus, humans should monitor the perceptual input for action effects that do not meet their expectations. Unexpected effects should elicit an orienting response that makes the given event available to behavioural control. The present study supports this notion. Moreover, the study corroborates the inseparability of action and perception as conceptualised in the common coding theory (Prinz, 1990; Hommel et al., 2001; see also Schubotz and von Cramon, 2001). The common coding principle suggests that perception and action are represented in a common medium, and that actions are controlled in terms of their intended sensory effects, rather then in terms of a specific movement (ideomotor principle). A motor plan represents sensory events the agent intends to bring about in the environment. Thus, anticipated action effects constitute an integral part of the context to which a sensory event is compared when checked for deviance from prediction. When this prediction is not fulfilled, the orienting response is pronounced.
4.
Experimental procedures
4.1.
Methods
4.1.1.
Subjects
Ten subjects (six females, four males; mean age 24 years) participated in a single session lasting about 1.5 h. All subjects
BR A IN RE S E A RCH 1 1 83 ( 20 0 7 ) 7 4 –8 2
were naive with respect to the purpose of the experiment. None had a history of neurological disease or trauma.
4.1.2.
Stimuli
Timing of stimuli was controlled by an IBM-compatible computer, interfaced to a 22-in. IIYAMA VISION MASTER PRO 510 monitor and to two loudspeakers in front of the participants.
4.1.2.1. Acquisition phase. A small white cross presented in the centre of the screen on a black background served as fixation point. Two sinus tones were used as action effects: 1940 Hz (later used as the standard stimulus) and 500 Hz (later used as the deviant stimulus). 4.1.2.2. Three-tone oddball task. We used three different sinus tones: 1940 Hz (standard), 2000 Hz (target), and 500 Hz (deviant). The three tones correspond to those used by Nittono (2006) and by Comerchero and Polich (1999). All tones were 70 ms in duration, had a sharp on- and off-set, and were presented at about 60 dB SPL. 4.1.3.
Procedure
During the whole experiment, subjects were monitored with a camera to check whether they kept the eyes open. The experiment was divided into an acquisition phase and a subsequent oddball phase (see Fig. 1).
4.1.3.1. Acquisition phase. The acquisition phase comprised two 200-trial blocks separated by a break of approximately 20 min during which the EEG recording was prepared. Splitting the acquisition phase in two sub-blocks was a compromise between the necessities of keeping the time short between the application of the electrodes and the beginning of the data recording, on the one side, and between the acquisition phase and the oddball phase, on the other side. In each block, subjects had to press one of two response keys with the index finger of the left or the right hand. Subjects performed 200 keypresses at a pace of about once per 2 s. We made subjects use different hands instead of different fingers of the same hand, because we believe that action–effect learning is easier for motor codes concerning two different hands than for motor codes concerning two different fingers. The response keys were horizontally arranged. The distance between the keys was 13 cm. The subjects were verbally instructed to choose freely which key to press but were instructed to use the keys about equally often and in a random order (i.e., to avoid fixed sequences). Each keypress (left/right) triggered a particular effect tone (1940 Hz/500 Hz). The tone was presented immediately after the keypress (stimulus onset asynchrony [SOA] = 0 ms). The action–effect mapping was counterbalanced across subjects. Keypresses that were too fast (b1000 ms) or too slow (N3500 ms) were fed back to the participants by a visual warning message. The number of left versus right keypresses was displayed on the screen at the end of each block. 4.1.3.2. Oddball phase. Subjects performed a three-tone oddball task. Stimulus presentation was triggered by subjects' voluntary keypresses. Subjects were required to perform one of the two keypresses of the acquisition phase (left/right) at a
81
pace of about once per 2 s. Once again, they were instructed to choose freely which key to press while trying to use the keys about equally often and in a random order. No further instructions regarding the relationship between the two phases were given. Subjects were not told to consciously anticipate the forthcoming tones. Each keypress triggered one of the three stimuli with an onset asynchrony of 0 ms. The standard stimulus (1940 Hz) was presented with a probability of .75, target (2000 Hz) and deviant stimuli (500 Hz) were presented with a probability of .125 each. Subjects were required to silently count the target stimuli. The oddball phase was run in six blocks of 120 keypresses. Whereas in every block 90 standard stimuli were presented, the actual number of target and deviant stimuli varied from 14 to 16 across blocks (otherwise the counting of the targets would be redundant). After each block, subjects had to report the number of target stimuli to the experimenter.
4.1.4.
Electrophysiological recording
Continuous EEG was acquired through the ActiveTwo Biosemi™ electrode system from 64 scalp positions, digitized at 256 Hz. There was no high pass filtering during recording since the hardware is completely DC coupled. Low-pass filtering during recording was performed digitally in the ADC's decimation filter (hardware bandwidth limit), which has a 5th order sinc response with a −3 dB point at 51.2 Hz, cf. http:// www.biosemi.com/faq/adjust_filter.htm. To control for ocular artefacts, an electro-oculogram (EOG) was recorded both vertically from above and below the left eye (vEOG) and horizontally from the outer canthi of both eyes (hEOG). EEG was re-referenced off-line to linked mastoids and low-pass filtered at 60 Hz. There was no off-line high-pass filtering. Epochs containing EEG or EOG over ±75 μV were removed automatically. Muscle artefacts and segments with eye movements were removed by visual inspection.
Acknowledgments The authors wish to thank Cornelia Belger and Mandy Kosel for assistance in data acquisition, Christina Jaeger, Antje Hollaender and Clemens Maidhof for helpful comments and Henrik Grunert for building the response device. We also thank Nikolaus Steinbeis for checking and improving the English.
REFERENCES
Alho, K., 1995. Cerebral generators of mismatch negativity (MMN) and its magnetic counterpart (MMNm) elicited by sound changes. Ear Hear. 16, 38–51. Braver, T.S., Barch, D.M., Gray, J.R., Molfese, D.L., Snyder, A., 2001. Anterior cingulate cortex and response conflict: effects of frequency, inhibition and errors. Cereb. Cortex 11, 825–836. Comerchero, M., Polich, J., 1999. P3a and P3b from typical auditory and visual stimuli. Clin. Neurophysiol. 110, 24–30. Courchesne, E., Hillyard, S.A., Galambos, R., 1975. Stimulus novelty, task relevance and the visual evoked potential in man. Electroencephalogr. Clin. Neurophysiol. 39, 131–143. Donchin, E., Coles, M.G.H., 1988. Is the P300 component a manifestation of context updating? Behav. Brain Sci. 11, 357–374.
82
BR A IN RE S EA RCH 1 1 83 ( 20 0 7 ) 7 4 –82
Elsner, B., Hommel, B., 2001. Effect anticipation and action control. J. Exp. Psychol. Hum. Percept. Perform 27, 229–240. Elsner, B., Hommel, B., 2004. Contiguity and contingency in action–effect learning. Psychol. Res. 68, 138–154. Escera, C., Alho, K., Schröger, E., Winkler, I., 2000. Involuntary attention and distractibility as evaluated with event-related brain potentials. Audiol. Neuro-Otol. 5, 151–166. Friedman, D., Cycowicz, Y.M., Gaeta, H., 2001. The novelty P3: an event-related brain potential (ERP) sign of the brain's evaluation of novelty. Neurosci. Biobehav. Rev. 25, 355–373. Giard, M.H., Perrin, F., Pernier, J., Bouchet, P., 1990. Brain generators implicated in the processing of auditory stimulus deviance: a topographic event-related potential study. Psychophysiology 27, 627–640. Herwig, A., Prinz, W., Waszak, F., in press. Two modes of sensorimotor integration in intention-based and stimulus-based action. Q. J. Exp. Psychol. Hommel, B., Müsseler, J., Aschersleben, G., Prinz, W., 2001. The theory of event coding (TEC): a framework for perception and action planning. Behav. Brain Sci. 24, 849–878. Katayama, J., Polich, J., 1998. Stimulus context determines P3a and P3b. Psychophysiology 35 (1), 23–33. Knight, R., 1996. Contribution of human hippocampal region to novelty detection. Nature 383, 256–259. Knight, R.T., Scabini, D., 1998. Anatomic bases of event-related potentials and their relationship to novelty detection in humans. J. Clin. Neurophysiol. 15, 3–13. Kornhuber, H.H., Deecke, L., 1965. Hirnpotentialänderungen bei Willkürbewegungen und passiven Bewegungen des Menschen: Bereitschaftspotential und reafferente Potentiale. Pflugers Arch. Gesamte Physiol. Menschen Tiere 284, 1–17. Kujala, T., Tervaniemi, M., Schröger, E., 2007. The mismatch negativity in cognitive and clinical neuroscience: theoretical and methodological considerations. Biol. Psychol. 74, 1–19. Kunde, W., 2001. Response–effect compatibility in manual choice reaction tasks. J. Exp. Psychol. Hum. Percept. Perform 27, 387–394. Liebenthal, E., Ellingson, M.L., Spanaki, M.V., Prieto, T.E., Ropella, K.M., Binder, J.R., 2003. Simultaneous ERP and fMRI of the auditory cortex in a passive oddball paradigm. NeuroImage 19, 1395–1404. Näätänen, R., 1990. The role of attention in auditory information processing as revealed by event-related potentials and other measures of cognitive function. Behav. Brain Sci. 13, 201–232. Näätänen, R., 1991. Mismatch negativity (MMN) outside strong attentional focus: a commentary on Woldorff et al. Psychophysiology 28, 478–484. Näätänen, R., 1992. Attention and Brain Function. Erlbaum, Hillsdale, NJ. Nittono, H., 2006. Voluntary stimulus production enhances deviance processing in the brain. Int. J. Psychophysiol. 59, 15–21. Nittono, H., Ullsperger, P., 2000. Event-related potentials in a self-paced novelty oddball task. NeuroReport 11, 1861–1864.
Nittono, H., Hamada, A., Hori, T., 2003. Brain potentials after clicking a mouse: a new psychophysiological approach to human–computer interaction. Hum. Factors 45, 591–599. Opitz, B., Rinne, T., Mecklinger, A., von Cramon, D.Y., Schroger, E., 2002. Differential contribution of frontal and temporal cortices to auditory change detection: fMRI and ERP results. NeuroImage 15, 167–174. Paavilainen, P., Simola, J., Jaramillo, M., Näätänen, R., Winkler, I., 2001. Preattentive extraction of abstract feature conjunctions from auditory stimulation as reflected by the mismatch negativity (MMN). Psychophysiology 38, 359–365. Polich, J., 2003. Theoretical overview of P3a and P3b. In: Polich, J. (Ed.), Detection of Change: Event-Related Potential and fMRI Findings. Kluwer Academic, Boston, pp. 83–98. Prinz, W., 1990. A common coding approach to perception and action. In: Neumann, O., Prinz, W. (Eds.), Relationships between Perception and Action: Current Approaches. Springer, Berlin, pp. 167–201. Prinz, W., 1997. Perception and action planning. Eur. J. Cogn. Psychol. 9, 129–154. Posner, M., Petersen, S., 1990. The attention system of the human brain. Annu. Rev. Neurosci. 13, 25–42. Rinne, T., Antila, S., Winkler, I., 2001. Mismatch negativity is unaffected by top-down predictive information. NeuroReport 12 (10), 2209–2213. Schubotz, R.I., von Cramon, D.Y., 2001. Functional organization of the lateral premotor cortex: fMRI reveals different regions activated by anticipation of object properties, location and speed. Cogn. Brain Res. 11, 97–112. Sokolov, E.N., 1963. Perception and the Conditioned Reflex. Pergamon Press, Oxford, UK. Squires, K.C., Squires, N.K., Hillyard, S.A., 1975. Decision-related cortical potentials during an auditory signal detection task with cued observation intervals. J. Exp. Psychol. Hum. Percept. Perform 1, 268–279. Tervaniemi, M., Maury, S., Näätänen, R., 1994. Neural representations of abstract stimulus features in the human brain as reflected by the mismatch negativity. NeuroReport 5, 844–846. Ullsperger, M., Nittono, H., von Cramon, D.Y., 2007. When goals are missed: dealing with self-generated and externally induced failure. NeuroImage 35, 1356–1364. Walton, M.E., Devlin, J.T., Rushworth, M.F., 2004. Interactions between decision making and performance monitoring within prefrontal cortex. Nat. Neurosci. 7, 1259–1265. Woldorff, M., Hillyard, S.A., Gallen, C.C., Hampson, S.R., Bloom, F.E., 1998. Magnetoencephalographic recordings demonstrate attentional modulation of mismatch-related neural activity in human auditory cortex. Psychophysiology 35, 283–292. Woods, D.L., 1990. The physiological basis of selective attention: implications of event-related potential studies. In: Rohrbaugh, J.W., Parasuraman, R., Johnson Jr., R. (Eds.), Event-Related Potentials: Basic Issues and Applications. Oxford University Press, New York, pp. 178–209.
a v a i l a b l e a t w w w. s c i e n c e d i r e c t . c o m
w w w. e l s e v i e r. c o m / l o c a t e / b r a i n r e s
Research Report
Effect anticipation modulates deviance processing in the brain Florian Waszak a,⁎, Arvid Herwig b a b
Laboratoire Psychologie de la Perception, CNRS and Université Paris Descartes, Paris, France Max Planck Institute for Human Cognitive and Brain Sciences, Department of Psychology, Leipzig, Germany
A R T I C LE I N FO
AB S T R A C T
Article history:
Humans constantly perform actions to achieve desired goals in the environment. However,
Accepted 31 August 2007
only very little is known about how actions influence stimulus processing. The present
Available online 16 September 2007
study addresses the question as to how performing an action that is associated with a particular auditory effect influences deviance processing in the brain. In the first part of the
Keywords:
experiment, subjects performed left and right keypresses that were always followed by one
Event-related potential
of two tones, establishing an association between the particular action and the perceptual
Voluntary action
code of the effect tone. In the second part subjects were required to perform random series
Action anticipation
of left and right keypresses. The action triggered randomly one of the experimental stimuli
Orienting response
of a typical oddball task (i.e., most of the time a standard tone and, rarely, a perceptually
P300
deviant tone). Deviant and standard stimuli were the same tones used as effect tones in the
P3a
first phase of the experiment. Deviant stimuli elicited a larger P3a when the action that
Mismatch negativity
triggered stimulus presentation was associated with the standard tone than when it was
Oddball task
associated with the deviant tone. This indicates a larger orienting response in the former
Ideomotor action
case. The findings suggest that the context to which incoming sensory information is compared in order to detect deviant stimuli is codetermined by the sensory effects humans anticipate their actions to have. © 2007 Elsevier B.V. All rights reserved.
1.
Introduction
People are almost constantly exposed to a multitude of external stimuli. Since the brain cannot analyse all of the incoming information to the same extent, humans select the most important aspects of their environment for in-depth processing. Yet, humans are not in danger to miss important or potentially perilous events in the surroundings, because novel or salient stimuli elicit an orienting response, i.e., an involuntary shift of attention to the new or unexpected stimulus (e.g., Sokolov, 1963). For example, when driving a car people might well attend to the radio but a sudden change in the background noise nonetheless triggers an orienting response towards the potential danger.
Hence, humans distinguish between events that are novel or unexpected and events that have already been experienced or were to be expected. Evidently, whether or not a stimulus is considered novel or unexpected depends heavily upon the context in which it is embedded. In most experimental paradigms investigating the orienting response, the subject takes the role of a passive observer who is presented with a series of stimuli, some of which are deviant with respect to the stimulus context. However, humans are hardly ever mere passive observers. Rather, they act on their surroundings in order to achieve desired goals and the stimulus context is strongly modulated by their actions. The ideomotor theory addresses this kind of “intentionbased” action that is meant to produce some internally pre-
⁎ Corresponding author. Laboratoire Psychologie de la Perception, CNRS – Université Paris Descartes, Centre Biomédical des Saints Pères, 45 rue des Sts Pères, 75270 Paris cedex 06, France. E-mail address: [email protected] (F. Waszak). 0006-8993/$ – see front matter © 2007 Elsevier B.V. All rights reserved. doi:10.1016/j.brainres.2007.08.082
BR A IN RE S E A RCH 1 1 83 ( 20 0 7 ) 7 4 –8 2
specified effect (see for example Prinz, 1997). The ideomotor approach claims that performing an action leaves behind an association between the action's motor code and the sensory effects the action produces (“action-effect bindings”). These associations are bi-directional and can, therefore, be used to retrieve an action by “anticipating” its effects (e.g., Elsner and Hommel, 2001; Herwig et al., in press; Kunde, 2001). Actually, the common coding approach (e.g., Prinz, 1990, 1997) assumes that perceived and to-be-produced events share a common representational medium. In this medium, both actions and perceptual events are represented in an abstract format. As a consequence, perceived events and actions destined to produce a sensory effect can interact with each other directly (for a review see Hommel et al., 2001). From this point of view, humans need not only monitor the input for stimuli that are deviant with respect to the context of perceived stimuli, but also for unexpected action effects. If they do so, then the “context”, to which the effect stimulus is compared, is not only defined by the perceived stimuli of the stimulus context. It is also defined by the anticipated consequences of the agent's actions (which, according to the common coding principle, have the same representational format as perceived stimuli). In the present study, we adopt an ideomotor/commoncoding perspective to investigate the effect of intention-based action production (or rather effect production) on deviance processing in the brain. In one of the most prominent paradigms for the study of the orienting response, two classes of stimuli are presented, a frequently occurring stimulus that defines the context – the standard stimulus – and an infrequently occurring stimulus that is meant to elicit the orienting response – the deviant stimulus. There are several variants of this “oddball task”, differing primarily in the task the subjects are required to perform, e.g., whether they have to react to the oddballs, to count them silently, or to watch a silent movie and, thus, to ignore the stimuli (see, for example, Friedman et al., 2001). The majority of studies have used the auditory modality, such that we know a lot about electrophysiological correlates of orienting to deviant auditory events (see Näätänen, 1990; Friedman et al., 2001). The earliest event-related potential (ERP) indicating the detection of a change in an otherwise invariant context of stimulus events is the mismatch negativity (MMN; see Alho, 1995; Näätänen, 1992). The MMN has a fronto-central topography and a latency of 120–250 ms post-stimulus. It is probably generated in and around the primary auditory cortex (Alho, 1995) and, as a secondary source, in the frontal cortex (e.g., Giard et al., 1990). It has been proposed that the frontal cortex receives input from the sensory-specific MMN generator indicating a potentially relevant event and calling for an involuntary orienting response. If the stimulus is sufficiently deviant, the MMN gives rise to a P300. The P300 is considered to consist of two different components that are probably mutually related and may be elicited in tandem, the P3a and P3b (Courchesne et al., 1975; Friedman et al., 2001; Polich, 2003). The P3a has a fronto-central distribution and peaks at about 300 ms post-stimulus. It is elicited by contextually deviant or novel stimuli, and is considered to reflect the frontal lobe function related to orienting of attention (see Posner and Petersen, 1990). The P3b has a more posterior
75
distribution and peaks later. It is elicited by infrequently occurring stimuli that are task-relevant, or involve a decision. It has been proposed that the P3b results from memory updating operations involving the hippocampal formation and the parietal cortex (e.g., Knight, 1996). In the present study, we used the three-tone oddball paradigm, which elicits the P3a and P3b components relatively independently (Comerchero and Polich, 1999; Katayama and Polich, 1998). In this paradigm, subjects are presented with a series of stimuli of either of three classes, a high-probability stimulus – the standard – and two different low-probability stimuli – the target and the deviant. The subjects' task was to silently count the target stimuli. The perceptual difference between standard and target stimuli was small (1940 Hz vs. 2000 Hz), whereas the difference between standard and taskirrelevant deviant stimuli was large (1940 Hz vs. 500 Hz). The P300 to high-deviant (but task-irrelevant) stimuli consists mainly of the P3a component; whereas the P300 to task-relevant (but low-deviant) stimuli consists mainly of the P3b component (see Comerchero and Polich, 1999; Katayama and Polich, 1998). Importantly, Nittono (2006) made subjects perform a threetone oddball task in two conditions that already shed some light on the question as to whether action production affects deviance processing (see also Nittono and Ullsperger, 2000; Nittono et al., 2003). In the self condition, the stimuli were presented in response to subjects' voluntary key presses. In this condition, subjects were instructed to press a pre-specified key with the index finger not faster than once every 2 s. Each keypress triggered one of the three stimuli. In the auto condition, the stimuli were presented automatically at the same inter-stimulus intervals as those recorded in the preceding self condition. Nittono (2006) showed that the P3a elicited by highdeviant task-irrelevant stimuli were enhanced in the self condition compared to the auto condition. By contrast, the P3b to low-deviant task-relevant stimuli was the same in both conditions. This was also true for the MMN. As Nittono (2006) pointed out, the enhanced P3a he observed in the self condition is not necessarily due to an effect of voluntary effect production. Rather, it might be that voluntary effect production merely serves to increase the allocation of attention to the oddball task. However, Nittono also points out that the results may indicate (but not prove beyond doubt) that the neural context to which a stimulus is compared is partially defined by the anticipated effect of the action that triggers the stimulus. According to the ideomotor principle, the perceptual representation of a forthcoming action effect is activated when people intend to produce it by voluntary action. It has been shown that when people frequently experience a perceptual event after a particular action a bidirectional link between action and effect is formed through associative learning mechanisms (Elsner and Hommel, 2001, 2004; Herwig et al., in press). It is plausible to assume that an action is associated to the stimulus event that the subject most often experiences after the execution of the action and that the action, therefore, at some point activates the representation of that stimulus. If so, the action subjects performed in the self condition of Nittono's (2006) experiment would have activated the representation of the high-probability standard stimulus. This anticipatory activation of the standard stimulus, in turn, would have made a deviant
76
BR A IN RE S EA RCH 1 1 83 ( 20 0 7 ) 7 4 –82
stimulus more salient, resulting in a larger orienting response and, therefore, an enhanced P3a. The present study addresses the question as to how performing an action associated to a particular auditory effect affects deviance processing in the brain as reflected by P3a and MMN. Our experiment comprised two phases. First, subjects undergo an acquisition phase, in which subjects performed self-selected keypresses that were always followed by a certain tone (e.g. left keypress → high pitch tone [1940 Hz, later used as the standard stimulus]; right keypress → low pitch tone [500 Hz, later used as the deviant stimulus]). This acquisition phase was meant to establish an association between the particular action code and the perceptual code of the effect stimulus, as demonstrated for example by Elsner and Hommel (2001, 2004) and Herwig et al. (in press). In the second phase, subjects performed a three-tone oddball task, as described above. Importantly, in our paradigm, stimulus presentation was triggered by the subjects' voluntary keypresses. As in the first phase, subjects were required on each trial to choose between a left and a right keypress. The action triggered randomly one of the three types of stimulus. The standard stimulus (1940 Hz) was presented with a probability of .75, target (2000 Hz) and deviant (500 Hz) stimuli were presented with a probability of .125 each. Subjects were required to silently count the target stimuli. Fig. 1 illustrates the experiment.
Fig. 1 – Illustration of the paradigm. In the acquisition phase, left/right keypresses are continuously followed by certain tones, leaving behind a bi-directional association between the action's motor code and the sensory code of the effect. In the oddball phase, subjects' keypresses trigger the presentation of a standard, a target or a deviant tone. According to the ideomotor theory of action control, the voluntary selection of the keypresses involves the anticipation of the effect tone associated to the given action.
Importantly, depending on the action performed on the given trial, the subject anticipated either the standard or the deviant stimulus. Depending on the stimulus that was actually presented, the subject's anticipation could or could not be fulfilled. Our reasoning is the following. According to the common coding approach, effect anticipations are represented in the same domain as perceived stimuli. As a consequence, the anticipation of an action effect constitutes one component of the neural context incoming stimulation is compared to, just as the perceived stimuli that constitute the stimulus context. If the subject anticipates the standard stimulus, then the occurrence of a deviant stimulus should be all the more unexpected and, thus, elicit a larger orienting response. Likewise, if the subject anticipates the deviant stimulus, then the occurrence of a deviant stimulus should be less unexpected. As a consequence, ERPs indicating deviance processing in the brain should be larger in the former case than in the latter. To be precise, on the basis of the findings from Nittono (2006) we expected to find an increased P3a for deviant stimuli when the “anticipated” action outcome is the standard stimulus than when the anticipated action outcome is the deviant stimulus. By contrast, since Nittono failed to find a modulation of the MMN by voluntary action production in the experiment described above, we predicted that the MMN does not interact with the “anticipated” action outcome. Notice that our paradigm does not confound effect anticipation and allocation of attention, since all stimuli in the oddball phase are triggered by voluntary keypresses.
2.
Results
2.1.
Electrophysiological data analysis
Trials that were too close (b1 s) or too remote (N3.5 s) from the previous trials were excluded from the analysis (1.1%). Stimulus-related event-related potentials (ERPs) were calculated timelocked to the onset of the three types of stimuli presented in the oddball phase. The epoch between 200 ms before and 800 ms after stimulus onset was averaged separately for each subject, site, stimulus type (standard, target, deviant), and anticipated effect. The latter factor refers to the perceptual code activated by the actions performed in the oddball phase. If, during the acquisition phase, the given action was followed by the standard stimulus, performing this action during the oddball phase should activate the perceptual code of the standard stimulus (ST-anticipating). If the action has been associated to the deviant stimulus, the action should activate the perceptual code of the deviant stimulus (D-anticipating). (Notice that this factor is counterbalanced across the key position, since the action → effect mapping of the acquisition phase was counterbalanced across subjects.) Stimulus-related ERPs were computed relative to a 200 ms prestimulus baseline voltage level. To examine movement-related ERPs, the epoch between 1000 ms before and 300 ms after action onset (which coincides with the stimulus onset) was averaged collapsing data across the three stimulus types presented in the oddball phase. Movement-related ERPs were computed relative to the 200 ms baseline voltage level between 1000 ms and 800 ms before stimulus/action onset.
BR A IN RE S E A RCH 1 1 83 ( 20 0 7 ) 7 4 –8 2
Amplitudes of the P300s were measured at electrodes Fz, Cz, and Pz and averaged across the time windows 248–420 ms for deviant stimuli and 300–540 ms for target stimuli. The same procedure was done – for both time windows – for the ERPs to standard stimuli. For deviant and target stimuli, peak latencies within the same time windows were measured. Moreover, we measured mean amplitudes of the interval between 100 ms and 148 ms after stimulus-onset for deviant and standard stimuli at electrodes Fz, Cz, and Pz. Usually, the MMN is represented by the difference waveform between deviant and standard stimuli in this time window. Statistical analyses are often run on the difference waveforms. This typical representation of the MMN is shown in Fig. 2c. However, notice that in our paradigm the anticipated effect (D-antici-
77
pating vs. ST-anticipating) could, in principle, modulate both the waveforms elicited by deviant stimuli and also the waveforms elicited by the standard stimuli. By subtracting standard from deviant waveforms, the information which waveform is affected is lost. Therefore, we preferred to analyse the unsubtracted waveforms, including Stimulus Type as a factor in the analysis. These unsubtracted waveforms are shown in Fig. 2b. Notice also that the results do not differ in either way of analysis. For the P300 to deviant stimuli (P3a), the mean amplitudes of the corresponding time window were entered into an ANOVA including the factors Stimulus Type (standard, deviant), Effect Anticipation (ST-anticipating, D-anticipating) and Site (Fz, Cz, Pz). The same was done for the P300 to target
Fig. 2 – (a and b) Grand mean ERP waveforms for target (P3b) and deviant stimuli (P3a; both waveforms are shown together with ERPs for standard stimuli) separately for ST- and D-anticipating actions, and for electrodes Fz, Cz, and Pz. (c) Difference waveforms calculated by subtracting the ERPs to standard stimuli from the ERPs to deviant stimuli. The P3a is larger for deviant stimuli following ST-anticipating actions. The ordinates indicate the stimulus-onset (which coincides with movement-onset).
78
BR A IN RE S EA RCH 1 1 83 ( 20 0 7 ) 7 4 –82
stimuli (P3b) and the unsubtracted waveforms of the MMN. Peak P300 latencies for target and deviant stimuli were analysed by two separate ANOVAs including the factors Effect Anticipation (ST-anticipating, D-anticipating) and Site (Fz, Cz, Pz). Violations of sphericity were corrected using the Huynh– Feldt ε (to facilitate reading, the uncorrected degrees of freedom are provided).
2.2.
Behavioural data
Mean keypress intervals in the oddball phase ranged from 1267 to 2245 ms across participants (mean = 1738 ms, SD = 272 ms). Keypress intervals were not different for ST-anticipating and D-anticipating keypresses (1736 ms vs. 1740 ms, respectively). T-tests showed the mean response ratios for left and right keypresses not to deviate from .5. (.505 vs. .495 for ST-anticipating and D-anticipating trials, respectively; SE = 0.8; t(9) = 1.26). Only a few errors occurred in the counting task. The mean deviation from the correct number (the total of 90 targets for the six blocks) was 3.40.
2.3.
ERPs
Fig. 2a and b show grand mean ERP waveforms. Both target (panel a) and deviant stimuli (panel b) elicited large P300s. As also illustrated in Fig. 3 (panel b), the waveform elicited by target stimuli has the typical characteristics of the P3b: a peak latency of about 440 ms and a posterior topography. (In Fig. 3, the topographic maps are collapsed across Effect Anticipation.) The corresponding ANOVA on the amplitudes yielded a significant main effect of Stimulus Type [F(1,9) = 41.43, p b 0.001] and Site [F(2,18) = 51.27, p b 0.001] qualified by a significant interaction Stimulus Type × Site [F(2,18) = 11.14, p b 0.001]. Two-
tailed t-tests confirmed the posterior topography of the observed P300: they showed the mean P3b amplitudes collapsed across Effect Anticipation to increase from frontal to parietal sites (Fz vs. Cz: p b 0.001; Fz vs. Pz: p b 0.001; Cz vs. Pz: p b 0.01). The ANOVA run on the peak latencies yielded a significant effect of Site [F(2,18) = 8.20, p b 0.01], indicating that the P3b peaks earlier at frontal than at parietal sites. Notice that the P3b was not the focus of the present study, since the manipulation of Effect Anticipation did not concern target stimuli, but only deviant and standard stimuli. We will turn to deviant stimuli now. The waveform in response to deviant stimuli shows the typical characteristics of the P3a: a central topography and a peak latency of about 310 ms post-stimulus (see Fig. 3a). Importantly, the mean P3a amplitudes to deviant stimuli were larger when the stimulus followed an ST-anticipating action than when following a D-anticipating action. Standard stimuli were not affected by whether the subjects predicted the deviant or the standard stimuli. The effect of Effect Anticipation on the P3a is illustrated best in Fig. 2c, which shows the P3a as the difference waveform calculated by subtracting the ERPs to standard stimuli from the ERPs to deviant stimuli. The corresponding ANOVA run on the amplitudes corroborates the pattern of results. The ANOVA yielded significant main effects of Stimulus Type [F(1,9) = 59.63, p b 0.001], Site [F(2,18) = 18.13, p b 0.001] and Effect Anticipation [F(1,9) = 6.55, p b 0.05]. The main effects were qualified by a significant interaction Stimulus Type × Effect Anticipation [F(1,9) = 6.87, p b 0.05]. As concerns the topography of the P3a, two-tailed t-tests showed the mean amplitudes collapsed across Effect Anticipation to be larger at central and parietal sites than at frontal sites, but not to differ between central and parietal sites (Fz vs. Cz: p b 0.01; Fz vs. Pz: p b 0.02; Cz vs. Pz: p N 0.5). As concerns the interaction Stimulus Type × Effect Anticipation, separate ANOVAs for
Fig. 3 – Topographic maps of the P3a to deviant stimuli (a) and P3b to target stimuli (b) collapsed across ST- and D-anticipating actions.
BR A IN RE S E A RCH 1 1 83 ( 20 0 7 ) 7 4 –8 2
deviant and standard ERPs showed the main effect of Effect Anticipation to be significant only for deviant ERPs but not for standard ERPs [F(1,9) = 8.1, p b 0.02 and F(1,9) b 1, p = 0.92, respectively]. The ANOVA run on the peak latencies did not yield any significant effect. Fig. 2b shows an early negative potential peaking about 120 ms after stimulus onset that was larger for deviant than for standard stimuli. Fig. 2c shows the difference waveform between deviant and standard stimuli. The early peak at about 120 ms represents the MMN. The amplitude of this potential was not affected by Effect Anticipation and had a centroparietal topography.1 The only significant effect of the corresponding ANOVA was the interaction Stimulus Type × Site [F(2,18) = 5.66, p b 0.02]. One-tailed t-tests showed the difference between standard and deviant stimuli to be significant for electrodes Cz and Pz (Fz: t = 0.29, df = 9, p = 0.4; Cz: t = 1.97, df = 9, p b 0.05; Pz: t = 3.2, df = 9, p b 0.01). Movement-related ERPs show the typical slow negative shift preceding movement onset at central sites. Running t-tests revealed that this “readiness potential” (Kornhuber and Deecke, 1965) was the same for ST-anticipating and D-anticipating actions.
3.
Discussion
The main results of the present study can be summarized as follows. Deviant non-target stimuli elicited a P300 with a more anterior topography and shorter peak latency than the P300 elicited by low-deviant target stimuli. Thus, the P300s elicited by the two types of oddball probably consist primarily of the P3a and P3b components, respectively (Comerchero and Polich, 1999). The P3a in response to deviant stimuli was larger when the stimulus was triggered by an action that, in the acquisition phase, was continuously followed by the standard stimulus than when triggered by an action that, in the acquisition phase, was continuously followed by the deviant stimulus. Moreover, the MMN peaking about 120 ms poststimulus (and also the P3b) were the same for stimuli triggered by ST- and D-anticipating actions. These findings are in accordance with our hypotheses derived from the study of Nittono (2006) mentioned in the introduction.
3.1.
P3a and P3b
As outlined above, the P3a is considered to reflect a switch of attention towards a deviant stimulus that presumably serves to make the event available to consciousness and behavioural control (Knight and Scabini, 1998; Squires et al., 1975; Woods, 1990; Friedman et al., 2001). Evidently, for a stimulus to be considered as “deviant” it must be compared to the “context” in which it occurs. Most studies investigating deviance processing presume that the context upon which attentional processes operate is provided by the neural representation of 1
The MMN is typically larger at Fz than Cz (see Kujala et al., 2007). We are not aware of whether the centro-parietal distribution observed in the present study is due to the deviant stimuli being initiated by voluntary keypresses and do not want to engage in speculations.
79
the stimulus environment. A stimulus that is not predicted given the current stimulus context elicits an orienting response. The present study extends this notion to stimulus deviance that results from the mismatch between the anticipated and the actual effect of a voluntary action. In the introduction, we suggested that the context to which external stimulation is compared is partially provided by the neural representation of the perceptual effects that the agent predicts his actions to have in the environment. An action effect that does not correspond to the agent's anticipation should be considered as deviant and, thus, elicit an orienting response. The results presented above support this hypothesis. We explain the results along the lines of the ideomotor theory of action control. This theory proposes a model of how humans select a suitable action to achieve a particular goal. Ideomotor approaches assume that performing an action establishes a link between the action's motor code and the sensory effects the action produces (“action–effect bindings”; see Elsner and Hommel, 2001, 2004; Herwig et al., in press). Herwig et al. consider this integration of action and effect to take place whenever a particular intention-based, self-selected action and a particular sensory code are frequently co-activated. These associations are bi-directional and are used to retrieve an action by internally “evoking” its effects. In other words, intentionbased actions are selected with respect to their perceptual consequences and anticipating these consequences is an integral part of the control of voluntary action (e.g., Prinz, 1990, 1997; Hommel et al., 2001; Kunde, 2001; Herwig et al., in press). During the acquisition phase of the present study, the action codes of the left/right keypresses were associated with the representations of the ensuing effect tones (standard or deviant, respectively). As a consequence, voluntarily selecting and performing left/right keypresses during the oddball phase involved the anticipation of these effects. Hence, subjects performed keypresses that involve the anticipation of either the standard or the deviant tone. From this ideomotor perspective, the explanation of the modulation of the P3a by whether the subject performed an ST- or a D-anticipating action is straightforward. If the subject's action involved the anticipation of the standard tone and if this anticipation partially defines the neural context of the current processing episode, then the occurrence of the deviant tone is all the more unexpected and, as a consequence, elicits a larger orienting response (i.e., a larger P3a). If, by contrast, the subject's action involved the anticipation of the deviant tone, the occurrence of this tone is less unexpected, making the ensuing orienting response weaker. This mechanism helps people to focus attention to unintended action effects and to initiate an appropriate correction of the current stream of action. In the current context, a recent study from Ullsperger, Nittono, and von Cramon (2007) is of great interest. Data of Ullsperger et al. suggest that the posterior medial frontal cortex (pMFC) signals the need for adjustments when the consequences of an action are worse than intended (see also Walton et al., 2004 who suggest that the pMFC is involved in monitoring the outcomes of self-generated actions). Ullsperger et al. also suggest that the pMFC plays a role in the selection of appropriate compensatory actions. Since the malfunctions they simulated were rare and unexpected action outcomes,
80
BR A IN RE S EA RCH 1 1 83 ( 20 0 7 ) 7 4 –82
Ullsperger et al. argued that the activity in the pMFC they observed may reflect a general response to an oddball event; an assumption that is corroborated by the fact that infrequent trial types in a target detection task have been demonstrated to elicit pMFC activity (Braver et al., 2001). One may thus speculate that the increased P3a observed in the present study is due to increased activity in the pMFC elicited by unpredicted action effects. However, notice that, in contrast to action-relevant target events, passively perceived oddball events (that are not triggered by the subjects' actions, as in the present study) do not seem to engage the pMFC (Liebenthal et al., 2003; Opitz et al., 2002). Hence, if the increased P3a observed in the present study is due to activity in the pMFC, then the pMFC seems to be involved in monitoring the interaction between the agent and the environment, be it that the agent reacts to external stimuli (target detection task) or that the agent produces effects in the environment (present study). As mentioned before, in principle, the factor Effect Anticipation could influence both ERPs to deviant and to standard stimuli. However, the results reported above clearly show that only the P3a elicited by the former are affected by the subject's action. This shows that the context to which the incoming input is compared is merely codetermined by the anticipated effect, i.e., the anticipated effect only modulates the stimulus context, but cannot overrule it. Since the standard stimulus is in no way deviant compared to the stimulus context, it does not yield an orienting response, regardless of whether it was anticipated by the action or not. The P3b was not the focus of the present study and is not discussed further. Notice that the actions were associated to the standard and the deviant stimulus, but not to the target. Thus, it was not to be expected that the P3b to target stimuli would be modulated by the subjects' actions. We opted for the inclusion of target stimuli in order to show that the waveform elicited by the deviant stimulus unambiguously reflects a P3a. Moreover, we wanted to be able to directly compare our results to the study of Nittono (2006) who used a three-tone oddball paradigm.
3.2.
Mismatch negativity
The MMN is the first electrophysiological indicator of the detection of a change in an otherwise invariant context of stimuli (e.g., Näätänen, 1992). It is elicited even when the subject is not attending to the stimuli (Woldorff et al., 1998; Näätänen, 1991) and, in fact, is considered to trigger a switch of attention (reflected by the P3a) if the deviant stimulus event is intrusive (Donchin and Coles, 1988; Escera et al., 2000). The results presented above clearly show that the MMN was not affected by the effect subjects anticipate. This is in line with the findings of Nittono (2006) who showed that the MMN did not differ between tones initiated by the computer and tones initiated by a voluntary keypress. We believe this to be an interesting finding, since it has been shown that the MMN does not reflect the mere detection of deviation from static stimulus properties. Rather, the MMN reflects the detection of deviation from stimulus regularities, as indicated by the fact that it is elicited by violations of an abstract rule of auditory input (e.g. pitch repetition in a sequence of regularly descending sounds [Tervaniemi et al., 1994; see also Paavilai-
nen et al., 2001]). In this case, the context is represented by the abstract rule. Thus, a priori it would not be implausible to assume that the MMN is also sensitive to violations of effect anticipation, which, after all, must involve the application of action–effect rules. That this is not the case may be taken to demonstrate that the neural representation, on which the early mechanisms screening the input for deviant events work, is not yet enhanced with information about anticipated action effects. This conclusion is in line with an experiment reported by Rinne, Antila, and Winkler (2001). These authors investigated whether the MMN is affected by the subject's prior knowledge of the sound changes. They instructed subjects to produce a sequence of keypresses by pressing one key frequently and another infrequently. In one condition, the frequently performed keypress triggered the standard tone, the other keypress the deviant tone. In another condition, each keypress triggered the next tone of a prearranged standard/deviant sequence. Rinne et al. did not observe any difference in the MMN between the two conditions. This shows, in full accordance with the results of the present study, that top–down predictive information does not affect the MMN-generating processes.
3.3.
Conclusion
In order to interact effectively with the environment we have to set up perceptual expectations of our actions' effects on the environment. This anticipation optimises the accuracy and speed of complex action sequences, where actions often depend on the outcome of the preceding action. However, the effect of a goal-directed action does not always come out as expected. As a matter of course, unpredicted action effects indicate a major perturbance of the ongoing agent–environment interaction and indicate immediate need for action correction. Thus, humans should monitor the perceptual input for action effects that do not meet their expectations. Unexpected effects should elicit an orienting response that makes the given event available to behavioural control. The present study supports this notion. Moreover, the study corroborates the inseparability of action and perception as conceptualised in the common coding theory (Prinz, 1990; Hommel et al., 2001; see also Schubotz and von Cramon, 2001). The common coding principle suggests that perception and action are represented in a common medium, and that actions are controlled in terms of their intended sensory effects, rather then in terms of a specific movement (ideomotor principle). A motor plan represents sensory events the agent intends to bring about in the environment. Thus, anticipated action effects constitute an integral part of the context to which a sensory event is compared when checked for deviance from prediction. When this prediction is not fulfilled, the orienting response is pronounced.
4.
Experimental procedures
4.1.
Methods
4.1.1.
Subjects
Ten subjects (six females, four males; mean age 24 years) participated in a single session lasting about 1.5 h. All subjects
BR A IN RE S E A RCH 1 1 83 ( 20 0 7 ) 7 4 –8 2
were naive with respect to the purpose of the experiment. None had a history of neurological disease or trauma.
4.1.2.
Stimuli
Timing of stimuli was controlled by an IBM-compatible computer, interfaced to a 22-in. IIYAMA VISION MASTER PRO 510 monitor and to two loudspeakers in front of the participants.
4.1.2.1. Acquisition phase. A small white cross presented in the centre of the screen on a black background served as fixation point. Two sinus tones were used as action effects: 1940 Hz (later used as the standard stimulus) and 500 Hz (later used as the deviant stimulus). 4.1.2.2. Three-tone oddball task. We used three different sinus tones: 1940 Hz (standard), 2000 Hz (target), and 500 Hz (deviant). The three tones correspond to those used by Nittono (2006) and by Comerchero and Polich (1999). All tones were 70 ms in duration, had a sharp on- and off-set, and were presented at about 60 dB SPL. 4.1.3.
Procedure
During the whole experiment, subjects were monitored with a camera to check whether they kept the eyes open. The experiment was divided into an acquisition phase and a subsequent oddball phase (see Fig. 1).
4.1.3.1. Acquisition phase. The acquisition phase comprised two 200-trial blocks separated by a break of approximately 20 min during which the EEG recording was prepared. Splitting the acquisition phase in two sub-blocks was a compromise between the necessities of keeping the time short between the application of the electrodes and the beginning of the data recording, on the one side, and between the acquisition phase and the oddball phase, on the other side. In each block, subjects had to press one of two response keys with the index finger of the left or the right hand. Subjects performed 200 keypresses at a pace of about once per 2 s. We made subjects use different hands instead of different fingers of the same hand, because we believe that action–effect learning is easier for motor codes concerning two different hands than for motor codes concerning two different fingers. The response keys were horizontally arranged. The distance between the keys was 13 cm. The subjects were verbally instructed to choose freely which key to press but were instructed to use the keys about equally often and in a random order (i.e., to avoid fixed sequences). Each keypress (left/right) triggered a particular effect tone (1940 Hz/500 Hz). The tone was presented immediately after the keypress (stimulus onset asynchrony [SOA] = 0 ms). The action–effect mapping was counterbalanced across subjects. Keypresses that were too fast (b1000 ms) or too slow (N3500 ms) were fed back to the participants by a visual warning message. The number of left versus right keypresses was displayed on the screen at the end of each block. 4.1.3.2. Oddball phase. Subjects performed a three-tone oddball task. Stimulus presentation was triggered by subjects' voluntary keypresses. Subjects were required to perform one of the two keypresses of the acquisition phase (left/right) at a
81
pace of about once per 2 s. Once again, they were instructed to choose freely which key to press while trying to use the keys about equally often and in a random order. No further instructions regarding the relationship between the two phases were given. Subjects were not told to consciously anticipate the forthcoming tones. Each keypress triggered one of the three stimuli with an onset asynchrony of 0 ms. The standard stimulus (1940 Hz) was presented with a probability of .75, target (2000 Hz) and deviant stimuli (500 Hz) were presented with a probability of .125 each. Subjects were required to silently count the target stimuli. The oddball phase was run in six blocks of 120 keypresses. Whereas in every block 90 standard stimuli were presented, the actual number of target and deviant stimuli varied from 14 to 16 across blocks (otherwise the counting of the targets would be redundant). After each block, subjects had to report the number of target stimuli to the experimenter.
4.1.4.
Electrophysiological recording
Continuous EEG was acquired through the ActiveTwo Biosemi™ electrode system from 64 scalp positions, digitized at 256 Hz. There was no high pass filtering during recording since the hardware is completely DC coupled. Low-pass filtering during recording was performed digitally in the ADC's decimation filter (hardware bandwidth limit), which has a 5th order sinc response with a −3 dB point at 51.2 Hz, cf. http:// www.biosemi.com/faq/adjust_filter.htm. To control for ocular artefacts, an electro-oculogram (EOG) was recorded both vertically from above and below the left eye (vEOG) and horizontally from the outer canthi of both eyes (hEOG). EEG was re-referenced off-line to linked mastoids and low-pass filtered at 60 Hz. There was no off-line high-pass filtering. Epochs containing EEG or EOG over ±75 μV were removed automatically. Muscle artefacts and segments with eye movements were removed by visual inspection.
Acknowledgments The authors wish to thank Cornelia Belger and Mandy Kosel for assistance in data acquisition, Christina Jaeger, Antje Hollaender and Clemens Maidhof for helpful comments and Henrik Grunert for building the response device. We also thank Nikolaus Steinbeis for checking and improving the English.
REFERENCES
Alho, K., 1995. Cerebral generators of mismatch negativity (MMN) and its magnetic counterpart (MMNm) elicited by sound changes. Ear Hear. 16, 38–51. Braver, T.S., Barch, D.M., Gray, J.R., Molfese, D.L., Snyder, A., 2001. Anterior cingulate cortex and response conflict: effects of frequency, inhibition and errors. Cereb. Cortex 11, 825–836. Comerchero, M., Polich, J., 1999. P3a and P3b from typical auditory and visual stimuli. Clin. Neurophysiol. 110, 24–30. Courchesne, E., Hillyard, S.A., Galambos, R., 1975. Stimulus novelty, task relevance and the visual evoked potential in man. Electroencephalogr. Clin. Neurophysiol. 39, 131–143. Donchin, E., Coles, M.G.H., 1988. Is the P300 component a manifestation of context updating? Behav. Brain Sci. 11, 357–374.
82
BR A IN RE S EA RCH 1 1 83 ( 20 0 7 ) 7 4 –82
Elsner, B., Hommel, B., 2001. Effect anticipation and action control. J. Exp. Psychol. Hum. Percept. Perform 27, 229–240. Elsner, B., Hommel, B., 2004. Contiguity and contingency in action–effect learning. Psychol. Res. 68, 138–154. Escera, C., Alho, K., Schröger, E., Winkler, I., 2000. Involuntary attention and distractibility as evaluated with event-related brain potentials. Audiol. Neuro-Otol. 5, 151–166. Friedman, D., Cycowicz, Y.M., Gaeta, H., 2001. The novelty P3: an event-related brain potential (ERP) sign of the brain's evaluation of novelty. Neurosci. Biobehav. Rev. 25, 355–373. Giard, M.H., Perrin, F., Pernier, J., Bouchet, P., 1990. Brain generators implicated in the processing of auditory stimulus deviance: a topographic event-related potential study. Psychophysiology 27, 627–640. Herwig, A., Prinz, W., Waszak, F., in press. Two modes of sensorimotor integration in intention-based and stimulus-based action. Q. J. Exp. Psychol. Hommel, B., Müsseler, J., Aschersleben, G., Prinz, W., 2001. The theory of event coding (TEC): a framework for perception and action planning. Behav. Brain Sci. 24, 849–878. Katayama, J., Polich, J., 1998. Stimulus context determines P3a and P3b. Psychophysiology 35 (1), 23–33. Knight, R., 1996. Contribution of human hippocampal region to novelty detection. Nature 383, 256–259. Knight, R.T., Scabini, D., 1998. Anatomic bases of event-related potentials and their relationship to novelty detection in humans. J. Clin. Neurophysiol. 15, 3–13. Kornhuber, H.H., Deecke, L., 1965. Hirnpotentialänderungen bei Willkürbewegungen und passiven Bewegungen des Menschen: Bereitschaftspotential und reafferente Potentiale. Pflugers Arch. Gesamte Physiol. Menschen Tiere 284, 1–17. Kujala, T., Tervaniemi, M., Schröger, E., 2007. The mismatch negativity in cognitive and clinical neuroscience: theoretical and methodological considerations. Biol. Psychol. 74, 1–19. Kunde, W., 2001. Response–effect compatibility in manual choice reaction tasks. J. Exp. Psychol. Hum. Percept. Perform 27, 387–394. Liebenthal, E., Ellingson, M.L., Spanaki, M.V., Prieto, T.E., Ropella, K.M., Binder, J.R., 2003. Simultaneous ERP and fMRI of the auditory cortex in a passive oddball paradigm. NeuroImage 19, 1395–1404. Näätänen, R., 1990. The role of attention in auditory information processing as revealed by event-related potentials and other measures of cognitive function. Behav. Brain Sci. 13, 201–232. Näätänen, R., 1991. Mismatch negativity (MMN) outside strong attentional focus: a commentary on Woldorff et al. Psychophysiology 28, 478–484. Näätänen, R., 1992. Attention and Brain Function. Erlbaum, Hillsdale, NJ. Nittono, H., 2006. Voluntary stimulus production enhances deviance processing in the brain. Int. J. Psychophysiol. 59, 15–21. Nittono, H., Ullsperger, P., 2000. Event-related potentials in a self-paced novelty oddball task. NeuroReport 11, 1861–1864.
Nittono, H., Hamada, A., Hori, T., 2003. Brain potentials after clicking a mouse: a new psychophysiological approach to human–computer interaction. Hum. Factors 45, 591–599. Opitz, B., Rinne, T., Mecklinger, A., von Cramon, D.Y., Schroger, E., 2002. Differential contribution of frontal and temporal cortices to auditory change detection: fMRI and ERP results. NeuroImage 15, 167–174. Paavilainen, P., Simola, J., Jaramillo, M., Näätänen, R., Winkler, I., 2001. Preattentive extraction of abstract feature conjunctions from auditory stimulation as reflected by the mismatch negativity (MMN). Psychophysiology 38, 359–365. Polich, J., 2003. Theoretical overview of P3a and P3b. In: Polich, J. (Ed.), Detection of Change: Event-Related Potential and fMRI Findings. Kluwer Academic, Boston, pp. 83–98. Prinz, W., 1990. A common coding approach to perception and action. In: Neumann, O., Prinz, W. (Eds.), Relationships between Perception and Action: Current Approaches. Springer, Berlin, pp. 167–201. Prinz, W., 1997. Perception and action planning. Eur. J. Cogn. Psychol. 9, 129–154. Posner, M., Petersen, S., 1990. The attention system of the human brain. Annu. Rev. Neurosci. 13, 25–42. Rinne, T., Antila, S., Winkler, I., 2001. Mismatch negativity is unaffected by top-down predictive information. NeuroReport 12 (10), 2209–2213. Schubotz, R.I., von Cramon, D.Y., 2001. Functional organization of the lateral premotor cortex: fMRI reveals different regions activated by anticipation of object properties, location and speed. Cogn. Brain Res. 11, 97–112. Sokolov, E.N., 1963. Perception and the Conditioned Reflex. Pergamon Press, Oxford, UK. Squires, K.C., Squires, N.K., Hillyard, S.A., 1975. Decision-related cortical potentials during an auditory signal detection task with cued observation intervals. J. Exp. Psychol. Hum. Percept. Perform 1, 268–279. Tervaniemi, M., Maury, S., Näätänen, R., 1994. Neural representations of abstract stimulus features in the human brain as reflected by the mismatch negativity. NeuroReport 5, 844–846. Ullsperger, M., Nittono, H., von Cramon, D.Y., 2007. When goals are missed: dealing with self-generated and externally induced failure. NeuroImage 35, 1356–1364. Walton, M.E., Devlin, J.T., Rushworth, M.F., 2004. Interactions between decision making and performance monitoring within prefrontal cortex. Nat. Neurosci. 7, 1259–1265. Woldorff, M., Hillyard, S.A., Gallen, C.C., Hampson, S.R., Bloom, F.E., 1998. Magnetoencephalographic recordings demonstrate attentional modulation of mismatch-related neural activity in human auditory cortex. Psychophysiology 35, 283–292. Woods, D.L., 1990. The physiological basis of selective attention: implications of event-related potential studies. In: Rohrbaugh, J.W., Parasuraman, R., Johnson Jr., R. (Eds.), Event-Related Potentials: Basic Issues and Applications. Oxford University Press, New York, pp. 178–209.