- Email: [email protected]
ERP evidence of early cross-modal links between auditory selective attention and visuo-spatial memory
Brain and Cognition 74 (2010) 273–280
Contents lists available at ScienceDirect
Brain and Cognition journal homepage: www.elsevier.com/locate/b&c
ERP evidence of early cross-modal links between auditory selective attention and visuo-spatial memory Marie D. Bomba a,b, Anthony Singhal c,⇑ a
Neurosciences and Mental Health Program, The Hospital for Sick Children, Canada Department of Kinesiology & Health Science, York University, Toronto, ON, Canada c Department of Psychology and Centre for Neuroscience, University of Alberta, Edmonton AB, Canada b
a r t i c l e
i n f o
Article history: Accepted 20 August 2010 Available online 14 October 2010 Keywords: ERP Attention Memory Nde Ndl P300 Cross-modal Spatial Dual-task
a b s t r a c t Previous dual-task research pairing complex visual tasks involving non-spatial cognitive processes during dichotic listening have shown effects on the late component (Ndl) of the negative difference selective attention waveform but no effects on the early (Nde) response suggesting that the Ndl, but not the Nde, is affected by non-spatial processing in a dual-task. Thus to further explore the nature of this dissociation and whether the Nd waveform is affected by spatial processing; fourteen adult participants performed auditory dichotic listening in conjunction with visuo-spatial memory in a cross-modal dual-task paradigm. The results showed that the visuo-spatial memory task decreased both the Nde and Ndl waveforms, and also attenuated P300 and increased its latency. This pattern of results suggests that: (1) the Nde reflects a memory trace that is shared with vision when the information is spatial in nature, and (2) P300 latency appears to be influenced by the discriminability of stimuli underlying the Nde and Ndl memory trace. Ó 2010 Elsevier Inc. All rights reserved.
1. Introduction An important question to answer in cognitive neuroscience involves elucidating the nature of the functional relationship between attention and working memory. There is a general agreement that these processes reflect complementary cognitive systems that often operate together to facilitate certain complex behaviour (Cabeza et al., 2003; de Fockert, Rees, Frith, & Lavie, 2001; Engle, 2002; Kane & Engle, 2003; LaBar, Gitelman, Parrish, & Mesulam, 1999). However, the specific interactions between these systems is not fully understood particularly when information is encountered in more than one sensory modality. Since it is known that there are strong links between audition and vision (Driver & Spence, 1998; Eimer & Schroger, 1998), the focus of the present experiment was to further explore the relationship between attention and working memory with event-related potentials (ERPs) in a cross-modal dual-task paradigm that combined auditory selective attention, and visuo-spatial working memory. When ERPs are recorded during a classic version of the dichotic listening paradigm (Hillyard, Hink, Schwent, & Picton 1973), and the ERP from the unattended stimuli is subtracted from the attended ERP, an endogenous negativity is observed that has two parts: an early negative difference (Nde), and a late negative differ⇑ Corresponding author. E-mail address: [email protected] (A. Singhal). 0278-2626/$ - see front matter Ó 2010 Elsevier Inc. All rights reserved. doi:10.1016/j.bandc.2010.08.007
ence (Ndl). These two components are thought to reflect overlapping, but distinct processes associated with auditory selective attention. Nde appears maximal at frontocentral electrodes, and typically peaks between 70 ms and 200 ms after stimulus onset. Nde has been shown to reflect the processing of simple stimulus features such as pitch (Alho, 1992; Woods, 1990), and evidence suggests that it is generated by a neural source within the auditory cortex (Giard, Perrin, Pernier, & Peronnet, 1988; Rif, Hari, Hamalainen, & Sams, 1991; Woldorff et al., 1993). It has been proposed that Nde reflects a temporary feature recognition system that consists of a series of overlapping memory traces that function to determine the suitability of the incoming stimulus for further processing (Näätänen, 1992). According to this theory, the simple physical characteristics of the stimuli are compared to a representation held in memory that contains the relevant stimulus features. This matching process terminates when the current stimulus is found to differ from the template. Therefore, the closer the feature-match is between the incoming stimulus and the template; the longer the process takes, which leads to a larger amplitude and longer latency of Nde. It has been argued that this comparison process is reflected by a processing negativity (PN), and the Nd represents the manifestation of the difference between the two PNs that reflect relevant stimuli and irrelevant stimuli respectively (Näätänen, 1992). However, it has been suggested that the selection of stimuli for further processing may not entirely depend upon physical characteristics. For instance it has been proposed
274
M.D. Bomba, A. Singhal / Brain and Cognition 74 (2010) 273–280
that Nde is sensitive to perceptual context, and that selective processing of the stimuli occurs at different perceptual levels (Alain, Achim, & Richer, 1993; Alain & Arnott, 2000; Alain & Woods, 1994; Arnott & Alain, 2002). The nature of the information processing mechanisms underlying the frontally distributed Ndl is not as well understood as that of Nde. It has been shown that Ndl peaks in the range 300–600 ms after stimulus onset, and Näätänen, (1992) has suggested that it involves frontal lobe generators. The Ndl is reduced by practice (Shelley et al., 1991; Woods, 1990) and it has been shown to reflect the analysis of multidimensional stimuli (e.g., Woods & Alain, 2001; Woods, Alho, & Algazi, 1994). Teder-Salejarvi and Hillyard (1998) reported that Ndl amplitude was largest in response to stimulus locations adjacent to an attended location in a horizontal array. This indicates that Ndl may reflect the extended processing of stimuli in order to verify their relative spatial locations. Moreover, Ndl is more sensitive than Nde to the magnitude of the physical difference between the attended and unattended stimuli (Hansen & Hillyard, 1980), and it is larger at longer inter-stimulus intervals (Hansen & Hillyard 1984; Näätänen, Gaillard, & Varey 1981). Based on this evidence, Näätänen (1990) suggested that Ndl may represent the ‘‘further processing” of the incoming stimulus, as well as selective rehearsal of the to-be-attended stimulus; which he termed ‘‘the attentional trace”. More recently, the roles of Nde and Ndl during auditory selective attention have been investigated with a series of experiments that employed a dual-task interference paradigm in which a primary visual task was used to probe the Nd components elicited during a secondary dichotic listening task. The dual-task paradigm is a useful tool in attention research because performance costs associated with the introduction of perceptual-cognitive load can be attributed to limitations within the attention system. These limitations have been suggested to reveal either central processing bottlenecks (Pashler, 1984), capacity sharing limitations, or crosstalk interference between specific cognitive processes (Pashler & Johnson, 1998). ERP studies that make use of dual-task paradigms generally rely on the assumption that changes in the latency and amplitude of ERP waveforms under load, compared to single task conditions; reflect the interference effects in the neural processing underlying performance deficits (Gopher & Donchin, 1986). Relying on this logic, the first of these dual-task studies employed a simulated aircraft flying task and showed that, compared to the dichotic task performed alone, the amplitude of Ndl, but not the Nde, was reduced both by the introduction of the flying task, and by an increase in its difficulty (Singhal, Doerfling, & Fowler, 2002). A second finding of interest was that P300 was reduced in concert with the Ndl. The dissociation between the Nde and Ndl was interpreted as indicating that the latter reflects amodal processing rather than unimodal auditory processing because the interference with the auditory Ndl was caused by a visual task. Furthermore, it was suggested that the relationship between Ndl and P300 may be closer than had been previously thought (Singhal et al. 2002). However, due to the complexity of the flying task; the question of which of the various cognitive demands of the flying task was responsible for the reduction of Ndl was left unanswered. One solution to address this question was to employ simpler visual tasks containing fewer components, thereby allowing more specific inferences to be drawn about the locus and nature of the interference. Accordingly, Singhal and Fowler (2004), Singhal and Fowler (2005) employed both the varied-set (short-term memory, STM) and fixed set (long-term memory, LTM) versions of Sternberg’s (1975) visual memory scanning tasks to investigate the cross-modal relationship between Ndl and memory processes. This study showed that Ndl amplitude was reduced by the varied-but not the fixed-set version of the scanning task whereas P300 was sensitive to both, while Nde was unaffected by either version. This
pattern of results suggested that Ndl reflects short-term memory processes, but not long-term memory processes, and supported the idea that Nde is a modality specific waveform. Moreover, the suggestion was made that within Baddeley’s (1986, 2000) view of working memory (WM); P300 may reflect a general aspect of WM perhaps associated with task difficulty (Kok, 2001), whereas the Ndl may reflect specific processes within the phonological loop (Singhal & Fowler, 2004). Additional support for this position was provided by Ramirez, Bomba, Singhal, and Fowler (2005), who investigated the effects of a modified visual covert attention switching paradigm (e.g. Posner, Nissen, & Ogden, 1978) on the Nd components. This study showed that the valid/invalid condition of Posner’s task compared to a neutral control condition, decreased Ndl, but did not affect Nde. Furthermore, P300 dissociated from the Ndl because its amplitude was decreased by both the neutral control and the valid/invalid conditions. It was concluded that Ndl is likely sensitive to attention switching and both Ndl and P300 share sensitivity to an amodal system that is not reflected by Nde. However, since the auditory dichotic listening task to generate the Nd waves employed by Singhal, Fowler and colleagues was primarily a spatial task, the question of whether Ndl is sensitive to spatial attentional control, or more general control mechanisms was not answered. Accordingly, Meehan, Singhal, and Fowler (2005) employed a modified version of the Posner task, comparing interference caused by a spatial attention switching task with interference caused by a letter matching task (Posner & Mitchell, 1967) so that successive lower- and upper-case letters of the alphabet appeared at the same spatial location, with the former serving as a cue for the latter. The results showed that Ndl was sensitive to interference from the spatial but not the letter matching task suggesting that Ndl reflects spatial attentional control, and not deeper non-spatial attentional control mechanisms. Once again, Nde remained unaffected by the experimental manipulations, and when considered with the other previously mentioned dual-task studies; suggests that this component reflects processes that are not compromised by cross-modal dual-task load. To summarize the dual-task work of Singhal, Fowler and colleagues, Ndl was found to be sensitive to tasks that involve working memory and spatial attention control processes. The pattern of Ndl data dissociated from P300, and importantly for the present study; Nde was found to be insensitive to all of the visual task manipulations. Thus, these studies strongly support Näätänen’s (1990) speculation that Ndl involves processing associated with an attentional trace that shares processing resources with other forms of working memory processes. However, these studies also show insensitivity of Nde to the load imposed by the same cognitive tasks, which is not in agreement with Näätänen’s (1992) theory that the Nde reflects a self-terminating matching process between incoming stimuli and the representations of to-be-attended stimulus feature retained in a memory buffer. However, one other possibility is that Nde does reflect a working memory trace – a spatial working memory trace. None of the aforementioned dual-task studies employed spatial memory in the competing tasks, yet the auditory dichotic listening task employed in these studies clearly required a spatial strategy. Thus, we reasoned that it would be important to probe the spatial Nd waves with a memory task involving spatial information. It has been shown in the literature that an effective way to engage spatial working memory processes is with variations of the visuo-spatial n-back task. An important fact to consider with the n-back task is that it is considered to be more complex than the delayed-response task because the memory demands of the n-back task can be parametrically manipulated while keeping the other task factors relatively constant (McEvoy, Smith, & Gevins, 1998). Indeed, in a large scale meta-analysis study of the n-back task in functional neuroimaging studies, Owen, McMillan, Laird, and Bullmore (2005) showed that
M.D. Bomba, A. Singhal / Brain and Cognition 74 (2010) 273–280
there was a high degree of consistency among the working memory networks in frontal and parietal cortex that were activated by various n-back tasks. Furthermore, the spatial variants of the n-back tasks activated unique networks in right dorsolateral prefrontal, lateral pre-motor, and posterior parietal regions compared to non-spatial n-back tasks (Owen et al., 2005), which encompass a neural network strongly considered to be involved in spatial working memory processes (Mesulam, 2000; Rowe, Toni, Josephs, Frackowiak, & Passingham, 2000). Thus, our purpose in this study was to examine whether a functional relationship exists between spatial working memory processes and the auditory Nde. To this end we employed a visuo-spatial n-back task performed in conjunction with dichotic listening in a cross-modal dual-task paradigm. Finally, since a potential benefit of the dual-task paradigm is the ability to differentiate between effects associated with the introduction of a second task from those associated with an increase in its difficulty; we sought to manipulate difficulty of the n-back task in order to examine whether the processes underlying the Nde reflect spatial working memory demands within a single modality (task difficulty), as well as across modalities (introduction of dual-task). 2. Methods 2.1. Participants Fourteen paid volunteers (eight female); age range 18–31 years participated in this experiment. All reported normal hearing with normal or corrected-to-normal vision. Ethics approval was obtained, and the participants gave their informed consent.
275
2.4. Recording and quantification of EEG Silver–silver chloride (Ag–AgCl) electrodes were attached at Fz, F3, F4, Cz, C3, C4, Pz, T3, and T4, referenced to linked ears, and grounded to the forehead with an impedance of 5 k X or less. Electro-oculographic (EOG) electrodes were attached to the outer canthus and supra-orbital region of the right eye. The signals were amplified (GRASS Neurodata Acquisition System 12B) with a bandpass of 0.1–30 Hz ( 6 dB, 9 dB/octave rolloff). The EEG and EOG were digitized every 5 ms for a 1500 ms epoch, which included a 100 ms pre-stimulus baseline. The EEG data were corrected for saturation and eye movements (Gratton, Coles, & Donchin, 1983). Peak picking software (CED Spike 2) was used to measure the components defined by the following: P300 was measured from the attended deviant tones and defined as the most positive peak in the range 270–600 ms. The difference waves were derived as follows: Nde = attended standards waveforms – unattended standards waveforms, and was defined as the most negative peak in the range 70–200 ms; Ndl = attended standards waveforms – unattended standards waveforms, and defined as the most negative peak in the range 300–500 ms. We calculated mean amplitude values for all ERP components by averaging the peak value with the value 5 ms prior to the peak and the value 5 ms post peak. Following visual inspection of the waveforms, we employed paired t-tests on normalized data (McCarthy & Wood, 1985) to determine at which midline electrodes the respective averaged ERP peaks were largest in the baseline auditory condition. We then performed statistical analyses on the mean amplitude data at those latencies.
2.2. Visuo-spatial task The participants fixated on the centre of a white cross presented in the centre of a 55.88 cm computer monitor. The arms of the cross subtended a visual angle of 3.4°. There were two conditions: Lag 1 and Lag 2. In the first condition, the cue for task initiation was the appearance of an open arrowhead at the end of one arm of the cross. After 3 s the cue disappeared and a solid arrowhead appeared simultaneously and randomly at one of the other three arm positions. The task was to respond to the position of the initial open arrow with the index finger on the dominant hand using a modified joystick. The response triggered the disappearance of the solid arrowhead and the simultaneous and random appearance of another solid arrowhead at one of the three other locations, which required a response to the second arrow – and so on. In the Lag 2 condition, two open arrowheads appeared in succession, followed by a closed arrowhead, which required a response to the spatial location of the first open arrowhead. We recorded reaction time (RT) and errors from this task. 2.3. Dichotic listening task The auditory task consisted of 12 blocks of 250 non-overlapping tones (50 ms duration, 10 ms ramp, 60 dB SPL) presented through stereophonic earphones (KOSS 4 A). The frequent standards (1000 Hz, 80% probability) and the infrequent deviants (1500 Hz, 20% probability) were presented randomly as a rectangular distribution with a mean inter-tone interval (offset to onset) of 700 ms and a range of 500–900 ms to either the left or right ear for a total of 125 tones/ear/block. The participants were required to pay attention to a designated ear and report the infrequent deviants with a button press with their non-dominant hand, RT, errors, and EEG activity were recorded from this task. Errors included misses and false alarms, but the latter were so rare that the two categories were pooled for the purposes of analysis.
2.5. Design and procedure We employed a repeated measures design. EEG activity from the dichotic listening task was collected in three conditions: one control and two experimental. In the control condition the dichotic listening task was performed alone (Aud-control). No visual stimuli were presented during this condition other than a white fixation cross at the centre of the monitor. In the two experimental conditions, the visuo-spatial task was performed in conjunction with the auditory task (Aud + Lag 1, and Aud + Lag 2). In two additional behavioural control conditions, the visuo-spatial task was performed alone (Lag 1 and Lag 2). We employed a subsidiary dualtask design where the participants were instructed to treat the visual task as primary and the auditory task as secondary. The five conditions were presented in counterbalanced order between sessions determined by Latin square. The ear of attention was also counterbalanced. The duration of each block of auditory tones was approximately 2.9 min with a short pause in between each block. In the dual conditions, the presentation of the visuo-spatial task was synchronized with the presentation of the auditory stimulus blocks. The timing was maintained when the visual task was presented alone. Thus, there was a total of approximately 1300 responses/condition. The two tasks were run on separate computers, and the timing of stimulus presentation between the two tasks was uncorrelated; thus the precise interleaving of the stimuli across modalities was not recorded. Prior to the experiment, the participants were trained on both tasks until error rates were below 15%. In the dual-task conditions, the participants were instructed to treat the visual task as primary and the auditory task as secondary. The data were analyzed using one-way, or two-way repeated measures ANOVAs with Greenhouse Geisser epsilon corrections, followed by contrasts corrected with the modified Bonferroni procedure (Keppel, 1991). Only the data collected from Fz, Cz, and Pz are reported in this paper.
276
M.D. Bomba, A. Singhal / Brain and Cognition 74 (2010) 273–280
Table 1 Mean correct reaction time (RT; ms) and accuracy (% correct) for each task and condition. Standard errors are shown in parentheses. Measure
Task alone controls Auditory task (Aud) Visual task (Lag 1) Visual task (Lag 2) Dual experimental conditions Auditory Auditory + visual Lag 1 Auditory + visual Lag 2 Visual Visual Lag 1 + auditory Visual Lag 2 + auditory
RT
Accuracy
325 (9) 230 (27) 225 (29)
98.3 (0.3) 98.7 (0.3) 95.7 (0.6)
466 (18) 509 (20)
92.7 (1.4) 86.6 (2.4)
296 (22) 348 (27)
97.0 (0.6) 90.7 (1.1)
increased in the dual condition compared to the alone condition and the increase was larger for Lag 2 compared to Lag 1. The analysis of the accuracy data revealed a similar pattern [F (1, 13) = 14.05, P < 0.002]. 3.3. Auditory task A one-way repeated measures ANOVA on RT revealed significant slowing [F (2, 26) = 153.04, T = 0.0570, P < 0.00001]. The slowing was observed from the Aud control condition to the Aud + Lag 1 condition (P < 0.01), and further slowing from Aud + Lag 1 to Aud + Lag 2 (P < 0.01). The same pattern was observed in the decrease in accuracy [F (2, 26) = 18.82, T = 0.649, P < 0.0002; both contrasts, P < 0.01]. 3.4. ERP measures
3. Results 3.1. Behavioural measures Table 1 contains the mean correct RT and accuracy data for each condition.
Fig. 1 shows the grand average waveforms elicited by the attended and unattended standard tones during dichotic listening at Fz and Cz. Fig. 2 shows the Nde and Ndl components derived from the original waveforms. 3.5. Nde and Ndl
3.2. Visual task A 2 (alone/dual) 2 (Lag 1/Lag 2) ANOVA on the RT data revealed an interaction [F (1, 13) = 9.25, P < 0.002], showing that RT
The mean peak amplitudes for Nde and Ndl at Fz and Cz are shown in Fig. 3. Both components were largest at Cz (Nde, t (13) = 4.99, P < 0.0002; Ndl, t (13) = 2.18, P < 0.04). A one-way re-
Fig. 1. Grand average waveforms for the auditory attended and unattended standards at Fz and Cz for the auditory control (auditory), dual Lag 1 (Aud cont + Lag 1), dual Lag 2 (Aud cont + Lag 2) conditions. The arrow in the upper left panel indicates the time of stimulus onset.
M.D. Bomba, A. Singhal / Brain and Cognition 74 (2010) 273–280
277
Fig. 2. Grand average auditory Nd (negative difference) waveforms at Fz and Cz for the auditory control (auditory), dual Lag 1 (Aud cont + Lag 1), dual Lag 2 (Aud cont + Lag 2) conditions. The Nd early (Nde) and Nd late (Ndl) are indicated, and the arrow in the upper panel indicates the time of stimulus onset.
P300 amplitude was largest at Pz (t (13) = 5.92, P < 0.0004); mean amplitudes: Aud = 11.59 lv; Aud + Lag 1 = 9.33 lv; Aud + Lag 2 = 8.89 lv. A one-way repeated measures ANOVA on P300 amplitude revealed a decrease at the Pz electrode, which was due to the difference between the Aud-control and the Aud + Lag 2 condition [F (2, 26) = 4.32, E = 0.598, P < 0.04; contrast, P < 0.03]. In the case of P300 latency, a one-way repeated measures ANOVA revealed an increase in latency as a cost of concurrence (Aud-control versus Aud + Lag 1) at Pz. [P < 0.01; Pz: F (2, 26) = 5.3, E = 0.747, P < 0.02; contrast, P < 0.01]. 4. Discussion 4.1. Dual-task assumptions
Fig. 3. Mean auditory negative difference (Nd) amplitude (microvolts ± SEM) at the Cz electrode for the auditory control (auditory), dual Lag 1 (Aud cont + Lag 1), dual Lag 2 (Aud cont + Lag 2) conditions.
peated measures ANOVA on Nde amplitude at Cz revealed a cost of concurrence (Aud-control versus Aud + Lag 1) [F (2, 26) = 9.22, E = 0.633, P < 0.001]; contrast, P < 0.02]. The Ndl analysis at Cz also revealed a cost of concurrence [F (2, 26) = 10.87, E = 0.972, P < 0.0004]; contrast, P < 0.002. 3.6. P300 Fig. 4 shows the grand average waveforms elicited by the attended deviant tones at Fz, Cz, and Pz. It was revealed that
The assumption that the visual task would impose a processing load on the auditory task, and vice versa, was confirmed by the behavioural data. The introduction of the visual task increased the time to detect targets as well as the number of target errors committed. Furthermore, the time to detect targets increased as the level of difficulty of the visual task increased from Lag 1 to Lag 2. On the other hand, the introduction of the auditory task degraded both speed and accuracy on the visual task. 4.2. Nde and Ndl The Nd results from this experiment are particularly interesting on two fronts. First, we present a novel finding where auditory Nde is reduced by a dual-task consisting of visually presented information involving spatial memory. Second, we
278
M.D. Bomba, A. Singhal / Brain and Cognition 74 (2010) 273–280
Fig. 4. Grand average waveforms for the auditory attended targets at Fz, Cz and Pz for the auditory control (auditory), dual Lag 1 (Aud cont + Lag 1), dual Lag 2 (Aud cont + Lag 2) conditions. The arrow in the top panel indicates the time of stimulus onset.
show clear-cut interference of the visuo-spatial task on Ndl. On the face of it, these data are in agreement with the idea that Nd (both Nde and Ndl) reflect a series of memory traces (Näätänen, 1992; Woods & Alain, 2001). It has been previously argued that the auditory attention system underlying the Nd waves could be separated into two distinct parts: one that involves modality specific operations (Nde), and one that is higher order in nature and is sensitive to amodal processes such as working memory (Meehan et al., 2005; Singhal & Fowler, 2004, 2005; Singhal et al., 2002). Taken together, the two Nd findings presented here suggest that this auditory attention system is more sensitive to amodal processes than has been previously considered. Our data suggest that the bottleneck of interference in our paradigm between auditory selective attention and visuo-spatial memory may occur earlier than has previously been speculated. Furthermore, the locus of interference may be dependent upon the type
of information that is under analysis. In other words, in contrast to our earlier work, we observed an early locus of interference in the system (70–150 ms). This is a novel finding in the dual-task literature and one possibility is that it is because the auditory attention and visual memory systems share basic resources for spatial information that do not have to be mutually recruited in situations involving cross-modal tasks that combine spatial and non-spatial information. Thus, while Ndl reflects higher-order mechanisms of attentional control and working memory that can operate outside of a particular modality; Nde is more specific in terms of its sensitivity to visual information. This line of argument is in agreement with studies that show that auditory and visual spatial integration can occur in primary sensory cortex (Meienbrock, Naumer, Doehrmann, Singer, & Muckli, 2007; Molholm et al., 2002). Indeed, the type of interference that we observed could be brought about by two possible convergent
M.D. Bomba, A. Singhal / Brain and Cognition 74 (2010) 273–280
pathways between auditory and visual cortex: (1) feed-forward visual input may reach multi-sensory regions such as along the superior temporal sulcus (STS) and then be transmitted via feedback loops to early auditory cortex (Kumar, Stephan, Warren, Friston, & Griffiths, 2007; Smiley et al., 2007); or (2) a direct feedforward pathway from early visual areas to primary auditory cortex (Clavagnier, Falchnier, & Kennedy, 2004; Falchier, Clavagnier, Barone, & Kennedy, 2003; Rockland & Ojima, 2003). One considerable limitation of our study in addressing the question of cortical pathway interactions is that we did not collect ERPs from the visual task as was done previously (Singhal & Fowler, 2004; Singhal & Fowler, 2005), and thus we cannot examine the pattern of interference that the auditory task had on visual task ERPs. Furthermore, in our study the precise timing of the cross-modal trial interleaving is not known, and we did not vary the level of difficulty of the auditory task, which might have imposed additional effects on the processing limitations within the auditory (Muller-Gass & Schröger, 2007) and visual systems (Norman & Bobrow, 1975). Finally, we cannot rule out the possibility that the pattern of interference observed on Nde is due to supramodal interference that involves spatial attentional control mechanisms (Eimer & Van Velzen, 2002). While, the studies conducted by Ramirez et al. (2005) and Meehan et al. (2005) did not find interference effects between visuo-spatial attentional control and auditory Nde, an important difference in their tasks was that they did not involve explicit memory encoding, storage, and retrieval processes. It may be the case that the attentional trace underlying Nde consists of a strong memory component that is not tapped by a visual task unless that task involves strong memory operations of spatial information.
4.3. Nd and P300 An additional question in this experiment was whether P300 and Ndl would be influenced in a parallel manner by the introduction of the visual task as has been previously shown (Singhal et al., 2002), or whether they would dissociate from one another (Ramirez et al., 2005; Singhal & Fowler, 2004). The visuo-spatial task in the present experiment did not have a major impact upon P300 component amplitude. Specifically, there was a decrease in P300 with the introduction of the visual task at the Pz electrode but no further effect as the task increased in difficulty from Lag 1 to Lag 2. This finding is similar in nature to the Ndl results and is in agreement with Singhal et al. (2002), who showed a parallel amplitude effect between these two components. Still, it is important to note that P300 latency was also affected in this experiment. A clear-cut slowing of P300 was observed as a cost of concurrence at Fz and Pz, and a strong trend in this direction at Cz. The slowing of P300 as a result of the introduction of a second task has reliably been shown before (Fowler, 1994), but has not been previously described in relation to Nd effects. Here we have observed P300 slowing in conjunction with a decrease in Nde and Ndl amplitudes. It has been argued that P300 latency is sensitive to the discriminability of the attended stimulus (Leuthold & Sommer, 1998; Magliero, Bashore, Coles, & Donchin, 1984), and it is likely that the introduction of the visuo-spatial task influenced the discriminability of the stimuli underlying the memory trace reflected in Nd amplitude. Thus the finding of a decreased Nde and Ndl may not be entirely isolated from the observed P300 latency effects. Along this line of argument, it may be the case that Nde amplitude relies on low level processes closely related to processes involved in the generation of the P300 response. If these processes are delayed due to discriminability effects, the result may be a conjunction of a slowed P300 and an attenuated Nde. A close relationship between Nde and P300 has been speculated before
279
(Näätänen, 1990), and here we show additional evidence for this link. Finally, it should be noted that we did not observe parallel effects between the ERP and the behavioural data in this study. The ERP effects were associated with the introduction of the visuo-spatial n-back task, but not by an increase in its difficulty. Whereas, we observed auditory RT and accuracy effects associated with both the introduction of the visual task as well as an increase in its difficulty. This likely suggests that the auditory ERP in our study reflect some of the processes associated with cross-modal spatial working memory load, but not those associated with changes in task difficulty within the visual modality. In summary, we have presented ERP evidence of early cross-modal interactions between the auditory selective attention system and the visual spatial memory systems. Further research is needed to examine whether these ERP patterns are reproducible between non-spatial selective attention and working memory.
Acknowledgment The authors thank Graeme Armstrong at the University of Alberta for his assistance with the ERP data analyses.
References Alain, C., Achim, A., & Richer, F. (1993). Perceptual context and the selective attention effect on auditory event-related brain potentials. Psychophysiology, 30, 572–580. Alain, C., & Arnott, R. (2000). Selectively attending to auditory objects. Frontiers in Biosciences, 1, 202–212. Alain, C., & Woods, D. (1994). Signal clustering modulates auditory cortical activity in humans. Perception & Psychophysics, 56, 501–516. Alho, K. (1992). Selective attention in auditory processing as reflected by eventrelated brain potentials. Psychophysiology, 29, 247–263. Arnott, S., & Alain, C. (2002). Effects of perceptual context on event-related brain potentials during auditory spatial attention. Psychophysiology, 39, 625–632. Baddeley, A. (1986). Working memory. Oxford: Clarendon Press. Baddeley, A. (2000). The episodic buffer: A new concept of working memory. Trends in Cognitive Sciences, 4, 417–423. Cabeza, R., Dolcos, F., Prince, S. E., Rice, H. J., Weissman, D. H., & Nyberg, L. (2003). Attention-related activity during episodic memory retrieval: A cross-function fMRI study. Neuropsychologia, 41, 390–399. Clavagnier, S., Falchnier, A., & Kennedy, H. (2004). Long-distance feedback projections to area V1: Implications for multisensory integration, spatial awareness, and visual consciousness. Cognitive, Affective, & Behavioral Neuroscience, 4, 117–126. de Fockert, J., Rees, G., Frith, C., & Lavie, N. (2001). The role of working memory in visual selective attention. Science, 291, 1803–1806. Driver, J., & Spence, C. (1998). Crossmodal attention. Current Opinion in Neurobiology, 8, 245–253. Eimer, M., & Schroger, E. (1998). ERP effects of intermodal attention and crossmodal links in spatial attention. Psychophysiology, 35, 313–327. Eimer, M., & Van Velzen, J. (2002). Crossmodal links in spatial attention are mediated by supramodal control processes: Evidence from event-related potentials. Psychophysiology, 39, 437–449. Engle, R. W. (2002). Working memory capacity as executive attention. Current Directions in Psychological Science, 11, 19–23. Falchier, A., Clavagnier, S., Barone, P., & Kennedy, H. (2003). Anatomical evidence of multimodal integration in primate striate cortex. Journal of Neuroscience, 22, 5749–5759. Fowler, B. (1994). P300 as a measure of workload during a simulated aircraft landing task. Human Factors, 36, 670–683. Giard, M., Perrin, F., Pernier, J., & Peronnet, F. (1988). Several attention related waveforms in auditory areas: A topographic study. Electroencephalography and Clinical Neurophysiology, 69, 371–384. Gopher, D., & Donchin, E. (1986). Workload: An examination of the concept. In K. R. Boff, L. Kaufhan, & J. P. Thomas (Eds.), Handbook of perception and human performance, Vol. 2, cognitive processes and performance. New York: John Wiley and Sons. Gratton, G., Coles, M. G. H., & Donchin, E. (1983). A new method for off-line removal of ocular artifact. Electroencephalography and Clinical Neurophysiology, 55, 468–484. Hansen, J., & Hillyard, S. (1980). Endogenous brain potentials associated with selective auditory attention. Electroencephalography and Clinical Neurophysiology, 49, 277–290. Hansen, J., & Hillyard, S. (1984). Effects of stimulation rate and attribute cueing on event-related potentials. Psychophysiology, 21, 394–405.
280
M.D. Bomba, A. Singhal / Brain and Cognition 74 (2010) 273–280
Hillyard, S., Hink, R., Schwent, V., & Picton, T. (1973). Electrical signs of selective attention in the human brain. Science, 182, 177–180. Kane, M. J., & Engle, R. W. (2003). Working-memory capacity and the control of attention: The contributions of goal neglect, response competition, and task set to Stroop interference. Journal of Experimental Psychology: General, 132, 47–70. Keppel, G. (1991). Design and analysis: A researcher’s handbook (3rd ed.). Englewood Cliffs, NJ: Prentice Hall. Kok, A. (2001). On the utility of P3 amplitude as a measure of processing capacity. Psychophysiology, 38, 557–577. Kumar, S., Stephan, K., Warren, J., Friston, K., & Griffiths, T. (2007). Hierarchical processing of auditory objects in humans. PLOS Computational Biology, 3, 1–9. LaBar, K. S., Gitelman, D. R., Parrish, T. B., & Mesulam, M. (1999). Neuroanatomic overlap of working memory and spatial attention networks: A functional MRI comparison within subjects. NeuroImage, 10, 695–704. Leuthold, H., & Sommer, W. (1998). Postperceptual effects and P300 latency. Psychophysiology, 35, 34–46. Magliero, A., Bashore, T. R., Coles, M. G. H., & Donchin, E. (1984). On the dependence of P300 latencies on stimulus evaluation. Psychophysiology, 21, 171–186. McCarthy, G., & Wood, C. (1985). Scalp distributions of event-related potentials: An ambiguity associated with analysis of variance models. Electroencephalography and Clinical Neurophysiology, 62, 203–208. McEvoy, L. K., Smith, M. E., & Gevins, A. (1998). Dynamic cortical networks of verbal and spatial working memory: Effects of memory load and task practice. Cerebral Cortex, 8, 563–574. Meehan, S., Singhal, A., & Fowler, B. (2005). The late Nd reflects a memory trace containing amodal spatial information. Psychophysiology, 42, 531–539. Meienbrock, A., Naumer, M., Doehrmann, O., Singer, W., & Muckli, L. (2007). Retinotopic effects during spatial audio–visual integration. Neuropsychologia, 45, 531–539. Mesulam, M.-M. (2000). Attentional networks, confusional states, and neglect syndromes. In M.-M. Mesulam (Ed.), Principles of behavioral and cognitive neurology (pp. 174–256). New York: Oxford University Press. Molholm, S., Ritter, W., Murray, M., Javitt, D., Schroedera, C., & Foxea, J. (2002). Multisensory auditory–visual interactions during early sensory processing in humans: A high-density electrical mapping study. Cognitive Brain Research, 14, 115–128. Muller-Gass, A., & Schröger, E. (2007). Perceptual and cognitive task difficulty has differential effects on auditory distraction. Brain Research, 1136, 169–177. Näätänen, R. (1990). The role of attention in auditory information processing as revealed by event-related brain potentials and other brain measures of cognitive function. Behavioral and Brain Sciences, 13, 201–288. Näätänen, R. (1992). Attention and brain function. Hillsdale, NJ: Erlbaum. Näätänen, R., Gaillard, A., & Varey, C. (1981). Attention effects on auditory ERPs as a function of interstimulus interval. Biological Psychology, 13, 173–187. Norman, D., & Bobrow, D. (1975). On data-limited and resource-limited processes. Cognitive Psychology, 7, 44–64. Owen, A. M., McMillan, K., Laird, A., & Bullmore, E. (2005). N-back working memory paradigm: A meta-analysis of normative functional neuroimaging studies. Human Brain Mapping, 25, 46–59. Pashler, H. (1984). Dual-task interference in simple tasks: Data and theory. Psychological Bulletin, 116, 220–244. Pashler, H., & Johnson, J. C. (1998). Attentional limitations in dual-task performance. In H. Pashler (Ed.), Attention (pp. 155–189). Hove, England: Psychology Press.
Posner, M. I., & Mitchell, R. F. (1967). Chronometric analysis of classification. Psychological Review, 74, 392–409. Posner, M. I., Nissen, M. J., & Ogden, W. C. (1978). Attended and unattended processing modes: The role of set for spatial attention. In H. L. Pick & E. Saltzman & Social Science Research Council (Eds.), Committee on cognition, modes of perceiving and processing information (pp. 137–158). Hillsdale, NJ: L. Erlbaum Associates. Ramirez, J., Bomba, M., Singhal, A., & Fowler, B. (2005). Influence of a visual spatial attention task on auditory early and late Nd and P300. Biological Psychology, 68, 121–134. Rif, J., Hari, R., Hamalainen, M., & Sams, M. (1991). Auditory attention affects two different areas in the human supratemporal cortex. Electroencephalography and Clinical Neurophysiology, 79, 464–472. Rockland, K., & Ojima, H. (2003). Multisensory convergence in calcarine visual areas in macaque monkey. International Journal of Psychophysiology, 50, 19–26. Rowe, J. B., Toni, I., Josephs, O., Frackowiak, R. S., & Passingham, R. E. (2000). The prefrontal cortex: Response selection or maintenance within working memory? Science, 288, 1656–1660. Shelley, A., Ward, P., Michie, P., Andrews, S., Mitchell, P., Catts, S., et al. (1991). The effect of repeated testing on ERP components during auditory selective attention. Psychophysiology, 28, 496–510. Singhal, A., Doerfling, P., & Fowler, B. (2002). Effects of a dual task on the N100–P200 complex and the early and late Nd attention waveforms. Psychophysiology, 39, 236–245. Singhal, A., & Fowler, B. (2004). The differential effects of Sternberg short- and longterm memory scanning on the late Nd and P300 in a dual-task paradigm. Cognitive Brain Research, 21, 124–132. Singhal, A., & Fowler, B. (2005). The effects of memory scanning on the late Nd and P300: An interference study. Psychophysiology, 42, 142–150. Smiley, J., Hackett, T., Ulbert, I., Karmas, G., Lakatos, P., Javitt, D., et al. (2007). Multisensory convergence in auditory cortex, I. Cortical connections of the caudal superior temporal plane in macaque monkeys. The Journal of Comparative Neurology, 502, 894–923. Sternberg, S. (1975). Memory scanning: New findings and current controversies. Quarterly Journal of Experimental Psychology, 27, 1–32. Teder-Salejarvi, W., & Hillyard, S. (1998). The gradient of spatial auditory attention in free field: An event-related potential study. Perception and Psychophysics, 60, 1228–1242. Woldorff, M., Gallen, C., Hampson, S., Hillyard, S., Pantev, C., Sobel, D., et al. (1993). Modulation of early sensory processing in human auditory cortex during auditory selective attention. Proceedings of the National Academy of Sciences, USA, 90, 8722–8726. Woods, D. (1990). The physiological basis of selective attention: Implications of event-related potential studies. In J. Rohrbaugh, R. Parasuraman, & J. Johnson (Eds.), Event-related brain potentials (pp. 178–209). New York: Oxford University Press. Woods, D., & Alain, C. (2001). Conjoining three auditory features: An event-related potential study. Journal of Cognitive Neuroscience, 13, 449–509. Woods, D., Alho, K., & Algazi, A. (1994). Stages of auditory feature conjunction: An event-related potential study. Journal of Experimental Psychology: Human Perception and Performance, 20, 81–94.
Contents lists available at ScienceDirect
Brain and Cognition journal homepage: www.elsevier.com/locate/b&c
ERP evidence of early cross-modal links between auditory selective attention and visuo-spatial memory Marie D. Bomba a,b, Anthony Singhal c,⇑ a
Neurosciences and Mental Health Program, The Hospital for Sick Children, Canada Department of Kinesiology & Health Science, York University, Toronto, ON, Canada c Department of Psychology and Centre for Neuroscience, University of Alberta, Edmonton AB, Canada b
a r t i c l e
i n f o
Article history: Accepted 20 August 2010 Available online 14 October 2010 Keywords: ERP Attention Memory Nde Ndl P300 Cross-modal Spatial Dual-task
a b s t r a c t Previous dual-task research pairing complex visual tasks involving non-spatial cognitive processes during dichotic listening have shown effects on the late component (Ndl) of the negative difference selective attention waveform but no effects on the early (Nde) response suggesting that the Ndl, but not the Nde, is affected by non-spatial processing in a dual-task. Thus to further explore the nature of this dissociation and whether the Nd waveform is affected by spatial processing; fourteen adult participants performed auditory dichotic listening in conjunction with visuo-spatial memory in a cross-modal dual-task paradigm. The results showed that the visuo-spatial memory task decreased both the Nde and Ndl waveforms, and also attenuated P300 and increased its latency. This pattern of results suggests that: (1) the Nde reflects a memory trace that is shared with vision when the information is spatial in nature, and (2) P300 latency appears to be influenced by the discriminability of stimuli underlying the Nde and Ndl memory trace. Ó 2010 Elsevier Inc. All rights reserved.
1. Introduction An important question to answer in cognitive neuroscience involves elucidating the nature of the functional relationship between attention and working memory. There is a general agreement that these processes reflect complementary cognitive systems that often operate together to facilitate certain complex behaviour (Cabeza et al., 2003; de Fockert, Rees, Frith, & Lavie, 2001; Engle, 2002; Kane & Engle, 2003; LaBar, Gitelman, Parrish, & Mesulam, 1999). However, the specific interactions between these systems is not fully understood particularly when information is encountered in more than one sensory modality. Since it is known that there are strong links between audition and vision (Driver & Spence, 1998; Eimer & Schroger, 1998), the focus of the present experiment was to further explore the relationship between attention and working memory with event-related potentials (ERPs) in a cross-modal dual-task paradigm that combined auditory selective attention, and visuo-spatial working memory. When ERPs are recorded during a classic version of the dichotic listening paradigm (Hillyard, Hink, Schwent, & Picton 1973), and the ERP from the unattended stimuli is subtracted from the attended ERP, an endogenous negativity is observed that has two parts: an early negative difference (Nde), and a late negative differ⇑ Corresponding author. E-mail address: [email protected] (A. Singhal). 0278-2626/$ - see front matter Ó 2010 Elsevier Inc. All rights reserved. doi:10.1016/j.bandc.2010.08.007
ence (Ndl). These two components are thought to reflect overlapping, but distinct processes associated with auditory selective attention. Nde appears maximal at frontocentral electrodes, and typically peaks between 70 ms and 200 ms after stimulus onset. Nde has been shown to reflect the processing of simple stimulus features such as pitch (Alho, 1992; Woods, 1990), and evidence suggests that it is generated by a neural source within the auditory cortex (Giard, Perrin, Pernier, & Peronnet, 1988; Rif, Hari, Hamalainen, & Sams, 1991; Woldorff et al., 1993). It has been proposed that Nde reflects a temporary feature recognition system that consists of a series of overlapping memory traces that function to determine the suitability of the incoming stimulus for further processing (Näätänen, 1992). According to this theory, the simple physical characteristics of the stimuli are compared to a representation held in memory that contains the relevant stimulus features. This matching process terminates when the current stimulus is found to differ from the template. Therefore, the closer the feature-match is between the incoming stimulus and the template; the longer the process takes, which leads to a larger amplitude and longer latency of Nde. It has been argued that this comparison process is reflected by a processing negativity (PN), and the Nd represents the manifestation of the difference between the two PNs that reflect relevant stimuli and irrelevant stimuli respectively (Näätänen, 1992). However, it has been suggested that the selection of stimuli for further processing may not entirely depend upon physical characteristics. For instance it has been proposed
274
M.D. Bomba, A. Singhal / Brain and Cognition 74 (2010) 273–280
that Nde is sensitive to perceptual context, and that selective processing of the stimuli occurs at different perceptual levels (Alain, Achim, & Richer, 1993; Alain & Arnott, 2000; Alain & Woods, 1994; Arnott & Alain, 2002). The nature of the information processing mechanisms underlying the frontally distributed Ndl is not as well understood as that of Nde. It has been shown that Ndl peaks in the range 300–600 ms after stimulus onset, and Näätänen, (1992) has suggested that it involves frontal lobe generators. The Ndl is reduced by practice (Shelley et al., 1991; Woods, 1990) and it has been shown to reflect the analysis of multidimensional stimuli (e.g., Woods & Alain, 2001; Woods, Alho, & Algazi, 1994). Teder-Salejarvi and Hillyard (1998) reported that Ndl amplitude was largest in response to stimulus locations adjacent to an attended location in a horizontal array. This indicates that Ndl may reflect the extended processing of stimuli in order to verify their relative spatial locations. Moreover, Ndl is more sensitive than Nde to the magnitude of the physical difference between the attended and unattended stimuli (Hansen & Hillyard, 1980), and it is larger at longer inter-stimulus intervals (Hansen & Hillyard 1984; Näätänen, Gaillard, & Varey 1981). Based on this evidence, Näätänen (1990) suggested that Ndl may represent the ‘‘further processing” of the incoming stimulus, as well as selective rehearsal of the to-be-attended stimulus; which he termed ‘‘the attentional trace”. More recently, the roles of Nde and Ndl during auditory selective attention have been investigated with a series of experiments that employed a dual-task interference paradigm in which a primary visual task was used to probe the Nd components elicited during a secondary dichotic listening task. The dual-task paradigm is a useful tool in attention research because performance costs associated with the introduction of perceptual-cognitive load can be attributed to limitations within the attention system. These limitations have been suggested to reveal either central processing bottlenecks (Pashler, 1984), capacity sharing limitations, or crosstalk interference between specific cognitive processes (Pashler & Johnson, 1998). ERP studies that make use of dual-task paradigms generally rely on the assumption that changes in the latency and amplitude of ERP waveforms under load, compared to single task conditions; reflect the interference effects in the neural processing underlying performance deficits (Gopher & Donchin, 1986). Relying on this logic, the first of these dual-task studies employed a simulated aircraft flying task and showed that, compared to the dichotic task performed alone, the amplitude of Ndl, but not the Nde, was reduced both by the introduction of the flying task, and by an increase in its difficulty (Singhal, Doerfling, & Fowler, 2002). A second finding of interest was that P300 was reduced in concert with the Ndl. The dissociation between the Nde and Ndl was interpreted as indicating that the latter reflects amodal processing rather than unimodal auditory processing because the interference with the auditory Ndl was caused by a visual task. Furthermore, it was suggested that the relationship between Ndl and P300 may be closer than had been previously thought (Singhal et al. 2002). However, due to the complexity of the flying task; the question of which of the various cognitive demands of the flying task was responsible for the reduction of Ndl was left unanswered. One solution to address this question was to employ simpler visual tasks containing fewer components, thereby allowing more specific inferences to be drawn about the locus and nature of the interference. Accordingly, Singhal and Fowler (2004), Singhal and Fowler (2005) employed both the varied-set (short-term memory, STM) and fixed set (long-term memory, LTM) versions of Sternberg’s (1975) visual memory scanning tasks to investigate the cross-modal relationship between Ndl and memory processes. This study showed that Ndl amplitude was reduced by the varied-but not the fixed-set version of the scanning task whereas P300 was sensitive to both, while Nde was unaffected by either version. This
pattern of results suggested that Ndl reflects short-term memory processes, but not long-term memory processes, and supported the idea that Nde is a modality specific waveform. Moreover, the suggestion was made that within Baddeley’s (1986, 2000) view of working memory (WM); P300 may reflect a general aspect of WM perhaps associated with task difficulty (Kok, 2001), whereas the Ndl may reflect specific processes within the phonological loop (Singhal & Fowler, 2004). Additional support for this position was provided by Ramirez, Bomba, Singhal, and Fowler (2005), who investigated the effects of a modified visual covert attention switching paradigm (e.g. Posner, Nissen, & Ogden, 1978) on the Nd components. This study showed that the valid/invalid condition of Posner’s task compared to a neutral control condition, decreased Ndl, but did not affect Nde. Furthermore, P300 dissociated from the Ndl because its amplitude was decreased by both the neutral control and the valid/invalid conditions. It was concluded that Ndl is likely sensitive to attention switching and both Ndl and P300 share sensitivity to an amodal system that is not reflected by Nde. However, since the auditory dichotic listening task to generate the Nd waves employed by Singhal, Fowler and colleagues was primarily a spatial task, the question of whether Ndl is sensitive to spatial attentional control, or more general control mechanisms was not answered. Accordingly, Meehan, Singhal, and Fowler (2005) employed a modified version of the Posner task, comparing interference caused by a spatial attention switching task with interference caused by a letter matching task (Posner & Mitchell, 1967) so that successive lower- and upper-case letters of the alphabet appeared at the same spatial location, with the former serving as a cue for the latter. The results showed that Ndl was sensitive to interference from the spatial but not the letter matching task suggesting that Ndl reflects spatial attentional control, and not deeper non-spatial attentional control mechanisms. Once again, Nde remained unaffected by the experimental manipulations, and when considered with the other previously mentioned dual-task studies; suggests that this component reflects processes that are not compromised by cross-modal dual-task load. To summarize the dual-task work of Singhal, Fowler and colleagues, Ndl was found to be sensitive to tasks that involve working memory and spatial attention control processes. The pattern of Ndl data dissociated from P300, and importantly for the present study; Nde was found to be insensitive to all of the visual task manipulations. Thus, these studies strongly support Näätänen’s (1990) speculation that Ndl involves processing associated with an attentional trace that shares processing resources with other forms of working memory processes. However, these studies also show insensitivity of Nde to the load imposed by the same cognitive tasks, which is not in agreement with Näätänen’s (1992) theory that the Nde reflects a self-terminating matching process between incoming stimuli and the representations of to-be-attended stimulus feature retained in a memory buffer. However, one other possibility is that Nde does reflect a working memory trace – a spatial working memory trace. None of the aforementioned dual-task studies employed spatial memory in the competing tasks, yet the auditory dichotic listening task employed in these studies clearly required a spatial strategy. Thus, we reasoned that it would be important to probe the spatial Nd waves with a memory task involving spatial information. It has been shown in the literature that an effective way to engage spatial working memory processes is with variations of the visuo-spatial n-back task. An important fact to consider with the n-back task is that it is considered to be more complex than the delayed-response task because the memory demands of the n-back task can be parametrically manipulated while keeping the other task factors relatively constant (McEvoy, Smith, & Gevins, 1998). Indeed, in a large scale meta-analysis study of the n-back task in functional neuroimaging studies, Owen, McMillan, Laird, and Bullmore (2005) showed that
M.D. Bomba, A. Singhal / Brain and Cognition 74 (2010) 273–280
there was a high degree of consistency among the working memory networks in frontal and parietal cortex that were activated by various n-back tasks. Furthermore, the spatial variants of the n-back tasks activated unique networks in right dorsolateral prefrontal, lateral pre-motor, and posterior parietal regions compared to non-spatial n-back tasks (Owen et al., 2005), which encompass a neural network strongly considered to be involved in spatial working memory processes (Mesulam, 2000; Rowe, Toni, Josephs, Frackowiak, & Passingham, 2000). Thus, our purpose in this study was to examine whether a functional relationship exists between spatial working memory processes and the auditory Nde. To this end we employed a visuo-spatial n-back task performed in conjunction with dichotic listening in a cross-modal dual-task paradigm. Finally, since a potential benefit of the dual-task paradigm is the ability to differentiate between effects associated with the introduction of a second task from those associated with an increase in its difficulty; we sought to manipulate difficulty of the n-back task in order to examine whether the processes underlying the Nde reflect spatial working memory demands within a single modality (task difficulty), as well as across modalities (introduction of dual-task). 2. Methods 2.1. Participants Fourteen paid volunteers (eight female); age range 18–31 years participated in this experiment. All reported normal hearing with normal or corrected-to-normal vision. Ethics approval was obtained, and the participants gave their informed consent.
275
2.4. Recording and quantification of EEG Silver–silver chloride (Ag–AgCl) electrodes were attached at Fz, F3, F4, Cz, C3, C4, Pz, T3, and T4, referenced to linked ears, and grounded to the forehead with an impedance of 5 k X or less. Electro-oculographic (EOG) electrodes were attached to the outer canthus and supra-orbital region of the right eye. The signals were amplified (GRASS Neurodata Acquisition System 12B) with a bandpass of 0.1–30 Hz ( 6 dB, 9 dB/octave rolloff). The EEG and EOG were digitized every 5 ms for a 1500 ms epoch, which included a 100 ms pre-stimulus baseline. The EEG data were corrected for saturation and eye movements (Gratton, Coles, & Donchin, 1983). Peak picking software (CED Spike 2) was used to measure the components defined by the following: P300 was measured from the attended deviant tones and defined as the most positive peak in the range 270–600 ms. The difference waves were derived as follows: Nde = attended standards waveforms – unattended standards waveforms, and was defined as the most negative peak in the range 70–200 ms; Ndl = attended standards waveforms – unattended standards waveforms, and defined as the most negative peak in the range 300–500 ms. We calculated mean amplitude values for all ERP components by averaging the peak value with the value 5 ms prior to the peak and the value 5 ms post peak. Following visual inspection of the waveforms, we employed paired t-tests on normalized data (McCarthy & Wood, 1985) to determine at which midline electrodes the respective averaged ERP peaks were largest in the baseline auditory condition. We then performed statistical analyses on the mean amplitude data at those latencies.
2.2. Visuo-spatial task The participants fixated on the centre of a white cross presented in the centre of a 55.88 cm computer monitor. The arms of the cross subtended a visual angle of 3.4°. There were two conditions: Lag 1 and Lag 2. In the first condition, the cue for task initiation was the appearance of an open arrowhead at the end of one arm of the cross. After 3 s the cue disappeared and a solid arrowhead appeared simultaneously and randomly at one of the other three arm positions. The task was to respond to the position of the initial open arrow with the index finger on the dominant hand using a modified joystick. The response triggered the disappearance of the solid arrowhead and the simultaneous and random appearance of another solid arrowhead at one of the three other locations, which required a response to the second arrow – and so on. In the Lag 2 condition, two open arrowheads appeared in succession, followed by a closed arrowhead, which required a response to the spatial location of the first open arrowhead. We recorded reaction time (RT) and errors from this task. 2.3. Dichotic listening task The auditory task consisted of 12 blocks of 250 non-overlapping tones (50 ms duration, 10 ms ramp, 60 dB SPL) presented through stereophonic earphones (KOSS 4 A). The frequent standards (1000 Hz, 80% probability) and the infrequent deviants (1500 Hz, 20% probability) were presented randomly as a rectangular distribution with a mean inter-tone interval (offset to onset) of 700 ms and a range of 500–900 ms to either the left or right ear for a total of 125 tones/ear/block. The participants were required to pay attention to a designated ear and report the infrequent deviants with a button press with their non-dominant hand, RT, errors, and EEG activity were recorded from this task. Errors included misses and false alarms, but the latter were so rare that the two categories were pooled for the purposes of analysis.
2.5. Design and procedure We employed a repeated measures design. EEG activity from the dichotic listening task was collected in three conditions: one control and two experimental. In the control condition the dichotic listening task was performed alone (Aud-control). No visual stimuli were presented during this condition other than a white fixation cross at the centre of the monitor. In the two experimental conditions, the visuo-spatial task was performed in conjunction with the auditory task (Aud + Lag 1, and Aud + Lag 2). In two additional behavioural control conditions, the visuo-spatial task was performed alone (Lag 1 and Lag 2). We employed a subsidiary dualtask design where the participants were instructed to treat the visual task as primary and the auditory task as secondary. The five conditions were presented in counterbalanced order between sessions determined by Latin square. The ear of attention was also counterbalanced. The duration of each block of auditory tones was approximately 2.9 min with a short pause in between each block. In the dual conditions, the presentation of the visuo-spatial task was synchronized with the presentation of the auditory stimulus blocks. The timing was maintained when the visual task was presented alone. Thus, there was a total of approximately 1300 responses/condition. The two tasks were run on separate computers, and the timing of stimulus presentation between the two tasks was uncorrelated; thus the precise interleaving of the stimuli across modalities was not recorded. Prior to the experiment, the participants were trained on both tasks until error rates were below 15%. In the dual-task conditions, the participants were instructed to treat the visual task as primary and the auditory task as secondary. The data were analyzed using one-way, or two-way repeated measures ANOVAs with Greenhouse Geisser epsilon corrections, followed by contrasts corrected with the modified Bonferroni procedure (Keppel, 1991). Only the data collected from Fz, Cz, and Pz are reported in this paper.
276
M.D. Bomba, A. Singhal / Brain and Cognition 74 (2010) 273–280
Table 1 Mean correct reaction time (RT; ms) and accuracy (% correct) for each task and condition. Standard errors are shown in parentheses. Measure
Task alone controls Auditory task (Aud) Visual task (Lag 1) Visual task (Lag 2) Dual experimental conditions Auditory Auditory + visual Lag 1 Auditory + visual Lag 2 Visual Visual Lag 1 + auditory Visual Lag 2 + auditory
RT
Accuracy
325 (9) 230 (27) 225 (29)
98.3 (0.3) 98.7 (0.3) 95.7 (0.6)
466 (18) 509 (20)
92.7 (1.4) 86.6 (2.4)
296 (22) 348 (27)
97.0 (0.6) 90.7 (1.1)
increased in the dual condition compared to the alone condition and the increase was larger for Lag 2 compared to Lag 1. The analysis of the accuracy data revealed a similar pattern [F (1, 13) = 14.05, P < 0.002]. 3.3. Auditory task A one-way repeated measures ANOVA on RT revealed significant slowing [F (2, 26) = 153.04, T = 0.0570, P < 0.00001]. The slowing was observed from the Aud control condition to the Aud + Lag 1 condition (P < 0.01), and further slowing from Aud + Lag 1 to Aud + Lag 2 (P < 0.01). The same pattern was observed in the decrease in accuracy [F (2, 26) = 18.82, T = 0.649, P < 0.0002; both contrasts, P < 0.01]. 3.4. ERP measures
3. Results 3.1. Behavioural measures Table 1 contains the mean correct RT and accuracy data for each condition.
Fig. 1 shows the grand average waveforms elicited by the attended and unattended standard tones during dichotic listening at Fz and Cz. Fig. 2 shows the Nde and Ndl components derived from the original waveforms. 3.5. Nde and Ndl
3.2. Visual task A 2 (alone/dual) 2 (Lag 1/Lag 2) ANOVA on the RT data revealed an interaction [F (1, 13) = 9.25, P < 0.002], showing that RT
The mean peak amplitudes for Nde and Ndl at Fz and Cz are shown in Fig. 3. Both components were largest at Cz (Nde, t (13) = 4.99, P < 0.0002; Ndl, t (13) = 2.18, P < 0.04). A one-way re-
Fig. 1. Grand average waveforms for the auditory attended and unattended standards at Fz and Cz for the auditory control (auditory), dual Lag 1 (Aud cont + Lag 1), dual Lag 2 (Aud cont + Lag 2) conditions. The arrow in the upper left panel indicates the time of stimulus onset.
M.D. Bomba, A. Singhal / Brain and Cognition 74 (2010) 273–280
277
Fig. 2. Grand average auditory Nd (negative difference) waveforms at Fz and Cz for the auditory control (auditory), dual Lag 1 (Aud cont + Lag 1), dual Lag 2 (Aud cont + Lag 2) conditions. The Nd early (Nde) and Nd late (Ndl) are indicated, and the arrow in the upper panel indicates the time of stimulus onset.
P300 amplitude was largest at Pz (t (13) = 5.92, P < 0.0004); mean amplitudes: Aud = 11.59 lv; Aud + Lag 1 = 9.33 lv; Aud + Lag 2 = 8.89 lv. A one-way repeated measures ANOVA on P300 amplitude revealed a decrease at the Pz electrode, which was due to the difference between the Aud-control and the Aud + Lag 2 condition [F (2, 26) = 4.32, E = 0.598, P < 0.04; contrast, P < 0.03]. In the case of P300 latency, a one-way repeated measures ANOVA revealed an increase in latency as a cost of concurrence (Aud-control versus Aud + Lag 1) at Pz. [P < 0.01; Pz: F (2, 26) = 5.3, E = 0.747, P < 0.02; contrast, P < 0.01]. 4. Discussion 4.1. Dual-task assumptions
Fig. 3. Mean auditory negative difference (Nd) amplitude (microvolts ± SEM) at the Cz electrode for the auditory control (auditory), dual Lag 1 (Aud cont + Lag 1), dual Lag 2 (Aud cont + Lag 2) conditions.
peated measures ANOVA on Nde amplitude at Cz revealed a cost of concurrence (Aud-control versus Aud + Lag 1) [F (2, 26) = 9.22, E = 0.633, P < 0.001]; contrast, P < 0.02]. The Ndl analysis at Cz also revealed a cost of concurrence [F (2, 26) = 10.87, E = 0.972, P < 0.0004]; contrast, P < 0.002. 3.6. P300 Fig. 4 shows the grand average waveforms elicited by the attended deviant tones at Fz, Cz, and Pz. It was revealed that
The assumption that the visual task would impose a processing load on the auditory task, and vice versa, was confirmed by the behavioural data. The introduction of the visual task increased the time to detect targets as well as the number of target errors committed. Furthermore, the time to detect targets increased as the level of difficulty of the visual task increased from Lag 1 to Lag 2. On the other hand, the introduction of the auditory task degraded both speed and accuracy on the visual task. 4.2. Nde and Ndl The Nd results from this experiment are particularly interesting on two fronts. First, we present a novel finding where auditory Nde is reduced by a dual-task consisting of visually presented information involving spatial memory. Second, we
278
M.D. Bomba, A. Singhal / Brain and Cognition 74 (2010) 273–280
Fig. 4. Grand average waveforms for the auditory attended targets at Fz, Cz and Pz for the auditory control (auditory), dual Lag 1 (Aud cont + Lag 1), dual Lag 2 (Aud cont + Lag 2) conditions. The arrow in the top panel indicates the time of stimulus onset.
show clear-cut interference of the visuo-spatial task on Ndl. On the face of it, these data are in agreement with the idea that Nd (both Nde and Ndl) reflect a series of memory traces (Näätänen, 1992; Woods & Alain, 2001). It has been previously argued that the auditory attention system underlying the Nd waves could be separated into two distinct parts: one that involves modality specific operations (Nde), and one that is higher order in nature and is sensitive to amodal processes such as working memory (Meehan et al., 2005; Singhal & Fowler, 2004, 2005; Singhal et al., 2002). Taken together, the two Nd findings presented here suggest that this auditory attention system is more sensitive to amodal processes than has been previously considered. Our data suggest that the bottleneck of interference in our paradigm between auditory selective attention and visuo-spatial memory may occur earlier than has previously been speculated. Furthermore, the locus of interference may be dependent upon the type
of information that is under analysis. In other words, in contrast to our earlier work, we observed an early locus of interference in the system (70–150 ms). This is a novel finding in the dual-task literature and one possibility is that it is because the auditory attention and visual memory systems share basic resources for spatial information that do not have to be mutually recruited in situations involving cross-modal tasks that combine spatial and non-spatial information. Thus, while Ndl reflects higher-order mechanisms of attentional control and working memory that can operate outside of a particular modality; Nde is more specific in terms of its sensitivity to visual information. This line of argument is in agreement with studies that show that auditory and visual spatial integration can occur in primary sensory cortex (Meienbrock, Naumer, Doehrmann, Singer, & Muckli, 2007; Molholm et al., 2002). Indeed, the type of interference that we observed could be brought about by two possible convergent
M.D. Bomba, A. Singhal / Brain and Cognition 74 (2010) 273–280
pathways between auditory and visual cortex: (1) feed-forward visual input may reach multi-sensory regions such as along the superior temporal sulcus (STS) and then be transmitted via feedback loops to early auditory cortex (Kumar, Stephan, Warren, Friston, & Griffiths, 2007; Smiley et al., 2007); or (2) a direct feedforward pathway from early visual areas to primary auditory cortex (Clavagnier, Falchnier, & Kennedy, 2004; Falchier, Clavagnier, Barone, & Kennedy, 2003; Rockland & Ojima, 2003). One considerable limitation of our study in addressing the question of cortical pathway interactions is that we did not collect ERPs from the visual task as was done previously (Singhal & Fowler, 2004; Singhal & Fowler, 2005), and thus we cannot examine the pattern of interference that the auditory task had on visual task ERPs. Furthermore, in our study the precise timing of the cross-modal trial interleaving is not known, and we did not vary the level of difficulty of the auditory task, which might have imposed additional effects on the processing limitations within the auditory (Muller-Gass & Schröger, 2007) and visual systems (Norman & Bobrow, 1975). Finally, we cannot rule out the possibility that the pattern of interference observed on Nde is due to supramodal interference that involves spatial attentional control mechanisms (Eimer & Van Velzen, 2002). While, the studies conducted by Ramirez et al. (2005) and Meehan et al. (2005) did not find interference effects between visuo-spatial attentional control and auditory Nde, an important difference in their tasks was that they did not involve explicit memory encoding, storage, and retrieval processes. It may be the case that the attentional trace underlying Nde consists of a strong memory component that is not tapped by a visual task unless that task involves strong memory operations of spatial information.
4.3. Nd and P300 An additional question in this experiment was whether P300 and Ndl would be influenced in a parallel manner by the introduction of the visual task as has been previously shown (Singhal et al., 2002), or whether they would dissociate from one another (Ramirez et al., 2005; Singhal & Fowler, 2004). The visuo-spatial task in the present experiment did not have a major impact upon P300 component amplitude. Specifically, there was a decrease in P300 with the introduction of the visual task at the Pz electrode but no further effect as the task increased in difficulty from Lag 1 to Lag 2. This finding is similar in nature to the Ndl results and is in agreement with Singhal et al. (2002), who showed a parallel amplitude effect between these two components. Still, it is important to note that P300 latency was also affected in this experiment. A clear-cut slowing of P300 was observed as a cost of concurrence at Fz and Pz, and a strong trend in this direction at Cz. The slowing of P300 as a result of the introduction of a second task has reliably been shown before (Fowler, 1994), but has not been previously described in relation to Nd effects. Here we have observed P300 slowing in conjunction with a decrease in Nde and Ndl amplitudes. It has been argued that P300 latency is sensitive to the discriminability of the attended stimulus (Leuthold & Sommer, 1998; Magliero, Bashore, Coles, & Donchin, 1984), and it is likely that the introduction of the visuo-spatial task influenced the discriminability of the stimuli underlying the memory trace reflected in Nd amplitude. Thus the finding of a decreased Nde and Ndl may not be entirely isolated from the observed P300 latency effects. Along this line of argument, it may be the case that Nde amplitude relies on low level processes closely related to processes involved in the generation of the P300 response. If these processes are delayed due to discriminability effects, the result may be a conjunction of a slowed P300 and an attenuated Nde. A close relationship between Nde and P300 has been speculated before
279
(Näätänen, 1990), and here we show additional evidence for this link. Finally, it should be noted that we did not observe parallel effects between the ERP and the behavioural data in this study. The ERP effects were associated with the introduction of the visuo-spatial n-back task, but not by an increase in its difficulty. Whereas, we observed auditory RT and accuracy effects associated with both the introduction of the visual task as well as an increase in its difficulty. This likely suggests that the auditory ERP in our study reflect some of the processes associated with cross-modal spatial working memory load, but not those associated with changes in task difficulty within the visual modality. In summary, we have presented ERP evidence of early cross-modal interactions between the auditory selective attention system and the visual spatial memory systems. Further research is needed to examine whether these ERP patterns are reproducible between non-spatial selective attention and working memory.
Acknowledgment The authors thank Graeme Armstrong at the University of Alberta for his assistance with the ERP data analyses.
References Alain, C., Achim, A., & Richer, F. (1993). Perceptual context and the selective attention effect on auditory event-related brain potentials. Psychophysiology, 30, 572–580. Alain, C., & Arnott, R. (2000). Selectively attending to auditory objects. Frontiers in Biosciences, 1, 202–212. Alain, C., & Woods, D. (1994). Signal clustering modulates auditory cortical activity in humans. Perception & Psychophysics, 56, 501–516. Alho, K. (1992). Selective attention in auditory processing as reflected by eventrelated brain potentials. Psychophysiology, 29, 247–263. Arnott, S., & Alain, C. (2002). Effects of perceptual context on event-related brain potentials during auditory spatial attention. Psychophysiology, 39, 625–632. Baddeley, A. (1986). Working memory. Oxford: Clarendon Press. Baddeley, A. (2000). The episodic buffer: A new concept of working memory. Trends in Cognitive Sciences, 4, 417–423. Cabeza, R., Dolcos, F., Prince, S. E., Rice, H. J., Weissman, D. H., & Nyberg, L. (2003). Attention-related activity during episodic memory retrieval: A cross-function fMRI study. Neuropsychologia, 41, 390–399. Clavagnier, S., Falchnier, A., & Kennedy, H. (2004). Long-distance feedback projections to area V1: Implications for multisensory integration, spatial awareness, and visual consciousness. Cognitive, Affective, & Behavioral Neuroscience, 4, 117–126. de Fockert, J., Rees, G., Frith, C., & Lavie, N. (2001). The role of working memory in visual selective attention. Science, 291, 1803–1806. Driver, J., & Spence, C. (1998). Crossmodal attention. Current Opinion in Neurobiology, 8, 245–253. Eimer, M., & Schroger, E. (1998). ERP effects of intermodal attention and crossmodal links in spatial attention. Psychophysiology, 35, 313–327. Eimer, M., & Van Velzen, J. (2002). Crossmodal links in spatial attention are mediated by supramodal control processes: Evidence from event-related potentials. Psychophysiology, 39, 437–449. Engle, R. W. (2002). Working memory capacity as executive attention. Current Directions in Psychological Science, 11, 19–23. Falchier, A., Clavagnier, S., Barone, P., & Kennedy, H. (2003). Anatomical evidence of multimodal integration in primate striate cortex. Journal of Neuroscience, 22, 5749–5759. Fowler, B. (1994). P300 as a measure of workload during a simulated aircraft landing task. Human Factors, 36, 670–683. Giard, M., Perrin, F., Pernier, J., & Peronnet, F. (1988). Several attention related waveforms in auditory areas: A topographic study. Electroencephalography and Clinical Neurophysiology, 69, 371–384. Gopher, D., & Donchin, E. (1986). Workload: An examination of the concept. In K. R. Boff, L. Kaufhan, & J. P. Thomas (Eds.), Handbook of perception and human performance, Vol. 2, cognitive processes and performance. New York: John Wiley and Sons. Gratton, G., Coles, M. G. H., & Donchin, E. (1983). A new method for off-line removal of ocular artifact. Electroencephalography and Clinical Neurophysiology, 55, 468–484. Hansen, J., & Hillyard, S. (1980). Endogenous brain potentials associated with selective auditory attention. Electroencephalography and Clinical Neurophysiology, 49, 277–290. Hansen, J., & Hillyard, S. (1984). Effects of stimulation rate and attribute cueing on event-related potentials. Psychophysiology, 21, 394–405.
280
M.D. Bomba, A. Singhal / Brain and Cognition 74 (2010) 273–280
Hillyard, S., Hink, R., Schwent, V., & Picton, T. (1973). Electrical signs of selective attention in the human brain. Science, 182, 177–180. Kane, M. J., & Engle, R. W. (2003). Working-memory capacity and the control of attention: The contributions of goal neglect, response competition, and task set to Stroop interference. Journal of Experimental Psychology: General, 132, 47–70. Keppel, G. (1991). Design and analysis: A researcher’s handbook (3rd ed.). Englewood Cliffs, NJ: Prentice Hall. Kok, A. (2001). On the utility of P3 amplitude as a measure of processing capacity. Psychophysiology, 38, 557–577. Kumar, S., Stephan, K., Warren, J., Friston, K., & Griffiths, T. (2007). Hierarchical processing of auditory objects in humans. PLOS Computational Biology, 3, 1–9. LaBar, K. S., Gitelman, D. R., Parrish, T. B., & Mesulam, M. (1999). Neuroanatomic overlap of working memory and spatial attention networks: A functional MRI comparison within subjects. NeuroImage, 10, 695–704. Leuthold, H., & Sommer, W. (1998). Postperceptual effects and P300 latency. Psychophysiology, 35, 34–46. Magliero, A., Bashore, T. R., Coles, M. G. H., & Donchin, E. (1984). On the dependence of P300 latencies on stimulus evaluation. Psychophysiology, 21, 171–186. McCarthy, G., & Wood, C. (1985). Scalp distributions of event-related potentials: An ambiguity associated with analysis of variance models. Electroencephalography and Clinical Neurophysiology, 62, 203–208. McEvoy, L. K., Smith, M. E., & Gevins, A. (1998). Dynamic cortical networks of verbal and spatial working memory: Effects of memory load and task practice. Cerebral Cortex, 8, 563–574. Meehan, S., Singhal, A., & Fowler, B. (2005). The late Nd reflects a memory trace containing amodal spatial information. Psychophysiology, 42, 531–539. Meienbrock, A., Naumer, M., Doehrmann, O., Singer, W., & Muckli, L. (2007). Retinotopic effects during spatial audio–visual integration. Neuropsychologia, 45, 531–539. Mesulam, M.-M. (2000). Attentional networks, confusional states, and neglect syndromes. In M.-M. Mesulam (Ed.), Principles of behavioral and cognitive neurology (pp. 174–256). New York: Oxford University Press. Molholm, S., Ritter, W., Murray, M., Javitt, D., Schroedera, C., & Foxea, J. (2002). Multisensory auditory–visual interactions during early sensory processing in humans: A high-density electrical mapping study. Cognitive Brain Research, 14, 115–128. Muller-Gass, A., & Schröger, E. (2007). Perceptual and cognitive task difficulty has differential effects on auditory distraction. Brain Research, 1136, 169–177. Näätänen, R. (1990). The role of attention in auditory information processing as revealed by event-related brain potentials and other brain measures of cognitive function. Behavioral and Brain Sciences, 13, 201–288. Näätänen, R. (1992). Attention and brain function. Hillsdale, NJ: Erlbaum. Näätänen, R., Gaillard, A., & Varey, C. (1981). Attention effects on auditory ERPs as a function of interstimulus interval. Biological Psychology, 13, 173–187. Norman, D., & Bobrow, D. (1975). On data-limited and resource-limited processes. Cognitive Psychology, 7, 44–64. Owen, A. M., McMillan, K., Laird, A., & Bullmore, E. (2005). N-back working memory paradigm: A meta-analysis of normative functional neuroimaging studies. Human Brain Mapping, 25, 46–59. Pashler, H. (1984). Dual-task interference in simple tasks: Data and theory. Psychological Bulletin, 116, 220–244. Pashler, H., & Johnson, J. C. (1998). Attentional limitations in dual-task performance. In H. Pashler (Ed.), Attention (pp. 155–189). Hove, England: Psychology Press.
Posner, M. I., & Mitchell, R. F. (1967). Chronometric analysis of classification. Psychological Review, 74, 392–409. Posner, M. I., Nissen, M. J., & Ogden, W. C. (1978). Attended and unattended processing modes: The role of set for spatial attention. In H. L. Pick & E. Saltzman & Social Science Research Council (Eds.), Committee on cognition, modes of perceiving and processing information (pp. 137–158). Hillsdale, NJ: L. Erlbaum Associates. Ramirez, J., Bomba, M., Singhal, A., & Fowler, B. (2005). Influence of a visual spatial attention task on auditory early and late Nd and P300. Biological Psychology, 68, 121–134. Rif, J., Hari, R., Hamalainen, M., & Sams, M. (1991). Auditory attention affects two different areas in the human supratemporal cortex. Electroencephalography and Clinical Neurophysiology, 79, 464–472. Rockland, K., & Ojima, H. (2003). Multisensory convergence in calcarine visual areas in macaque monkey. International Journal of Psychophysiology, 50, 19–26. Rowe, J. B., Toni, I., Josephs, O., Frackowiak, R. S., & Passingham, R. E. (2000). The prefrontal cortex: Response selection or maintenance within working memory? Science, 288, 1656–1660. Shelley, A., Ward, P., Michie, P., Andrews, S., Mitchell, P., Catts, S., et al. (1991). The effect of repeated testing on ERP components during auditory selective attention. Psychophysiology, 28, 496–510. Singhal, A., Doerfling, P., & Fowler, B. (2002). Effects of a dual task on the N100–P200 complex and the early and late Nd attention waveforms. Psychophysiology, 39, 236–245. Singhal, A., & Fowler, B. (2004). The differential effects of Sternberg short- and longterm memory scanning on the late Nd and P300 in a dual-task paradigm. Cognitive Brain Research, 21, 124–132. Singhal, A., & Fowler, B. (2005). The effects of memory scanning on the late Nd and P300: An interference study. Psychophysiology, 42, 142–150. Smiley, J., Hackett, T., Ulbert, I., Karmas, G., Lakatos, P., Javitt, D., et al. (2007). Multisensory convergence in auditory cortex, I. Cortical connections of the caudal superior temporal plane in macaque monkeys. The Journal of Comparative Neurology, 502, 894–923. Sternberg, S. (1975). Memory scanning: New findings and current controversies. Quarterly Journal of Experimental Psychology, 27, 1–32. Teder-Salejarvi, W., & Hillyard, S. (1998). The gradient of spatial auditory attention in free field: An event-related potential study. Perception and Psychophysics, 60, 1228–1242. Woldorff, M., Gallen, C., Hampson, S., Hillyard, S., Pantev, C., Sobel, D., et al. (1993). Modulation of early sensory processing in human auditory cortex during auditory selective attention. Proceedings of the National Academy of Sciences, USA, 90, 8722–8726. Woods, D. (1990). The physiological basis of selective attention: Implications of event-related potential studies. In J. Rohrbaugh, R. Parasuraman, & J. Johnson (Eds.), Event-related brain potentials (pp. 178–209). New York: Oxford University Press. Woods, D., & Alain, C. (2001). Conjoining three auditory features: An event-related potential study. Journal of Cognitive Neuroscience, 13, 449–509. Woods, D., Alho, K., & Algazi, A. (1994). Stages of auditory feature conjunction: An event-related potential study. Journal of Experimental Psychology: Human Perception and Performance, 20, 81–94.