- Email: [email protected]
Human auditory cortex is activated by omissions of auditory stimuli
Brain Research 745 Ž1997. 134–143
Research report
Human auditory cortex is activated by omissions of auditory stimuli T. Raij ) , L. McEvoy, J.P. Makela ¨ ¨ 1, R. Hari Brain Research Unit, Low Temperature Laboratory, Helsinki UniÕersity of Technology, FIN 02150 Espoo, Finland Accepted 10 September 1996
Abstract Cortical signals associated with infrequent tone omissions were recorded from 9 healthy adults with a whole-head 122-channel neuromagnetometer. The stimulus sequence consisted of monaural Žleft or right. 50-ms 1-kHz tones repeated every 0.2 or 0.5 s, with 7% of the tones randomly omitted. Tones elicited typical responses in the supratemporal auditory cortices. Omissions evoked strong responses over temporal and frontal areas, independently of the side of stimulation, with peak amplitudes at 145–195 ms. Response amplitudes were 60% weaker when the subject was not attending to the stimuli. Omission responses originated in supratemporal auditory cortices bilaterally, indicating that auditory cortex plays an important role in the brain’s modelling of temporal characteristics of the auditory environment. Additional activity was observed in the posterolateral frontal cortex and in the superior temporal sulcus, more often in the right than in the left hemisphere. Keywords: Auditory cortex; Internal representation; Magnetoencephalography; Human; Brain; Omission
1. Introduction Our brains continuously construct and update models of the external world in order to interpret and predict environmental events. Association cortices evidently have a key role in constructing these internal representations, yet activation of sensory cortices, indicating not only the presence but also the absence of stimuli, necessarily provides critical information. Changes in sensory input are often more informative than monotonous, repetitive stimuli, and the cerebral cortex reacts to them strongly. In the auditory modality, a rare change in almost any parameter of a repeating sound elicits a mismatch response ŽMMR. generated mainly in auditory cortex w15,24x. An unexpected omission of a stimulus from a regular series of stimuli also activates the brain, producing an omission response in electroencephalographic ŽEEG. w3,5,19,32,35x, magnetoencephalographic ŽMEG. w18,30x and intracerebral recordings w2x. The response is time-locked to the expected stimulus occurrence
but not attributable to the stimulus immediately preceding the omission. The neural generators of omission responses are not fully understood. Simson et al. w33x suggested in an EEG study that human visual and auditory omissions evoke early modality-specific activity in the secondary sensory cortices, and later activity in the inferior parietal and frontal association areas. MEG, which is selectively sensitive to activity of fissural cortex and does not suffer from the distorting effects of the skull and scalp, offers more precise source localization than does EEG. Joutsiniemi and Hari w18x, using a 7-channel MEG device, reported an attention-sensitive component of the auditory omission response, apparently generated in the posterolateral frontal cortex. The availability of magnetometers with over 100 channels now makes it possible to record MEG activity over the whole scalp simultaneously. We used a whole-scalp neuromagnetometer to identify the human cortical areas activated by auditory omissions. 2. Materials and methods
)
Corresponding author. Fax: Ž358 . Ž0 . 451-2969; e-mail: [email protected] 1 Present address: Central Military Hospital, P.O. Box 50, FIN 00301 Helsinki, Finland.
2.1. Subjects and stimuli Nine healthy members of the laboratory staff Žage 23–50 years Žmean 32., one left-handed, one woman. were stud-
0006-8993r97r$17.00 Copyright q 1997 Elsevier Science B.V. All rights reserved. PII S 0 0 0 6 - 8 9 9 3 Ž 9 6 . 0 1 1 4 0 - 7
T. Raij et al.r Brain Research 745 (1997) 134–143
ied. One subject suffers from intermittent tinnitus that was not present during the measurements. All subjects were experienced in MEG experiments. Monaural, 50-ms Žincluding 15-ms rise-fall times., 1kHz tones of 95–101 dB SPL were delivered to the subject through plastic tubes and earpieces at interstimulus intervals ŽISIs. of 0.2 s Ž210 ms. and 0.5 s Ž495 ms.. In four subjects, an ISI of 1 s Ž1010 ms. was also used. Seven percent of the tones were randomly omitted, with at least three tones occurring between successive omissions. The subject was instructed either to ignore the stimuli and read a self-selected text Ž‘ignore’. or to count the omissions silently and motionlessly, with eyes open Ž‘attend’.. The order of stimulus blocks across subjects was randomized with respect to the ear stimulated Žleftrright., ISI Ž0.2r0.5r1 s., and task Žattendrignore.. Each subject was studied in 5–10 of the 12 possible conditions. However, all were presented with the left-ear attended stimuli at 0.5-s and 0.2-s ISIs. The measurements were performed during several sessions, each lasting about an hour, on different days. For four subjects, 1–3 conditions were repeated on different days to assess response stability over time.
2.2. Recording and data analysis Cerebral magnetic signals were recorded with a wholescalp 122-channel planar SQUID Žsuperconducting quantum interference device. magnetometer ŽNeuromag-122 TM w1x. inside a magnetically shielded room. The instrument measures two orthogonal tangential derivatives, E BzrE x and E BzrE y, of the magnetic field at 61 measurement sites. Planar gradiometers detect the largest signal just above a local source current. The electro-oculogram ŽEOG. was recorded from electrodes lateral and inferior to the left eye, and epochs contaminated by eyeblinks or eye movements Žsignal amplitudes exceeding "150 mV. were automatically discarded from the averaged data. Subjects were not instructed to avoid blinking, since this may affect the amplitude of the auditory cortical responses w36x. The signals were band-pass filtered at 0.03–100 Hz, digitized at 397 Hz and averaged on-line. The responses were averaged in three categories: ‘tones’ Žexcluding tones immediately following an omission., ‘omissions’, and ‘tones-after-omissions’ Žtones immediately following an omission., with at least 100 single responses in each category. The length of the analysis period depended on category and ISI and varied from 1600 ms for omission responses at 1-s ISI to 300 ms for tones at 0.2-s ISI. The averaged signals were digitally low-pass filtered at 40 Hz and response amplitudes were measured with respect to a prestimulus baseline Ž50 ms for ISIs of 0.5 and 1 s, 30 ms for the 0.2-s ISI.. Prior to each measurement, the position of the head
135
with respect to the sensor array was calculated from signals produced by weak currents fed into three small coils attached to the scalp. The position of these coils with respect to the nasion and the two preauricular points was measured with a 3-D digitizer ŽIsotrak 3S1002, Polhemus Navigator Sciences, Colchester, VT, USA.. This information was used to align the subject’s MEG and MRI Žmagnetic resonance imaging. data Žfor details, see w10x.. Peak amplitudes and latencies were measured over each hemisphere from the sensor pair showing the largest response after ‘ vector-summing’ the responses of the two orthogonal derivatives:wŽ E BzrE x . 2 q Ž E BzrE y . 2 x1r2 . For tones and tones-after-omissions, the peak amplitudes and latencies of the 100-ms response ŽN100m. were measured over each hemisphere. For omissions, the amplitude and latency of the first prominent peak was measured over each hemisphere. In the attend condition, only values exceeding the root-mean-square noise level of the prestimulus baseline by a factor of 2 were accepted. Data in the ignore condition often did not meet this criterion. In these cases the amplitude values in the ignore condition were measured at the same latency as in the attend condition. Sources of the averaged responses were identified by calculating the three-dimensional locations, strengths and orientations of equivalent current dipoles ŽECDs. that would generate a dipolar magnetic field pattern as similar to the measured pattern as possible, using the least-squares method Žfor a detailed description of the source estimation, see w11x.. The head was modelled as a spherical volume conductor, fitted to the outer surface of the brain over the Sylvian fissures and vertex on each subject’s MRI. For tone responses, a single source was identified for the largest response, N100m, in each hemisphere separately. Since the amplitude of N100m decreases dramatically as ISI decreases w14x, the source location was determined from the longest-ISI signals available Ž1 s or 0.5 s.. A subset of 18 channels over each temporal region was used to determine the ECD, and only dipoles accounting for at least 85% of the local field variance were accepted Žfor details, see w11x.. For omission responses, source locations were estimated in attend conditions from a subset of 12–24 channels around the signal maxima. Again, only dipoles accounting for at least 85% of the local field variance were accepted. If the field was not satisfactorily explained by a single dipole, additional dipoles were included in the model. At the dipole strength maximum, each accepted dipole had to add at least 10% to the whole-head goodness-of-fit Ž g%. value, and the source had to be at least twice as strong as the largest value during the baseline period. The accepted dipoles were then used in a multi-dipole model where dipole strength was allowed to vary over time. Peak dipole strengths and latencies were measured for each dipole in the attend condition. Thereafter the corresponding amplitude values were measured in the ignore condition at the same latencies using the same model.
136
T. Raij et al.r Brain Research 745 (1997) 134–143
3. Results 3.1. EÕoked responses Fig. 1 shows the 122-channel responses of Subject 5 to left-ear tones and omissions with the three ISIs in the attend condition. The responses to tones are strongest over the right, contralateral temporal area, and consist of major deflections around 100 and 200 ms ŽN100m and P200m., preceded by a small response around 60 ms ŽP60m.. These responses have been previously shown to be generated in the supratemporal auditory cortex Žfor a review, see w12x.. N100m and P200m clearly increase in amplitude as ISI is increased. Over the left, ipsilateral temporal region, the only clear deflection is a P200m. Over this hemisphere, P200m is not affected by ISI. The attended tone omissions produced broad, clear responses, again largest over the temporal areas, and somewhat stronger over the right than the left hemisphere. In this subject, omission responses started about 100 ms after the expected tone onset and reached their first peak at 120–190 ms. Response amplitude was not systematically affected by ISI.
Fig. 2A shows the largest responses over both hemispheres for Subject 5 in the 3 ISI conditions. Responses to tones, omissions and tones-after-omissions are shown for left and right ear stimuli in both attend and ignore conditions. The omission response is largest with the 0.5-s ISI and is clearly stronger in the attend than in the ignore condition. It is larger over the right hemisphere without any difference between left- and right-ear stimulation. Clear omission responses, again stronger in the attend than in the ignore condition, can also be seen at the 0.2-s ISI. At the 1-s ISI, omission responses are observable over the right hemisphere, although weaker, and with a somewhat smaller effect of attention. Tone responses, unlike omission responses, are larger with longer ISIs and are usually stronger over the hemisphere contralateral to the stimuli. Responses to tones-after-omissions are larger than responses to tones preceded by other tones. Fig. 2B shows, for all subjects, the largest omission responses over each hemisphere at the 0.5-s ISI. All subjects, except S9, show clear omission responses ŽSubject 1 only over the right hemisphere., although with substantial interindividual variability in response morphology. For most subjects, the responses are bilateral with no
Fig. 1. 122-channel MEG responses of Subject 5 to tones Žleft, n s 1400. and omissions Žright, n s 100.. In each channel pair, the upper trace shows the latitudinal and the lower the longitudinal field derivative. The stimuli were presented to the left ear with ISIs of 0.2 s Ždashed line., 0.5 s Žsolid thick line. and 1 s Žsolid thin line.. Typical P60m, N100m and P200m responses to tones over the contralateral hemisphere are shown enlarged in inset b. Inset a shows the enlarged ipsilateral responses. The same channel pairs are shown enlarged in insets c and d for the omission responses.
T. Raij et al.r Brain Research 745 (1997) 134–143
difference between left and right ear stimuli. The responses are clearly stronger in the attend than in the ignore condition for all but one subject ŽS1; no ignore data were obtained for S4.. N100m responses were identified in eight of the nine subjects, bilaterally in five. N100m amplitude and latency were analyzed only for the left-ear 0.5-s ISI. N100m was, on average, 12 ms earlier Žnot significant. and 39% stronger Ž F Ž1,7. s 8.0 P - 0.05. over the right, contralateral hemisphere than over the left. N100m was 23% Ž F Ž1,7. s 10.7; P - 0.05. larger when preceded by omissions than by tones, but the peak latency was the same in both categories. Neither amplitude nor latency was significantly affected by attention.
137
Omission responses generally consisted of a transient deflection reaching its first peak around 145–195 ms after the predicted time of stimulus occurrence, followed by a broader, sometimes sustained response until the onset of the next stimulus Žcf. Fig. 2B.. The omission response amplitude, unlike the tone response amplitude, was heavily dependent on attention, being on average 60% weaker in the ignore than attend condition Ž F Ž1,4. s 239; P - 0.001.. It did not differ according to ear of stimulation or hemisphere, but was on average 33 ms earlier with the 0.2-s than the 0.5-s ISI Ž t Ž7. s y3.7; P - 0.01.. Omission response morphology was replicable across days. There was some amplitude variability across sessions, but peak latencies were quite consistent.
Fig. 2. A: responses Žvector-sums of channel pairs. of Subject 5 over the left and right hemispheres ŽLH, RH. to tones, omissions and tones-after-omissions, for left and right ear stimulation ŽLE, RE., in attend and ignore conditions Žatt, ign.. Responses are shown at the 1-s Župper., 0.5-s Žmiddle. and 0.2-s Žlower. ISI. Data are displayed as continuous, although averages were calculated separately for the three categories of tones, omissions, and tones-after-omissions. The dashed vertical lines mark the time of the expected stimulus occurrence. B: omission responses Žvector-sums of channel pairs. of all subjects at the 0.5-s ISI.
138
T. Raij et al.r Brain Research 745 (1997) 134–143
3.2. Source locations Omissions produced largest deflections over the same cerebral regions as tones, and the field patterns evoked by omissions closely resembled those evoked by tones. We thus first examined the extent to which the generator of N100m to tones could also explain the omission responses. Fig. 3A shows, for one subject, the magnetic field patterns for N100m at the timepoints used to search for the ECDs, and the locations of these dipoles superimposed on the subject’s MRI. These ECDs were then used to explain the omission response by keeping the dipole locations constant but allowing the orientations to vary over time. In Fig. 3B, the predicted waveforms from this 2-dipole model are superimposed on the measured 122-channel omission responses. Over both midtemporal areas, the model predicts accurately the omission responses, as well as responses to tones-after-omissions. However, the model does not ac-
count for parts of the omission response over the right anterotemporal area. A third source, in the right superior temporal sulcus ŽSTS. adequately accounts for the remaining part of the response. Fig. 3C shows the magnetic field pattern used to estimate the location of this source and the location of the source superimposed on the subject’s MRI. This approach demonstrates that a major part of the omission response can be explained by sources in the supratemporal auditory cortices, but that additional areas may also be activated by omissions. As a more conventional approach, we modelled directly the sources of omission responses. In all subjects, the main sources were found in the supratemporal cortices, typically bilaterally. Fig. 4 shows, for the eight subjects producing omission responses, the N100m sources and examples of omission response sources to both left and right ear stimuli. The sources of the omission response were on average within 8 " 4 mm Žmean " S.D.. from the N100m sources
Fig. 3. A: magnetic field patterns Žstep 20 fT. of N100 m to attended left-ear tones ŽISI 0.5 s. and the corresponding ECDs Žarrows.. The N100m ECDs, superimposed on the subject’s MRI, are also shown. B: measured 122-channel omission responses Žthin lines. superimposed on signals predicted by bilateral N100m generators, which were allowed to change orientation as a function of time Žthick lines.. The inserts show one channel pair, with measured signals superimposed on predictions for the 2- and 3-dipole models. C: field patterns of omission responses at 465 ms and the STS source superimposed on the subject’s MRI.
T. Raij et al.r Brain Research 745 (1997) 134–143
in both hemispheres, without any systematic difference in any direction. Although supratemporal sources often accounted for the major part of the omission response, additional sources were needed to fully explain the whole-head data. In six subjects additional sources were identified within the STS Žsee Fig. 6., as was shown for Subject 6 in Fig. 3; each of these sources improved the goodness-of-fit of the model
139
by at least 10% Žsee below.. One subject showed bilateral STS activation, four had a source only in the right STS and one only in the left. In addition to temporal-lobe sources, frontal sources were evident in all subjects in at least one condition. Fig. 5 shows the 122-channel responses of Subject 2 along with the predicted signals from a 2-dipole model, with one dipole in each temporal cortex. Over the left hemisphere,
Fig. 4. Supratemporal generators of responses to tones Žspheres. and omissions Žipsilateral stimuli: triangles; contralateral stimuli: squares. of all subjects. Examples shown are the sources of clearest omission responses to left and right ear stimuli in each hemisphere of each subject. The dipoles are visible through F 20 mm in the MRIs. Subject 4 produced atypical N100m responses bilaterally, Subjects 7 and 8 over the left hemisphere, resulting in rejection of this part of their data.
140
T. Raij et al.r Brain Research 745 (1997) 134–143
the model predicts well the measured data. In contrast, the right-hemisphere signals are not explained by the temporal-lobe sources alone. The addition of a third source clearly improves the explanation. This source was identified on the basis of a dipolar field pattern over the anterior right hemisphere at 140 ms Žsee bottom right corner of Fig. 5., and it agrees with a source in the right posterolateral frontal cortex. Similar source locations were evident also at the two other ISIs for this subject. In all subjects, the major sources outside the temporal lobes were located in posterolateral frontal cortex, slightly anterior to the central sulcus. Fig. 6 shows these sources in both hemispheres of all subjects, superimposed on a schematic sagittal brain slice. Frontal activity was observed in 25 measurements in the right hemisphere and in 11 in the left. Four subjects had bilateral frontal sources. The source locations were more variable across subjects in the left than in the right hemisphere. Fig. 6 also shows the STS sources in all subjects. STS sources were observed in
Fig. 6. All frontal and STS sources from all subjects, displayed on a schematic MRI slice separately for the left and the right hemisphere. All subjects had at least one frontal source. The individual source locations in this figure were determined by finding the Sylvian fissure and central sulcus in each brain separately, and superimposing the sources on the schematic brain slice with respect to these landmarks. In four subjects, the central sulcus was localized by finding the generator of the 20-ms magnetic response to median nerve stimulation Žknown to be in primary somatosensory cortex, embedded within the central sulcus.; in the remaining four subjects, the central sulcus was localized on anatomical grounds.
Fig. 5. Measured Žthin lines. and predicted Žthick lines. 122-channel omission responses of Subject 2 to left-ear stimuli in the attend condition, 0.2-s ISI. The predictions in the whole-head picture were derived from a model consisting of two temporal-lobe dipoles, one in each hemisphere. One right frontal channel is shown enlarged on the left. The panel in the middle shows the measured and predicted responses from the same channel when a third, frontal dipole was added to the model. The magnetic field pattern Žstep 20 fT. over the right hemisphere at the time of the frontal source estimation is shown on the right.
two measurements in the left hemisphere and seven in the right. As suggested already by the 60% smaller amplitudes of the omission responses in the ignore than in the attend condition Žsee above., the activation strengths of all source areas were significantly affected by attention, being on average 70% weaker in the ignore than in the attend condition Žpaired t-tests for each area separately P - 0.01, collapsed across 0.5-s and 0.2-s ISIs, hemisphere and stimulated ear.. The latencies and amplitudes did not differ between contra- and ipsilateral supratemporal responses, nor between contra- and ipsilateral frontal responses Žpaired t-tests, collapsed across ear and 0.5-s and 0.2-s ISI.. The mean Ž"S.D.. latency of maximum activation in supratemporal and frontal areas was approximately the
T. Raij et al.r Brain Research 745 (1997) 134–143
same Ž180 " 80 ms and 190 " 40 ms, collapsed across 0.5-s and 0.2-s ISIs, hemisphere and stimulated ear.; activity of the superior temporal sulcus source peaked later Ž350 " 115 ms..
4. Discussion Our results show that the absence of a tone in an otherwise regular sound sequence may activate the human auditory cortex bilaterally. This activation is not caused by afferent input from the sensory receptors but rather reflects an internal expectation, built up over the prior pattern of stimulation, that a stimulus should occur. When this expectancy is not met, a response is produced signalling the deviation from the predicted event. The response is tightly time-locked to the expected stimulus occurrence, illustrating the ability of auditory cortex to model the temporal input pattern. The response is also heavily dependent on attention, indicating that the tuning of auditory cortex to the temporal pattern of stimulation is an active process. The main generators of the omission response were in the auditory areas of the supratemporal cortex in the immediate vicinity of those generating responses to tones. Tones are known to elicit larger and earlier responses in the contralateral than the ipsilateral hemisphere w13,21,27,29x. Omissions, however, elicited activity of the same magnitude and latency in the supratemporal cortices of both hemispheres, suggesting that the expectancy of future stimuli is not ear specific. Although the main generators of the omission response were in the supratemporal cortex, omissions also elicited activity within STS in six out of nine subjects. Activation of the supratemporal and STS cortex by infrequent sound omissions has recently been suggested in a combined MEG and fMRI study w23x. STS is known to contain both auditory and modality non-specific association cortex w7,17x. Auditory association cortex in the STS is involved in auditory short term memory w6x: bilateral ablations of the lateral surface of the superior temporal gyrus and the upper bank of the STS severely impair monkeys’ performance on a delayed auditory, but not visual, matching to sample task, despite intact auditory discrimination. Short term memory is certainly necessary for forming expectations of future stimuli based on prior input patterns. Outside the temporal lobes, tone omissions activated posterolateral frontal cortex in all subjects. Supratemporal auditory cortex and association cortex in the STS have connections to frontal association areas w17,26x and the lateral frontal cortex is an important node in the general neural network subserving attention w22x. Thus, the frontal areas activated by omissions in our study may be related to activation of general attention and target detection systems. In EEG recordings, stimulus omissions typically elicit a vertex-negative response at about 200 ms ŽN2., followed by a vertex-positive P3 deflection at 300–1000 ms w31–35x.
141
The generators producing the N2 response have been related both to modality-specific processing and to more general orienting reactions. The generators of the P3 response also probably include modality-specific and nonspecific brain regions w4,8,9x. The source areas found in the present study are not necessarily directly related to generators of the N2 and P3 responses because of the well-known different sensitivities of the EEG and MEG to tangentially vs. radially oriented currents w11x. In two of our subjects, 15-channel EEG scalp mapping Žreferenced to nose. was performed simultaneously with the MEG recordings in the 0.5-s ISI condition. When attending to the stimuli, one subject showed a large P3 Žpeak at 355 ms. only, while the other produced both N2 Ž195 ms. and P3 Ž360 ms. deflections. None of these responses coincided with any major deflections in the MEG responses or with peaks in the source amplitude waveforms. It thus seems that the electric N2–P3 complex receives major contributions from radially oriented currents which are silent in MEG recordings, and that its major generator areas differ from those found in our MEG recordings. However, the N2–P3 complex most likely receives some contribution also from the sources identified in the present study. The observed supratemporal and STS sources would produce EEG signals broadly distributed around vertex: due to the main current directions in these areas, the supratemporal sources would produce a vertexnegative deflection which might contribute to the early phase of the N2 response, whereas the STS source could contribute to the P3 response. The frontal sources, due to their current direction, could further add to the parietal positivity. In the present study, the morphology of the omission response was quite variable across subjects. This could reflect individual differences in source orientations. Since MEG is mainly sensitive to tangential currents, differences in radial versus tangential source orientation across subjects would appear as differences in response strength. Response variation is also likely to be related to individual differences in the accuracy of estimating time intervals. MMRs, which signal the automatic detection of a change in some feature of a repetitive stimulus w24x, have been elicited by changes in temporal rhythm, when, e.g., a stimulus occurs earlier than expected w16,20,25x. MMRs are also produced by decreases in stimulus intensity. A stimulus omission can be interpreted as a maximum intensity decrease and thus should produce a MMR. However, it is unlikely that the response elicited by stimulus omissions reflects the same brain processes that generate MMRs. First, MMRs are generally considered to be automatic processes, occurring irrespective of attention whereas omission responses strongly depended on attention Žsee also w18x.. Secondly, the generator of the MMR in the supratemporal cortex is typically located about 1 cm anterior to that of N100m, whereas no such difference was observed between the generators of the omission response
142
T. Raij et al.r Brain Research 745 (1997) 134–143
and N100m. Finally, bilateral frontal activity elicited by omissions in this study has not been reported in recordings of MMRs, nor did we observe right inferior parietal lobe activity to stimulus omissions, as has been reported for MMRs to changes in frequency, duration, or ISI w20x. Omission responses show that a template is formed in sensory cortex of expected events. Such a template can act as a cortical filter through which incoming sensory information is evaluated in an attention-dependent way. Our data also indicate that this processing is right-hemisphere dominant, as has been previously suggested for automatic change detection w20x. Acknowledgements A preliminary report of these data has appeared in abstract form w28x. This study has been financially supported by the Academy of Finland, the Sigrid Juselius ´ Foundation, the Foundation for Medicine in Finland and the BIRCH Large-Scale Facility ŽEU’s Human Capital and Mobility Programme. in the Low Temperature Laboratory of the Helsinki University of Technology. The MRIs were recorded at the Radiology Department of the Helsinki University Central Hospital.
w10x
w11x
w12x
w13x
w14x
w15x
w16x
w17x
w18x
References w19x w1x Ahonen, A.I., Hamalainen, M.S., Kajola, M.J., Knuutila, J.E.T., ¨ ¨¨ Laine, P.P., Lounasmaa, O.V., Parkkonen, L.T., Simola, J.T. and Tesche, C.D., 122-channel SQUID instrument for investigating the magnetic signals from the human brain, Physica Scripta, T49 Ž1993. 198–205. w2x Alain, C., Richer, F., Achim, A. and Saint Hilaire, J.-M., Human intracerebral potentials associated with target, novel and omitted auditory stimuli, Brain Topogr., 1 Ž1989. 237–245. w3x Barlow, J.S., Some observations on the electrophysiology of timing in the nervous system, Electroenceph. Clin. Neurophysiol., 27 Ž1969. 545. w4x Baudena, P., Halgren, E., Heit, G. and Clarke, J.M., Intracerebral potentials to rare target and distractor auditory and visual stimuli. III. Frontal cortex, Electroenceph. Clin. Neurophysiol., 94 Ž1995. 251–264. w5x Bullock, T.H., Karamursel, S., Achimowicz, J.Z., McClune, M.C. ¨ and Basar-Eroglu, C., Dynamic properties of human visual evoked and omitted stimulus potentials, Electroenceph. Clin. Neurophysiol., 91 Ž1994. 42–53. w6x Colombo, M., D’Amato, M.R., Rodman, H.R. and Gross, C.G., Auditory association cortex lesions impair auditory short-term memory in monkeys, Science, 247 Ž1990. 336–338. w7x Galaburda, A. and Sanides, F., Cytoarchitectonic organization of the human auditory cortex, J. Comp. Neurol., 190 Ž1980. 597–610. w8x Halgren, E., Baudena, P., Clarke, J.M., Heit, G., Liegois, C., ´ Chauvel, P. and Musolino, A., Intracerebral potentials to rare target and distractor auditory and visual stimuli. I. Superior temporal plane and parietal lobe, Electroenceph. Clin. Neurophysiol., 94 Ž1995. 191–220. w9x Halgren, E., Baudena, P., Clarke, J.M., Heit, G., Marinkovic, K., Devaux, B., Vignal, J.-P. and Biraben, A., Intracerebral potentials to rare target and distractor auditory and visual stimuli. II. Medial,
w20x
w21x
w22x
w23x
w24x w25x
w26x
w27x
w28x
lateral and posterior temporal lobe, Electroenceph. Clin. Neurophysiol., 94 Ž1995. 229–250. Hamalainen, M., Anatomical correlates for magnetoencephalogra¨ ¨¨ phy: integration with magnetic resonance images, Clin. Phys. Physiol. Meas., 12 Suppl. A Ž1991. 29–32. Hamalainen, M., Hari, R., Ilmoniemi, R.J., Knuutila, J. and Lounas¨ ¨¨ maa, O.V., Magnetoencephalography – theory, instrumentation and applications to noninvasive studies of the working human brain, ReÕ. Mod. Phys., 65 Ž1993. 413–497. Hari, R., The neuromagnetic method in the study of the human auditory cortex. In F. Grandori, M. Hoke and G.L. Romani ŽEds.., Auditory EÕoked Magnetic Fields and Potentials, AdÕances in Audiology, Vol. 6, Karger, Basel, 1990, pp. 222–282. Hari, R. and Makela, ¨ ¨ J.P., Modification of neuromagnetic responses of the human auditory cortex by masking sounds, Exp. Brain Res., 71 Ž1988. 87–92. Hari, R., Kaila, K., Katila, T., Tuomisto, T. and Varpula, T., Interstimulus-interval dependence of the auditory vertex response and its magnetic counterpart: Implications for their neural generation, Electroenceph. Clin. Neurophysiol., 54 Ž1982. 561–569. Hari, R., Hamalainen, M., Ilmoniemi, R., Kaukoranta, E., ¨ ¨¨ Reinikainen, K., Salminen, J., Alho, K., Naatanen, R. and Sams, M., ¨¨ ¨ Responses of the primary auditory cortex to pitch changes of tone pips: neuromagnetic recordings in man, Neurosci. Lett., 50 Ž1984. 127–132. Hari, R., Joutsiniemi, S.-L., Hamalainen, M. and Vilkman, V., ¨ ¨¨ Neuromagnetic responses of human auditory cortex to interruptions in a steady rhythm, Neurosci. Lett., 99 Ž1989. 164–168. Jones, E.G. and Powell, T.P.S., An anatomical study of converging sensory pathways within the the cerebral cortex of the monkey, Brain, 93 Ž1970. 793–820. Joutsiniemi, S.-L. and Hari, R., Omissions of auditory stimuli may activate frontal cortex, Eur. J. Neurosci., 1 Ž1989. 524–528. Klinke, R., Fruhstorfer, H. and Finkenzeller, P., Evoked responses as a function of external and stored information, Electroenceph. Clin. Neurophysiol., 25 Ž1968. 119–122. Levanen, S., Ahonen, A., Hari, R., McEvoy, L. and Sams, M., ¨ Deviant auditory stimuli activate human left and right auditory cortex differently, Cereb. Cortex, 6 Ž1996. 288–296. McEvoy, L., Hari, R., Imada, T. and Sams, M., Human auditory cortical mechanisms of sound lateralization: II. Interaural time differences at sound onset, Hear. Res., 67 Ž1993. 98–109. Mesulam, M.-M., Large-scale neurocognitive networks and distributed processing for attention, language and memory, Ann. Neurol., 28 Ž1990. 597–613. Miyauchi, S., Takino, R., Sasaki, Y., Putz, ¨ B. and Okamura, H., Visualization of information processing in the human brain: recent advances in MEG and functional MRI, 10th Tokyo Institute of Psychiatry International Symposium, Book of Abstracts Ž1995. 72– 73. Naatanen, R., Attention and Brain Function, Lawrence Erlbaum ¨¨ ¨ Associates, Hillsdale, New Jersey, 1992. Nordby, H., Roth, W.T. and Pfefferbaum, A., Event-related potentials to time-deviant and pitch-deviant tones, Psychophysiology, 25 Ž1988. 249–261. Pandya, D.N., Hallett, M. and Mukherjee, S.K., Intra- and interhemispheric connections of the neocortical auditory system in the rhesus monkey, Brain Res., 14 Ž1969. 49–65. Pantev, C., Lutkenhoner, B., Hoke, M. and Lehnertz, K., Compari¨ ¨ son between simultaneously recorded auditory-evoked magnetic fields and potentials elicited by ipsilateral, contralateral and binaural tone burst stimulation, Audiology, 25 Ž1986. 54–61. Raij, T., Makela, ¨ ¨ J.P., McEvoy, L. and Hari, R., Human auditory cortex is activated by omissions of auditory stimuli, 10th International Conference on Biomagnetism (Biomag96) Abstracts Ž1996. 83.
T. Raij et al.r Brain Research 745 (1997) 134–143 w29x Reite, M., Zimmerman, J.T. and Zimmerman, J.E., Magnetic auditory evoked fields: Interhemispheric asymmetry, Electroenceph. Clin. Neurophysiol., 51 Ž1981. 388–391. w30x Rogers, R.L., Papanicolaou, A.C., Baumann, S.B. and Eisenberg, H.M., Late magnetic fields and positive evoked potentials following infrequent and unpredictable omissions of visual stimuli, Electroenceph. Clin. Neurophysiol., 83 Ž1992. 146–152. w31x Ruchkin, D.S., Sutton, S., Munson, R., Silver, K. and Macar, F., P300 and feedback provided by absence of the stimulus, Psychophysiology, 18 Ž1981. 271–282. w32x Simson, R., Vaughan, H.G. and Ritter, W., The scalp topography of potentials associated with missing visual or auditory stimuli, Electroenceph. Clin. Neurophysiol., 40 Ž1976. 33–42.
143
w33x Simson, R., Vaughan, H.G. and Ritter, W., The scalp topography of potentials in auditory and visual discrimination tasks, Electroenceph. Clin. Neurophysiol., 42 Ž1977. 528–535. w34x Sutton, S., Braren, M., Zubin, J. and John, E.R., Evoked potential correlates of stimulus uncertainty, Science, 150 Ž1965. 1187–1188. w35x Sutton, S., Tueting, P., Zubin, P. and John, E.R., Information delivery and the sensory evoked potential, Science, 155 Ž1967. 1436–1439. w36x Verleger, R., The instruction to refrain from blinking affects auditory P3 and N1 amplitudes, Electroenceph. Clin. Neurophysiol., 78 Ž1991. 240–251.
Research report
Human auditory cortex is activated by omissions of auditory stimuli T. Raij ) , L. McEvoy, J.P. Makela ¨ ¨ 1, R. Hari Brain Research Unit, Low Temperature Laboratory, Helsinki UniÕersity of Technology, FIN 02150 Espoo, Finland Accepted 10 September 1996
Abstract Cortical signals associated with infrequent tone omissions were recorded from 9 healthy adults with a whole-head 122-channel neuromagnetometer. The stimulus sequence consisted of monaural Žleft or right. 50-ms 1-kHz tones repeated every 0.2 or 0.5 s, with 7% of the tones randomly omitted. Tones elicited typical responses in the supratemporal auditory cortices. Omissions evoked strong responses over temporal and frontal areas, independently of the side of stimulation, with peak amplitudes at 145–195 ms. Response amplitudes were 60% weaker when the subject was not attending to the stimuli. Omission responses originated in supratemporal auditory cortices bilaterally, indicating that auditory cortex plays an important role in the brain’s modelling of temporal characteristics of the auditory environment. Additional activity was observed in the posterolateral frontal cortex and in the superior temporal sulcus, more often in the right than in the left hemisphere. Keywords: Auditory cortex; Internal representation; Magnetoencephalography; Human; Brain; Omission
1. Introduction Our brains continuously construct and update models of the external world in order to interpret and predict environmental events. Association cortices evidently have a key role in constructing these internal representations, yet activation of sensory cortices, indicating not only the presence but also the absence of stimuli, necessarily provides critical information. Changes in sensory input are often more informative than monotonous, repetitive stimuli, and the cerebral cortex reacts to them strongly. In the auditory modality, a rare change in almost any parameter of a repeating sound elicits a mismatch response ŽMMR. generated mainly in auditory cortex w15,24x. An unexpected omission of a stimulus from a regular series of stimuli also activates the brain, producing an omission response in electroencephalographic ŽEEG. w3,5,19,32,35x, magnetoencephalographic ŽMEG. w18,30x and intracerebral recordings w2x. The response is time-locked to the expected stimulus occurrence
but not attributable to the stimulus immediately preceding the omission. The neural generators of omission responses are not fully understood. Simson et al. w33x suggested in an EEG study that human visual and auditory omissions evoke early modality-specific activity in the secondary sensory cortices, and later activity in the inferior parietal and frontal association areas. MEG, which is selectively sensitive to activity of fissural cortex and does not suffer from the distorting effects of the skull and scalp, offers more precise source localization than does EEG. Joutsiniemi and Hari w18x, using a 7-channel MEG device, reported an attention-sensitive component of the auditory omission response, apparently generated in the posterolateral frontal cortex. The availability of magnetometers with over 100 channels now makes it possible to record MEG activity over the whole scalp simultaneously. We used a whole-scalp neuromagnetometer to identify the human cortical areas activated by auditory omissions. 2. Materials and methods
)
Corresponding author. Fax: Ž358 . Ž0 . 451-2969; e-mail: [email protected] 1 Present address: Central Military Hospital, P.O. Box 50, FIN 00301 Helsinki, Finland.
2.1. Subjects and stimuli Nine healthy members of the laboratory staff Žage 23–50 years Žmean 32., one left-handed, one woman. were stud-
0006-8993r97r$17.00 Copyright q 1997 Elsevier Science B.V. All rights reserved. PII S 0 0 0 6 - 8 9 9 3 Ž 9 6 . 0 1 1 4 0 - 7
T. Raij et al.r Brain Research 745 (1997) 134–143
ied. One subject suffers from intermittent tinnitus that was not present during the measurements. All subjects were experienced in MEG experiments. Monaural, 50-ms Žincluding 15-ms rise-fall times., 1kHz tones of 95–101 dB SPL were delivered to the subject through plastic tubes and earpieces at interstimulus intervals ŽISIs. of 0.2 s Ž210 ms. and 0.5 s Ž495 ms.. In four subjects, an ISI of 1 s Ž1010 ms. was also used. Seven percent of the tones were randomly omitted, with at least three tones occurring between successive omissions. The subject was instructed either to ignore the stimuli and read a self-selected text Ž‘ignore’. or to count the omissions silently and motionlessly, with eyes open Ž‘attend’.. The order of stimulus blocks across subjects was randomized with respect to the ear stimulated Žleftrright., ISI Ž0.2r0.5r1 s., and task Žattendrignore.. Each subject was studied in 5–10 of the 12 possible conditions. However, all were presented with the left-ear attended stimuli at 0.5-s and 0.2-s ISIs. The measurements were performed during several sessions, each lasting about an hour, on different days. For four subjects, 1–3 conditions were repeated on different days to assess response stability over time.
2.2. Recording and data analysis Cerebral magnetic signals were recorded with a wholescalp 122-channel planar SQUID Žsuperconducting quantum interference device. magnetometer ŽNeuromag-122 TM w1x. inside a magnetically shielded room. The instrument measures two orthogonal tangential derivatives, E BzrE x and E BzrE y, of the magnetic field at 61 measurement sites. Planar gradiometers detect the largest signal just above a local source current. The electro-oculogram ŽEOG. was recorded from electrodes lateral and inferior to the left eye, and epochs contaminated by eyeblinks or eye movements Žsignal amplitudes exceeding "150 mV. were automatically discarded from the averaged data. Subjects were not instructed to avoid blinking, since this may affect the amplitude of the auditory cortical responses w36x. The signals were band-pass filtered at 0.03–100 Hz, digitized at 397 Hz and averaged on-line. The responses were averaged in three categories: ‘tones’ Žexcluding tones immediately following an omission., ‘omissions’, and ‘tones-after-omissions’ Žtones immediately following an omission., with at least 100 single responses in each category. The length of the analysis period depended on category and ISI and varied from 1600 ms for omission responses at 1-s ISI to 300 ms for tones at 0.2-s ISI. The averaged signals were digitally low-pass filtered at 40 Hz and response amplitudes were measured with respect to a prestimulus baseline Ž50 ms for ISIs of 0.5 and 1 s, 30 ms for the 0.2-s ISI.. Prior to each measurement, the position of the head
135
with respect to the sensor array was calculated from signals produced by weak currents fed into three small coils attached to the scalp. The position of these coils with respect to the nasion and the two preauricular points was measured with a 3-D digitizer ŽIsotrak 3S1002, Polhemus Navigator Sciences, Colchester, VT, USA.. This information was used to align the subject’s MEG and MRI Žmagnetic resonance imaging. data Žfor details, see w10x.. Peak amplitudes and latencies were measured over each hemisphere from the sensor pair showing the largest response after ‘ vector-summing’ the responses of the two orthogonal derivatives:wŽ E BzrE x . 2 q Ž E BzrE y . 2 x1r2 . For tones and tones-after-omissions, the peak amplitudes and latencies of the 100-ms response ŽN100m. were measured over each hemisphere. For omissions, the amplitude and latency of the first prominent peak was measured over each hemisphere. In the attend condition, only values exceeding the root-mean-square noise level of the prestimulus baseline by a factor of 2 were accepted. Data in the ignore condition often did not meet this criterion. In these cases the amplitude values in the ignore condition were measured at the same latency as in the attend condition. Sources of the averaged responses were identified by calculating the three-dimensional locations, strengths and orientations of equivalent current dipoles ŽECDs. that would generate a dipolar magnetic field pattern as similar to the measured pattern as possible, using the least-squares method Žfor a detailed description of the source estimation, see w11x.. The head was modelled as a spherical volume conductor, fitted to the outer surface of the brain over the Sylvian fissures and vertex on each subject’s MRI. For tone responses, a single source was identified for the largest response, N100m, in each hemisphere separately. Since the amplitude of N100m decreases dramatically as ISI decreases w14x, the source location was determined from the longest-ISI signals available Ž1 s or 0.5 s.. A subset of 18 channels over each temporal region was used to determine the ECD, and only dipoles accounting for at least 85% of the local field variance were accepted Žfor details, see w11x.. For omission responses, source locations were estimated in attend conditions from a subset of 12–24 channels around the signal maxima. Again, only dipoles accounting for at least 85% of the local field variance were accepted. If the field was not satisfactorily explained by a single dipole, additional dipoles were included in the model. At the dipole strength maximum, each accepted dipole had to add at least 10% to the whole-head goodness-of-fit Ž g%. value, and the source had to be at least twice as strong as the largest value during the baseline period. The accepted dipoles were then used in a multi-dipole model where dipole strength was allowed to vary over time. Peak dipole strengths and latencies were measured for each dipole in the attend condition. Thereafter the corresponding amplitude values were measured in the ignore condition at the same latencies using the same model.
136
T. Raij et al.r Brain Research 745 (1997) 134–143
3. Results 3.1. EÕoked responses Fig. 1 shows the 122-channel responses of Subject 5 to left-ear tones and omissions with the three ISIs in the attend condition. The responses to tones are strongest over the right, contralateral temporal area, and consist of major deflections around 100 and 200 ms ŽN100m and P200m., preceded by a small response around 60 ms ŽP60m.. These responses have been previously shown to be generated in the supratemporal auditory cortex Žfor a review, see w12x.. N100m and P200m clearly increase in amplitude as ISI is increased. Over the left, ipsilateral temporal region, the only clear deflection is a P200m. Over this hemisphere, P200m is not affected by ISI. The attended tone omissions produced broad, clear responses, again largest over the temporal areas, and somewhat stronger over the right than the left hemisphere. In this subject, omission responses started about 100 ms after the expected tone onset and reached their first peak at 120–190 ms. Response amplitude was not systematically affected by ISI.
Fig. 2A shows the largest responses over both hemispheres for Subject 5 in the 3 ISI conditions. Responses to tones, omissions and tones-after-omissions are shown for left and right ear stimuli in both attend and ignore conditions. The omission response is largest with the 0.5-s ISI and is clearly stronger in the attend than in the ignore condition. It is larger over the right hemisphere without any difference between left- and right-ear stimulation. Clear omission responses, again stronger in the attend than in the ignore condition, can also be seen at the 0.2-s ISI. At the 1-s ISI, omission responses are observable over the right hemisphere, although weaker, and with a somewhat smaller effect of attention. Tone responses, unlike omission responses, are larger with longer ISIs and are usually stronger over the hemisphere contralateral to the stimuli. Responses to tones-after-omissions are larger than responses to tones preceded by other tones. Fig. 2B shows, for all subjects, the largest omission responses over each hemisphere at the 0.5-s ISI. All subjects, except S9, show clear omission responses ŽSubject 1 only over the right hemisphere., although with substantial interindividual variability in response morphology. For most subjects, the responses are bilateral with no
Fig. 1. 122-channel MEG responses of Subject 5 to tones Žleft, n s 1400. and omissions Žright, n s 100.. In each channel pair, the upper trace shows the latitudinal and the lower the longitudinal field derivative. The stimuli were presented to the left ear with ISIs of 0.2 s Ždashed line., 0.5 s Žsolid thick line. and 1 s Žsolid thin line.. Typical P60m, N100m and P200m responses to tones over the contralateral hemisphere are shown enlarged in inset b. Inset a shows the enlarged ipsilateral responses. The same channel pairs are shown enlarged in insets c and d for the omission responses.
T. Raij et al.r Brain Research 745 (1997) 134–143
difference between left and right ear stimuli. The responses are clearly stronger in the attend than in the ignore condition for all but one subject ŽS1; no ignore data were obtained for S4.. N100m responses were identified in eight of the nine subjects, bilaterally in five. N100m amplitude and latency were analyzed only for the left-ear 0.5-s ISI. N100m was, on average, 12 ms earlier Žnot significant. and 39% stronger Ž F Ž1,7. s 8.0 P - 0.05. over the right, contralateral hemisphere than over the left. N100m was 23% Ž F Ž1,7. s 10.7; P - 0.05. larger when preceded by omissions than by tones, but the peak latency was the same in both categories. Neither amplitude nor latency was significantly affected by attention.
137
Omission responses generally consisted of a transient deflection reaching its first peak around 145–195 ms after the predicted time of stimulus occurrence, followed by a broader, sometimes sustained response until the onset of the next stimulus Žcf. Fig. 2B.. The omission response amplitude, unlike the tone response amplitude, was heavily dependent on attention, being on average 60% weaker in the ignore than attend condition Ž F Ž1,4. s 239; P - 0.001.. It did not differ according to ear of stimulation or hemisphere, but was on average 33 ms earlier with the 0.2-s than the 0.5-s ISI Ž t Ž7. s y3.7; P - 0.01.. Omission response morphology was replicable across days. There was some amplitude variability across sessions, but peak latencies were quite consistent.
Fig. 2. A: responses Žvector-sums of channel pairs. of Subject 5 over the left and right hemispheres ŽLH, RH. to tones, omissions and tones-after-omissions, for left and right ear stimulation ŽLE, RE., in attend and ignore conditions Žatt, ign.. Responses are shown at the 1-s Župper., 0.5-s Žmiddle. and 0.2-s Žlower. ISI. Data are displayed as continuous, although averages were calculated separately for the three categories of tones, omissions, and tones-after-omissions. The dashed vertical lines mark the time of the expected stimulus occurrence. B: omission responses Žvector-sums of channel pairs. of all subjects at the 0.5-s ISI.
138
T. Raij et al.r Brain Research 745 (1997) 134–143
3.2. Source locations Omissions produced largest deflections over the same cerebral regions as tones, and the field patterns evoked by omissions closely resembled those evoked by tones. We thus first examined the extent to which the generator of N100m to tones could also explain the omission responses. Fig. 3A shows, for one subject, the magnetic field patterns for N100m at the timepoints used to search for the ECDs, and the locations of these dipoles superimposed on the subject’s MRI. These ECDs were then used to explain the omission response by keeping the dipole locations constant but allowing the orientations to vary over time. In Fig. 3B, the predicted waveforms from this 2-dipole model are superimposed on the measured 122-channel omission responses. Over both midtemporal areas, the model predicts accurately the omission responses, as well as responses to tones-after-omissions. However, the model does not ac-
count for parts of the omission response over the right anterotemporal area. A third source, in the right superior temporal sulcus ŽSTS. adequately accounts for the remaining part of the response. Fig. 3C shows the magnetic field pattern used to estimate the location of this source and the location of the source superimposed on the subject’s MRI. This approach demonstrates that a major part of the omission response can be explained by sources in the supratemporal auditory cortices, but that additional areas may also be activated by omissions. As a more conventional approach, we modelled directly the sources of omission responses. In all subjects, the main sources were found in the supratemporal cortices, typically bilaterally. Fig. 4 shows, for the eight subjects producing omission responses, the N100m sources and examples of omission response sources to both left and right ear stimuli. The sources of the omission response were on average within 8 " 4 mm Žmean " S.D.. from the N100m sources
Fig. 3. A: magnetic field patterns Žstep 20 fT. of N100 m to attended left-ear tones ŽISI 0.5 s. and the corresponding ECDs Žarrows.. The N100m ECDs, superimposed on the subject’s MRI, are also shown. B: measured 122-channel omission responses Žthin lines. superimposed on signals predicted by bilateral N100m generators, which were allowed to change orientation as a function of time Žthick lines.. The inserts show one channel pair, with measured signals superimposed on predictions for the 2- and 3-dipole models. C: field patterns of omission responses at 465 ms and the STS source superimposed on the subject’s MRI.
T. Raij et al.r Brain Research 745 (1997) 134–143
in both hemispheres, without any systematic difference in any direction. Although supratemporal sources often accounted for the major part of the omission response, additional sources were needed to fully explain the whole-head data. In six subjects additional sources were identified within the STS Žsee Fig. 6., as was shown for Subject 6 in Fig. 3; each of these sources improved the goodness-of-fit of the model
139
by at least 10% Žsee below.. One subject showed bilateral STS activation, four had a source only in the right STS and one only in the left. In addition to temporal-lobe sources, frontal sources were evident in all subjects in at least one condition. Fig. 5 shows the 122-channel responses of Subject 2 along with the predicted signals from a 2-dipole model, with one dipole in each temporal cortex. Over the left hemisphere,
Fig. 4. Supratemporal generators of responses to tones Žspheres. and omissions Žipsilateral stimuli: triangles; contralateral stimuli: squares. of all subjects. Examples shown are the sources of clearest omission responses to left and right ear stimuli in each hemisphere of each subject. The dipoles are visible through F 20 mm in the MRIs. Subject 4 produced atypical N100m responses bilaterally, Subjects 7 and 8 over the left hemisphere, resulting in rejection of this part of their data.
140
T. Raij et al.r Brain Research 745 (1997) 134–143
the model predicts well the measured data. In contrast, the right-hemisphere signals are not explained by the temporal-lobe sources alone. The addition of a third source clearly improves the explanation. This source was identified on the basis of a dipolar field pattern over the anterior right hemisphere at 140 ms Žsee bottom right corner of Fig. 5., and it agrees with a source in the right posterolateral frontal cortex. Similar source locations were evident also at the two other ISIs for this subject. In all subjects, the major sources outside the temporal lobes were located in posterolateral frontal cortex, slightly anterior to the central sulcus. Fig. 6 shows these sources in both hemispheres of all subjects, superimposed on a schematic sagittal brain slice. Frontal activity was observed in 25 measurements in the right hemisphere and in 11 in the left. Four subjects had bilateral frontal sources. The source locations were more variable across subjects in the left than in the right hemisphere. Fig. 6 also shows the STS sources in all subjects. STS sources were observed in
Fig. 6. All frontal and STS sources from all subjects, displayed on a schematic MRI slice separately for the left and the right hemisphere. All subjects had at least one frontal source. The individual source locations in this figure were determined by finding the Sylvian fissure and central sulcus in each brain separately, and superimposing the sources on the schematic brain slice with respect to these landmarks. In four subjects, the central sulcus was localized by finding the generator of the 20-ms magnetic response to median nerve stimulation Žknown to be in primary somatosensory cortex, embedded within the central sulcus.; in the remaining four subjects, the central sulcus was localized on anatomical grounds.
Fig. 5. Measured Žthin lines. and predicted Žthick lines. 122-channel omission responses of Subject 2 to left-ear stimuli in the attend condition, 0.2-s ISI. The predictions in the whole-head picture were derived from a model consisting of two temporal-lobe dipoles, one in each hemisphere. One right frontal channel is shown enlarged on the left. The panel in the middle shows the measured and predicted responses from the same channel when a third, frontal dipole was added to the model. The magnetic field pattern Žstep 20 fT. over the right hemisphere at the time of the frontal source estimation is shown on the right.
two measurements in the left hemisphere and seven in the right. As suggested already by the 60% smaller amplitudes of the omission responses in the ignore than in the attend condition Žsee above., the activation strengths of all source areas were significantly affected by attention, being on average 70% weaker in the ignore than in the attend condition Žpaired t-tests for each area separately P - 0.01, collapsed across 0.5-s and 0.2-s ISIs, hemisphere and stimulated ear.. The latencies and amplitudes did not differ between contra- and ipsilateral supratemporal responses, nor between contra- and ipsilateral frontal responses Žpaired t-tests, collapsed across ear and 0.5-s and 0.2-s ISI.. The mean Ž"S.D.. latency of maximum activation in supratemporal and frontal areas was approximately the
T. Raij et al.r Brain Research 745 (1997) 134–143
same Ž180 " 80 ms and 190 " 40 ms, collapsed across 0.5-s and 0.2-s ISIs, hemisphere and stimulated ear.; activity of the superior temporal sulcus source peaked later Ž350 " 115 ms..
4. Discussion Our results show that the absence of a tone in an otherwise regular sound sequence may activate the human auditory cortex bilaterally. This activation is not caused by afferent input from the sensory receptors but rather reflects an internal expectation, built up over the prior pattern of stimulation, that a stimulus should occur. When this expectancy is not met, a response is produced signalling the deviation from the predicted event. The response is tightly time-locked to the expected stimulus occurrence, illustrating the ability of auditory cortex to model the temporal input pattern. The response is also heavily dependent on attention, indicating that the tuning of auditory cortex to the temporal pattern of stimulation is an active process. The main generators of the omission response were in the auditory areas of the supratemporal cortex in the immediate vicinity of those generating responses to tones. Tones are known to elicit larger and earlier responses in the contralateral than the ipsilateral hemisphere w13,21,27,29x. Omissions, however, elicited activity of the same magnitude and latency in the supratemporal cortices of both hemispheres, suggesting that the expectancy of future stimuli is not ear specific. Although the main generators of the omission response were in the supratemporal cortex, omissions also elicited activity within STS in six out of nine subjects. Activation of the supratemporal and STS cortex by infrequent sound omissions has recently been suggested in a combined MEG and fMRI study w23x. STS is known to contain both auditory and modality non-specific association cortex w7,17x. Auditory association cortex in the STS is involved in auditory short term memory w6x: bilateral ablations of the lateral surface of the superior temporal gyrus and the upper bank of the STS severely impair monkeys’ performance on a delayed auditory, but not visual, matching to sample task, despite intact auditory discrimination. Short term memory is certainly necessary for forming expectations of future stimuli based on prior input patterns. Outside the temporal lobes, tone omissions activated posterolateral frontal cortex in all subjects. Supratemporal auditory cortex and association cortex in the STS have connections to frontal association areas w17,26x and the lateral frontal cortex is an important node in the general neural network subserving attention w22x. Thus, the frontal areas activated by omissions in our study may be related to activation of general attention and target detection systems. In EEG recordings, stimulus omissions typically elicit a vertex-negative response at about 200 ms ŽN2., followed by a vertex-positive P3 deflection at 300–1000 ms w31–35x.
141
The generators producing the N2 response have been related both to modality-specific processing and to more general orienting reactions. The generators of the P3 response also probably include modality-specific and nonspecific brain regions w4,8,9x. The source areas found in the present study are not necessarily directly related to generators of the N2 and P3 responses because of the well-known different sensitivities of the EEG and MEG to tangentially vs. radially oriented currents w11x. In two of our subjects, 15-channel EEG scalp mapping Žreferenced to nose. was performed simultaneously with the MEG recordings in the 0.5-s ISI condition. When attending to the stimuli, one subject showed a large P3 Žpeak at 355 ms. only, while the other produced both N2 Ž195 ms. and P3 Ž360 ms. deflections. None of these responses coincided with any major deflections in the MEG responses or with peaks in the source amplitude waveforms. It thus seems that the electric N2–P3 complex receives major contributions from radially oriented currents which are silent in MEG recordings, and that its major generator areas differ from those found in our MEG recordings. However, the N2–P3 complex most likely receives some contribution also from the sources identified in the present study. The observed supratemporal and STS sources would produce EEG signals broadly distributed around vertex: due to the main current directions in these areas, the supratemporal sources would produce a vertexnegative deflection which might contribute to the early phase of the N2 response, whereas the STS source could contribute to the P3 response. The frontal sources, due to their current direction, could further add to the parietal positivity. In the present study, the morphology of the omission response was quite variable across subjects. This could reflect individual differences in source orientations. Since MEG is mainly sensitive to tangential currents, differences in radial versus tangential source orientation across subjects would appear as differences in response strength. Response variation is also likely to be related to individual differences in the accuracy of estimating time intervals. MMRs, which signal the automatic detection of a change in some feature of a repetitive stimulus w24x, have been elicited by changes in temporal rhythm, when, e.g., a stimulus occurs earlier than expected w16,20,25x. MMRs are also produced by decreases in stimulus intensity. A stimulus omission can be interpreted as a maximum intensity decrease and thus should produce a MMR. However, it is unlikely that the response elicited by stimulus omissions reflects the same brain processes that generate MMRs. First, MMRs are generally considered to be automatic processes, occurring irrespective of attention whereas omission responses strongly depended on attention Žsee also w18x.. Secondly, the generator of the MMR in the supratemporal cortex is typically located about 1 cm anterior to that of N100m, whereas no such difference was observed between the generators of the omission response
142
T. Raij et al.r Brain Research 745 (1997) 134–143
and N100m. Finally, bilateral frontal activity elicited by omissions in this study has not been reported in recordings of MMRs, nor did we observe right inferior parietal lobe activity to stimulus omissions, as has been reported for MMRs to changes in frequency, duration, or ISI w20x. Omission responses show that a template is formed in sensory cortex of expected events. Such a template can act as a cortical filter through which incoming sensory information is evaluated in an attention-dependent way. Our data also indicate that this processing is right-hemisphere dominant, as has been previously suggested for automatic change detection w20x. Acknowledgements A preliminary report of these data has appeared in abstract form w28x. This study has been financially supported by the Academy of Finland, the Sigrid Juselius ´ Foundation, the Foundation for Medicine in Finland and the BIRCH Large-Scale Facility ŽEU’s Human Capital and Mobility Programme. in the Low Temperature Laboratory of the Helsinki University of Technology. The MRIs were recorded at the Radiology Department of the Helsinki University Central Hospital.
w10x
w11x
w12x
w13x
w14x
w15x
w16x
w17x
w18x
References w19x w1x Ahonen, A.I., Hamalainen, M.S., Kajola, M.J., Knuutila, J.E.T., ¨ ¨¨ Laine, P.P., Lounasmaa, O.V., Parkkonen, L.T., Simola, J.T. and Tesche, C.D., 122-channel SQUID instrument for investigating the magnetic signals from the human brain, Physica Scripta, T49 Ž1993. 198–205. w2x Alain, C., Richer, F., Achim, A. and Saint Hilaire, J.-M., Human intracerebral potentials associated with target, novel and omitted auditory stimuli, Brain Topogr., 1 Ž1989. 237–245. w3x Barlow, J.S., Some observations on the electrophysiology of timing in the nervous system, Electroenceph. Clin. Neurophysiol., 27 Ž1969. 545. w4x Baudena, P., Halgren, E., Heit, G. and Clarke, J.M., Intracerebral potentials to rare target and distractor auditory and visual stimuli. III. Frontal cortex, Electroenceph. Clin. Neurophysiol., 94 Ž1995. 251–264. w5x Bullock, T.H., Karamursel, S., Achimowicz, J.Z., McClune, M.C. ¨ and Basar-Eroglu, C., Dynamic properties of human visual evoked and omitted stimulus potentials, Electroenceph. Clin. Neurophysiol., 91 Ž1994. 42–53. w6x Colombo, M., D’Amato, M.R., Rodman, H.R. and Gross, C.G., Auditory association cortex lesions impair auditory short-term memory in monkeys, Science, 247 Ž1990. 336–338. w7x Galaburda, A. and Sanides, F., Cytoarchitectonic organization of the human auditory cortex, J. Comp. Neurol., 190 Ž1980. 597–610. w8x Halgren, E., Baudena, P., Clarke, J.M., Heit, G., Liegois, C., ´ Chauvel, P. and Musolino, A., Intracerebral potentials to rare target and distractor auditory and visual stimuli. I. Superior temporal plane and parietal lobe, Electroenceph. Clin. Neurophysiol., 94 Ž1995. 191–220. w9x Halgren, E., Baudena, P., Clarke, J.M., Heit, G., Marinkovic, K., Devaux, B., Vignal, J.-P. and Biraben, A., Intracerebral potentials to rare target and distractor auditory and visual stimuli. II. Medial,
w20x
w21x
w22x
w23x
w24x w25x
w26x
w27x
w28x
lateral and posterior temporal lobe, Electroenceph. Clin. Neurophysiol., 94 Ž1995. 229–250. Hamalainen, M., Anatomical correlates for magnetoencephalogra¨ ¨¨ phy: integration with magnetic resonance images, Clin. Phys. Physiol. Meas., 12 Suppl. A Ž1991. 29–32. Hamalainen, M., Hari, R., Ilmoniemi, R.J., Knuutila, J. and Lounas¨ ¨¨ maa, O.V., Magnetoencephalography – theory, instrumentation and applications to noninvasive studies of the working human brain, ReÕ. Mod. Phys., 65 Ž1993. 413–497. Hari, R., The neuromagnetic method in the study of the human auditory cortex. In F. Grandori, M. Hoke and G.L. Romani ŽEds.., Auditory EÕoked Magnetic Fields and Potentials, AdÕances in Audiology, Vol. 6, Karger, Basel, 1990, pp. 222–282. Hari, R. and Makela, ¨ ¨ J.P., Modification of neuromagnetic responses of the human auditory cortex by masking sounds, Exp. Brain Res., 71 Ž1988. 87–92. Hari, R., Kaila, K., Katila, T., Tuomisto, T. and Varpula, T., Interstimulus-interval dependence of the auditory vertex response and its magnetic counterpart: Implications for their neural generation, Electroenceph. Clin. Neurophysiol., 54 Ž1982. 561–569. Hari, R., Hamalainen, M., Ilmoniemi, R., Kaukoranta, E., ¨ ¨¨ Reinikainen, K., Salminen, J., Alho, K., Naatanen, R. and Sams, M., ¨¨ ¨ Responses of the primary auditory cortex to pitch changes of tone pips: neuromagnetic recordings in man, Neurosci. Lett., 50 Ž1984. 127–132. Hari, R., Joutsiniemi, S.-L., Hamalainen, M. and Vilkman, V., ¨ ¨¨ Neuromagnetic responses of human auditory cortex to interruptions in a steady rhythm, Neurosci. Lett., 99 Ž1989. 164–168. Jones, E.G. and Powell, T.P.S., An anatomical study of converging sensory pathways within the the cerebral cortex of the monkey, Brain, 93 Ž1970. 793–820. Joutsiniemi, S.-L. and Hari, R., Omissions of auditory stimuli may activate frontal cortex, Eur. J. Neurosci., 1 Ž1989. 524–528. Klinke, R., Fruhstorfer, H. and Finkenzeller, P., Evoked responses as a function of external and stored information, Electroenceph. Clin. Neurophysiol., 25 Ž1968. 119–122. Levanen, S., Ahonen, A., Hari, R., McEvoy, L. and Sams, M., ¨ Deviant auditory stimuli activate human left and right auditory cortex differently, Cereb. Cortex, 6 Ž1996. 288–296. McEvoy, L., Hari, R., Imada, T. and Sams, M., Human auditory cortical mechanisms of sound lateralization: II. Interaural time differences at sound onset, Hear. Res., 67 Ž1993. 98–109. Mesulam, M.-M., Large-scale neurocognitive networks and distributed processing for attention, language and memory, Ann. Neurol., 28 Ž1990. 597–613. Miyauchi, S., Takino, R., Sasaki, Y., Putz, ¨ B. and Okamura, H., Visualization of information processing in the human brain: recent advances in MEG and functional MRI, 10th Tokyo Institute of Psychiatry International Symposium, Book of Abstracts Ž1995. 72– 73. Naatanen, R., Attention and Brain Function, Lawrence Erlbaum ¨¨ ¨ Associates, Hillsdale, New Jersey, 1992. Nordby, H., Roth, W.T. and Pfefferbaum, A., Event-related potentials to time-deviant and pitch-deviant tones, Psychophysiology, 25 Ž1988. 249–261. Pandya, D.N., Hallett, M. and Mukherjee, S.K., Intra- and interhemispheric connections of the neocortical auditory system in the rhesus monkey, Brain Res., 14 Ž1969. 49–65. Pantev, C., Lutkenhoner, B., Hoke, M. and Lehnertz, K., Compari¨ ¨ son between simultaneously recorded auditory-evoked magnetic fields and potentials elicited by ipsilateral, contralateral and binaural tone burst stimulation, Audiology, 25 Ž1986. 54–61. Raij, T., Makela, ¨ ¨ J.P., McEvoy, L. and Hari, R., Human auditory cortex is activated by omissions of auditory stimuli, 10th International Conference on Biomagnetism (Biomag96) Abstracts Ž1996. 83.
T. Raij et al.r Brain Research 745 (1997) 134–143 w29x Reite, M., Zimmerman, J.T. and Zimmerman, J.E., Magnetic auditory evoked fields: Interhemispheric asymmetry, Electroenceph. Clin. Neurophysiol., 51 Ž1981. 388–391. w30x Rogers, R.L., Papanicolaou, A.C., Baumann, S.B. and Eisenberg, H.M., Late magnetic fields and positive evoked potentials following infrequent and unpredictable omissions of visual stimuli, Electroenceph. Clin. Neurophysiol., 83 Ž1992. 146–152. w31x Ruchkin, D.S., Sutton, S., Munson, R., Silver, K. and Macar, F., P300 and feedback provided by absence of the stimulus, Psychophysiology, 18 Ž1981. 271–282. w32x Simson, R., Vaughan, H.G. and Ritter, W., The scalp topography of potentials associated with missing visual or auditory stimuli, Electroenceph. Clin. Neurophysiol., 40 Ž1976. 33–42.
143
w33x Simson, R., Vaughan, H.G. and Ritter, W., The scalp topography of potentials in auditory and visual discrimination tasks, Electroenceph. Clin. Neurophysiol., 42 Ž1977. 528–535. w34x Sutton, S., Braren, M., Zubin, J. and John, E.R., Evoked potential correlates of stimulus uncertainty, Science, 150 Ž1965. 1187–1188. w35x Sutton, S., Tueting, P., Zubin, P. and John, E.R., Information delivery and the sensory evoked potential, Science, 155 Ž1967. 1436–1439. w36x Verleger, R., The instruction to refrain from blinking affects auditory P3 and N1 amplitudes, Electroenceph. Clin. Neurophysiol., 78 Ž1991. 240–251.