Evoked potentials to auditory movement sensation in duplex perception

Clinical Neurophysiology 114 (2003) 1316–1331 www.elsevier.com/locate/clinph

Evoked potentials to auditory movement sensation in duplex perception Ilan Laufer, Hillel Pratt* Evoked Potentials Laboratory, Technion – Israel Institute of Technology, Gutwirth Building, 32000 Haifa, Israel Accepted 13 March 2003

Abstract Objective: The purpose of this study was to examine the processing of auditory movement sensation accompanying duplex perception in binaural hearing. Methods: Stimuli were formant transitions (presented to the front, left or right of the subject) and base (presented to the front), that fused to result in vowel –consonant – vowel (V– C– V) sequences /aga/ and /ada/. An illusion of auditory movement (duplex sensation) accompanied the fusion of these V – C– V sequences when the spatial locations of the formant transitions and base were different. Ten right-handed, adult, native Hebrew speakers discriminated each fused stimulus, and the brain potentials associated with performance of the task were recorded from 21 electrodes. The processing of auditory movement was studied by a factorial design (ANOVA) and statistical non-parametric mapping (SnPM) of low resolution electromagnetic tomography (LORETA) images of the net-fusion response. Brain regions implicated in auditory movement processing were expected to be associated with the lateralized formant location, which gave rise to duplex perception. In addition, the time-course of significant activation in brain areas that differentiated between fusion conditions was determined. Results: The posterior parietal, anterior cingulate and premotor cortices were found to be implicated in duplex processing. Auditory cortex involvement was also evident, and together with the latter two brain regions was affected by right-ear advantage. Conclusions: Duplex perception resulting from fusion of spatially separate sounds forming an auditory object results in activation of a network of brain regions reflecting enhanced allocation of attention and the effect of language processing. q 2003 International Federation of Clinical Neurophysiology. Published by Elsevier Science Ireland Ltd. All rights reserved. Keywords: Motion perception; F-complex; Right parietal cortex; Cingulate gyrus; Low-resolution electromagnetic tomography

1. Introduction Auditory spatial processing relies on binaural and monaural cues that are essential for the perception of sound movement. The involvement of the auditory cortex in sound motion perception, its relationships with other brain areas as well as the specificity of these areas to auditory movement perception are still debatable. Some animal lesion studies corroborate the role of the auditory cortex in sound localization (Cranford et al., 1971, Heffner and Masterton, 1975; Jenkins and Merzenich, 1984) and perception of sound movement (Altman and Kalmykova, 1986). However, evidence for the involvement of the auditory cortex was not always found (e.g. Weeks et al., 1999), while sound localization deficits following lesions in cortical areas other than the auditory cortex, in frontal and parietal regions (De Renzi et al., 1984, 1989), * Corresponding author. Tel.: þ972-4-8292-321; fax: þ 972-4-8229-949. E-mail address: [email protected] (H. Pratt).

suggest their possible involvement in sound motion perception. Recently, converging data from positron emission tomography (PET) and functional magnetic resonance imaging (fMRI) studies (Griffiths et al., 1998, 2000) have demonstrated that sound motion perception relies on a network of areas which includes regions beyond the primary auditory cortex comprising the right parietal cortex, the bilateral premotor areas and the frontal eye fields. Results from a recent magnetoencephalography (MEG) study (Xiang et al., 2002) corroborate the previous results, that implicate a network of areas in processing sound motion. In that study, the network was suggested to include the right parietal cortex and the left and right superior temporal cortices. Evidence from other PET studies (Bushara et al., 1999; Weeks et al., 1999) has demonstrated that activation in posterior parietal, lateral prefrontal and inferior temporal regions was also associated with auditory spatial processing. A comparison of areas activated during auditory and visual spatial processing found both

1388-2457/03/$30.00 q 2003 International Federation of Clinical Neurophysiology. Published by Elsevier Science Ireland Ltd. All rights reserved. doi:10.1016/S1388-2457(03)00083-X

I. Laufer, H. Pratt / Clinical Neurophysiology 114 (2003) 1316–1331

modality-specific and multimodal spatial localization areas. Modality-specific areas comprised adjacent areas of the superior parietal lobule (BA 7), and the middle frontal gyrus (BA 9). In addition, the inferior temporal and frontal gyri (BA 20, 47) were found to be auditory specific. All of these brain regions were bilaterally activated. Multimodal spatial localization areas included the inferior parietal lobule (BA 40) activated bilaterally, the right medial frontal cortex (BA 6) as well as the right inferior temporal cortex (BA 20). Additional supra-modal activation areas included the right thalamus and the right and left cerebellum. It is noteworthy that no involvement of the auditory cortex was found in movement processing. Although modality-specific and multimodal spatial localization areas involve bilateral activation, converging evidence from both PET and fMRI studies suggests a crucial role for the right parietal association cortex during auditory spatial processing. These studies found the involvement of areas beyond the auditory cortex which are all situated in the right hemisphere: the right posterior parietal cortex (Griffiths et al., 1998; Bushara et al., 1999; Weeks et al., 1999), the right posterior planum temporale (Baumgart et al., 1999), and the right insula (Griffiths et al., 1994). This evidence is in accord with the role attributed to the posterior parietal cortex as an interface between sensory and motor structures and as integrating visual, auditory and vestibular signals in the representation of abstract spatial information (Andersen, 1995). Moreover, unilateral neglect is often among the sequelae of right parietal damage causing deficits in the localization of sounds in the left hemifield (Petersen et al., 1994; Griffiths et al., 1996; Soroker et al., 1997). Neurophysiological evidence suggests that the right hemisphere carries a primary role in allocating attention to both hemifields, whereas the left hemisphere is responsible for controlling attention only in the right hemifield. This interhemispheric allocation of attention exists both in the visual (Corbetta et al., 1993; Mangun et al., 1994) and auditory modalities (De Renzi et al., 1989; Soroker et al., 1997). In a previous study (Laufer and Pratt, 2003) we examined the effect of sound source location on fusion of speech elements. The duplex paradigm was used to examine fusion in the context of speech, and the brain activity associated with net-fusion, which we termed the ‘F-complex’, was derived. The duplex paradigm (Mann and Liberman, 1983) involves presentation of formant transitions that are lateralized with respect to the base with which they fuse to result in a simultaneous perception of a phonetic entity and a chirp-like echo. An illusion of sound movement to the left or right accompanies the echo-sensation associated with the lateralized formants. In addition to the behavioral perception of movement, an additional finding associated our previous study with auditory movement processing. Apart from the ‘C-potentials’ (comprising of the N1 –P2 – N2b complex elicited in response to change) by fusion, a lateral temporal contribution, the

1317

T-complex (Wolpaw and Penry, 1975, 1977), was also evident at temporal sites. The T-complex is conventionally defined as a positive – negative (Ta –Tb) response around 100 and 150 ms, respectively. It was shown to be larger contralateral to the stimulus and to demonstrate right-sided preponderance (Wolpaw and Penry, 1975, 1977). However, other studies showed that the conventional definition of the complex is rather loose. For example, Connolly (1993) used a different peak amplitude measuring technique than that used by Wolpaw and Penry (1977) and found that the positive component, the Ta, tended to be larger ipsilateral to the stimulus rather than contralaterally. Furthermore, the negativity, and not the positivity, of the T-complex, was shown to be larger on the right in the context of instrumental tones (Jones and Byrne, 1998; Jones et al., 1998; Jones and Perez, 2001). Abnormalities of this negativity, measured by the N1c component (an equivalent of the Tb) were found to be associated with auditory associative cortex dysfunction in children with autism (Bruneau et al., 1999) and with language impairment (Tonnquist-Uhle´n, 1996). The positive component of the complex, Ta, was also found to be associated with language impairment in the latter study. In our previous study (Laufer and Pratt, 2003) the T-complex was characterized by a negative – positive– negative complex, and resembled the F-complex. In addition, the T-complex was symmetrically distributed about the midline. This contrasts with some previous results that described the T-complex as having prominence over the contralateral hemisphere as well as being larger over the right hemisphere (Jones and Byrne, 1998; Jones et al., 1998). However, results from a study that manipulated the interaural time differences in dichotic presentation of noise, resulting in an image shift (Jones et al., 1991), may reconcile these apparent contradictory results. In that study, a symmetrically distributed temporal negative potential was elicited in response to the sensation of the image shift. Thus, the symmetrically distributed T-complex found in our previous study suggests that perception of auditory movement affected the processing of fusion. In the present study subjects performed an ‘Oddball’ paradigm in which they had to discriminate between two vowel –consonant – vowel (V – C –V) sequences /ada/ (deviant) and /aga/ (frequent). Each V – C –V was a result of fusion of speech elements (base and formant transitions extracted from natural /ada/ and /aga/) that were unintelligible when presented alone. The base was always positioned directly in front of the subject’s head while the formant transitions were either lateralized to the left (leftfusion), to the right (right-fusion) or spatially coincided with the base (front-fusion). When the formants were lateralized, the echo-sensation created the illusion of movement towards the spatial position of the formants, whereas when the formants (of either /ada/ or /aga/) coincided with the base (front-fusion), no movement sensation was perceived.

1318

I. Laufer, H. Pratt / Clinical Neurophysiology 114 (2003) 1316–1331

Auditory movement arising from the duplex effect created an echo-sensation that accompanied the perception of the two V – C – V sequences in the lateralized fusion conditions, left- and right-fusion. In contrast, in frontfusion, there was no duplex and thus the perception of the sequence was not accompanied by an echo-sensation. This echo-sensation was in itself an attention-catching event and thus the response to the perception of illusory movement was confounded with novelty attentional effects reflected in the FN2b – FP3b complex (e.g. Novak et al., 1990). Thus, the complex of the net-response to fusion is confounded with an attention effect. By assigning equal probability to left, right and front formants, their novelty was the same. To control for the effect of novelty (not included in this report), we used the ‘Oddball’ paradigm where assigning the /ada/ stimuli to the infrequent category and /aga/ as the frequent. The primary aim of the study was to examine whether movement sensation in the duplex phenomenon is associated with activation of specific brain regions comprising the network implicated in sound motion processing. Specifically, both right-fusion and left-fusion were expected to be associated with processing of auditory movement, since the lateralized formants in each condition created an illusion of image shift towards the respective side, while in frontfusion, where the formants coincided with the base, no image shift was expected to occur. Enhanced activation was expected to occur with fusion involving lateralized formants in all or part of the following areas: the posterior parietal (superior and inferior parietal lobules), prefrontal, and inferior temporal cortex. A moving sound is characterized by its velocity (in the illusory movement perception) and its physical attributes such as spectral content and intensity. Accordingly, in our study both the auditory cortex and the right parietal cortex can be expected to be activated simultaneously (Xiang et al., 2002): motion cues created by the duplex percept are passed to the right parietal cortex when fusion occurs, while at the same time, acoustic features of fusion (the acoustic-change response) are being processed by the auditory cortex. Moreover, the response from the right parietal cortex may be related to the allocation of attention to the hemispace in which movement (an attention-capturing event) occurs. Therefore, we expect that the highest level of activity in the right parietal cortex will occur at the latency range of the complex associated with the duplex perception. This complex is considered to be part of the larger MMN – N2b – P3a complex, assumed to reflect stimulus-change detection and involuntary attention switching toward a deviant event (Novak et al., 1990; Pekkonen et al., 1996; Tervaniemi et al., 1997; Ritter et al., 1999). The F-complex can be expected to be followed by a positivity (a positive component or complex) associated with the detection of an unexpected event, i.e. the occurrence of the deviant sequence. The secondary aim of the study was to examine the time-

course of activation in specific brain regions associated with sound motion processing.

2. Methods The details of the experiment were reported in an earlier study, in which the F-complex was initially extracted and described (Laufer and Pratt, 2003). 2.1. Subjects Ten right-handed, adult, native Hebrew speakers (4 women and 6 men) participated in the study. Their ages ranged between 21 and 30 years and none reported hearing impairment or neurological deficits. 2.2. Stimuli In order to extract the elements of fusion (base and formant transitions), a selective filtration procedure was applied separately on two V –C – V sequences /ada/ and /aga/, which were recorded from a male native Hebrew speaker, using a 44.1 kHz sampling rate and 16 bit resolution. To create the formant transitions, the /ada/ and /aga/ sequences were high-pass filtered in a time window of 90 ms following the release burst in each sequence (250 ms after /ada/ onset, 230 ms after /aga/ onset). The cut-off frequency was 900 Hz with a slope of 48 dB/ octave. The base was created using the same filtration parameters, and was the result of low-pass selective filtration applied on the /ada/ sequence, in the specific time window of 90 ms following the release burst. After filtration, the base comprised of the initial vowel /a/, the voiced silent period preceding the stop /d/, the complete F1, and the steady-state portion of F2, F3 and F4, respectively. The base was truncated to 460 ms duration (see Table 1, for the frequency parameters of the second vowel of the base as well as of each of the formants, respectively). The base and formant transitions were placed in a 3-dimensional (3D) virtual reality room using a human head-related transfer function (Tucker – Davis Technologies, Gainesville, FL, USA, System II). The software emulates perception of a sound source in 3D space. The base was kept stationary in front of the subject, while each of the formant transitions, extracted from /da/ and /ga/, either spatially coincided with the base or was placed to the left or to the right of the subject. All 3 sound source locations were positioned at a virtual distance of 1.2 m and level with the subjects’ ears. Prior to its mixing with the base, the onset of formant transitions was aligned with the onset of the /d/ burst in the original /ada/ sequence. Fusions of base with two formant transitions (/da/ and /ga/) at 3 spatial positions (left, right, front) comprised the 6 stimuli that were

I. Laufer, H. Pratt / Clinical Neurophysiology 114 (2003) 1316–1331

1319

Table 1 Base and formants frequency parameters (a) Base: second-vowel formants parameters. Frequencies are given in Hz Average F0

F1

110

Onset 250

Steady state 600

F2

F3

Central frequency 1370

Central frequency 2370

(b) Beginning and ending frequencies (Hz) of /da/ and /ga/ formant transitions /da/

/ga/

F2 Beginning 1615

F3 End 1460

Beginning 2370

F2 End 2480

presented to the subjects. The 3 fusion locations were termed: front-fusion, left-fusion and right-fusion. 2.3. Procedure An Oddball paradigm was used to record the responses to the /ada/ and /aga/ sequences. Rare /ada/ (probability of 17%) stimuli were embedded among frequent standard /aga/ stimuli (probability of 83%). The formant locations were balanced within each set of standard and frequent stimuli, and the presentation order was randomized. Stimuli were presented at 80 dB sound pressure level (SPL) by earphones. The interstimulus interval varied randomly between 1.8 and 2.2 s. In 4 additional Oddball paradigms the separate base and each of the /da/ formants (front, left and right) replaced one of the /ada/ sequences as follows: the separate /da/ formants (left, front and right) replaced the corresponding fused /ada/ sequence, while the base replaced the frontfusion resultant /ada/. The subject’s task was to push one button in response to the occurrence of the rare stimulus (/ada/) and another in response to the standard stimulus (/aga/). On hearing the separate base or formants, the subject was told to press the button assigned for /ada/. 2.4. Electrophysiological recordings The ERPs were recorded from an electrode cap with tin electrodes placed according to the 10 –20 system (ElectroCap International Inc., Eaton, OH, USA). Activity was recorded (Ceegraph IV Biologic Systems Corp. IL, USA) from the following 19 locations: Fp1, Fp2, F7, F3, Fz, F4, F8, T3, C3, Cz, C4, T4, T5, P3, Pz, P4, T6, O1, O2. Three additional electrodes external to the cap were used: two were attached to the left and right mastoids (A1 and A2), and one below the left eye, referenced as Fp1, to control for eye movements (EOG). In total, EEG was recorded from 21 electrodes. All EEG electrodes were referenced to the center of the chin. An electrode on the left forearm served as

Beginning 1590

F3 End 1280

Beginning 2175

End 2160

ground. Impedance at each electrode was maintained below 5 kV. EEG (£ 100,000) and EOG (£20,000) channels were amplified, digitized with a 12 bit A/D converter at a rate of 256 samples/s, filtered (0.1 – 100 Hz, 6 dB/octave) and stored for off-line analysis. 2.5. ERP derivation EEG processing involved segmentation (2 300 to 1500 ms, relative to sequence onset), eye movement correction (Attias et al., 1993) and artifact rejection (^ 150 mV). Average waveforms (2 100 to 550 ms, relative to fusion onset) for the /ada/ duplex stimuli and for each of the separate fusion elements were computed per subject as well as across subjects to obtain grand mean waveforms. After averaging, the data were low-pass filtered (FIR rectangular filter with a cut-off frequency of 14 Hz) and baseline (average amplitude across the 100 ms preceding fusion) corrected. 2.6. Difference waveform derivation The electrophysiological net-responses to the rare /ada/ fusion, the F-complex (Laufer and Pratt, 2003) were determined by subtracting the potentials to the fused sound from the sum of the potentials to transition and to base. The subtraction procedure was performed for each of the 3 spatial locations involved in the fused /ada/ stimuli: front, left and right. The resulting components, N1, P2 and N2b obtained by the subtraction procedure were termed as follows: FN1, FP2 and FN2b, respectively, where ‘F’ stands for ‘fusion’, and hence the complex comprising these components was termed as the ‘F(fusion)-complex’ (Fig. 1, top). In addition to the F-complex, a positive component that comprised of two positive peaks that followed the F-complex was also derived by the subtraction procedure and was most prominent at Fz (Fig. 1, bottom). This positive complex will be termed here as dP3, since it was derived by

1320

I. Laufer, H. Pratt / Clinical Neurophysiology 114 (2003) 1316–1331

2.7. Current density estimations and ANOVA

Fig. 1. Top: The F-complex at lateral central and temporal electrodes. The complex recorded at the temporal sites resembles that recorded at central sites both in terms of latency and polarity. The negative–positive–negative complex at the temporal sites is the T-complex, which is conventionally referred to as a negative– positive displacement. Note the double-peaked dP3, which immediately follows the complex, c.f. T4. The arrow denotes the onset of fusion. Bottom: The F-complex and the positive complex as recorded from Fz. Note the double-peaked dP3. The differences in peak amplitudes of the positive complex between fusion conditions were not significant. The latency shift associated with the second positive peak (dP470) was significant, being prolonged with front-fusion. The arrow at the beginning of the trace denotes fusion onset.

using a subtraction procedure, and is therefore a difference (d) waveform. The results of analyzing the F-complex were reported elsewhere (Laufer and Pratt, 2003). Three-way repeated measures analysis of variance (ANOVA) with Frontality (frontal, central, parieto-temporal), Laterality (left –right hemisphere, midline) and Formant location (left, front and right) was performed on the latency and peak amplitude values of both positive peaks of the dP3 component. The electrodes that were included in the ANOVA were: Fp1, Fp2, Fz, Cz, Pz, T3, C3, T4, C4. Electrodes Pz, T3 and T4 were grouped together and termed the ‘parieto-temporal’ electrodes.

Low resolution electromagnetic tomography (LORETA; Pascual-Marqui et al., 1994, 1999, Vitacco et al., 2002) is a functional brain imaging method that estimates the distribution of current density in the brain, displaying it in a 3D Talairach space. It computes current density assuming that in each voxel the current density should be as close as possible to the average current density of the neighboring voxels (smoothness assumption). LORETA was used to compute the current density values associated with each formant location and with each component of the F-complex and positive complex that followed, as detailed below. A factorial statistical design was used to assess the interaction between formant location and specific regions of interest implicated in the auditory movement perception arising from the duplex effect. To that end, average current density was determined for a time zone surrounding each of the F-complex components and the following positive complex in each formant location, and was then subjected to ANOVA with repeated measures. For each component the average current density was computed across the voxels comprising the following 7 regions of interest: BA 6 (medial frontal gyrus), BA 7 (superior parietal lobule), BA 8 (middle/medial frontal gyrus), BA 24, 32 (anterior cingulate gyrus), BA 41, 42 (superior temporal gyrus). The average current density in the auditory cortex was collapsed in the voxels comprising both BA 41 and BA 42. The average current density surrounding each component peak, in each formant location and region of interest was computed as follows: first, the grand mean averages of the different waveforms for each of the formant locations were subjected to current density analysis by LORETA, using the 21 channels of AEP (Fig. 2). Secondly, the F-complex components FN1, FP2, FN2b, and the following positive complex: dP360, dP470 were determined based on peaks of the global field power (GFP). The GFP peak for each component was determined on the basis of its proximity to the ERP peak as identified at Cz and Fz for the F-complex and positive complex components, respectively. The GFP peak latencies (from fusion onset) for each F-complex and positive complex components, were as follows: FN 1-117 ms; FP2-187 ms; FN 2b -250 ms; dP360-363 ms; dP470-453 ms. Thirdly, LORETA average pre-stimulus and post-fusion current density values were calculated separately for 3 time intervals surrounding each GFP component peak, for voxels of the specific regions of interest. Each time interval comprised of 11 sampling points (43 ms): (1) the closest to each peak extended 5 before and after the peak, and was therefore termed the‘peak’ period; (2) 11 sampling points before the peak period (pre-peak); and, (3) 11 sampling points after the peak period (post-peak). The end-points of the ‘peak’ period overlapped with the highest and lowest end-points of the pre-peak and post-peak periods, respectively. Table 2 details the end-points of each period surrounding each peak. The

I. Laufer, H. Pratt / Clinical Neurophysiology 114 (2003) 1316–1331

1321

interest (7 regions of interest £ 2 hemispheres) and epoch (pre-stimulus and post-fusion average current density). The purpose of this ANOVA was to compare the post-fusion average current density level in each region of interest for each component (F-complex and positive complex components) to a pre-stimulus (baseline) activation level and identify regions of interest with activity significantly above noise. The pre-stimulus activity level was defined as a randomized homogeneous distribution of the average current density during the pre-stimulus baseline period (100 ms). Average pre-stimulus (baseline) current density was computed separately for each of the 3 periods (pre-peak, peak, post-peak) and was collapsed on the voxels comprising each of the regions of interest. The second repeated measures ANOVA assessed the effect of duplex sensation (echo perception in the side associated with the lateralized formants) on specific regions of interest. This ANOVA included the following factors: cortical region of interest (7 brain regions), formant location (front, left, right) and hemisphere (left/right). GreenhouseGeisser corrections for multiple comparisons and violations of sphericity were employed as appropriate using epsilon (E). In all, 6 repeated measures ANOVAs were conducted separately. 2.8. LORETA t test comparisons

Fig. 2. The 21-channel AEPs that were used for LORETA current density computations. The entire period recorded in response to ‘front-fusion’ is presented, including the pre-stimulus baseline (a total of 1600 ms). The GFP derived from these data is shown below. The F-complex is demarcated between the two vertical lines that cross the troughs of the negative components FN1 and FN2b, at about 95 and 260 ms, respectively. Note that the peaks of the F-complex correspond to the GFP deflections: the peaks of FN1 and FN2b correspond to GFP peaks, while FP2 corresponds to a trough in the GFP. This reflects the wide distribution of FP2 across electrodes, compared to FN1 and FN2b that are more restricted in scalp distribution.

division into 3 separate periods enabled a thorough scanning of the time period adjacent to each peak. For each of the 3 periods (pre-peak, peak, post-peak), two ANOVAs were conducted. The first was a two-way repeated measures ANOVA with two factors: cortical region of

In order to validate the results of the factorial design (ANOVA), the effect of formant location on duplex processing was additionally assessed by LORETA (PascualMarqui et al., 1994, 1999, Vitacco et al., 2002). LORETA makes use of statistical non-parametric mapping (SnPM) to compare current density distributions between two experimental conditions. The SnPM assigns a threshold value (t statistic value) by which the statistical significance of the difference in the current densities of corresponding voxels may be evaluated. We used SnPM to compare current densities between fusion conditions. Source-current density analysis was based on the 21-channel recording. A major drawback of LORETA multiple statistical univariate tests pertains to Holmes correction (Holmes Table 2 The 3 periods surrounding each of the F-complex and positive complex components: pre-peak, peak and post-peak periods. Each period was 40 ms in duration Component

FN1 FP2 FN2b dP360 dP470

Period Pre-peak

Peak

Post-peak

60–100 130–170 190–230 305–345 395–435

100–140 170–210 230–270 345–375 435–475

140 –180 210 –250 270 –310 375 –415 475 –515

1322

I. Laufer, H. Pratt / Clinical Neurophysiology 114 (2003) 1316–1331

et al., 1996) for multiple comparisons, which is applied only for the number of voxels but not for the number of timeframes. The comparison between current density distributions in response to the 3 formant locations was conducted over a period of 150 time-frames (586 ms) starting from fusion onset. In each t test comparison between the 3 formant locations, only time-frames that were assigned with a probability below P , 0:05 for at least 5 contiguous timeframes were thresholded according to their t statistic value and were then averaged to obtain one LORETA solution comprising 2394 voxels. This method reduces the probability that time-frames assigned with significance by chance alone due to alpha inflation would be included in the analysis. In order to plot the time-course of activation in specific regions of interest possible spurious activation was removed as follows: post-fusion current density values in each of 2394 voxels over a period of 77 time-frames (300 ms) were compared to a pre-stimulus activity level by a LORETA t test. Differences across all time-frames in this comparison reached significance ðP , 0:05Þ. Voxels were thresholded according to the t statistic of P , 0:05, and voxels that failed, were nullified. This thresholding procedure was conducted for each time-frame separately. As in the case of the LORETA t test comparisons among formant locations, the probability of obtaining significant contiguous time-frames over an extended period of time by chance alone is highly improbable.

3. Results 3.1. Positive complex Fig. 1 (bottom) shows the F-complex followed by a double-peaked positive d (difference) P3 complex, with dP360 and dP470 as peaks (ca. 360 and 470 ms, respectively). The two positive peaks could be clearly observed at Fz. The amplitude of the double-peaked dP3 complex tended to be the largest with right-fusion and the smallest with front-fusion, but this tendency did not reach significance. The latency of dP470 was prolonged with front-fusion (ca. 492 ms) relative to left-fusion (ca. 453 ms) and rightfusion (ca. 460 ms). This fusion-type effect was significant [Fð2; 18Þ ¼ 5:90, P , 0:05]. A scalp frontality effect was found for the latency of dP360 [Fð2; 18Þ ¼ 10:80, P , 0:01]. Post-hoc Newman – Keuls comparisons ðP , 0:05Þ indicated that the latency of dP360 was shorter at frontal sites (344 ms) than in central (370 ms) or parieto-temporal sites (377 ms). In addition, a frontality, a laterality and a frontality £ laterality interaction effects were found on the amplitudes of dP 360 ([Fð2; 18Þ ¼ 28:92, P , 0:0001]; [Fð2; 18Þ ¼ 14:54, P , 0:001]; [Fð4; 36Þ ¼ 14:64, P , 0:001], respectively) and dP470 ([Fð2; 18Þ ¼ 11:36, P , 0:01]; [Fð2; 18Þ ¼ 3:97, P , 0:05]; [Fð4; 36Þ ¼ 16:53, P , 0:01], respectively). Newman – Keuls post-hoc

analyses ðP , 0:05Þ found the amplitude of both peaks to be largest at the frontal electrodes, and that the interaction was due to a reduction in the amplitudes of both dP360 and dP470, mainly in Pz. 3.2. Current density estimations and ANOVA The formant location effect failed to reach significance in the ANOVAs that were conducted for the peak and postpeak periods (Table 2). Therefore, only the results obtained for the pre-peak period are reported here. The significance of current density during the pre-peak period was assessed with a repeated measures ANOVA with the factors region of interest (7 regions of interest £ 2 hemispheres) and epoch (pre-stimulus, post-fusion). Results showed significant main effects of region of interest, epoch and a significant interaction effect between region of interest and epoch for the latencies of FN1 ([Fð13; 117Þ ¼ 33:68, P , 0:0001]; [Fð1; 9Þ ¼ 32:32, P , 0:001]; [Fð13; 117Þ ¼ 14:39, P , 0:05], respectively), FP2 ([Fð13; 117Þ ¼ 32:95, P , 0:000001]; [Fð1; 9Þ ¼ 72:18, P , 0:0001]; [Fð13; 117Þ ¼ 19:37, P , 0:01], respectively), FN 2b ([Fð13; 117Þ ¼ 34:90, P , 0:000001]; [Fð1; 9Þ ¼ 62:12, P , 0:0001]; [Fð13; 117Þ ¼ 21:02, P , 0:01], respectively), dP360 ([Fð13; 117Þ ¼ 46:91, P , 0:000001]; [Fð1; 9Þ ¼ 220:36, P , 0:000001]; [Fð13; 117Þ ¼ 30:01, P , 0:001], respectively) and dP470 ([Fð13; 117Þ ¼ 46:54, P , 0:000001]; [Fð1; 9Þ ¼ 166:36, P , 0:000001]; [Fð13; 117Þ ¼ 25:14, P , 0:0001], respectively). Post-hoc Newman – Keuls comparisons ðP , 0:05Þ indicated that the significant interaction effect between region of interest and epoch was due to enhanced average post-fusion current density in the auditory cortex (BA 41, 42) and medial frontal gyrus (BA 6) bilaterally. In addition, the post-hoc comparisons showed that average current density in all regions of interest was significantly above baseline except for the left BA 8 (middle/medial frontal gyrus). The effect of formant location on the perception of movement was assessed by repeated measures ANOVA for the pre-peak period with factors: region of interest (7 brain regions), formant location (left, front, right) and hemisphere (left/right). Results showed significant main effects of region of interest and hemisphere for the latencies of FN 1 ([Fð6; 54Þ ¼ 30:88, P , 0:0001];[Fð1; 9Þ ¼ 13:35, P , 0:01], respectively), FP2 ([Fð6; 54Þ ¼ 39:97, P , 0:000001];[Fð1; 9Þ ¼ 13:51, P , 0:01], respectively), FN2b ([Fð6; 54Þ ¼ 44:87, P , 0:00001]; [Fð1; 9Þ ¼ 6:08, P , 0:05], respectively), FP370 ([Fð6; 54Þ ¼ 57:30, P , 0:000001]; [Fð1; 9Þ ¼ 7:17, P , 0:05], respectively) and FP470 ([Fð6; 54Þ ¼ 62:17, P , 0:000001]; [Fð1; 9Þ ¼ 11:93, P , 0:01], respectively). The interaction between region of interest and hemisphere only tended towards significance for the latencies of FN1 [Fð6; 54Þ ¼ 5:70, P ¼ 0:050044], FN2b [Fð6; 54Þ ¼ 4:81, P ¼ 0:050054] and dP 360 [Fð6; 54Þ ¼ 4:01, P ¼ 0:060091]. A significant interaction was found for the

I. Laufer, H. Pratt / Clinical Neurophysiology 114 (2003) 1316–1331

1323

In order to compare the activation patterns between the auditory (BA 41, 42) and premotor cortices (BA 6), the time-course of average current density was plotted for these specific brain regions over a 300 ms epoch duration. We used a non-parametric t test to compare the pre-stimulus average level of activity with post-fusion activation on a voxel-by-voxel basis for each time-frame. Subthreshold voxels were nullified according to a t value threshold level of P ¼ 0:05 (Fig. 4). In general, both brain regions were activated simultaneously, activation peaked concurrently in both, and peaks of activity coincided with the F-complex components (FN1, FP2, FN2b). More specifically, over the left hemisphere (Fig. 4) the level of activation in the auditory cortex was enhanced,

Fig. 3. Region of interest £ formant location interaction. Note the enhanced right-ear advantage effect evident in the auditory (BA 41, 42) and premotor (BA 6) cortices. A significant interaction was also found in the anterior cingulate cortex (BA 24).

latencies of FP2 [Fð6; 54Þ ¼ 8:04, P , 0:05] and dP470 [Fð6; 54Þ ¼ 7:43, P , 0:01]. Post-hoc Newman – Keuls comparisons ðP , 0:05Þ showed that the interaction between region of interest and hemisphere was due to enhanced average current density over the left hemisphere in the following regions of interest: inferior and superior parietal lobules (BA 40, 7) medial frontal gyrus (BA 8) and auditory cortex (BA 41, 42). Left-hemisphere advantage was also found in the latency range of the F-complex components (FN1, FN2b) and dP360 in which the interaction effect only tended towards significance. In addition, for FN2b, a significant main effect of formant location was found [Fð2; 18Þ ¼ 6:52, P , 0:05], as well as a significant interaction effect between formant location and region of interest [Fð12; 108Þ ¼ 4:53, P , 0:05]. Post-hoc Newman – Keuls comparisons ðP , 0:05Þ revealed that the interaction effect was mainly due to enhanced average current density in both the auditory (BA 41, 42) and the premotor cortices (BA 6) with right-fusion (echo-sensation on the right side) in comparison to both left- and frontfusion (no echo-sensation, Fig. 3). An additional brain region in which activity was significantly enhanced with right-fusion relative to both left- and front-fusion included the anterior cingulate gyrus (BA 24), but this increase was not as salient as in the auditory (BA 41, 42) and premotor (BA 6) cortices. Enhanced average activation level with right-fusion also occurred in the inferior parietal lobule (BA 40), but only compared to left-fusion (Fig. 3).

Fig. 4. Time-course of activation in the auditory (BA 41, 42) and premotor (BA 6) cortices over the right and left hemispheres. Activation level was plotted after using non-parametric t test comparisons to compare prestimulus with post-fusion level of activation. Left hemisphere: Time-course of activation in the left hemisphere in the auditory (BA 41, 42) and premotor (BA 6) cortices. Note the enhanced activity in the auditory cortex in the latency range of FN1 and FN2b as well as the concurrently occurring peaks during the activation of both brain regions. Right hemisphere: Timecourse of activation over the right hemisphere in the auditory (BA 41, 42) and premotor (BA 6) cortices. Note the similar level of activation in both brain regions.

1324

I. Laufer, H. Pratt / Clinical Neurophysiology 114 (2003) 1316–1331

especially at the latencies of FN1 and FN2b. In addition, the level of activation in the premotor cortex (BA 6) was similar in both hemispheres and was enhanced compared to the auditory cortex following the FN1 peak. The level of activation in the auditory cortex (BA 41, 42) was enhanced only in the left hemisphere (Fig. 4). The level of activation in the premotor cortex was more intermittent in the right hemisphere, between 200 and 300 ms post-fusion (Fig. 4). 3.3. Current density estimations and LORETA t test comparisons LORETA t test comparisons were conducted between current density distributions in response to the 3 formant locations using the SnPM analysis, over a period of 150 time-frames (586 ms) starting from fusion onset. The analyses resulted in a statistical map of voxels with t values indicating significance beyond a threshold of 0.05. Thus, the statistical map showed only those brain regions in which current densities were significantly different between fusion conditions. In general, the results obtained by the LORETA t test comparisons corroborated those obtained by the factorial design: enhanced activity with right-fusion (echo-sensation on the right-side). Moreover, almost all significant differences among formant locations appeared in the latency region that started about 50 –65 ms before the FN2b peak, up to 20 ms before the FN2b peak (ca. 260 ms, Fig. 5). This latency range coincided exactly with the pre-peak period, the only period in which a significant interaction was found between formant location and region of interest when the factorial design was employed. However, there were also some differences between the results of the two statistical methods: involvement of the auditory cortex (BA 41, 42) in duplex processing was obtained only when the factorial design was employed, and involvement of the superior parietal lobule (BA 7) was only revealed when the LORETA t test comparisons were utilized. In addition, activity with left-fusion was significantly enhanced in comparison to front-fusion only when the LORETA t test comparisons were employed. LORETA t tests of voxel-by-voxel comparisons indicated structures differentially affected between formant locations (Fig. 5). When right- and left-fusion (motion on the right and left side, respectively) were separately compared with front-fusion (no motion), motion was associated with increased activity in the medial frontal (BA 6, 8), as well as in the cingulate (BA 24, 32) gyri (Fig. 5, right-front; left-front). In case of right-fusion enhanced activation was also located in the right inferior parietal lobule (BA 40). The comparison between right-fusion and left-fusion (Fig. 5, left –right a and b) showed that right-fusion was associated with increased activity in the right superior parietal (BA 7, 5) as well as in the right inferior parietal

(BA 40) lobules, i.e. more activity with perception of movement on the right side.

4. Discussion 4.1. Summary of results The primary purpose of the study was to examine whether the sensation of movement accompanying the duplex effect manifests in activation of brain structures associated with the processing of sound motion. Specifically, we expected to find differences in structures that are assumed to be part of a network implicated in sound motion processing: the posterior parietal, prefrontal, and inferior temporal cortices. A factorial statistical design (ANOVA) revealed the effect of formant location only for the period extending from 60 up to 20 ms before the FN2b peak. The ANOVA results indicated the involvement of the following brain regions in duplex effect processing: the auditory (BA 41, 42), the anterior cingulate (BA 24) and the premotor (BA 6) cortices, as well as the inferior parietal lobule (BA 40). Enhanced activation was prominent in both the auditory (BA 41, 42) and premotor (BA 6) cortices in both hemispheres. The interaction effect between formant location and region of interest revealed that the enhanced activation in both the auditory (BA 41, 42) and the premotor (BA 6) cortices was more prominent with right-fusion (movement sensation in the right side). In addition, a tendency towards activity enhancement in the left hemisphere was found in the auditory cortex (BA 41, 42) as well as in the inferior and superior parietal lobules (BA 40, 7). The LORETA t test comparisons corroborated the above results. Enhanced activation with duplex (lateralized movement) sensation coincided exactly with the pre-peak period, the only period in which a significant interaction was found between formant location and region of interest. In addition, right-ear advantage in fusion was also demonstrated: enhanced activation in comparison to both front- (no movement sensation) and left-fusion (movement on the left side) was found with right-fusion. Enhanced activation with right-fusion included the anterior cingulate (BA 32) and premotor (BA 6) cortices as well as the (right) inferior parietal lobule (BA 40). These are the same brain regions that the factorial design revealed to be affected by formant location. 4.2. Right-ear advantage These results show that with right-fusion average current density is enhanced in comparison to both left- and frontfusion. Right-ear advantage in speech fusion is well documented in psychoacoustic studies and was also demonstrated electrophysiologically in our previous study (Laufer and Pratt, 2003). The right-ear advantage found in the

I. Laufer, H. Pratt / Clinical Neurophysiology 114 (2003) 1316–1331

1325

Fig. 5. LORETA images of voxel-by-voxel, time-frame by time-frame t test comparisons, showing only voxels for which t values indicated significance ðP , 0:05Þ. Right-front: Comparing right-fusion and front-fusion. Note that right-fusion (motion), when compared to front-fusion (no motion), was associated with increased activity in the right inferior parietal lobule (BA 40), the right post-central gyrus (BA 1, 2) as well as in the medial frontal (BA 6, 8) and cingulate gyri (BA 32) between 195 and 211 ms post-fusion, approximately 65 – 49 ms before the FN2b peak. Left-front: Comparing left-fusion and front-fusion. Note that left-fusion (motion), when compared to front-fusion (no motion), was associated with increased activity in the left and right cingulate gyri (BA 24, 32) and in the medial frontal gyrus (BA 6) between 211 and 262 ms after fusion onset, starting approximately 50 ms before the FN2b peak and ending at the peak (around 260 ms post-fusion). Left – right a: The comparison between left-fusion and right-fusion (both associated with motion). Note that right-fusion was associated with increased activity in the inferior and superior parietal lobules (BA 40, 7) as well as in the post-central gyrus (BA 5) between 219 and 238 ms post-fusion, approximately 41 – 22 ms before the FN2b peak. A difference between the latter formant locations was also found at a later latency range, between 390 and 422 ms (left – right b), partly overlapping with the latencies of the positive complex peaks (ca. dP360 and dP470), and was also located in the inferior parietal lobule (BA 40). Brain slices at the level of the maximal t value are indicated by the black arrows. However, for the right-front comparison, the brain slice including supra-threshold values at the right superior parietal cortex was chosen. The vertical markers on the traces to the right of the images demarcate the time interval in which significance was obtained. The arrow denotes fusion onset. The x, y, z coordinates (Talairach and Tournoux, 1988) are in millimeters. The origin is in the anterior commisure. The ‘ 2 ’ and ‘ þ ’ signs denote: left and right in the x-axis; posterior and anterior in the y-axis; inferior and superior in the z-axis. L, left; R, right; A, anterior; P, posterior.

1326

I. Laufer, H. Pratt / Clinical Neurophysiology 114 (2003) 1316–1331

context of speech fusion is in line with previous studies that demonstrated left-hemisphere advantage in processing abruptly changing non-speech sounds (Brown et al., 1995; Johnsrude et al., 1997) as well as speech (Belin et al., 1996, Fiez et al., 1995). In general, these studies show that shorter transitions or frequency glides cause an enhanced activation in the left hemisphere in comparison with longer formant transitions or glides. More recent studies (Lie´geois-Chauvel et al., 1999; Shtyrov et al., 2000) in which the differences in the processing of speech and non-speech sounds were examined found that both hemispheres are implicated in speech processing, but each taps a different aspect of the speech signal: left auditory cortex is specialized in temporal resolution, whereas right auditory cortex in spectral resolution (Zatorre et al., 2002). 4.3. Right parietal cortex The results of the factorial design (ANOVA) showed that the inferior parietal lobule (BA 40) as well as the premotor areas (BA 6) constitute part of a network involved in sound motion analysis. Enhanced activation with right-fusion was evident in these brain regions in both hemispheres (with only a tendency towards left-hemisphere enhancement in BA 40). These results are in agreement with previous findings showing that both the left and right inferior parietal lobules as well as the premotor cortex (BA 6) are multimodal spatial localization areas (Bushara et al., 1999, see Section 4.6). Results of the LORETA t tests found involvement of the same regions, but in case of the inferior parietal lobule (BA 40) differences among formant locations were located in the right hemisphere. In addition, the right superior parietal lobule (BA 7) was also found to be implicated in motion processing. These results corroborate previous fMRI and PET findings (Griffiths et al., 1998, 2000), showing that the right inferior and superior parietal lobules (BA 40, 5, 7, Figs. 3 and 5) are implicated in auditory movement processing. In an earlier study (Griffiths et al., 1998) subjects heard moving stimuli from the midline towards one side, in a balanced mixture (50:50), or a stationary sound in the control condition. In our study an illusion of source movement to the left and right (in a balanced mixture) was also created by the lateralized formants, while the base was kept stationary at a midline position. In the more recent fMRI study (Griffiths et al., 2000), the probability of movement to one side (0.80) was enhanced in comparison to the other side (0.20). Similar brain regions were found to be implicated in sound movement processing in the latter study as in the case of the balanced condition: right and left premotor cortices, the right parietal cortex, and a reduced activation in the left parietal cortex (Griffiths et al., 2000). Recently, Xiang et al. (2002) reported similar results using MEG. They examined responses to moving sounds (of

different durations) from left to right or vice versa. In that study, the responses to the moving sounds were localized to the left and right temporal cortices as well as the right parietal cortex. One of the components (M280), elicited in response to the longer duration moving sound, was consistently located in the right parietal cortex. This response had a longer latency than that of M180 (also generated in the right parietal cortex), which followed the shorter duration sound. 4.4. Brodmann area 6 – premotor cortex The results of our study indicated that the premotor cortex (BA 6) was affected by formant location (Figs. 3 and 5). Motor planning is known to be associated with the posterior parietal cortex (e.g. Andersen, 1995) as well as with the premotor cortex. Previous findings imply that the latter area may be involved in the processing of sensory space in preparation for movement (Graziano et al., 1997a, b; Griffiths et al., 2000). The neglect phenomenon is also attributed to deficits in premotor circuits (Rizzolatti and Gallese, 1996) involved with the orientation of attention to specific spatial positions. Our results also indicate that the premotor area (BA 6) was differentially activated by fusion conditions: enhanced activation in this brain region was observed with right- as well as with left-fusion (Figs. 3 and 5). This latter finding corroborates earlier results showing that motor planning is implicated in auditory movement processing. Furthermore, as in the study of Griffiths et al. (2000), our study also indicated that the control of eye movement is involved in this process (Fig. 5): BA 8 is part of the pyramidal motor system and the premotor area. This Brodmann area includes the frontal eye fields, situated in the lower section of this brain region, known to be involved in the control of voluntary conjugate eye movement. The right parietal cortex, the bilateral premotor areas as well as the left and right superior temporal cortices have been suggested to constitute a network responsible for an interaction between perceptual and motor processes involved in the perception of a stimulus and in preparation for movement (Griffiths et al., 2000; Xiang et al., 2002). The results of the current study also suggest an interaction between the auditory (BA 41, 42) and the premotor cortex (BA 6). The ANOVA results revealed a significant interaction effect between formant location and region of interest which was mainly due to enhanced activation in the auditory (BA 41, 42) and the premotor (BA 6) cortices, especially with right-fusion. This result may indicate that the premotor area is also affected by inputs not only associated with motion information but also by the physical features of the auditory stimulus, i.e. those associated with speech (e.g. abruptly changing formants). In summary, activation in the premotor area (BA 6) was affected not only by sound motion processing, but also by

I. Laufer, H. Pratt / Clinical Neurophysiology 114 (2003) 1316–1331

right-ear advantage, well documented in psychoacoustic studies (see Section 4.2) and demonstrated electrophysiologically in our previous study (Laufer and Pratt, 2003). 4.5. Involvement of auditory cortex In previous studies (e.g. Bushara et al., 1999; Griffiths et al., 2000) the auditory cortex was not found to be implicated in sound motion processing. In contrast, the results obtained in our study showed involvement of the auditory cortex (BA 41, 42) in processing sound motion. A moving auditory object has physical (e.g. frequency, intensity) as well as dynamic features (motion information) that the brain may process in parallel (Xiang et al., 2002). Therefore, the effect of movement processing might have been confounded by the effect of spectral content of the speech percept, i.e. the fast-changing formant transitions. Thus, the enhanced activation level in the auditory cortex (BA 41, 42) as well as in the inferior parietal lobule (BA 40) may well have reflected the mixed effects of both spectral content and motion cues. This may have resulted in activity enhancement in both brain regions with a tendency towards left-hemisphere dominance. The time-course of activation in the auditory (BA 41, 42) and premotor (BA 6) cortices (Fig. 4) confirmed our hypothesis that in case of duplex perception the brain processes the physical features of the sound as well as the motion information associated with it in a parallel manner. Activity in both brain regions appeared in parallel and peaks of activation appeared concurrently (Fig. 4). In addition, both the time-course of activation in the auditory cortex as well as the ANOVA results showed that the level of activation in the left auditory cortex is enhanced, in agreement with left-ear advantage (LEA) found in the context of speech processing (Belin et al., 1996; Fiez et al., 1995; Lie´geois-Chauvel et al., 1999; Shtyrov et al., 2000). This latter finding is in line with the possibility that auditory cortex activation was also affected by the spectral content of the moving percept, i.e. fast transient changes in the formants, in addition to the effect of formant location (enhancement with right-fusion). 4.6. Other brain structures involved in sound motion processing The anterior cingulate cortex (BA 24, 32) was also affected by right-ear advantage (Figs. 3 and 5). Previous findings (Griffiths et al., 2000) suggest that anterior cingulate cortex activation may reflect an increased attentional load in sound movement processing, and that activation of this brain region may be associated with eye movement control in visual spatial processing. Using PET it was demonstrated (Bushara et al., 1999) that auditory and visual spatial processing activated brain regions in which both modality-specific and non-specific regions could be identified. Modality-specific localization

1327

areas were located in the superior parietal lobule (BA 7) and the middle frontal gyrus (BA 9). These areas match those found in our study to be differentially activated between fusion conditions. However, additional auditory-specific localization areas, which were not found in our study, including the right inferior temporal gyrus (BA 20) and the left inferior frontal gyrus (BA 47), were also implicated in the PET study. Areas implicated in multimodal spatial localization found in the latter study were congruent with the same areas that emerged in our results, namely the premotor cortex (BA 6) and the left and right inferior parietal lobule (BA 40). Additional supra-modal areas not found in our study included the right inferior temporal cortex, the right thalamus, as well as the right and left cerebellum (Bushara et al., 1999). 4.7. The anterior cingulate (BA 24, 32) and premotor cortices Right-ear advantage affected both the anterior cingulate (BA 24, 32) and premotor cortices (BA 6) in processing duplex. Recent findings (Paus, 2001; Winterer et al., 1999) suggest that the anterior cingulate cortex is an interface between limbic and cortical systems and its level of activation serves as an index for the degree of engagement in task performance. It has also been suggested that activation of the anterior cingulate cortex is affected by error detection (e.g. Dehaene et al., 1994) or unpredictable conflict situation (Carter et al., 1998, 2000). Early activation (120 –150 ms after stimulus onset) of the motor portion of the anterior cingulate cortex (BA 24) together with the supplementary motor area (BA 6) was demonstrated during a choice-reaction task using fMRI (Winterer et al., 1999). In that study a significant correlation was found between degree of anterior cingulate cortex activation and reaction time, further strengthening the connection between level of activation in this brain region and the degree of task engagement. In our study, enhanced activation in the auditory cortex (BA 41, 42), inferior parietal lobule (BA 40), anterior cingulate (BA 24, 32) and premotor (BA 6) cortices was observed with right-fusion. In the latter two brain regions, activity enhancement was not accompanied with left-hemisphere advantage as in the case of the auditory cortex (BA 41, 42) and inferior parietal lobule (BA 40). This dissociation between right-ear advantage and lefthemisphere advantage evident in the premotor (BA 6) and anterior cingulate cortex (BA 24, 32) suggests that these two brain regions receive inputs primarily from the auditory cortex: the auditory (BA 41, 42) and premotor (BA 6) cortices were the two brain regions most affected by rightear advantage (Fig. 3) and the time-course of activation in these brain regions (Fig. 4) clearly demonstrated that they were activated in a parallel manner. The enhanced activation in the auditory cortex over the

1328

I. Laufer, H. Pratt / Clinical Neurophysiology 114 (2003) 1316–1331

left hemisphere, because of the speech stimuli (formant transitions) used, and the similarity between the patterns of activation in the auditory and premotor cortices may explain the effect of right-ear advantage on the premotor cortex although level of activation in this region was similar over both hemispheres. It is possible that not only motion cues were transferred from the auditory cortex to the premotor cortex, but also information on the significance of the stimuli (speech signals). This information was processed by the premotor (BA 6) and anterior cingulate (BA 24) cortices and probably became more prominent with right-fusion (Fig. 3). According to this explanation, the formants were more salient with right-fusion, enhancing the subject’s allocation of resources indexed by activity enhancement in the premotor and anterior cingulate cortices. This explanation is also in agreement with the role attributed to the anterior cingulate cortex as reflecting the degree of the subject’s task engagement (e.g. Paus, 2001). 4.8. A network activated during increased attentional load The main finding of this study is the evidence for rightear advantage in speech fusion. Moreover, as indicated above, the significant interaction effect between formant location and region of interest was mainly due to enhanced activation in the auditory, premotor and anterior cingulate cortices with right-fusion. Therefore, these brain regions might have been affected by a mixture of factors which include both sound motion cues associated with duplex as well as the physical features unique to speech stimuli (e.g. abruptly changing formants). By the same token, right parietal cortex activation (demonstrated in the LORETA t test comparisons, Fig. 5) may have also been affected by the physical nature of the stimuli used, and not only by motion cues. This implies that in processing duplex, two effects are being confounded: the effect of the spectral content of the formant transitions affecting hemispheric dominance, and the effect of duplex sensation (illusory movement) caused by the lateralization of the formants. The enhanced activation level at the inferior (BA 40) and superior (BA 7) parietal lobules was larger over the left hemisphere. This result corroborates the assumption that the formants appearing on the right gained additional amount of processing because of their affinity to speech. Recently, parts of the posterior frontal cortex, the presupplementary motor area and the rostral part of the dorsolateral premotor cortex, were shown to be activated during the performance of numerical verbal and spatial non-motor mental operation tasks (Hanakawa et al., 2002). The posterior parietal cortex was activated during all 3 tasks. In addition, premotor cortex activation was localized to the superior precentral sulcus, located dorsomedial to the frontal eye fields. These results are in agreement with the results obtained in our study, which apart from inferior and superior right parietal cortex activation also showed the involvement of the medial frontal gyrus, specifically of the

premotor cortex (BA 6) and possibly of the frontal eye fields (BA 8, Fig. 5) in duplex processing. Our results may therefore imply that the brain regions thought to be implicated in spatial processing that include the posterior parietal lobe, prefrontal regions and the frontal eye fields (Bushara et al., 1999; Griffiths et al., 2000) are affected not only by sound motion per se but also by attentional allocation of resources directed towards the moving sound which becomes an ‘attention-catching’ auditory object. The above assumption is supported by the activation of the anterior cingulate cortex (BA 24, 32; Figs. 3 and 5) which may reflect an increased attentional load because of movement perception (Griffiths et al., 2000). Moreover, the results of our study showed that formant location affected fusion processing during the latency range of the FN2b, a component which reflects deviance detection (Novak et al., 1990). Therefore, our results corroborate attention-related activation of right parietal, premotor and anterior cingulate cortices in sound motion processing. These regions are implicated in the allocation of attention to specific spatial locations where movement is perceived (Griffiths et al., 1997, 1998). The attention-related activation by a moving sound gains additional support by neglect theories (e.g. Halligan et al., 1989; Halligan and Mashall, 1994) that postulate that neglect, a frequent sequel of right-hemisphere damage, is a consequence of a deficit in a system responsible for allocation of attention to the neglected hemispace (Posner et al., 1984, 1987), as well as to a failure in mapping external space into a neural system (Bisiach and Luzatti, 1978). 4.9. The positive complex: dP360, dP470 The F-complex in response to deviants, was followed by a positive complex comprising a double-peaked d (difference) P3, with dP360 and dP470 as peaks (ca. 360 and 470 ms, respectively) which was frontally distributed. The frontal distribution of the double-peaked dP3 indicates that it is a variant of the P3a, which conventionally reflects the processing of rare target and distracter stimuli (Halgren et al., 1998). The P3a is considered to be a part of the MMN –N2b – P3a complex reflecting stimulus-change detection and involuntary attention switching towards the deviant event (Novak et al., 1990; Pekkonen et al., 1996; Tervaniemi et al., 1997; Ritter et al., 1999). Our results also showed that the latency of dP470 was prolonged with front-fusion relative to left- and right-fusion. This finding indicates that the left- and right-fusion conditions (motion sensation) are perceived as more deviant, or more attention-catching events than the stationary front-fusion condition. A double-peaked dP3 was also elicited in a previous study in which subjects performed the Complex Tone Test (Tenke et al., 1993). Subjects were presented with a dichotic pair of tones followed by a binaurally presented probe tone. The subjects had to decide whether the probe matches one (‘same’ condition) or none (‘different’ condition) of the

I. Laufer, H. Pratt / Clinical Neurophysiology 114 (2003) 1316–1331

dichotic tones. Subjects with strong LEA had larger amplitudes of both P350 and P550 over the right hemisphere in response to the probe stimulus. The results were interpreted to reflect a common process shared by both positive peaks, which may represent an initial evaluation followed by a reevaluation of stimulus information. The authors further suggested that the positive peaks may reflect a comparison of the probe to the memory template of the preceding dichotic pair. In that study, the late positive complex, the P350 and P550 peaks, showed a parietal scalp distribution. However, the difference in waveforms (derived by subtracting waveforms elicited in the ‘same’ judgment condition from their ‘different’ counterparts) showed that at frontocentral sites the amplitude of P550 (and a slow wave component) was larger for the different than the same judgment conditions made during the test. Our results also showed that the later positive peak, dP470, was affected by the fusion condition. Although in the study of Tenke et al. (1993) a variant of the P3b component was elicited, compared to a variant of the P3a (dP360 and dP470) in our study, there are similarities between the results of the two studies. First, the N2b – P3b complex is also attributed to deviant stimuli (although it is more readily elicited when attention is allocated to the task). Secondly, in both studies, the later positive peaks (P550 and dP470) were affected by stimulus-type and the effect was frontally prominent. Finally, in both studies, a comparison process between incoming stimuli and a template held in memory may well have occurred. In our study this comparison might have involved a template of /da/, and the different matching process might have occurred by comparing the lateralized fused sequences to that template. The results of our previous study (Laufer and Pratt, 2003) further support this suggestion. In that study the FN2b component was more negative in response to left-fusion. This finding was attributed to a matching process between the template of /da/ and the incoming /da/, which was probably worse in the case of left-fusion. Thus, when the difference between the template and the incoming stimulus was enhanced (in the case of left-fusion), the amplitude of FN2b was enhanced as well (see also D’Arcy et al., 2000).

5. Conclusion In general, this study showed that a phonetic fused auditory object, which results in duplex perception, activates a network of brain regions implicated in sound motion processing in the latency of the FN2b component, which is associated with the processing of an ‘attention-engaging’ novel or deviant event. The results further demonstrate that activation in the premotor (BA 6), as well as in the anterior cingulate (BA 24, 32) cortices and inferior parietal lobule (BA 40) was enhanced with right-fusion. In contrast with previous findings, an involvement of the auditory cortex in movement processing was also found. Its

1329

implication in duplex processing was affected by right-ear advantage which was probably confounded with the processing of motion cues associated with illusory movement perception, i.e. the echo-sensation that accompanied fusion. The results of the present study indicate that illusory auditory movement processing and novelty attentional effects are highly interrelated. Brain regions conventionally assumed to be implicated in sound motion processing probably reflect allocation of attention to the moving sound (an attention-catching event) and may constitute a network activated by enhanced attentional load.

Acknowledgements Dr Liat Kishon-Rabin’s help with stimulus selection and useful comments in the planning of this study and Nitza Horev’s help in stimulus selection and analysis are gratefully acknowledged. This study was partially supported by the Rappaport Family Institute for Research in the Medical Sciences and by Technion V.P.R. Fund – Ashkenazy Handicap Research Fund.

References Altman JA, Kalmykova IV. Role of the dog’s auditory cortex in discrimination of sound signals simulating sound source movement. Hear Res 1986;24:243 –53. Andersen RA. Encoding of intention and spatial location in the posterior parietal cortex. Cereb Cortex 1995;5:457–69. Attias J, Urbach D, Gold S, Shemesh Z. Auditory event related potentials in chronic tinnitus patients with noise induced hearing loss. Hear Res 1993;71:106–13. Baumgart F, Gaschler-Markefski B, Woldorff MG, Heinze HJ, Scheich H. A movement-sensitive area in auditory cortex. Nature 1999;400:724 –6. Belin P, Zilbovicius P, Crozier S, Masure M-C, Remy P, Samson Y. Passive and active processing of formant transitions: a PET study. Soc Neurosci Abstr 1996;22:1698. Bisiach E, Luzatti C. Unilateral neglect of representational space. Cortex 1978;14:129–33. Brown CP, Fitch RH, Tallal P. Gender and hemispheric differences for auditory temporal processing. Soc Neurosci Abstr 1995;21:440. Bruneau N, Roux S, Adrien JL, Barthe´le´my C. Auditory associative cortex dysfunction in children with autism: evidence from late auditory evoked potentials (N1 wave – T complex). Clin Neurophysiol 1999;110: 1927–34. Bushara KO, Weeks RA, Ishii K, Catalan M-J, Tian B, Rauschecker JP, Hallett M. Modality specific frontal and parietal areas for auditory and visual spatial localization in humans. Nat Neurosci 1999;2:759–66. Carter CS, Braver TS, Botvinick MM, Noll D, Cohen JD. Anterior cingulate cortex, error detection, and the online monitoring of performance. Science 1998;280:747 –9. Carter CS, MacDonald AM, Botvinick M, Ross LL, Stenger VA, Noll D, Cohen JD. Parsing executive processes: strategic vs. evaluative functions of the anterior cingulate cortex. Proc Natl Acad Sci USA 2000;97:1944–8. Connolly JF. The influence of stimulus intensity, contralateral masking and handedness on the temporal N1 and the T complex components of the auditory N1 wave. Electroenceph clin Neurophysiol 1993;86:58–68.

1330

I. Laufer, H. Pratt / Clinical Neurophysiology 114 (2003) 1316–1331

Corbetta M, Miezin FM, Shulman GL, Petersen SE. A PET study of visuospatial attention. J Neurosci 1993;13:1202 –26. Cranford J, Ravizza R, Diamond IT, Whitfield IC. Unilateral ablation of the auditory cortex in the cat impairs complex sound localization. Science 1971;172:286–8. D’Arcy RCN, Connolly JF, Crocker SF. Latency shifts in the N2b component track phonological deviations in spoken words. Clin Neurophysiol 2000;111:40 –4. Dehaene S, Posner MI, Tucker DM. Localization of a neural system for error detection and compensation. Psychol Sci 1994;5:303– 5. De Renzi E, Gentilini M, Pattacini F. Auditory extinction following hemisphere damage. Neuropsychologia 1984;22:733–44. De Renzi E, Gentilini M, Barbieri C. Auditory neglect. J Neurol Neurosurg Psychiatry 1989;52:613–7. Fiez JA, Raichle ME, Miezin FM, Petersen SE. PET studies of auditory and honological processing: effects of stimulus characteristics and task demands. J Cogn Neurosci 1995;7:357–75. Graziano MSA, Hu XT, Gross CG. Coding the locations of objects in the dark. Science 1997a;277:239–41. Graziano MSA, Hu XT, Gross CG. Visuospatial properties of ventral premotor cortex. J Neurophysiol 1997b;77:2268–92. Griffiths TD, Bench CJ, Frackowiak RS. Human cortical areas selectively activated by apparent sound movement. Curr Biol 1994;4:892–5. Griffiths TD, Rees A, Witton C, Shakir RA, Henning GB, Green GG. Evidence for a sound movement area in the human cerebral cortex. Nature 1996;383:425 –7. Griffiths TD, Rees A, Witton C, Cross PM, Shakir RA, Green GG. Spatial and temporal auditory processing deficits following right hemisphere infarction. A psychophysical study. Brain 1997;120(Pt 5):785–94. Griffiths TD, Rees G, Rees A, Green G, Witton C, Rowe D, Buechel C, Turner R, Frackowiak RSJ. Right parietal cortex is involved in the perception of sound movement in humans. Nat Neurosci 1998;1:74– 9. Griffiths TD, Green GG, Rees A, Rees G. Human brain areas involved in the analysis of auditory movement. Hum Brain Mapp 2000;9:72–80. Halgren E, Marnikovic K, Chauvel P. Generators of the late cognitive potentials in auditory and visual oddball tasks. Electroenceph clin Neurophysiol 1998;106:156 –64. Halligan PW, Mashall JC. Toward a principled explanation of unilateral neglect. Cogn Neuropsychol 1994;11:167 –206. Halligan PW, Mashall JC, Wade DT. Visuospatial neglect: underlying factors and test sensitivity. Lancet 1989;14(2):908–11. Hanakawa T, Honda M, Sawamoto N, Okada T, Yonekura Y, Fukuyama H, Shibasaki H. The role of rostral Brodmann area 6 in mental-operation tasks: an integrative neuroimaging approach. Cereb Cortex 2002;12: 1157–70. Heffner H, Masterton B. Contribution of auditory cortex to sound localization in the monkey (Macaca mulatta). J Neurophysiol 1975; 38:1340–58. Holmes AP, Blair RC, Watson JD, Ford I. Nonparametric analysis of statistic images from functional mapping experiments. J Cereb Blood Flow Metab 1996;16:7–22. Jenkins WM, Merzenich MM. Role of cat primary auditory cortex for sound-localization behavior. J Neurophysiol 1984;52:819–47. Johnsrude IS, Zatorre RJ, Milner BA, Evans AC. Left-hemisphere specialization for the processing of acoustic transients. NeuroReport 1997;8:1761–5. Jones SJ, Byrne C. The AEP T-complex to synthesized musical tones: left – right asymmetry in relation to handedness and hemisphere dominance. Electroenceph clin Neurophysiol 1998;108:355–60. Jones SJ, Perez N. The auditory ‘C-process’ analyzing the spectral envelope of complex sounds. Clin Neurophysiol 2001;112:965–75. Jones SJ, Pitman JR, Halliday AM. Scalp responses to coherence and discoherence of binaural noise and change in the interaural time difference: a specific binaural evoked potential or a ‘mismatch’ response? Electroenceph clin Neurophysiol 1991;80:147– 54. Jones SJ, Longe O, Vaz Pato M. Auditory evoked potentials to abrupt pitch

and timbre change of complex tones. Electrophysiological evidence of ‘streaming’? Electroenceph clin Neurophysiol 1998;108:131– 42. Laufer I, Pratt H. The electrophysiological net response (‘F-complex’) to spatial fusion of speech elements forming an auditory object. Clin Neurophysiol 2003;114:818–34. Lie´geois-Chauvel C, de Graaf JB, Laguitton V, Chauvel P. Specialization of left auditory cortex for speech perception in man depends on temporal coding. Cereb Cortex 1999;9:484 –96. Mangun GR, Luck SJ, Plager R, Loftus W, Hillyard SA, Handy T, Clark VP, Gazzaniga MS. Monitoring the visualworld: hemispheric asymmetries and subcortical processes in attention. J Cogn Neurosci 1994;6: 267 –75. Mann VA, Liberman AM. Some differences between phonetic and auditory modes of perception. Cognition 1983;14:211 –35. Novak GP, Ritter W, Vaughan Jr HG, Wiznitzer ML. Differentiation of negative event-related potentials in an auditory discrimination task. Electroenceph clin Neurophysiol 1990;75:255–75. Pascual-Marqui RD, Michel CM, Lehmann D. Low resolution electromagnetic tomography: a new method for localizing electrical activity in the brain. Int J Psychophysiol 1994;18:49–65. Pascual-Marqui RD, Lehmann D, Koeing T, Kochi K, Merlo MCG, Hell D, Koukkou M. Low resolution brain electromagnetic tomography (LORETA) functional imaging in acute, neuroleptic-naı¨ve, firstepisode, productive schizophrenia. Psychiatry Res: Neuroimaging 1999;90:169–79. Paus T. Primate anterior cingultae cortex: where motor control, drive and cognition interface. Nat Rev 2001;2:417–24. Pekkonen E, Rinne T, Reinikainen K, Kujala T, Alho K, Na¨a¨ta¨nen R. Aging effects on auditory processing: an event-related potential study. Exp Aging Res 1996;22:171–84. Petersen SE, Corbetta M, Miezin FM, Shulman GL. PET studies of parietal involvement in spatial attention: comparison of different task types. Can J Exp Psychol 1994;48:319–38. Posner MI, Walker JA, Friedrich FJ, Rafal RD. Effects of parietal injury on covert orienting of attention. J Neurosci 1984;4:1863–74. Posner MI, Inhoff AW, Friedrich FJ, Cohen A. Isolating attentional systems: a cognitive anatomical analysis. Psychobiology 1987;15: 107 –21. Ritter W, Sussman E, Deacon D, Cowan N, Vaughan Jr HG. Two cognitive systems simultaneously prepared for opposite events. Psychophysiology 1999;36:835–8. Rizzolatti G, Gallese V. Mechanisms and theories of spatial neglect. In: Boller F, Grafman J, editors. Handbook of neuropsychology, vol. 1. Amsterdam: Elsevier; 1996. p. 223–46. Shtyrov Y, Kujala T, Palva S, Ilmoniemi RJ, Na¨a¨ta¨nen R. Discrimination of speech and of complex nonspeech sounds of different temporal structure in the left and right cerebral hemispheres. Neuroimage 2000;12:657–63. Soroker N, Calamaro N, Glicksohn J, Myslobodsky MS. Auditory inattention in right-hemisphere-damaged patients with and without visual neglect. Neuropsychologia 1997;35:249–56. Talairach J, Tournoux P. Co-planar stereotaxic atlas of the human brain. 3-dimensional proportional system: an approach to cerebral imaging. Stuttgart: George Thiem Verlag; 1988. Tenke CE, Bruder GE, Towey JP, Leite P, Sidtis JJ. Correspondence between brain ERP and behavioral asymmetries in a dichotic complex tone test. Psychophysiology 1993;30:62 –70. Tervaniemi M, Schro¨ger E, Na¨a¨ta¨nen R. Pre-attentive processing of spectrally complex sounds with asynchronous onsets: an event-related potential study with human subjects. Neurosci Lett 1997;227:197 –200. Tonnquist-Uhle´n I. Topography of auditory evoked long-latency potentials in children with severe language impairment: the T complex. Acta Otolaryngol (Stockh) 1996;116:680–9. Vitacco D, Brandeis D, Pascual-Marqui R, Martin E. Correspondence of event-related potential tomography and functional magnetic resonance imaging during language processing. Hum Brain Mapp 2002;17:4–12. Weeks RA, Aziz-Sultan A, Bushara KO, Tian B, Wessinger CM, Dang N,

I. Laufer, H. Pratt / Clinical Neurophysiology 114 (2003) 1316–1331 Rauschecker JP, Hallett M. A PET study of auditory spatial processing. Neurosci Lett 1999;12(262):155–8. Winterer G, Ziller M, Dorn H, Frick K, Muleret C, Dahhan N, Herrmann WM, Coppola R. Cortical activation, signal-to-noise ratio and stochastic resonance during information processing in man. Clin Neurophysiol 1999;110:1193–203. Wolpaw JR, Penry JK. A temporal component of the auditory evoked response. Electroenceph clin Neurophysiol 1975;39:609– 20.

1331

Wolpaw JR, Penry JK. Hemispheric differences in the auditory evoked response. Electroenceph clin Neurophysiol 1977;43:99 –102. Xiang J, Chuang S, Wilson D, Otsubo H, Pang E, Holowka S, Sharma R, Ochi A, Chitoku S. Sound motion evoked magnetic fields. Clin Neurophysiol 2002;113:1 –9. Zatorre RJ, Belin P, Penhune VB. Structure and function of auditory cortex: music and speech. Trends Cogn Sci 2002;6:37– 46.