Task-related coupling from high- to low-frequency signals among visual cortical areas in human subdural recordings

International Journal of Psychophysiology 51 (2004) 97–116

Task-related coupling from high- to low-frequency signals among visual cortical areas in human subdural recordings Andreas Bruns*, Reinhard Eckhorn Physics Department, Neurophysics Group, Philipps-University, Renthof 7, Marburg D-35032, Germany Received 8 January 2003; received in revised form 20 June 2003; accepted 17 July 2003

Abstract Cortical cooperativity during cognitive demands includes high- and low-frequency activities, which raises the question whether there are interdependencies between fast and slow processes and how they are reflected in electrical brain signals. We had the opportunity to record signals intracranially from occipital visual areas in an epileptic patient and quantified inter-areal signal coupling while the patient performed a visual delayed-match-to-sample task. We computed coherence, phase consistency and amplitude envelope correlation and we also determined inter-frequency coupling through correlation between low-frequency signal components and amplitude envelopes of high-frequency components. There was a pronounced task-related increase of correlation between gamma-band (28–70 Hz) signal envelopes from a superior (occipital) and low-frequency (0–3.5 Hz) signals from an inferior (occipital) visual area, lasting for approximately 1 s and possibly reflecting a short-term memory encoding process. The correlational delay between envelopes and low-frequency components was 40 ms. In contrast, coherence, phase consistency and envelope correlation showed event-, but no task-related changes of intra-areal and no changes of inter-areal coupling. Our data suggest a specific effect of gamma-activity in the superior onto low-frequency activity in the inferior area. We argue that temporal dispersion of conduction delays might prevent coherent transmission of high-frequency signals and thus account for the absence of gamma-coherence. As such dispersion is a general property of long-range projections, envelope-to-signal correlation possibly reflects a general neuronal mechanism. Hence, our method provides a powerful tool for detecting such inter-areal interactions not visible with conventional linear coupling measures. 䊚 2003 Elsevier B.V. All rights reserved. Keywords: Gamma; Theta; Envelope; Correlation; Coherence; Amplitude; Phase; Non-linear; Intracranial

1. Introduction For understanding the principles of cognitive processing in the brain, it is crucial to gain knowl*Corresponding author. Tel.: q49-6421-28-24164x26631; fax: q49-6421-28-27034. E-mail addresses: [email protected] (A. Bruns), [email protected] (R. Eckhorn).

edge of the neuronal mechanisms of cortico– cortical interactions. Enhanced gamma-range (30– 70 Hz) activity in intracranial, in scalp EEG and in MEG recordings is a candidate broadly discussed as reflecting neuronal processes of cognitive relevance (reviews, e.g. Basar-Eroglu et al., ´ 1999; Singer, 1999; Tallon-Baudry 1996; Sauve, ¨ and Bertrand, 1999; Eckhorn, 2000; Muller et al., 2000; Karakas et al., 2001). Investigation of per-

0167-8760/04/$ - see front matter 䊚 2003 Elsevier B.V. All rights reserved. doi:10.1016/j.ijpsycho.2003.07.001

98

A. Bruns, R. Eckhorn / International Journal of Psychophysiology 51 (2004) 97–116

Fig. 1. Visual stimulation protocol (illustrations not to scale). A black square frame (118=118) on a grey background (58 cdym2) served as baseline period for data analysis. Task-relevant stimuli were taken from a set of 16 different abstract complex geometric patterns (38=38, mean luminance 44 cdym2 ). Patterns were non-semantic in order to avoid verbal encoding and subvocal rehearsal. The second stimulus disappeared upon the subject’s response, and the next trial began after 2–2.5 s. In the figure task, subjects had to compare the stimuli and to respond by pressing one of two buttons (yes when objects were identical and no when they were not). In the spatial task, the objects’ positions were compared relative to the double line which formed one of the four edges of the square frame. To control for effects of passive visual stimulation or finger movement, subjects also performed a control task in which they had to alternate between yes and no from trial to trial, regardless of the stimuli. In order to obtain a sufficiently long motor-response-free period after the second stimulus onset in the control task, subjects were instructed to let some time elapse before responding, such that response times would be similar to those in the other tasks. Correct response probability by chance was 50% in all tasks. In one session, each task comprised four blocks of 16 trials each, with blocks of different tasks alternating in pseudorandom order. Trials and blocks were statistically balanced for frequency of the 16 available objects and the 16 possible spatial configurations.

ception- or task-related coupling between signals in the gamma-range has, therefore, been pursued in many studies, and in fact, several cases of this type of coupling have been reported so far (e.g. Lachaux et al., 1999; Miltner et al., 1999; Rodriguez et al., 1999; Haig et al., 2000; Klemm et al., 2000). Especially in intracranial recordings, however, coupling among gamma-signals has often been found to be restricted to a spatial range between a few millimeters and approximately 1 cm and to drop to noise levels across larger distances (Bullock et al., 1995; Menon et al., ¨ 1996; Steriade et al., 1996; Jurgens et al., 1996; Gross and Gotman, 1999; Frien and Eckhorn, 2000), at least when using conventional linear coupling measures like cross-correlation or spectral coherence. We have, therefore, previously proposed amplitude-envelope correlation as one alternative measure that can detect coupling between gamma-amplitude modulations despite the absence of coherence (Bruns et al., 2000). Apart from this, however, it has been proposed that high-frequency population signals are mainly associated with more

local processes, whereas long-range interactions are mediated by low-frequency signals (Schanze and Eckhorn, 1997; von Stein and Sarnthein, 2000). The reason for a limited signal coupling range at high frequencies might be that different axon diameters and thus conduction velocities within a population of projection fibres might lead to an increasing temporal dispersion with increasing cortical distance, so that coherent long-range signal transmission is limited to lower frequencies. Therefore, given that local and global processes manifest themselves through high- and low-frequency signals, respectively, global interaction of local processes should cause interdependence and hence some kind of coupling between high- and low-frequency signals, instances of which have increasingly often been reported recently (Bullock et al., 1997; Schanze and Eckhorn, 1997; von Stein et al., 2000; Schack et al., 2002). We found a special form of such inter-frequency interaction, namely the correlation between low-frequency signal components and amplitude envelopes of highfrequency signal components, between non-phase-

A. Bruns, R. Eckhorn / International Journal of Psychophysiology 51 (2004) 97–116

coupled patches in the visual cortex of awake monkeys (Bruns et al., 2001). We, therefore, wondered whether this coupling type might more generally occur across larger distances between sites not showing coherent activities. Hence, in the present study we applied this coupling measure to human subdural recordings from different visual areas, apart from quantifying coupling between identical frequencies by means of coherence, phase consistency and amplitude envelope correlation. For studying interactions between cortical areas, it would be most desirable to have both the high spatial resolution of intracortical microelectrodes and the complete spatial coverage attainable with scalp EEG recordings. Subdurally implanted electrode grids constitute an interesting compromise between these extremes. On the one hand, as they are in direct contact with the cortical surface, their spatial integration profile drops off much faster than for scalp electrodes, which leads to a higher spatial resolution with integration areas in the order of 1 cm2 (scalp integration areas: G10 cm2). On the other hand, unlike most microelectrode arrays, subdural grids often contain tens of contacts at regular spacing and typically cover regions comprising two or more cortical areas. In addition, muscle or eyeblink artifacts are negligible in subdural as opposed to scalp recordings. For addressing the question of inter-areal coupling mechanisms, we, therefore, investigated intracranial recordings obtained from epileptic patients with subdural electrodes implanted for clinical diagnostics. We used a visual delayed-match-to-sample paradigm with different tasks designed to differentially activate the ventral and dorsal visual subsystems, respectively, (e.g. Ungerleider and Mishkin, 1982; Haxby et al., 1991). The aim of the experiment was not to provide support for any of the current concepts concerning the putative roles of the visual pathways. We rather wanted to achieve a functional dissociation that would allow us to distinguish cognitive processes associated with a certain short-term memory task from stimulus-driven cortical responses. Part of this work has been presented previously in abstract form (Bruns et al., 2001).

99

2. Methods 2.1. Experimental setup 2.1.1. Subjects Subdural recordings were obtained from epileptic patients undergoing presurgical evaluation in the Bethel Epilepsy Surgery Program (Bielefeld, Germany). In order to study cortico–cortical interactions between visual areas, we needed subjects with electrodes situated over at least two different visual areas. For identifying such candidates, we used the results of electrical cortical stimulation, which is routinely carried out by neurologists during clinical diagnostics. One out of seven epileptic patients we investigated met these requirements. In this patient (an 18-year-old right-handed woman), 78 electrodes were placed over several regions of the right hemisphere (Fig. 2). Subjects gave informed consent for the experiment, which was approved by the ethical committee of the Bethel Epilepsy Center and conducted in accordance with the Declaration of Helsinki, Tokyo and Venice. 2.1.2. Experimental protocol We designed a visual delayed-match-to-sample paradigm in which the same physical stimuli were used for three different tasks requiring selective attention, respectively, on the stimuli’s shape (figure task), spatial arrangement (spatial task) or neither feature (control task) (Fig. 1). The experiment was conducted in the patient’s hospital room, with the patient seated upright in bed and the room mesopically lit. Stimuli were presented on a computer screen at 1-m distance (frame rate 100 Hz; visual angle 188=148; 800=600 pixels). Presentation of stimuli, capture of the subject’s responses and reaction times and generation of trigger signals for off-line signal processing were done by a personal computer. The subject performed two successive experimental sessions with a break between them and a total duration of approximately 90 min, during which subdural signals were recorded. In order to become familiar with the experimental procedure, a training session was performed 1 week before, with electrodes not yet implanted.

100

A. Bruns, R. Eckhorn / International Journal of Psychophysiology 51 (2004) 97–116

2.1.3. Signal recording Electrodes (Ad-Tech Medical Instruments, Racine, WI, USA) were platinum–iridium discs with a contact diameter of 2.4 mm (impedance 5– 20 kV at 10 Hz), arranged in rectangular arrays with nearest-neighbor center-to-center distance 10 mm. Subdural signals were recorded with a standard system (Vangard EEG system, Cleveland Clinic Foundation; bandpass 0.016–70 Hz, 12 dBy octave; sampling rate 200 Hz; amplitude resolution 14 bit). For off-line signal analysis, the average across all channels was used as reference. 2.2. Data analysis 2.2.1. General remarks Only trials with correct responses and without instrumental artifacts (identified by visual inspection) were included in the analysis. In order to capture different aspects of the data and to demonstrate the influence of the analysis method on the results, the recorded signals were characterized by several different measures: spectral amplitude, coherence, phase consistency, envelope correlation and envelope-to-signal correlation. All measures were based on complex demodulation of the signals, i.e. on their analytic amplitudes and phases. In order to track the dynamics of these parameters, we always performed a time-resolved, event-related analysis by first aligning trials to the onset of the first stimulus (defining ts0 on the time axis) and then determining the temporal evolution of amplitudes and phases. 2.2.2. Complex demodulation of signals In the following, the recorded raw signal will be denoted by sX,k(t), where X is the electrode at which the signal was measured and k(s1,...,N) is the number of the single trial. The first step of our analysis was to determine, by means of complex demodulation, the so-called analytic signals within several different frequency bands. (The analytic signal, which is a complex quantity, represents the raw signal’s time-dependent amplitude and phase in a given frequency band.) In detail, each trial sX,k(t) was digitally bandpass-filtered in the frequency domain around a center frequency f, with cut-off frequencies (3-dB points) at fyDf and fq

Df, respectively. The transfer function had sigmoidal flanks of width Df each and a plateau between them. The spectrum of the filtered trial was set to zero for negative frequencies, multiplied by 2 and transformed back to the time domain, yielding the analytic signal:

sX,k(t,f).

(1)

The analytic signal can also be obtained via the Hilbert transform in the time or frequency domain (e.g. Panter, 1965). In fact, the described procedure is formally equivalent to that approach, but is computationally more efficient. The choice of the bandwidth Df has influence on the outcome of the analysis. In order to allow comparisons of low-frequency signals with other frequency bands (envelope-to-signal correlation, see below), we wanted the analytic signals at all center frequencies to reflect amplitude and phase modulations on the same temporal scale. (Assuming a certain slow process, which is reflected in low-frequency signals on the one hand and in amplitude envelopes of high-frequency signals on the other hand, the best signal-to-noise ratio is achieved in the analysis when envelopes and lowfrequency signals have the same bandwidth.) For this reason, the filter bandwidths were chosen to be the same for all center frequencies. The results presented here did not critically depend on the exact value of Df. For the present report, we chose Dfs3.5 Hz, which represents a reasonable compromise between time- and frequency-resolution. Therefore, analytic signals were computed for center frequencies f between 3.5 and 84 Hz in steps of 3.5 Hz, so that successive frequency bands overlapped by 50% of their total bandwidths. For envelope correlation and envelope-to-signal correlation, we started at 10.5 Hz as the lowest center frequency (i.e. 7 Hz as the lowest cut-off frequency) because the envelope concept only makes sense if the frequency range of the underlying bandpass-signal is substantially higher than the frequency range of the envelope itself (at least something like twice as high). The time-dependent analytic amplitude aX,k(t,f), i.e. the amplitude envelope, of each raw signal sX,k(t) within a given frequency band f"Df was

A. Bruns, R. Eckhorn / International Journal of Psychophysiology 51 (2004) 97–116

derived from the respective analytic signal sX,k(t,f) according to aX,k(t,f)sNsX,k(t,f)N.

sX,kŽt,f. )sX,kŽt,f.)

.

(3)

All measures applied in our analysis were derived from the spectro-temporal representation of the analytic signal (Eq. (1)), analytic amplitude (Eq. (2)) andyor analytic phase (Eq. (3)), which in turn were in effect based on the Hilbert transform (see above). The same approach has been previously used by other authors to determine the time-varying amplitude (e.g. Clochon et al., 1996) or phase (e.g. Tass et al., 1998) of electrophysiological signals. Alternatively, these parameters may be obtained by short-time Fourier analysis (which is the most common approach) or by wavelet analysis (amplitude: e.g. Tallon-Baudry et al., 1996; phase: e.g. Lachaux et al., 1999). When the different techniques are matched with respect to their time-frequency resolution, they yield largely similar results, which has been explicitly shown for Hilbert and wavelet analysis (Le Van Quyen et al., 2001). As we preferred constant filter bandwidths in our analysis (see above), the wavelet approach seemed less suitable to us, since it is typically used with a proportional relationship between bandwidth and frequency. 2.2.3. Spectral amplitude For each of the three tasks and for each electrode X, an estimator of event-related spectral amplitude was determined by simply averaging the singletrial analytic amplitude (Eq. (2)) across the N trials of the respective task: aXŽt,f.s

1 N 8)aX,kŽt,f.). N ks1

(1)) before computing the amplitude would have extracted only signal components phase-locked to the stimulus.

(2)

Correspondingly, the time-dependent analytic phase wX,k(t,f) in that frequency band was obtained by ignoring (i.e. normalizing to) the amplitude, which yielded the complex expression expŽiØwX,kŽt,f..s

101

(4)

Averaging the single-trial analytic signals (Eq.

2.2.4. Coherence For each of the three tasks and for each electrode pair {X,Y}, an estimator of event-related squared coherence was determined using the traditional definition of coherence (Bendat and Piersol, 1971) with Bartlett estimation, i.e. averaging across the N trials of the respective task:

) k2XYŽt,f.s

N

)

U 8sX,kŽt,f.ØsY,kŽt,f.

ks1

N

8

2

,

N

)sX,kŽt,f.) Ø

ks1

2

8

)sY,kŽt,f.)

2

(5)

ks1

using the analytic signals from Eq. (1); the asterisk denotes the complex conjugate. The systematic bias of coherence values due to the limited number of contributing trials was corrected by subtracting the term Ž1yk2XYŽf,t..yN (Benignus, 1969). For better comparability with the other coupling measures, in the following we will use non-squared 2 coherence kXY(f,t) instead of kXY Žf,t. for displaying the results. Nevertheless, non-squared coherence is still a positive (no complex) number—it is obtained as the square-root of Eq. (1). The Bartlett method yields high coherence values at a given frequency when at that frequency the two signals show covarying amplitudes and a consistent phase lag across trials. This phase consistency is a necessary condition, i.e. even perfectly covarying amplitudes cannot cause any significant coherence when the phases are uncoupled. Pure phase coupling without amplitude covariation, on the other hand, will lead to intermediate coherence values. It is, therefore, sometimes instructive to investigate phase and amplitude effects separately. 2.2.5. Phase consistency Phase consistency FXY(t,f) could be easily derived from coherence by ignoring the signals’ amplitudes, i.e. by substituting the phase expression (Eq. (3)) for the analytic signal (Eq. (1)) in Eq. (5). For each of the three tasks and for each

A. Bruns, R. Eckhorn / International Journal of Psychophysiology 51 (2004) 97–116

102

electrode pair {X,Y}, this leads to

)

F2XYŽt,f.s

1 N 8expŽi N ks1

)

2

ØwX,kŽt,f.. expŽyiØwY,kŽt,f.. .

(6)

We used the same bias-correction for phase consistency as for coherence and we will use nonsquared values for displaying the results. The phase consistency measure given by the definition in Eq. (6) need not be regarded as a special case of coherence (like above), but can also be derived from a more general phase locking condition (Rosenblum et al., 1996). It has been based on different phase estimation techniques and has been termed differently by different authors (Tass et al., 1998: Hilbert approach, ‘synchronization index’; Lachaux et al., 1999: wavelet approach, ‘phase locking value’). Anyhow, Eq. (6) is perfectly equivalent to those definitions. 2.2.6. Envelope (-to-envelope) correlation A reasonably high phase consistency requires that the phase jitter over trials be less than approximately one fourth of a cycle, which in the gammafrequency range corresponds to a temporal precision of a few milliseconds. Looking for a method which requires temporally less precise coupling (cf. Section 1), we had previously found amplitude envelope correlation to be a promising measure (Bruns et al., 2000). It constitutes a timeresolved cross-correlation not between the signals proper, but between their amplitude envelopes in the same or in different frequency bands, thus disregarding phase effects and concentrating on the dynamic amplitude modulations of the signals. Tracking the temporal evolution of envelope correlation was achieved by using a sliding analysis time-window. In the following, such a timew T TE window will be denoted by Itsxty , tq F, T 2 2G y being its length, and t being its center position on the time axis. In other words, the envelope timecourses from Eq. (2) were subdivided into segments aX,k(t,f) (tgIt) of length T. In the present

study, we used Ts320 ms and a step width of T s80 ms between analysis time-windows, so 4 that successive envelope segments overlapped by 75%. For each electrode pair {X,Y}, each center frequency f, each time-window It and each single trial k, the correlation between corresponding envelope segments was determined after subtracting the segments’ means and correlation values were normalized to segment energies:

8 a9X,kŽt,f.Øa9Y,kŽt,f. rXY,kŽt,f.s

tgIt

yE9X,kŽt,f.ØE9Y,kŽt,f.

(7)

,

s

where a9X,kŽt,f.saX,kŽt,f.yaX,kŽt,f. ŽtgIt. denotes an envelope segment with its mean sub2 tracted, and E9X,kŽt,f.s8tgI a9X,k Žt,f. is the energy t

of that segment. Finally, correlation values were averaged across trials, using Fisher’s Z transform FZT(r)stanhy1(r): B

E 1 N FZTŽrXY,kŽt,f..F. 8 D N ks1 G

rXYŽt,f.sFZTy1C

(8)

This measure can be easily extended to correlation between envelopes for different center frequencies { f X,f Y}, revealing inter-frequency interactions rXY(t,f X,f Y) at the same location or between different locations. This extension, however, will not be used in the following presentation. 2.2.7. Envelope-to-signal correlation Having in mind that envelope correlation may result from a mutual or a co-modulation of activities at two cortical sites, we wondered whether the ‘modulating signal’ might also be directly measurable as a low-frequency potential. Thus, as an extension of amplitude envelope correlation, we computed a time-resolved cross-correlation not between envelopes, but between envelopes of highfrequency signals at one recording location and raw low-frequency signals at the same or at another location (envelope-to-signal correlation). This was accomplished by replacing the envelope computa-

A. Bruns, R. Eckhorn / International Journal of Psychophysiology 51 (2004) 97–116

tion step for the second recording site (Y) by mere lowpass-filtering with cut-off frequency Df. In other words, for the recording from the second electrode the bandpass-filtering was performed with center frequency fs0 Hz and not followed by a complex demodulation. Eq. (7) thus becomes

8 a9X,kŽt,f.Øs9Y,kŽt,Df. rXY,kŽt,f.s

tgIt

yE9X,kŽt,f.ØE9Y,kŽt,Df.

,

(9)

with s9Y,kŽt,Df.ssY,kŽt,Df.ysY,kŽt,Df. ŽtgIt. denoting a segment of the lowpass-filtered signal with its mean subtracted and E9Y,kŽt,Df.s 8tgI s92Y,kŽt,Df. being the energy of that segment. t

Averaging of correlation values was done according to Eq. (8). Note that the low-frequency-signal part in Eq. (9) is not frequency-dependent, so that statements about certain frequency ranges (e.g. ‘envelope-to-signal correlation in the beta-range’) will always refer to the envelope contribution only. 2.2.8. Handling time-lags in correlations When evaluating cross-correlation functions not only at the central position (i.e. zero time-lag), but also at non-zero time-lags Dt, the temporal overlap of segments decreases as the time-lag increases. In order to avoid this in the correlations among envelopes or between envelopes and lowfrequency signals, we used the following scheme: starting from any given analysis interval It centered around t, we successively moved this time-window backward by Dty2 in recording X and forward by Dty2 in recording Y (or vice versa), thus preserving the whole correlation length T for all time-lags Dt. Accordingly, envelope and signal energies for normalization were determined separately for every single time-lag. In other words, for non-zero time-lags, the signal segments to be correlated were not exactly taken from the current analysis interval It (centered around t), but from intervals slightly shifted forward or backward from this position by Dty2. Consequently, it is important not to confuse the aforementioned step width between successive analysis intervals on the one hand with the timelag between corresponding signal segments from the two electrodes on the other hand. The former

103

T s80 ms, whereas the latter 4 was varied within the range Dts"100 ms in steps of 10 ms. was chosen to be

2.2.9. Testing against shuffled data Sometimes an event-related coupling change might not be genuine, but only a consequence of a generally increased or decreased coupling probability caused by structural changes in the signals. We, therefore, repeated our computations on shuffled data sets with the average temporal structure of the signals preserved. Shuffled means that signal segments from electrode Y were not taken from the same trials as those from electrode X, but instead for each X trial, one Y trial was randomly drawn at equal probability. This procedure differs from the classical shift predictor (Perkel et al., 1967) in two respects: first, a given Y trial can be repeatedly drawn, while some trials might not be drawn at all and second, the Y trial is also allowed to be the correct one, i.e. the same as the X trial. In a bootstrap procedure we then repeated the shuffling for Nrandom randomizations and thus estimated a distribution for the coupling value in question. We determined the deviation of the true coupling value from this distribution and finally obtained the corresponding significance value. 2.2.10. Frequency bands For a compact presentation of our results, we subdivided the frequency axis into four ranges: 0– 3.5, 7–14, 14–28 and 28–70 Hz. In the following, the upper three ranges will be referred to as alpha, beta-, and gamma-range, respectively. They are the frequency ranges providing the amplitude envelopes, e.g. for envelope-to-signal correlation. The 0–3.5 Hz range will be called the lowfrequency range and only refers to the lowpassfiltered signals. All amplitude and coupling values were first computed separately for each center frequency, and then averaged across all frequency bands within the frequency range in question. Note especially that, e.g. ‘gamma-envelope-to-signal correlation’ does not refer to one broad-band or one mean narrow-band envelope for the whole gamma-range, but is the mean correlation of a given low-frequency signal with all narrow-band

104

A. Bruns, R. Eckhorn / International Journal of Psychophysiology 51 (2004) 97–116

envelopes having their center frequencies in the gamma-range. Center frequencies lying on the borders of a frequency range were not taken into account (since this would have extended the range by one additional bandwidth to each side). Thus, the alpha-range consisted only of the frequency band centered approximately 10.5 Hz, while the beta-range comprised the bands with center frequencies 17.5, 21 and 24.5 Hz, and the gammarange comprised all center frequencies from 31.5 to 66.5 Hz. Since the gamma-range phenomena to be described in the present report were broad-band effects and showed no particular preferences for certain sub-ranges, further subdivision of the gamma-frequency range is not necessary for the presentation of our results. 3. Results 3.1. Psychophysical measures Of the seven subjects investigated, one perfectly met the demands for the present study. The total performance of this subject in the two sessions was 96% in the figure task, 81% in the spatial task, and 98% in the control task. The total numbers of trials thus included in the analysis were 77, 73 and 75, respectively. Reaction times were 1129"216 ms (figure), 1751"430 ms (spatial), and 1598"234 ms (control) (medians"mean absolute deviations). (Reaction times in the control task resulted from the explicit instruction to let some time elapse before responding, cf. caption Fig. 1.) 3.2. Distinguishing visual areas Precise cortical electrode positions could not be determined, since no magnetic resonance imaging (MRI) data were available for this study. However, three-dimensional positions of electrodes relative to each other and to the skull could be reconstructed from two radiographs, using a method proposed by Metz and Fencil (1989) and Fencil and Metz (1990). The fact that different visual areas were covered by some of the electrodes could be derived from clinical diagnostics and off-line data analysis.

During clinical pre-surgical diagnostics, physicians made a coarse functional classification of cortical regions according to the clinical symptoms elicited by electrical stimulation of each of the subdural electrodes. The resulting map is shown in Fig. 2a. The most interesting locations for the present study are the twenty posterior electrodes that covered visual areas. We will, therefore, focus on recordings from these electrodes in the following. Note that the clinical descriptions ‘early visual region’ and ‘visual association cortex’ are comparatively vague and need not have exact equivalents in the widely accepted anatomic-functional classification of visual areas. It seems plausible to assume that at least two of the locations at which cranial nerve irritations occurred can also be assigned to the ‘visual association cortex’, because they were surrounded by the latter. Spectral amplitude data supported the clinical distinction between two different regions, roughly corresponding to the ten superior and the ten inferior electrodes, respectively, (Fig. 2b). While no significant event-related changes of spectral amplitude at any frequency occurred in the recordings from the early visual region, most of the recordings from visual association cortex showed strong amplitude increases in the gamma-range (up to 300% of baseline level between 60 and 80 Hz) and slight decreases below 20 Hz during presence of the visual stimuli. These responses appeared both during the first and the second stimulus presentation in all three tasks. Further, even more convincing evidence for distinguishing between two functionally different cortical regions, separated by a relatively sharply defined border, came from the analysis of amplitude-envelope correlation (Fig. 2c). Eight of the superior electrodes, largely corresponding to the early visual region, showed uniformly high mutual gamma-range envelope correlation coefficients (e.g. r¯ XYs0.35"0.08 during the baseline period in the figure task, cf. coupling matrix in Fig. 2c). Envelopes at the twelve remaining locations, however, did not take part in this coupling, nor were they coupled among each other (r¯ XYs0.06" 0.04). This spatial coupling pattern showed neither pronounced task-specificity nor major event-relat-

A. Bruns, R. Eckhorn / International Journal of Psychophysiology 51 (2004) 97–116

105

Fig. 2. Cortical area classification. Top row: schematic lateral views onto the right hemisphere, with positions of all 78 electrodes (A) or of the 20 electrodes over visual cortical areas only (B, C). (a) Area classification as inferred from the neuropsychological symptoms elicited by electrical stimulation of each single electrode during clinical diagnostics. (b) Top: Spatial distribution of eventrelated gamma-amplitude increase during stimulus presentations relative to baseline. Bottom: Example time-frequency maps of spectral amplitude in the figure task for one electrode not showing the gamma-amplitude increase (blue arrow) and one electrode showing it very clearly (red arrow). (c) Top: Mean gamma-envelope-to-envelope correlation of each posterior electrode to the 19 remaining posterior electrodes, respectively, during the baseline interval in all tasks. Bottom: Coupling matrix showing all baseline gamma-envelope-to-envelope correlation values among the 16 electrodes classified as area A or B. Numbering of electrodes: nos. 1–8, area A; nos. 9–17, area B. Dotted lines in the matrix mark the inter-areal border, so that the sub-matrices represent intra-areal A-to-A (lower left) and B-to-B (upper right) coupling and inter-areal A-to-B (upper left) and B-to-A (lower right) coupling, respectively. Apart from this, the numbering roughly corresponds to the order in which one encounters the electrodes on a way from the most posteriorysuperior to the most anterioryinferior location. Note the clear difference between effects at area A and area B electrodes.

106

A. Bruns, R. Eckhorn / International Journal of Psychophysiology 51 (2004) 97–116

ed variations in time and, therefore, seems to reflect basic connectivity properties of the underlying tissue. Thus, clinical diagnostics together with spectral amplitude and amplitude-envelope correlation analysis strongly suggest that the posterior electrodes in the subject were located over (at least) two different, clearly separable visual areas. From the available data, we cannot tell for certain which visual areas these were. However, as we were mainly interested in how inter-areal interactions might generally manifest themselves in the data, it will be sufficient in the following to know that we recorded from early and higher visual areas along the ventral pathway (assuming a hierarchical increase from occipital to infero-temporal locations). For further discussion, the areas will be neutrally referred to as area A and area B. We defined as area A the eight superior locations showing high mutual gamma-range envelope correlation, and as area B the one superior and the seven inferior locations showing a significant gamma-range response of spectral amplitude. Thus, further analysis concentrated on recordings from these 16 electrodes. 3.3. Coherence, phase consistency and envelope correlation Generally, the spatial patterns of coherence, phase consistency and envelope-to-envelope correlation showed no prominent event-related or even task-specific changes. For all three measures, therefore, the spatial structures of coupling values, averaged across the baseline periods of the three tasks, provide a good description. These mean coupling patterns are depicted in Fig. 3a–c in the form of matrices visualizing the coupling among the 16 selected electrodes for the alpha-, beta- and gamma-frequency ranges. The three measures yielded qualitatively similar results, which we quantified with the mean pairwise, normalized correlation coefficient between coupling matrices of the different measures (disregarding the trivial values on the diagonal, which would cause misleadingly high similarities). These inter-measure consistencies were on average 0.98 between coherence and phase consistency, 0.88 between coher-

ence and envelope-to-envelope correlation, and 0.83 between phase consistency and envelope-toenvelope correlation. However, the measures covered different ranges between high and low coupling values. Average contrasts (mHymL)y (mHqmL) between the means mH and mL of the highest and the lowest deciles (without diagonal values), respectively, were 0.90 for envelope correlation, 0.73 for coherence and 0.69 for phase consistency. Thus, coupling patterns were most clearly structured with envelope-to-envelope correlation, whereas phase consistency showed the smallest differences between high and low coupling values. The contrast between areas A and B, in particular, depended not only on the coupling measure, but also on the frequency range. While mean coupling within area B and between areas A and B essentially decreased with increasing frequency, coupling within area A showed an opposite tendency in the higher frequency range, so that areas were best distinguishable in the gammarange. For envelope-to-envelope correlation, we also computed coupling matrices for non-zero correlation time-lags (results not shown here). Essentially, however, correlation functions were maximal at zero time-lag and coupling matrices became less structured in both directions on the time-lag axis. Due to the envelopes’ limited bandwidth, peaks in correlation functions were correspondingly wide (full width at half maximum approx. 100 ms). The only noteworthy event-related coupling changes (not shown) were transient increases (of approx. 500 ms duration) of coupling within area B in the alpha-range after stimulus onsets and decreases of coupling within area A in the gammarange after stimulus changes (on- and offsets) in all three tasks. In agreement with the abovementioned general contrast differences between the three coupling measures, these changes were largest in envelope correlation, less prominent in coherence and almost absent in phase consistency. 3.4. Envelope-to-signal correlation Compared to the other three coupling measures, envelope-to-signal correlation was generally lower and, more importantly, showed qualitatively differ-

A. Bruns, R. Eckhorn / International Journal of Psychophysiology 51 (2004) 97–116

107

Fig. 3. Similar coupling patterns of different measures, except envelope-to-signal correlation. Mean baseline wy1.5 s, y0.5 sx coupling matrices (analogous to the one in Fig. 2c) for the 16 classified electrodes, for different coupling measures (rows) and frequency ranges (columns). Prominent deviations from these baseline patterns during task performance were only seen with envelope-to-signal correlation (Fig. 4). (a) Non-squared coherence, cf. Eq. (5) in the text. (b) Phase consistency, cf. Eq. (6). (c) Envelope-to-envelope correlation, cf. Eqs. (7) and (8). (d) Envelope-to-signal correlation, cf. Eqs. (9) and (8). Note that, in contrast to the other coupling measures, diagonal values do not stand out from the others, i.e. spatial closeness or remoteness is not decisive for this coupling type. Note also that the measure and, therefore, the matrices are not symmetrical, cf. Eq. (9).

ent coupling patterns (Fig. 3d). Inter-measure consistencies of envelope-to-signal correlation with coherence, phase consistency and envelope-toenvelope correlation were only 0.27, 0.22 and 0.43, respectively (as opposed to the above-mentioned high consistencies among the latter measures). Most strikingly, the baseline patterns of

envelope-to-signal correlation were spatially more homogeneous in that there were no particular ‘coupling patches’. This was especially true for the gamma-range, while in the alpha-range areaA-to-A and area-A-to-B coupling was slightly larger than B-to-A and B-to-B coupling. In neither case, however, coupling patterns were as clearly

108

A. Bruns, R. Eckhorn / International Journal of Psychophysiology 51 (2004) 97–116

structured as with coherence or envelope-to-envelope correlation. Systematic structural differences within matrices were so low that they approached the order of magnitude of stochastic fluctuations. This can be seen from the fact that baseline coupling patterns were less reproducible across the three tasks with envelope-to-signal correlation than with the other measures: While baseline inter-task consistency (defined analogously to inter-measure consistency) for gamma-range coherence, phase consistency and envelope-to-envelope correlation was 0.98, 0.97 and 0.98, respectively, it was only 0.76 for gamma-envelope-to-signal correlation. However, envelope-to-signal correlation was the only of the employed measures that revealed an instance of very specific task- and event-related inter-areal interaction. This interaction showed itself as a temporary increase of area-A-to-B envelope-to-signal correlation during the figure task. It started approximately 0.5 s after the first stimulus onset, reached a plateau between 1 s and 2 s (i.e. during the end of the first stimulus presentation and the beginning of the inter-stimulus interval) and vanished afterwards within 150 ms. The coupling was asymmetric in the sense that only areaA-to-B coupling and no B-to-A coupling was observed. When normalized to baseline fluctuations (i.e. to baseline S.D.) or to the overall variability of correlation values, the increase was most pronounced in the gamma-range (not shown). We additionally computed envelope-to-signal correlations for non-zero time-lags and found that area-A-to-B coupling was maximal at 40 ms timelag between envelopes and low-frequency signals (i.e. envelopes were leading in time). We, therefore, used this time-lag for further analysis of envelope-to-signal correlation. Coupling matrices analogous to those in Fig. 3d, but for a time-lag of 40 ms, are depicted in Fig. 4 for selected time and frequency intervals during the figure task. Coupling patterns in the spatial and in the control task at all times were similar to the respective baseline or inter-stimulus patterns in the figure task (first and third column) and are, therefore, not shown. At 40 ms time-lag the increase (second column) in gamma-range envelope-to-signal correlation (top row) appeared not only in area-A-toB, but also in A-to-A coupling.

For quantitative description, we tested envelopeto-signal correlations for significant deviations from the respective values in shuffled data (cf. Section 2, P-0.0001, Nrandoms300). Fig. 5a shows the incidence of recording pairs with such deviations separately for different area combinations (A-to-A, A-to-B, B-to-A and B-to-B) and allows several statements. First, the figure task was the only task to exhibit a notable increase in significant coupling (Fig. 5a, second row) compared to the baseline period (Fig. 5a, first row). Second, the increase was most extensive in the gamma-range, less significant in the beta-range and virtually absent in the alpha-range. There were, however, no narrow sub-ranges within the gammarange causing the overall gamma-range correlation. All center frequencies within the gamma-range rather showed similarly high correlations, which justifies the above definition of this broad range. Third, a strong area-A-to-B vs. B-to-A asymmetry was only observed in the gamma-range, whereas beta-range coupling was spatially less specific. Note especially (Fig. 4, top row) that inter-areal gamma-envelope-to-signal coupling between many locations as far apart as 5 cm or more was much higher than any local coupling (even at the same location) within area B and that diagonal values in the coupling matrices in general did by no means stand out from the rest. In addition, to testing correlations against shuffled data, we looked for significant deviations from the respective baseline values (P-0.0001, Mann–Whitney test). The incidence of electrode pairs with significant coupling during the interval w1 s, 2 sx is shown in Fig. 5b; the results are qualitatively the same as in Fig. 5a, second row. As a measure of the results’ reproducibility, we determined inter-session consistencies (defined analogously to the other two consistency measures) between the two experimental sessions. Intersession consistency was 0.96 for gammaenvelope-to-signal correlation during the interval w1 s, 2 sx in the figure task (Fig. 4, top row, second column). Corresponding inter-session consistencies for coherence, phase consistency and envelope-toenvelope correlation were 0.98, 0.95 and 0.97, respectively.

A. Bruns, R. Eckhorn / International Journal of Psychophysiology 51 (2004) 97–116

109

Fig. 4. Inter-areal envelope-to-signal correlation during visual object processing. Figure shows envelope-to-signal correlation matrices at 40 ms time-lag (cf. Eqs. (9) and (8)) for different frequency ranges (rows) and peri-stimulus time intervals (columns) during the figure task (matrices analogous to the one in Fig. 2c). The prominent pattern seen in the second column (top row) reveals interareal A-to-B (but no B-to-A) and intra-areal A-to-A coupling. It started approximately 500 ms after onset of the first stimulus, reached its maximum between 1 and 2 s, and then rapidly disappeared within the following 150 ms. It was most clear-cut for gamma-envelopes (top row) and grew less significant with decreasing frequency (middle and bottom rows). Matrices in the spatial and in the control task were similar to the ones shown here, except that the increase in the second column was absent, and matrices during this period instead were comparable to the respective baseline patterns. Matrices for the time intervals not shown were comparable to the baseline or delay period matrices (first and third column).

For better comparability with the other coupling measures, only absolute values of envelope-tosignal correlation have been depicted in the figures. In fact, correlation values between recordings from superior and recordings from inferior electrodes (i.e. most area-A-to-B correlations) were positive, whereas intra-areal (A-to-A) correlations were negative. In other words, high gamma-amplitudes within area A were accompanied by small lowfrequency activations and vice versa. One could suppose that the different signs were simply due

to different time-lags of the respective correlation functions, but the latter were not oscillatory and had only one unique peak (either positive or negative). The phenomenon of task-specific transient longrange coupling between gamma-envelopes and low-frequency signals raises the question whether there were any accompanying amplitude or coupling modulations within the gamma-range or within the low-frequency range, respectively. Increases in gamma-amplitude in area B during

110

A. Bruns, R. Eckhorn / International Journal of Psychophysiology 51 (2004) 97–116

Fig. 5. Task-, frequency- and spatial specificity of inter-areal envelope-to-signal correlation. Bars give numbers of recording pairs showing a significant increase of envelope-to-signal correlation at 40 ms time-lag, evaluated for different frequency ranges (columns). Within each diagram, incidences are shown separately for the three tasks (groups of bars) and the four different area combinations (colors). Extensive area-A-to-A and A-to-B coupling specifically occurred during the figure task in the gamma-range (and to a lesser degree in the beta-range). (a) Test of actual against shuffled data (P-0.0001, Nrandom s300, cf. Section 2) during the baseline interval (upper row) and the time interval of maximal inter-areal coupling (lower row). (b) Test of actual data during maximalcoupling interval against actual data during baseline interval (P-0.0001, Mann–Whitney test).

stimulus presentations (Fig. 2b) were most pronounced in the figure task, but also very prominent in the other tasks, and only partly overlapped with the first half of the increase in envelope-to-signal correlation. At lower frequencies, amplitude showed decreases during stimulus presentations (again most pronounced in the figure task) and prolonged increases during inter-stimulus intervals in the figure and in the spatial task at the more superior electrodes of area B (Fig. 2b). The afore-

mentioned decreases of gamma-envelope-to-envelope correlation after stimulus changes again were not task-specific, nor did any of them coincide with the increase in envelope-to-signal correlation. Correlation between low-frequency signals showed increases of approximately 1 s duration after both stimulus onsets within area A and within area B in the figure task and within area B also in the other tasks. However, there was only marginal overlap in time with the increase in envelope-to-

A. Bruns, R. Eckhorn / International Journal of Psychophysiology 51 (2004) 97–116

signal correlation. Inter-areal correlation between low-frequency signals showed no systematic eventrelated changes. 4. Discussion The most important finding of the presented work is the pronounced task-related increase of correlation between envelopes of gamma-signals from an early visual area and low-frequency signals from a higher visual area. The increase occurred at the end of the first stimulus presentation in the figure task, lasted for approximately 1 s and was maximal at a correlational delay of 40 ms between gamma-envelopes and low-frequency signals. It was not visible with common linear correlation measures like coherence or phase consistency. (Note that the term ‘gamma’ only refers to the frequency range and does not necessarily denote oscillatory phenomena.) 4.1. Area classification We found convergent evidence from clinical diagnostics, spectral amplitude and envelope-toenvelope correlation analysis that at least two functionally different visual areas were distinguishable in the posterior cortex of the presented subject. Localization of the inter-areal border was quite consistent across the three approaches. Accordingly, it did not exactly coincide with the gap between the superior and the inferior electrode grid, but was obviously located within the superior grid. In the present study, we chose for area classification the results from gamma-envelope-toenvelope correlation and spectral amplitude for several reasons. First, they were derived from the recorded signals and thus delivered an objective criterion. Second, unlike clinical diagnostics, they were based on the very data set which we then searched for inter-areal coupling. Third, especially the fact that the border between low and high coupling did not coincide with the spatial gap, but occurred within the superior electrode grid, suggests that this was really a structural border and not a coupling gap resulting from distance. Thus, area A lay beneath the eight superior electrodes showing high mutual coupling. The exact extent

111

of area B is not as obvious, because the spatial profile of gamma-amplitude changes was less clear-cut than the profile of gamma-envelope-toenvelope correlation. However, the question whether one or the other of the excluded electrodes actually belonged to area B is far from being crucial for the results of this study. It seems strange that area A, the ‘early visual region’, did not show any spectral amplitude responses to visual stimulation, while area B as part of the ‘visual association cortex’ did, which might question the clinical classification. In fact, the electrodes’ positions relative to the skull (not shown) only partly support the latter and are rather compatible with the superior grid covering V2d, V3 andyor V3a (Van Essen and Drury, 1997; Smith et al., 1998) and the inferior grid covering V2v, VP andyor V4v (Van Essen and Drury, 1997; McKeefry and Zeki, 1997). Anyhow, area A was by no means unresponsive to visual stimuli: intraareal gamma-coherence and gamma-envelope-toenvelope correlation generally decreased after stimulus changes and intra-areal low-frequency signal correlation increased during stimulus presentations in the figure task. In general, coupling within area A (maintained as well as visually induced) especially in the gamma-range was much higher than within area B or among signals from any of the remaining 58 electrodes (not shown). Event-related potentials in response to visual stimuli were seen both in area A and B. 4.2. Reliability and significance Although no further patient has been available with electrodes at corresponding cortical positions, our results are very reliable for the presented subject. They show high inter-session consistencies, i.e. they were highly reproducible in a second experimental session, and in particular the specific increase in envelope-to-signal correlation was highly significant when tested against baseline values as well as against shuffled data. It is unlikely that the results are severely distorted by reference signal effects. Subdural electrodes with their high spatial resolution show only negligible lateral overlap of their integration areas (cf. Section 1). This is corroborated by the fact

112

A. Bruns, R. Eckhorn / International Journal of Psychophysiology 51 (2004) 97–116

that our data provide many examples of neighboring electrodes showing virtually no coherence between their signals (e.g. in area B or in the other areas that were not dealt with in the present report; Fig. 2a). Therefore, the average across all 78 channels should provide a reasonable reference signal, making distortions from this source rather unlikely. The increase in envelope-to-signal correlation is certainly not an epiphenomenon resulting from event-related changes of gamma-amplitude or lowfrequency signals. The coupling was most pronounced between gamma-envelopes in area A and low-frequency signals in areas A and B. Gammaamplitude as such, however, did not exhibit any significant event-related changes in area A at all. Similarly, no event-related potentials occurred at any electrode during the time interval in question (not shown). Thus, envelope-to-signal coupling was not only non-stimulus-locked (which already follows from its absence in shuffled data), but it was not even accompanied by conspicuous changes of the signal components involved. The only event-related amplitude or signal changes were observed at other electrodes andyor were not taskspecific andyor did not coincide with envelope-tosignal coupling in time. 4.3. Functional interpretation of envelope-to-signal correlation The event-related increase of gamma-envelopeto-signal correlation reached its maximum between 1 and 2 s after the first stimulus onset and showed several specificities. First, it was task-specific in that it occurred only in the figure task. Second, it was frequency-specific, since coupling of alphaand beta-envelopes to low-frequency signals was weaker and spatially less specific. Third, spatial specificity mainly refers to the fact that lowfrequency signals in area B were more involved in inter- than in intra-areal coupling and particularly showed much higher coupling to envelopes at distant sites than to envelopes in the same place. Fourth, inter-areal gamma-envelope-to-signal coupling was asymmetric in the sense that envelopes in area A were coupled to low-frequency signals in area B, but not vice versa. Fifth, coupling was

directional, because it reached its maximum at a time-lag of 40 ms between envelopes and lowfrequency signals. Neither area-A-to-B nor B-to-A coupling were observed at negative time-lags, i.e. envelopes were always leading in time. The task-specificity suggests that the observed coupling was not simply evoked by visual stimulation, but reflects cognitive processes associated with the special requirements of the figure task. This is compatible with the concept that shape information, as opposed to spatial information, is processed in the ventral pathway (e.g. Ungerleider and Mishkin, 1982, Haxby et al., 1991). The time interval of maximal coupling comprised the end of the first stimulus presentation and the beginning of the inter-stimulus interval, which presumably was also about the time when the subject memorized the first stimulus, after having fully perceived and consciously extracted its prominent features. Thus, envelope-to-signal correlation in the present study might reflect an encoding process in visual short-term memory. In fact, the latter is often considered to be at least partly represented at sites compatible with the position of the inferior electrode grid (Courtney et al., 1996; Ungerleider et al., 1998; Hong et al., 2000). Furthermore, lowfrequency activity (being one of the ‘coupling partners’ in the present study) has repeatedly been associated with short-term memory processes (e.g. ´ et al., 1994; Sarnthein et al., 1998; KliBuzsaki mesch, 1999; Tesche and Karhu, 2000; Raghavachari et al., 2001). The prolonged increase of amplitude below the alpha-range during the interstimulus interval in area B (Fig. 2b) by the way is consistent with the gating of theta amplitude observed by Raghavachari et al. (2001) in human intracranial recordings during the Sternberg task. 4.4. Possible mechanisms underlying envelope-tosignal correlation Envelope-to-signal correlation represents a special type of coupling between high- and lowfrequency signal components. Recent studies have shown that such inter-frequency coupling is not unusual for inter-areal interaction. In a visual go-

A. Bruns, R. Eckhorn / International Journal of Psychophysiology 51 (2004) 97–116

yno-go-experiment with cats, von Stein et al. (2000) found gamma-to-thetayalpha coupling between a primary and a higher visual area (areas 17 and 7) during presentation of behaviorally relevant stimuli. Instead of envelope-to-signal correlation, they used cross-bicoherence (Fourierbased), which quantifies phase coupling between different spectral components. It has been argued that coupling between high and low frequencies might reflect the integration of local high-frequency and global low-frequency processes (e.g. Schanze and Eckhorn, 1997; von Stein et al., 2000). Cross-bicoherence (model-based) was also used by Schack et al. (2002) in a human scalp EEG study on short-term memory. They found gamma-to-theta coupling (with ‘gamma’ denoting the frequency range 20–30 Hz, which corresponds to the upper beta-range in our study) between scalp positions Fp1 and Fz, which they associated with parts of the prefrontal cortex and the limbic system, respectively. The authors demonstrated that this phase coupling was probably due to an increased coherence between gamma-envelopes at Fp1 and theta-signals at Fz, and thus was the consequence of an amplitude modulation effect. In our study, task-specificity and transience of the increase in envelope-to-signal correlation suggest the existence of inter-areal projections which were not permanently active, but became functionally effective when required. Since in our data no mean–amplitude change in area A occurred, this kind of functional gating was obviously not directly accomplished by area A activity exceeding or falling below a certain threshold. It rather had to be actively controlled, probably by a third process (outside the region covered by the electrodes), which mediated the internal state of the system, e.g. selective attention to shape. Thus, activity in area A, even if it did not exhibit systematic eventrelated changes by itself, could have had influence on activity in area B at certain times and could not so at others. The correlation time-lag suggests that the process associated with the envelopes influenced the process associated with the low-frequency signals (and not vice versa). Starting from this assumption, we can speculate about the possible underlying neuronal mechanisms. In short, envelope-

113

to-signal correlation may result from a transmission of temporally modulated gamma-activity via a pathway with temporal dispersion. For a detailed explanation, let us consider a sub-population of area A neurons involved in local gamma-activity whose amplitude is slowly modulated in the lowfrequency range, not necessarily from outside, but rather as a consequence of their own dynamics. This idea is closely related to properties of visual cortical neurons (particularly in layers 2y3) that have been described by Anderson et al. (2000). They found that the membrane potentials of these neurons are switching aperiodically, at rates in the low-frequency range, between a low and a high state, the latter being associated with increased gamma-activity. These switchings do not occur independently in each neuron, but reflect the behavior of networks on the millimeter scale and hence should be measurable with subdural electrodes. We further assume that mean output spike rates of the considered area A sub-population are modulated according to the amplitude of their gamma-activity (Volgushev et al., 2002). Now consider a time interval when the population is, by the above-mentioned control mechanism, enabled to transmit its activity to area B. During periods of high area A activity, there will be high input, i.e. many superposed post-synaptic potentials in area B, producing a high sum potential on the cortical surface. Correspondingly, periods of low area A activity will produce only small sum potentials in area B. At the same time, periods of high and low area A gamma-activity will produce high and low amplitudes of the corresponding recorded gamma-signal, respectively. Thus, there should be a correlation between gamma-envelopes in area A and low-frequency signals in area B. Finally, different individual axonal andyor activation delays within the projecting population will cause temporal dispersion of the transmitted signals (Nowak and Bullier, 1997) and thus account for the absence of inter-areal gamma-coherence. For example, a dispersion in the order of "10 ms (i.e. "25% of the observed correlational delay) will barely affect envelope-to-signal correlation, but will severely reduce signal coherence above 25 Hz.

114

A. Bruns, R. Eckhorn / International Journal of Psychophysiology 51 (2004) 97–116

The correlation time-lag between envelopes and low-frequency signals was not smaller for intrathan for inter-areal coupling. In fact, the time-lag did generally not depend on inter-electrode distance and intra-areal time-lags even showed a tendency toward slightly higher values than interareal time-lags (not shown). This cannot be completely explained on the basis of our data. One possibility is that the intra-areal correlations were not genuine, but only a ‘mirror image’ of the interareal correlations due to volume conduction. Electrodes were situated over the occipito-lateral convexity of the cortex, with area A and area B electrodes partly facing each other through the brain at a somewhat oblique angle. Thus, area A electrodes might have partly registered the (inverted) low-frequency signals from area B, while area B electrodes did not register area A gammasignals, probably because the area of constructive superposition was not as large as for the lowfrequency signals. However, the (albeit slight) differences of optimal time-lags and overall time courses between intra- and inter-areal coupling (not shown) question this interpretation. Another explanation could be that area A indirectly projected to area B and also onto itself via one or more cortico–thalamo–cortical pathways. This would explain the similar intra- and inter-areal correlation time-lags. Furthermore, the additional activation delays in such an indirect pathway would account for the comparatively large dispersion of conduction delays postulated in our mechanistic model. The thalamus could have been directly involved in the gating of this transthalamic cortico–cortical communication, as proposed by Guillery and Sherman (2002). With respect to the positive inter-areal, but negative intra-areal correlation (cf. Section 3), one would have to conclude that the projection targets in area A and area B were qualitatively different, e.g. lay in different cortical layers, producing surface potentials of different signs. In any case, the interaction was probably no simple direct projection from one site to another. We rather tend to assume a more complex system of inter-areal connections, possibly involving multiple pathways at the same time, which in effect led to the observed coupling.

5. Conclusion We presented a case of highly specific and significant inter-areal correlation between gammaenvelopes and low-frequency signals. Cortico– cortical interaction here was much more determined by specific signal transmission between distant recording sites than by spatial neighborhood. The interaction was directed from a lower to a higher visual area, it was cognitively relevant and possibly gated by some top-down process playing a role in short-term memory encoding. Envelope-to-signal correlation of the observed type, however, may more generally reflect transmission of temporally modulated activity from a source to a target area in any situation. Since the mechanism of neuronal cooperativity presumably underlying this coupling measure is very basic, envelope-to-signal correlation may be of general interest for studying cortico–cortical interactions in intracranial as well as in scalp recordings, particularly because such non-linear coupling is not detectable with the more common coupling measures. Acknowledgments We thank Dr Hennric Jokeit and Dr Alois Ebner for their cooperation and organizational help at the Bethel Epilepsy Center. We also thank the Bethel Epilepsy Center EEG staff, especially Ralf Dernbach, for technical assistance in recording the subdural signals. This work was supported by DFG grant EC 53y9-3 given to R.E. References Anderson, J., Lampl, I., Reichova, I., Carandini, M., Ferster, D., 2000. Stimulus dependence of two-state fluctuations of membrane potential in cat visual cortex. Nat. Neurosci. 3, 617–621. ¨ ¨ Basar-Eroglu, C., Struber, D., Schurmann, M., Stadler, M., Basar, E., 1996. Gamma-band responses in the brain: a short review of psychophysiological correlates and functional significance. Int. J. Psychophys. 24, 101–112. Bendat, J.S., Piersol, A.G., 1971. Random Data: Analysis and Measurement Procedures. Wiley and Sons, New York. Benignus, V.A., 1969. Estimation of the coherence spectrum and its confidence interval using the fast Fourier transform. IEEE Trans. Aud. Electroacoust. AU-17, 145–150.

A. Bruns, R. Eckhorn / International Journal of Psychophysiology 51 (2004) 97–116 Bruns, A., Eckhorn, R., Jokeit, H., Ebner, A., 2000. Amplitude envelope correlation detects coupling among incoherent brain signals. Neuroreport 11, 1509–1514. Bruns, A., Eckhorn, R., Gail, A., Brinksmeyer, H.J., Schanze, T., 2001. Directional coupling of gamma-envelopes and theta-signals between separate neuronal populations in human and monkey visual cortex. Soc. Neurosci. Abstr. 27, no. 821.36. Bullock, T.H., Achimowicz, J.Z., Duckrow, R.B., Spencer, S.S., Iragui-Madoz, V.J., 1997. Bicoherence of intracranial EEG in sleep, wakefulness and seizures. Electroencephalogr. Clin. Neurophysiol. 103, 661–678. Bullock, T.H., McClune, M.C., Achimowicz, J.Z., IraguiMadoz, V.J., Duckrow, R.B., Spencer, S.S., 1995. EEG coherence has structure in the millimeter domain: subdural and hippocampal recordings from epileptic patients. Electroencephalogr. Clin. Neurophysiol. 95, 161–177. ´ Buzsaki, G., Bragin, A., Chrobak, J.J., Nadasdy, Z., Sik, A., Hsu, M., et al., 1994. Oscillatory and intermittent synchrony in the hippocampus: relevance to memory trace formation. ´ ´ R.R., Singer, W., Berthoz, A., In: Buzsaki, G., Llinas, Christen, Y. (Eds.), Temporal Coding in the Brain. Springer, Berlin, pp. 145–172. ´ Clochon, P., Fontbonne, J.M., Lebrun, N., Etevenon, P., 1996. A new method for quantifying EEG event-related desynchronization: amplitude envelope analysis. Electroencephalogr. Clin. Neurophysiol. 98, 126–129. Courtney, S.M., Ungerleider, L.G., Keil, K., Haxby, J.V., 1996. Object and spatial visual working memory activate seperate neural systems in human cortex. Cereb. Cortex 6, 39–49. Eckhorn, R., 2000. Cortical Processing by fast synchronization: high frequency rhythmic and non-rhythmic signals in the visual cortex point to general principles of spatiotemporal coding. In: Miller, R. (Ed.), Time and the Brain. Gordon and Breach, Lausanne, pp. 169–201. Fencil, L.E., Metz, C.E., 1990. Propagation and reduction of error in three-dimensional structure determined from biplane views of unknown orientation. Med. Phys. 17, 951–961. Frien, A., Eckhorn, R., 2000. Functional coupling shows stronger stimulus dependency for fast oscillations than for low-frequency components in striate cortex of awake monkey. Eur. J. Neurosci. 12, 1466–1478. Gross, D.W., Gotman, J., 1999. Correlation of high-frequency oscillations with the sleep–wake cycle and cognitive activity in humans. Neuroscience 94, 1005–1018. Guillery, R.W., Sherman, S.M., 2002. Thalamic relay functions and their role in corticocortical communication: generalizations from the visual system. Neuron 33, 163–175. Haig, A.R., Gordon, E., Wright, J.J., Meares, R.A., Bahramali, H., 2000. Synchronous cortical gamma-band activity in taskrelevant cognition. Neuroreport 11, 669–675. Haxby, J.V., Grady, C.L., Horwitz, B., Ungerleider, L.G., Mishkin, M., Carson, R.E., et al., 1991. Dissociation of object and spatial visual processing pathways in human extrastriate cortex. Proc. Natl. Acad. Sci. USA 88, 1621–1625.

115

Hong, K.S., Lee, S.K., Kim, J.Y., Kim, K.K., Nam, H., 2000. Visual working memory revealed by repetitive transcranial magnetic stimulation. J. Neurol. Sci. 181, 50–55. ¨ ¨ Jurgens, E., Eckhorn, R., Frien, A., Wolbern, T., 1996. Restricted coupling range of fast oscillations in striate cortex of awake monkey. In: Elsner, N., Schnitzler, H.U. (Eds.), ¨ Gottingen Neurobiology Report 1996, Georg Thieme, Stuttgart, p. 418. ¨ Karakas, S., Basar-Eroglu, C., Ozesmi, C., Kafadar, H., Erzen¨ ¨ 2001. Gamma response of the brain: a multifuncgin, O.U., tional oscillation that represents bottom-up with top-down processing. Int. J. Psychophys. 39, 137–150. Klemm, W.R., Li, T.H., Hernandez, J.L., 2000. Coherent EEG indicators of cognitive binding during ambiguous figure tasks. Conscious Cogn. 9, 66–85. Klimesch, W., 1999. EEG alpha and theta oscillations reflect cognitive and memory performance: a review and analysis. Brain Res. Rev. 29, 169–195. Lachaux, J.P., Rodriguez, E., Martinerie, J., Varela, F.J., 1999. Measuring phase synchrony in brain signals. Hum. Brain Mapp. 8, 194–208. Le Van Quyen, M., Foucher, J., Lachaux, J.P., Rodriguez, E., Lutz, A., Martinerie, J., et al., 2001. Comparison of Hilbert transform and wavelet methods for the analysis of neuronal synchrony. J. Neurosci. Method 111, 83–98. McKeefry, D.J., Zeki, S., 1997. The positions and topography of the human colour center as revealed by functional magnetic resonance imaging. Brain 120, 2229–2242. Menon, V., Freeman, W.J., Cutillo, B.A., Desmond, J.E., Ward, M.F., Bressler, S.L., et al., 1996. Spatio–temporal correlations in human gamma band electrocorticograms. Electroencephalogr. Clin. Neurophysiol. 98, 89–102. Metz, C.E., Fencil, L.E., 1989. Determination of three-dimensional structure in biplane radiography without prior knowledge of the relationship between the two views: theory. Med. Phys. 16, 45–51. Miltner, W.H.R., Braun, C., Arnold, M., Witte, H., Taub, E., 1999. Coherence of gamma-band EEG activity as a basis for associative learning. Nature 397, 434–436. ¨ Muller, M.M., Gruber, T., Keil, A., 2000. Modulation of induced gamma band activity in the human EEG by attention and visual information processing. Int. J. Psychophys. 38, 283–299. Nowak, L.G., Bullier, J., 1997. The timing of information transfer in the visual system. In: Rockland, K.S., Kaas, J.H., Peters, A. (Eds.), Extrastriate Cortex in Primates. Cerebral Cortex, Vol. 12. Plenum Press, New York. Panter, P., 1965. Modulation, Noise and Spectral Analysis. McGraw-Hill, New York. Perkel, D.H., Gerstein, G.L., Moore, G.P., 1967. Neuronal spike trains and stochastic point processes. II. Simultaneous spike trains. Biophys. J. 7, 419–440. Raghavachari, S., Kahana, M.J., Rizzuto, D.S., Caplan, J.B., Kirschen, M.P., Bourgeois, B., et al., 2001. Gating of human theta oscillations by a working memory task. J. Neurosci. 21, 3175–3183.

116

A. Bruns, R. Eckhorn / International Journal of Psychophysiology 51 (2004) 97–116

Rodriguez, E., George, N., Lachaux, J.P., Martinerie, J., Renault, B., Varela, F.J., 1999. Perception’s shadow: longdistance synchronization of human brain activity. Nature 397, 430–433. Rosenblum, M.G., Pikovsky, A.S., Kurths, J., 1996. Phase synchronization of chaotic oscillators. Phys. Rev. Lett. 76, 1804–1807. Sarnthein, J., Petsche, H., Rappelsberger, P., Shaw, G.L., von Stein, A., 1998. Synchronization between prefrontal and posterior association cortex during human working memory. Proc. Natl. Acad. Sci. USA 95, 7092–7096. ´ K., 1999. Gamma-band synchronous oscillations: recent Sauve, evidence regarding their functional significance. Conscious Cogn. 8, 213–224. ¨ Schack, B., Vath, N., Petsche, H., Geissler, H.G., Moller, E., 2002. Phase-coupling of theta–gamma EEG rhythms during short-term memory processing. Int. J. Psychophys. 44, 143–163. Schanze, T., Eckhorn, R., 1997. Phase correlation among rhythms present at different frequencies: spectral methods, application to microelectrode recordings from visual cortex and functional implications. Int. J. Psychophys. 26, 171–189. Singer, W., 1999. Neuronal synchrony: a versatile code for the definition of relations? Neuron 24, 49–65, 111–125. Smith, A.T., Greenlee, M.W., Singh, K.D., Kraemer, F.M., Hennig, J., 1998. The processing of first- and second-order motion in human visual cortex assessed by functional magnetic resonance imaging (fMRI). J. Neurosci. 18, 3816–3830. Steriade, M., Amzica, F., Contreras, D., 1996. Synchronization of fast (30–40 Hz) spontaneous cortical rhythms during brain activation. J. Neurosci. 16, 392–417.

Tallon-Baudry, C., Bertrand, O., Delpuech, C., Pernier, J., 1996. Stimulus specificity of phase-locked and non-phaselocked 40 Hz visual responses in human. J. Neurosci. 16, 4240–4249. Tallon-Baudry, C., Bertrand, O., 1999. Oscillatory gamma activity in humans and its role in object representation. Trends Cognit. Sci. 3, 151–162. Tass, P., Rosenblum, M.G., Weule, J., Kurths, J., Pikovsky, A., Volkmann, J., et al., 1998. Detection of n:m phase locking from noisy data: application to magnetoencephalography. Phys. Rev. Lett. 81, 3291–3294. Tesche, C.D., Karhu, J., 2000. Theta oscillations index human hippocampal activation during a working memory task. Proc. Natl. Acad. Sci. USA 97, 919–924. Ungerleider, L.G., Courtney, S.M., Haxby, J.V., 1998. A neural system for human visual working memory. Proc. Natl. Acad. Sci. USA 95, 883–890. Ungerleider, L.G., Mishkin, M., 1982. Two cortical visual systems. In: Ingle, D.J., Goodale, M.A., Mansfield, R.J.W. (Eds.), Analysis of Visual Behavior. MIT Press, Cambridge, MA, pp. 549–586. Van Essen, D.C., Drury, H.A., 1997. Structural and functional analyses of human cerebral cortex using a surface-based atlas. J. Neurosci. 17, 7079–7102. Volgushev, M., Pernberg, J., Eysel, U.T., 2002. A novel mechanism of response selectivity of neurons in cat visual cortex. J. Physiol. 540, 307–320. ¨ von Stein, A., Chiang, C., Konig, P., 2000. Top-down processing mediated by interareal synchronization. Proc. Natl. Acad. Sci. USA 14 748–14 753. von Stein, A., Sarnthein, J., 2000. Different frequencies for different scales of cortical integration: from local gamma to long range alphaytheta synchronization. Int. J. Psychophys. 38, 301–313.