Neuromagnetic studies of human vision: Noninvasive characterization of functional neural architecture

Physica D 42 (1990) 411-427 North-Holland

NEUROMAGNETIC STUDIES OF HUMAN VISION: NONINVASIVE CHARACTERIZATION OF FUNCTIONAL NEURAL ARCHITECTURE John S. GEORGE, Cheryl J. AINE and Edward R. FLYNN Neuromagnetism Laboratory, Life Sciences and Physics Divisions, Los Alamos National Laboratory, Los Alamos, NM 87544, USA

Biological solutions to the computational problems of vision appear to emphasize parallel processing within a hierarchy of visual areas specialized for different features of the visual array. This paper summarizes recent studies concerning the organization of the human visual system using noninvasive magnetic recording techniques. The results illustrate principles of visual system organization previously demonstrated in animal single unit studies- sequential activation of multiple visual areas- as well as new observations not previously evident in animal studies- different (apparent) cortical sources of evoked responses to different spatial frequencies.

I. Introduction

Vision is an excellent example of the rich interplay between computational and biological approaches to the understanding of complex information processing problems. Biological solutions to the computational problems of vision appear to emphasize parallel processing within multiple, anatomically segregated areas of visual cortex, which are specialized for different dimensions or features of the visual array [1, 2]. Biological systems may represent optimal or near-optimal solutions to such problems, having developed over billions of years of evolution. Organisms ranging from insects to humans exhibit a variety of visual capabilities that substantially outperform existing computational systems. The convergent evolution of functionally similar visual systems from separate genetic stem-lines suggests that the requirement to efficiently encode and process visual information can serve as a powerful selective influence [3, 4]. In model systems in which optimal information encoding was used as a criterion to guide network evolution, resulting network models exhibited functional organization similar to that seen in biological systems [5-7]. For these reasons, studies of biological solutions to the computa0167-2789/90/$03.50 © Elsevier Science Publishers B.V. (North-Holland)

tional problems of vision may offer important clues and constraints for purely computational approaches. Neurophysiological and anatomical studies of experimental animals have provided a wealth of data concerning the structure and function of the visual system. Several observations appear as recurring themes across species and techniques and are considered "organizational principles": The visual system employs multiple-step, hierarchical processing within and between a number of anatomically distinct areas [1, 8-11]. However, as in other biological networks, the functional architecture within the visual system is massively parallel and the system employs multiple, parallel information streams [12-16]. A major function of the neural circuitry is to produce specialized representations of information which subserve specific processing needs. Presumably, many or all of these patterns should be found in the human visual system. Thus far, however, the limitations of experimental techniques applicable for human studies have prevented the direct characterization of microscopic functional organization. Evoked response techniques have proven useful tools for the study of sensory information process-

412

J.S. George et al./ Neuromagnetic studies of human vision

ing by human subjects [17, 18]. Neuroelectric and neuromagnetic measurements provide complementary functional views of neural activity with ms temporal resolution. Scalp-recorded electrical potential measurements (event related potentials or ERPs) display a complex temporal waveform consisting of a series of positive and negative peaks referred to as "components". While a variety of stimulus and behavioural task manipulations have been shown to affect the kinetics or amplitudes of specific response components, limitations of ERP methodology have prevented unequivocal determination of neural sources for many components of interest. Neuromagnetic field distributions appear to be produced by intraceUular current and unlike electrical potentials, appear to be relatively unaffected by the skull and scalp. Neuromagnetic sources often can be modeled adequately as one or more equivalent current dipole elements in a uniformly conducting sphere. It is also possible to reconstruct a two- or three-dimensional current distribution consistent with an observed field distribution which satisfies some numerical criteria such as the minimum number of dipoles, maximum entropy, or minimum norm. To the extent that a model's assumptions are valid, sources can be localized with considerable precision from magnetic measurements. In this paper we describe how neuromagnetic mapping techniques have been used to characterize evoked responses to sinusoidal gratings presented at various locations in the visual field. In this effort, we attempt to discover salient features of the functional architecture of the human visual system.

2. Background The neural process of vision begins in the retina, which transduces light energy into a cellular electrical response. The input state of the retina is a cellular mosaic consisting of two classes of pho-

toreceptors, rods and cones, which are distinguished by their morphology and by their response characteristics [9, 20]. Rods are sensitive at much lower light levels than cones, producing statistically reliable responses to single photons [21, 22]; however, both rods and cones adapt their sensitivity over several orders of magnitude [23, 24]. Coupled with adaptation, the differential sensitivity of these subsystems provides the visual system with an extended dynamic range difficult to match with an electronic sensor. There are three types of cones, each characterized by its sensitivity to a particular band of wavelengths. The broad, overlapping absorbance spectra of cones, with peaks roughly corresponding to red, green and blue, are the basis for color encoding by the visual system [25-27]. The packing density of photoreceptors, the ratio of cones to rods, and visual acuity vary across the retina, with highest values in the central, "foveal" region [28-30]. This design coupled with mechanisms to orient the eye provides information processing flexibility and efficiency while minimizing hardware requirements in the retina and subsequent processing stages. Considerable information processing occurs in the retinal neural network, which may serve to maximize the information content of the signal. One important transformation performed by the retina is encoding the analog signal of the primary receptors into the frequencyencoded format characteristic of much of the central nervous system. Axons of the retinal ganglion cells form the optic nerve which transmits information from the retina through the optic chiasm to the Lateral Geniculate Nuclei (LGN). Relay neurons arising in this structure project to the primary visual area in occipital cortex, also known as cytoarchitectural area 17, or V1. Cortical neurons in V1 project to a chain of secondary visual areas including V2, V3, V4, MT, MST and others, spanning occipital, parietal and temporal cortex [2, 12, 14]. Although information processing and transfer between cortical areas is sequential and hierarchical, there are extensive feedback pathways, and functionally de-

J.S. George et al./ Neuromagnetic studies of human vision

fined subsystems within each visual area which give rise to multiple parallel information streams. A hierarchy of specialized representations: In classical neurophysiology, the function of visual system neurons is described in terms of properties of the neuron's receptive field. The receptive field is the limited region of visual space which can directly influence the firing of the cell. Retinal ganglion cells and the so-called "simple" cells of L G N and cortex are characterized by center/ surround receptive fields consisting of a central region which may be excitatory or inhibitory surrounded by an annular region which is antagonistic [31]. Cells with such receptive fields typically encode information about luminance and chromatic contrast based on relative differences in light intensity or wavelength, rather than absolute levels. One consequence of this encoding at an early stage is that subsequent information processing operations are relatively insensitive to variables such as the intensity or wavelength composition of ambient illumination. Simple cells exist at the bottom of a hierarchy of cells which respond to progressively more complex visual features. For example, some cortical neurons have receptive fields specific for oriented bars and edges, probably driven by collections of simple cells. Higher-order feature detectors such as neurons sensitive to comers or to motion in a particular direction within the receptive field are also observed in primary visual cortex. These are local operations performed in parallel by relatively specialized circuits. In secondary visual areas receptive field properties may be highly specific, and there are many varieties. One of the most striking organizational features of the early cortical visual areas is the maintenance of retinotopic order: local spatial relationships between objects in the visual field are preserved in the cortical projection even though global order is (necessarily) distorted [32]. Columnar structures collecting groups of related neurons are superimposed onto the retinotopic map. These include ocular dominance columns, consisting of neurons driven by one eye; hypercolumns, which

413

contain the full complement of feature detector types, driven from a particular region of retina; blobs and interblobs and thick stripes and other histologically or functionally defined microsystems within the visual cortex [31].

3. Neuromagnetic studies Of vision In an attempt to discover organizational principles (or organizational details) we have applied neuromagnetic measurement techniques to the study of human visual processing. The effort has employed traditional psychophysical paradigms as well as analytical strategies driven by computational models of multiple-source configurations. The process necessarily has been iterative; .by reference to existing knowledge of neurophysiology, we have been able to develop more sophisticated analytical tools. By applying these tools we are able to make inferences concerning the neurophysiology underlying our experimental observations. As in ERP measurements, the initial data produced by neuromagnetic measurements is a timevarying voltage representing the physical quantity being measured, in this case the second-order spatial difference or "derivative" of the radial component of the magnetic field around the head. All of the information available in the measurement is embedded in this time/space matrix. For some purposes the data are represented as functions of time at each spatial location. By taking a temporal snapshot of waveform amplitudes over the ensemble of measurement locations it is possible to construct field maps which may disclose the position and orientation of evoked neural activity at that instant. In such maps, a simple current dipole produces balanced extrema of positive and negative magnetic flux, flanking, and orthogonal to the projection of the source into the measurement surface. In our initial studies we attempted to localize neural sources associated with empirically defined temporal components identified in previous ERP studies, assuming a single equivalent dipole source. One complication which became

414

J.S. George et a l l Neuromagnetic studies of human vision

apparent early in our analyses was the existence of multiple simultaneous sources of neuromagnetic activity. In some cases this situation was obvious in a single field map; in other instances it was inferred from a temporal sequence of maps. Because multiple source modeling appears crucial for interpretation of neuromagnetic visual evoked response data, our approach is discussed in some detail within this paper. We expected that retinotopic order would be a primary determinant of the pattern of evoked neuromagnetic fields. Based on the retinotopic organization of primary visual cortex observed in monkeys as well as the "cruciform model" of cortical organization in humans [33-36], we expected to find significant differences in the location a n d / o r orientation of neuromagnetic sources as a function of field of stimulation. We were also interested to see whether neuromagnetic techniques could detect the effects of more subtle stimulus manipulations such as spatial frequency (SF). Given the progressive increase in receptive field size as a function of increasing eccentricity from the center of the retina [37, 38], we expected to find "preferences" for different spatial frequencies at central and peripheral stimulus locations. Anatomical and physiological studies of animals (for reviews, see refs. [39-41]) as well as psychophysical and ERP studies in humans [42-48] suggest the existence of spatial frequency selective channels within the visual system. We hoped to distinguish such channels on the basis of latency a n d / o r amplitude differences in temporal response components.

4. Experimental and analytical procedures Neuromagnetic responses to visual stimuli were monitored with a 7-channel hexagonal sensor array. Each sensor consisted of a second-order gradiometer coupled to a superconducting quantum interference device (SQUID). Responses were averaged from 25 or more presentations of each stimulus. Three or more blocks of trials were

conducted at each sensor array location and six or more array positions were collected during a single experiment, which typically spanned two recording sessions. Amplitude data from all sensors (42-112 in these studies) were used to compute isofield contour or pseudocolor maps of magnetic field distribution at 10 ms intervals from 80-360 ms following stimulus presentation. If two or more field extrema of opposite polarity were observed in a map, a least-squares procedure was applied to fit a single current dipole model to the empirical distribution, and the percentage of variance accounted for by the model was assessed. Maps of theoretical and residual field distributions were prepared for visual inspection. In some cases residual maps were featureless and appeared to reflect measurement noise. In other cases residual maps contained dipole-like features with amplitudes above estimated noise levels, suggesting the existence of additional sources. In such cases, or where three or more peaks were present in the empirical field maps, multiple dipoles were fit to a single map [49]. Magnetic resonance brain images (MRIs) were obtained for some subjects. Images were acquired using full volumetric scanning techniques, producing a series of 32 slices at 4.5 mm thickness in each of three views. During MRI, oil-containing capsules were attached at reference locations to a bathing cap also worn during neuromagnetic data acquisition. By identifying the inion and left and right periauricular reference points in each slice series, it was possible to reconcile coordinate systems between volumetric series and with the neuromagnetic coordinate system. Slices were selected and pixel coordinates were calculated by computer program. Neuromagnetic and M R / coordinate systems were reconciled by comparison of common anatomical landmarks, and the calculated source of neuromagnetic activity was located on the MRI images [50]. Stimuli consisted of 100% contrast, intensitymodulated sinusoidal gratings randomly presented at various locations in the visual field. Gratings ranged from I to 8 cycles per degree and appeared

415

J.S. George et a l l Neuromagnetic studies of human vision

in a rectangular window 2.0 ° H x 1.5 ° V. Gratings were displayed for 100 ms at approximately 1 s intervals. Background intensity was chosen so that nominal average luminance was the same in the presence or absence of the grating. Raster displays were generated by microcomputer, and projected onto a translucent screen. A system of mirrors relayed the images into a magnetically shielded chamber where experiments were conducted. In some experiments subjects were required to respond by pressing a fiber optic coupled switch for each occurrence of a grating of specified spatial frequency and location, in order to manipulate the attentional state of the subject.

ERP

N140

log 200

ATTEND - NOT ATTEND - - -

140

6.0 340 +

.2 rG) > 3.0 o .,o m

°L -20Oft

100 +

~-'~f

v

"

"l,'-

v

~'

-~

A

5. Results Electrical and magnetic evoked responses were collected simultaneously. Magnetic waveforms evoked by a small 1 cpd sinusoidal grating presented at an eccentricity of 2 ° in the lower right quadrant of the visual field are illustrated in fig. 1. Selected electrical and magnetic waveforms are illustrated in the upper panel; for these measurement locations, approximately corresponding temporal " c o m p o n e n t s " can be identified in electrical and magnetic responses. We have adopted nomenclature consistent with ERP terminology; the " P I " or "P100" E R P component is a positive peak (in occipital electrode recordings) occurring at ~ 100 ms post-stimulus. The following negative peak is the N1; the next positive peak (at --200 ms) is the P2, which is followed by the N2. Observed event related field (ERF) distributions were typically much more focal than corresponding ERP distributions. Note the polarity inversion in the 100 ms component of the E R F waveform over 2 - 4 cm on the head surface. Fig. 2 illustrates source modeling of observed neuromagnetic field distributions using one and two dipole models. Field maps for time points corresponding to peaks in ERP waveforms are illustrated in column 1. Each distribution was fit by a single current dipole model using least squares

ERF

v

~

100 -

340 -~

0.0

-6.0

-5.0

4.0 cm

left of

-3.0

-2.0

-I .0

inion

Fig. 1. Electrical and magnetic waveformsevoked by a stimulus at 2° in the lower right quadrant of the visual field. Upper panel: Selected event related potential (ERP) and event related neuromagnetic field (ERF) waveforms. Temporal components are identified by labels and the traces. Lower panel: An ensemble of magnetic response waveforms. Complete maps were constructed from at least six array placements, extending the equilateral triangular sampling grid over an area of approximately 10 × 12 cm on the occipital and left parietal surfaces of the head. Labels indicate positive and negative extrema for 100 and 340 ms components. Amplitude scales are in units of femtotesla (10-zs T).

procedures, and the predicted field distribution for the empirical sampling matrix was calculated. Calculated maps are shown in column 2. Fields observed at 280 ms were fit by a two-dipole model. In most of these examples, a single equivalent current dipole accounts for a reasonable percentage of the variance ( a v e r a g e = 8 5 % ; r a n g e = 76-93%) in the field maps. A distinct apparent source is associated with each of the time slices chosen to correspond to " c o m p o n e n t " peaks in the E R P waveform. Early sources (through ap-

416

J.S. George et al./ Neuromagnetic studies of human vision

Empirical a

Fields Theoretical

-7-6-5-¢-3-2-10123

8-7-6-5-4-3-2-10123

'--

S

E z

Fields

~

"

'!

p r o x i m a t e l y 260 ms) a p p e a r to b e in or very n e a r o c c i p i t a l cortex. H o w e v e r , in field m a p s s p a n n i n g larger regions of the h e a d surface we f o u n d evid e n c e for other, n o n - o c c i p i t a l sources with activity o v e r l a p p i n g the early o c c i p i t a l sources ( u n p u b lished observations). T h e c a l c u l a t e d source of the P1 is n e a r the source of the P2; N1 a n d N 2 sources are also similar. T h e 340 m s source (which b e c o m e s a p p a r e n t at 280 ms) a p p e a r s to b e p a r i etal in origin; this source was m u c h s t r o n g e r in a t t e n d e d t h a n in n o n - a t t e n d e d stimuli.

o O3

-t

"

5.1. Visual f i e l d effects -7-11-5-4-3-2-'l

0

-4 11-7-6-~-4-3-2-I

2

0

~

2

3

B

E e~

,.,

' ~

,

o

_o, -3 •

-7-(~-5-4--,~-2-10

-4

I

21

-7 -6-5

-,4-3-2

-I

1

2

3

-'I ?,

5

-4

70

S

4 3 2 1 0 1

-7-6-5-4-,~-2-I

0

'

*tJ,

t,,

-7-4-5-4-1-2-1

1

3

0

I

3

'N -7-6-5-4-3-2-1

0

I

-7-6-5-4-3-2-~

0

t

8

6

4

E

3

o ¢r 03 Q.

2 I

b -1 -2 -3

-7-6-5-4-3-2-1

O

I

2

3

cm left/right of inion

-4

2

cm left/right of inion

W e e x a m i n e d n e u r o m a g n e t i c field m a p s associa t e d with the P1 E R P c o m p o n e n t in all subjects. These m a p s d i s c l o s e d m a g n e t i c field d i s t r i b u t i o n s c o n s i s t e n t with a c u r r e n t d i p o l e source in the left h e m i s p h e r e w h e n the right visual field was stimul a t e d a n d a right h e m i s p h e r e source w h e n the left visual field was s t i m u l a t e d . T h e e s t i m a t e d n e u r o m a g n e t i c sources were l o c a t e d o n M R I scans for each subject. I n all cases this p r o c e d u r e suggested a c o n t r a l a t e r a l source in o c c i p i t a l cortex, a l t h o u g h precise l o c a t i o n s v a r i e d b e t w e e n individuals. I n o t h e r e x p e r i m e n t s we c o m p a r e d r e s p o n s e s to central versus p e r i p h e r a l (right visual field, R V F ) stimulation. P e r i p h e r a l stimuli c o n s i s t e n t l y p r o d u c e d d e e p e r source fits for the earliest r e s p o n s e

• Fig. 2. Left column: Neuromagnetic (isocontour) field maps for subject CA, evoked by an attended stimulus at 2° in the right visual field. Maps are Mercator (surface) projections reconstructed from field amplitude measures at a number of sensor locations over the head surface. The inion, a small boney projection at the posterior base of the scull is located at x = 0, y = 0 in these plots. Maps are at times corresponding to component peaks in the ERP temporal waveform. Right column: Calculated field distributions based on fitted parameters for a single equivalent current dipole (ECD) model. The observed field distribution at 280 ms was fit with two simultaneous dipole sources.

417

J.S. George et al./ Neuromagnetic studies of human vision

components, consistent with one of the qualitative predictions of the cruciform model, as well as with previous neuromagnetic data [35, 51]. At later times we often observed a more lateral source; this might reflect the cortical topology of one or more secondary visual areas. In many sequences of field maps we were able to resolve multiple neuromagnetic sources with distinct temporal envelopes. For example, fig. 3 illustrates selected time slices for a stimulus presented in right visual field, near but not on the central fixation point. From the initial map for the stimulus we inferred a source in the hemisphere contralateral to the visual field of stimulation. In later maps we detected evidence for ipsilateral activation: a second dipole-like source which became apparent within 20 ms of initial (observed) Empirical

Fields

Theoretical

I=

°

cortical activity. With 40 ms of the initial response, the strength of these sources was well balanced so that field distributions associated with right or left visual field stimuli appeared similar. The contralateral projection of the visual field was expected based on human neuroanatomy and animal single unit electrophysiology. The subsequent ipsilateral activation may reflect interhemispheric transfer of information by the fibers of the corpus callosum, which connect regions near the cortical projection of the vertical meridian of the visual field. In the case above, the intermediate field map clearly reflects the existence of multiple simultaneous sources. While it is possible to fit such a map in isolation using a two-dipole model, the distribution may be ambiguous. The fitting procedure will

Fields

,

0

"-° --,o.4-.-,-,

•

;

•

,

,,

,.-...-.,,.-,,

•

~

4

•

•

-,tqJ ,,,4J ,,4 - 4

,.

,

,

.

.

t

4

•

•

£ -io-g

-g

,-4

-•

~

•

Theoretical

Fields

2-Dipole

Theoretical

E o

_,,'2,'~,'_, _,,;';

.

.

.

4

•

0

.~..,,~

L~

u

-10

.4

-8

-4

-s

o

cm left/right

t

4

•

•

-to

-I

4

-4

-|

o

•

of Jnion

Fig. 3. A temporal sequence of field maps for subject MO evoked by a stimulus at 2° in the right visual field. Note that these maps span a larger region of right hemisphere than the previous example. Top two maps were fit with a single ECD source. Lower maps were fit with two simultaneous dipoles, and appear to reflect the delayed activation of a secondary source in ipsilateral occipital cortex.

J.S. George et aL / Neuromagnetic studies of human vision

418

converge on one of several possible dipole pairs, depending on the initial parameter estimates. In this case we can make a parsimonious selection between these possibilities, based on simpler field distributions observed at earlier or later times. In other cases a single dipole-like field distribution was observed over a number of time steps, but the location and orientation of the apparent source evolved in time. Fig. 4 illustrates a sequence of E R F maps obtained in one subject in response to a 1 cpd grating presented at 2 ° in the right visual field. Poststimulus latency is indicated below each map. From 90-140 ms the location and strength of field extrema appeared to systematically shift. A similar sequence was observed from 160 to 240 ms. At around 280 ms another dipole-like source became apparent. This source was much stronger in attended that non-attended stimuli. The top panel in fig. 5 illustrates the location and orientation of calculated sources for the interval 90-140 ms poststimulus. Evoked field maps for each 10 ms interval were fit with the singledipole model. Equivalent dipole source locations and orientations are illustrated in orthographic projections (upper panels). Note the apparent path of activation suggested by successive apparent sources. In the head centered coordinate system used for these studies, the - x axis is centered on the inion (a boney protrusion near the rear of the skull), the + y axis passes through a reference point on the left hemisphere -- 2 cm above and in

90 ms

00 m s

110 m s

front of the ear canal, and the + z axis exits the top of the head. Because confidence intervals for source calculations are difficult to determine analytically, Monte Carlo techniques were used to assess the potential scatter in calculated sources due to measurement noise [52]. Noise with a distribution estimated from the prestimulus baseline was added to model amplitude measures for each sensor, and the resulting field distribution was fitted. Fifty source calculations for each timepoint are illustrated in the lower panel of fig. 5. Note that even with measurement noise, modeling procedures were able to resolve at least three distinct (non-overlapping) sources. Recent studies by Kaufman and coworkers [53] suggest that brain noise is diminished during evoked activity so that our estimate is probably an upper limit. This procedure does not account for errors in absolute localization due to uncertainty of sensor position with respect to the head; however, the experimental design insures that relative differences in source locations can be reliably determined. Evolving field patterns observed from 90 to 140 ms might reflect a single migrating source; however, they might also be explained by the combination of field distributions associated with two or more discrete, temporally overlapping sources. Fig. 6 summarizes model studies designed to test whether observed field distributions might be explained by a combination of two distinct sources. Neuromagnetic field maps in the upper panels

120 m s

130 m s

140 m s

CM LEFT/RIGHT OF INION

Fig. 4. A temporal sequence of field maps for subject C A evoked by a stimulus at 2 ° in the right visual field, from the same data set illustrated in fig. 2. Note the shift of field extrema apparent in the sequence.

J.S. George et aL/ Neuromagnetic studies of human vision

419

(Left Hemisphere) 10.0

SAGITTAL

CORONAL

8.0 6.0

Z

",•40m s

"•140m s

4.0

,,o raN%, °

2.0

m,

90 mS i

0.0

HORIZONTAL

.0

I

I

-2.0

I

i

-4.0

i

;

-6.0

:

- 0.0

-8.0

X

-2.0 -4.0

140 m s

X

"~

120mi

-6.0

~r

90 m s

-8.0 -10.0

i

I

I

t

2.0

0.0

I

!

4.0

t

i

I

8.0

6.0

0.0

Y 10.0

SAGITTAL

CORONAL 8.0 6.0

Z ~ 1 4 0

4.0

~1~

ms

140 m s

2.0 0.0

I

HORIZONTAL

0.0

i

-2 . 0

i

i

-4.0

I

i

-6 . 0

t

i

-8 . 0

i

-10.0

X

-2.0 -4.0

X -6.0 -8.0

-10.0

I

0.0

i

2,0

t

i

I

4.0

t

6,0

t

i

8.0

i

10.0

Y Fig. 5. Top panels: Orthographic projections of the location and orientation of ECD sources calculated from a single dipole model fit to field maps at 10 ms intervals (the data set illustrated in fig. 4). Note the apparent path of source migration suggested by this analysis. Lower panels: Scatter plots in orthographic projection of source locations calculated in Monte Carlo simulations of the effect of measurement noise. Each cluster is an ensemble of 50 calculations on field distributions derived by adding noise with a distribution estimated from the prestimulus baseline to the observed field maps at 90, 120 and 140 ms poststimulus. The stimulus is at 2 ° in the right visual field.

J.S. George et al./ Neuromagnetic studies of human vision

420

_~

*

50 80

I

4-

*

100

2 Dipole

I

*

120

I

140

Latency (msec)

Fig. 6. Computer modeling to account for the sequence of field maps observed from 90 to 140 ms. The graph compares the percent of field variance accounted for by each of three models: the unconstrained single-source model; a single-source model, in which model ECD parameters were constrained to vary linearly between 90 and 140 ms values; and a two-dipole model using an optimal combination of 90 and 140 ms sources.

Table 1 Three source constrained fits (percent of empirical variance accounted for by model). A comparison of three classes of source model: an unconstrained single dipole fit to each field map; a single dipole constrained to vary linearly between specified locations and orientations; and a linear combination of two specified dipoles. In the upper half of the table, specified dipole pairs were best fitting single dipoles for 90/120 and 120/140 ms field maps. In the lower half, dipole pairs were 90/110 and 110/140 ms. Latency (ms) 90

100

110

120

130

original fit

-

92%

85%

-

66%

constrained 1-dipole

-

89%

85%

-

65%

constrained 2-dipole

-

89%

83%

-

65%

-

92%

-

73%

66%

constrained 1-dipole

-

89%

-

72%

65%

constrained 2-dipole

-

89%

-

70%

65%

original fit

140

illustrate the modeling process for a representative timepoint: 110 ms. The first panel is the empirical field map. The second panel is a single-dipole model where dipole location and orientation is a weighted average of fitted parameters for 90 and 140 ms field distributions. The third panel is twodipole model where the resulting distribution is a weighted linear combination of 90 to 140 ms theoretical distributions. The weighting ratio and field amplitude multiplier were adjusted to produce optimal fits in each case. The lower panel compares goodness of fit measures between constrained single and double dipole sources over the range of times. The single migrating source model was the better explanation for observed intermediate field distributions. The superiority of the single-dipole model lead us to ask whether three distinct, temporally overlapping sources could better explain the 90-140 ms evoked field distribution. Our modeling strategy was similar to that described above. 90 and 140 ms calculated sources were accepted as endpoints. The intermediate source was taken as the optimal single-dipole fit for 110 or 120 ms timepoints. Table 1 lists the percent of variance accounted for by each of three models applied to selected latencies: the original unconstrained single-source model, the single source with linearly constrained parameters, or the optimal linear combination of two (specified) dipole sources. Note that there is little difference in goodness of fit between constrained single-source and dualsource models, particularly when 120 ms was accepted as the intermediate source. The unconstrained single-source model performed slightly better; however, this may reflect a less than optimal selection of intermediate source parameters used in constrained fits. Thus while the observed field distributions may reflect a continuously migrating focus of neural activation, the data are reasonably explained by three discrete, temporally overlapping sources. Neuromagnetic sources suggested by these modeling procedures were located on magnetic resonance images. Calculated sources for 90, 120 and

J.S. George et al./ Neuromagnetic studies of human vision

140 ms evoked field distributions are illustrated in fig. 7. Locations of slices are indicated by crosshairs. The silhouette was produced from the summed thresholded images of each of the slices in a series and are provided as a visual coordinate reference. We estimate the accuracy of sensor location measurements used in this study to be + 0.5 to 1.0 cm, better near the inion. Improved procedures will allow millimeter resolution. Even given the present degree of uncertainty, suggested sources are anatomically reasonable. The 90 ms source appears to lie along the calcarine fissure (primary visual cortex). The 120 ms equivalent source is deeper, located near the anterior extent of striate cortex, possibly in area 18. The 140 ms source is higher and may lie along the parietal occipital sulcus. The general patterns of response observed in the case described above were consistently seen across subjects and under a variety of experimental conditions. We have begun to extend these spatial/temporal analyses to other subjects. Dur-

421

ing the initial cycle of activation ( - - 9 0 - 1 5 0 ms), the apparent path of activation is typically curved, suggesting at least three sources m u s t be involved. In most subjects the second cycle of activation (180-240 ms) follows a similar path, suggesting repetitive activation of the same anatomical structures.

5. 2. Effects of visual field and spatial frequency Experiments involving the manipulation of spatial frequency were intended to serve (at least) two purposes. First, we sought evidence for the existence of spatial frequency selective channels in the human visual pathway. Second, we considered the study to be a test of the capability of the neuromagnetic technique to distinguish functional differences beyond those of gross anatomical projection. We examined amplitudes and source moment of neuromagnetic responses as a function of visual field and spatial frequency of stimulation. In each

Fig. 7. Neuromagnetic sources at 90, 120 and 140 ms located on MRI. Calculated source locations are indicated by circles; the 90 ms source is the lowest in all three views. Slices were selected and pixel coordinates were calculated by software after reference points were identified in each series of images. Locations of selected slices are indicated by crosshairs. Silhouttes are provided as a visual coordinate reference. Note that by radiological convention, computed sections are displayed as though viewed from in front of (coronal), below (axial or horizontal), or to the right of (sagittal) the head.

J.S. George et a l l Neuromagnetic studies of human vision

422

Central Field Stimulation Left Hemisphere

1

Right Hemisphere

Subl. CA

"

I

I

L2M

I

I

i i

I R2M

I I

I I

,, I

0 ~

(if)

4OO .2O0

---

HI Spatial Frequency(5cpd) LO Spatial Frequency(lcpd)

Latency

{reset)

Inlon

Fig. 8. Differences in evoked field waveforms as a function of spatial frequency of the sinusoidal grating stimulus. Waveforms are from two contiguous sensor array positions spanning the midline of the head, over occipital cortex.

field map we were able to identify a pair of adjacent sensor locations which contained extrema for an initial response component for both 5 cpd (HI) and 1 cpd (LO) spatial frequencies (SF). Waveform amplitudes were sampled at selected timepoints and averaged across these two locations. HI SF elicited larger responses in CVF while LO SF produced larger responses for RVF presentations. This trend was also observed for source moments (a measure of dipole strength) calculated on the basis of dipole model fits for latencies of 90-130 ms. HI SF elicited a stronger early response in CVF while LO SF elicited a stronger and earlier response in RVF. These observations were consistent with our predictions based on the variation of receptive field size across the retina. When examining ensembles of waveforms, we expected to see differences in amplitudes and peak latencies of specific components as a function of spatial frequency. We expected that the spatial distribution of peaks would be a function of the anatomical locations of sources, and would therefore reflect only the retinotopic location of the visual stimulus, at least within primary visual areas. We were surprised to observe differential el-

fects of spatial frequency on the shapes and spatial distributions of early E R F response components elicited by stimuli presented at the same location in the visual field. Fig. 8 illustrates averaged waveforms for one subject collected at two adjacent array locations over occipital cortex, when the central visual field was stimulated. Solid lines are responses to HI SF. Dashed lines are responses to LO SF. Such differences as a function of spatial frequency were reproducible and were consistently observed across subjects. We also observed corresponding differences in E R F maps as a function of spatial frequency. Plate I presents field maps for subject JMG at indicated response latencies, when the central field was stimulated. Left-hand maps are for LO SF CVF stimuli; fight-hand maps are HI SF CVF. All maps are displayed on the same coordinate grid. Upon initial examination, field maps for different spatial frequencies in one location were more notable for their similarities than differences. However, in this and other subjects, significant differences were consistently observed in the distributions of earliest apparent components (90-100 ms), and in response components appearing at around 190 ms. For this and other subjects,

J.S. George et aL / Neuromagnetic studies of human vision

423

Plate I. Field maps for subject JMG at selected time points illustrating effects of spatial frequency on evoked field distributions. Lefthand maps are for LO SF CVF stimuli and right-hand maps for HI SF CVF stimuli. The first row is at 90 ms, the second row at 160 ms and the third at 200 ms. The maps are surface projections spanning an area 8 cm left and 4 cm right of, and 8 cm above and 2 cm below the inion, a boney projection near the base of the pasterior skull. Red signifies magnetic flux emerging from the head while blue signifies flux re-entering.

424

J.S. George et al./ Neuromagnetic studies of human vision

HI and LO SF stimuli produced very similar response maps for latencies from -- 130-180 ms. By analyzing time sequences of response maps for several subjects showing differences as a function of spatial frequency, we identified three apparent reasons for discrepant maps. In the earliest maps (typically 80-100 ms) there appeared to be a difference in the relative strength or number of dipole sources. In other subjects (not illustrated here) HI and LO spatial frequencies produced qualitatively similar field maps, but Monte Carlo analyses disclosed significant differences in the location a n d / o r orientation of a single apparent source. Based on analyses previously described, such a finding might reflect differential activation of two or more simultaneous sources with overlapping field distributions. By 120 ms these differences had largely disappeared. In some subjects significant differences were again observed around 130 ms; however, subsequent time slices suggest that these simply reflected response latency differences as a function of spatial frequency. Observed distributions at --150 ms were similar for HI and LO spatial frequency. By 190 ms differences in the number, extent, or locations of negative peaks suggest a significant difference in the location and orientation or relative strengths of equivalent dipole sources as a function of spatial frequency.

6. Discussion Neuromagnetic techniques permit high-resolution spatial and temporal characterization of patterns of neural activity evoked by sensory stimulation and allow discrimination of differences in evoked responses due to major (visual field) or more subtle (spatial frequency) manipulations of visual stimuli. As expected from retinotopic organization of striate cortex, we observed significant differences in ERF distribution as a function of field of stimulation. Initial evoked sources consistently appeared in the hemisphere contralateral to the field of stimulation. As predicted from classi-

cal models of primary visual cortex organization, early responses to peripheral stimuli appeared to originate deeper in cortex than responses to central stimuli. For stimuli near (but not spanning) the midline of the visual fidd we were able to detect subsequent ipsilateral activation, possibly mediated by the corpus collosum. Such interconnection would be expected on the basis of animal studies, and presumably prevents perceptual discontinuities which might otherwise arise. For example, motion across the midline of the visual field is perceived as continuous; an object in the central visual field can be recognized on the basis of a gestalt of features in the left and right hemispheres. While appropriate correlations might be extracted at a higher processing level, such a strategy could introduce perceptual delays for central vision, and would establish boundaries which could distort local processing. Although we were able to estimate equivalent current dipole sources for neuromagnetic field distributions associated with each of the identified components of the ERP waveform, we have discovered that analyses which focus on peaks in the temporal waveform may be misleading. For example, source locations calculated for the P1 and N1 ERP components suggest two discrete sources; however, field maps spanning the P1-N1 components display a pattern of temporal evolution consistent with a migrating focus of activation. Alternatively, such sequences may reflect a complex of spatially discrete, temporally overlapping sources. In the example discussed here, at least three sources must be postulated to adequately explain the data. Simpson et al. [54], using a multiple-dipole model for ERP data, have postulated eight dipole sources with sequential, overlapping activation to account for a time sequence spanning 60-280 ms poststimulus. Based on our initial spatial/temporal source analyses of other subjects we conclude that at least three sources must be postulated to explain the initial cycle of observed activity ( - 90-150 ms). In most subjects a second cycle of activation (180-240 ms) follows a similar anatomical path.

J.S. George et al./ Neuromagnetic studies of human vision

In these experiments, we estimate our error in the measurement of sensor location to be +0.5 cm. Although this degree of uncertainty prevents unequivocable identification of the anatomical substrates of the evoked responses, the most parsimonious explanation for our observations is the sequential activation of several areas within occipital cortex. The initial calculated source was near the calcarine fissure in striate cortex, consistent with activation of primary visual cortex. Subsequent sources were in regions suspected to be secondary visual areas, on the basis of anatomical analogy with experimental primates and studies of perceptual effects of human lesions. In any case, our observations support the idea that human visual information processing is sequential to some extent, and distributed across a number of anatomically discrete loci. If this sequence of apparent sources reflects the hierarchy of visual areas, it might be possible to dissect the pathway further through manipulation of visual stimulus parameters. For example, although retinotopic order is present in both V1 and V2, the precise pattern of projections and location and topology of the areas is significantly different. Also, a tendency toward progressive functional specialization is apparent in the characteristic information processing activities associated with secondary visual areas. Higher-order systems may discard some types of information, in order to effectively perform specialized processing tasks. Single unit recordings in monkeys have suggested that processing of color information is a primary function of area V4, and that area MT is particularly involved in processing motion in the visual field [8, 11, 12]. By manipulating the information content of stimuli we may be able to differentially activate such specialized areas, and trace the flow of information through the distributed system. In this report, we described some consequences of manipulating the spatial frequency of stimuli at varying locations in the visual field. We chose to examine spatial frequency for a number of reasons. The size of ganglion cell receptive fields varies across the retina with larger fields in the

425

periphery; this difference alone should establish differential sensitivity to spatial frequencies. There is also a systematic difference in the distribution of cell types across the retina. Finally, psychophysical and evoked potential studies have suggested but do not prove the existence of specialized spatial frequency channels within the visual system. In our experiments, response amplitudes, latencies and calculated current dipole moments varied as a function of eccentricity of the grating within the field and demonstrated an expected statistical interaction between spatial frequency and visual field. CVF "preferred" higher spatial frequency gratings, while RVF "preferred" LO, although respenses to the LO SF grating were strong for both fields of stimulation. We also observed differences in neuromagnetic field distributions as a function of spatial frequency. Based on hypotheses about the consequences of retinal organization on cortical responses, we had expected to find differences in the latencies of components of their relative strengths as a function of spatial frequency and visual field location of the experimental stimuli. We did not expect to find differences in the shapes or characteristic features of evoked field distributions as a function of spatial frequency, since we expected that source location and orientation would be dominated by retinotopic order of the cortex. However, such differences were consistently seen. These appeared to reflect differences in the number a n d / o r relative strength of dipole sources, differences in component latencies, and differences in the location and orientation of sources. Observed differences in evoked field maps might reflect the preferential activation of distinct subsystems within the primary visual pathway as a function of spatial frequency. Recent studies of non-human primates have provided evidence for two distinct streams of visual processing which exist in parallel through areas V1 and V2: the "parvocellular" and "magnocellular" systems. Differences between the systems begin at the retina with anatomically and functionally distinct types

426

J.S. George et a l l Neuromagnetic studies of human vision

of ganglion cells (types A and B); type A cells are larger, have larger receptive fields and respond m o r e transiently than type B cells. The cellular p a t h w a y s are anatomically segregated into interleaved laminae at the L G N (type A cells project to magnocellular layers, type B to parvocellular) which project to anatomically segregated structures within the cortex. The magnocellular system operates at lower spatial resolution and is insensitive to color; however, it is substantially more sensitive to luminance contrast and to temporal aspects of a stimulus. Parvocellular neurons in the L G N are typically sensitive to color contrast, and this sensitivity is maintained in a subsystem of the cortical parvocellular pathway. Consistent with a p r o p o s e d role in high-resolution form perception, the parvocellular system incorporates a r o u n d 90% of cortical neurons. So far, our studies involving m a n i p u l a t i o n of stimulus luminance contrast is consistent with this interpretation; however, at this stage o f analysis it is not clear whether observed differences arise within or between visual areas. I n lower animals, sensory information processing is often highly distributed and highly specialized. F o r example, the retina of the fly performs m o t i o n processing operations which are handled by m u c h later stages of the h u m a n visual pathway. Because h u m a n s perform so m a n y separate analyses of the visual data stream, we might assume that early processing would be generic. However, our studies of spatial frequency processing clearly suggest that specialized and anatomically segregated representations of the visual world exist in parallel at relatively low levels within the h u m a n visual system. These specialized subsystems must reflect a (local) optimal solution to the problem of e n c o d i n g a n d processing visual information.

Acknowledgements T h e authors wish to thank Ivan Bodis-Wollner for his collaboration in the design and execution of some of the spatial frequency experiments re-

ferred to here, and for his insightful discussions of their interpretation; Chris W o o d for useful editorial oversight; and a cast of volunteer subjects w h o contributed their time and patience in extended m a p p i n g procedures. T h e work was supported by the US A r m y Research Institute, D e p a r t m e n t of Energy C o n t r a c t W - 7 4 0 5 - E N G - 3 6 , and the V A / L A N L / U N M m a g n e t o e n c e p h a l o g r a p h y program.

References [1] D. Van Essen, Ann. Rev. Neurosci 2 (1979) 227-263. [2] D.C. Van Essen and J.H.R. Maunsell, Trends in Neurosci. (1983) 370-375. [3] A. Kaneko, Ann. Rev. Neurosci. 2 (1979) 169-191. [4] H.B. Barlow, Proc. R. Soc. London B 212 (1981) 1-34. [5] R. Linsker, Proc. Natl. Acad. Sci. US 83 (1986) 7508-7512. [6] R. Linsker, Proc. Natl. Acad. Sci. US 83 (1986) 8390-8394. [7] R. Linsker, Proc. Natl. Acad. Sci. US 83 (1986) 8779-8783. [8] P. Lennie, Vis. Res. 20 (1980) 561-594. [9] S.M. Zeld, Nature. 274 (1978) 423-428. [10] D.H. Hubel and T.N. Wiesel, J. Physiol. 160 (1962) 106-154. [11] D.H. and T.N. Wiesel, J. Neurophysiol. 28 (1965) 229-289. [12] M.S. Livingston and D.H. Hubel, Science 240 (1988) 740-749. [13] D.C. Van Essen, in: Cerebral Cortex Vol. 3, eds. A. Peters and E.F. Jones (Plenum, New York, 1985) pp. 259-329. [14] R.W. Rodieck, Ann. Rev. Neurosci. 2 (1979) 193-225. [15] K.A. Macko, C.D. Jarvis, C. Kennedy, M. Miyaoka, M. Shinohara, L. Sokoloff and M. Miskin, Science 218 (1982) 394-397. [16] M. Wilson, in: Handbook of Behavioral Neurobiology, Vol. 1, ed. R.B. Masterton (Plenum, New York, 1978) pp. 209-247. [17] Bodis-WoUner, ed., Ann. NY Acad. Sci. 388 (1982). [18] J.E. Desmedt, Visual Evoked Potentials in Man: New Developments (Clarendon Press, Oxford, 1977). [19] B.B. Boycott and J.E. Dowling, Philos. Trans. R. Soc. London 255 (1969) 109-184. [20] G. Wald, Science 162 (1968) 230-239. [21] S. Hecht, S. Shlaer and M.H. Pirenne, J. Gen. Physiol. 25 (1942) 819-840. [22] D.A. Baylor, T.D. Lamb and K.-W. Yau, J. Physiol. (London) 288 (1979) 589-611. [23] G.L. Fain, J. Physiol. (London) 261 (1976) 71-101. [24] J.A. Coles and S. Yamane, J. Physiol. (London) 247 (1975) 189-207. [25] J.K. Bowmaker and H.J.A. Dartnall, J. Physiol. (London) 298 (1980) 501-511. [26] P.K. Brown and G. Wald, Science 144 (1964) 45-52. [27] W.B. Marks, W.H. Dobelle and E.F. MacNichol Jr., Science 143 (1964) 1181-1183.

J.S. George et al./ Neuromagnetic studies of human vision [28] E.T. Rolls and A. Cowey, Exp. Brain Res. 10 (1970) 298-310. [29] V.H. Perry and A. Cowey, Vis. Res. 25 (1985) 1975-1810. [30] O.D. Creutzfeldt, in: Biophysics, eds, W. Hoppe, W. Loman, H. Markl and H. Ziegler (Springer, Berlin, 1983). [31] S.W. Kufl]er, J.G. NichoUs and A.R. Martin, From Neuron to Brain (Sinauer, Sunderland, MA, 1984). [32] E.L. Schwartz, Science 227 (1985) 1066. [33] P.M. Daniel and D. Whitteridge, J. Physiol. (London) 159 (1961) 203-221. [34] A. Cowey, J. Neurophysiol. 27 (1961) 366-393. [35] D.A. Jeffreys and J.G. Axford, Exp. Brain Res. 16 (1972) 1-21. [36] T.M. Darcey, J.P. Ary and D.H. Fender, Prog. Brain Res. 54 (1980) 128-134. [37] Y. Fukada and J. Stone, J. Neurophysiol. 37 (1974) 749-772. [38] F.M. De Monasterio and P. Gouras, J. Physiol. (London) 251 (1975) 167-195. [39] O. Braddick, F.W. Campbell and J. Atkinson, in: Handbook of Sensory Physiology, Vol. 8, eds. R. Held, H.W. Leibowitz and H. Teuber (Springer, Berlin, 1978). [40] B.G. Breitmeyer and L. Ganz, Psychol. Rev. 83 ( 1 9 7 6 ) 1-36 [41] L. Maffei, in: Handbook of Sensory Physiology, Vol. 8, eds. R. Held, H.W. Leibowitz and H. Teuber New York: (Springer, Berlin, 1978). [42] C. Blakemore and F.W. Campbell, J. Physiol. (London) 203 (1969) 237-260. [43] F.W. Campbell and L. Maffei, J. Physiol. (London) 207 (1970) 635-652.

427

[44] C. Blakemore, J.P.J. Muncey and R.M. Ridley, Vis. Res. 13 (1973) 1915-1931. [45] R.L. DeValois, H. Morgan and D.M. Snoddedy, Vis. Res. 14 (1974) 75-81. [46] D.J. Tolhurst, Vis. Res. 15 (1975) 1143-1149. [47] M.R. Hatter, V.L. Towle and M.F. Musso, Vis. Res. 16 (1976) 1111-1117. [48] R. Blake and E. Levinson, Exp. Brain Res. (1977) 221-222. [49] C. Aine, J. George, P. Medvick, S. Supek, E. Flynn and I. Bodis-Wollner, in: Advances in Biomagnetism, eds. S.J. Williamson, M. Hoke, M. Kotani and G. Stroink (Plenum, New York), in press. [50] J.S. George, P.S. Jackson, D.M. Ranken and E.R. Flynn, in: Advances in Biomagnetism, eds. S.J. Williamson, M. Hoke, M. Kotani and G. Stroink (Plenum, New York), in press. [51] E. Maclin, Y.O. Okada, L. Kaufman and S.J. Williamson, Nuovo Cimento 2 (1983) 410-419. [52] P.A. Medvick, P.S. Lewis, C. Aine and E.R. Flynn, in: Advances in Biomagnetism, eds. S.J. Williamson, M. Hoke, M. Kotani and G. Stroink (Plenum, New York), in press. [53] B.J. Schwartz, C. Salustri, L. Kaufman and S.J. Williamson, in: Advances in Biomagnetism, eds. S.J. Williamson, M. Hoke, M. Kotani and G. Stroink (Plenum, New York), in press. [54] G.V. Simpson, M. Scherg, W. Ritter and H.G. Vaughan Jr., Paper presented at EPIC IX: International Conference on Event-Related Potentials of the Brain, May 28, 1989, Noordwijk, The Netherlands.