Complex-cell receptive field models

Progress in NeurobiologyVol. 31, pp. 285 to 309, 1988 Printed in Great Britain. All rights reserved

0301-0082/88/$0.00 + 0.50 Copyright © 1988 Pergamon Press plc

COMPLEX-CELL RECEPTIVE FIELD MODELS HEDVA SPITZER* a n d SHAUL HOCHSTEIN'{" *Dept of Biomedical Engineering, Technion-Israel Institute of Technology, Haifa 3200, Israel t Dept of Neurobiology, Institute of Life Sciences, Hebrew University, Jerusalem, Israel

(Received 14 August 1987)

CONTENTS I. Introduction 2. Classical definitions of complex cells, classical stimulation paradigms, and the classical receptive field models 2.1. Hubel and Wiesel's classifications 2.2. Deviations from the classical classification schemes 3. Complex cells as neuronal filters; sinusoidal grating stimulation in the study of cortical cell properties 3.1. The Fourier methodology as applied to the visual cortex 3. I.I. Systems analysiw-sinusoidal grating stimulation 3.1.2. Contrast reversal grating stimulation 3.1.3. Drifting sinusoidal grating stimulation 3.2. Spatial and temporal Fourier components 3.2.1. Linearity tests 3.2.2. Temporal and spatial linearity and nonlinearity 4. Receptive field models and the temporal forms and spatial phase dependences they predict 5. Cell type classification based on the responses to sinusoidal grating stimulation 5.1. Spatial frequency tuning 5.2. Spatial phase~modulated vs unmodulated responses 5.3. Receptive field discreteness and the Fourier even:odd harmonics ratio 6. Parallel or hierarchical processing? 7. Complex cell response-form dependence on stimulus parameters 7.1. Cat 7.1.1. Response-form dependence on spatial phase 7.1.2. Spatial frequency dependence 7.2. Monkey 8. A complex ceU receptive field model 8.1. The two subunit model 8.2. Model response dependence on spatial phase and frequency 8.3. Model spatial frequency tuning curve 9. Various cell types produced by variations of receptive field structure, using a single summation rule 10. Conclusion Acknowledgements References

1. INTRODUCTION It is over 25 years since the inception of Hubel and Wiesel's pioneering work (Hubel and Wiesel, 1959) in which they described and classified the visual cells of the primary mammalian cortex, and suggested models for their different receptive fields. Since then a large number of reports have appeared that characterize the properties of these visual cortical cells in a more quantitative way. The accumulation of more data, including data that derive from different methods of stimulation and analysis, has led to the discovery of receptive field properties that are not always consistent with the original models suggested by Hubel and Wiesel (1959, 1962, 1968). The aim of this review is to review the main concepts concerning visual complex cell receptive 285

285 286 286 287 287 288 288 289 289 289 289 290 291 294 295 296 296 297 300 300 300 300 301 303 304 304 305 305 307 307 307

field models, emphasizing our own recent model (Spitzer and Hochstein, 1985b), in light of the different descriptions of complex cell properties that exist in the literature. David Hubel said in his Nobel prize address, Complex cells come in a wide variety of subtypes... In the past 10 or 15 years the subject of cortical receptive field types has become rather a jungle, partly because the terms simple and complex are used differently by different people and partly because the categories themselves are not cleanly separated. We hope that this review will serve to help beat a path through this jungle and to help clean up, at least in part, the separation between these categories. One of the questions we will address is what characteristic unifies complex cells into a single class and what

286

H. SPITZERand S. HOCHSTEIN

characteristics differentiate them into a wide variety of subtypes.

(A)

2. CLASSICAL DEFINITIONS OF COMPLEX CELLS, CLASSICAL STIMULATION PARADIGMS, AND THE CLASSICAL RECEPTIVE FIELD MODELS The cells in the primary visual cortex are classically divided into three categories called simple, complex and hypercomplex, according to the different characteristics of their receptive fields. (We will not discuss here the circularly symmetric cells found in Monkey striate cortex.) There are many reports classifying visual cells as belonging to each of these categories according to various criteria (Hubel and Wiesel, 1959, 1962, 1968, 1977; Bishop and Henry, 1972; Palmer and Rosenquist, 1974; Singer et al., 1975; Cynader et al., 1976; Schiller et al., 1976a; Gilbert, 1977; Henry, 1977; Henry et al., 1979; Kato et al., 1978; Berman et al., 1982). We discuss first the classical description of these neurons introduced by Hubel and Wiesel, and then additional schemes suggested by later studies.

2.1. HUBEL AND WIESEL'SCLASSIFICATIONS Hubel and Wiesel gave the name simple cells to cells that exhibited the following properties: (1) They were subdivided into distinct excitatory and inhibitory parts (2) there was summation within the separate excitatory and inhibitory regions (3) there was antagonism between excitatory and inhibitory regions; and (4) it was possible to predict responses to stationary or moving spots of various shapes from a map of the excitatory and inhibitory areas. Hubel and Wiesel did not find a clear cut definition like the above which was suitable for describing complex cells. They wrote t h a t " . . , cells [which] were termed 'complex'... responded to variously-shaped stationary or moving forms in a way that could not be predicted from maps made with small circular spots". They added that "the principles of summation and mutual antagonism, so helpful in interpreting simple fields, did not generally hold". Complex cells were regarded generally by these authors, and by others in their wake, as giving more or less identical responses to the onset and the offset of the stimulus anywhere in the receptive field--an O N - O F F type response, and a homogeneous receptive field (but see below). Furthermore, complex cell receptive fields were usually larger than simple cell fields. Thus, complex cells have four defining properties: nonlinearity (which prevents the prediction of the response to compound stimuli from knowledge of the responses to simple stimuli); O N - O F F responses to local stimuli (or frequency-doubling); a homogeneous receptive field (the absence of a single area of peak sensitivity in the receptive field); and a large receptive field area. Many subsequent reports postulated as a working definition for complex cells the presence of an overlap of the ON and OFF regions throughout the receptive

.

I~ ~I~ _

.

.

.

.

.

.

~' -,_

_

L FIG. 1. Hubel and Wiesel's models for cortical simple cells (A) reproduced from Fig. 19 of Hubel and Wieset, 1962, and for cortical complex cells (B) reproduced from Fig. 20 of Hubel and Wiesel, 1962. Simple cells are seen as deriving excitatory input from LGN units with center-surround receptive field organization (presumably X-cells) while complex cells are seen as deriving excitatory inputs from simple cells. Though the summation rules are not given, presumably the simple ceils sum linearly their LGN inputs while complex cells sum nonlinearly their inputs. See text. field, even though Hubel and Wiesel were careful to describe also a small proportion of complex ceils that showed discrete zones in part of their receptive fields (Hubel and Wiesel, 1962; see also introduction in Dean and Tolhurst, 1983). When Hubel and Wiesel (1962) summarized their results, however, they wrote that "complex fields, however, differed from simple fields in that a stimulus was effective wherever it was placed in the field, provided that the orientation was appropriate". Thus, the emphasis placed by Hubel and Wiesel was on the uniformity of the effectiveness of the receptive field rather than on its O N - O F F character, or lack of discreteness. The classical model that was given to account for the spatial structure of the receptive fields of simple cells was convergence of geniculate fibers having receptive fields with ON or OFF centers aligned in a specific orientation and restricted to an appropriate retinal region. This model is reproduced in Fig. I(A). The inhibitory flanks of the simple cell receptive field were seen as built from the outlying inhibitory surround regions of the same geniculate fields. According to Hubel and Wiesel's model these flanks could be reinforced by appropriately placed OFF centers of additional geniculate cells. This model would lead to disinhibitory outer surrounds due to the surrounds of these OFF cells (see Maffei and Fiorentini, 1972; also Singer and Creutzfeldt, 1970; Hubel and Wiesel, 1961). The properties of complex cells, on the other hand, are not easily accounted for by simply supposing that these complex receptive fields are fed directly by lateral geniculate ceils (Hubel and Wiesei, 1962), since their spatial frequency and orientation proper-

COMPLEX-CELL RECEPTIVE FIELD MODELS

ties cannot be explained on the basis of their receptive fields as determined by spot stimuli alone. Hubel and Wiesel assumed a hierarchical model in which simple cells are afferent to complex cells. The latter determine the bar width and axis of orientation which are optimal for visual stimulation of the receptive fields of complex cells, as displayed in Fig. 1 (B). Actually, this presumed hierarchy of cortical cells could be used as a distinguishing property of simple and complex cells. However, relatively few studies measured this property directly (Gilbert, 1977; Gilbert and Wiesel, 1979; Bullier and Henry, 1979a; Bullier et al., 1982; Toyama et al., 1977, 1981a, b; Tanaka, 1983). Other studies have suggested the existence, or nonexistence, of a hierarchy not on the basis of direct experimental evidence. Rather, they have seen hierarchy as a mechanism on which to base their model. We will discuss the question of hierarchy vs parallel processing separately (Section 6). We will not discuss the classifying properties of hypercomplex cells and related subgroups in this report. These are now seen to be cells whose receptive fields are "end-stopped", a property found in subgroups of both simple and complex cells. Most of what we will say about simple and complex cells applies also to hypersimple and hypercomplex cells as well.

2.2. DEVIATIONSFROM THE CLASSICAL CLASSIFICATIONSCHEMES The first reports that challenged Hubel and Wiesel's classification were of Henry and his colleagues. They actually suggested a finer classification scheme (Henry et al., 1972, 1978, 1983; Henry, 1977), and introduced a modified nomenclature, replacing the hierarchical names with initial letters. The simple cell, for example, became an S cell. However, they altered the definition of this group of cells, requiring only that their receptive fields should be made up of discrete regions where stimulation would induce either an ON or an OFF response. The complex cells were subdivided into a number of subgroups. In the C group (the closest to the original complex cells) they included only cells called complex by Hubel and Wiesel and only those complex cells that have uniform receptive fields in which there are mixed ON and OFF discharges along the entire receptive field. Another subgroup of complex cells was the A group described as having a non-uniform receptive field including distinct areas of ON or OFF discharge. This group was distinguished from the simple or S cell by its summation characteristics (Henry, 1977; Gilbert, 1977). The B group of cells have small receptive fields in which aspect they are similar to simple cells, but they respond to a flashing bar with an O N - O F F type of response, similar to complex cells (Henry, 1977). Schiller and his colleagues (1976a, b) classified distinct classes of cells: the S type, CX type and T type, as well as four subgroups of the S type. Their classification was based on different cell properties such as: orientation tuning, optimal direction of movement, overlap of the subfields according to their

287

responses to either a light or dark edge or both, and temporal modulation. Heggelund (1981b, 1985) found that complex receptive fields could be subdivided into antagonistic subregions like simple cells when tested with a dual stimulus technique. He also found linear spatial summation within the subregions just as found in simple cells. These findings suggested that complex cells may differ from simple cells mainly in the ON, OFF, or O N - O F F nature of their subregions rather than in the type of summation between them. Each class of cells has been described also in terms of other response properties, even though these were not the key distinguishing properties. Such characteristics include spontaneous activity, dimensions of discharge regions, inhibitory regions, orientation specificity, direction specificity, velocity dependence, cortical layer, etc. (Hubel and Wiesel, 1962; Goodwin and Henry, 1975; Pollen and Ronner, 1975; Henry et al., 1974; Rose and Blakemore, 1974; Heggelund and Albus, 1978; Heggelund, 1981a, b; Movshon, 1975; Hammond and MacKay, 1975, 1977; Palmer and Rosenquist, 1974; Finlay et al., 1976; Gilbert, 1977). The contribution of the studies reviewed in this section to the cumulative understanding of cortical cell receptive field properties naturally complicated the picture. Taken together they emphasized the discrepancies between the results of different classification criteria. Thus, the cumulative result of these studies was not a unified agreed-upon classification procedure, nor even a decision on which property of the receptive field was essential for this definition. Still, the existence of a dichotomy of cortical cells was generally accepted. The attempt to find a single defining characteristic stemmed from the desire to uncover its visual information processing function. The wealth of cell properties found to vary in these cells precluded a simple and definitive understanding of the mechanism underlying the dichotomy. In the following section we discuss the introduction of another simulation method that dealt with this question in a different manner.

3, C O M P L E X CELLS AS NEURONAL FILTERS; SINUSOIDAL GRATING STIMULATION IN THE STUDY OF CORTICAL CELL PROPERTIES The characterizations and models described until now were derived from responses of cortical neurons to light and dark bar stimuli. Parallel to these studies there are numerous reports based on cortical responses to stimuli which comprise a complete basis set. The Fourier set of sinusoidal gratings is the most popular basis set (not without reason; see below) and we shall devote the majority of this review to this stimulation method. However, other basis sets have been used as well. For example, the set of Gabor functions has received a lot of attention lately (Marcelja, 1980; Sakitt and Barlow, 1982; Kulikowski et al., 1982; Pollen and Ronner, 1983; Pollen et al., 1984; Zeevi and Ponat, 1984; Palmer and Jones, 1984; Palmer et al., 1985; Jones and Palmer, 1987a, b). These functions are similar to sinusoidal gratings but are multiplied by a Gaussian function so

H. SPITZER and S. HOCHSTEIN

288

that they have a limited spatial extent though of course a broader spatial frequency bandwidth. Thus, they seem to match neuronal receptive fields better. Another basis set is the set of orthogonal two dimensional black and white Walsh patterns (see Section 3.2.2). Finally, even the set of points of light in Cartesian coordinate space forms a complete basis set: they are orthogonal patterns and any complex pattern may be constructed from them. These basis sets have mainly been used to study the spatial (and spatiotcmporal) profiles of cortical units. We shall emphasize the Fourier technique which is more widespread in its usage and which has been used to analyze the summation characteristics within cortical receptive fields. 3.1. THE FOURIER METHODOLOGYAS APPLIED TO THE VISUAL CORTEX

3.1.1. Systems analysis--sinusoidal grating stimulation A full description of sinusoidal grating stimulation and of spatial frequency analysis is available in the literature (see for example the recent review by Shapley and Lennie, 1985). The introduction of the sinusoidal stimulation paradigm was accompanied by the application of analytic methodologies that were more quantitative and afforded new insights into the mechanisms that are responsible for the neuronal properties that have been described above. For example, the degree of temporal and spatial summation linearity of the receptive field could be calculated from these analyses. The use of spatial frequency gratings became quite a popular tool for three main reasons: First, as mentioned above, this method of stimulation is close to the mathematical idea of a complete and orthogonal basis set. In this way, the visual cells may be treated as neuronal filters of visual signals, and by this method it is possible to characterize the properties of the neuronal filter, in this case the receptive field structure. Second, there are advantages to choosing this Fourier basis as a set for stimulation, the main one being that it is easy to test if the filter is a linear one in the time and/or the space domain: When a sinusoidal input is applied to a linear filter the output will also be sinusoidal and of the same frequency as the input. The relative amplitude of the sinusoidal modulation of this linear output may be different from that of the input and it may also be shifted in phase. Thus, this visual stimulation paradigm is in effect a Fourier analysis of the receptive field of the unit. The Fourier analysis is direct since each stimulus used (each sinusoidal grating) is a single Fourier component and tests the filter characteristics of the neuron to that component. (See a wider discussion in the reviews by De Valois and De Valois, 1980; and Shapley and Lennie, 1985.) Third, visual cortical cells are tuned over the dimension of spatial frequency, as are retinal ganglion and LGN cells so that their tuning parameters may be studied and compared; this comparison is facilitated by the possibility of limiting it to a few tuning curve parameters, the spatial frequency of

peak sensitivity and the tuning bandwidth. The fineness of the tuning to spatial frequencies has even been evoked as proof that these cells are spatial frequency detectors rather than bar or edge detectors as suggested by Hubel and Wiesel. For example, Albrecht et al. (1980) showed that striate cortical cells are more responsive to gratings than to bars or edges, and are much more selective along the dimension of spatial frequency than along the dimension of bar width. Any function may be introduced as a linear sum of sinusoidal waves of different frequencies, phases and amplitudes. The Fourier transform yields those coefficients which are complex numbers, containing information on the amplitude and phase of each sine wave component. (If only the amplitudes of the waves are represented as a function of the frequencies the result is called the amplitude spectrum.) One may predict the response to any arbitrary stimulus as follows: analyze the stimulus itself into its sine (and cosine) Fourier components, pass each component through the receptive field filter (multiplying the amplitudes of the stimulus and the receptive field profile for each spatial or temporal frequency, respectively, and adding their spatial or temporal phases), and finally perform an inverse Fourier analysis of the result. Thus, one may calculate the response of a linear neuron to any arbitrary stimulus on the basis of the experimentally derived responses to the complete set of sinusoidal stimuli. Theoretically, an analogous procedure could be used for any complete and orthogonal basis set. For example, an "n x n" grid of spots of light forms a complete basis set (Cartesian coordinate space). We may construct any arbitrary stimulus from the sum of properly placed spots of light of proper intensity and timing. Therefore, we should be able to derive the response to any stimulus from the sum of the responses to these spots of light. This is indeed an appropriate method for studying the retina and LGN (see above), and has been used with considerable success also for simple cells (Jones and Palmer, 1987). However, as Hubel and Wiesel already noted, the responses of cortical complex cells to complex stimuli cannot be predicted from their responses to small spots of light, mainly because these cells are not very sensitive to spots of light and because they may respond to them in an O N - O F F fashion. We should not be surprised by the failure of this technique for a nonlinear system since it is based on the assumption of linearity of the filter. However, if the Cartesian spots-of-light basis set fails, why then may we use the Fourier basis set on this nonlinear system? This is because one nonlinearity of the cortex is a selection for elongated, oriented stimuli. Spots are generally ineffective stimuli. Thus, the Cartesian coodinate set of spots is not an appropriate basis set for studying these receptive fields. One can use the Fourier spatial frequency set of grating stimuli since these pass the nonlinear filter. Still, one must view with caution the implications of these studies as the full power of linear systems analysis is only valid in linear systems. Generally one may assume some small range over which any system is linear. A more interesting case is when even though the system is not linear as a whole (over all stimulus variables), it is still linear over one

289

COMPLEX-CELL RECEPTIVE FIELD MODELS

or more variable. For example, the neuronal response may be nonlinear as a function of light intensity but may still be linear as a function of spatial phase. Another case where linear analysis may be used in a nonlinear system is when the type of nonlinearity may be identified, for example as a rectification stage, so that beyond a certain level of one of the variables the system can be described as linear or semi-linear. Expanding on the last example, if a full-wave rectification stage renders a response frequencydoubled and thus highly temporally nonlinear (the fundamental frequency is now absent and only even Fourier components are present), then the response amplitude dependence on spatial phase may still be sinusoidal--a definite sign of linear spatial summation. We summarize briefly the two methods that are commonly used with sinusoidal grating stimulation, namely, contrast reversal gratings and drifting gratings, and in the following section we analyze the temporal and spatial Fourier components of the histogram responses.

tivity function (Enroth-Cugell and Robson, 1966). The spatial frequency response can be also determined without adjusting the contrast (see Shapley and Lennie, 1985). 3.2. SPATIAL AND TEMPORAL FOURIER COMPONENTS

3.2.1. Linearity tests To study the linearity of a receptive field one may perform a Fourier analysis of the neuron response over the stimulation temporal cycle for standing grating stimulation, and over the spatial cycle for moving stimulus stimulation. (In fact, for a drifting sinusoidai grating, the spatial and the temporal cycles are related since their ratio is the drift velocity.) Like any other function, the neuron response may be represented by a linear sum of sinusoidal waves of different frequencies, phases and amplitudes. The nth temporal Fourier component amplitude F,(n) is the r.m.s, of the cosine and sine components, 1 ([- 2N

-]2

3.1.2. Contrast reversal grating stimulation A contrast-reversal sine-wave grating is a standing sine wave of spatial frequency fx and spatial-phase whose contrast C is temporally sine-wave modulated at a frequency f,. Thus, the illumination at any point on the screen is,

S(x, t) -~ Co + Csin(2nf~t )sin(2nfxx + 6#) where Co is the mean illumination. In order to get the full spatial picture of the filter (in this case the receptive field structure), it is necessary to collect data for the different spatial phases for each spatial frequency chosen. The term contrastreversal grating may be used also for a standing grating whose contrast modulation is not sinusoidai (but rather, for example, square-wave). Contrastreversal gratings with sinusoidal modulation are also called counter-phase gratings. 3.1.3. Drifting sinusoidal grating stimulation A drifting sinusoidal grating is a moving sine wave of spatial frequency, fx and spatial phase, 2nftt:

S(x, t) = Co + Csin(2nfxX + 27~f,t) The main difference between these two methods of stimulation is that in the standing sine wave the whole receptive field "sees" the grating stimulation change its contrast in the same direction at the same time, whereas during drifting grating stimulation the stimulus is "scrolled" over the receptive field. From the responses to contrast reversal stimulation one can differentiate between the temporal properties and the spatial properties of the receptive field whereas it is not easy to derive this information from the responses to drifting sinusoidal gratings. We shall discuss below the predictions for different receptive field models through hypothetical response histograms derived from the two stimulation methods. For both methods, if the spatial frequency is varied and the contrast of the grating is adjusted, it is possible to determine the spatial frequency sensi-

F 2N

q2)1/2

+ L,.ZR,,)sin(2, nt/2N)j

where R(I) is the response histogram amplitude at time t (histogram bin number t), the response is sampled 2N times per stimulation cycle, and n is the Fourier component number which runs from 0 to N. A similar procedure may be applied for the histogram responses derived from moving gratings. Most response histograms (especially when using contrast reversal standing gratings) appear to contain half-wave or full-wave rectified sinusoidai forms, or combinations of these, where the x-axis extends one grating stimulation temporal cycle. Figure 2 displays theoretical responses of these types and the amplitudes of their temporal Fourier components. The more transient the response, the greater the importance of the higher Fourier components. Note that response histograms that contain two equal peaks (right column in Fig. 2) do not have any odd harmonics. The response histograms to drifting gratings are similar to those above but often appear to be superimposed on an elevated maintained response level. This change in the response histogram will be expressed in the amplitude spectrum only by a larger 0th harmonic ( D e component). The relative amplitudes of the rest of the harmonics will not be changed. Enroth-Cugell and Robson (1966) introduced the procedure of electrophysiological tests for temporal and spatial lincarity. Using stationary or moving grating stimulation, they studied the response dependence of retinal ganglion cells on spatial phase. For a linear system this dependence should be sinusoidal since a sinusoidal input must lead to a sinusoidal output in a linear system. This sinusoidal dependence includes a null point, a spatial phase of the grating where all the excitatory influences of the grating stimulation are exactly cancelled by the inhibitory influences. They found X-cells to be spatially linear in their receptive fields in this way and Y-cells

290

H. SPITZERand S. }-IocI~TEIN FOURIER TRANSFORM

HISTOGRAM

FOURIER TRANSFORM

HISTOGRAM

ot. OL_ o0 °i,o, ,.o ~

~ 0.~

,.o

~- o.~ .

o

~2.0

tnl.0

2.0

El.0

0.5

1.0

0.5

o

-~ 0 ~

~ o "= h ;

°

.-~ z

o

t,

.-i.o

~" L.O

~

~

o

~o.~ o ,

~

L

32

64

Bin number

0

16

32

Fourier

components

~

'== i.o

~l.

~ 0

.

I

32

64

Bin number

0

t., ....

;

16

32

Fourier

components

FIG. 2. Idealized response histograms and their temporal Fourier transforms. Top row demonstrates Fourier analysis of response histograms which are sinusoids following half-wave rectification (left column) or following full-wave rectification (right column). Lower histograms contain successively more

"transient-like" time courses, gradually emphasizing higher Fourier components while the fundamental and second harmonic components are diminished (left and right columns, respectively). Note that the two-peaked time courses contain only even Fourier harmonics, whereas the single-peaked time courses contain odd components as well. (Reproduced from Spitzer and Hochstein, 1985a).

to be nonlinear. Hochstein and Shapley (1976a, b) expanded this linearity testing methodology, using contrast reversal sinusoidal gratings. They used a series of spatial frequencies to study the acuity limits of the linear and nonlinear components of the receptive fields. In this way they were able to show that Y-cells sum nonlinearly a multitude of scattered subunits coextensive with the classical linear surround area. When studying Fourier analyses of response histograms, one often uses only the first harmonic (fundamental) for testing the linear component, and the second harmonic, for counterphase gratings, or DC component, for drifting gratings, for testing the nonlinear component of the receptive fields (Hochstein and Shapley, 1976a, b; Movshon et al., 1978a, b, c; De Valois et al., 1979; Dean, 1981; Dean and Tolhurst, 1983; Jones and Palmer, 1987). As discussed in the following section, when responses are rectified or transient it is preferable to include in these tests the higher order harmonics. 3.2.2. Temporal and spatial linearity and nonlinearity Temporal linearity is not necessarily linked to spatial summation linearity. It is the spatial phase dependence that directly measures the linearity of the summation of spatially distinct receptive field areas. A response time-course histogram which shows two peaks, for example, may be related to spatial summation nonlinearity, but may also be found in receptive fields which have spatial summation linearity.

A local nonlinearity is any stimulus-response transformation which does not obey the linearity rules for stimuli at a single location (in particular, a sinusoidal response time-course to sinusoidal modulated stimuli). Locally, it is the contrast which is changed sinusoidally as a function of time so that these deviations of the sinusoidal wave of the output are due to contrast nonlinearities. When the stimulus-response transformation obeys the linearity rules for stimuli in any one location but the transformation for the sum of stimuli in more than one location is nonlinear, the nonlinearity is a spatial summation nonlinearity. Temporal nonlinearity is measured by the presence of Fourier harmonics other than the fundamental in the response to sinusoidally modulated stimuli. All cortical responses contain many other harmonics in their responses, partially deriving from the different degrees of transientness of their responses and partially from their lack of spontaneous activity (see details below, Section 4). The lack of spontaneous activity prevents reduction of the response, and produces a rectified response which contains also higher Fourier harmonics (see demonstration in Fig. 2). Similarly, the more transient the response, the greater the weight of the higher Fourier components: As demonstrated in Fig. 2, the amplitude of the first and/or second harmonic is reduced for more transient responses in a normalized Fourier transform. One would wish to emphasize in the analysis of the neuron response, the degree of phase-locking to the stimulus temporal frequency (or spatial frequency)

COMPLEX-CELL RECEPTIVE FIELD MODELS

regardless of the presence of a temporal nonlinearity. Thus, in order to compare responses having different time courses (one peak versus two peaks) irrespective of their degrees of transientness, it is preferable to study not only the Fourier fundamental and second harmonic components, but rather to take into account all the Fourier Harmonics (Spitzer and Hochstein, 1985a, b; Baker and Hess, 1984; Baker and Cynader, 1986). All the odd and all the even component are (r.m.s.) summed and analyzed separately (see Figs 10-13 for examples of this analysis methodology). We call these sums the odd and even "portions" of the response, respectively. Using this method, the odd and the even Fourier portions are tested separately for their spatial phase dependences. When the response has even harmonics whose amplitudes vary sinusoidally with the spatial phase of counter-phase grating stimulation, we may say that the receptive field is temporally nonlinear but has linear spatial summation. In order to measure the spatial linearity, second, stimulus spatial-phase, Fourier analyses are performed separately for the odd and even response portions (Spitzer and Hochstein, 1985b). The n th component of this spatial phase Fourier transform is defined as,

L¢= l

+

[F,(4)sin(2nn¢/2v)]2 ¢=1

Note that for the responses to contrast reversal gratings (as opposed to drifting gratings) one may thus separate the spatial dependence from the temporal dependence. Having derived quantitative measures for the odd and even portions of the response to contrast reversal grating stimulation at each spatial phase, one may derive the spatial summation characteristics of the receptive field regardless of the response amplitude since the degree of linearity is measured as a percentage. The linearity of spatial summation is reflected in the spatial phase dependence of the odd and even portions. Since the odd portion peak jumps 180 degrees temporal phase for a shift of 180 degrees in stimulus spatial phase, we may assign a positive value to the odd portion amplitude whenever the peak appears in one temporal phase and a negative value when it appears in the 180 degree shifted position. For a receptive field with linear spatial summation we then obtain a sinusoidal dependence of the odd portion on spatial phase. The same procedure cannot be applied to the even portion as this part of the response has equal peaks in each half temporal cycle and there is no jump on the basis of which to assign a negative value. The spatial phase dependence of the even portion of the response of a receptive field with linear spatial summation therefore has the form of a full-wave rectified sinusoid. A second Fourier analysis may be performed on the odd and the even portions' spatial phase dependence (Spitzer and Hoehstein, 1985a, b). Spatial linearity is expressed by the odd portion containing mainly a fundamental component and the even portion (resulting only from a threshold rectification following linear summation)

291

containing a significant second harmonic component (as does a half-wave rectified sinusoid). 4. RECEPTIVE FIELD MODELS AND THE TEMPORAL FORMS AND SPATIAL PHASE DEPENDENCES THEY PREDICT In this section we will outline the different response histograms that are expected when applying grating stimulation of the two methods described above to cells with receptive fields corresponding to the different receptive field models that have been proposed in the literature. We shall refer to these models and their expected responses when we cite the physiological data below (e.g. in Section 5 where we describe the responses of cortical neurons to sinusoidal grating stimuli). Figure 3 presents the different shapes which histograms are expected to have in response to these two methods of stimulation for different receptive field models. In addition, we show, for each model, the spatial phase dependence of the odd and even portion of the response to counter-phase grating stimulation. The diagrams in Fig. 3 are schematic without reference to their relative amplitudes. The response of a linear visual neuron to standing or drifting grating stimulation should have a sinusoidal form of the same temporal frequency as the stimulus. However, most cortical neurons do not have a maintained level of background activity, especially not the simple cells (Pettigrew et aL, 1968; Rose and Blakemore, 1974; Movshon et al., 1978a). Thus, their responses appear half-wave rectified: They respond during half the sinusoidal stimulation cycle and are silent during the other half. The left column of Fig. 3 presents the theoretically expected response histogram from a simple cell to standing and drifting gratings (top and bottom rows, respectively). The two forms of grating stimulation induce similar shapes (time-courses) of response histogram, but using the drifting grating stimulation method it is not possible to differentiate between the temporal properties and the spatial properties (by testing the responses to each "frozen" spatial phase separately), as described above. The spatial phase dependence of the response to counter-phase grating stimulation is presented in the middle row of Fig. 3. For the simple cell the shape of the response is independent of spatial phase so that the magnitudes of the odd and the even portions of the response maintain a constant ratio. However, as mentioned above, there are different conventions as to the way the amplitudes of these portions are plotted; a negative value is assigned to the odd portion when the peak appears in the second half of the response histogram. Thus, the spatial phase dependence of the odd portion has the form of a sinusoid. The same procedure cannot be applied for the even portion as this part of the response has equal peaks in each half temporal cycle. Thus, the spatial phase dependence has the form of a full-wave rectified sinusoid. The second column of Fig. 3 presents the shapes of the histograms expected from the classical receptive field model for complex cells. This model assumes that many input subunits (classically assumed to be derived from simple cell receptive fields) are summed

292

H. SP]TZERand S. HOCHSTEIN simple cell

classic i complex

Y- like response

mixed B- cell

cel I

intermediate cell

rv'q &

O.

o Q. U)

E

FIG. 3. Response histogram shapes expected for stimulation by counter-phase (upper rows) and drifting (bottom row) gratings for different receptive fields models. The center row demonstrates, for each model, the spatial phase dependence of the odd and the even portion of the response to counter-phase grating stimulation. The response of a linear visual neuron (left column) should have a sinusoidal form of the same temporal frequency as the stimulus. The lack of background activity in simple cells makes their responses appear half-wave rectified. The second column presents the histograms expected from the classical receptive field model for complex cells which assumes that many input subunits are summed nonlincarly to form the large, homogeneous, ON-OFF responding, complex cell receptive field. The third column displays the cortical responses expected from cells whose only inputs are from LGN Y-cells. Different responses result from stimulation with low and with high spatial frequency gratings (as shown in the two responses displayed in each row of this column). The fourth column demonstrates the expected responses for B ceils which have only one subregion that consists of an ON and an OFF subunit which are spatially coincident, but whose responses are summed nonlinearly. The fifth and sixth columns demonstrate the expected responses for complex cells according to the sub-unit model, for the sub-types called mixed cells and intermediate cells, respectively. See text. nonlinearly to form the large, homogeneous, O N O F F responding, complex cell receptive field. During stimulation with a standing grating, all the subunits see either the (local temporal) peak or the valley of the stimulus intensity at the same time. Since there may be many cycles of the stimulation grating within the homogeneous receptive field there are an equal number of peaks and valleys and hence an equal number of regions responding to the two opposite phases of the contrast and we expect to get equal peaks during the two halves of the temporal cycle of the grating stimulation. Thus, the response to counter-phase grating stimulation will have only even harmonic components. The assumption of homogeneity and overlap of the subunits in the spatial domain predicts an independence of this response (containing only an even

portion) as a function of the spatial phase. Thus, the response to counter-phase grating stimulation is independent of spatial phase (Fig. 3, middle row) and the response to a drifting grating is a DC elevation of the neuronal activity (Fig. 3, bottom row). This classical receptive field model, originally proposed on the basis of the responses to light bar stimuli, received additional experimental support from the responses of some cortical cells to drifting spatial frequency grating stimulation. Movshon and Tolhurst (1978b) and Schiller et al. (1976c) reported that these responses of complex cells were usually unmodulated. However, a few of these cells showed a modulated response at the temporal frequency of the test grating when applying low spatial frequency stimulation. Glezer and his colleagues (1982) used masks to test

COMPLEX-CELL RECEPTIVE FIELD MODELS

the responses of different parts of the receptive fields of complex cells, and found strongly modulated responses which were spatial-phase dependent. They suggested that complex cells receive only a few inputs, that these are themselves well-modulated, and that complex cells then sum these inputs in different phase relationships and thus produce different levels of modulated and unmodulated activity to a drifting sine-wave grating. Recently, Bonds (1984, 1987) found that complex cell unmodulated responses to drifting gratings appeared modulated when the GABA blocker bicucullin was applied locally to the cortex. Glezer and his colleagues (1982) suggested that complex cells receive LGN Y-cell input and that their receptive field properties derive from this source. The third column of Fig. 3 displays the predicted responses of cortical complex cells based on the Hochstein and Shapley (1976b) model for Y-cell receptive fields. The latter described a linear mechanism which is built from the center and surround regions that are opponent to each other and sum linearly. The nonlinear mechanism was described as made up of smaller subunits, homogeneously spread over the receptive field surround, whose responses are half-wave rectified before summation. A complex cell receptive field model which is based on LGN Y-cell input to complex cells will predict responses of different characters to low and to high spatial frequency stimulation (as shown in the two responses displayed in each row of this column). Stimulation with low spatial frequencies may reflect mainly the linear mechanism which derives from the large linear center and surround areas in the LGN, while stimulation with high spatial frequencies may reflect mainly the nonlinear mechanism which derives from the small subunits which can elicit a response in any location across the receptive field. This distinction will be more apparent for contrast reversal stimulation than for drifting grating stimulation since in the latter the response shape is more sensitive to the neuron receptive field temporal properties and to the velocity of the drift of the grating. The distinction would be greater in the spatial phase dependence which is isolated by the counter-phase grating stimulation method. For low spatial frequencies we would expect to get a linear-summation spatial phase dependence whereas for high spatial frequencies we would expect to get a nonlinear-summation spatial phase dependence. The fourth column of Fig. 3 describes the expected response histograms and the spatial phase dependence for B cells which were described as a subgroup of complex cells (Henry, 1977; Kulikowski and Murray, 1985). The B cells have one subregion that consists of an ON and an OFF subunit which are spatially coincident (are at the same location) but whose responses are summed nonlinearly (e.g. follow•ng rectification). From contrast reversal grating stimulation we expect to get equal responses to the two halves of the stimulation temporal cycle and therefore double-peaked responses. The spatial phase dependence would then be a full-wave rectified sinusoid. (The response of each subunit is sinusoidal and the nonlinear sum of the responses of the two subunits is just the absolute value of the response of J.PN. 31/4--C

293

each.) This full-wave rectified response shape and spatial phase dependence would be expected also to a drifting sinusoidal grating in those cases (velocities) where the responses are not too sustained. The final two columns of Fig. 3 describe the expected response histograms and spatial phase dependence for complex cells according to our own recent model (Spitzer and Hochstein, 1985b; see Section 8 for more details). In this model, complex cells are described as having a few nonlinearlysumming spatially-disparate subunits each of which is derived from, or is similar to, an X-cell-like receptive field (for the sub-type called "mixed" cells, Fig. 3, fifth column) or a B-cell-like receptive field (for the sub-type called "intermediate" cells, Fig. 3, right column). Complex cells, as simple cells, are seen to derive their major excitatory input from LGN Xcells. On the other hand, complex cells, unlike simple cells, sum the effects of these inputs nonlinearly, probably with a half-wave rectification stage preceding the summation so only positive influences are summed. Figure 4 demonstrates how a single "mixed" receptive field model may elicit a response with one peak (ON or OFF) to stimulation of one spatial frequency and two peaks (an O N - O F F response) with stimulation of another spatial frequency. If the distance between the subunits is exactly a grating cycle, then the subunits will be stimulated identically, and the response will appear exactly as it might with a single linear subunit--namely with a single peak and a sinusoidal response dependence on phase. At other spatial frequencies, the distance between the subunits may correspond to half a grating cycle so one subunit will see an increase in illumination while the other sees an equal decrease in illumination. The nonlinear summation leads to an O N - O F F type of response (Fig. 4). In the same way, intermediate spatial frequencies will elicit O N - O F F responses which at some spatial phases have unequal peaks. The fifth column of Fig. 3 displays the three types of response histogram shape. It is this mixture of simple-like and classic-complex-like behaviour which gives this receptive field the name "mixed". For drifting grating stimulation we expect to see histograms of different shapes for gratings of different spatial frequencies. The fifth column (bottom three histograms) gives examples of the responses to different spatial frequencies in which the distance between the subunits is one, three-quarters and onehalf of a stimulation cycle, respectively. Complex cell receptive fields may contain two regions whose receptive field subunits, one ON and one OFF are overlapping. A stimulus over a single region, the region of overlap, would cause an O N - O F F response. Thus, complex cell O N - O F F responses depend on the nonlinear summation of the receptive field subunits. These subunits may be coextensive, overlapping (the predictions for this case are presented in the sixth column), leading to O N - O F F responses to stimulation with light spots or light bars which are turned ON and OFF, or they may be spatially disparate, leading to distinct ON and OFF responses to spots and bars and O N - O F F responses only to compound stimuli (for example, counter-phase gratings). The model predicts that the

294

H. 8PITZERand S. HOCHSTEIN

receptive field

counterphase grating stimulus response histogram FIG. 4. A single receptive field model will elicit a response with one peak (ON or OFF) with stimulation of one spatial frequency and with two peaks (ON--OFF) with stimulation of another spatial frequency. Model complex cells are described as having two nonlinearly-summing spatially-disparatesubunits each of which is similar to an X-cell-like receptive field (for mixed cells) or a B-cell-like receptive field (for intermediate cells). Complex cells sum the effects of these inputs nonlinearly, following half-wave rectification, so that only positive influences are summed. If the distance between the subunits equals a grating cycle, then both subunits will be stimulated identically, and the response will appear as if there were only one subunit (bottom row). If the distance between the subunits corresponds to half a grating cycle, one subunit will see an increase in illumination while the other sees an equal decrease in illumination. The nonlinear summation then results in an ON43FF type of response (top row). Intermediate spatial frequencies will elicit ON-OFF responses which have unequal peaks (middle row).

spatial phase dependence will be a function of the spatial frequency. The full repertoire of response forms and their spatial phase dependence will be presented in Section 7. The sixth column of Fig. 3 describes the expected response histograms and spatial dependence for complex cells of the "intermediate"; sub-type (Spitzer and Hochstein, 1985a, b). In this model, complex cells are described as having a few (perhaps two) nonlinearlysumming spatially-disparate subunits, each of which is of the O N - O F F type, or a B-cell-like receptive field. In this case the responses at the two halves of each histogram are identical because the responses for increasing and decreasing illuminations are assumed to be the same---each subunit has an O N - O F F type of response. These suggested complex cell receptive fields somewhat agree with the results and the model that Pollen and his colleagues suggested (Pollen et al., 1978; Pollen and Ronner, 1983). They found a modulated as well as an unmodulated level of activity present in the responses to drifting gratings, in about half of the cells studies. They suggested a "model complex cell" to explain their results, in which input of two pairs of simple cells are converted to a complex cell, one pair consists of ON-center cells with odd and even symmetry and the Other pair of OFF-center cells (Pollen and Ronner, 1983). They showed that such a model receptive field could predict periodically

spaced response peaks riding above an increased firing level. Recently, additional evidence was found that supports the "mixed" cell receptive field model. Mullikin et al. (1984b) and Palmer et aL (1985) report that cat area 17 periodic "simple" cell receptive fields are composed of four to six excitatory regions alternating in space with up to seven inhibitory regions (see Fig. 5). They emphasized that in these cells the width and placement of excitatory regions is such that the profile of light sensitivity as a function of distance perpendicular to the optimal orientation is obviously periodic. The spatial linearity of these receptive fields was not tested. Thus it is possible that "mixed" cells belong to the same group as these periodic simple cells, as well as the "silent periodic" receptive fields described by Kulikowski and Bishop (1982).

5. CELL TYPE CLASSIFICATION BASED ON THE RESPONSES TO SINUSOIDAL GRATING S T I M U L A T I O N Using the Fourier methods described above, a variety of criteria have been suggested to distinguish between the cell receptive field types found in visual cortex. We review these studies here and discuss a theoretical implication of their results in the following section.

COMPLEX-CELL RECEPTIVE FmLD MODELS

4.0d

A

Response

I 0

ON

- -

plane

-

295

-

B Ha3£

-

l

1.28 sec

FIG. 5. Response plane of a periodic cell. (A) The response of a cat cortical unit to a flashing light bar stimulus of size 0.1 x 1.7 degrees. (B) The hand plot of the receptive field of this neuron derived from the data of A. This receptive field exhibited a number of spatially discrete (nonoverlapping) excitatory and inhibitory regions. The response plane was tested in 40 positions and I0 ms bins. Average of 23 passes. (Reproduced from Mullikin et al., 1984b).

5.1. SPATIAL FREQUENCYTUNING Spatial frequency tuning of cortical neurons has received wide attention because of its relevance to the idea that the visual system contains "channels" tuned to different spatial frequencies (Robson, 1966; Campbell and Robson, 1968; Blakemore and Campbell, 1969; Graham, 1977; Maffei and Fiorentini, 1973). These channels were assumed to be the components resulting from the visual system's performing a Fourier analysis of the image. It has been shown by many groups (Campbell et al., 1969; Maffei and Fiorentini, 1973; Ikeda and Wright, 1975; Movshon et al., 1978c; De Valois et al., 1979, 1982) that the response sensitivity as a function of spatial frequency (the response transfer function) and the range of responsiveness (the response bandwidth) are similar for simple and complex cells (see also the review by Shapley and Lennie, 1985). Many physiologists refer to a "piece-wise Fourier analysis" or to local Fourier coefficients that are represented in the responses of single cells. The average bandwidth reported for neurons of both cats and monkeys was approximately 2 octaves (Movshon et al., 1978c; King-Smith and Kulikowski, 1975; Kulikowski and Bishop, 1982; Glezer et al., 1976, 1985). Many of the tuning curves that have been reported have a single peak. Glezer and his colleagues were first to find deviations from such curves in complex cell responses (Glezer et al., 1976). They found that a number of neurons had an additional maximum in the spatial frequency tuning curves. Figure 6 shows an example of a tuning curve with an additional peak (reproduced from Fig. 7 of Glezer et al., 1976). A group of complex cells was described by other laboratories called "periodic cells" which have similar properties (Pollen and Ronner, 1975; Pollen et al., 1978, 1984; Kulikowski and Bishop, 1982; Hochstein and Spitzer, 1984; Spitzer and Hochstein, 1985a). Glezer and his colleagues suggested a complex receptive field model that is consistent with both the line spread results and the spatial frequency characteristics of the complex cells they recorded from. This complex cell receptive field model assumed discrete excitatory regions with partially discrete inhibitory

i o o --

(a)

%

X

50

,,I

OI

I I I I,I 2 3 4.s 7

1 , , I .... I

0.2

O.S

t.O

c/deg

(h)

40

,,s

E

20

I

0.2

,,I,,,,I

0.5

I

I , ~1 .....

2

5

c/deg

FIG. 6. Examples of two spatial frequency transfer functions for two complex cell receptive fields that exhibit an additional maximum to moving rectangular gratings of different spatial frequencies. All the gratings were drifted across the receptive field at the velocity that gave the maximum response for that cell, namely, 7.65 dog/~¢ (graph(a)) and 3 deg/sec (graph(b)). (Reproduced from Glezcr et ai., 1976.)

296

H. SPITZERand S. HOCHSTEIN

areas. Pollen and Ronner (1975) arrived at a similar receptive field model, assuming a periodic component in the complex receptive fields. Glezer et al. concluded (by comparing the calculated curves with experimental data) that complex receptive fields have spatial frequency tuning characteristics which make them good candidates for being spatial frequency filters. (The spatial summation among the different zones in the receptive fields was not discussed or examined in that report.) 5.2. SPATIALPHASE--MODULATEDVS UNMODULATED RESPONSES

Maffei and Fiorentini (1973) found that complex cells generally responded to drifting gratings with an increase in their maintained discharge rate, while simple cells had a response that was modulated at the temporal rate of the stimulation. Until now, the expression "modulated response" when referring to both simple and complex cells has been used to mean a response at the temporal frequency of the test grating. There are four conventional methods used to measure the degree of this modulation. The first, a quantitative estimate of the amplitude of the modulated component of each average response histogram, f(x), was obtained from its autocorrelation (Pollen et al., 1978; Andrews and Pollen, 1979; Pollen and Ronner, 1982; Glezer et al., 1980). The second, the relative modulation, is the ratio of the amplitudes of the response component at the fundamental frequency, f0, and the zero frequency DC response component, fl (Movshon et al., 1978a, b; De Valois and De Valois, 1980; De Valois et al., 1982; Dean, 1981; Dean and Tolhurst, 1983; Jones and Palmer, 1987). The third, the extent to which neuronal responses to moving gratings are modulated, was obtained by using a mean variation score (Schiller et al., 1976b). This score, or index of modulation, is calculated for each histogram from the difference between the number of responses in each bin of the histogram and the mean overall response. The fourth, quantifying the spatial-summation linearity of the responses, is found by performing a second spatial phase Fourier transformation of the odd and even portions separately and determining the fractions of the response energy in the fundamental or second harmonic, respectively (Spitzer and Hochstein, 1985a, b; and Section 8.2, below, in this review). The first three measures were applied to responses that derived from moving grating stimulation. The suggested distinction between the two classes of cells by the "relative modulation" was derived from moving grating stimulation. Cells were classified as simple when the AC:DC ratio was greater than one, and as complex if this ratio was less than one (De Valois and De Valois, 1980; De Valois et al., 1982). This measure is suggested as a means by which one can quantify the extent to which a neuron's response is modulated by a repetitive stimulus. It seems to us, however, that there are two problems with the interpretation of the results deriving from the use of this measure as presented: the AC portion measured was the fundamental Fourier harmonic and not the sum of all the (non-DC) harmonics. Thus, one should be

careful when calling this the "modulated" part of the response. Furthermore, this AC:DC ratio cannot reflect the spatial phase dependence of the regions within the whole stimulation repertoire of the receptive fields (Spitzer and Hochstein, 1985a, b) since some complex cells exhibit a spatial phase dependence in their even portion which is the dominant portion of their responses. The third method provides a qualitative measure, as it does not give an estimation of how the output response is related to the input stimulus so that it will not be sensitive to the degree of spatial nonlinearity. Pollen and Ronner (1982) tested the responses of simple and complex cells to moving sine wave gratings and square wave gratings of different spatial frequencies. They showed in their different figures (see Fig. 7 for an example) that simple cells yielded modulated responses to the sine wave gratings, as expected. The unexpected results were to the square wave gratings where nonlinear phenomena were seen in simple cells. Stimulation at some spatial frequencies, and not necessarily at the high frequencies, caused responses which contained a dominant second harmonic. The second phenomenon that was seen is that the temporal Fourier transform profile was not independent of stimulation spatial frequency. The third phenomenon is that the dominant response component was not the fundamental harmonic, at all spatial frequencies. These three phenomena are deviations from the properties of a linear filter that would be expected in simple cell receptive fields. De Valois and his colleagues used Enroth-Cugell and Robson's (1966) X-Y distinction, and Hochstein and Shapley's (1976a, b) methodology and found a similar dichotomy between the simple and the complex classes (De Valois et al., 1982). They found, for example, that their complex cell data, in particular the dependence of the complex cell response on grating spatial phase, are consistent with one of the fundamental properties of the complex cell, namely, that it fires similarly to a stimulus regardless of stimulus location within the receptive field. They concluded that this similarity confirms Hubel and Wiesers receptive field models for the simple and the complex cells (see Fig. 3, column 2, classical complex cell). Almost all of the reports mentioned that complex cells elicited mainly O N - O F F responses both to light and dark bars and elicited two peaks to contrast reversal gratings. The appearance of a two-peaked response is reflected in the dominance of the second or even harmonic Fourier portion. Jones and Palmer (1987) report that cells defined as simple by the relative modulation test (AC: DC) also have little or no overlap in the areas of their receptive fields which respond to light and to dark spot stimuli ( O N - O F F areas). 5.3. RECEPTIVEFIELD DISCRETENESSAND THE FOURIER EVEN:ODD HARMONICSRATIO Dean and Tolhurst (1983) found a criterion based on two indices which allows classification differentiating between simple and complex cell receptive fields. The two indices are: (1) discreteness, which is a measure of the degree of separation Of the

COMPLEX-CELL RECEPTIVE FIELD MODELS 0095 c/Oe9.

t

I

0.18 c / d e g

Cyc~I

4

.

I 1 cycle *

11 CyCleI

297

close to one, the cell was regarded as exhibiting linear spatial summation. Even though this classification criterion successfully divides cortical cells into two groups, we have some hesitation about the meaning of the second index. It is easy to think of receptive field models which incorporate nonlinear spatial summation but still predict spatial summation indices near unity--for example, a receptive field that contains two discrete regions whose responses are rectified before summation. Recently, another method for differentiating simple and complex cell receptive fields was suggested based on a single index (Spitzer and Hochstein, 1986, 1987). Complex cells were defined as those cells in which the responses to contrast reversal grating stimuli of various stimulus parameters usually contain two peaks, reflected in the dominance of the even Fourier portion. Simple cells, on the other hand, were defined as always having responses with one peak, reflected in the dominance of the odd Fourier portion. Thus, an even:odd Fourier portion ratio larger than unity is a defining characteristic of complex cells, differentiating them from simple cells. This even:odd portions ratio measure probably tests the same receptive field properties as tested by the two indices measured by Dean and Tolhurst (1983). Figure 8 (center and right) displays receptive field spatial frequency dependencies of the odd and even portions of the responses of simple and complex cells. The odd portion predominates in simple cells and the even portion in complex cells throughout the spatial frequency range where the neuron is sensitive.

6. PARALLEL OR HIERARCHICAL PROCESSING?

)

a

1 cycle

FIG. 7. Responses of an "off-center" simple ceil to drifting

square-wave gratings of various spatial frequencies. All the gratings were drifted at the same velocity. The horizontal calibrations represent the extent of one full cycle of the test stimulus square-wave grating. Note the different number of peaks that are in one cycle for the different spatial frequencies. (Reproduced from Pollen and Ronner, 1982.)

excitatory and the inhibitory regions, and (2) spatial summation ratio, which aims at measuring the degree of spatial summation within each region. The discreteness measure was calculated from the responses to bright and dark bars whereas the spatial summation measure was calculated from the neuron's responses to bars and to moving sinusoidal gratings. They estimated the equivalent optimum stimulus bar width as half of the period of the moving sinusoidal grating which gave a maximum neuron response. This was compared to the width of the bar stimulus which gave the maximal response (Movshon et al., 1978a). If a neuron yielded a ratio of these measures

There has been much debate in the literature as to whether information processing in the visual cortex is parallel or hierarchical. Hubel and Wiesel originally suggested a hierarchical model where complex cells derive their input from simple cells. The power of this model lies in that it accounts for the complex cell's preference for narrow bars (or high spatial frequency gratings) despite its having a broad receptive field and a response relative independence of spatial location of a stimulus bar within this receptive field. The hierarchical model is problematic, however. There is evidence of monosynaptic input to both some simple and some complex cells (Henry et al., 1979; Singer et al., 1975; Stone and Dreher, 1973; Stone, 1983; Toyama et al., 1973, 1981a, b). Furthermore, complex cells respond to some stimuli to which simple cells do not respond. For example, Hammond and his colleagues (Hammond and Mackay, 1976, 1977; Hammond and Smith, 1982) showed that only complex cells are responsive to motion of visual noise. A differential sensitivity like this could not derive from a hierarchical mechanism. Several groups asked how the distinction between the X and Y pathways relates to the distinction between simple and complex cells. Stone and Dreher (1973) suggested that LGN X-cells drive cortical simple cells whereas LGN Y-cells drive complex cells. They based their suggestion on the observation that some complex cells had short latencies to electrical

298

H. SPITZERand S. HOCHSTEIN 30

LGN - Y

20

•

.-°'"'('<~I

",!

25 15 IO

,•

I

Simple

,#~o/'Q', Complex

"I 5

IsO

30 /

•

75

.-"

tt i

"°I

25 °°"

2O

~ l

, "

Ioo 75

fie ,0 20

eo

3O

4o

2O I0

,,,o,\ ~

'~

~ I0

t

i

i i

o~,s o~ d, o~ o~ o= ,., ,.,

o~=s ~

*:, o'~ o'., ~

,~ ~2

~, 0~ 03 o12 o~, o'., /.6 ;.2

Spoliol Frequency (c/deg) FIG. 8. Spatial frequency transfer functions for LGN Y-cells (left graph), simple ceils (center graph) and complex cells (fight graph). Response amplitudes are plotted as a function of spatial frequency for the odd (flied circles) and the even (open circles) Fourier portions of the responses of a few cells of each cell type. Note that, compared to the even portion, the odd portion is larger in simple cells and smaller in complex cells, while in LGN Y-cells the odd portion is larger at low spatial frequencies and smaller at high spatial frequencies. (Left graph reproduced from So and Shapley, 1981;center and right from Spitzer and Hochstein, 1985a.)

stimulation and should therefore be monosynaptic to LGN units. Some cells respond with multiple inputs of varying latency to a single electrical pulse (Singer et aL, 1975; Bullier and Henry, 1979a, b; see also the review by Lennie, 1980). Other studies emphasize the similarities of response cl~aracteristics of LGN X-cells and cortical simple cells on the one hand, and of LGN Y-cells and cortical complex cells on the other. Movshon et al. (1978a, b, c), De Valois and De Valois (1980) and De Valois et al. (1982) found similar nonlinear spatial summation properties of cortical complex cells and LGN Y-cells. From studies of response planes (Palmer and Davis, 1981; Citron et al., 1982), it was found that LGN Y-cells and cortical complex cells have spatially homogeneous receptive fields and contain large spatially overlapping excitatory and inhibitory domains, whereas the receptive fields of LGN X-cells and simple cells are spatially heterogeneous (contain distinct ON and OFF areas). On the other hand, simple receptive fields were described as being X-like or Y-like, based on the spatiotemporal organization of genieulate X and Y receptive fields (Mullikin et al., 1984a). These authors su~ested that the X and Y systems, which originate in the retina, are maintained in parallel at the level of simple cells in striate cortex. Retinal ganglion and geniculate X-cells are indeed quite linear in both their temporal and spatial phase

dependence, and simple cells are also spatially linear, whereas Y-cells and complex cells are nonlinear in both these aspects (Hochstein and Shapley, 1976a, b; So and Shapley, 1979, 1981; Movshon et al., 1978a, b, c; De Valois and De Valois, 1980; De Valois et al., 1982; Spitzer and Hochstein, 1985a, b). However, similarities can only suggest but cannot prove a claim such as the parallel processing hypothesis (Spitzer and Hochstein, 1985a). In particular, nonlinearities present in cortical receptive fields could derive from nonlinearities present in the LGN input to these cortical cells, or alternatively could derive de novo from cortical processing. Toyama et al. (1981a, b) and Tanaka et al. (1983) found no excitatory input from simple to complex cells. On the other hand, they found pairs of cells from the simple and the complex groups that receive overlapping inputs from X and Y LGN cells. They concluded that the differentiation of response types at the cortex depends on intrinsic effects of intracortical connections, rather than on extrinsic inputs of LGN excitation. These findings seem to refute both the parallel (X to simple; Y to complex) and the hierarchical (LGN to simple to complex) models. The spatial frequency dependences of the odd and even Fourier portions of the responses of cortical simple and complex cells are displayed in Fig. 8 (reproduced from Spitzer and Hochstein, 1985a). These spatial frequency dependences are compared

299

COMPLEX-CELL RECEPTIVE FIELD MODELS

0.38 cldeg

1.0 c/deg

O"

22.5"

45"

675' Relative phase 90"

1125'

135'

1575' 05

1

~ 0

05

~-.,--~,~te~ 1

Time (sec)

FIG. 9. The responses of a complex cell to stationary sinusoidal gratings of two spatial frequencies, 0.38 cpd and 1.0 epd, square-wave modulated in time at 1 Hz and positioned at the indicated spatial phase. Note that the response amplitude and histogram shape (relative amplitude of the two peaks) both vary as a function of the spatial phase, especially for the 0.38 cpd grating. The gratings had a contrast of 0.5. (Reproduced from Movshon et al., 1978b.)

with the spatial frequency dependence in LGN cells (Fig. 8, left), as reproduced from So and Shapley (1981). They found that LGN Y-cells yield responses in which the first harmonic dominates at low spatial frequencies, and the second harmonic at high spatial frequencies; the second harmonic has a three times higher high-spatial frequency cut-off than the fundamental Fourier component. Furthermore, at high spatial frequencies the second harmonic, and thus the entire response, has no spatial phase dependence. These two properties have been found to be consistent characteristics for all retinal ganglion and LGN Y-cells reported and may derive from the fact that the Y-cell's nonlinear subunits are a third as broad as the linear center mechanism (Hochstein and Shapley, 1976a, b; So and Shapley, 1981). Thus, the 3:1 ratio and the lack of spatial phase dependence of the second harmonic may be referred to as the Y-cell signature. Both simple and complex cells lack this distinctive Y-cell signature. For simple cells the odd portion is as large as, or larger than the even portion, throughout the spatial frequency range

of cell response. The presence of the small but considerable even portion which is nearly absent in LGN X-cells may derive from cortical rectification. On the other hand, for complex cells, the even portion is generally larger than the odd portion. There are complex cells that show a periodicity of the even:odd portions ratio and most complex cells showed spatial phase dependences in the even portion for some spatial frequencies. The response odd portion, however, was present and smaller than the even portion throughout the spatial frequency range of cell response. The simple conclusion that may be drawn is that the cortical nonlinearity does not derive from LGN Y-cell nonlinearity. This in itself suggests that the parallel channels model cannot apply in its Xsimple, Y-complex form. It does not however imply that the system is hierarchical, since all cortical neurons could receive direct functional LGN input. The same conclusions were found recently by Ferster (1987) using an entirely different experimental technique. He recorded intracellulafly from cortical simple and complex cells and found only X-mediated

300

H. SPITZERand S. HOCHSTEIN

synaptic input throughout area 17 and Y-input only in area 15. Thus, functional Y-cell input to area 17 appears to be questionable. 7. COMPLEX CELL RESPONSE-FORM DEPENDENCE ON STIMULUS PARAMETERS

7.1. CAT

7. I. 1. Response-form dependence on spatial phase Cat cortical cell receptive fields exhibit a variety of response-form dependences on stimulus parameters. We describe these in this section, together with recent results from monkey cortex (Spitzer et al., 1987) in light of the results of other research groups and in terms of a complex cell model, outlined in the following section. Movshon and his colleagues (Movshon et al., 1978b, c) emphasize that complex cell responses to contrast reversal gratings (modulated with a temporal sinusoid or square-wave) were usually more or less purely O N - O F F . Figure 9, which is reproduced from their paper, demonstrates that at a low spatial frequency the amplitude of the response depended on spatial phase and the amplitudes of the two peaks of the same response are not always identical. The figure (A)

also shows that the response histogram shape (relative amplitude of the two peaks) varies as a function of spatial phase (Fig. 9, left) much more strongly than at a higher spatial frequency where response histogram shape is more uniformly O N - O F F (Fig. 9, right). Figure 10A, B displays histogram responses of a complex neuron to sinusoidal standing counter-phase gratings of 0.3 cpd and &4 cpd as a function of spatial phase. For 0.4 cpd gratings, at some spatial phases (Fig. 10B), the responses consist mainly of two equal peaks and at others mainly of a single peak, while at still other phases the responses have a mixed form with two peaks of unequal amplitude. The degree to which the response is mixed was calculated by the ratio of the amplitudes of the even and the odd portions. (For the responses presented in Fig. 10B this ratio varies from 1.0:l to 3.4:1; Spitzer and Hochstein, 1985a.) Figure 11A, B shows the spatial phase dependence of the odd and the even portions of the responses of Fig. 10A, B. The odd portion is nicely fit by the sinusoidal function that is drawn as a smooth curve (Fig. IIB); the degree of similarity to this harmonic is 87%. On the other hand the similarity of the even portion to the second harmonic is small, 14.6%.

SF =O.3cpd hist odd even

f~

2nd

(B)

SF=O.4cpd hist odd

even 2rid

:ZZz::=::-:: I

I

I

I

1

0o

90 °

180 °

270 °

360 °

¢ ( spatial

phase)

FIG 10. Complex cell time-course response histograms are demonstrated as a function of counterphase stimulation spatial phase. The response histograms and their reconstituted portions are presented. Each row of histograms is for thirteen spatial phases in steps of 30 degrees, as indicated. The reconstituted portions of the original histograms, are based on the odd or the even components, or only the fundamental or second harmonic component, respectively. The tic harmonic was added to each reconstitution. In the upper set of histograms, response histograms from a single neuron are presented in response to contrast reversal grating stimulation of spatial frequency 0.3 cpd (temporal frequency 2 Hz). Maximal grating contrast was 0.2. The response histograms are quite homogeneous in their shape, mainly double-peaked, so that the even portion is predominate. Although the odd portion is small for these responses, it varies sinusoidally as a function of the spatial phase. At this spatial frequency the amplitudes of the odd and the even portions maintain a constant ratio. The even portion varies as a full-wave rectified sinusoid with spatial phase. In the lower set of histograms responses are shown for stimulation with a 0.4 clad sinusoidal grating. The responses are not single peaked (as in simple cell responses) and do not have two equal peaks (as in nearly the case in the upper half of this figure), but rather display two unequal peaks, a mixed-response time course. The relative amplitudes of these peaks vary with spatial phase, so that some responses have nearly equal peaks (e.g. at 120-150 and 330 degrees) and others have predominantly one peak (e.g. 30--60and 210 degrees). The shape of the response is analyzed by separating its odd and even Fourier portions; their amplitudes are plotted as a function of spatial phase in Fig. 11.

COMPLEX-CELL

RECEPTIVE

SF-CX3c~ IO

!

i

I

|

odd

i

I

!

|

I

7.2. MONKEY Recently, Spitzer and colleagues studied the responses of striate cortex neurons to a set of orthogonal, two-dimensional, black and white (Walsh) patterns in an awake, fixating monkey (Spitzer et al., 1984, 1987). As part of the analysis of the results of this study, they abutted the ON responses to each stimulus pattern to the responses generated by its contrast reversal stimulus mate, even though they had not been presented sequentially to the animal. In this way, they were able to compare these results to the previously analyzed responses recorded from anesthetized cats (Movshon et al., 1978a, b; Pollen and Ronner, 1982; Glezer et al., 1982; Spitzer and Hochstein, 1985a, b). An analogous procedure was applied also for the offset of the patterns. For most of the monkey complex cells the shape of the response histograms depended on the stimulus parameters. The abutted histograms exhibited a variety of shapes in the reconstructed histograms: two equal peaks, two unequal peaks, and a single peak, to the different Walsh patterns. The offset responses (the responses to turning off the stimulus) showed

'

'

/

!

'

Ji°l

-0.5_

!

I

i

i

I

!

!

• •

,

,

,

'

i

l

l

t

l

l

J

l

,

l

i

,

,

l

•

•

i ' '

,

J

i

l

,

,

i

•

,

,

t l l l U l i , l u u

•

.5

even

i

.

sF.~4~ l l l

I~[

I

1 L L.

•e

(B)

I

301

MODELS

independent of spatial frequency (2.5:1 for the 0.3cI~ and 2.7:1 for the 0.6cpd), but the spatial phase dependence is different at the two spatial frequencies (Fig. 12B).

7.1.2. Spatial frequency dependence The response time course not only often varies as a function of spatial phase but also can vary as a function of spatial frequency. Figure 10A shows response histograms from the same neuron whose responses are displayed in Fig. 10B to grating stimulation of another spatial frequency, 0.3 clad. The responses to this spatial frequency are more homogeneous, mainly double-peaked. In Fig. 11A, the odd and even portions of the responses of Fig. 10A are displayed as a function of spatial phase. The even harmonics portion is much larger at this spatial frequency, indicating a change in the form of the response histogram. In addition to the change of the shape of the histogram there is also a different spatial phase dependence of the even portion. At this spatial frequency, the even portion has a larger dependence on the spatial phase, and the dependence resembles a full-wave rectified sinusoid. Most complex cells which showed these properties (response histogram shape dependence on spatial phase and spatial frequency), also showed a variation of the spatial phase dependence as a function of spatial frequency. Such variation of the spatial phase dependence was also found for many cells that did not show strong variations in the shape of the histogram responses. Figure 12A, B illustrates this phenomenon. The shapes of the response histograms are uniform (Fig. 12A), the even:odd ratio is quite

(A)

FIELD

• 0 0

•

• • •

lo - - ~ 5 ~ i l l l t l | l l l l e O" 90" 180" 270"

i 360"

(~'

l

i I 90"

|

| i 180"

|

l i 270"

i 560"

Spollol Phase ( 4 ) FIG. 1I. Spatial phase dependence of the odd and even portions of the responses to spatial frequency gratings of different spatial frequencies. Normalized odd and even portion amplitudes of the responses of Fig. 10 are plotted as a function of stimulation-grating spatial phase. At 0.4 cpd (lower graph) the odd portion dependence has a sinusoidal form (87%), whereas the even portion is more uniform and has a complex dependence on spatial phase. At 0.3 cpd (upper graph) the odd portion dependence is also sinusoidal, but here the even portion has a full-wave rectified sinusoidal dependence on spatial phase.

302

H, SPITZERand S. HOCHSTEIN (A) SF=O.3cpd

LL L LLLLLLLLLLLL L

hist odd

even

m..,mm.,m,

mm

. ~ , . a l I m llm m m l l m , . ~ m m

SF=O.6cpd hist

U.LLLLu, ,- LL u.LLLL

even

2n_ I

I

Iu

I 0•

M,U.M.tkU, M,M.M,I I 90 •

I 180 •

I... 270 °

(spatial (B)

(37) 1.0

~E

~_<

i

I

phase )

odd

SF =0.3clxI i

I

v

I 360"

even i-"

I

T

1

I

i

l

1

I

I

i

I

i

0.5

u)

.>

~

I

I

'

I

0 0

-0.5

,

i

I

i

I

[

I

i

I

I

I

I

i

i

i

i

i

i

i

I

l

l

SF = 0 . 6 c p d T

,

l

|

l

|

I

I

1

1

1

1

1

1.0 -!

•

t l

v~

I

I

;

0

-0.5

t

0°

I

I

90"

I

I

I,,

le0 °

I

I

I

270"

I

I

I

360*

0°

Spotiol Phose

I

I

90"

I

I

I, 180"

,

i i 270"

I 360"

(~ )

FIG. 12. Intermediate cell responses at two spatial frequencies. The response histograms (A) and spatial phase dependences of the odd and even Fourier portions (B) are shown for stimulation at two spatial frequencies. At 0.3 cpd (upper histograms and graphs) the odd portion has a sinusoidal form (72%) whereas the even portion amplitude is more uniform. At 0.6 ~ (lower histograms and graphs) the odd portion has a sinusoidal form (77%) while the even portion also has a strong dependence on spatial phase. (Reproduced from Spitzer and Hochstein, 1985a.)

COMPLEX-CELLRECEPTIVEFIELD MODELS

again the same phenomenon, a large variety in the shape of the histograms. A significant similarity was found between the ON response set and the OFF response set from the same cell, and the even:odd ratios were close for these two groups. When the offset responses were large (more than 1/7th the ON responses), then the ON and the OFF (offset) response histogram shapes were very similar to each other. This similarity of the ON and the OFF responses (offset responses) led the authors to suggest this phenomenon as a possible source for the negative after-image effect. The expression " O F F " response is commonly used in the literature with three different alternative meanings that might originate from three different possible mechanisms and types of effects (Spitzer et al., 1987): (1) A response that is elicited after turning off a light stimulus. In other words, a visual response to a change in contrast (derivative), taking the sign into account. (2) Suppression of an ongoing activity where the " O F F " response is a reflection of the overshoot of the subarea that was released from inhibition. (Such a mechanism can be the factor responsible for receptive field linear spatial summation.) (3) A response that is elicited by a light stimulus whose illumination is below the average illumination. By using contrast reversal grating stimulation it is possible to test the spatial summation properties of the receptive field and thus to differentiate between the second and the third mechanisms. However, it is not possible to differentiate between the first and third mechanisms with contrast reversal gratings. This report on the responses to Walsh pattern stimuli enabled one to confront and test separately the responses to a below-average light illumination stimulus and the responses that were elicited by turning off the visual stimulus. Thus, the authors hoped that this study would give a quantitative answer to the confusion in the literature between the OFF responses at stimulus offset and the responses to contrast reversal stimuli, and thus to differentiate between the first and third mechanism.

303

where R,(t) is the response of subunit s at time t, f a x ) is the receptive-field weighting function of the subunit, and S(x, t) is the stimulus intensity at position x and time t. In the model a receptive field is proposed with nonlinear spatial summation between two identical subunit rows. The nonlinearity is introduced by a half-wave rectification stage before the summation. This rectification stage is expressed by taking the average of the response function R,(t ) with its absolute value. Call the positions of the two subunits x = _+2 and denote their responses by the subscripts + 2 and - 2 , respectively. The neuron response is then, R+a(t) + ]R+a(t)l + R_a(t) + IR_~(t)] R(t) = 2 The stimuli on the two subunits may be related in one of three possible ways: (I) They may be identical, (2) they may be of equal intensity but opposite sign, or (3) they may be different. In the first case, the response histogram will contain a single peak; in the second case, the histogram response will have two equal peaks; and in the third case the response will contain two unequal peaks, a response which we will say has the mixed shape. These three cases occur when the distance between the subunits is exactly a grating cycle, so that both subunits are stimulated identically; when the distance between the subunits corresponds to half a grating cycle, so that only a single subunit is effective at any one time (the rectification only allows positively signed responses to enter the summation stage); and finally, when the distance between the subunits is neither an integral multiple of the grating period nor just a half wave more or less than an integral multiple, so that the response will contain two peaks of unequal amplitudes for some spatial phases (See Fig. 4, middle row).

8.2. MODEL R£SPONSE DEPENDENCEON SPATIAL PHASE AND FREQUENCY

8. A COMPLEX CELL RECEPTIVE FIELD MODEL The complex cell characteristics that were described above, in particular the response temporal form dependence on stimulus parameters, may ap.pear puzzling. They may be more readily understood in terms of a model (Spitzer and Hochstein, 1985b). 8.1. THE TWO SUBUNIT MODEL The complex cell receptive field in its simplest form is assumed to contain two (rows of) subunits. Each subunit is composed of X-cell-like, DOG-shaped (difference of two gaussian shaped), input regions. Each acts as a linear filter and its response may be described by the inner product (point-by-point multiplication) of the stimulation function, S(x, t), and the sensitivity function, f,(x), of the receptive field.

R,(t ) =

(x) S(x, t ) dx

Since a difference was found between the spatial phase dependence of the Fourier components in linear X- and in nonlinear Y-retinal ganglion cells (Enroth-Cugell and Robson, 1966; Hochstein and Shapley, 1976a, b) it is of interest to analyze separately the linear and nonlinear parts of the complex cell response (the odd and the even portions) as a function of the spatial phase. Figure 13A, B, C presents the response histogram forms expected from the model and their odd and even portions, respectively. Each row of Fig. 13A contains histograms for a fixed spatial frequency co, as a function of spatial phase 4). From this summary figure it is easy to see that as the spatial frequency is varied, the response time course changes from a single-peaked response to a response with two peaks of equal amplitude. A Fourier transformation is performed on the histograms and then an inverse Fourier transformation of the odd and even components, separately. Figure 13B and C displays the reconstituted odd and even portions of the histograms of Fig. 13A.

304

H. SPITZERand S. HOCHSTEIN A: Model Response

o.oo - - L L L - _ - A A . t o.o8 -A-LU.a.-. LIJLA= 0.25

~l~llAk.llA-~..Ij_IdlJJ,

0.33 I J J ~ L , d L , ~ , ~ t ~ L ~ , , A ~ d M L 0.42 ~ d i S A , , ~--- ~--- --~- ~-------~---~ --~" - ~ "~ M

0.50 J ~ M h ~ - - . ~ w ~ M B: Odd Portion I

"<

3

o oo - - , , -dk,l b ¢ , , , - - - - ~ d l ~ --,, ¢,1b¢,,, ---,.~J~J--

---

--~tk, l,~--,Jdtdl,.t

J--

0.08

o. I r 0.25 0.33 0.42

=l L,~II~8.,L,1..Adl Jl~l,,,~ -l~L"a"A-'aL.'Im'dl"~',-M',~ll'-~"amn l l m ab"o~"- ~ ' ~ l l l l l ' J ~ - ~ ' " ~ =~l~l

0,50 l l l - -

_

O

~ l l l l - -

i

l

l

C: Even Portion

~M~ ~ M ~ a M ~ a ~ M ~ I M ~ a A

A'= ~ M ~

~M~~

0 . 3 3 ~u~ a U ~ , M ~ M ~ ~u~~ M ~ ~ M ~ dM~ 0 42 ~ u ~ u ~ " ~ u M M t ~ M ~ " ~ u ~ d M ~ 0.50 ~u~ ~u~ ~ u ~ ~u~ ~u~~ M ~ M ~

0°

90*

180"

Isin(~b + 2nco2 )1 + Isin(~b - 2nco).)l One variation of the model is receptive fields with two subunits each of which is already made up of a B-cell type area, responding in O N - O F F fashion to local stimuli (see below, Section 9). The functional dependence of I-cell responses on spatial phase, is very similar of that of M-cell responses. Since each subunit consists of ON and O F F regions with equal weighting functions the response histogram will be always with two equal peaks, containing only the even harmonics. The spatial phase dependence for the even portion of the Model (M) cell was described above as the sum of two unrelated full-wave rectified sinusoids, and since this dependence is only a function of the spatial distance, 22, between the subunits, the same spatial phase dependence will result also for the I-cell. 8.3. MODEL SPATIAL FREQUENCYTUNING CURVE

O. Ill/' ~ , ~ a ~ & A , t l ~ = - " ~ 8 ~ l ~ , M , ~ , 0.25

tude dependence is a single full-wave rectified sinusoid. In the general mixed case, the amplitude of the even harmonics has a more complex dependence on phase, namely the sum of two full-wave rectified sinusoids of unrelated phases,

270*

~(Spatial phase) Fro. 13. Response histogram forms expected from a two

subunit receptive field model. (A) Each row shows histograms for a fixed spatial frequency, co, as a function of spatial phase, varying from 0 to 360 degrees in steps of 30 degrees. Each column is for a fixed spatial phase, and varying spatial frequency, increasing from k/2d to (k + 1/2)/2d, where k is a positive nonzero integer and 2d is the distance between the subunits. (B) Reconsituted odd portions of response histograms of A. These are inverse Fourier transform of the odd components (and DC components) of the histograms. Note the sinusoidal dependence on spatial phase. (C) Reconstituted even portions of the response histograms of A, representing the nonlinear aspect of the histograms. Note the full-wave rectified sinusoidal dependence on spatial phase of the responses in the top and bottom rows. Other rows have a dependence that is the sum of two full-wave rectified sinusoids (see equations in the text; reproduced from Spitzer and Hochstein, 1985b.) (The DC, 0th order, component was added to both inverse transformations.) The model predicts that the spatial phase dependence of the amplitude of the odd portion will be a sinusoidal function of the spatial phase at all spatial frequencies (Fig. 13B). On the other hand, the amplitude of the even portion will correspond to a simple function of spatial phase only at co = n / 2 2 and co = ( n +0.5)/22, (where n is any integer) and is never zero except at these spatial frequencies (see equation 23 in Spitzer and Hochstein, 1985b). At these particular frequencies, the response histograms have a fixed shape across the different spatial phases, and thus only an amplitude dependence on spatial phase. The function that describes this ampli-

From the equations of the model (see Spitzer and Hochstein, 1985b, equations 5-8), one may derive the parameters of the tuning curves of the odd and the even Fourier harmonic portions. It may be shown that the odd portion tuning curve lies below the even portion tuning curve which acts as its envelope. This envelope function of the tuning curve is fixed by the shape of a single subunit (in the case that all the subunits are identical). It is the Fourier cosine transform component (for a symmetric receptive field subunit) of the receptive field of a single subunit that determines this tuning curve envelope. The description of the odd and the even portions as a function of the spatial frequency is summarized in Fig. 14. The value of the even harmonics portion as a function of spatial frequency gives a measure of the amplitude of the envelope of the odd harmonics portion tuning curve. The change of the odd portion (the dashed curve in Fig. 14) as a function of the spatial frequency reflects the shape of the response histogram. When the odd portion reaches zero, the shape of the histogram response will be with two equal peaks, according to the receptive field model. When the odd portion reaches the roof of the envelope, the response histogram will be with a single peak. Between these two points, the mixed shape will appear.

9. VARIOUS CELL TYPES PRODUCED BY VARIATIONS

OF

RECEPTIVE

FIELD

STRUCTURE, USING A SINGLE S U M M A T I O N RULE The cortical cell receptive field model outlined above is meant as a working hypothesis for analysis of the wide variety of cell types and sub-types found in the primary visual cortex. The basic premise is that cortical receptive fields are built of a small number of X-cell-like inputs summed either linearly or nonlinearly. Using different spatial organizations of these

COMPLEX-CELL RECEPTIVE FIELD MODELS

~b--~E~,,~k.

1.0

0 odd portion norm

/./

._o

,/

',,

it I

.

~o.5

• 0

c GI~.-

,

,,!,,,

; tl

~ o 0.05

--

0.1

0.2

t

0.4

0.8

ou(spatial frequency), cpd

FIG. 14. The spatial frequency dependence of the odd and the even components of the responses expected from a two subunit receptive field model, as demonstrated in Fig. 13. At spatial fr~luencies for which the distance between the subunits is just equal to an integral number of grating cycles plus one-half cycle, the response has two peaks and the odd portion nears zero. At spatial frequencies for which this distance is just an integral number of grating cycles, the response has a single peak and the odd portion reaches its maximal value compared to the even portion at that spatial frequency. Between these spatial frequencies the response odd portion has an intermediate amplitude. input units one may construct receptive fields which match many of those which have been studied, and whose properties have been reviewed here. Some of these are demonstrated in Fig. 15. One way in which the receptive fields of these cells may be formed is as follows. The simple cell receptive field may be regarded as a specific case of the model receptive field in which one of the subunits has degenerated, leaving only a single, half-wave rectified (sub)unit. In Fig. 15 we show a simple cell with one row of LGN X-like DOG inputs. However, cortical units should be referred to as simple no matter what their receptive field profile as long as the summation between the inputs is linear. Henry's type-B cell (Bishop et al., 1977) may be seen as having two subunits, one ON and the other OFF, as in the model. In this case, however, the distance between the two subunits, 2A approaches zero. For this cell type, the condition R + ~ ( t ) = R _ ~ ( t ) , always holds true, i.e. the responses of the two subunits are always equal but of opposite sign. The response in this case will be double peaked with peaks of equal amplitudes to all the different stimulus parameters. In addition, the spatial phase dependence will be a fixed function, always a full-wave rectified sinusoid, regardless of spatial frequency. When the two subunits are one ON and one OFF but are spatially disparate (ideally at a distance equal to the 1/e-width of the subunits), the neuron responds to edges or zero-crossings (Marr, 1982; Hochstein and Spitzer, 1984). Thus, the detection of zero-crossings no longer depends on the null response of retinal ganglion or L G N cells, but rather on the peak firing of small-field cortical complex cells. (These cells are called M - in Fig. 15.) What happens as subunits are added to the model receptive field? If they appear at regular intervals of

305

2~ they do not change the model's predictions (as summarized in Fig. 13A, B, C) except that a weighting function must be introduced before summing the responses. If the distances are irregular, the response shape tends towards the mixed form. As the number of subunits increases, with irregular spacing, we approach the double-peaked response with no amplitude dependence on spatial phase, as occurs in the classical complex response (called C in Fig. 15). Many complex cells showed a fixed temporal shape of the responses even though their response amplitudes were spatial phase dependent for spatial frequency grating stimulation; see Fig. 12A, B for demonstrations of this response type. The model may be used to describe this phenomenon as well: assume a receptive field consisting of two identical subunits, but in this case each is a B-type receptive field consisting of two overlapping ON and OFF (DOG) subunits (I in Fig. 15). I0. CONCLUSION The original classifications of simple and complex cells as defined by Hubel and Wiesel included the notion of (simple) predictability of the response to compound stimuli from the responses to spot stimuli for simple cells and the absence of such predictability for complex cells. This may be seen as equivalent to the notion of the linearity of spatial summation within the receptive field for simple cells and a nonlinearity in the summation characteristics of complex cells, though Hubel and Wiesel did not actually use the terms linear and nonlinear. These concepts were only emphasized in mammalian vision research starting with the work on retinal ganglion cells by Enroth-Cugell and Robson (1966), later expanded by Hochstein and Shapley (1976a, b), and extended to LGN cells and cortical cells (for example by De Valois et al., 1979; Glezer et al., 1980; Movshon et al., 1978a, b; Shapley and Hochstein, 1975; So and Shapley, 1981; and Spitzer and Hochstein, 1985a). Complex cell receptive fields have been analyzed in many ways, and numerous models have been suggested to understand them. These analytic methods have included the number and type of sub-regions, the degree of overlap of these sub-regions, the receptive field spatial summation, the predictability of the responses to gratings from the responses to bar stimuli, the degree of modulation of the responses to gratings, and the similarity of the receptive fields to LGN Y-cell receptive fields. We believe that major features of complex cell receptive fields may now be summarized in the following way (Spitzer and Hochstein, 1985b): Complex cells derive their major excitatory input from LGN X-cells but sum the effects of these inputs nonlinearly, probably including a half-wave rectification stage before the summation, so that only net positive influences of the receptive field subunits are summed. Thus, stimuli which affect first one subunit (during the first phase of the stimulus) and then another (at the second phase of the stimulus) produce frequency-doubled responses in complex cells. (In simple cells these stimuli would induce antagonistic influences which would cancel.) This two-peaked response may occur in two ways: The subunits may be coextensive, overlapping, lead-

306

H. SPITZER and S. HOCHSTEIN

DOG

o

V

::

M

+lI

' •l •

4

^

• •

•

~

¢

o

•

i

•

e

•

+

•

o •

o

t A 6 o O ~ x 4 4~ ~a ax me

$

I

•

•

~

4 o

6

• •

FIG. 15. Variations in the spatial structure of complex cell receptive fields based on the proposed model. Mixed cells have two receptive field subunits of the same sign (ON-ON or OFF--OFF) or of opposite sign (ON-OFF). The distance between the subunits may vary from cell to cell. In particular if the distance is zero and the subunits are of opposite sign, we have a B cell with ON-OFF responses to modulated stimuli but a linear dependence on spatial phase. Intermediate ceils have two B-cell-like subunits so that their responses are ON-OFF but their amplitudes vary with spatial phase and frequency as mixed cells. Finally, classic complex ceils have many scattered subunits so that responses to counter-phase gratings are always ON-OFF and have no spatial phase dependence.

ing to O N - O F F responses to spots of light or light bars which are turned O N and O F F (as in B cells or I cells), or they may be spatially disparate, leading to distinct O N or O F F responses to spots and doublepeaked responses only to ~lome c o m p o u n d stimuli (as in M cells). According to this view all complex cells share the characteristic mechanism of nonlinear spatial summation between subunits, distinguishing them from simple cell receptive fields. However, the number of these subunits and their spatial arrangement may vary from neuron to neuron giving rise to the multifarious nature of the complex cell group.

Acknowledgements--We thank Moshe Gur, Peter Hillman, Dov Sagi, and especially Robert Shapley, for their useful comments and sug$cstions. H.S. thanks Jeffrey Prawer for generous help in English language corrections of early drafts of the manuscript. We thank the authors and journals whose figures are reproduced here with their permission. This work was supported by the Lady Davis foundation and grants from the U,S.-Israel Binational Science Foundation (BSF).

REFERENCES ALBmECHT,D. G., DE VALOIS,R. L. and TrtOItELL,L. (1980) Are bars or gratings the optimal stimuli? Science 2tFI, 88-90.

COMPLEX-CELLRECEPTIVEFIELDMODELS ANDREWS, B. W. and POLLEN,D. A. (1979) Relationship between spatial frequency selectivity and receptive field profile of simple cells. J. Physiol., Lend. 287, 163-176. BXKER, C. L. and CYNADE~.,M. S. (1986) Spatial receptive field properties of direction-selective neurons in cat striate cortex. J. Neurophysiol. 55, 1136-1152. BAKER,C. L. and HESS, R. F. (1984) Linear and non-linear components of human electroretinogram. J. Neurophysiol. 51, 952-967. B ~ o , N , N., P^YNE, B. R., LABOR, D. R. and MURPHY, E. H. (1982) Functional organization of neurons in cat striate cortex: variations in ocular dominance and receptive field type with cortical lamina and location in visual field. J. NeurophysioL 48, 1362-1377. BISHOP, P. O. and HENRY, G. H. (1972) Striate neurons: receptive field concepts. Invest. Ophthalmol. 11, 346-354. BISHOP, P. O., COOMnS, J. S. and HENRY, G. H. (1971) Responses to visual contours: spatial-temporal aspects of excitation in the receptive fields of simple striate neurons. J. Physiol., Lond. 219, 625-657. BISHOP, P. O., COOMBS, J. S. and HENRY, G. H. (1973) Receptive fields of simple cells in the cat striate cortex. J. Physiol., Lend. 231, 31-60. BLAKEMORE, C. and CAMPBELL, F. W. (1969) On the existence of neurons in the visual system selectively sensitive to the orientation and size of retinal images. J. Physiol., Lond. 203, 237-260. BONDS, A. B. (1984) Spatial phase dependence of orientation-selective inhibition in cat striate cortex. Invest. Ophthalmol. Vis. Sei. 25, 277. BONDS, A. B. (1988) The role of inhibition in the specification of orientation selectivity of cells in the cat striate cortex. Vis. Neurosci. In press. BULLIER,J. and HENRY, G. H. (1979a) Neural paths taken by afferent streams in striate cortex of the cat. J. Neurophysiol. 42, 1264-1270. BULLIERJ. and HENRY,G. H. (1979b) Laminar distribution of first order neurons and afferent terminals in cat striate cortex. J. Neurophysiol. 42, 1271-1281. BULLIER, J., MUSTARI, M. J. and HENRY, J. H. (1982) Receptive-field transformations between LGN neurons and S-cells of cat striate cortex. J. Neurophysiol. 47, 417-438. CAMPBELL,F. W. and ROBSON,J. G. (1968) Application of Fourier analysis to the visibility of gratings. J. Physiol., Lond. 197, 551-556. CAMPBELLF. W., CARPENTER,R. H. S. and LEVINSON,J. Z. (1969) Visibility of aperiodic patterns compared with that of sinusoidal gratings. J. Physiol., Lond. 204, 283-298. CITRON, M. C., ERENSON,R. C. and IDE, L. S. (1982) Spatial and temporal receptive-field analysis of the cat's geniculate pathway. Vision Res. 21, 385-396. CYNADER, M., BERMAN, N. S. and HE]N, A. H. (1976) Recovery of function in cat visual cortex following prolonged deprivation. Expl Brain Res. 25, 139-156. D~N, A. F. (1981) The relationship between response amplitude and contrast for cat striate cortical neurones. J. Physiol., Lond. 318, 413-427. DEAN, A. F. and TOLHURST,D. J. (1983) On the distinctness of simple and complex cells in the visual cortex of the cat. J. Physiol., Lond. 344, 305-325, 1983. DE VALOIS, R. L. and DE VALOIS, K. K. (1980) Spatial vision. A. rev. Psychol. 31, 309-341. DE VALOIS,K. K., DE VALOIS,R. L. and YUND, E. W. (1979) Responses of striate cortex cells to grating and checkerboard patterns. J. Physiol., Lond. 291, 483-505. DE VxLOlS, R. L., ALBRECHT,D. G. and THOI~LL, L. G. (1982) Spatial frequency selectivity of cells in macaque visual cortex. Vision Res. 22, 545-559. ENROTH-CUGELL,C. and ROSSON,.I.G. (1966) The contrast sensitivity of retinal ganglion cells of the cat. J. Physiol., Lond. 187, 517-552. FERsTEP,, D. (1987) Segregation of X and Y afferents into

307

areas 17 and 18 of cat visual cortex. Soc. Neurosci. Abstr. 13, 1449. FINLAY, B. L., SCHILLER,P. H. and VOLMAN,S. F. (1976) Quantitative studies of single-cell properties in monkey striate cortex. IV. Corticotectal cells. J. Neurophysiol. 39, 1352-1361. GILBERT,C. D. (1977) Laminar differences in receptive field properties of cells in cat primary visual cortex. J. Physiol., Lond. 268, 391-422. GILBERT, C. D. and WmSEL, T. N. (1979) Morphology and intracortical projections of functionally characterized neurons in the cat visual cortex. Nature, Lond. 280, 120-125. G~ZER, V. D. (1985) Spatial and spatialfrequency characteristicsof receptive fields of visual cortex and piecewise Fourier analysis. In: Models of Visual Cortex. Eds. D. ROSE and V. G. DOSSON. John Wiley & Sons: London. GLEZER,V. D. and COOPERMAN,A. M. (1977) Local spectral analysis in the visual cortex. Biol. Cybernetics 28, 101-108. GLEZ2X, V. D., COOPEmVtAN,A. M., IVANOV, V. A. and TSl-mXBACH, T. A. (1976) An investigation of spatial frequency characteristics of complex receptive fields in the visual cortex of the cat. Vision Res. 16, 789-797. GlaZER, V. D., TSl-mXBACH,T. A., GAUS~L~N, V. E. and BOND~KO, V. M. (1980) Linear and non-linear properties of simple and complex receptive fields in area 17 of the cat visual cortex. Biol. Cybernetics 37, 195-208. GLEZER, V. D., TSI~RBACH,T. A., GAU~LMAN,V. E. and BONDAgKO,V. M. (1982) Spatio-temporal organization of receptive fields of the cat striate cortex. Biol. Cybernetics 43, 35--49. GOODWIN, A. W. and HENRY, G. H. (1975) Direction selectivity of complex cells in comparison with simple cells. J. Neurophysiol. 38, 1524-1540. GR.~HAM, N. (1977) Visual detection of aperiodic spatial stimuli by probability summation among narrowband channels. Vision Res. 17, 637-652. GROSSBERG, S. and MINGOLLA, E. (1985) Neuronal dynamics of perceptual grouping: textures, boundaries, and emergent segmentations. Perception Psychophys. 38, 141-171. HAMMOND, P. and MACKAY, D. M. (1976) Differential responses of cat visual cortical cells to textured stimuli. Expl Brain Res. 22, 427-430. HAMMOND, P. and MACKAY, D. M. (1977) Differential responsiveness of simple and complex cells in cat striate cortex to visual texture. Expl Brain Res. 30, 275-296. HAMMOND,P. and SMITH,A. T. (1982) On the sensitivity of complex cells in feline striate cortex to relative motion. Expl Brain Res. 47, 457-460. HEC,GELUND, P. (1981a) Receptive field organization of simple cells in cat striate cortex. Expl Brain Res. 42, 89-98. HEC,GELUND, P. (1981b) Receptive field organization of complex cells in cat striate cortex. Expl Brain Res. 42, 99-107. HEC,GELUND, P. (1985) Receptive field organization of simple and complex cells. In: Models oftbe Visual Cortex. Eds. D. ROSE and V. G. DOnSON. John Wiley & Sons: London. HEGGELUND,P. and ALBUS,K. (1978) Response variability and orientation discrimination of single cells in striate cortex of cat. Expl Brain Res. 32, 197-211. HEGGELUND,P. and MooRs, J. (1983) Orientation selectivity and the spatial distribution of enhancement and suppression in receptive fields of cat striate cortex cells. Expl Brain Res. 52, 235-247. HENRY, G. H. (1977) Receptive field classes of cells in the striate cortex of the cat. Brain Res. 133, 1-28. HENRY, G. H. and BISHOP, P. O. (1972) Striate neurons: receptive field organization. Invest. Ophthalmol. 11, 467-473.

308

I"I. SPITZERand S. HOCHSTEIN

HENRY, G. H., D~mraL B. and BisHoP, P, O. (1974) Orientation specifity of cells in cat striate cortex. J. Neurophysiol. 37, 1394-1409. HENRY, G. H., GOODWIN, A. Y. and BISHOP, P. O. (1978) Spatial summation of responses in receptive fields of single cells of cat striate cortex. Expl Brain Res. 32, 245-266. HENRY, G. H., H,~a~vEY,A. R. and LUNO, J. S. (1979) The afferent connections and laminar distribution of cells in cat striate cortex. J. comp. Neurol. 187, 725-744. HENRY, G. H., MUST~I M. G. and BCLLmR, J. (1983) Different geniculate inputs to B and C cells of cat striate cortex. Expl Brain Res. 52, 179-189. HOCmT~IN, S. and SrO,PLEY, R. M. (1976a) Quantitative analysis of retinal ganglion cell classification. J, Physiol., Lond. 262, 237-264. HocP,Sa~IN, S. and SHAPLEY, R. M. (1976b) Linear and nonlinear spatial subunits in Y cat retinal ganglion cells. d. Physiol., Lond. 262, 265-284. HOCHSTEIN, S. and SPITZER, H. (1984) Zero-crossing detectors in primary visual cortex? Biol. Cybernetics 51, 195-199. HOCHST~Ir~, S. and SPITZER, H. (1985) One, few, infinity: linear and nonlinear processing in the visual cortex. In: Models of the Visual Cortex. Eds. D. ROSE, and V. G. DOgSON. John Wiley & Sons: London. HUBEL, D. H. and WIESELT. N. (1959) Receptive fields of single neurons in the cat's striate cortex. J. Physiol., Lend. 148, 547-591. HUBEL, D. H. and WmSEL, T. N. (1961) Integrative action in the cat's lateral geniculate body. J. Physiol., Lend. 155, 385-398. HUBEL, D. H. and WimEL, T. N. (1962) Receptive fields, binocular interaction and functional architecture in the cat's visual cortex. J. Physiol., Lend 160, 106-154. HUBEL, D. H. and WraSSE, T. N. (1968) Receptive fields and functional architecture of monkey striate cortex. J. Physiol., Lend. 195, 215-243. HCaEL, D. H. and Wra~L, T. N. (1977) Ferrier Lecture. Functional architecture of macaque monkey visual cortex. Prec. R. Soc. Ser. B. 198; 1-59. Ird~DA, H. and WglOrrr, M. J. (1975) Spatial and temporal properties of sustained and transient neurones in area 17 of the cat's visual cortex. Expl Brain Res. 22, 363-383. JosFs, J. P. and PhL~,l~, L. A. (1987a) The two dimensional spatial structure of simple receptive fields in cat striate cortex. J. Neurophysiol. 58, 1187-1211. JONES, J. P. and PALMER, L. A. (1987b) An evaluation of the two-dimensional Gabor filter model of simple receptive fields in cat striate cortex. J. Neurophysiol. 58, 1233-1258. KATO, H., BISHOP, P. O. and OgaAN, G. A. (1978) Hypercomplex and simple/complex classifications in cat striate cortex. J. NeurophysioL 41, 1071-1095. KING-SMITH, P. E. and KULIKOWSKI, J. J. (1975) The detection of gratings by independent activation of line detectors. J. Physiol., Lend, 247, 237-271. KULIKOWSK1J. J. and BISHOP, P. O. (1982) Silent periodic cells in the striate cortex. Vision Res. 22, 191-200. KULIKOWSKI, J. J. and MugrtAv, I. J. (1985) Contribution of striate cortical cells to pattern, movement and colour processing. In: Models of the Visual Cortex. Eds. D. ROSE and V. G. DO~ON. John Wiley & Sons: London. LENNm, P. (1980) Parallel visual pathways: a review. Vision Res. 20, 561-594. M^r~EI, L. and FIORENTIN1, A. (1972) Retinogeniculate convergence and analysis of contrast. J. Neurophysiol. 35, 65-72. MArt~l, L. and FIORENTIm,A. (1973) The visual cortex as a spatial frequmacy analyzer. Vision Res. 13, 1255-1267. MArl~I, L. and FIOREN'nNI, A. (1977) Spatial frequency rows in the striate visual cortex. Vision Res. 17, 257-264. MAFFEI, L., MORRONE, C., PIRCmO, M. and SANDINI, G.

(1979) Responses of visual cortical cells to periodic and non-periodic stimuli. J. Physiol., Lend. 296, 27--47. MARCEtaA, S. (1980) Mathematical description of the responses of simple cortical cells. J. opt. Soc. Am. 70, 1297-1300. MA~, D. (1982) Vision. W.H. Freeman: San Francisco. MOVSHON, J. A. (1975) The velocity tuning of single units in cat striate cortex. J. Physiol., Lend. 283, 79-100. MOVSHON, J. A., THOMPSON, I. D. and TOLm.mST, D. J. (1978) Spatial summation in the receptive fields of simple cells in the cat's striate cortex. J. Physiol., Lend. 283, 53-77. MOVSHON, J. A., THOMPSON, I. D. and TOLHURST, D. J. (1978b) The receptive field organization of complex cells in the cat's striate cortex. J. Physiol., Lend. 283, 79-99. MOVSHON, J. A., THOMPSON, I. D. and TOL.URST, D. J. (1978c) Spatial and temporal contrast sensitivity of neurones in area 17 and 18 of the cat's visual cortex. J. Physiol., Lend. 283, 101-120. MULLmIN, W. H., JONES, J. P. and PALMER,L. A. (1984a) Receptive-field properties and laminar distribution of X-like and Y-like simple cells in cat area 17. J. Neurophysiol. 52, 350-371. MULLm1N, W. H., JONES, J. P. and PALMEa, L. A. (1984b) Periodic simple cells in cat area 17. J. Neurophysiol. 52, 372-387. ORBAN, G. A. (1984) Neuronal operations in the visual cortex. Springer-Verlag, Berlin. PALMER, L. A. and DAVIS, T. P. (1981) Receptive-field structures in cat striate cortex. J. Neurophysiol. 46, 260-276. PALMER, L. A. and JONES,J. P. (1984) Quantitative analysis of cat simple receptive fields in two spatial and spatial frequency dimensions. Soc. Neurosci. Abstr. 10, 800. PALMEa, L. A. and ROSENQUIST,A. C. (1974) Visual receptive fields of single striate cortical units projecting to the superior coUiculus in the cat. Brain Res. 67, 27-42. PALMER, L. A., JONES, J. P. and MULLIKINW. H. (1985) Functional organization of simple receptive fields. In: Models of the Visual Cortex. Eds. D. ROSE and V. G. DOBSON. John Wiley & Sons: London. PETTIGREW, J. D., NIKARA, T. and BISHOP, P. O. (1968) Responses to moving slits by single units in cat striate cortex. Expl Brain Res. 6, 373-390. POLLEN, O. A. and RONNER, S. F. (1975) Periodic changes across the receptive fields of complex cells in the striate and parastriate cortex of the cat. J. Physiol., Lend. 245, 667-697. POLLEN, D. A. and RON~R S. (1982) Spatial computation performed by simple and complex cells in the visual cortex of the cat. Vision Res. 22, 101-118. POLLEN, D. A. and RONNER, S. F. (1983) Visual cortical neurons as localized spatial frequency filters. IEEE Trans. Sys. Man. Cyb. SMC-13, 907-918. POLLEN, D. A., ANDREWS,B. W. and FELDON, S. E. (1978) Spatial frequency selectivity of periodic complex cells in the visual cortex of the cat. Vision Res. 18, 665-682. POLLEN, D. A., NAGLER M., DAUGMAN,J., KRONAUER,R. and CAVANAUGH,P. (1984) Use of Gabor elementary functions to probe receptive field substructure of posterior inferotemporal neurons in the owl monkey. Vision Res. 24, 233-241. ReaSON, J. G. (1966) Spatial and temporal constrast sensitivity functions of the visual system. J. opt. Soc. Am. 56, 1176-1186.

ROSE, D. (1979) Mechanisms underlying the receptive field properties of neurons in cats visual cortex. Vision Res. 19, 533-544. ROSE, D. and BLAKEMORE,C. (1974) An analysis of orientation selectivity in the cat's visual cortex. Expl Brain Res. 20, 1-17. SAKITT,B. and BAgLOW,H. B. (1982) A model for economic

COMPLEX-CELLRECEPTIVEFIELD MODELS cortical encoding of the visual image. Biol. Cybernetics 43, 97-108. SCHILLER,P. H., FINLAY,B. L. and VOLMAN,S. F. (1976a) Quantitative studies of single-cell properties in monkey striate cortex. I. Spatiotemporal organization of receptive fields. J. Neurophysiol. 39, 1288-1319. SCHILLER,P. H., FINLAY,]3. L. and VOLMAN,S. F. (1976b) Quantitative studies of single-cell properties in monkey striate cortex. III. Spatial frequency. J. Neurophysiol. 39, 1334-1351. SCHILLER,P. H., FiNLAY, B. L. and VOLMAN,S. F. (1976c) Quantitative studies of single-cell properties in monkey striate cortex. V. Multivariate statistical analysis and models. J. Neurophysiol. 39, 1362-1374. SHAPLEV,R. and LENNm,P. (1985) Spatial frequency analysis in the visual system. A. Rev. Neurosci. 8, 547-583. StmRMAN, S. M., WATKINS,D. W. and WILSON,J. R. (1976) Further differences in receptive field properties of simple and complex cells in cat striate cortex. Vision Res. 16, 919-927. SINGER, W. and CI~UTZ~LDT, O. (1970) Reciprocal lateral inhibition in on and off center neurons in the lateral geniculate body of the cat. Expl Brain Res. 10, 311-330. SINGER, W., TROTTER, F. and CYNADER, M. (t975) Organization of cat striate cortex: a correlation of receptivefield properties with afferent connections. J. Neurophysiol. 38, 1080-1098. So, Y. T. and SHAPLEY,R. M. (1979) Spatial properties of X and Y cells in the lateral geniculate nucleus of the cat and conduction velocities of their inputs. Expl Brain Res. 36, 533-550. So, Y. T. and SHAPLEY,R, M. (1981) Spatial tuning of cells in and around lateral geniculate nucleus of the cat: X and Y relay cells and perigeniculate interneurons. J. Neurophysiol. 45, 107-120. SPITZER, H. and HOCHSTEIN, S. (1985a) Simple and complex-cell response dependencies on stimulation parameters. J. Neurophysiol. 53, 1244-1265.

JP.N.31/4~D

309

SPITZF~, H. and HOCHSTFJN, S. (1985b) A complex-cell receptive-field model. J. Neurophysiol. 53, 1281-1301. SPITZl~, H. and HOCHSTEIN,S. (1987) Visual receptive fields of cat conical neurons lack the distinctive geniculate Y cell signature. Isr. J. Med. Sci. 23, 69-74. Sprrz~, H., RICtlMOt~'D,B. J. and OPTICAN, L. M. (1984) Linearity of response in monkey striateneurons depends on stimulus parameters. Invest. Ophthalmol. Vis. Sci. 25, 32. SPITZER, H., RICHMOND, B. J. and OPTICAN, L. M. (1988) Linearity of onset and offset responses in monkeys neurons depends on stimulus parameters. J. Neurophysiol., submitted. STONE, J. (1983) Parallel Processing in the Visual System. Plenum: New York. STONE,J. and DREtmR, B. (1973) Projection of X and Y cells of the cat's lateral geniculate nucleus to area 17 and 18 of visual cortex. J. Neurophysiol. 36, 551-567. TANAKA, K. (1983) Cross-correlation analysis of geniculostriate neuronal relationships in cats. J. Neurophysiol. 49, 1303-1318. TOYAMA,K., MACKAWA,K. and TOKEDA,T. (1973) Analysis of neuronal circuitry for two types of visual cortical neurons classified on the basis of their responses to photic stimuli. Brain Res. 61, 395-399. TOYAMA, K., MACKAWA,K. and TOKEDA, T. (1977) Convergence of retinal inputs onto visual cortical cells: I. A study of the cells monosynaptically excited from the lateral geniculate body. Brain Res. 137, 207-220. TOYAMA,K., KIMURA,M. and TANAKAK. (1981a) Crosscorrelation analysis of interneuronal connectivity in the cat visual cortex. J. Neurophysiol. 46, 191-201. TOYAMA, K., KIMURA, M. and TANAKA, K. (1981b) Organization of cat visual cortex as investigated by the cross-correlation technique. J. Neurophysiol. 46, 202-214. ZEEVi, Y. and PORAT, M. (1984) Combined frequencyposition scheme of image representation in vision. J. opt. Soc. Am. (A)I, 1248.