Chapter 22 Emergence of feature selectivity from lateral interactions in the visual cortex

C H A P T E R 22

Emergence of Feature Selectivity from Lateral Interactions in the Visual Cortex

U. E R N S T and K. P A W E L Z I K

M. T S O D Y K S

Institute for Theoretical Physics, University of Bremen, Kufsteiner Str., D-28334 Bremen, Germany

Department of Neurobiology, Weizmann Institute, Rehovot 76100, Israel

Handbook of Biological Physics Volume 4, edited by F. Moss and S. Gielen

9 2001 Elsevier Science B.V. All rights reserved

969

Contents

1.

Introduction

2.

Models

3.

4.

.................................................

....................................................

2.1.

Neuronal populations

2.2.

Dynamics of one column

........................................

2.3.

Coupled columns

...........................................

A simple model of visual cortex 3.1.

Orientation preference

3.2.

Direction preference maps

3.3.

R e c e p t i v e fields

Discussion

.....................................

........................................ .....................................

............................................

..................................................

Abbreviations

................................................

Acknowledgements References

......................................

.............................................

.....................................................

970

971 974 975 976 979 984 985 991 995 996 999 999 999

1. Introduction

It was a major breakthrough for the investigation of neurobiological mechanisms underlying brain function, when David Hubel and Thorsten Wiesel (Nobel price for Medicine 1981) discovered that neurons in the primary visual cortex become most strongly activated by elongated visual stimuli moving across the visual field [1,2]. Fig. 1a shows a typical recording of a simple cell of layer 4 in area 17 of the cat. A long bar of light moving across a screen leads to an increased number of action potentials only if: 9 the bar crosses a particular location, the so-called receptive field, 9 the bar has a particular orientation, and 9 a particular direction of motion. The responses are weaker or even vanish if one or more of these conditions are changed. It appears as if the neuron was selective for a particular set of features of a stimulus and for this reason one speaks of feature selectivity of the response. The neuron is said to be tuned for the feature, which is quantified by plotting the dependency of the number of spikes on the features, the so-called tuning curves (Fig. l b and c) show examples for orientation tuning and direction tuning curves, respectively). Hubel and Wiesel also discovered that the selectivity for location and for orientation varies gradually when the recording site moves smoothly from one cortical location to the next parallel to the cortical surface ("horizontally", "tangentially", see Fig. 2). In this way the responses realize mappings of retinal places and of orientations. With novel, recently developed methods of optical imaging of intrinsic signals Grinvald and Bonhoeffer [6] and Blasdel [7] uncovered the precise layout of these maps (Fig. 2b). It turned out that the selectivities for particular orientations and also for directions of stimuli are arranged smoothly across the cortex except for some points and lines where they change abruptly. These discontinuities are called pinwheels and fractures, respectively [6,8], see Fig. 2b). While the electrophysiological experiments of Hubel and Wiesel and many others had already shown that the mapping of retinal location to cortex is on average also smooth ("retinotopy") a very recent series of experiments revealed that the retinotopic mapping is correlated with the orientation map [4,5] in a way that a movement in cortex which is associated with a change of 90 ~ in orientation preference entails on average a movement of the receptive field by one receptive field size (Fig. 3). When Hubel and Wiesel discovered the response properties of the neurons in primary visual cortex, they also offered a simple explanation for the selectivity for orientation (Fig. 4). The idea was, that neurons selective for a particular orientation are connected to neurons in the lateral geniculate nucleus (LGN), that provide the input to the visual cortex, and whose (unoriented) receptive fields are arranged in an 971

972

U. Ernst et al.

I

I <

[I 0

1

I

I

90

180

270

360

I IIIIII

-!5

III IIII1111 I[ IIIIIL I

I

<

II 0

[

I

I

90

180

270

360

Fig. 1. Orientation and direction preference of cortical cells. (a) Oriented bars moving across the receptive fields (black boxes) of neurons evoke response which are stronger when stimulating with the preferred orientation of the nerve cell (see examples of spike trains on the right). (b) The rate A of one neuron in dependence of the stimulus orientation ~ yields the tuning curve of the neuron. The response in (b) is only orientation selective, while the response in (c) displays direction selectivity.

elongated manner. Despite the simplicity of this model it has surprisingly not been experimentally settled until now whether this picture is correct for the adult animal (see, however [9]). An alternative picture assumes that intracortical mechanisms strongly influence the selectivity for orientation such that a small bias provided by the structure of input connections would suffice to generate the full orientation preference map [10,11]. But also in this picture, the structure of the orientation maps is laid down in specific patterns of input connections. It is usually assumed that these patterns of input connections emerge by activity-dependent development [12-14]. It came as a blow to the idea that the structure of the selectivity maps emerge by activity-dependent self-organization of the input connections, when it was shown, that the maps which develop for the two eyes are very similar (if not identically) also when the animal never experienced vision through both eyes simultaneously [15,16]. Very recently it could be shown that the selectivity for ori-

973

Emergence of feature selectivity from lateral interactions in the visual cortex

b

. . . . . ::~::~:i::i::: ~:;~:

iiii iiiiii iii~i~i

I I/~-,~\

: ,~:~, .....

...................

I I/~

. . . . .

1

...... ,I m m

Fig. 2. (a) Columnar organization of the visual cortex, as proposed in the ice-cube model by Hubel and Wiesel [3]. The cortical surface appears to be divided into columns sharing similar response properties, as e.g., the orientation (colored bars) or the ocular dominance (R = right eye, L = left eye) of a stimulus. (b) "Real" orientation maps, however, show a more complicated architecture with singularities (pinwheels) and fractures (optical imaging of area 17 of the cat, data from T. Bonhoeffer).

1.6

-Oc(DO

i

.N'~ t.,

-•8 ,..E

HI II

i.2

c

ill

..LI_

r-n"

0.8

E~

o.~

9

gl

II

,In

an i

I

II

mild

i

i

. 'iI~

i iBm i~lI i

i im

> OO

0.4

E "~

0.2

u_,~ n " ~,.~,

5"~m.Hi

ali

b. |

o o

i

~s

4s

eT.s

9o

C h a n g e in Orientation (degrees) Fig. 3. The distance of the centers of two neighboring receptive fields is a linear function of the difference of preferred orientations of the corresponding neurons. The slope of this function is approximately 1 receptive field per 90 ~ of OP change, such that neurons with orthogonal orientation preferences have nonoverlapping receptive fields [4,5]. entation also emerges identically for both eyes, even if the eyes had no visual input at all [17]. These experimental results challenge the models for the development of feature selectivity which are based on the activity-dependent a d a p t a t i o n of the patterns of afferent connections alone. It appears that some yet u n k n o w n factor in the cortex seems to be responsible in the very young animal for the identity and the r e m a r k a b l e stability of the development of orientation maps under the various rather drastical

974

U. Ernst et al.

receptive fi

Fig. 4. Classical model for orientation preference in a simple cell, adapted from Hubel and Wiesel [1]. One neuron in the cortex (right) receives synaptic afferent input from many thalamic neurons (middle), whose receptive fields are aligned to match a specific stimulus orientation (left). Within this framework, inputs from simple cells with parallel receptive fields, converging onto a single postsynaptic cell, could explain the phase-independent response of complex cells to moving gratings.

experimental conditions. It has been speculated that the factors are genetic in origin [15]. It is, however, unclear what this could mean (see e.g. [18]). In this chapter, we review models for the dynamics of activities in cortex which are based on stereotyped intracortical interactions. These models stood at the very beginning of the mathematical description of collective phenomena in the brain. Nevertheless, the dynamics of these models may have far-reaching consequences and can explain a variety of experimental findings. In particular we will show that they might provide a novel explanation for the early development of feature maps in the visual cortex.

2. Models

The organization of the brain has a high degree of complexity. Every square millimeter of the cortical sheet contains over 106 neurons, each of them receiving input from at least 104 synapses. If we want to understand how these units interact and generate the dynamics seen in the experiments, it is very useful to reduce the amount of complexity. The goal is to build a model which is minimalistic in the sense that it captures the properties we want to analyze, but lacks all the details which may obscure the basic mechanisms. Despite the huge number of neurons, it appears that if one is only interested in mean firing rates, nerve cells within an approximate 0.1 rnm x 0.1 mm-column of the cortex respond very similarly to stimuli. Therefore, one can attempt to analyze cortical dynamics in terms of population activities of certain groups (columns) of neurons. To our opinion, it is surprising how many experimental results may be explained by simulating and analyzing such a simple model of visual cortical circuitry.

Emergence of feature selectivityfrom lateral interactions in the visual cortex

975

2.1. Neuronal populations First, one has to derive differential equations for the fraction of excitatory and inhibitory neurons being active at time t at position r = (rx,ry) T in the neuronal tissue. These activities will be termed Ae(r,t) and Ai(r, t), respectively. Let us first consider the activation dynamics of a single column, neglecting the coupling to other columns in the cortex. In 1972, Wilson and Cowan derived differential equations for

A (t) and A (t)

Ae(t + z) - [ 1 - f f

Ae(,')

x Se(fta(t Ai (t + v) = 1 -

- t')[weeAe(t')-wieAi(t')+Ie(t')]dt'),

Ai (t') dt' T

x Si(ft

ot(t - tl)[weiAe(t ') -wiiAi(t l) +Ii(tl)]dtl).

The fractions of active neurons at time t + z, where z denotes an absolute refractory period, are given by the products of the fraction of neurons having been not active for the time 9 (first term on the right-hand side) with the gain functions Se and Si. These gain functions model the excitability of the populations depending on the total synaptic input I. The synaptic input I has three contributions, one from each population with corresponding coupling constant wxx (x c {e, i}), and one from external sources as e.g., from the LGN. The first index of Wxx denotes the presynaptic, and the second index the post-synaptic population w~ is connecting to the function at models the synaptic response characteristics, which in the simplest cases is an exponential, ~(t) ~ exp(-t). These equations are not very convenient for computational purposes, so their complexity was further reduced by "time coarse-graining", an exponentially decaying time average over Ae (a similar expresssion is obtained for Ai):

Ae(t) "- ~1 A- - (t)

.-

1

ot(t- {)Ae({)d{,

f t

-

at',

oo

yielding the set of differential equations: dAe

Ze dt = -Ae q-- (ke - reA---~)Se(weeA--~- wieA--il-+ Ie), dAi

"Ci d---t-= -A~ + (ki - riA---~)Si(weiAe- wiiA---i~-+-Ii).

(1)

(2)

Without loss of generality, Eqs. (1) and (2) could be rescaled such that the time coarse-graining constants ke and ki become 1, ke = ki - 1. As the equations are freely scalable, we will omit to give absolute values for the parameters and variables. Only

976

U. Ernst et al.

at points where relations between parameters play an important role, we will discuss their implications for the biological length or time scales. F r o m now on, as in the original publication of Wilson and Cowan [19], we will use Ae (Ai) instead of the terms Ae (Ai). The functions Se and Si summarize the distributions of e.g., different neuronal thresholds or different synaptic weights, or the system's response to noisy inputs. It can be expected that Se and Si increase monotonically with their argument, and saturate above a certain activation level. If the distributions are unimodal, it has been shown that Se and Si indeed have the shape of a sigmoid function, as e.g., the logistic curve 1

1

Slog(I) := 1 + exp[--Zs(I -- If)] - 1 + exp(Zslf) " Another common choice is a threshold-linear function Slin Slin(I) " - s . ( I - / f )

for I > If, and 0 otherwise.

Equivalently, Se and Si could be interpreted as the firing rate of a single neuron depending on its total synaptic input. In the original work of Wilson and Cowan [19], the authors consider the dynamics of the fraction of neurons being active at time t, and therefore the activation or gain function has to be limited to values below 1. Alternatively, considering the mean activity (firing rate) of a neuronal population, the gain function could also have a threshold-linear shape as Slin. In Fig. 5, four different gain functions are plotted: the logistic curve Slog, a threshold-linear gain function Slin, the gain function Siaf of an integrate-and-fire neuron (see chapter by Meunier and Segev in this book), and the response curve for a pyramidal neuron Spy~ in the cortex [20]. It can be seen that all these curves share some common properties. Up to some firing threshold If, the response is small, then increases with some average slope s, until some saturation value is reached. Note, however, that the response curve typical for cortical neurons is surprisingly linear, and that most cortical neurons normally do not operate close to their saturation l e v e l - so Slin seems to be a good approximation to work with. At this point, we should note that the rate dynamics as described above has its functional limitations: the dynamics of single spikes and their correlations within one population or column have been averaged out. As can be seen in Eqs. (1) and (2), the time constants for the population dynamics are re and zi, and fast oscillatory responses on a smaller timescale, typically occurring when a stimulus is switched on [21], are not obtained by using the simplified dynamics. It has been shown that the full dynamics is able to switch instantaneously between different attractors of the population dynamics [22], which may lead to interesting dynamical phenomena beyond the scope of this chapter.

2.2. Dynamics of one column By setting dAe/dt =-dAi/dt- O, one can find the fixed points and analyze their stability in the phase space of Ae and Ai. To demonstrate the qualitative differences

Emergence of feature selectivity from lateral interactions in the visual cortex

100

977

j s." /

N

::

50

I,SJ .Z'," 9

---....

f.0

_ .... ~ t

0

~S

...... ,

1 I[nA]

Slog Sgr S,,o Siaf

2

Fig. 5. The response curve or gain function S models the excitability or rate of one neuron in dependence of its input current I. Above some threshold, S normally increases monotonically with I. Sial, integrate-and-fire neuron; Slog, logistic curve as used by Wilson and Cowan [19]; Spyr, pyramidal cell in the cortex [20] and Slin, threshold-linear neuron. Parameters are s = 7 0 H z / n A and I f = 0 . 3 n A and Slin; s = 1 4 0 H z / n A and I f = l nA for Slog; s = 1/(RCIf) = 60 Hz/nA and If = 0.3 nA for an integrate-and-fire neuron obeying the differential equation RC d V / d t = - V + R I for its membrane potential V (C = membrane capacitance, R = membrane resistance). between a threshold-linear and a sigmoidal gain function, we first consider only one population (excitatory or inhibitory) within a column, with the simplified dynamics of Eqs. (1) and (2) and r - 0 dA

"c - ~ - - A + S ( w A + I).

(3)

The fixed points can be found by solving Ao - S(wAo + I) for A0; if one has to deal with two populations, one has to solve a system of two fixed point equations. One population. There are two different regimes depending on the gain parameter s and the coupling constant w; the weak coupling regime sw < 1, and the strong coupling regime sw >~ 1. In the weak coupling regime, with both S = Slin and S = Slog, we find a stable fixed point A0 whose absolute value increases monotonically with increasing input I (Fig. 6a and b). The only difference between the two gain functions is that with Slog, A0 saturates at higher values of I. In the strong coupling regime, with S = Slin, there is either one stable fixed point at A0 - 0, or the activity increases beyond all limits. With S = Slog, depending on I either one stable fixed point near 0, one stable fixed point near maximum activity, or both of these fixed points coexist. This behavior results in hysteresis: with intermediate I, depending on the initial or previous activation level A, one of the fixed points either at the low activity or at the high activity level is reached (Fig. 6c and d).

978

U. Ernst et al.

b

< 0 0

0

,//

/

<

||

t .dr

0 I

qh,,J

0 I

Fig. 6. Fixed points of the dynamics of a single column. The figures show the fixed points A0 (open circles, stable; stars, unstable) as revealed by the intersections of the gain functions S with the identity. For (a) and (c) the threshold-linear gain function Slin, and for (b) and (d) the sigmoidal gain function Slog was used in (a) and (b) the weak coupling regime, and (c) and (d) in the strong coupling regime. The colors red, green, and blue mark increasing input levels I. In (a) and (b), the dynamics has one stable fixed point for each/, while in (c), only the fixed point at A = 0 may be stable - otherwise, the activity diverges. In (d) the dynamics can have up to two fixed points with medium input levels; here, the system undergoes hysteresis and the activity is limited by the saturating gain function.

Two populations. With two populations, Eqs. (1) and (2) yield two isoclines intersecting at the fixed points of the activation dynamics. Their stability can then be derived by linearization of Eqs. (1) and (2) around these fixed points and solving the characteristic equation. Using Sli,, the activation dynamics is very simple. There is no hysteresis in the system, and either a stable fixed point exists at Ae >/0 and Ai >/0, or the activation diverges because the interaction is too strong. With Slog, there is the possibility of multiple hysteresis phenomena. Increasing the constant input, one finds either one, two, or three stable fixed points existing simultaneously (Fig. 7a and b). The existence of hysteresis is very important, because it can implement a form of short-term memory: brief pulses of external input can excite a column, which remains activated after the input has decayed, due to the dynamics of the internal couplings. Additionally, there is a parameter range where the model can exhibit (damped) oscillations in the population activity. These solutions of the differential equations correspond to the existence of limit cycles in phase space. Limit cycles occur if there is only a single unstable fixed point of the dynamics, and if the input is sufficiently

Emergence of feature selectivity from lateral interactions in the visual cortex

979

b

0.5 0.4

0

A.

0.4

-0.5

0

I

I

1

e

Fig. 7. Hysteresis in the Wilson-Cowan model. (a) shows intersecting isoclines for three different excitatory input currents Ie (red, Ie - 0; green, Ie = 0.5; blue, Ie = 1). Fixed points are marked as in Fig. 6. (b) Depending on the excitatory input current Ie, either one (blue), two (green), or three fixed points (red) are stable, and the initial conditions determine which one is selected. Adapted from Wilson and Cowan [19], parameters were Wee = 13, Wie - - 4, Wei - - 2 2 ,

Wii "- 2, Se - - 1.5,

Si = 6, If, e ---- 2 . 5 ,

If, i = 4 . 3 ,

re = ri - - 1, "1;e - - 10, ~i = 5,

and Ii = 0. high. It can be shown that limit cycles occur naturally in coupled neuronal populations [19]. It has been speculated, that these oscillations explain the rhythmicity seen in the frequency bands of E E G recordings, or other oscillatory phenomena in brain activity. In the following, we will concentrate on nonoscillatory solutions.

2.3. Coupled columns To simulate more than a local cortical column, Wilson and Cowan extended their model and examined a chain of coupled neuronal populations [23]. The activation A is now a function of time and space, A(t) ~ A(r, t), and the synaptic input now depends not only on the activities of the populations in the same column, but also on the activities in all other columns. The products in Eqs. (1) and (2) therefore have to be replaced by the convolution of the activities with the corresponding coupling kernels Wee (r - r'), Wei (r - l't), ~/ie (r - r'), and H/ii(r - r'), where [W 9 A](r, t) := j c

W(r - r')A(r')dr' /7(

Te

~Ae(r,

t)

~-----------~- -Ae(r, t) + (ke - reAe(r, t))Se([Wee * Ae] (r, t)

-[g(e ~i

* Ai](r, t ) + Ie(r, t)),

(4)

8Ai(r, t) ~----------~ = -Ai(r, t) + (ki - riAi(r,t))Si([~i * Ae](r,t)

-[Wii * Ai](r, t ) +

Ii(r, t)).

The delay of synaptic transmission from r to r ~ has hereby been neglected.

(5)

980

u. Ernst et al.

The choice of the coupling functions Wxx is crucial for the dynamics of the system. A common assumption is that excitatory couplings prevail on short distances [[r - r']], while inhibitory interactions dominate on larger distances. This leads to a coupling function having the shape of a Mexican hat (Fig. 8). It is questionable if this assumption is really fulfilled in the visual cortex. It has been shown that longrange horizontal connections spanning several hypercolumns exist [24--31], while inhibitory interactions have a limiting range of about one hypercolumn [32,33]. These long-ranging axons, however, are not distributed homogeneously but form dense clusters in columns having a similar orientation preference as the neuron from which they originate. Due to the typical structure of an orientation map in the visual cortex, it may still be possible that the interaction profile has indeed the shape of a Mexican h a t - at least in the young animal, where long-ranging excitatory connections have not been developed yet. For convenience, we will also assume that the coupling functions are chosen such that fc

d r ' W ~ ( r - r') = Wx~. TX

In many cases, it is reasonable to reduce Eqs. (4) and (5) introducing the following simplifications: first, let us assume that the neurons have a vanishing absolute refractoriness, r i - - r e - - 0 . Second, axons originating from one population should contact excitatory and inhibitory neurons in a given distance with equal relative probability, Wee cx Wei and Wie o( g~i- Our last assumption is that the remaining parameters and the gain functions in Eqs. (4) and (5) are identical for inhibitory and

r,.

x !

x v

-50

0 r-r x

50 x

Fig. 8. Excitatory couplings We(rx- r'x) (green) having a shorter length scale than inhibitory couplings ~ ( r x - r'x) (red) lead to a coupling function W ( R ~ - r'~)= We- N (black) having the shape of a mexican hat. Parameters of the coupling functions chosen (see Eqs. (7) and (8)) are We = 14, wi = 12.5, ere = 5.6, cyi = 10, and d = 1.

Emergence of feature selectivity from lateral interactions in the visual cortex

981

excitatory populations, re = r i , ke = ki, and Se = Si. Under these assumptions, Eqs. (4) and (5) are redundant and can be replaced by a single integro-differential equation

~A(r, t)

17

~---7---= -A(r, t) + S([We * A](r, t) - [ < 9 A](r, t) + I(r, t)).

In the following section, we will assume a threshold-linear gain function S make the following choice for the coupling functions We and N:

_ We(r - r')

We

(2 g)d/2(Y d e x p

__

Wi

(2 rt)d/zcr~ exp

(r - r')

(,r-r',,2) 2 cy2

--

(r-r',,2) --

2 cy2

(6) =

Slin

and

(7) '

(8)

"

The results are qualitatively identical for other choices, exceptions will be discussed. s will be termed the gain, If the firing threshold, We the excitatory coupling strength, wi the inhibitory coupling strength, and Cre and cri, with ~e < c~i, the excitatory and inhibitory coupling length scales, respectively, d denotes the dimensionality of the neuronal tissue.

2.3.1. Dynamics of coupled columns Constant input." Linear, marginally stable, and diverging regimes. For simplicity, let us first consider a one-dimensional chain of length lx, with periodic boundary conditions. This chain is stimulated with a constant external input I ( r , t ) = I0 = const. > If. What activation dynamics do we expect from the model in Eq. (1.6)? It is simple to calculate that there exists one single spatially homogeneous fixed point A0 > 0, if wi > We - 1/s (for arbitrary coupling functions) A0(r, t) = A0 =

(Io - l d

1 - S(We -

wi)"

Is this fixed point stable? After linearization of the dynamics and solving the characteristic equation by applying the Fourier transformation, one obtains the following spectrum of eigenvalues )v(k):

1 I

a(k) = --+-

,17 ,17

weexp

If there exists one k for which stable. For our choice of eye < )~(kl) is always negative for V ' - c r e2/cr2 < 1. Therefore it is This is the case if

2

-wiexp

( 2 2)1 -

2

.

)v(k) > 0, then the homogeneous fixed point is un~i, 2 has one or two extrema at kl = 0 and k2 > 0. A0 > 0, and k2 does not exist for wi < we/V, crucial to know if)v(k2) is also negative ifwi > we/V.

U. Ernst et al.

982

wi > We ~" V. [(1 - V)-s] -}-1.

(9)

Summarizing these considerations, we have three stability conditions B1-B3: 9 B 1: Existence of a homogeneous fixed point for Wi > We - - 1Is. 9 B2: The homogeneous fixed point is stable for wi < we/V. 9 B3: For wi > we/V, the homogeneous fixed point is stable if condition (9) is fulfilled. Due to these conditions, the phase diagram of the activation dynamics depending on the lateral coupling strengths shows three different regions (Fig. 9a). If the inhibitory coupling is strong enough, the homogeneous fixed point is stable and every perturbation decays exponentially (linear regime) (Fig. 9b). If the excitatory weights become much stronger, the fixed point becomes unstable and the smallest perturbation of the homogeneous external input leads to an exponentially increasing activity. Due to the threshold in the gain function S, however, there is a subregion where the activity converges into an inhomogeneous stable state, the so-called marginally stable regime (Fig. 9c), which will be described later. If the excitatory interaction becomes too strong, the nonlinearity of the coupling function does not suffice to limit the diverging activity (diverging regime). We want to emphasize that this divergence could be avoided by using a saturating gain function as e.g., 5'1o8. In cortical neurons, however, neurons barely operate near their limits - so the distinction between the divergent and marginally stable states could help to find an operating regime where the activity level of the cortical neurons is regulated only by the network, and not by the internal dynamics of a single element.

Nonconstant input in the marginally stable regime. In the linear regime, spatially inhomogeneous input leads to a similar activity distribution, because afferent input dominates over lateral feedback. In the marginally stable state, however, the activation dynamics becomes much more interesting. Let us assume that we have a neuronal chain of length lx with periodic boundary conditions, and that there is a positive perturbation in the input at the position r~x = lx/2. Two possible realizations of this input would be:

' 1 = 10(1 +

cos(2

(lo)

(rx -

Due to the excitatory interactions prevailing on short distances, this perturbation becomes enhanced, while the activity in the surround becomes suppressed by the inhibition. This leads to a localized activation blob centered around 1~/2. If the afferent input is suprathreshold everywhere, other blobs appear in a specific distance which is determined by the length scales of the excitatory and inhibitory interactions. In Fig. 9c, this distance is about half of the size of the chain, such that two activation clusters appear. This picture does not change significantly in higher dimensions: in a two-dimensional cortex, the activation clusters typically

Emergence of feature selectivity from lateral interactions in the visual cortex

983

marginally stable regime

100

0 -2

-4 0

1

2

0

linear regime

1

2

50 divergent regime 0 0

I

n

,

i

20

30 W

0

,

i

40

50

1

2

60

e

b

c

20

200 I

n

m

m

m

m

m

|

m

u

m

u

m

u

<

m

m

m

m

m

n

<

(I)

.=

0 0

50 r

X

I O0

0 0

50

100

F

X

Fig. 9. (a) Phase diagram in dependence of the excitatory and inhibitory coupling strengths We and wi. The conditions B1-B3 partition the phase space in three regions: in the upper region, the homogeneous fixed point is stable (b), and in the lower region, no fixed point exists and the activity diverges exponentially. The region in between shows a different behavior. Here, the homogeneous fixed point is unstable, so each minimal local perturbation of an otherwise constant synaptic input leads to pattern formation, which is stable (c) or unstable, depending on the actual strength of the inhibitory coupling. The green line separates the linear from the marginally stable, and the red line marks a lower boundary of the marginally stable regime. The blue lines are numerical estimates of the phase boundaries. (b) and (c) show successive activity profiles A(rx, t) after the system has been stimulated with a homogeneous input with a small perturbation, at times t = 1.25, 45 (dotted), t--3.75, 50 (dashed), and t = 50, 55 (solid), respectively. Parameters for the simulation were We = 30, 45, We = 80, 32, lx = 100, CYe • 5.6, (5"i - - - 10, z = 5, s -- 100, I = 1, At = 0.25. The insets in (a) display typical eigenspectra X(k) for the three cases. a r r a n g e in a h e x a g o n a l p a t t e r n (Fig. 10). If the inhibitory interaction extends over larger distances, or even does n o t decay significantly as in [34], then the n e t w o r k implements some sort o f a winner-takes-all n e t w o r k with global inhibition. Only one blob will a p p e a r at the location with strongest feedback and afferent input, a n d all other n e u r o n s will be inhibited.

U. Ernst et al.

984

80

m '~'

e 0

' 40

~>~

0

t

O O

O

O

O O

, 0 b:. A(r x, ry)

0

9

O ql

0 O

O 0 0

4

O

O

p

O 9

O

0

:i4

0 0

9

Q

4

0 rx

Fig. 10. Stationary activation pattern A(rx, ry) in a two-dimensional homogeneous model cortex, obtained with a uniform stimulus. The blobs arrange in a regular hexagonal pattern. The activity A is coded in shades of grey, see color bar. Simulation parameters are Ix = 8 8 , ly = 105, W e = 45, W i = 60, O"e = 2.8, O'i = 5 , T = 5 , S = 100, I 0 - 1, and At = 1.

There are two other interesting dynamical states in this model leading to propagating waves or blobs of velocity ~'~b [35-37]. In the first state, a movement of a periodic stimulus with velocity ~s as modeled e.g., by I = I0(1 + e cos(2 ~(0.5 + ~st +

rx/lx))

(11)

drags the blobs into the direction of movement. Depending on the time scale of the lateral dynamics and the modulation amplitude ~ of the stimulus, the activation either follows the stimulus perfectly with [~b -- fls, or misses some cycles (fib < fls). In the second case, a small asymmetry in the input leads to a self-propagating wave. Here, a necessary condition is ~ << 0, because otherwise the blob becomes pinned at the position of maximal input. Traveling waves typically occur when the inhibitory input is large [37].

3. A simple model of visual cortex In the previous section, we discussed that a simple model of coupled neuronal populations can exhibit interesting dynamical properties: from simple fixed points to pattern formation, and from network oscillations to traveling waves. Here, we want to make the connection from the model to various experimental observations in the primary visual cortex. We will show that the rate dynamics formalism can explain a variety of dynamical properties of real neurons in the visual system.

Emergence of feature selectivity from lateral interactions in the visual cortex

985

3.1. Orientation preference Neurons in V1 respond perferentially to moving stimuli having a specific orientation 9 . Recording the firing rate A of a cortical neuron depending on the orientation of the stimulus in its receptive field yields the so-called orientation tuning curve A (~). It has been observed that the width of A(~) is nearly independent of the contrast or stimulus intensity I (orientation tuning invariance). One possible mechanism for orientation preference was proposed by Hubel and Wiesel: one cortical neuron receives input from a number of thalamic neurons whose receptive fields are aligned to match one specific stimulus orientation, see Fig. 4. Although the synaptic input indeed shows a tuning matching the preferred orientation of the cortical cell, the aspect ratio of this tuning is far too weak to explain the sharpness of the orientation tuning in cortex (see, however, [38,39]). One method to sharpen this tuning is to introduce some inhibitory intracortical interaction suppressing the response of the cell to nonoptimally tuned stimuli, but still the data could not be explained because the orientation tuning width now depends on the stimulus intensity. A better idea is to use excitatory and inhibitory intracortical interactions, as has been demonstrated in the work of Ben-Yishai and Sompolinsky [10]: Let us again assume we have a one-dimensional chain of coupled cortical columns with length lx. We now identify this chain with a hypercolumn in the visual cortex by assuming a linear dependence between position and orientation preference, ~(rx)= rcrx/lx. This assumption is plausible because orientation preference varies smoothly with position within the cortex, as can be seen in typical examples of orientation preference maps (see Fig. 2b). Accordingly, each stimulus with orientation ~s within the receptive field of this hypercolumn is modeled as a thalamic input

I(rx) = I0[1 + gcos(2((I)(rx) - (I)s))] to the cortex, z << 1 is a weak bias of the input, as e.g., provided by the spatial arrangement of the receptive fields of the LGN cells as proposed by Hubel and Wiesel, encoding the orientation preference of the cortical columns. From the previous section, we know that the model, working in the marginally stable regime, will enhance this perturbation e in the afferent input, leading to a stable activation blob centered around the neuron with orientation preference ~s. Columns with orientation preferences not similar to ~s become suppressed. As the model is symmetric with respect to an exchange of (I)s and ~(rx), the activation profile A(rx) can be identified with the orientation tuning curve A(~s). From Eq. (6), it is easy to see that the shape of A(~s) does not depend on the stimulus contrast I0, provided that the afferent bias ~ is weak. From this evidence, it can be concluded that lateral interactions may play an important role in shaping the responses of cortical neurons in V1, leading to contrast-invariant tuning curves. Within this framework, some other properties of the visual system could be explained easily. It is well known that small angles between two adjacent line segments are overestimated (Fig. 1 l a). Due to the inhibitory interactions, two activation blobs can coexist only if they are separated by a distance roughly equivalent to

986

U. Ernst et al.

a

b

< 0

r

•

Fig. 11. (a) Small angles between line segments are overestimated by our visual system; we perceive a tilt in the vertical lines despite they have been laid out in parallel. (b) Possible explanation for the phenomenon shown in (a). If difference in orientation A~ is identified with difference in cortical position Arx (compare with Fig. 2a), two line segments crossing at small angles lead to a bimodal input distribution I(rx) (thin line). The distance between the two maxima in this distribution is smaller than the minimal distance between two blobs, which appear shifted to the left or to the right, relative to the input. Thus, the position of their maxima in A(rx) may suggest for higher cortical areas that the stimulus displays a larger angle than present.

the length scale of these couplings. Choosing a stimulus with two maxima at angles corresponding to a smaller distance in the cortex, the model would predict that either only one single blob will appear, or two blobs within their minimal interblob distance. In the latter case, this repulsive effect could be interpreted as a much wider stimulus angle by higher cortical areas. Besides this phenomenon, other dynamical properties of the columnar activation dynamics have been directly related to the perception of illusory movements of ambiguous stimuli [40]. In these experiments, the observer sees flashing spots positioned on the corners of an invisible even-sided rectangle. In the first stimulation condition, the leftmost and rightmost two corners, and in the second condition, the uppermost and lowermost two corners are flashed alternately. The observer perceives these stimuli as an illusory movement in the horizontal and vertical directions, respectively. In a third stimulus condition, however the opposite corners (upper left and lower right, or upper right and lower left) are flashed alternately. This stimulus is ambiguous, it can be either perceived as a horizontal or a vertical illusory movement of the spots. Typically, an observer switches between these two percepts periodically. One could imagine that in the visual cortex each flashing light evokes one activation blob. If the stimulus changes, the blobs, instead of vanishing and reappearing at the new position of the spots, could be dragged to their new positions. This dragging of an activation cluster over the cortical surface could be interpreted as an illusory movement of the stimulus itself. With the ambiguous stimulus condition,

Emergence of feature selectivity from lateral interactions in the visual cortex

987

small perturbations would determine into which direction the blobs are dragged first, horizontally or vertically, and this behavior would persist until this attractor of the dynamics is left due to e.g., external influences or noise.

3.1.1. Consequences of disorder The pattern formation mechanism of the model has some other interesting properties which may have far-reaching consequences for cortical development. As we have seen, a small perturbation in the input becomes amplified and leads to the formation of a blob which is centered around the maximum of the input. What happens, if there are also perturbations in the couplings or in the density of the neurons, as one would expect in a "real" cortex? To investigate this question, we applied a small r a n d o m jitter rl(rx, ry) on the weights matrix, such that there will be positions where the coupling is stronger than at other positions. In Fig. 12, we simulated the model with and without jitter, applying the same input centered at different cortical positions. As can be seen in Fig. 12a, the neuronal response follows exactly the stimulus, in the first case, the system is marginally stable in the sense that a stable solution centered at position rx, is also a stable solution centered at position rx + Ar~. In the second case (Fig. 12b),

150

150

x_ x

x_ x

v

v

<

<

0

50 r X

100

0

50 r

100

x

Fig. 12. Superpositions of activation profilesA(rx) of a chain of coupled cortical columns to different afferent inputs (Eq. (10)). Each of the afferent inputs has a maximum at one specific location r'x. These locations have been chosen to be distributed equidistantly (in the simulations, the input has been shifted by equal distances, with periodic boundary conditions). While in (a), the network was homogeneous, in (b), random disorder has been introduced by applying a random displacement of rl(rx ) -- rand(l) on the columnar positions. In (a), the positions of the clusters are exclusively determined by the input maximum, in (b), the marginally stable continuum of attractors has been broken up into a finite set of attractors located at positions with maximal cortical feedback. Here, system disorder and input perturbation determine the neuronal response. The response to the stimulus centered in the middle of the chain is marked in red. Parameters were We = 45, Wi = 60, (5"e - - - 2 0 , O'i = 40, z = 5, s = 100, Io = 0.5, e = 0.05, At = 1, cyl = 5.

988

U. Ernst et al.

however, this continuum of stable solutions breaks down into a discrete set of attractors: the position of the blob is now determined by both the maximum or the specificity of the input, and the position in cortex where the (excitatory) feedback is strong [34]. This clustering phenomenon is qualitatively independent on the type of disorder: random neuronal positions, distributions of neuronal parameters, variations in the synaptic weights or neuronal density - all of these sources of quenched disorder lead to the same observations. It should be noted, that these variations are to be expected in a real biological system, so this phenomenon seems to be the generic case. As we will see in the next section, it is of interest how the inhomogeneities or impurities in the system affect the response not only to static, but also to moving stimuli. Without inhomogeneities, we already mentioned that a movement of the stimulus leads to a movement of the cortical activation pattern. What are the factors determining the velocity ~"~b of this pattern? First, the amount of noise 1"1 will determine the strength of a stimulus needed to delocate the blobs. Also, the modulation amplitude e of the stimulus, and its periodicity )~ compared to the length scale of the blob pattern, will have an impact on the movement. We simulated Eq. (6) on a two-dimensional cortical surface, applying input of the form I(r, t) = I0[1 + ecos(2 rc(rx + ~st)/A)] thus realizing a drifting sinusoidal grating close to stimuli used in experiments. Fig. 13 shows the resulting velocities ~'~b for fixed 11 and different A. It can be seen that with all other parameters held constant, movement is fastest if A has the same order of magnitude as the length scales of the lateral couplings. Additionally, movement is most pronounced if the velocity ~2s is small enough to match the dynamic's time constant, and if the stimulus modulation amplitude e is strong, which is what one would expect. The aforementioned condition on A, however, requires an intuitive explanation. With small gratings, it is clear that the stimulus modulation is smoothed out by the lateral feedback. But using a wide grating, the blobs also remain stationary. This counterintuitive behavior only occurs if multiple blobs are present. While one blob is dragged in one direction, the next blob positioned in the direction of the movement may receive more afferent input at its opposite side. Thus the two blobs counteract while trying to decrease their distance, which is not possible due to the existence of a minimum distance allowed by the length scales of the lateral couplings. While the blobs seem to remain stationary if one averages the activation dynamics over time, the stimulus movement nevertheless induces small oscillatory movements around the blob's positions. The dynamics resembles a particle in a potential well driven by a periodic force too weak to enable the particle to escape. This behaviour might underly the directional specificity of cortical neurons, as explained in the next sections.

3.1.2. Emergence of cortical maps The main result of the last section is that the position of the blobs is determined both by the local strength of the intracortical feedback, in conjunction with the local

Emergence of feature selectivity from lateral interactions in the visual cortex

a

A=4

989

b

A=8

1 cO _Q

0

0

1

4

E

0

16 192 64 ~ [.10-3]

4

~

0

192 o~ ~. [.10-3]

s

S

A=16

~

A=32

1

1

m""-n

cO

o9

t_______, x'3

r

0 1

4 0

192 o~ ~

[.10-3]

4 ~

s

0

192 u~ ~

[.10-3] s

Fig. 13. Movement ~')b of the blobs, normalized to the movement of the stimulus, f~s, for several parameter combinations: (a)-(d) for A - 4, 8, 16, 32, respectively. In dependence of the effective stimulus modulation amplitude e and the stimulus velocity f2s, the blobs either remain more or less stationary (f2b ~ 0), or are being dragged by the stimulus (f2b > 0). If ~'~b -- 1, the blobs move with the same velocity as the stimulus. Parameters as in Fig. 10.

strength of the afferent input. If one presents different stimuli like gratings of various orientations, then the pattern of blobs resulting from the lateral dynamics should be different for each orientation. This lead us to the idea that both orientation and direction selectivity of single neurons, as well as the shape of the corresponding maps, might emerge from a stereotyped pattern of intra-cortical connections instead of a bias in the afferent weights, and are therefore independent from visual experience. We tested this hypothesis on a two-dimensional model with r a n d o m inhomogeneities. Each neuron has an isotropic input field receiving afferent stimulation from a specific circular region of the visual field (VF).

I(r, t) - 2

/0/v 2

g O ' a ft

F

exp(

2crzfr

990

U. Ernst et al.

where G(R) is the visual stimulus (typically a sinusoidal grating or an oriented bar) at position R in the VF, and CYaff is the length scale of the arborization of the afferent connections. Due to the retinotopic organization of the cortex, we choose R(r) := r, and we assume also periodic boundary conditions for the visual field R to avoid perturbation of the results by geometrical constraints [18]. For these reasons, the VF had necessarily the same aspect ratio as the cortex, but it is possible to simulate different magnification factors of the retinotopic mapping by simply rescaling the stimulus. Without lateral connections within the network, the units do not exhibit any orientation or direction selectivity. We considered two types of stimuli G - full-sized sinusoidal gratings Gs and localized spots and bars G3. In the first case, the stimulus is a sinusoidal grating with period A, orientation (I), and velocity f/s Gs(R) "- 1 + e-cos

(2 rt(q~ + ~st)) A '

qx := Rx cos(O) + Ry sin(O), where ~ models the contrast of the stimulus, R - (R~, Ry)T a position in the VF, and t is the time. In the second case, the stimulus is a circular spot of size (Yl, centered at Rc GI(R) "- (1 - e ) + e exp ( -

IIR-RII2) 2 c~2

where Rc runs over all positions where localized stimuli were presented.

3.1.3. Orientation preference maps Similar to the experimental procedure, full-sized gratings moving in N = 16 different directions covering the full circle, (I)(n) = 2rm/N, were presented to the network. For each of these stimuli, after an initial time interval of To = 100 which was long enough for the activity patterns to build up, one single condition activity map An(r), n = 1,... ,N has been obtained by averaging the resulting activities A,(r, t) over a time interval of T = 800 time steps of At = 0.25 (integration with Runge-Kutta fourth-order)

[To+T --Tat-To An(r, 1

A,(r)

--

0.

As predicted, for each stimulus a different activation pattern has been obtained. For a specific neuron, this results in different activation levels depending on the orientation of the stimulus: the neuron spontaneously shows up an orientation and direction preference. The preferred direction | is obtained by vectorially summing up the averaged activities for each direction of motion. Similarly, the preferred orientation @(r) is obtained by the same procedure after first averaging the activity over two opposite directions of motion for each orientation of the grating (see e.g. [8])

Emergence of feature selectivity from lateral interactions in the visual cortex

1 O(r)

-

991

N

~ Z An(r) exp(iO(n)), n=l

1

N

D(r) = ~ Z An (r) exp (2 i(I)(n)), n=l

|

= arg{O(r)},

@(r) = arg{D(r)}. Why do the responses of the columns show a preference for certain stimulus orientations, while there is no structure or bias in the afferent connections? In terms of the symmetry breaking introduced by small inhomogeneities in the weight matrix, each stimulus selects a different subset of all possible attractors. The feedback of the lateral couplings both attenuates the irregularities in the system, and applies a spatial bandpass filter on the cortical response [41,42]. This bandpass filtering by the Mexican-hat type of interaction is responsible for the emergence of orientation preference maps. The neurons not only achieve a preference for one stimulus orientation, but orientation preference also changes smoothly across the cortical surface. The excitatory interactions are responsible for the smoothness of the maps (cooperation), while inhibitory interactions realize a sort of competition. Typically for the mapping of a periodical quantity onto a manifold, the orientation preference map (I)(r) shows characteristic singularities like pinwheels (places where you can find cells of all orientation preferences nearby) and fractures (elongated curves where cells change abruptly their orientation preference), Fig. 14. In this respect, the model is similar to the algorithms proposed by [41,42] explaining the structure of cortical maps by low-pass filtering noise. In the models described in these pages, however, the filter here is related to the dynamics of the biologically plausible lateral interactions which also have much wider capabilities to account for experimental evidence like contrast invariance of orientation tuning and other neuronal properties described in the previous sections. Most importantly, the positions of active patches were robust against random initial conditions due to the stabilizing effect of the inhomogeneities in the lateral connections [43]. 3.2. Direction preference maps

Having a closer look at the single condition maps An, we see that the neurons also exhibit a preference for a certain direction of the stimulus movement (Fig. 17a). The corresponding direction map (Fig. 15) closely resembles experimental data (Fig. 16). Where does this preference comes from? As we already know from the preceding section, moving the stimulus implies a periodic movement of the activation clusters. The existence of the inhomogeneities in the network now introduces barriers like hills in a potential landscape, as in the example with the particle in a potential well discussed above. If the force of the

992

U. Ernst et al.

.........t O

-

ep

J.

"

":

"

~0

..

r..-J

m

S

! % %

i

Fig. 14. (a) Single condition maps An for gratings drifting in n = 8 different orientations (see colored bars). (b) Orientation preference map O(r) obtained by vectorially summing up the single condition maps shown in (a). Parameters were lx = ly = 128, ere = 5.6, O'i = 10 W e --- 45, Wi = 60, O ' a ft - - 4, (q) = 1 (white noise), I0 = 10, s = 100, 9 = 5, At = 1, f~s = 0.2, and A ~ 18, which is approximately the size of a blob. stimulus m o v e m e n t does not suffice to drag the blob over the potential barrier into the basin o f the neighboring attractor, the stimulus m o v e m e n t will only induce an oscillation o f the blob a r o u n d a position which is identified relative to its stationary location in the long-time average. In Fig. 13, there exist regions where the blob moved, ~"~b > 0, or r e m a i n e d stationary, ~"~b = 0. The conditions s u p p o r t i n g a fast m o v e m e n t are: 9 a strong m o d u l a t i o n e o f the sinusoidal stimulus, because o f the c o m p e t i t i o n between the stimulus m o v e m e n t a n d the localization strength o f d o m i n a n t lateral interactions,

Emergence of feature selectivity from lateral interactions in the visual cortex

993

R

#,

g % m

|

grad(~(r))

* ID(r)i

Fig. 15. (a) Direction preference map | The preferred direction is shown color-coded according to the color bars on the left. (b) Relation between direction and orientation map. Discontinuities in the orientation preference map, as obtained from a gradient transform g r a d ( ~ ) - V/(O(I)/Orx)2+ (O~/~ry) 2 with periodic boundary conditions and periodic argument (I)(r), are coded in shades of yellow superimposed on the map of the directional selectivity ]D(r)[ coded in grey shades. Dark colors represent low, and bright colors represent high absolute values (normalized color table). Same parameters as in Fig. 14.

Fig. 16. (a) Direction preference map and (b) relation between direction and orientation map found in experimental studies from Shmuel and Grinvald [8]. Same representation as in Fig. 15, the sizes of the rectangles are approximately 3 mm z 2.5 mm. 9 a stimulus velocity f~s with a time constant similar to the rate dynamics, and 9 a convenient grating period A in the same range or larger than the typical interblob distance. Because all of these conditions have to be fulfilled to move the blobs over an inhomogeneous cortex, it is very improbable that the blobs will move being in a regime where lateral interactions are strong. Instead, most likely the shifts from the basin of attraction will lead to direction preference: moving the stimulus to the left, the average activation is shifted to the left, and moving the stimulus to the right, the average activation also shifts to the right. This effect provides a novel explanation

994

U. Ernst et al.

0

O A

IV

25

50

m

A(---)(r)

(A+i+,A - )/2

Fig. 17. (a) Two activity maps A+(r) obtained by stimulation with a grating moving up (top left) or down (top right) yield a the mean activation A(r) = (A+ + A - ) / 2 stimulating with a specific orientation (bottom). The single differential directional map is shown by the yellow and green arrows, their lengths coding the selectivity A + - A- for one of the two conditions. (b) shows the equivalent differential map from experiments by Shmuel and Grinvald [8]. Same parameters as in Fig. 14.

for the direction selectivity of the neurons, not relying on special types of neurons, [44] or asymmetries in the afferent connections, either leading to delayed s u m m a t i o n of action potentials [45,46], or to a sharpening of these asymmetries by cortical

Emergence of feature selectivity from lateral interactions in the visual cortex

995

feedback [47] - here, the direction selectivity is caused simply by intracortical inhomogeneities. From these observations, it is possible to explain another experimental result. It has been observed that patches having similar orientation preferences split into two subpopulations having opposite direction preferences (Fig. 17). As a patch of similar orientation preferences corresponds to an activation cluster evoked by a specific stimulus orientation, it is obvious that the shift of the cluster in both stimulus directions divides this patch into two subregions being more strongly activated by the one or by the opposite direction of movement. Necessarily, the deformation being roughly symmetrical, there has to be a narrow region within the activated area where the activities for stimuli moving in either of the two directions are the same [48]. These narrow regions of weak directional preference typically connect pinwheels or fractures in the orientation preference map (Figs.15b and 16b). A similar relationship between PO and PD cortical maps was recently observed in an experimental study [8]. By subtracting the two activity patterns for opposite directions one obtains the differential map (Fig. 17a, length of arrows). Comparing the average amplitude of this map with the corresponding differential map for two orthogonal orientations (not shown), one finds that the orientation map was about twice as strong as the direction map, in agreement with the experimental results obtained for kittens [8]. The relative weakness of the PD map in the network is a consequence of the strong pinning of the activity patches at the particular locations for a given orientation. As a result, the amplitude of the oscillatory movement of patches, and thus the amplitude of the corresponding differential map, is small.

3.3. Receptive fields Recently, several experiments have been carried out to determine the relationship between cortical maps and receptive fields. It has been found that on average, receptive fields are nonoverlapping for neurons more than half a hypercolumn apart. That means, if one measures the receptive fields of neurons positioned in a straight line in the cortex, receptive fields move about two average receptive field diameters per hypercolumn, i.e. a change in orientation preference of 180~ In the model, receptive fields should not be mixed up with the afferent input fields. The input field only determines which afferent input a column receives, but the activation response of the column is determined by the concerted action of all neighboring columns. It has to be expected that the receptive field as measured by the column's response to localized stimuli shown at specific cortical postions does not match the shape of the input field. To obtain the receptive field sizes and shapes, we applied a small nonoriented stimulus G1 at positions R c VF on a regular lattice, sampling the whole VF, thus leading to the activity distributions A(r, R). The receptive field at r was then to be analyzed as follows: with sign(x) = 1 for x > 0 and 0 otherwise, the sizes (average diameters) d(r) of the receptive fields in units of the lattice unit length were computed as

996

u. Ernst et al.

d(r) - ~ / ~ / J v v sign(A(r, R)) dR / 9 Accordingly, the receptive field centers C(r) were given by C(r)-jvvR.A(r,R)dR/fvvA(r,R)dR. It is known that even a very small localized stimulus provides broad sub-threshold excitation to the cortical area [27]. To account for this observation, it is convenient to choose the stimulus radius large enough for the afferent input to cover a big area in cortex, but at the same time small enough to allow only one activation blob to appear. Accordingly, the position where the blob appears will be the location where the lateral feedback is strongest. Without inhomogeneities, the neuronal response is centered at the maximum of the input distribution. In this case, one receptive field has the size of a typical blob (Fig. 18a), because units which are located at the opposite sides of the patch will have nonoverlapping receptive fields. Introducing inhomogeneities, the receptive field size can change from very small values (at positions with weak lateral feedback) up to the size of one hypercolumn (at positions with very strong lateral feedback), see Fig. 18b. Nevertheless, the mean receptive field size equals the size of half a hypercolumn. Correspondingly, the smallest distance between two units with nonoverlapping receptive fields should be about one half of the size of the hypercolumn. This relation can indeed be observed in simulations (Fig. 19b). We note here, that this relation relies on the intracortical interaction only, and not on a specific arrangement or length scale of feedforward connections from the thalamus. It would be very difficult to explain that relation by the dynamics of feedforward interactions.

4. Discussion

In this chapter we reviewed the dynamics of population activities within the framework originally developed by Wilson and Cowan [19]. In chains and twodimensional neuronal layers a Mexican-hat shaped coupling induces localized activation patterns. This simple dynamics can be related to response properties of neurons in primary visual cortex. We showed that this approach can also explain the shape of orientation and direction maps as well as the relation of columnar structures to receptive field size and movement. In particular this chapter demonstrates that lateral interactions can play a crucial role for the emergence of typical response properties of neurons and columns in primary visual cortex. The tuning properties of neurons responding to oriented moving stimuli result from the interplay between excitation on a short length scale, and inhibition dominating at larger distances. While the excitatory feedback amplifies the response, the inhibition sharpens the tuning and suppresses

Emergence of feature selectivity fi'om lateral interactions in the visual cortex

997

V 20

40 A(r, R)

R

20 U ~ . " ~ ~ ~'~

40

A(r, R)

R

Fig. 18. (a) Activations A(r,R) in a simulation with a homogeneous distribution of weights, and (b) in a simulation with a jittered weight matrix (data and parameters from Fig. 12), due to input distributions with maxima at positions R. While in (a), receptive fields A(r = rconst.,R) are of equal size and shape at all cortical positions r, in (b), receptive fields vary tremendously in size and shape, especially at positions where neuronal densities are small or, equivalently, cortical feedback is weak.

responses of columns having different orientation preferences. Within this flamework, several psychophysical experiments can be interpreted in terms of lateral interactions. Recent simulations demonstrate that local intra-cortical connections could also be responsible for the orientation and direction selectivity of single

U. Ernst et al.

998

3

'

Oo oo o

o ~o ~

2

'oo ~176176 o ~ ~o oJ'- o o~ ~ ~o

o

8

o o

,~o~176 ~,~o

-

,,oo

~#~oO ~ ~ o - ~ ~ Oo Oo~o o~o o

O.o~

<3

o

;~[~~B'~epog~"-oo

,,% cPO"

~~i~l~~

~

ooo o

_ o o o

~o o oo o

o@

o

o

o

O'U- 0

, O ~ ~ o O oO %0 0

0

0

~o

'

0

0

100

200

300

Fig. 19. The distance, AR, of the centers of two neighboring receptive fields is an approximately linear function of the difference of preferred orientations, A~, of the corresponding neurons. Compare with Fig. 3. Parameters as in Fig. 14, stimulated with nonoriented stimuli G1 with ~ = 1 and (5"1 = 5.

cortical neurons as well as the geometrical shape of the corresponding maps at the very early stages of cortical development. This hypothesis is reinforced by the result that the overall appearance of the simulated maps, and the interrelation between them was the same as of the experimentally observed maps. These maps emerge by a symmetry breaking induced by small inhomogeneities in the network, which are amplified by the feedback and smoothened by the band-pass filtering property of the coupling function. Formally, the inhomogeneities reduce the continuum of marginally stable states to a finite number of attractors, of which the oriented input chooses a subset as locations for the activation clusters. Since no activity-specific development of neither afferent nor lateral interactions was involved, the shape of the maps in our model is strictly a function of the pattern of connections within the network. It is possible that this mechanism of orientation and direction selectivity arises early during development and only subsequently the afferent and horizontal connections are adapted in an activity-dependent way. This development could be driven by either visual experience or by spontaneous activity waves in the retina which were observed in kittens even before opening of the eyes [49]. In the maturing cortex the connections then become orientation specific in register with the orientation preference of the neuron's responses [50,51,25,52,9]. If the general layout of local intra-cortical connections remains the same at the initial stages of development, our model could explain the remarkable stability of visual maps observed in the experiments [15-17]. Our results are also compatible with the finding that kittens raised with strabismus have only a single PO map that is continuous across the segregated monocular domains of the two eyes [53].

Emergence of feature selectivity from lateral interactions in the visual cortex

999

Abbreviations EEG, Electro encephalogram Hz, Hertz LGN, Lateral Geniculate Nucleus mm, millimeter nA, nano Amp6re OP, Orientation preference RF, receptive field t, time V, membrane potential V1, part of visual cortex (area 17) VF, visual field

Acknowledgements The work on the cortex model and cortical maps presented here was done in close collaboration with Carmit Sahar-Pikielny. We would also thank S. Hochstein, A. Shmuel, S. L6wel, W. Singer, and F. Wolf for valuable discussions; and Stan Gielen for his patience waiting for the manuscript. This work was supported in part by grants from the Max-Planck-Gesellschaft (K.P.) and the Deutsche Forschungsgemeinschaft (U.E.) through SFB 185 and SFB 517. References 1. 2. 3. 4. 5. 6. 7. 8. 9. 10. 11. 12. 13. 14. 15. 16. 17. 18. 19. 20.

Hubel, D. and Wiesel, T. (1992) J. Physiol. 160, 106-154. Hubel, D. and Wiesel, T. (1968) J. Physiol. 195, 215-244. Hubel, D. (I989) in: Eye, Brain, and Vision. Scientific American Library, New York. Das, A. and Gilbert, C. (1997) Nature 387, 594-598. Ernst, U., Pawelzik, K., Tsodyks, M. and Sejnowski, T. (1999) Neur. Comp. 11, 375-379. Bonhoeffer, T. and Grinvald, A. (1986) Nature 353, 429-431. Blasdel, G. and Salama, G. (1986) Nature 321, 579-585. Shmuel, A. and Grinvald, A. (1996) J. Neurosci. 16, 6945-6964. Ferster, D., Chung, S. and Wheat, H. (1996) Nature 380, 249- 252. Ben-Yishai, R., Bar-Or, R. and Sompolinsky, H. (1995) PNAS 92, 3844-3848. Somers, D., Nelson, S. and Sur, M. (1995) J. Neurosci. 15, 5449-5465. Obermayer, K., Ritter, H. and Schulten, K. (1990) PNAS 87, 8345-8349. Wolf, F. and Geisel, T. (1998) Nature 395, 73-78. Scherf, O., Pawelzik, K., Wolf, F. and Geisel, T. (1999) Phys. Rev. E 59, 6977-6993. G6decke, I. and Bonhoeffer, T. (1996) Nature 379, 251-254. G6decke, I., Kim, D.-S., Bonhoeffer, T. and Singer, W. (1997) Europ. J. Neurosci. 9, 1754-1762. Crair, M., Gillespie, D. and Stryker, M. (1998) Science 279, 566-570. Wolf, F., Bauer, H.-U., Pawelzik, K. and Geisel, T. (1996) Nature 382, 306. Wilson, H. and Cowan, J. (1972) Biophys. J. 12, 1-24. Prince, D. and Huguenard, J. (1988) in: Neurobiology of Neocortex, eds P. Rakic and W. Singer. pp. 153-176, Wiley Chichester, New York, Brisbane, Toronto, Singapore. 21. Bethge, M., Pawelzik, K. and Geisel, T. (1999) Neurocomputing 26-27, 1-7. 22. Lin, J., Pawelzik, K., Ernst, U. and Sejnowski, T. (1998) Network 9, 333-344.

1000

23. 24. 25. 26. 27. 28. 29. 30. 31. 32. 33. 34. 35. 36. 37. 38. 39. 40. 41. 42. 43. 44. 45. 46. 47. 48. 49. 50. 51. 52. 53.

U. Ernst et al.

Wilson, H. and Cowan, J. (1973) Biol. Cyb. (Kybernetik) 13, 55-80. Mitchison, G. and Crick, F. (1982) PNAS 79, 3661-3665. Malach, R., Amir, Y., Harel, M. and Grinvald, A. (1993) PNAS 90, 10469-10473. Hirsch, J. and Gilbert, C. (1991) J. Neurosci. 11, 1800-1809. Grinvald, A., Lieke, E., Frostig, R. and Hildesheim, R. (1994) J. Neurosci. 14, 2545-2568. Schmidt, K., Kim, D.-S., Singer, W., Bonhoeffer, T. and L6wel, S. (1997) J. Neurosci. 17, 5480-5492. Weliky, M., Kandler, K., Fitzpatrick, D. and Katz, L. (1995) Neuron 15, 541-552. Gilbert, C. (1996) Cur. Op. Neurobiol. 6, 269-274. Das, A. and Gilbert, C. (1995) Nature 375, 780-784. Dalva, M. and Katz, L. (1994) Science 265, 255-258. Kisv~irday, Z., T6th, E., Rausch, M. and Eysel, U. (1997) Cereb. Cortex 7, 605-618. Tsodyks, M. and Sejnowski, T. (1995) Int. J. Neural Syst. 6, 81-86. Amari, S. (1977) Biol. Cyb. 27, 77-87. Amit, D., Brunel, N. and Tsodyks, M. (1994) J. Neurosci. 14, 6435--6445. Ben-Yishai, R., Hansel, D. and Sompolinsky, H. (1997) J. Comp. Neurosci. 4, 57-77. Troyer, T., Krukowski, A., Priebe, N. and Miller, K. (1998) J. Neurosci. 18, 5908-5927. Ferster, D. and Miller, K. (2000) Neural mechanisms of orientation selectivity in the visual cortex, Ann. Rev. Neurosci. 23. Yantis, S. and Nakama, T. (1999) Nature Neurosci. 1, 508-512. Rojer, A. and Schwartz, E. (1990) Biol. Cyb. 62, 381-391. W6rg6tter, F. and Niebur, E. (1993) Biol. Cyb. 70, 1-13. Tsodyks, M. and Sejnowski, T. (1995) Network 6, 111-124. Livingstone, M. (1998) Neuron 20, 509-526. Mineiro, P. and Zipser, D. (1998) Neur. Comp. 2, 353-371. Single, S., Haag, J. and Borst, A. (1997) J. Neurosci. 17, 6023-6030. Suarez, H., Koch, C. and Douglas, R. (1995) J. Neurosci. 15, 6700-6719. Swindale, N., Matsubara, J. and Cynader, M. (1987) J. Neurosci. 7, 1414-1427. Maffei, L. and Galli-Resta, L. (1990) PNAS 87, 2861-2864. Gilbert, C. and Wiesel, T. (1989) J. Neurosci. 9, 2432-2442. L6wel, S. and Singer, W. (1992) Science 255, 209-212. Reid, R. and Alonso, J.-M. (1995) Nature 378, 281-284. L6wel, S. and Singer, W. (1998) in: Perceptual Learning, eds M. Fahle and T. Poggio. MIT press, Boston. (in press).