Dynamical computational properties of local cortical networks for visual and motor processing: A bayesian framework

J Physiology (Paris) ( 1996) 90.251-262 OElsevier, Paris

Dynamical computational properties of local cortical networks for visual and motor processing: A bayesian framework E Koechlin, JL Anton, Y Burned* INSERM-CREARE, Universite Pierre-et-Marie-Curie, 9, quai St-Bernard. 75005 Paris. France

Summary - A major unsolved question concerns the interaction between the coding of information in the cortex and the collective neural operations (such as perceptual grouping, mental rotation) that can be performed on this information. A key property of the local networks in the cerebral cortex is to combine thalamocortical or feedforward information with horizontal cortico-cortical connections. Among different types of neural networks compatible with the known functional and architectural properties of the cortex, we show that there exist interesting bayesian solutions resulting in an optimal collective decision made by the neuronal population. We suggest that thatamo-cortical and corticocortical synaptic plasticity can be differentially modulated to optimize this collective bayesian decision process. We take two examples of cortical dynamics, one for perceptual grouping in MT, and the other one for mental rotation in Ml. We show that a neural implementation of the bayesian principle is both computationally efficient to perform these tasks and consistent with the experimental data on the related neuronal activities. A major implication is that a similar collective decision mechanism should exist in different cortical regions due to the similarity of the cortical functional architecture. computation

/ neocortex I bayesian probability

I decision process / feedforward

Introduction Neurons in sensory cortical areas respond to a specific attribute of a stimulus when it is presented in the receptive field of each neuron. This leads to the notion of a cortical map, which is an isomorphic representation of the cortesponding physical space. The main property of cortical maps is that they encode in parallel local attributes (for example shape, color, movement) by the frequency of discharge of each neuron of the map. Nevertheless, this type of coding does not make it easy to understand interaction with the cognitive processes influencing it such as perceptual grouping or mental rotation. It has been proposed that mental operation such as feature binding could be reflected in the synchronous activity of populations of neurons during oscillations observed in the primary visual cortex (Singer, 1993). Several models have been proposed in which neurons synchronously firing are assumed to respond in coherence with a same global attribute (Von der Maslburg and Bienenstock, 1986; Crick and Koch, 1990). They assume that local information is encoded by the frequency of discharge of individual neurons, while global information resulting from perceptual grouping is implemented by synchronous fling, different perceptions being differentiated by the phase of discharge. However, it has also been proposed that the cortex could use a different strategy, the population vector, to encode sensorial, motor, and sensorimotor representation (Georgopolous et al, 1986, 1989; Caminiti et al, 1991). Sensorial and motor features can be repre*Correspondence

and reprints

connectivity

/ visual processing

sented by directions in vectorial spaces. A neuron has a preferred direction, and a unitary vector in this direction is associated with each neuron. The sum of these individual, unitary vectors, weighted by the discharge frequency of the corresponding cell, is the population vector, whose direction represents the global percept or the movement to be performed. Some experiments suggest that the population vector dynamics reflects the underlying cortical operations, such as a mental rotation. Recordings from the motor cortex of monkeys performing mental rotation tasks have shown that the population vector rotates before the onset of the movement, during the mental rotation (Georgopoulos et al, 1989; Lurito et al, 1991). other experiments and related models have shown that it is necessary to take into account not only the dynamics of the population vector, but also the dynamics of the population of vectors to understand the underlying visuo-motor operations (Caminiti et al, 1991; Bumod et al, 1992). We propose to use a more general framework, the population bayesian decision, which provides a simple and general relation between the coding of information and the collective neural operations (grouping, rotation, coordinate transformation, etc) that can be performed on this information.

The first hypothesis is that a collective hayesian decision process is performed by a cortical neuron population A major property of the local networks in the cerebral cortex is to combine feedforward information (as stressed by the population vector scheme) with hori-

258

zontal (or lateral) cortico-cortical connections (as stressed by synchronous firing schemes). Interactions between both types of connections have specific properties (see for example models in Heeger, 1993; Douglas et al, 199.5; Somers et al, 1995; Bringuier and Fregnac, 1996): i) the excitatory cortico-cortical connections have a gating effect on feedforward inputs; ii) there exists an excitatory-inhibitory balance in lateral interactions between cortical pyramidal neurons, with a flat lateral inhibition compared to a tuned lateral excitation; and iii) inhibition has a divisive effect, resulting in a normalization. Among different types of neural networks compatible with these functional and architectural properties, there exist interesting bayesian solutions resulting in an optimal collective decision by the neuronal population (see Apperadix). In essence, such a bayesian property means that a neuron responds to its feedforward input only if it received prior lateral inputs, corresponding to an expectation from other neurons in the population. After few iterations. the population activity represents the collective response to the stimulus which is the most compatible with the knowledge encoded in the set of synaptic weights.

The second hypothesis is that synaptic plasticity is modulated to optimize the collective bayesian decision process It is possible to demonstrate in such a bayesian process that the summed activity in the population has a major significance which can be compared to the energy function in artificial neural networks. When the feedforward inputs are maintained constant, this parameter converges towards the maximal eigenvalue of a matrix formed by the product of feedforward inputs and lateral cortico-cortical weights, and the population activity converges towards a related eigenvector. Thus, the asymptotic summed activity reflects the optimal coherence between the feedforward inputs and the lateral cortico-cortical weight distributions. A bayesian learning rule is therefore to maximize this pammeter by the activity-dependent plasticity of feedforward and lateral weights. Approximations of the two leaming rules, on feedforward and lateral connections, can be obtained by using a standard gradient descent algorithm.

The adaptive bayesian process is computationally powerful for a large variety of sensory, motor, and cognitive operations The computational efficacy of the bayesian decision can be demonstrated on natural moving images. For

example, extracting objects from background is not an easy task for a neural net, since the local computations of velocity require the presence of local features, and are biased by the orientations of these local features due to the aperture problem. Figure 1A represents an image with three different objects, moving as a result of both their own movements and the movement of the camera. Figure IB shows the results of local computation of motions by a neural network with only feedforward connections. There is a high variability due to the fact that the local computation is biased by the orientations of the local features (aperture problem). Figure 1C illustrates the efficacy of a map of processing units combining feedforward and lateral interactions with a bayesian logic. The three objects and their movement emerge after few iterations from the collective bayesian decision process.

The collective bayesian decision process results in a dual coding principle With the underlying bayesian model, it is possible to decompose neuronal activity in a given cortical area in two factors: I) representation of information; and 2) mental operation on this representation. The distribution of cell activity encodes attribute information, while the summed activity reflects the significance or pertinence of the encoded attribute in direct relation with the mental operation performed by the cerebral cortex (Koechlin and Burnod, 1996). The bayesian framework extends the population vector and synchronous firing coding schemes. The direction of the population vector appears as a simplification of the distribution of activities in the population, and its norm as a measure of the summed activity. Synchronous firing appears as a particular case of coherence between the feedforward inputs and the lateral cortico-cortical weight distributions.

A major implication is that a similar collective decision mechanism should exist in different cortical regions due to the similarity of the cortical functional architecture The collective neuronal dynamics resulting from the local bayesian process can be compared with available experimental data on neuronal activities in different cortical areas. As for any other neural net, this results from an activation rule (the local bayesian computation), and a learning rule (the corresponding learning scheme). The distributions of input activities and sy-

Fig 1. Examples of perceptual grouping on natural moving images. A. Image with three moving overlapping objects. B. Motion field extracted by a map of motion filter units with only fredforward connections. Arrows indicate population velocity kectors for each location in the image. No object emerges. C. The bame map of motion filter units with additional lateral connections implementing the bayeaian procesh described in the Appemlix. The extracted motion field now \egments the three moving objects quite well.

naptic weights should depend upon each cortical region, but the collective dynamical patterns should mainly depend upon the bayesian logic of the task. For example, one can predict that when a stimulus appears, the dynamical patterns will depend upon the previous population activity only if the stimulus is ambiguous. We have shown that several predictions of the collective bayesian decision are confirmed by experimental results on neuronal activities in different cortical areas, primary motor (Ml), visual (MT), or associative (IT) (Koechlin and Bumod, in preparation). We will give here two examples on two apparently different operations which have been analyzed in behaving monkeys at the cellular and psychophysical level: 1) extraction

of coherent motion in the VI-MT pathway: 2) mental rotation in the motor areas. Exatnple

I: E.rtraction

qf coherent

ttlotion

The same local neural process which was efficient to process natural moving images (as shown in fig 1) can also fit neural activities in the V 1-MT motion pathways and the related psychophysics (Koechlin et al, submitted). Dynamic random dot stimuli have been used to study the link between neuronal responses in MT and the perceptual detection thresholds of a coherent motion. Experiments in which monkeys were required to

260

identify the direction of coherent motion in random dot displays revealed that the responses of single neurons in MT closely matched the psychometric functions obtained when the percentage of dots moving coherently was varied (Newsome er al, 1989; Britten et al, 1992). Examples of stimuli used by these authors are shown in figure 2A. In these experiments, a two alternative forced-choice paradigm was used and monkeys were required to report whether the coherent motion was left or rightward. Monkeys were found to be able to reliably report the correct motion even when the percentage of motion correlated dots was less than 10%. For each neuron, a ‘neurometric’ function was calculated representing the probability to infer a correct response from the neuron activity and was found for certain neurons to closely match the related psychometric functions. Furthermore, the proportion of correct response increased, and the neuronal and behavioral detection thresholds decreased, when the stimulation duration was increased. The neural net implementing the local bayesian process extracts coherent motion in similar dynamic random dot stimuli as monkeys do. The response of the neural network was determined by calculating the proportion of the activity supported by the units coding for the direction of the coherent motion. This proportion represents the probability to produce a correct response on the basis of the activity of the processing units in an ideal two-way forced choice paradigm. Figure 2B shows how this proportion varies when the percentage of correlated dots is increased. The general shape of the response curve and detection threshold are consistent with the neuronal and behavioral thresholds reported by Britten et al (1992). Furthermore, as these authors found, the proportion of correct response increased, and the neuronal and behavioral detection thresholds decreased when the stimulation duration was increased (‘rig 20 This temporal phenomenon is explained by the bayesian property of remanence. Since the feedforward motion signals produced by the incoherently moving dots are ambiguous, the responses of the processing units to these signals are strongly biased by their prior responses. Only the coherently moving dots produces non-ambiguous feedforward local motion signals. When the stimulation time increases, there are more and more locations which are temporary occupied by coherently moving dots. Consequently, the unit responses to incoherently moving dots are more and more biased in favor of the coherent motion when the stimulation time increases. Example

2: Mental

rotatim

The same process that results in perceptual grouping can generate apparently different operations such as

-A

B l-

099

-

1

a1

10

Coherence

100

level (%)

C

o,g 10% 038

5%

0,7

036 20

I

I

I

I

1

40

60

80

100

120

Stimulation time (iterations) Fig 2. Cumulative extraction of coherent motion by populations of neurons implementing a bayesian process. A. Random dot stimuli similar to those used by Britten et al (1992) with different percentages of correlated dots indicating a coherent motion. B. Proportion of correct responses produced by the model for different percentages of correlated dots. This parameter represents the proportion of neuronal activity in the populations related to the coherent motion direction compared to the opposite direction. The performance of the model closely matches the data of B&ten et al (1992). C. Time course of model performances for two percentages of correlated dots. Consistently with experimental data (Britten et al, 1992), the proportion of correct detection of coherent motion increases with the duration of the stimulation.

261

Time Fig 3. Model dynamics of population activities in mental rotation. Top. The plot shows the time course of neuron activities in the modeled population when the direction coded in the input shifts from 45 degrees to 135 degrees. Front axis, neurons indexed by their preferred direction; depth axis. processing time: vertical axis, neuronal activity. The population activity moves continuously from the initial to the final input direction. Bottom. Rotation of the related population vector. Note the increase of the norm during the rotation.

ment onset, from its initial direction towards the movement direction; ii) the norm, or summed population activity, increases in parallel with the input onset; and iii) the activities of neurons with intermediate preferred directions increase, and then decrease during the rotation process, A straightforward prediction of the bayesian process is that the spatio-temporal pattern of activities is critically dependent upon the shape of the input distribution, and can shift more abruptly when this distribution becomes narrower. Experimental results on dynamical activities in the cortex can be modeled with other types of neural networks (Amari, 1977; Amit et al, 1994), since these models which are underconstrained show equivalent properties. So we do not claim that the population bayesian decision model is the best, but we suggest that this framework has the advantage to be both general and operational to analyze population activities in relation with the behavioral task. It can help to interpret collective dynamical properties of cortical neurons, whatever the cortical area, in relation with the bayesian logic of the task. It is based on a single and general hypothesis on the interaction between feedforward and lateral cortico-cortical connections, which is compatible with existing models of the local cortical circuitry in the primary visual cortex.

Appendix A bayesian process is implemented the following activation rule: X,

mental rotation. Furthermore, the resulting dynamical patterns fit experimental data obtained on neuronal activities in the primary motor area during a mental rotation which is used to execute a movement a certain angle away from the direction indicated by a visual stimulus. In such mental rotation tasks, monkeys are required to infer the appropriate direction for movement which is not directly given by the target itself, but which is rotated by a constant angle away from the visual stimulus. The neuronal dynamics in the model result from the lateral weights distribution. The selforganization with the bayesian learning rules encodes a cosine-like correlation between the neuron’s preferred directions, as observed in Ml (Georgopoulos et al, 1993). The processing unit activities then reproduce experimental observations in area Ml during the rotation tasks (fig 3): i) the population vector, ie the vector sum of the neuron’s preferred directions weighted by their activity, rotates, after the stimulus and before the move-

(?I + 1) =

Si

C

in discrete time by

ri, Xi (n)/C

-U,(n)

where xi(n) is the activity of the neuron i in the population, Si the related feedforward input, and rv the lateral connection weight from neuron j to neuron i. In continuous time, an equivalent process is given by the following equation:

$ =s, C i

rii x, (t) -xi

(t)

C

x, (t)

I

These activation rules derive from the assumptions: 1) that the probability, that neuron i is active, is related to the probability that the input Si occurs conditionally to the prior activity in the population; and 2) that the lateral weights encode such statistical dependencies.

References Amari S (1977) Dynamics of pattern formation type neural fields. Bid Cybem 21. 71-87

in lateral-inhibition

262

Amit DJ, Brunei D, Tsodyks MV (1993) Correlationa of cortical hebbian reverberations: Theory 1’s experiment. J Ne~rosci 14. 6435-6445

Georgopoulos AP, Taira M, Luhashin A (1993) Cognitive neurophysiology of the motor cortex. Sciencr 260. 37-51

Britten KH, Shadlen MN. Nrwsome WT. Movshon JA ( 1992) The analysis of visual motion: A comparison of neuronal and psychophysical performance. J Nrzrrosci 12, 3745-4765

Heeger DJ (1993) Modeling normalized. half-squared 1885-1898

Bumod Y. Grandguillaume P. Otto I. Ferraina S. Johnson PB. Caminiti R ( 1992) Visuomotor transformations underlying arm movements toward visual targets: a neural network model of cerebral cortical operationa. J Neurosc~i I?. 1435-I 453

Koechlin E. Burnod Y (1996) Dud] population coding in the neocortex. A model of interaction between representation and attention in the visual cortex. J Cogr~ Neurosci 8. 383-399

Caminiti R, Johnson PB. Galli C, Ferraina S. Burnod Y ( 1991) Making arm movements within different parts of space: the premotor and motor cortical representation of a coordinate system for reaching to visual targets. .J Nr~rosci I I, 19X-31 I Crick F. Koch C (1990) Towards a neurobiological sciousness. Srr7lirl Neuvosci 2, 263-275

simple-cell direction selectivity with linear operator. J N&troph,v.sio/ 70,

Lurito JT, Georgakopoulos T, Georgopoulos AP (1991) Cognitive spatial motor processes. 7. The making of arm movements at an angle from a stimulus direction: Studies of motor cortical activity at the single cell and population level. E.rp Brain Rr.s 87. 56’

theory of con-

Douglas RJ. Koch C, Mahowald M, Martin KAC, Suarez HH ( 1995) Recurrent excitation in neocortical circuit. S&IIL.C 269. 981-985 Fregnac Y, Bringuier V ( 1996) Spatio-temporal dynamics of synaptic integration in cat visual cortical receptive fields. /,I: Biolo~ic~ol Unsic ~rx/ Corllp”tc”i”““/ TIluo~ r!f’ Visiorr (Aertscn A, Braitcnberg V. eds) Elsevier, Amsterdam. 143-199 Georgopoulos AR Schwartz AB, Kettner RE (1986) Neuronal population coding of movement direction. &?rr~c,r 133. 1416-1419 Georgopoulos AP. Lurito JT, Petrides M. Schwarttr AB. Massey JT ( 1989) Mental rotation of the neuronal population vector. Scirncc 213. 734

Newsanr WT. Brittcn KN. Morahan WT (1989) Neural correlates of a perceptual decision. N&rre 341. 51 Singer W ( 1993) Synchronization or cortical activity and its putative role in int’ormation processing and learning. Anru~ Rap Pl~~siol 5s. 349-374 Somers DC, Nelson SB. Sur M ( 1995) An emergent model of orientation selectivity in cat visual cortical simple cells. J Neumsci 15. 5438 Von der Maslburg C, Bienenstock E ( 1986) Statistical coding and shot?term synaptic plasticity: A scheme for knowledge representation in the brain. Nero Srn’r. vol FX. 137-272