The superior colliculus and spatiotemporal translation in the saccadic system

Neural Networks, Vol, 3, pp. 75 86, 1990

()g93-6080/90 $3.00 + .00 Copyright ,~ 1990 Pergamon Press plc

Printed in the USA. All rights reserved.

O R I G I N A L CONTRIB UTION

The Superior Colliculus and Spatiotemporal Translation in the Saccadic System DOUGLAS B. TWEED AND TUTIS VILIS University of Western Ontario

(Received 16 December 1988; revised and accepted 19 July 1989)

Abstract--The superior colliculus (SC) plays an important part in generating saccadic eye movements, sending signals coding desired eye rotation to the brainstem. These signals must be translated from the topographic (spatial) representation used in the SC to the firing frequency (temporal) code used downstream. We show that a model of the saccadic system using the quaternion representation of eye rotations yields a spatiotemporal translation with all the experimentally observed properties: activation of a particular site in the SC generates a saccade of a particular amplitude and direction; activation of multiple sites evokes a vector average (weighted by activity levels) of the saccades coded by the individual sites; the intensity and temporal profile of activation determine saccade speed but not metrics. The feature of the model that is essential to these results is a particular sort of redundancy in the quaternion representation, coupled with multiplicative downstream handling of SC outputs.

Keywords--Eye movements, Listing's law, Quaternions, Saccades, Spatiotemporal translation, Superior colliculus, Temporal coding, Topographic coding. back model developed by Robinson and coworkers (van Gisbergen, Robinson, & Gielen, 1981b), saccades are generated by a brain stem feedback circuit which is driven by an input coding the desired position of the eye in the head. But the signals descending from the SC code, not desired eye position, but desired eye displacement (Robinson, 1972; Schiller & Stryker, 1972; Sparks & Mays, 1980). The second problem is that the SC and downstream centres code saccade-related information in different forms: downstream structures code most information temporally, in the firing frequencies of neurons; while in the SC most information is coded spatially (topographically), by the locations of the active neurons. For example, motoneurons employ temporal coding: their firing rates correlate with eye position. In contrast, the size and direction of a saccade evoked by a single site in the SC depends entirely on the location of the site, and not on the stimulus intensity (Robinson, 1972; Schiller & Stryker, 1972). Neither SC nor downstream structures employ one method of coding exclusively: among burst and tonic neurons, some information, namely the direction of eye velocity and eye displacement, is coded spatially, by the ratios of activity among different cell populations; and saccade speed (Berthoz, Grantyn, & Droulez, 1986; Munoz & Guitton, 1987; Rohrer, White, & Sparks, 1987), is coded by

1. INTRODUCTION

There is abundant evidence that the superior colliculus (SC), a nucleus in the dorsal midbrain, plays an important part in generating the rapid gaze shifts called sacades. In cats and monkeys, stimulation of the SC evokes saccades (Robinson, 1972; Schiller & Stryker, 1972), lesions impair saccade generation (Hikosaka & Wurtz, 1985, 1986), and anatomical tracers reveal projections to known saccadic centres (Harting, 1977). But the actual role of the SC in saccade generation is unclear because the experimental data, especially on stimulation-evoked saccades, do not fit well with current models for the saccadic system. We shall deal with two problems which must be solved by any unified model of the SC and downstream saccadic centres. The first problem is that the SC does not appear to provide a suitable input to the downstream centres. According to the local feed-

Acknowledgements--We thank L. van Cleeff for drawing the figures. This study was supported by the Medical Research Council of Canada Grant MT9335. D. Tweed is a Fellow of the Medical Research Council. T. Vilis is a Medical Research Council Scientist. Requests for reprints should be sent to Douglas Tweed, Department of Physiology, University of Western Ontario, London, Ontario N6A 5C1, Canada.

75

76

D. B. 'Fweed and 7". Vitis

firing frequency in the SC. However, the important point is that some information, namely saccade size, changes form between the SC and the downstream structures: it is translated from spatial to temporal code. A crucial question regarding this translation is: How does a higher activity level in the SC produce a faster (Munoz & Guitton, 1987) but not a larger saccade (Robinson, 1972)? Most SC models (e.g., Keller, 1980; Scudder, 1988) incorrectly predict that stronger stimulation should yield larger saccades. Other models (du Lac & Knudsen, 1987; Tweed & Vilis, 1985), have been devised to solve this puzzle, with partial success. The main result of the present paper is that if we model the saccadic system in three dimensions, using a four-component quaternion representation for eye position (described in Tweed & Vilis, 1987), then a spatiotemporal translation with the observed properties emerges automatically.

dl

02

2. PROPERTIES OF THE SUPERIOR COLLICULUS tO ~

2.1. Topographic Coding The deep layer of the SC is a topographic map of saccades. Electrical stimulation of this layer evokes saccades whose amplitudes and directions depend on the site stimulated, but not on other stimulus parameters such as intensity, frequency, and duration, as long as these exceed certain threshold values (Robinson, 1972; Schiller & Stryker, 1972). Sites coding different saccades are distributed in an orderly map across the SC. Saccade-related burst neurons in the deep layer fire with stronger bursts for saccades nearer the cells' optimal amplitudes and directions (Sparks & Mays, 1980). Saccades evoked by stimulation are also largely independent of eye position except at the edges of the oculomotor range.

2.2. Weighted Vector Averages Stimulation of one SC site evokes a saccade whose metrics are independent of stimulus intensity, but when two sites are stimulated simultaneously, the evoked saccade depends on the stimulus intensities at both sites. Looking at the horizontal and vertical coponents of saccades using search coils, Robinson (1972) observed that the saccades evoked by twosite stimulation are weighted vector averages; that is, if site~ coding displacement d~ is stimulated with intensity il, and site2 coding d2 with intensity i2, the resulting displacement is approximately (ildl +/~d2)/ (il q- i2) (Figure 1). The intensity of neural activity at a SC site also influences saccade dynamics:

15°

circled "2": (d, + 3~)t(1 + 3)--

6°upend

right.

stronger stimulation of the SC yields faster (though not larger) saccades (Munoz & Guitton. 1987); saccade-related burst neurons in the SC fire harder for faster saccades of the same size (Rohrer et al., 1987); and in owls, saccade-like head movements evoked by two-site stimulation can be faster than normal movements of the same size and direction (du Lac & Knudsen. 1987). In summary, the SCappears to code desired eye displacements by "hills" of activity whose locations determine saccade amplitude and direction and whose heightsmthe firing frequencies of ceils in the active patch and breadths determine saccade speed. The advantage of this arrangement is that the level and spread of activity in the SC need not be precisely controlled, because stronger excitation will merely produce a faster movement and vector averaging of a large active patch will not produce a larger movement.

3. THE SACCADE GENERATOR 3.1. Local Feedback Medei The SC projects to regions of the brain stem that are known to participate in saccade generation, To model these downstream circuitS, we shall adapt the local feedback model devised b y Robinson ~ d coworkers to describe the horizontal component of the system (van Gisbergen et al., 1981b). In thatmodel,

Superior Colliculus and Saccades

77 H

FIGURE 2. One-dimensional local feedback model for saccedes. The oculomotor integrator acts on the horizontal eye velocity command H from the horizontal short lead burst neurons to yield an internal estimate of current horizontal eye position H. This position signal goes to the motoneurons, but is also fed back and subtracted from a signal coding the desired position of the eye, H*, to yield an error signal E which drives the short lead burst neurons until the eye is on target. Burst neurons are represented by a roughly sigmoid curve to indicate that their velocity commands are a nonlinear function of the error signal.

eye velocity commands originating from short lead burst neurons in the pons reach the extraocular muscle motoneurons both directly and through an integrator which provides the eye position signal required to hold the eye steady after the saccade is over (Figure 2). There is persuasive evidence that the saccade generator, consisting of burst neurons and integrator, is feedback-driven (e.g., Becker et al., 1981). In the local feedback model the output of the oculomotor integrator, coding current eye position, is subtracted from a signal coding desired eye position to yield a motor error estimate that drives the burst neurons until the target is reached (Figure 2).

the comparator. But plots of horizontal saccades in the absence of the integrator show normal-looking short durations.

4. SC MODELS 4.1. Displacement Feedback Model Several writers (including Hepp & Henn, 1983; Jurgens, Becker, & Kornhuber, 1981; Sparks, Mays, & Porter, 1987; Tweed & Vilis, 1985) have proposed that the signal from the SC, AH*, coding desired eye displacement (horizontal displacement in the onedimensional model in Figure 3a), provides the input to the comparator in the saccade generator. To be consistent with this input, the feedback signal is not H, eye position, but AH, the horizontal displacement of the eye since the beginning of the current saccade, which is computed by integrating the velocity signal from the short lead burst neurons, and resetting the integrator in the feedback loop to zero after every saccade. The motor error signal is the difference between desired and actual displacement.

4.2. Scudder Model In Scudder's model (1988) (Figure 3b), the output of the SC is a frequency-coded signal which, when

AH"m~

3.2. Eye Position or Eye Displacement Feedback? There is some tension between the local feedback model and the known properties of the SC. The findings that the saccade-related burst neurons in the SC have latencies about 8 ms longer than the pontine and mesencephalic short lead burst cells, that the SC projects to regions containing short lead burst cells, and that SC lesions impair saccade generation all suggest that the SC is an important input to the saccade generator. But the SC provides a desired displacement signal while according to the local feedback model the saccade generator needs a desired craniotopic eye position signal. Evidence that the input to the comparator of the saccadic system is actually a desired displacement signal from the SC is reviewed in Tweed and Vilis (1985). If the input is desired displacement and not desired position, the feedback signal cannot be an actual eye position signal from the oculomotor integrator. This idea is supported by experiments in which the integrator of the horizontal system was completely deactivated (Cannon & Robinson, 1987). In this situation, the local feedback model would predict very long-duration horizontal saccades, because the abolished horizontal eye position signal from the integrator would never match the desired position signal coming to

E [7-I reset

(a)

-I

•

AH

i~ I

H

(b) FIGURE 3. One-dimensional SC models. (a) Displacement feedback model. The input to the comparator is a signal from the SC coding the desired horizontal rotation of the eye AH*, in frequency form. The feedback signal codes AH, the displacement of the eye since the start of the saccade, which is computed by integrating the velocity signal from the burst cells and ~esetting the integrator to zero between saccades. (b) Scudder model. The velocity command from the burst cells is subtracted from a SC signal whose integral is the desired eye displacement. The difference is integrated to yield the error signal that drives the burst neurons. In both these models, more spikes coming from the SC will result in a larger saccade.

78

D. 13, Tweed and T. Vili~

integrated over its entire duration, yields the desired displacement of the eye. A velocity signal from the short lead burst neurons is subtracted from the SC output, and the difference is integrated to yield the error signal that drives the saccade. The saccade continues until the integral of burst neuron activity equals the total integral of SC output: the desired displacement. The model can be viewed as a variation on the displacement model, obtained by switching the order of integration and subtraction. 4.3. Flaw in Current Models The models in Figures 3a and 3b share a serious flaw: they posit that signals from the SC represent desired displacement in frequency or numbers of spikes; both models therefore predict that greater SC activity should yield larger saccades. But in fact, stimulation at a given SC locus consistently evokes normal saccades of the same amplitude and direction, regardless of whether the stimulation is of high or low intensity, of brief duration, or continued for the duration of five saccades. Surely these different electrode-induced firing patterns do not all have the same frequencies or total integrals. Thus neither model offers a mechanism for weighted averaging or for how stronger stimulation yields faster but not larger saccades. 5. T H R E E - D I ~ S I O N A L SACCADE GENERATOR The above problems can be solved by shifting to a three-dimensional view of oculomotor control. As a first step, we shall review our recent three-dimensional local feedback model of the saccade generator (Tweed & Vilis, 1987), which represents eye position using a four-component rotational operator called a quaternion (first applied to eye position by Westheimer, 1957). There are many ways to represent angular position and velocity as sets of real numbers (i.e., as frequency modulation on several channels) but as we have argued (Tweed & Vilis, 1987), the simplest models are obtained using quaternions and angular velocity vectors. For mathematical background see the Appendix, and also Tweed and Vilis (1987), Tait (1890), Brand (1948). A quaternion model of the saccade generator that preserves the structure of Robinson's one-dimensional model (Figure 2) is illustrated in Figure 4. At the centre of the model, three pools of short lead burst neurons--horizontal in the pons (van Gisbergen et al., 1981b) and two nonhorizontal in the midbrain (Hepp et al., 1988; King & Fuchs, 1979), all represented by a single box in the flow diagram-code the three components of the eye velocity command. An important difference from the one-di-

q"

q

()

~RE 4. T h r e e - d i ~ cadre. The o /2 from

the ~

local ~

reimS1 vector of ~

sltion Ilignai q. A ~

~

model for u c -

burm ~

is mui~ti~! W eye

eye ~

d ~ by q to ~ ~quaternion E ~ part E drives the burst oells.

q* is ~r

mensional model is that the velocity command is multiplied by a feedback signal coding eye position q (a quaternion) and the product is integrated. The position signal q is also fed back to the comparator of the saccadic feedback system, where desired eye position q* is divided by q to yield the motor error quaternion E. The error is the quotient of q* and q, not their difference, because composition of rotations is multiplicative, not additive. The last three components of E--called the vector part E--pass through a nonlinearity to yield the burst neuron signal o)/2 (one half the angular vel~ity of the eye). The saccade stops when the error quaternion E equals 1, which is the quaternion representing no displacement, or in this case no motor error. Looking at this another way, when E = t, then the error vector E = 0, and so the velocity vector to also becomes 0 and the eye comes to a halt.

6. THREE-DIMENSIONAL SC MODEL The quaternion model in Figure 4 shares an important flaw with Robinson's one-dimensional model: the feedback signal is eye position and the circuit therefore requires a desired eye position input, whereas the SC actually provides a desired displacement signal. One solution is a three-dimensional version of the displacement feedback model discussed in section 4.1. 6.1. 1 1 u ' e ~ ~ Feedback Model

Displacement

Assuming that each site in the SC codes a particular eye rotation independent of eye position when the site is activated, we shall regard each site as the trait quaternion r* representing that rotation.We propose that each output cell sends a desired displacement (i.e.. rotation) signal to the saccade generator, coded on four channels (axon branches) with relative strengths r~ : r~ : r2*-• r 3*. That is, the overall strengths of the synaptic connections where these four axon branches contact the generator are in the ratios

79

Superior Colliculus and Saccades reset

FIGURE 5. Three-dimensional displacement feedback model. Input to the multlpllcatlve compsrator Is a signal from the SC coding the desired rotation of the eye r*. The feedback signal codes r, the rotation of the eye since the start of the saccade.

shown. Thus when the cell is firing with frequency f, the signal to the generator is the quaternion f x (r~', r~', r~', r~'). Figure 5 is the flow diagram for a three-dimensional displacement feedback model. At the left, the desired total displacement quaternion r* from the SC is divided by a feedback signal r, the quaternion for the displacement so far in the saccade, to yield the error signal E. The displacement quaternion r cannot be computed from ~ (or//) with just a resetting integrator. Rather, o~/2 must be multiplied, prior to integration, by a feedback signal r coding displacement so far. 6.2. Properties of the Model

We saw that the one-dimensional displacement feedback model in Figure 3a incorrectly predicted larger (rather than just faster) saccades with stronger SC stimulation, and saccade summation (rather than averaging) when two SC sites are stimulated simultan e o u s l y . S u r p r i s i n g l y , its t h r e e - d i m e n s i o n a l counterpart delivers the correct predictions on both counts. The underlying reason is that all nonzero scalar multiples cq of a quaternion q represent the same rotation (see Appendix). Consider first the question of stimulus intensity. We proposed that each output neuron in the SC represents a unit quaternion r*, in the sense that when the cell's firing rate is f , the signals conveyed to the saccade generator by the four channels are f x r~', f × r~', f × r~' and f x r~'. Thus the output from the site is always f x r*, for some scalar f, and so weak or powerful stimulation, for a short or long duration, will always code a scalar multiple of r*; and since all scalar multiples of r* represent the same saccade displacement, the saccade amplitude and direction will always be the same. To see how this works, suppose that the SC locus coding the rotation r* is stimulated with double the "normal" intensity, so that the firing frequencies evoked in the generator by its four output channels are the components of 2r*. Since error computation is mutiplicative in this model, the result is a doubled

error signal. When the feedback signal r indicates that the eye has undergone the rotation r*, the error quaternion is 2r'r* - 1, which is 2 - - a pure scalar quaternion with no vector part. The error vector is therefore 0, and the eye stops after exactly the same rotation (r*) as it would have made, had the SC signal been r* or any other scalar multiple of that quaternion. We now consider what the model predicts for twosite stimulation, assuming the simplest possible hypothesis: that the displacements coded by the two sites, weighted by the activity levels at the two sites, converge additively on the comparator. Suppose one site coding the quaternion p* is stimulated with intensity f and another site coding r* is stimulated with intensity g. Then the summated input to the comparator is fp* + gr*. But since any scalar multiple of a quaternion represents the same rotation, this input codes the same displacement as the weighted average (fp* + gr*)/(f + g). Weighted averaging comes free with quaternions. For saccades smaller than 90 ° , the resulting movements would be indistinguishable from the vector-averaged saccades reported by Robinson (Figure 1). For example, suppose the eye is in primary position when two sites coding the rotations p* = 24 ° up and r* = 20 ° right are simultaneously activated with equal intensities (as in Figure 1, case 1). Then (expressing vectors in coordinates (x, y, z) with x forward, y left, z up) the signal reaching the comparator is p* + r* = cos 12° + sin 12°(0, -1, 0) + cos 10° + sin 10°(0, 0, - 1) = .9781 + (0, -.2079, 0) + .9848 + (0, 0, -,1736) = 1.9630 + (0, -.2079, -.1736) = 1.982[.9906 + (0, -.1049, -.0876)] = 1.982[cos 7.86 ° + sin 7.86°(0, -.7675, -.6410)]. (1) Thus the signal to the comparator is a scalar multiple of the unit quaternion representing a 2 x 7.86 ° = 15.71 ° rotation about the axis (0, -.7675, -.6410); that is, a 15.71 ° rotation up and right along the meridian tilted 50.13 ° counterclockwise from directly rightward (see Appendix for an explanation of the quaternion representation of rotations). In comparison, the strict vector average illustrated in Figure 1 is a 15.62 ° rotation along the 50.19 ° meridian. There is an important difference between this quaternion mechanism for weighted averaging, and a mechanism that actually averages SC outputs. In the quaternion model, activity levels are not actually divided out. As we saw, with double intensity stimu-

80

lation the input to the burst generator is 2r*, which yields a doubled error signal 2E which, in turn, produces a larger velocity command in the short lead burst neurons. In this model, then, higher-intensity stimulation of a SC site produces a faster, shorter duration saccade without changing saccade size or direction. And two-site stimulation can yield unusually rapid saccades. As mentioned in section 2, these are the observed results. Note also that the independence of stimulus intensity in single-site results is just a special case of weighted averaging. Moreover, cell recordings show that for natural saccades, the active region of the SC motor map comprises many neurons firing at various frequencies (Sparks & Mays, 1980), so the averaging of multiple active cells is a feature of normal SC functioning.

6.3. Essential Features of the Model

In the SC model of this paper, as in our saccade generator model (Tweed & Vilis, 1987), we have proposed that oeulomotor circuits carry eye position information in quaternion form. But in both cases, the quaternions themselves are not essential to the models, in the sense that the basic mechanisms could be implemented using other representations, as long as these shared certain key properties with quaternions. In the saccade generator model, for example, one essential feature of the quaternion representation was that it facilitates the conversion of velocity signals to position signals. What are the features of quaternions that are essential to vector averaging? Clearly one relevant property is a particular sort of redundancy. The neural representation of eye displacements in the SC is clearly redundant, in that all scalar multiples of a particular signal--that is, all activity levels at a particular SC site--produce the same rotation. Our SC model automatically incorporates this feature because quaternions have precisely this sort of redundancy: all scalar multiples of a given quaternion represent the same rotation. But r e d u n d a n c y is not e n o u g h to p r o d u c e weighted averaging. Downstream handling of the redundant signals must correctly sort out the information about saccade metrics, coded topographically, from the weighting factor coded in the activity level. In Figure 5, SC signals are input to a multiplieative feedback system. Multiplication is important because it makes the computed error vector E a linear function of the SC signal; linearity is the reason that scaling the SC signal results in scaling of the error vector. Linearity also means that the error computation distributes over addition, that is, if SC signals r~ and r~' generate error vectors E1 and E2, given the same feedback signal r, then the sum r~ + r~ will generate the error vector E~ + E2,

D. B~ Tweed and ]'. Vilis

The importance of distributivity is discussed in section 7.3. 7. CONNECTIONS WITH O ~ E R SC MODELS 7.1. The dn Lac-Knudsen Model

Other mechanisms have been proposed for vector averaging. Du Lac and Knudsen (1987) have suggested that an eye displacement is coded by the ratio of excitatory and inhibitory influences exerted by a SC site on downstream structures. Their model yields weighted average saccades by a mechanism based on redundancy: a one-dimensional rotation is expressed by the ratio of two numbers. Since the activity level at a locus presumably multiplies both terms in the fraction, leaving the ratio and therefore the coded displacement unchanged, this representation has the same invariance under scalar multiplication as quaternions do. One advantage of the quaternion approach over the du Lac-Knudsen ratios is that ~ t h quaternions, redundancy and multiplicative interactions arise automatically. Moreover, because activity levels of SC sites are divided out in the du Lac-Knudsen model, instead of driving a feedback system, their scheme does not account for the high speeds o f saccades evoked by high-intensity or two-Site stimulation of the SC. 7.2. The van Opstal-van Gisimrpn Model

A recent model by van Opstal and van Gisbergen (1989) matches, at least qualitatively, the two-site stimulation results using a simple mechanism based on lateral inhibition within the SC One advantage of this model is that it explains why, for SC stimulation at intensities near the threshold level, saccade amplitude does depend on intensity (van Opstal & van Gishergen, 1989). However. the quaternion model can also explain this finding in a natural way: at low activation levels, the error signal driving the saccade generator is just large enough to silence the pontine pause cells which inhibit short lead burst neurons (Beeker et al.. 1981); as the saccade proceeds, motor error eventually shrinks below the critical level and the pause cells terminate the saccade prematurely. Simulations of the model incorporating pause cells show that it makes saccade amplitude a nonlinear, saturating function of activation level near threshold. A problem for the van Opstat-van Gishergen model, and for any model that explains vector averaging based on intracollicutar mechanisms, is that there is another topographic map of saccades, located in the frontal cortex and known as the frontal

81

Superior Colliculus and Saccades

eye field (FEF); multiple-site stimulation here also produces vector averages (Robinson & Fuchs, 1969)--a result which can be explained by applying any of the above SC models to the F E F - - b u t in addition, vector averaging is also observed when one electrode is in the FEF and the other is in the SC (Schiller & Sandell, 1983). A quaternion model of the SC and FEF predicts this result, because the two quaternions still summate at the same comparator. Assuming that the SC and FEF project to the saccade generator in parallel, rather than in series (see Schiller & Sandell, 1983), it appears that the van Opstal-van Gisbergen model must postulate inhibitory connections between SC and FEF to explain the results of simultaneous stimulation in both regions. 7.3. Topographic Feedback Model In 1985, we proposed a topographic feedback model for the saccadic system which produced the correct results for two-site stimulation of the SC (Tweed & Vilis, 1985). In that paper, we noted that if each SC site sent its displacement signal to a different comparator, then the separate error signals from these comparators would compete to drive the saccade, and the sum of all the errors would become zero when the displacement was the weighted vector average of the displacements coded by the individual sites. Ouaternions remove the need for multiple comparators because they make the error computation muitiplicative; since multiplication distributes over addition, we have the useful relation (~ r,*)r ~

=

~(r,*r-~).

(2)

That is, whether the SC signals (r*) go to separate comparators (right side of (2)) or summate and feed into a single one (left side), the error computed is the same. 7.4. Three-Dimensional Scudder Model Could a three-dimensional version of Scudder's model also produce vector averages? It appears not, mainly because there is no straightforward threedimensional version of Scudder's model. The onedimensional version was obtained essentially by switching the order of the feedback integration and the subtraction in the displacement model. This switch worked because integration and subtraction are commuting operations. But in three dimensions, motor error is the quotient of, not the difference between, desired displacement and the integral of velocity times displacement. Division and integration do not commute, and so the Scudder model fails. It is possible that some variant of the Scudder model

may yet be correct, but we have not found one that combines simplicity and agreement with experiment as successfully as the three-dimensional displacement model. 8. LISTING'S LAW Any shift in the gaze direction can be accomplished by infinitely many different eye rotations. But the rotation r* coded by the SC must not only point the eye at the target, it must conform with Listing's law, which states that the torsional component of the eye position quaternion, ql, is always zero--q being expressed in head-fixed coordinates with the x-coordinate axis pointing forward (Westheimer, 1957). That is, el lies in Listing's plane (see Figure 6d). This section suggests one way that higher levels of the saccadic system might act on two-dimensional sensory information about target direction to select the correct SC site, and therefore the correct three-dimensional rotation r*. At the far left in Figure 6a, upstream from the SC, a signal codes the desired gaze vector g*, which has length 1 and lies along the desired direction of gaze. The length constraint reduces g* to two degrees of freedom. We assume that g* is expressed in head coordinates and is computed from retinal images and estimates of eye position, or from auditory or tactile information. Given any g*, the corresponding quaternion q* that satisfies Listing's Law is; q* = [X/(g, + 1)/2, 0,-g3/~/2(g, + 1), gSX/2(g, + 1)]. (3) This computation is performed by the box marked LL (for Listing's law) in Figure 6a. The resulting quaternion q* is then left multiplied by q-~ to yield r*; that is, the output of the LL operator interacts with a feedback signal coding actual eye position q, to determine which site in the SC will be activated. (The lag element, which indicates that the response of the SC to activity in the outer feedback loop is slow relative to the time scale of saccades, prevents interference with the local feedback guidance of saccades. Evidence for this lag is referenced in the Discussion.) According to Figure 6a, then, the mechanisms that create Listing's law are not at the level of the muscles or motoneurons, nor even at the burst cells as proposed by Robinson and Zee (1981), but are upstream from the SC. The rationale is that since Listing's law is a constraint on craniotopic eye position, it is probably implemented at some level where the saccade is still coded in terms of the desired craniotopic eye postion. And this way, the law could be incorporated into saccade planning and execution from the beginning. If this arrangement is correct, the SC must code a three-dimensional set of rotations r* rather than

82

D, B. t~weed arm T, Vi/i,~

the two-dimensional array usually assumed, because the eye rotations between positions fitting Listing's law form a three-dimensional set (Tweed & Vilis, 1987). This means that the axes of these rotations are not confined to any single plane. By measuring the instantaneous rotation axis of the eye during visually-elicited saccades made with the head stationary, we have confirmed that~saccade velocities have the horizontal, vertical, and torsional components that are required to keep eye position in accord with Listing's Law (Tweed & Vilis, in press). 9. DISCUSSION In this paper we have developed a three-dimensional model for saccade generation by the SC. According to this model, the SC sends a signal, coding desired overall eye displacement, as input to the saccade generator in the brain stem. This saccade generator is a feedback system in which a signal coding eye displacement so far is fed back and interacts multiplicatively with the SC signal to yield motor error. In our formulation, the SC is outside this feedback loop, and the desired eye displacement signal coded by the SC is essentially constant throughout the saccade, though the size and firing frequencies of the active site may be changing. (However, our mechanism for the spatiotemporal translation is equally consistent with the suggestion made by Waitzman, Ma, Optican, and Wurtz (1988), based on their finding of SC cells whose firing rates decline to zero in a tight temporal relation with saccadic motor error, that the comparator of the local feedback system is within the SC). The desired displacement is coded by a "hill" of activity whose location determines saccade size and direction, and whose height (i.e., the firing rates of the ceils in the active site) and breadth determine saccade speed. The resultant saccade is the weighted vector average of the saccades coded by the individual cells. This weight~ed averaging is an automatic consequence of using a quaternion representation for eye displacement in the model. We have also proposed that sensory information coding the two-dimensional direction of the saccade target interacts with eye position feedback upstream from the SC to determine which !SC site is activated. The resultant motor error signal rotates the eye toward the target so that the final eye position fits Listing's law. 9.1. Saccade Generation In this section we trace the generation of one saccade by the model in Figure 6a. Initial and desired craniotopic gaze vectors, g and g*, are drawn in Figure 6b. Vector parts of the c o r r e s p o n ~ eye position quaternions q and q* are shown lying in Listing's

plane in Figure 6d. To rotate the eye from q to q ~ preserving Listing's law, the eye velocity axis must tip backward out of Listing's plane (LP) as shown in Figure 6d. The LL operator computes q* from g*: the quotient q-lq* = r* is the desired eye rotation. whose axis tilts out of Listing's plane as required. The signal r* determines which SC site will be activated. This selection process has four important features. First, multiplicative interaction enables q* and the feedback signal q, both with zero torsional components, to generate a rotation r* with the required nonzero torsional component that is, a rotation whose axis lies out of Listing's plane. Second, the dependence of r* on q means that the SC site which is activated depends not only on retinal error but also on initial eye position. Third. the lag between q* and r*. representing the slow buildup and decay of hills of activity in the SC, prevents the outer feedback loop from interfering with local feedback guidance of saccades. Evidence that saccades are driven by a delayed and filtered version of a signal coding current target position has been obtained by Becket and Jurgens (1979) and van Gisbergen et al. (1981a). And fourth, if the subject has been actively fixating a target, then as the hill of activity representing r* builds up, another hill representing the null rotation (i.e.. fixation) is shrinking away (Figure 6c). Experimental evidence for a SC site coding the null rotation comes from the observation that some SC cells fire only when the monkey fixates a target light (Sparks & Mays, 1980). The purpose of the null site may be to inhibit saccades by activating pause cells and raising the threshold for triggering. In our model, the spatiotemporal translation occurs at the synapses between the SC output cells and the saccade generator. These output cells, which belong to the topographic map of e3 e displacements in the SC. each send four axon branches to the generator. The overall strengths of the synaptic connections made by these four branches are unequal: the relative strengths are the four components of a quaternion corresponding to the eve rotation represented by the SC cell (see Figure Oc). Since the SC may project to the saccade generator via vector burst neurons in the pons (Hepp & Henn. 1983), which also form a topographic map ol saccades, another possible site for the spatiotemporal translation is at the synapses between vector burst cells and generator. The saccade generator to which the SC (or vector burst) cells project is a multiplicative feedback system. m comparator computes a motor error vector E w t h e vector part of r - l r * This error signal in turn activates three populations of short lead burst neurons in the pons (horizontal cells) and the mesencephalon (nonhorizontal cells). As long as there are

Superior Colliculus a n d Saccades

83

reset

r~

,

()~

q~

~OJ,

(a)

1

Z

d

~JJ 1 o

oooooo

()

1 0 : 0 : 0

~

r° ~ :? ,o ) o :~

i'h

J

r" : r," : r~" : r~' LP \¸

(b)

(c)

"

~ ¢

(d)

FIGURE 6. Queternion model for the seccedic system. (a) Block diagram. A Listing's law (LL) operator computes the desired eye position quaternion q* from the desired gaze vector g*. Then the quotient of q* and an eye position feedback signal q determines which SC site r* is activated; the lag between the division operation and r* Indicates the sluggish response of the SC to upstream signals. The remainder of the model is as in Figure 5. (b) Initial and desired gaze vectors: g (up and right) and g* (up and left); initial and final gaze eccentricities are both 30°. (c) "Hills" of activity in the SC, with the SC depicted as a one-dimensional string of cells. The height of the line indicates the firing frequency of the cell beneath. Before the saccade, the subject was fixating a target, so there Is a hill of activity at the site coding the quaternion 1, representing the null rotation. The "null hill" is dwindling as a new hill, centred on the site representing the desired eye rotation r*, is rising. Axons of two SC output cells are shown, with four branches whose projection strengths to the downstream comparator are in the ratios of the components of the corresponding quaternions. (d) Vector parts of the initial and desired eye position quaternione q and q*. Both vectors lie in Llstlng's plane, indicating that the eye positions at the start and finish of the saccade both fit Listlng'e law. Tilting behind Llstlng's plane Is ~, the angular velocity vector of the eye, whose direction Is constant throughout the seccade. By rotating about this axis, the eye moves from q to q* with all Intermediate eye positions fitting LieUng's law.

no perturbations to the saccade, the velocity vector coded by the burst cells is parallel with r*, and therefore tilts out of Listing's plane as shown in Figure 6d. The error signal and velocity command diminish as r approaches r* and vanish when r - l r * = 1, at which point the eye is in the desired position q*. 9.2. Neural Network Implementation

To implement the flow diagram in Figure 6a as a neural network--that is, in terms of scalar signals and operations--we simply split each of the vector (or quaternion) signals into three (or four) parallel channels carrying the scalar components, and express the quaternion algebra in component terms as described in the Appendix. For example, Figure 7 shows a "neuronal" implementation of one channel of the spatiotemporal translation, in which signals from two active sites (coding quaternions p* and r*) interact with the feedback signal r to yield motor

error E. Applying eqns (A.3) and (A.4) from the Appendix, we find that the horizontal component (i.e., the third vector component) of E = r - X r * is given by E~ = ror~ -

r3rf* - q r * + r2r~.

(4)

In Figure 7, the four-component signalsp* + r* and r are shown converging on four scalar multipliers as dictated by this formula. The multipliers--two excitatory (white) cells and two inhibitory ( b l a c k ) project to an adder which outputs Ea. The other three components of E would come from three other circuits, much like this one, in parallel. This rather complicated depiction of one component of a single quaternion division shows how useful the quaternion notation is for keeping the flow diagrams compact and comprehensible. For an example of the circuit in action, suppose sites p* and r*, representing rotations of 12° and 6° left, are activated with intensities 2 and 1, respec-

84

D. tL /weed and 7~ VlliS

SC:Spatial Code

leftward rotation--the 2:1 weighted average of the rotations p* and r*. However, it is important to note that Figure 7, like any implementation in terms of scalars, is not a good depiction of the quaternion model, because the model is consistent with many other implementations. For example, the addition of SC signals and the division by r need not be implemented in two steps, on separate sets of neurons. More important, the fact that the equation for E~ involves multiplication does not imply the existence of individual multiplier neurons as shown in Figure 7. A forthcoming paper will show in detail how a wide range of multidimensional, nonlinear operations (such as quatern/on multiplication) can be performed quickly (in one synaptic delay: .5 ms) by small groups of very simple "neurons": cells which multiply their inputs by fixed weighting factors and sum the products, falling silent if the sum is below a fixed threshold.

0,0OO000O,(3

r3 e!

ro

9.3. Experimental Support for the Model

E3

Motor Error: Temporal Code FIGURE 7. INeuralnetworkl ~ n of M of thespa, t ~ ~ i ~ ~ O f ~ ~ ~ . ~t of ~ error, E,, I ~ ~ ~ and ~ The scalars p; to P3 ~ r ~ to r3 ot t.o SC o ~ MS ~ng'the ~ ~ s p* and r ; r0 to r3are the ~ p o n e n t s of the q ~ i o n r, the ~ ! coding eye rotation so far In the c ~ e n t ~ , coded in the firing frequencies of the feedback'axons.

tively. Then the four descending inputs to the multipliers are the components of the quaternion 2p* + r* = 2[cos 6° + (0, 0, sin 6°)] + cos 3°

+ (0, 0, sin 3°) = 1.989 + (0, 0, .209) + .999 + (0, 0, .052) = 2.988 + (0, 0, .261).

(5)

When the feedback quaternion r is cos5 ° + (0, 0, sin5 °) = .996 + (0, 0, .087), representing a 10° leftward rotation, application of (4) shows that E3 = .996(.261) - .087(2.988) = 0, Similar calculations show that the other vector components of E a r e also 0. Thus the error vector E driving the short lead burst cells is 0, and so the movement stops after a 10°

The model in Figure 6a provides simple explanations for several properties of the saccadic system. The redundant coding proposed for SC output cells, mwhich eye displacements are represented in quaternion form on four channels, can account for the observed correlations between single unit activity and saccade properties. The four channels would convey information about saccade metrics only in their relative effects on the saccade generator, so recordings from any one fibre, or from the cell of origin, would not be expected to correlate with saccade metrics. This prediction agrees with the findings of Sparks and Mays (1980). For saccades of the same size and direction, however, SC cells would be expected to fire harder for faster saccades, as observed by Munoz and Guitton (1987), and by Rohrer et al. (1987). Higher-intensity stimulation of a SC site would yield faster saccades, and sudden surges in cell firing frequency or stimulation intensity would correlate with surges in saccade velocity, as found by Munoz and Guitton (1987",. SC lesions would result in abnormally slow saccades (Hikosaka & Wurtz. 1985: Schiller, True, & Conway, 1980): since saccade velocity depends on the total activity driving the comparator, a lesion would reduce saccade speed by reducing the total SC output. The mechanism we have proposed for weighted vector averaging relates Robinson's double stimulus findings (1972) to other data on interactions ofelectrically-eticited saccades with visuaUy-elicited saccaries (Schiller & Sandell, 1983) or with fixation of a target light (Sparks & Mays, t983). In these situations, the saccade evoked by SC stimulation would be the weighted average of the rotations coded by

Superior Colliculus and Saccades

two hills of activity: one evoked by the electrode and one at the SC site coding the visually-elicited saccade or the null displacement during visual fixation (Sparks & Mays, 1980)--as in Figure 6c. Finally, the Listing's law mechanism in Figure 6a can account, not only for static eye positions fitting the law, but for the fact that during saccades the eye rotates about the unique fixed axis that brings the gaze to the target while preserving Listing's law throughout the movement (Tweed & Vilis, 1988). The hypothesis that Listing's law is generated upstream from the SC could be tested by electrical stimulation: if Listing's law is implemented upstream from the SC, stimulation of the SC should generally yield final positions violating the law; if Listing's law is implemented downstream, then saccades evoked by SC stimulation should end up in positions fitting the law, because the downstream Listing's law mechanism should alter the evoked rotation to preserve the law.

9.4. General Spatiotemporai Translation We showed that the essential feature of our spatiotemporal translation model is a particular sort of redundancy in the quaternion representation of eye rotations, coupled with downstream handling of the SC signals that can extract the displacement command. The representation is redundant because three-dimensional rotational information is carried in the ratios of activity among the four quaternion components, and so all scalar multiples of a given quaternion (i.e., all activity levels at a given SC site) represent the same eye rotation. SC outputs go to a multiplicative feedback system which automatically extracts the coded rotation and uses the activity level as a weighting factor. The essentials of this model could also be realized in other neural systems. Multichannel frequency codes which are unaffected by scaling, coupled with downstream multiplicative feedback systems which extract the coded variables, are plausible elements in any spatiotemporal translation in which firing frequency at a map site serves as a weighting factor for spatially-coded information.

REFERENCES Becker, W., & Jurgens, R. (1979). An analysis of the saccadic system by means of double step stimuli. Vision Research, 19, 967-983. Becker, W., King, W., Fuchs, A., Jurgens, R., Johansen, G., & Kornhuber, H. (1981). Accuracy of goal-directed saccades and mechanisms of error correction. In A. Fuchs & W. Becker (Eds.), Progress in oculomotor research (pp. 29-37). Amsterdam: Elsevier/North-Holland.

85 Berthoz, A., Grantyn, A., & Droulez, J. (1986). Some collicular efferent neurons code saccadic eye velocity. Neuroscience Letters 72, 289-294. Brand, L. (1948). Vector and tensor analysis. New York: J. Wiley. Cannon, S., & Robinson, D. (1987). Loss of the neural integrator of the oculomotor system from brain stem lesions in monkey. Journal of Neurophysiology 57, 1383-1409. du Lac, S., & Knudsen, E. (1987). The optic tectum encodes saccade magnitude in a push-pull fashion in the barn owl. Society for Neuroscience Abstracts, 13(112.10), 393. Harting, J. (1977). Descending pathways from the superior colliculus: An autoradiographic analysis in the rhesus monkey (Macaca mulatta). Journal of Comparative Neurology. 173. 583-612. Hepp, K., & Henn, V. (1983). Spatio-temporal recoding of rapid eye movement signals in the monkey paramedian pontine reticular formation (PPRF). Experimental Brain Research, 52, 105-120.

Hepp, K., Vilis, T., & Henn, V. (1988). The generation of rapid eye movements in three dimensions. Annals of the New York Academy of Science, 545, 140-153. Hikosaka, O., & Wurtz, R. (1985). Modification of saccadic eye movements by GABA-related substances. I. effect of muscitool and bicuculline in monkey superior colliculus. Journal of Neurophysiology, 53,266-291. Hikosaka, O.. & Wurtz, R. (1986). Saccadic eye movements following injection of lidocaine into the superior colliculus. Experirnental Brain Research, 61, 531-539. Jurgens, R., Becker, W., & Kornhuber, H. (1981). Natural and drug-induced variations of velocity and duration of human saccadic eye movements: evidence for a control of the neural pulse generator by local feedback. Biological Cybernetics, 39, 87-96. Keller, E. (1980). Oculomotor specificity within subdivisions of the brain stem reticular formation. In J. Hobson & M. Brazier (Eds.), The reticular formation revisited (pp. 227-240). New York: Raven Press. King, W., & Fuchs, A. (1979). Reticular control of vertical saccadic eye movements by mesencephalic burst neurons. Journal of Neurophysiology, 42, 861-876. Munoz, D., & Guitton, D. (1987). Tecto-reticulo-spinal-neurons have discharges coding the velocity profiles of eye and head orienting movements. Society for Neuros'cience Abstracts, 13(112.9), 393. Robinson, D. A. (1972). Eye movements evoked by collicular stimulation in the alert monkey. Vision Research, 12, 17951808. Robinson, D. A., & Fuchs, A. (1969). Eye movements evoked by stimulation of frontal eye fields. Journal of Neurophysiology, 32, 637-648. Robinson, D.. & Zee, D. (1981). Theoretical considerations in the function and circuitry of various rapid eye movements. In A. Fuchs & W. Becker (Eds.), Progress in oculomotor research (pp. 3-12) Amsterdam: Elsevier/North-Holland. Rohrer, W., White, J., & Sparks, D. (1987). Saccade-related burst cells in the superior colliculus: relationship of activity with saccadic velocity. Society for Neuroscience Abstracts, 13(303.12), 1092. Schiller, P., & Sandell, J. (1983). Interactions between visually and electrically elicited saccades before and after superior colliculus and frontal eye field ablations in the rhesus monkey. Experimental Brain Research, 49, 381-392. Schiller, P., & Stryker, M. (1972). Single-unit recording and stimulation in superior colliculus of the alert rhesus monkey. Journal of Neurophysiology, 35, 915-924. Schiller, P., True, S., & Conway, J. (1980). Deficits in eye movements following frontal eye-field and superior colliculus ablations. Journal of Neurophysiology , 44, 1175-1189.

D.B.

86

Scudder, C. (1988). A new local feedback model of the saccadic burstgenerator. Journal o f Neurophysiology, 59, 1455-1475. Sparks, D., & Mays, L. (1980). Movement fields of saccaderelated burst neurons in the monkey superior colliculus. Brain Research, 190, 39-50. Sparks, D., & Mays, L. (1983). Spatial localization of saccade targets. I. Compensation for stimulation-induced perturbations in eye position. Journal o f Neurophysiology, 49, 45-63. Sparks, D., Mays, L., & Porter, J. (1987). Eye movements induced by pontine stimulation: Interaction with visually triggered saccades. Journal o f Neurophysiology, 58, 300-317. Tait, P. (1890). A n elementary treatise on quaternions. Cambridge: Cambridge University Press. Tweed, D., & Vilis, T. (1985). A two dimensional model for saccade generation. Biological Cybernetics, 52, 219-227. Tweed, D., & Vilis, T. (1987). Implications of rotational kinematics for the oculomotor system in three dimensions. Journal o f Neurophysiology, 58, 832-849. Tweed, D., & Vilis, T. (1988). Rotation axes of saccades. Annals o f the New York Academy o f Sciences, 545, 128-139. van Gisbergen, J., Gielen, S., Cox, H., Bruijns, J., & Schaars, H. (1981a). Relations between metrics of saccades and stimulus trajectory in visual target tracking; implications for models of the saccadic system. In A. Fuchs & W. Becker (Eds.), Progress in oculomotor research (pp. 19-27). Amsterdam: Elsevier/North-Holland. van Gisbergen, J., Robinson, D., & Gielen, S. (1981b). A quantitative analysis of generation of saccadic eye movements by burst neurons. Journal o f Neurophysiology, 45, 417-442. van Opstal, J., & van Gisbergen, J. (1989) A nonlinear model for collicular spatial interactions underlying the metrical properties of electrically elicited saccades. Biological Cybernetics, 60, 171-183. Waitzman, D., Ma, T., Optican, L., & Wurtz, R. (1988). Superior colliculus neurons provide the saccadie motor error signal. Experimental Brain Research, 112, 1-4. Westheimer, G. (1957). Kinematics of the eye. Journal o f the Optometric Society o f America, 47, %7-974.

APPENDIX The Quaternion R ~ n t n l i o n

of Eye Rotations

A quaternion is a four-component object which is the sum of a scalar and a vector: q = q0 + q.

(A.1)

Any quaternion q can be written q = Iql[(cos(a/2) + nsin(a/2)]

(A.2)

l~,eed a n d T. Vitis

for some a and some unit vector n; a is called the angle of the quaternion, n its axis, and Iql, which ~/(qoq~ + q~q: + q2q2 + q3q3), its magnitude. Quaterniorm can be added and subtracted like four-component vectors. They can also be multiplied and divided. Using indices 0 through 3 for the four components of a quaternion, the formula for a quaternion product p = qr is: Po = qoro - qlr~ - q2r~

q~r~

p~ = qorl + q~ro + q2r~ - o r~ p: = qor2 + q2r~ "- q~r "- q~r~ p~ = qor3 + q~ro + qir,~ .... qJ~.

(A.3)

To divide by a quaternion q, one multiplies by the multip!icative inverse q-~. If q = q0 + q, then q-~ = (q,i " q)/] q [2; in particular, if q has magnitude 1 then q ' = q 0 - q.

(A.4)

Quaternions represent eye positions in terms of their rotational displacements from some reference position~ Such as primary position. If the eye is displaced from primary position by an a-degree rotation about the axis n (a vector of length L oriented according to the right hand rule), then we represent the position by the quaternion q of magnitude 1: q = cos(a/2) + nsin(a/2).

(A.5)

For example, an eye position 30° left of primary position has the quaternion q = cosl5 ° + sinl5°(0, 0, 1), where.the vector is expressed in a coordinate system with the first coordinate axis pointing forward, the second left and the third up. The reason quaternions are regarded as representing rotations is that, for any vector v and quaternion p, the vector v' = pvp I

(A.6)

is obtained by rotating v about the axis of p, through the angle of p. Note that multiplying p by any nonzero scalar c would not change the rotation represented by p: the' inverse of cp is p--!tc, and so c and 1/c cancel in the operation p v p - k To state this important result another way: all nonzero Scalar multiples o f a given quaternion represent the same rotation.

The resultant of two rotations can be computed using quaternion multiplication. That is, rotation p followed by rotation q yields the overall angular rotation qp. Since multiplying any quaternion by 1 leaves the quaternion unchanged, q = 1 is the unit quaternion that represents no displacement, In three dimensions, the rate of change of angular position depends not only on angular velocity co, but also on instantaneous eye position itself (regardless of the representation e ~ n for eye position), so that position signals cannot be computed by integrating velocity signalS. Eye position q can he computed from ¢o using the following formula (Tait, 1890; Tweed & Vilis, 1987): ?l = o~q/2.

(A.7)