R744
Dispatch
Extrastriate cortex: A signature of perception grows K.H. Britten
Neuronal activity in area MT of the extrastriate visual cortex is correlated with the choices monkeys make on perceptual tasks. New evidence suggests that this correlation is stronger on some tasks than others. Address:University of California at Davis Center for Neuroscience, 1544 Newton Court, Davis, California 95616, USA. E-mail:
[email protected]. Current Biology 2001, 11:R744–R746 0960-9822/01/$ – see front matter © 2001 Elsevier Science Ltd. All rights reserved.
Neurophysiologists love correlations. Correlation with a stimulus property is a window into the sensory information carried by a neuron or a population of neurons. Correlation with movement reveals the motor roles of neurons at the ‘business end’ of the nervous system. Perceptual correlates are intrinsically fascinating but tricky — perception is not necessarily tied in a one-to-one manner to the stimulus, nor need it always produce a particular motor outcome. Some of the most revealing experiments have found correlates of perception in the firing rates of sensory neurons responding to perceptually ambiguous stimuli. A new study by Dodd et al. [1] has revealed that the magnitude of such Figure 1 Trials sorted by monkey choice Non-pref
100
Pref
Response
80 60 40 20 Non-pref Pref
0 410
630 Trial number
6 4 2 0 2 4 Count Current Biology
Example of an MT cell showing a large and significant choice probability. Individual trials are shown by the dots; firing rates vary substantially from trial to trial. For all of these trials, the stimulus was the same — the colors indicate the monkey’s choice on the trial. One can see that the firing rate of the cell is usually higher on trials where the monkey later chose the cell’s preferred stimulus, which in this case was a clockwise rotation. The histograms to the right show the firing rate distributions for the same trials, color coded in the same manner. Choice probability indicates the degree of separation between these two distributions; for this neuron the choice probability was 0.79.
perceptually tied signals depends on the animal’s task. Although this study is similar in design to two previous studies [2,3], the results are revealingly different. In their new study, Dodd et al. [1] recorded the responses of neurons in the middle temporal area (MT) of the extrastriate visual cortex — known to be important in motion perception — while monkeys reported the direction of rotation of a projected, ‘transparent’ cylinder of rotating dots. Related displays have been widely used to study how motion cues the perception of three-dimensional shape. In the most revealing configuration, the dots convey no depth information, so the display is purely two-dimensional. Human subjects viewing such a display invariably report a vivid sense of rotation in depth, but the percept is ambiguous: either surface can be perceived as being in front. The two percepts correspond to the two possible directions of cylinder rotation that the monkeys were trained to report. The trick when using monkeys for such tasks, of course, is making sure they are being honest — there is no objectively right or wrong answer upon which to base their reward. To keep the monkeys honest, these fully ambiguous trials were embedded in a series of trials in which stereo disparity was added to the display to give the cylinder ‘true’ depth and thus a veridical direction of rotation. When very little disparity was added to the display, the correct rotation was difficult to perceive, and mistakes were frequent. The monkeys’ performance smoothly increased as disparity was added. One gets confidence that the monkeys were honestly reporting their percepts because their performance was statistically similar on the fully ambiguous trials and the trials where a little disparity was added. If the monkeys were adopting some other strategy on the ambiguous trials, then one would probably see some difference in the distribution of their choices — monkeys often show strong choice biases when they give up on a task. While the monkeys were performing this task, Dodd et al. [1] recorded from neurons in MT that represented the correct region of space and discriminated between the two directions of cylinder motion. For the interesting, perceptually ambiguous trials, the stimulus was constant: the same two-dimensional projection of a rotating cylinder. On some trials, however, the monkey reported a clockwise direction percept, and on others the opposite. Many neurons in MT responded differently depending on the monkey’s percept, as illustrated in Figure 1. The points in Figure 1 represent neuronal firing rates on individual trials as the block of trials progressed; green dots correspond to clockwise-report trials, while red dots correspond
Dispatch
to counterclockwise-report trials. The histograms show the resulting firing-rate distributions. Dodd et al. [1] used a statistic called the ‘choice probability’ to quantify the magnitude of the difference between the behavior of MT neurons in the two cases. This metric reports the sign and degree of separateness of the two distributions — if the neuron always fires more on ambiguous trials when the monkey reports the percept in the neuron’s preferred rotation direction, the choice probability is 1.0. If the firing rate is uncorrelated with the monkey’s report, the choice probability is 0.5. The average value for the authors’ sample of 93 neurons from two monkeys was 0.67. Thus, on average, neurons in MT predict the decision of the monkey on these perceptually ambiguous displays. This is the signature of perception in the firing rates of MT cells. Similar correlations with perception have been seen before in a variety of perceptual tasks [2–7]; their mere presence comes as no particular surprise. However, two findings in this new study [1] are novel by comparison with the previous work. In every other experiment of this sort, some cells have been correlated in the wrong direction — that is, their responses go up when the monkey perceives the neuron’s non-preferred stimulus. In the experiment reported by Dodd et al. [1], all neurons that showed a significant relationship did so in the more sensible ‘forward’ direction. The most similar experiment was that of Bradley et al. [2], who found about a quarter of the neurons that showed correlation to perception were correlated in the ‘backwards’ direction. Dodd et al. [1] attribute the differences between the two sets of results to differences in statistical methods; certainly very different analyses were done. More interesting, perhaps, was a substantial difference in the average magnitude of the choice probability. The most directly comparable numbers come from the very similar analysis in the work of Britten et al. [3], in the context of a closely related task, in which monkeys discriminated the direction of two-dimensional patterns containing weak motion mixed with noise. This study reported an average value of 0.55, substantially smaller than in the study of Dodd et al. [1]. This shows that the magnitude of the choice probability depends on the task being performed, or the stimulus used. The new results, then, are larger in magnitude and more consistent in sign than have been previously observed. To think about this, we must consider two possible sources for the choice probabilities. One possibility is that spikes in MT directly cause perception and thus decisions on the task. Firing rates in MT vary; this leads to variability in the resulting percept. The magnitude of the choice probability depends on a variety of things, such as the number of neurons used for the decision and their degree of correlation with each other [8]. The feedforward interpretation
R745
has been favored by many authors, including Dodd et al. [1]. To explain the increased magnitude of the choice probability in the cylinder task, one needs to invoke changes in the ‘readout’ rules by which neuronal signals are converted into percepts or decisions. Presumably, the changes occur because the cylinder task depends on two stimulus dimensions: motion and depth. MT is involved in perceiving both these dimensions [9,10]. The fact that MT signals need to be sorted on two dimensions in order to be useful on the cylinder task suggests that downstream mechanisms might very well use different rules, compared to the direction task where only one dimension is involved. The second possibility comes from the idea that feedback or ‘top-down’ influences affect the firing rates of MT cells. Such influences, which would presumably be mediated by the extensive feedback connections in extrastriate cortex, could include modulation related to the decision or attention directed selectively to one of the alternatives on the task. Decision-related activity is widespread in higher areas of parietal and frontal cortex [11,12], and attentional signals are profound in MT and elsewhere [13,14]. Consideration of the dynamics of the effects observed by Dodd et al. [1] may help to shed light on the underlying mechanism. In particular, decision-related activity would be expected to grow with time, as the monkey makes up its mind. Figure 2a shows the time course of the perceptually correlated activity in MT. There is a clear trend towards increasing differences as time progresses, not previously observed in other work on MT. For comparison, Figure 2b shows the same measurement from the direction discrimination task [3]. In this work, there is no corresponding change over time. The difference between the two studies is clear yet puzzling; one certainly would not expect entirely different mechanisms to be at work. The common feature in both data sets, however, is that the signal is present from the first 50 milliseconds of the stimulus — this seems unreasonably fast for a decision signal. The buildup of such signals indeed seems to occur quite slowly where it has been directly investigated [15,16]. The other top-down signal, from directed attention, is however consistent with such rapid dynamics. Biases in the amount of attention directed to each alternative can be present from the start of a trial, and indeed might grow during the trial. Such signals would probably be stray attention — in contrast to experiments where cues specifically direct animals’ attention — and thus might have quirky variability from experiment to experiment. In any case, the Dodd et al. [1] results place strong constraint on what attention can be modulating, if it is the source of choice probabilities. From their results, attention cannot be modulating any single feature, as the alternatives are distinguished by a conjunction of depth and motion.
R746
Current Biology Vol 11 No 18
Figure 2
the change take place asynchronously in sensory areas like MT and higher areas. With enough temporal resolution, simultaneous recording might be able to detect which structure changed state first. This structure would then have a claim on the privileged position at the tail of the causal arrow, leading us beyond correlation. Even if we cannot easily pin down the direction of causation, finding traces of conscious experience in the discharges of single neurons is a precious glimpse into the machinery of perception.
(a) Response
0.6 0.4 0.2
Difference
0.2
References 0.1
0 0
1.0 Time (sec)
2.0
Preferred choices Non-preferred choices
Difference
Response
(b) 0.4 0.3 0.2 0.1 0.08 0.04 0
0
1.0 Time (sec)
2.0 Current Biology
Time course of the firing rate differences that result in choice probabilities. (a) Results from the cylinder task used by Dodd et al. [1]. For this analysis, the firing rate histograms of all 36 neurons were individually normalized to the peak of the preferred-choice histogram, then averaged. The green curve depicts the rate function for preferredchoice trials; the red for the non-preferred-choice trials. These curves separate early, but continue to diverge through the two-second stimulus period. (b) Identical analysis for the direction discrimination task used by Britten et al. [3]. In this case, there is no sign of a trend of increasing difference with time.
Therefore, if attentional modulation is to explain the correlation with perception, it must be flexible enough to selectively modulate neurons according to their preferences on multiple dimensions. It seems intrinsically difficult to dissociate feedforward from feedback models on the basis of dynamics alone, but maybe modern multi-electrode methods could help settle the issue. The rate changes seen by Dodd et al. [1] seem gradual, but this might be an artifact of averaging. If they result from the abrupt switch of a bistable percept (which human subjects report for these stimuli), one might see
1. Dodd JV, Krug K, Cumming BG, Parker AJ: Perceptually bistable 3-D figures evoke high choice probabilities in cortical area MT. J Neurosci 2001, in press. 2. Bradley DC, Chang GC, Andersen RA: Encoding of threedimensional structure-from-motion by primate area MT neurons. Nature 1998, 392:714-717. 3. Britten KH, Newsome WT, Shadlen MN, Celebrini S, Movshon JA: A relationship between behavioral choice and the visual responses of neurons in macaque MT. Vis Neurosci 1996, 13:87-100. 4. Logothetis NK, Schall JD: Neuronal correlates of subjective visual perception. Science 1989, 245:761. 5. Leopold DA, Logothetis NK: Activity changes in early visual cortex reflect monkeys’ percepts during binocular rivalry. Nature 1996, 379:549-553. 6. Celebrini S, Newsome WT: Neuronal and psychophysical sensitivity to motion signals in extrastriate area MST of the macaque monkey. J Neurosci 1994, 14:4109-4124. 7. Thiele A, Hoffman K-P: Neuronal activity in MST and STPp, but not MT changes systematically with stimulus-independent decisions. NeuroReport 1996, 7:971-976. 8. Shadlen MN, Britten KH, Newsome WT, Movshon JA: A computational analysis of the relationship between neuronal and behavioral responses to visual motion. J Neurosci 1996, 16:1486-1510. 9. Salzman CD, Murasugi CM, Britten KH, Newsome WT: Microstimulation in visual area MT: effects on direction discrimination performance. J Neurosci 1992, 12:2331-2355. 10. DeAngelis GC, Cumming BG, Newsome WT: Cortical area MT and the perception of stereoscopic depth. Nature 1998, 394:677-680. 11. Kim JN, Shadlen MN: Neural correlates of a decision in the dorsolateral prefrontal cortex of the macaque. Nat Neurosci 1999, 2:176-185. 12. Snyder LH, Batista AP, Andersen RA: Intention-related activity in the posterior parietal cortex: a review. Vis Res 2000, 40:1433-1441. 13. Treue S, Martínez Trujillo JC: Feature-based attention influences motion processing gain in macaque visual cortex. Nature 1999, 399:575-579. 14. Reynolds JH, Pasternak T, Desimone R: Attention increases sensitivity of V4 neurons. Neuron 2000, 26:703-714. 15. Schall JD, Hanes DP, Thompson KG, King DJ: Saccade target selection in frontal eye field of macaque. I. Visual and premovement activation. J Neurosci 1995, 15:6905-6918. 16. Shadlen MN, Newsome WT: Neural basis of a perceptual decision in the parietal cortex (Area LIP) of the rhesus monkey. J Neurophysiol 2001, in press.