Recurrent Feedback Loops in Associative Learning

Neuron

Previews movements in Drosophila. The direct flight muscles attach to the base of the wing and have been implicated in controlling the temporal structure of song (Shirangi et al., 2013). The indirect flight muscles attach to the thorax and are important for power generation during flight (Moore et al., 2000). Coen et al. found a strong correlation between indirect flight muscle spiking activity and pulse amplitude, but no correlation between spiking activity and IPI. Together, these results suggest that the indirect flight muscles control pulse amplitude, whereas the direct flight muscles independently control song timing. Although the ability to modulate song intensity with distance makes intuitive sense, it remains to be seen how AMD contributes to mating success. Regardless, these studies reveal an intriguing level of sophistication to courtship song, with male fruit flies modulating their tune

following a control strategy that was previously thought to be the exclusive province of larger, more complex vertebrate brains. Future studies in the fly will enable this strategy to be understood at the circuit and algorithmic level, shedding new light on how animal communication is shaped by interactions between individuals.

Coen, P., Xie, M., Clemens, J., and Murthy, M. (2016). Neuron 89, this issue, 629–644. Kohatsu, S., and Yamamoto, D. (2015). Nat. Commun. 6, 6457. Kohatsu, S., Koganezawa, M., and Yamamoto, D. (2011). Neuron 69, 498–508. Moore, J.R., Dickinson, M.H., Vigoreaux, J.O., and Maughan, D.W. (2000). Biophys. J. 78, 1431–1440. Schuster, S., Strauss, R., and Go¨tz, K.G. (2002). Curr. Biol. 12, 1591–1594.

REFERENCES Agrawal, S., Safarik, S., and Dickinson, M. (2014). J. Exp. Biol. 217, 2796–2805.

Shirangi, T.R., Stern, D.L., and Truman, J.W. (2013). Cell Rep. 5, 678–686.

Brumm, H., and Slater, P.J.B. (2006). Anim. Behav. 72, 699–705.

von Philipsborn, A.C., Liu, T., Yu, J.Y., Masser, C., Bidaye, S.S., and Dickson, B.J. (2011). Neuron 69, 509–522.

Clowney, E.J., Iguchi, S., Bussell, J.J., Scheer, E., and Ruta, V. (2015). Neuron 87, 1036–1049.

Yamamoto, D., and Koganezawa, M. (2013). Nat. Rev. Neurosci. 14, 681–692.

Clyne, J.D., and Miesenbo¨ck, G. (2008). Cell 133, 354–363.

Zahorik, P., and Kelly, J.W. (2007). J. Acoust. Soc. Am. 122, EL143–EL150.

Coen, P., Clemens, J., Weinstein, A.J., Pacheco, D.A., Deng, Y., and Murthy, M. (2014). Nature 507, 233–237.

Zhou, C., Franconville, R., Vaughan, A.G., Robinett, C.C., Jayaraman, V., and Baker, B.S. (2015). eLife 4, 1–25.

Recurrent Feedback Loops in Associative Learning Abigail L. Person1 and Kamran Khodakhah2,3,4,* 1Department

of Physiology & Biophysics, University of Colorado School of Medicine, Aurora, CO 80045, USA P. Purpura Department of Neuroscience 3Saul R. Korey Department of Neurology 4Department of Psychiatry and Behavioral Sciences Albert Einstein College of Medicine, New York, NY 10461, USA *Correspondence: [email protected] http://dx.doi.org/10.1016/j.neuron.2016.01.037 2Dominick

In this issue of Neuron, Gao et al. (2016) report on a little-studied feedback pathway from the cerebellar nuclei back to the cerebellar cortex. They find that it contributes to associative conditioning and execution of learned movements, highlighting a role for local feedback loops in the brain.

The brain, like other computational devices, often uses feedback to fine-tune and optimize its computations. For example, iterative computational circuits frequently sample their own output to estimate error in their intended outcome and use this information to improve their computational accuracy in the next iteration. In addition to the traditional global feedback loop that originates from the final output of the circuit, local feedback loops also offer computational benefits.

Consider, for example, a computational device made of a number of discrete operational stages, each of which transforms the information it receives to generate its own output. In such a case, local feedback loops, embedded in each operational stage, offer tremendous flexibility and specificity for modulation of the overall computational capacity and accuracy of the device. In many cases, local feedback mechanisms are indispensable for circuit optimization,

for example, when the computation performed at each stage is not linear, or when additional (external) inputs are incorporated at each operational stage. In addition to providing a mechanism for error minimization, feedback can also be used to alter the gain of a circuit, or as a plasticity mechanism for learning. In artificial computational devices, feedback mechanisms work at both the local and global level. It is therefore

Neuron 89, February 3, 2016 ª2016 Elsevier Inc. 427

Neuron

Previews

Figure 1. A Local Feedback from the Cerebellar Nuclei to the Cerebellar Cortex Is Found to Facilitate Associative Learning Depicted is the simplified organization of the cerebellar circuitry.

no surprise that in the brain, where computations occur at many different scales, feedback loops appear to be used at almost every level: within single neurons, within cortical layers, between cortical layers, and between cortex and subcortical structures. In this issue of Neuron, Gao et al. (2016) use the many advantages of cerebellum as a circuit ideally suited for reverse engineering the brain (Khodakhah, 2015) to examine the properties and function of a local feedback loop from the cerebellar nuclei to the cerebellar cortex and provide convincing evidence for the role of this feedback loop for motor learning in associative conditioning. To appreciate the contribution of Gao et al. (2016), it would be helpful to consider the overall design of the cerebellum (Figure 1): the cerebellum is an evolutionarily conserved, layered structure emerging from the hindbrain that is critical for refining motor control, motor learning, and analogous cognitive sharpening. The input layer, the granule cell layer, houses half the neurons in the brain, with each granule cell receiving just four or so mossy fiber inputs. Granule cells project into the superficial

molecular layer where they branch to form parallel fibers, which innervate the spectacular dendrites of Purkinje cells, along with a smattering of molecular layer inhibitory interneurons (MLIs), and apical dendrites of inhibitory interneurons of the granule cell layer, Golgi cells. Purkinje neurons also receive olivary climbing fiber inputs that wrap around the dendrites like a vine and trigger associative plasticity of parallel fiber inputs onto Purkinje cells. Purkinje neurons in turn project to and inhibit the cerebellar nuclei, which form the sole output of most of the cerebellum. By in large, this circuit is feedforward, which makes the long-known (Houck and Person, 2014) but little-studied feedback pathway from the cerebellar nuclei to the cerebellar cortex so tantalizing: how is this local feedback used in motor learning and cerebellar computation? To address this question, Gao et al. (2016) use optogenetics to selectively manipulate activity in this pathway during a cerebellar-dependent learning task. Delay eyeblink conditioning is a cerebellar-dependent form of classical conditioning that exploits the ability of the circuit to associate arbitrary stimuli to generate a conditioned motor output. In the first part of their study, Gao et al. (2016) examine structural plasticity in nucleocortical pathway during motor learning. They corroborated recent work showing that the nucleocortical pathway innervates part of the cerebellar cortex involved in eyeblink conditioning (Houck and Person, 2015) and then go on to explore whether nucleocortical filopodia show structural plasticity during eyeblink conditioning as they have been shown to do with rotarod training (Ruediger et al., 2011). Indeed, comparing nucleocortical terminals in naive and trained mice revealed dramatic structural plasticity, with a nearly 70% increase in the number of filopodial extensions. The structural plasticity observed in this population of mossy fibers raises many important questions. Among the most important are the implications for the combinatorial pattern recognition of granule cells. A foundation of cerebellar theory revolves around the idea that granule cells respond to coincident inputs

428 Neuron 89, February 3, 2016 ª2016 Elsevier Inc.

from diverse mossy fiber sources, which converge onto sparse granule cell dendrites. This convergence is hypothesized to produce an expansion recoding by the granule cell layer that could be used by Purkinje neurons in pattern recognition (Marr, 1969; Albus, 1971). The combinatorial expansion theorized to be produced by granule cells could be modulated if mossy fiber filopodia extend to integrate with granule cells that receive other sources of information: if mossy fiber filopodia sprout to invade sensory glomeruli contacting granule cells, for example, combinatorial expansion may transform, as would receptive field properties of granule cells and their overlying Purkinje neurons. In addition to the coding implications of feedback mossy fiber sprouting, questions arise around the molecular underpinnings of the expansion. What drives filopodial outgrowth upon learning, and are the changes stable or transient? Regardless of the specific dynamics of mature mossy fiber filopodia, identifying the molecular drivers of this structural plasticity will be essential to fully understand cerebellar learning processes. One clue could come from cerebellar development studies. Cerebellar mossy fiber filopodial plasticity is seen during development and is regulated by BMP4 signaling (Kalinovsky et al., 2011). Future studies are thus poised to investigate whether these developmental mechanisms are preserved in adults during periods of learning and, if so, how those processes are engaged. It is interesting to note that filopodial extensions of other cerebellar mossy fiber sources targeting Lobules 5 and 9 sprout during cerebellar-dependent motor learning of a rotarod task (Ruediger et al., 2011). In the same study, it was noted that hippocampal mossy fibers also sprout filopodia during hippocampal-dependent learning, suggesting common mechanisms and computational principles associated with structural plasticity. There it was found that filopodia preferentially targeted inhibitory interneurons, which was not seen in the current study. The present observation of filopodial contacts onto granule cells is therefore novel and suggests that filopodial recruitment of feedforward

Neuron

Previews inhibition through Golgi cells is not the whole story. To look at how motor learning affects the inhibitory/excitatory (I/E) balance in Purkinje cells, Gao et al. (2016) examined feedforward inhibition into Purkinje cells and found a very intriguing pattern of Purkinje cell inhibition. Whereas before learning nucleocortical activation drove slightly more inhibition than excitation in Purkinje cells (I > E), after eyeblink condition the I/E difference was steeper, with a greater degree of inhibition onto Purkinje neurons. The pattern of high I/E ratio elicited by nucleocortical activation was in stark contrast to the pattern seen with electrical (i.e., nonspecific) mossy fiber activation, which evoked about half as much inhibition as excitation under basal conditions. Feedforward inhibition onto Purkinje neurons is presumably via granule cells driving MLI inhibition of Purkinje neurons. As such, the learning-related change in inhibition onto Purkinje cells points to yet another mechanism of cerebellar plasticity: that of increasing feedforward inhibition onto Purkinje cells by specific mossy fiber afferents. The finding of altered I/E balance after learning is an exciting advance given the degree of debate in the field about the significance of traditional ‘‘cerebellar LTD’’ mechanisms in learning. While pauses develop in Purkinje neurons during acquisition of conditioned eyeblink responses, disinhibiting downstream nuclear neurons that drive eyelid closure, the mechanistic underpinnings of these pauses are under intense scrutiny. While there is not enough room to review the debate here, the finding that inhibition is plastic onto Purkinje neurons, even if the increase is owed to polysynaptic changes upstream, could complement concomitant changes in excitation or operate in a parallel manner to other synaptic changes underlying learned responses. The final set of experiments reported here go on to show the functional role of nucleocortical feedback in delay eyelid conditioning. Optogenetic stimulation of the nucleocortical mossy fibers before conditioned responses are learned elicits no eyelid closure, but activation after learning enhances the amplitude and shortens the latency of the condi-

tioned response. Conversely, optogenetic silencing of the nucleocortical pathway delays and truncates learned eyelid closures. To make sense of these data, it is important to consider that the nucleocortical mossy fiber pathway is comprised of axon collaterals of cerebellar output neurons. As such, they send copies of motor-related signals to the cerebellar cortex (Houck and Person, 2015; Tolbert et al., 1978; Gao et al., 2016). What makes this pathway enticing to study in the context of eyeblink conditioning is that we know how the activity patterns of these neurons change during learning. Before a conditioned response is learned, nuclear neuron firing rate is relatively unmodulated by the conditioning stimulus. After the CR is learned, however, nuclear neurons increase their firing rates to drive anticipatory eyelid closure. Thus, the content of the feedback information sent to the cortex is changing—increasing—with training, indicative of a positive feedback loop. Therefore, both physiological and structural changes occur during learning. Since optogenetic activation of the pathway only shows effects after learning, it points to the structural plasticity rather than the activity changes alone, as critical for altering the gain of other sensory inputs to Purkinje cells. The findings have implications for the role of this local feedback loop in other cerebellar computations. If this local feedback is used to shape learned movements guided by the cerebellum, then silencing of the feedback nucleocortical branch during other cerebellardependent behaviors, such as locomotion (Machado et al., 2015), should alter coordination. The intriguing question is whether such a role could be observed only in newly learned skills, or whether all movements rely on analogous information to appropriately shape motor kinematics. Eyelid conditioning is convenient for its novelty and arbitrariness, since animals will not have naturally paired a tone with a reflex-eliciting stimulus. However, the paradigm is blind to the role of such information in naturally learned and previously integrated sensorimotor mappings. Future experiments would likely address these and related questions.

A question worthy of further scrutiny is whether the nucleocortical feedback loop can be considered to be a corollary discharge pathway. Corollary discharge, or efference copy as it is also known (Sperry, 1950; von Holst and Mittelstaedt, 1950), has long been hypothesized to play a critical role in motor control and sensory processing throughout the brain (Sommer and Wurtz, 2008). Within the cerebellum, corollary discharge information has been postulated to contribute to kinematic predictions made by Purkinje neurons (Ito, 2006). By bypassing slow sensory reafference in guiding feedforward motor control, copies of motor commands can rapidly provide feedback to enhance motor control. Several features of the nucleocortical pathway make it of particular interest in the context of corollary discharge feedback in motor control. For example, by definition corollary discharge pathways must convey motor command signals. In delay eyelid conditioning, cerebellar nuclear neurons encode the conditioned eyelid closure, thus the nucleocortical pathway could, at least in this context, provide feedback of conditioned motor commands to the cerebellar cortex. However, whether under physiological conditions the nuclei generally encode motor command signals remains to be established. The experiments presented by Gao et al. (2016) promise to be the tip of the iceberg in what will be a fast-moving field uncovering the specific role of local feedback loops in motor control, sensory processing, and even cognitive function. At long last, opto- and chemogenetic tools allowing for neuronal branch-specific manipulations open up these long-standing questions to experimental investigation. Stay tuned.

REFERENCES Albus, J.S. (1971). Math. Biosci. 10, 25–61. Gao, Z., Proietti-Onori, M., Lin, Z., ten Brinke, M.M., Boele, H.-J., Potters, J.-W., Ruigrok, T.J.H., Hoebeek, F.E., and De Zeeuw, C.I. (2016). Neuron 89, this issue, 645–657. Houck, B.D., and Person, A.L. (2014). Cerebellum 13, 378–385. Houck, B.D., and Person, A.L. (2015). J. Comp. Neurol. 523, 2254–2271. Ito, M. (2006). Prog. Neurobiol. 78, 272–303.

Neuron 89, February 3, 2016 ª2016 Elsevier Inc. 429

Neuron

Previews Kalinovsky, A., Boukhtouche, F., Blazeski, R., Bornmann, C., Suzuki, N., Mason, C.A., and Scheiffele, P. (2011). PLoS Biol. 9, e1001013. Khodakhah, K. (2015). Nature 526, 326–327. Machado, A.S., Darmohray, D.M., Fayad, J., Marques, H.G., and Carey, M.R. (2015). Elife 4, e07892.

Marr, D. (1969). J. Physiol. 202, 437–470.

Sperry, R.W. (1950). J. Comp. Physiol. Psychol. 43, 482–489.

Ruediger, S., Vittori, C., Bednarek, E., Genoud, C., Strata, P., Sacchetti, B., and Caroni, P. (2011). Nature 473, 514–518.

Tolbert, D.L., Bantli, H., and Bloedel, J.R. (1978). J. Comp. Neurol. 182, 39–56.

Sommer, M.A., and Wurtz, R.H. (2008). Annu. Rev. Neurosci. 31, 317–338.

von Holst, E., and Mittelstaedt, H. (1950). Naturwissenschaften 37, 464–476.

Reducing Speed and Sight: How Adaptive Is Post-Error Slowing? Markus Ullsperger1,2,3,* and Claudia Danielmeier4 1Institute

of Psychology, Otto von Guericke University Magdeburg, 39106 Magdeburg, Germany for Behavioral Brain Sciences, 39106 Magdeburg, Germany 3Donders Institute for Brain, Cognition and Behaviour, Radboud University Nijmegen, 6525 HR Nijmegen, Netherlands 4School of Psychology, University of Nottingham, Nottingham NG7 2RD, UK *Correspondence: [email protected] http://dx.doi.org/10.1016/j.neuron.2016.01.035 2Center

After errors, decision boundaries change, which results in post-error slowing of decisions. Purcell and Kiani (2016) report simultaneously decreased sensitivity to sensory information counteracts post-error increases in accuracy. Early post-error adjustments reflect a general orienting reflex rather than goal-directed adaptation.

After a mistake, people often slow down their subsequent actions. This post-error slowing (PES) (Rabbitt, 1966) has been found for various cognitive activities, such as reaction time tasks, lexical decisions, and typewriting. Is it adaptive to slow down after errors? It would, if it reflected a more cautious response mode providing the basis for more accurate behavior in forthcoming trials (Ridderinkhof et al., 2004). In line with this assumption, theories of cognitive control suggest that after errors, motor responses are inhibited, for example by raising the threshold that motor cortex activity needs to exceed to elicit an overt action (Botvinick et al., 2001). Such speed-accuracy tradeoff would entail higher accuracy (i.e., lower error likelihood) on post-error trials. However, only a few studies have shown associations of PES with increased post-error accuracy. Often post-error changes in accuracy and reaction time were uncoupled or even in opposition (Ullsperger et al., 2014, for a review), which is incompatible with a speedaccuracy account.

Purcell and Kiani (2016) have addressed this issue in experiments in humans and monkeys performing a perceptual decision task. Participants had to make a saccade according to the perceived predominant motion direction of moving dots. After feedback indicating an error, reaction times in the following trial were prolonged while accuracy did not change; in other words, both species showed PES that cannot be attributed to a simple speed-accuracy tradeoff. Drift-diffusion models (DDM) that accumulate and integrate noisy evidence toward decision bounds have successfully explained behavioral and neuronal data in multiple perceptual decision-making studies. By fitting a DDM to the data, the authors found that PES can be explained by a combination of an increased decision bound and a reduction of the accumulator’s sensitivity to perceptual information. The reduced sensitivity thus counteracted the to-be-expected increase in accuracy associated with raised decision bounds. Based on the behavioral data alone, one cannot distinguish

430 Neuron 89, February 3, 2016 ª2016 Elsevier Inc.

different potential causes for the observed changes in decision bound and sensitivity. Recordings of single-neuron activity in the monkey’s lateral intraparietal cortex (LIP), a region known to host neurons whose firing dynamics represent evidenceaccumulation-to-bound decision signals related to saccades, were used to further specify the mechanisms of PES. LIP responses showed typical ramp-like increases of firing rates when monkeys chose to fixate the target in the neuron’s response field. Interestingly, the static features of the ramp, particularly firing at start and end point, did not differ between posterror and post-correct trials; rather, the dynamics of the ramp itself changed. Again, this could be explained best by two contributing factors: a stimulus-independent decrease in urgency and a stimulusdependent decrease in sensitivity. Thus, instead of a static decision bound, an urgency signal appears to lead to a stimulus-independent increase of firing as time of evidence accumulation progresses. This urgency signal, which in