Decision making

Decision making

Available online at www.sciencedirect.com Decision making Editorial overview Kenji Doya and Michael N Shadlen Current Opinion in Neurobiology 2012, 2...

156KB Sizes 3 Downloads 275 Views

Available online at www.sciencedirect.com

Decision making Editorial overview Kenji Doya and Michael N Shadlen Current Opinion in Neurobiology 2012, 22:911–913 For a complete overview see the Issue

0959-4388/$ – see front matter, # 2012 Elsevier Ltd. All rights reserved. http://dx.doi.org/10.1016/j.conb.2012.10.003

Kenji Doya Okinawa Institute of Science and Technology, Neural Computation Unit, 1919-1 Tancha, Onna, Okinawa 904-0412, Japan e-mail: [email protected] Kenji Doya received his PhD in 1991 at U. Tokyo. He was a research associate at U. Tokyo, U. C. San Diego, and Salk Institute before joining Advanced Telecommunications Research Institute International (ATR) in 1994. In 2004, he was appointed as the principal investigator of Neural Computation Unit, Okinawa Institute of Science and Technology (OIST) and started Okinawa Computational Neuroscience Course (OCNC) as the chief organizer. As OIST re-established itself as a graduate university in 2011, he became a professor and the vice provost for research. He serves as the co-editor in chief of Neural Networks from 2008. He is interested in understanding the functions of basal ganglia and neuromodulators based on the theory of reinforcement learning. He is also a triathlete and finished Ironman Hawaii in 2006 and 2011.

Michael N Shadlen Professor of Neuroscience, Investigator, Howard Hughes Medical Institute, Columbia University Medical Center, 1051 Riverside Drive, PI Kolb Annex 873, New York, NY 10032, USA e-mail: [email protected] Michael N Shadlen received his PhD in 1985 from UC Berkeley and his MD in 1988 from Brown University. He was professor of physiology and biophysics at the University of Washington until 2012. He is an Investigator of the Howard Hughes Medical Research Institute and professor of neuroscience at Columbia University. Shadlen studies the neural mechanisms that underlie decision making, visual perception, and timing. His studies combine neurophysiology, behavioral, and computational techniques. Shadlen is also a neurologist and a jazz guitarist.

www.sciencedirect.com

Strictly speaking, we do not make decisions, decisions make us. – Jose Saramago, All the Names Almost any interesting cognitive function can be framed as a decision of some sort, because once the function admits flexibility, contingency, or a provisional plan, it embraces elements of deliberation and commitment. Similarly, many behaviors which we do not regard as obviously cognitive — which proceed without conscious thought or ideation — also rely on decision processes. Even for the most basic, existential act of waking up from sleep or anesthesia or a minimally conscious state approximating coma, some unconscious process decides to engage the world. The decision to do this in response to the baby’s cry but not the traffic or thunder attests to the sophistication of even this most basic operation of our mental lives. Viewed from this perspective, the neurobiology of decision making offers a window on to brain mechanisms that support cognition, and it allows us to appreciate the rudiments of cognition in simpler behaviors. Thus to some, the central importance of this area of inquiry is its exposure of principles of cognitive neuroscience. To others, the neurobiology illuminates many intrinsically interesting issues pertaining to decision-making itself: how we operate as agents in a marketplace (Levy and Glimcher; Takahashi; Rangel and Clithero; Adams et al.), society (Seo and Lee; Adams et al.), or how we view ourselves as moral agents (Roskies). This issue of COIN reflects a wide range of topics, models, techniques and interests in the field of decision making. Its contributions span moral philosophy, neuroeconomics, learning, foraging, perception and motor control in humans, nonhuman primates, and rodents. The articles draw upon brain imaging, EEG, MEG, single neuron recordings, mathematics, and philosophy. Although it is by no means comprehensive, there ought to be something in this issue for just about anyone with an interest in the topic. Yet despite this diversity, there are a variety of themes that link many of these articles. Several contributions focus on perception and action. This is not surprising since sensory psychophysics has always incorporated a decision stage to connect perception to choice and response time. The neurobiology of perceptual decisions, as exemplified in Romo et al. and Churchland and Ditterich, shifts the emphasis from the representation of sensory data to the accumulation of evidence bearing on a proposition, interpretation or categorization. This accumulation and a host of other factors are bundled into a decision variable that is but a hair-trigger (or threshold crossing) away from commitment to a choice. Current Opinion in Neurobiology 2012, 22:911–913

912 Decision making

Most experimentally tractable decisions involve some kind of action to indicate a choice. Indeed, much progress in our understanding of the neural mechanisms of decision-making derives from the study of neurons involved in action selection, planning, or canceling. The success of this program reminds us that cognitive functions like decision making evolved as an elaboration on a simpler sensory-motor design of the brain. Yet, there is something unsettling about this: decisions do not feel like they are about actions but about propositions. A commitment to a proposition may be communicated through an action, but the decision itself seems to concern something more abstract. We see two sides of this issue batted about by Cisek and Dehaene and Sigman. Cisek expands on a Gibsonian theme of affordances — the idea that even perceptual qualities are to be understood in terms of the way we grasp, approach, sit upon, and so forth — whereas Dehaene and Sigman argue forcefully for the limitations of this perspective. They speculate that neurobiology of cognitive decisions will require more elaborate architecture than we can grasp from affordances and plans of action. An open question is whether something qualitatively different is required — language or a central executive that can broadcast a message to many possible brain targets — or whether the basic mechanisms and architecture will look like provisional plans of action, albeit with rule or strategy substituted for action, what might be termed decisions about decisions about decisions — ultimately — to act or engage or explore elsewhere (Rushworth et al.; Botvinick). An appealing idea is that similar computational principles are at play in a wide variety of decisions, whether they ultimately concern propositions, perceptions, actions, value preferences or strategies. For propositions and perceptual decisions, Bayesian theory reigns supreme, but Bayes is not just about inference. As explained by Drugowitsch and Pouget, the Bayesian formulation countenances costs and therefore has the power to link perceptual-based and value-based decisions. Thus it is reassuring that the formalisms in Rangel and Clithero and Adams et al. which deal with value-based decisions should resemble those that have proven so useful in perceptual decisions [1]. Once costs and gains are brought into the fold, even social decisions seem like they might involve the same types of neural computations. Interestingly, Seo and Lee raise the possibility that social cues can enter decision making both as costs/gains and as contextual cues bearing on the type of decision to engage in the first place. The Bayesian formulation entices us to glimpse the principles behind sublime cognitive capacities, but they also expose parallels with the ostensibly mundane neural processes devoted to moving our bodies (Wolpert and Landy). It appears that the neurobiology of decision making has more than a superficial connection to the way the motor system chooses and implements Current Opinion in Neurobiology 2012, 22:911–913

movement trajectories, and how the motor system updates commands based on noisy proprioceptive and visual feedback it receives in flight. A more obvious decision for the motor system is to act in the first place (Desmurget and Sirigu) and to cancel an action that has been planned (Schall and Godlove). The contribution from Desmurget and Sirigu describes fascinating work on the neurobiology of volition, a topic that intersects traditional philosophical problems of free will and moral responsibility, as discussed in Roskies. Whereas it seems secure to posit that many types of decisions draw on common computational principles, it is less clear whether the same neural processes are involved. One possibility is that single neurons convert all elements of a decision — evidence, value, social costs, elapsed time — into a common currency of neural activity. This perspective is expressed forcefully in Levy and Glimcher based on human fMRI, and it is the subtext of many contributions in the issue. It may be challenging to reconcile this view with the compelling division of functions proposed in Rushworth et al.. This article provides a wonderful example of the type of hierarchical-butintertwined architecture required to make decisions about something at the moment, but to retain — for learning or further exploration or both — unchosen options and their associated values. As the field uncovers computational principles, it will be important to dig deeper into their mechanisms. How do neural circuits give rise to computations such as integration — which underlies evidence accumulation — valuation, setting bounds, detecting bound crossings and so forth. An immediate challenge is to model these processes with realistic neural elements. It is truly a challenge because there is so little known about how neurons or circuits of neurons can display firing rates that reflect the integration of evidence over time scales that are orders of magnitude greater than the membrane time constant. Contributions from Wang and Cain and SheaBrown expose the challenges and potential solutions to the neural integrator. In our view, this is one of the most important problems in cognitive neuroscience. An overly leaky neural integrator is the likely culprit behind the neurology of confusion, wandering attention, poor concentration and many other disorders of higher brain function. And an unstable integrator — runaway selfexcitation — may well be at the root of tics and epilepsy. The mechanisms involved in valuation and processing rewards may be more tractable. Clark et al. show in rodents that there are two separate but related systems for valuation: a mesolimbic dopamine system for reactive decisions and an orbitofrontal system for cognitive decisions. Takahashi demonstrates using PET with radio ligands that there are links between risk-seeking and dopamine and loss-aversion and norepinephrine. These www.sciencedirect.com

Editorial overview Doya and Shadlen 913

same neural systems are also believed to underlie the positive reinforcement that supports learning. Indeed, the theory of reinforcement learning (RL) has played an important role in many studies of decision making. RL relies on neural computations that would associate reward with behavioral ‘state’ and the menu of possible actions, but as argued by Dayan the translation of these theoretical entities into neural representations remains unclear. Even the definition of reward, which is arguably the most straightforward, may be nuanced once one acknowledges that the acquisition of novel information can be a reward in itself. While many decisions are reactive and almost automatic, we often spend time and energy deliberating about what can happen following different options and how good or bad they can be. Doll et al. review the dichotomy of model-based and model-free RL and highlight enigmatic overlaps of the neural substrates of the two systems. Rushworth et al. review the functions of different regions of the prefrontal cortex and suggest parallel valuation mechanisms involved in searching and deciding. Another theme that arises when synthesizing the variety of topics in this issue concerns the stochastic nature of choice. Neural mechanisms underlying perceptual decisions operate deterministically, although they appear stochastic because noise in the stimulus and brain lead to the appearance of probabilistic behavior. This noise is real and unavoidable, but when an animal chooses A over B on 70% of trials, it does not mean that the brain computed P(A is better) = 0.7 and rendered a random binary outcome that matches that probability. A proper understanding of the mechanism should explain why the decision variable corresponds to ‘A is probably better than B’ on 70% of trials. It is interesting to read contributions in this issue and wonder whether the theories seem to imply a matching of probabilities or the maximizing (with noise) that is the mainstay of signal detection theory and its extension to bounded evidence accumulation (e.g. diffusion models). It is interesting to read the piece on free will by Roskies with this same dialectic in mind. There is much missing from this issue. Invertebrate and rodent models of perceptual decision making are underrepresented despite rapid progress in this field [2–4]. This issue contains little information about the mechanism for establishing a termination criterion and sensing that this criterion is met by neurons that represent the decision

www.sciencedirect.com

variable. This seems like a tractable problem [5,6], but the fact is, we do not know which brain structures are responsible. Further, we do not know how — once a decision has terminated — neurons activate other neural structures to initiate a response, stop the integration process, or initiate another decision process. For example, we do not know how contextual cues lead to a decision to treat information about color, say, as a evidence bearing on where to look. We might postulate mechanisms that accumulate evidence and achieve some threshold level of activation. But after that, what are the neural events that bring a circuit into play and configure it so that the relevant evidence bears on the appropriate set of possible actions? These circuit-selection and circuit-configuration problems are likely to involve feedback and corticothalmo-cortical pathways that instantiate what might be termed a ‘decision to engage’ [7]. We hope the articles in this issue will excite readers by highlighting the progress of this burgeoning field and by drawing attention to the challenges and mysteries that lie ahead. We hope readers will delight in the insights while yearning for deeper understanding of the neural mechanisms. Decision-making brings neuroscience into the domain of ethics, philosophy and the law and thus strikes at what it is that makes us human. And if this sounds scary, fear not, for explaining is not the same thing as explaining away. Rather, the neurobiology illuminates the mechanisms that, as Saramago put it, make us who we are.

References 1.

Gold JI, Shadlen MN: The neural basis of decision making. Annu Rev Neurosci 2007, 30:535-574.

2.

Faumont S, Lindsay T, Lockery S: Neuronal microcircuits for decision making in C. elegans. Curr Opin Neurobiol 2012, 22:580-591.

3.

Uchida N, Kepecs A, Mainen ZF: Seeing at a glance, smelling in a whiff: rapid forms of perceptual decision making. Nat Rev Neurosci 2006, 7:485-491.

4.

Harvey CD, Coen P, Tank DW: Choice-specific sequences in parietal cortex during a virtual-navigation decision task. Nature 2012, 484:62-68.

5.

Lo CC, Wang XJ: Cortico-basal ganglia circuit mechanism for a decision threshold in reaction time tasks. Nat Neurosci 2006, 9:956-963.

6.

Machens CK, Romo R, Brody CD: Flexible control of mutual inhibition: a neural model of two-interval discrimination. Science 2005, 307:1121-1124.

7.

Shadlen MN, Kiani R: Consciousness as a decision to engage. In Characterizing Consciousness: From Cognition to the Clinic? Research and Perspectives in Neurosciences. Edited by Dehaene S, Christen Y. Berlin/Heidelberg: Springer-Verlag; 2011:27-46.

Current Opinion in Neurobiology 2012, 22:911–913