Role of Dopamine Neurons in Reward and Aversion: A Synaptic Plasticity Perspective

Neuron

Review Role of Dopamine Neurons in Reward and Aversion: A Synaptic Plasticity Perspective Marco Pignatelli1 and Antonello Bonci1,2,3,* 1Intramural

Research Program, Synaptic Plasticity Section, National Institute on Drug Abuse, Baltimore, MD 21224, USA H. Snyder Neuroscience Institute 3Department of Psychiatry Johns Hopkins University School of Medicine, Baltimore, MD 21205, USA *Correspondence: [email protected] http://dx.doi.org/10.1016/j.neuron.2015.04.015 2Solomon

The brain is wired to predict future outcomes. Experience-dependent plasticity at excitatory synapses within dopamine neurons of the ventral tegmental area, a key region for a broad range of motivated behaviors, is thought to be a fundamental cellular mechanism that enables adaptation to a dynamic environment. Thus, depending on the circumstances, dopamine neurons are capable of processing both positive and negative reinforcement learning strategies. In this review, we will discuss how changes in synaptic plasticity of dopamine neurons may affect dopamine release, as well as behavioral adaptations to different environmental conditions falling at opposite ends of a saliency spectrum ranging from reward to aversion. The ventral tegmental area (VTA) is a heterogeneous midbrain structure that plays a major role in regulating different adaptive brain functions related to reward and motivation processing. It is predominantly composed of dopaminergic neurons (55%– 65%), while the rest are mainly GABAergic (30%) and a small portion are glutamatergic cells (5%) (Hnasko et al., 2012; Steffensen et al., 1998). Here, we will focus on the main neuronal population of the VTA: dopamine neurons. Dopamine (DA) release from the VTA efferent projections takes place in a broad range of structures such as prefrontal cortex (PFC), nucleus accumbens (NAc), amygdala, and hippocampus (Barrot, 2014). One key feature of this dopaminergic pathway, known as the meso-cortico-limbic circuit, is its ability to reinforce natural rewarding behavior and attribute motivational salience to otherwise neutral environmental stimuli (Berridge and Robinson, 1998; Smith et al., 2011). Here, we propose that two forms of synaptic plasticity, known as long-term potentiation (LTP) and long-term depression (LTD), occurring onto DA neurons, represent a key fundamental mechanism capable of affecting reward and aversion processes, as well as changes in motivational salience accomplished via effects on VTA DA release exerted in the target areas. LTP and LTD are widely recognized as cellular substrates for learning and memory in the brain (Anggono and Huganir, 2012; Bliss and Collingridge, 1993; Collingridge et al., 2010). Since the discovery of synaptic plasticity, these forms of cellular memory have been the subject of intensive investigation in several brain regions that support cognitive processes, such as hippocampus and cortex. The functional significance of synaptic plasticity in the aforementioned structures has been extensively characterized (Bliss et al., 2014). In contrast, the cellular events underlying synaptic plasticity in DA neurons of the VTA, and the role of LTP and LTD in shaping DA-dependent behaviors, are still poorly understood.

In this review, we will focus on the significance of LTP and LTD within DA neurons of the VTA, with respect to neural processing and coding of both rewarding and aversive stimuli. First, we will provide an overview of the role of DA in learning and memory processes that occur when organisms adapt their behavior on the basis of associations with reward and aversion. Next, we will present recent findings that shed light on the circuit connectivity and functional heterogeneity of the VTA, highlighting the complexity and the diversity of a myriad of cellular regulators of reward and aversion. Finally, we will review the general rules governing synaptic plasticity of VTA DA neurons and the potential impact of a defined synaptic state in terms of DA release, or co-release capability and behavioral flexibility. Encoding and Modulation of Appetitive and Aversive Learning in VTA Dopamine Neurons Dopamine and Goal-Directed Behaviors DA is implicated in the execution of goal-directed behaviors and in the regulation of reward-related vigor (Beierholm et al., 2013; Ranaldi, 2014). These concepts have emerged mostly from behavioral studies showing that DA antagonists are capable of attenuating motivation to perform an action, even before motor responses occur (Wise, 2004). Interestingly, conditioned learning precedes and guides the performance of an instrumental act, as well as being DA-dependent (Darvas et al., 2014). In particular, fundamental goal-directed behaviors such as seeking food or water are not innate, but learned (Changizi et al., 2002). It is widely believed that stimuli provided by the surrounding environment can be selectively reinforced and thus guide the proper execution of directed and motivated behavior, even for a naive organism (Hall and Mayer, 1975; Johanson and Hall, 1979). Once stimulus-reward associations are formed, they can last long after the appropriate motivational drive states are present (Balleine and Dickinson, 1992; Mendelson, 1966; Morgan, 1974). Nevertheless, behavioral effects to conditioned responses are flexible and subject to extinction if paired with Neuron 86, June 3, 2015 ª2015 Elsevier Inc. 1145

Neuron

Review unrewarded trials (Wise et al., 1978) or if the experimental setting occurs without a supportive drive state (Mendelson, 1966; Morgan, 1974). In general terms, consumption of reward (food, mating, and drugs of abuse) produces a hedonic state that is thought to initiate learning processes devoted to consolidate specific goals or cues that predict availability of a reward or actions that permit its consumption (Wise and Kiyatkin, 2011). The overall concept is to promote a motivational state to allow an organism to purposely select the best way to achieve the successful procurement of its essential needs. Based on this assumption, the brain seems to store and estimate the value of certain actions based on prior experience. In other words, an animal uses stored values to predict and plan actions, based on past events that involved a wide range of variables, including reward and aversion (Montague et al., 2004). Dopamine Neuron Activity and Prediction Error The brain is wired to predict future outcomes. This fundamental extraordinary feature occurs in response to a myriad of discrete stimuli or situations, and it is mainly based on prior experience and specific patterns of response. However, when the outcome differs from what is expected, a ‘‘prediction error’’ occurs. Such prediction error, mainly supported by dopaminergic activity, is generally used by the brain to refine and optimize its future responses, as well as learn new behavioral strategies for all needs that are essential to ensure survival (Cohen et al., 2012; Hollerman and Schultz, 1998; Schultz, 2006; Schultz et al., 1993, 1997, 1998). The neural language of DA neurons at resting condition exhibits a consistent tonic pattern of firing. On top of this tonic activity, a brief phasic burst of spike activity can appear and be superimposed based on prior history of reward experience. Depending on the circumstances, these neurons are capable of coding three different functional states: a reward that is ‘‘as expected,’’ ‘‘better than expected,’’ or ‘‘worse than expected.’’ Reward falling in the category of ‘‘as expected’’ produce tonic activity, reward that are ‘‘better than expected’’ result in phasic bursts signal, and a pause in firing parallels reward that are ‘‘worse than expected.’’ Interestingly, a burst of action potentials is capable of releasing more DA in specific projection areas than the same number of spikes organized in spaced action potentials do (Schultz, 1986). These cellular phenotypes recapitulate the classic concept that, in mammals, single spikes firing in DA neurons are thought to play a permissive role in initiating movements (Romo and Schultz, 1990), while burst firing is correlated with arousal states and motivation. More importantly, mimicking a ‘‘better than expected’’ reward prediction error signal by optogenetic activation of DA neurons during the occurrence of reward delivery was sufficient to establish a long-lasting increase in reward-seeking behavior caused by cue-reward learned associations (Steinberg et al., 2013). Many studies have shown that burst firing activity is highly regulated by glutamatergic afferent inputs (Grace and Bunney, 1984a, 1984b; Grace et al., 2007). Not surprisingly, another substantial body of evidence supports a role for glutamate in learning mechanisms related to reward adaptive behaviors (Kauer, 2004; Lu¨scher and Malenka, 2011; Schultz, 2011). In fact, glutamatergic afferents reaching the VTA are susceptible to plasticity during reward and drug-associated learning 1146 Neuron 86, June 3, 2015 ª2015 Elsevier Inc.

(Chen et al., 2008; Stuber et al., 2008; Ungless et al., 2001). Using behavioral sensitization, a phenomenon that is based on escalating behavioral responses to repeated exposure to a drug, it was discovered that N-methyl-D-aspartate receptor (NMDAR) antagonists delivered into the VTA prevented the development of this behavioral phenomenon. In addition, alpha-amino-3-hydroxy-5-methyl-4-isoxazolepropionate receptor (AMPAR) blockade is capable of affecting the maintenance of behavioral sensitization (Jackson et al., 2000; Wolf, 1998). Therefore, pharmacological compounds administered within the VTA that antagonize the activity of AMPARs and NMDARs severely disrupt classical Pavlovian associative learning (Harris and Aston-Jones, 2003; Kelley et al., 2003; Stuber et al., 2008). The functional interpretation of these findings opens up several possible cellular scenarios, including disruption of cellular processes related to synaptic plasticity, intrinsic properties, or burst firing modalities of DA neurons. In fact, both LTP and the ability to generate burst firing rely on the integrity of NMDAR activity in DA neurons of the VTA. It is tempting to speculate that in these experimental settings a glutamatergic antagonist could affect both processes. Moreover, burst firing of DA neurons in the substantia nigra can play a permissive role in the proper occurrence of NMDA synaptic plasticity (Harnett et al., 2009). Presumably, this phenomenon might happen also for DA neurons of the VTA, thus engaging the first fundamental step toward the expression of the AMPAR mediated LTP during cocaine exposure (Argilli et al., 2008), or potentially during the occurrence of natural reward-related association. Aversion: The Other Face of Dopamine DA neurons are capable of encoding both reward and aversion. There is solid evidence that several types of rewarding stimuli, like food (McCutcheon et al., 2012; Roitman et al., 2004), intraoral infusion of a sucrose based solution (Roitman et al., 2008), drugs of abuse (Aragona et al., 2009; Stuber et al., 2005a, 2005b) and cues predicting their delivery (Daberkow et al., 2013; Phillips et al., 2003), all strongly, albeit transiently, increase the probability of DA release events. Given that animals are willing to perform tasks that lead to the consumption of reward, but are generally reluctant to deal with aversive stimuli (Miller and Hunt, 1944), it will be crucial to unequivocally differentiate between opposite hedonic evaluations, like reward and aversion, as a direct effect of selective coding occurring in DA neurons. There are several approaches that can be used to mimic aversive situations in experimental animal models. The methodologies involve a variety of conditions ranging from acute or chronic exposure to aversive stimuli (foot shocks, forced swimming test, hind paw injection of formalin, social defeat stress, fear conditioning) to stereotypical orofacial responses evoked by intraorally delivered solutions (Berridge, 2000; Chaudhury et al., 2013; Fadok et al., 2009; Friedman et al., 2014; Lammel et al., 2012; Tye et al., 2013). From these studies, a role for DA and DA neurons, in the modulation and coding of aversive life experiences clearly emerges. Importantly, DA neurons are active players during fear-related experiences (Fadok et al., 2010), facilitating the early stabilization of the memory trace of fear-related learning. Additionally, VTA DA neurons undergo activity-dependent plasticity mechanisms (Gore et al., 2014) and actively participate in

Neuron

Review the modulation of amygdala action potentials during Pavlovian fear conditioning and stress (Inglis and Moghaddam, 1999; Rosenkranz and Grace, 2002). Along these lines, a mouse model carrying a genetic inactivation of functional NMDA receptors on DA neurons displays a maladaptive conditioning to a cue predicting an aversive outcome, and concurrently this impairment in contingency alertness is associated with the development of a generalized anxiety-like phenotype (Zweifel et al., 2011). Consistent with a direct involvement of DA neurons in the coding and modulation of emotional memory formation and adaptive behavior, bidirectional control (via optogenetic inhibition or excitation) of genetically targeted midbrain DA neurons rapidly modulates prototypical symptoms caused by chronic stress (Chaudhury et al., 2013; Friedman et al., 2014; Tye et al., 2013). The latter approaches can directly test the involvement of the DA system in the behavioral control of aversion. However, optogenetic manipulations, like the ones performed in the previous studies, can indeed rescue phenotypic alterations in animals that underwent behavioral manipulation, such as depressive-like phenotypes, but such rescue first requires the induction of a non-physiological behavior (Friedman et al., 2014; Tye et al., 2013). In summary, to gain full knowledge about the exquisitely intricate physiology of DA neurons, it will be important in the future to compare reward and aversive responses within an ‘‘intact’’ mesolimbic system. Furthermore, the many groundbreaking contributions, briefly discussed in this review, shed light toward a better understanding of how, and where, reward and aversion processing takes place within the CNS. Scientists are not far from giving a precise identity and role to the various neuronal sub-populations located within the VTA, that are engaged in the coding of reward, aversion, or in facilitating contingency awareness, with the ultimate goal to identify potential targets for therapeutic intervention. Cracking the Dopamine Code with New Approaches to Understand Neuronal Heterogeneity DA neurons located within the VTA are clustered into specific anatomical niches and project to a single target region such as nucleus accumbens, prefrontal cortex, hypothalamus, basolateral amygdala, lateral habenula, pallidum, and bed nucleus of stria terminalis (Haber, 2014). The VTA is a very heterogeneous region in which dopaminergic, GABAergic and glutamatergic neurons intermingle. As previously mentioned in the introduction, dopaminergic neurons are the most abundant population (65%), followed by GABA neurons (30%), while glutamatergic neurons (5%) are the most underrepresented population (Morales and Root, 2014; Yamaguchi et al., 2007, 2011). Such anatomical complexity is paralleled by a functional and, perhaps, vesicular promiscuity, since dopaminergic neurons can co-release glutamate or GABA. The most powerful approaches used to disentangle the anatomical and physiological intricacies of the VTA are derived from the proper combination of transgenic technologies, optogenetics, electrophysiology, and immunocytochemistry. For example, the use of retrograde tracing allowed for a detailed functional identification of DA neurons residing within the VTA and projecting to NAc, PFC and basolateral amygdala (BLA)

(Lammel et al., 2011; Margolis et al., 2008). Using this approach, it has been shown that VTA neurons projecting to the lateral shell of NAc reside within the lateral portion of the VTA and exhibit large Ih currents, a hyperpolarization-activated cationic current, which represents an electrophysiological feature traditionally used to identify DA neurons. In addition, a recent study found that glutamatergic synapses display a low AMPAR/NMDAR ratio at resting state. Conversely, DA neurons projecting to the basolateral amygdala, PFC, and core of the NAc originate from the medial portion of the VTA. Moreover, these neurons do not show Ih currents, and the glutamatergic synaptic configuration displayed a high AMPAR/NMDAR ratio at resting conditions. Interestingly, salient appetitive events, like a passive exposure to cocaine, induced synaptic potentiation mostly within DA neurons projecting to the lateral shell, while an aversive event (hind paw injection of formalin) was capable of inducing synaptic potentiation, mostly within PFC projecting neurons (Lammel et al., 2011) (Figure 1). These distinct subsets of neurons receive different afferent synapses. NAc shell projection neurons receive excitatory afferents, both glutamatergic and cholinergic inputs, from the laterodorsal tegmentum (LDTg) and inhibitory afferents from the rostromedial tegmental nucleus (RMTg). In the case of dopaminergic neurons projecting to the PFC, the excitatory afferents are provided by glutamatergic input emanating from the lateral habenula. The lateral habenula also sends glutamatergic synapses onto GABAergic neurons of the RMTg that project back to NAc shell-projecting neurons. The behavioral implication of this intricate circuit has been determined in vivo by using optogenetic stimulation of specific presynaptic terminals, allowing an examination of the functional significance of input specificity, relative to the VTA. Optical stimulation of channelrhodopsin-2 expressing terminals in the VTA produced a robust conditioned place preference (CPP), a form of Pavlovian conditioning used to measure the motivational effects of experiences. On the other hand, light stimulation of the lateral habenula and RMTg terminals in the VTA produced conditioned place aversion (Lammel et al., 2012). Providing proper control of DA neuron firing rates in VTA is essential for the physiological expression of reward processing and motivational salience. The balance between excitatory and inhibitory drive has emerged as a fundamental mechanism for the control of firing rates and activity patterns of DA neurons. Glutamatergic input via AMPA and NMDA receptors expressed on DA neurons can shape DA release events. GABAergic synapses are also capable of regulating firing activity via activation of GABAA receptors located on DA neurons, thereby profoundly reducing their firing rate. Interestingly, there are several glutamatergic and GABAegic inputs, in addition to those already mentioned, orchestrating the activity of dopaminergic neurons of the VTA. Activation of pyramidal neurons of the PFC in vivo can induce bursting of VTA DA neurons (Tong et al., 1996). Lateral hypothalamus (Kempadoo et al., 2013) and BNST (Jennings et al., 2013) can also provide glutamatergic inputs. GABA neurons are located within the VTA, as well. Coordinated activity of these GABA cells, via selective optogenetic stimulation, can support conditioned place aversion and disrupt reward consumption (Tan et al., 2012; van Zessen et al., 2012). Surprisingly, this Neuron 86, June 3, 2015 ª2015 Elsevier Inc. 1147

Neuron

Review

Figure 1. Synaptic Plasticity and Dopaminergic Activity in Response to Rewarding or Aversive Processes Rewarding experiences like a passive exposure to cocaine are capable of producing synaptic potentiation mostly within VTA DA neurons projecting to NAc lateral shell (left) while aversive events, hind paw injection of formalin, cause synaptic potentiation mostly within PFC projecting neurons (right). These distinct subsets of DA neurons receive different afferents. NAc shell projection neurons receive afferents from LDTg and RMTg and modulate reward processing, while PFC projecting neurons receive excitatory afferents provided by LHb and they are implicated in aversion. The observed enhanced expression of AMPA receptors occurring onto specific subsets of DA neurons, respectively supporting reward and aversion, might be a cellular mechanism for burst generation and concurrent increased DA release.

population of cells can also innervate NAc. In addition, it has been recently demonstrated that these cells synapse onto accumbal cholinergic interneurons (CINs). More importantly, optogenetic manipulation of these terminals can induce a pause in the firing pattern of CINs in behaving mice and enhanced discrimination of stimulus previously paired with an aversive outcome (Brown et al., 2012). Thus, these data support the intriguing idea that GABA neurons within the VTA can ‘‘teach’’ NAc how to respond and promote actions, in regard to salient events. Given the functional importance of GABAergic neurons located within the VTA, it is important to note that they receive inhibitory input from medium spiny neurons of the NAc (Bocklisch et al., 2013), BNST (Jennings et al., 2013), and ventral pallidum (Hjelmstad et al., 2013). Interestingly, it has been discovered that synapses between medium spiny neurons expressing DA receptor type 1 of the NAc projecting to GABA neurons of the VTA can undergo synaptic plasticity, namely potentiation, after repeated in vivo experiences with cocaine. This synaptic potentiation is also capable of occluding the occurrence of homosynaptic, inhibitory long-term potentiation (Bocklisch et al., 2013). At the circuit level, this plastic adaptation 1148 Neuron 86, June 3, 2015 ª2015 Elsevier Inc.

induced a reduction in the activity of GABAergic neurons, thus causing a subsequent disinhibition of DA neurons. The circuits and neuronal population described above belong to anatomically and molecularly defined neurons. It is important to emphasize that beside the biological complexity conferred by the promiscuous release of neurotransmitter, the existence of a ‘‘hybrid’’ population of neurons within the VTA emerges as a very prominent reality that strongly determines the intrinsic circuitry of this system. These populations of cells reside within the VTA and project to the lateral habenula, expressing the prototypical markers of DA neurons, but are unable to release detectable levels of DA in the target region. Surprisingly, they can release GABA in the LH and, accordingly, promote reward-seeking behaviors (Stamatakis et al., 2013). It is remarkable to point out that, during the last few years, scientists have been able to add several fundamental anatomical and functional details to an already complicated and heterogeneous region. We are now facing a fascinating era in which we are combining together all these pieces of information, with the aim of clarifying and understanding the rules governing the reward circuits in physiological and pathological states.

Neuron

Review General Rules Governing Synaptic Plasticity in DA Neurons In vitro brain slice electrophysiology has been an invaluable tool to address synaptic level changes in the function of glutamatergic synapses occurring in the brain, following different circumstances. Several electrophysiological studies have revealed that learning induces long-lasting changes in the synaptic strength of central glutamatergic synapses. As discussed earlier, this activity-dependent modification of the efficacy of synaptic transmission is generally termed synaptic plasticity. It is widely recognized that this plasticity mechanism is a fundamental cellular candidate for the acquisition and maintenance of learning and memory traces in the brain. In the mammalian brain, given the diversity of the properties and functions of each region, several different forms of synaptic plasticity have been described. In this section, we attempt to provide a broad overview of the mechanisms of synaptic plasticity that have been described at excitatory synapses in the VTA. Glutamatergic synapses reaching the VTA are capable of expressing plasticity properties. Indeed, several studies have found that both LTP and LTD caused by evoked AMPAR currents can be induced in the VTA (Bonci and Malenka, 1999; Thomas et al., 2000). Older induction protocols, adapted from the hippocampus literature, held DA neurons at a depolarized potential for a long period of time, while stimulating afferents at high or low frequencies. Despite the fact that the older induction protocols were capable of producing plasticity, they were unable to recapitulate biologically relevant patterns of synaptic activity. Theoretically speaking, perhaps a more physiologically relevant induction protocol is represented by spike-timing–dependent plasticity (STDP). In STDP, the induction of plasticity is governed by specific rules that organize the order and precise temporal interval between pre and post-synaptic spikes, resembling the physiological activity of the neuron (Feldman, 2012). Based on these assumptions, the use of STDP has emerged in the VTA plasticity field over the last decade. Interestingly, both classical and STDP induction protocols are capable of producing LTP and, similarly to what has been reported in other brain regions (i.e., hippocampus), rely on NMDAR activation, as well as a subsequent increase in postsynaptic calcium. Conversely, LTD in the VTA is NMDAR independent. In fact, bath application of APV, the NMDA receptor antagonist, is not able to prevent lowfrequency stimulation-induced LTD (LFS-LTD). Surprisingly, however, Ca2+ signaling in the postsynaptic compartment is essential for the proper expression of LFS-LTD; in fact having the Ca2+ chelator BAPTA in the recording pipette prevents LFSLTD. Furthermore, bath application of DA or the D2 receptor agonist quinpirole blocks LFS-LTD (Thomas et al., 2000). The molecular mechanisms underlying this synaptic process are still poorly understood. Certainly, the functional recruitment of dendritic potassium channels by DA, or by quinpirole, is unlikely since these experiments were performed with TEA and cesium in the whole cell pipette, both potassium channel blockers. It is very intriguing that activation of mGlu1 (Group I metabotropic glutamate receptors) is also capable of producing LTD in the VTA and, more importantly, mGlu and LFS-LTD coexist and do not occlude each other, thus supporting the idea that they are mechanistically distinct (Bellone and Lu¨scher, 2005).

mGlu1-mediated LTD is also supported by elevation of intracellular Ca2+ levels, since having BAPTA in the pipette completely prevents LTD (Bellone and Lu¨scher, 2005). The expression mechanism of mGlu1-LTD in the VTA is quite peculiar. In fact, instead of relying on a reduced number of postsynaptic AMPA receptors, it relies on an exchange of AMPA receptors with a lower conductance, those that contain the GluA2 subunit (Bellone and Lu¨scher, 2005). We will discuss the importance of AMPAR composition in further detail below. Ultimately, the molecular players regulating the increase or decrease in synaptic strengths observed in LTP and LTD, respectively, are the AMPARs. This class of receptors can undergo differential trafficking or a change in subunit composition at the cell surface, depending on the type of synaptic plasticity occurring. In fact, depending on the synaptic mechanism engaged for the synaptic potentiation or depression of AMPAR-mediated excitatory synaptic transmission, we could imagine the following molecular scenarios: an increase or decrease in sensitivity, number of receptors, or changes in the expression of AMPAR subunits. AMPARs are fundamental ionotropic transmembrane receptors mediating fast excitatory synaptic transmission in the CNS. Assembly of these receptors requires four subunits (GluA1–GluA4), which can form hetero- or homomeric complexes (Gan et al., 2015). AMPARs are thought to exist as heteromeric complexes containing both GluA2/3 and GluA1 subunits. At resting state, the vast majority of AMPARs contain the GluA2 subunit, forming heteromeric receptors with GluA1 or GluA3. However, GluA2-lacking AMPARs can form, in which the tetrameters are composed of GluA1/1 or GluA1/3 AMPARs, in other words ‘‘GluA1-type.’’ GluA1-type AMPARs are permeable to calcium, which can be a permissive signaling event for the establishment of plasticity, while GluA2 containing receptors are Ca2+ impermeable (Isaac et al., 2007). Our understanding of how AMPAR subunit composition is regulated has been aided by both biochemical (crosslinking methods) and electrophysiological techniques. The use of crosslinking assays allowed the resolution of AMPAR subunit type present on the cell surface and also revealed the phosphorylation state of specific sets of AMPARs subunits. Importantly, depending on their phosphorylation state, AMPARs can increase or decrease channel conductance. Electrophysiological techniques have also taken advantage of a specific feature of GluA2-lacking AMPARs: that functional block occurs when intracellular polyamines are applied at a positive potential. This biophysical feature parallels a very unique inward rectifying current-voltage relationship that can be used to determine the presence or absence of GluA2-lacking AMPARs. Biochemical and electrophysiological approaches can then be used in combination, via the use of specific subunit-selective peptide antagonists. The elegance of the use of the latter approach resides in the possibility to dissect out the relative contribution of GluA1 and GluA2 AMPAR subunits in both in vitro and in vivo experimental conditions. A powerful and convenient behavioral framework for studying synaptic plasticity and the electrophysiological impact, in terms of behavioral adaptation, has been offered by the drug addiction field. Evidence that drugs of abuse are capable of producing Neuron 86, June 3, 2015 ª2015 Elsevier Inc. 1149

Neuron

Review Figure 2. Cocaine Exposure ‘‘Hijacks’’ Synaptic Plasticity Rules of VTA Dopamine Neurons Synaptic plasticity involves several orchestrated changes of synaptic efficacy supporting experience-dependent modifications of brain function. The increased efficacy of glutamatergic synapses onto DA neurons observed after cocaine exposure is supported by a redistribution of AMPA (GluR2 Lacking) and NMDA receptors (GluN2B and GluN3A containing). The observed changes in AMPA and NMDA subunit composition lead to changes in Ca2+ permeability that are capable of affecting synaptic rules governing activitydependent plasticity. As a consequence, antiHebbian pairing protocols, not sufficient to induce LTP in naive conditions, can produce LTP after cocaine exposure. Interestingly, both in vitro and in vivo activation of mGluR1 can reverse cocaineinduced synaptic adaptation via synaptic incorporation of GluR2 containing AMPA receptors. Whether activation of mGluR1 is also capable of reversing NMDA receptors redistribution is still unknown.

plasticity within DA neurons of the VTA was shown with a single non-contingent injection of cocaine. Our group discovered an increased AMPAR/NMDAR ratio, a ‘‘surrogate’’ measure of LTP, 24 hr after the treatment (Ungless et al., 2001). Additionally, it was demonstrated that excitatory synapses were not further strengthened, supporting the idea that the potentiation, and the subsequent occlusion phenomena, was due to the fact that cocaine-induced LTP shared the same mechanism as physiological LTP. This plastic neuroadaptation was observed 24h after the injection and persists for up to 5 days. Interestingly, several other salient events like food and sucrose, delivered after self-administration procedures, are capable of producing the same neuroadaptation (Chen et al., 2008). The fundamental difference is revealed by the duration of the synaptic potentiation. Plasticity triggered by sucrose, food, or pure appetitive Pavlovian learning and memory mechanisms is a very transient phenomenon that decays rapidly over time. In the case of cocaine, and particularly in animals trained to self-administer the drug, this LTP-like state persists for several weeks. Interestingly, cocaine-induced synaptic potentiation of AMPARs is mediated by prior recruitment of NMDA receptors. In fact, in vivo treatment with MK-801, the noncompetitive antagonist of the NMDARs, prevented the occurrence of synaptic plasticity. Notably, the increase in AMPAR/NMDAR ratio parallels an increased sensitivity to polyamines, which is indicative of the presence of GluA2-lacking AMPARs (Argilli et al., 2008; Bellone and Luscher, 2006). In addition, both in vitro and in vivo activation of mGlu1 are capable of reversing the early synaptic effects triggered by cocaine exposure, namely GluA2-lacking AMPARs, via de novo synthesis and synaptic incorporation of GluA2 containing receptors (Bellone and Luscher, 2006; Mameli et al., 2007) (Figure 2). 1150 Neuron 86, June 3, 2015 ª2015 Elsevier Inc.

Another level of complexity has been revealed by photo-uncaging glutamate at single synapses of DA neurons in animals that underwent in vivo cocaine exposure. The experiments showed reduced NMDAR-mediated currents (Mameli et al., 2011). Reduction in NMDAR-mediated currents can concurrently support and greatly amplify the observed increased in AMPA/NMDA ratio. It is important to point out that NMDARs are fundamental players in the induction phase of LTP and in several other integrative cellular processes. Thus, their expression or subunit composition can be a crucial component for the proper tuning of synaptic plasticity. DA neurons of animals exposed to cocaine display a larger sensitivity to the selective GluN2B inhibitor ifenprodil, compared to the selective GluN2A inhibitor zinc (Yuan et al., 2013). The latter discovery leads to the consideration of the ratio of GluN2A/GluN2B receptors at synapses. It is important to appreciate that switches in NMDAR subunit composition can potently regulate Ca2+ entry into the post-synaptic compartment and, more importantly, can affect the thresholds required for inducing synaptic plasticity (Kopp et al., 2007). Yuan et al. (2013) discovered that the observed change in subunit composition was strictly dependent on the insertion of GluN3A receptors, shown via a combination of GluN3A knockout animals and short hairpin RNA approaches. Synaptic insertion of GluN3A receptors lead to reduced Ca2+ permeability and magnesium sensitivity and is required for the development of a cocaine-induced increase in AMPAR/NMDAR ratio. The observed changes in AMPA and NMDA subunit composition, in terms of Ca2+ permeability, can reconcile the observation that after cocaine exposure, anti-Hebbian pairing protocols, not sufficient to induce LTP in naive conditions, are capable of inducing LTP (Mameli et al., 2011). Thanks to the advent of optogenetics, circuit level analysis has become a very fruitful and important way to investigate how neuroadaptations caused by appetitive events, like cocaine

Neuron

Review experience, occur within DA neurons. DA neurons can be classified depending on their different projection targets. While VTA DA neurons projecting to NAc medial shell display a robust increased AMPAR/NMDAR ratio 24 hr after cocaine injection, VTA DA neurons projecting to a lateral shell show only a subtle change in the AMPA/NMDA ratio. Surprisingly, VTA DA neurons projecting to the prefrontal cortex are insensitive to cocaine exposure and have unchanged AMPAR/NMDAR ratio 24h after cocaine treatment. Instead, they respond vigorously to an aversive experience (hind paw injection of formalin) with a remarkable increase in AMPAR/NMDAR ratio (Lammel et al., 2011). The functional segregation of DA neurons based on the projection targets is also supported by behavioral optogenetic manipulations. In fact, optogenetic stimulation of accumbens-projecting DA neurons support conditioned place preference, while optogenetic stimulation of prefrontal cortex-projecting DA neurons supports a vigorous conditioned place aversion (Lammel et al., 2012). Establishment of a maladaptive process of reward-related learning has been suggested as a core feature of drug addiction. Therefore, the ability of drugs of abuse to forge synaptic plasticity and ‘‘hijack’’ the natural reward process is an appealing reason for the development and, more importantly, for the persistence of behavioral responses associated with drugs of abuse. However, to date, there is no clear evidence that the observed initial synaptic plasticity changes triggered by substances of abuse represent a hallmark of addiction, but rather an initial step that will eventually lead to the induction of the disorder later on. Moreover, it has been shown that the persistence of synaptic plasticity occurring in the VTA is a requirement for the expression of a later synaptic adaptation that takes place in the NAc (Mameli et al., 2009). Potentially, synaptic adaptation in the ventral striatum might represent a gate for a later establishment of stimulusresponse habit formation occurring in the dorsal striatum (Belin and Everitt, 2008). Together, these observations raise the question of whether this cocaine-induced long-lasting potentiation may also reflect a disruption in the ability of DA neurons to code for reward and aversion during the occurrence of naturalistic behavior. Synaptic Plasticity and Dopaminergic Activity: Shaping DA Release Events The majority of information related to DA function in the brain has been provided by a combination of behavioral, pharmacological, genetic, and electrophysiological approaches. Surprisingly, none of the cited approaches can selectively provide subsecond, temporal resolution information related to DA release happening within ‘‘intact’’ forebrain terminal regions. The relatively recent introduction of voltammetric approaches, like fast scan cyclic voltammetry (FSCV), has radically changed the way we approach DA release in the brain, especially in regards to appetitive and aversive experiences. The power of this methodology is mostly represented by a capacity to quantitate phasic changes in DA concentration, within given target regions, occurring on a physiological timescale. Detecting DA concentration by using FSCV within the NAc of animals that are experiencing opposite hedonic valence revealed an opposite pattern of DA release. While the appetitive sucrose-based oral solution evoked a robust increase in DA

transients, the aversive quinine-based oral solution induced a reduction in DA release (Roitman et al., 2008). Interestingly, DA release does not only depend on the hedonic or the motivational states engaged by the behavioral framework used, but it is highly regulated by learned associations (Roitman et al., 2004). When animals received extensive Pavlovian conditioning training, where a conditioned stimulus (CS+) predicts the delivery of a reinforcer-unconditioned stimulus (US), phasic DA release within the NAc, that early in training appears to be primarily associated to the delivery of the reward, moves to the onset of the CS. The idea that learned associations can affect the occurrence of DA release within the NAc has also been shown for aversive experiences. In fact, phasic DA release events are suppressed by stimuli that have been associated with an aversive outcome. Conversely, phasic DA release events are increased by stimuli that precede the successful avoidance of an aversive event (Oleson et al., 2012). The lessons that we have learned so far account for an active interplay between learning and memory mechanisms and the regulation of specific patterns of DA release. The question left is how learning and memory mechanisms are connected with the occurrence of DA release. Interestingly, during reward-learned associations, DA release to reward predictive cues parallel an enhanced synaptic strength onto DA neurons (Stuber et al., 2008). Moreover, the observed synaptic potentiation develops over the course of cue-reward learning. It is worth mentioning that in conditioned appetitive learning, where environmental cues are associated with the receipt of a reward, food restriction can strongly increase the acquisition rate with which the animals learn how to perform the task. Notably, restriction of food intake is also a well-known strategy to enhance the reinforcing efficacy of drugs of abuse in experimental animal models (Carroll et al., 1979). The overall idea is that food restriction can enhance motivational or hedonic states that are important for learning and acquisition of behavioral tasks. Surprisingly, animals under food restriction protocols display not just an increase in behavioral responses to drugs of abuse, but also enhanced burst firing of DA neurons, increased AMPA/NMDA ratio and D2 autoreceptor-mediated desensitization in DA neurons (Branch et al., 2013). The timing in which synaptic plasticity occurs, and the fact that glutamatergic plasticity might affect the cellular mechanisms by which DA neurons produce DA release, is of great interest to the scientific community. In fact, the enhanced synaptic strength observed in the aforementioned conditions might be the cellular mechanism lying at the core of the transformation of neutral environmental stimuli to salient reward-predictive cues and potentially supports the establishment of specific hedonic and motivational states that are involved in learning processes. The Dual Role of Glutamatergic Input onto Dopamine Neurons Glutamatergic inputs are not just required for the induction of synaptic plasticity but they are also a fundamental player in the induction of burst firing of DA neurons. In particular, activation of NMDARs induces burst firing by regulating calcium transients that initiate and terminate bursts via Ca2+-activated potassium channels (Paladini and Roeper, 2014). Functional alterations, in the previously cited cellular mechanisms, can Neuron 86, June 3, 2015 ª2015 Elsevier Inc. 1151

Neuron

Review affect the way DA neurons produce burst firing during motivationally relevant conditions. Plasticity mechanisms occurring in response to aversive, appetitive, and associative learning conditions develop via a selective recruitment of synaptic AMPARs (Lammel et al., 2011; Saal et al., 2003; Ungless et al., 2001). Thus, it is likely that enhanced expression of AMPARs in DA neurons represents an additional cellular mechanism for promoting burst generation (Canavier and Landry, 2006; Komendantov et al., 2004). It is therefore plausible to hypothesize that a potentiated synaptic state onto DA neurons may allow the generation of a more depolarized membrane potential under specific circumstances and subsequently allow fast Mg2+ unbinding kinetics of NMDARs, thus increasing channel open probability and burst firing generation. The latter functional state may allow several behavioral conditions to generate uncontrolled burst firing, representing an unchecked route to specific forebrain effectors capable of supporting different psychiatric conditions. In support of this prediction is the demonstration that burst firing of DA neurons is elevated in awake rats that are experiencing passive exposure to cocaine (Koulchitsky et al., 2012). Unexpectedly, in a social defeat stress model, a behavioral condition with presumably an opposite hedonic valence in respect to cocaine exposure, susceptible mice showed a robust hyperactivity of VTA DA neurons. This cellular phenotype was caused by the upregulation of the hyperpolarization-activated cyclic nucleotide-gated type 2 (HCN2) channels (Ih), a protein regulating intrinsic excitability onto DA neurons. On the other hand, mice expressing a resilient phenotype displayed an even larger Ih and also increased potassium (K+) channel-mediated currents. Pharmacological treatment with the Ih potentiator lamotrigine, or increasing activity of DA VTA neurons via optogenetic manipulation in susceptible mice, is capable of reversing depression-related traits (Friedman et al., 2014). Interestingly, optogenetic stimulation occurring within DA neurons of the VTA mediates behavioral rescue via a homeostatic plasticity mechanism occurring between VTA-NAc projecting neurons (Friedman et al., 2014). Future studies will likely solve the question of how synaptic plasticity and intrinsic mechanisms regulating cellular excitability can affect, in a cooperative fashion, phasic DA release in a population of DA-projecting neurons. A glutamatergic regulation of DA release has been also suggested at the level of nerve terminals for PFC and NAc (Floresco et al., 1998; Jones et al., 2010; Takahata and Moghaddam, 1998). For example, dopaminergic and excitatory terminals are localized in close apposition both in the PFC and NAc. Anatomical studies identified that glutamatergic afferents from the BLA terminate in close apposition to DA terminals in the NAc (Johnson et al., 1994). On the other hand, DA terminals and excitatory afferents from the hippocampal formation are localized in close apposition in the PFC (Carr and Sesack, 1996; Smiley and Goldman-Rakic, 1993). DA release within NAc and PFC is highly sensitive to AMPARs blockade, suggesting that AMPARs are regulating DA release in a tonic fashion. This experimental data raise the possibility that AMPA receptors do not only reside within the somatic or dendritic shaft compartments of VTA DA neurons (Paquet et al., 1997), but could also be potentially expressed on DA terminals, where they might exert a control over DA release. However, electron microscopy studies were not 1152 Neuron 86, June 3, 2015 ª2015 Elsevier Inc.

able to identify AMPA receptors in striatal DA terminals (Bernard and Bolam, 1998; Bernard et al., 1997). How can it be possible that AMPA receptors are capable of modulating DA release in the striatum via action onto dopaminergic terminals if there is no evidence for expression? It has been suggested that H2O2 can act as a key regulator of this local AMPAR-mediated DA release (Avshalumov et al., 2000, 2008; Avshalumov and Rice, 2003). In this model, glutamatergic activity occurring onto AMPARs expressed in medium spiny neurons (MSNs) could lead to the production of H2O2, which could then diffuse to adjacent DA terminals, open KATP channels and regulate DA release. While there is no clear evidence for the existence of AMPARs in dopaminergic terminals in the NAc, there is solid evidence for the existence of mGlu receptors (mGluRs) (Paquet and Smith, 2003). mGluRs, mostly located at the perisynaptic sites, are engaged in the regulation of DA release in the striatum, where they are recruited in the case of glutamate spillover. Once functionally recruited, they could activate Ca2+-activated K+ channels and thus affect DA release (Zhang and Sulzer, 2003). All of these studies however only partially account for a local glutamatergic regulation of DA release into the NAc, at the level of dopaminergic terminals, under physiological conditions. In fact, little is known about the presence and the function of AMPARs onto dopaminergic terminals after exposure to reward-related stimuli, drugs of abuse, or aversive stimuli. Additionally, there seems to be no evidence about the presence of AMPA receptors on DA terminals located in the PFC, even though it is known that AMPA receptors blockade in the PFC significantly reduced cortical DA release (Takahata and Moghaddam, 2000). It is plausible to speculate that AMPA receptors may be actively transported to axonal terminals or incorporated there, after in loco de novo synthesis from preexisting mRNA, when the dopaminergic system is expressing synaptic plasticity processes triggered by salient conditions (Figure 3). The expression and synaptic recruitment of these AMPA receptors would be potentially regulated depending on the biological identity of each individual DA neuron, projection target, specific glutamatergic afferent, or, more importantly, by the emotional nature of the experience an animal faces. Cross talk between AMPARs and mGluRs at the axonal terminals can then shape DA release events within a given terminal region in a multifaceted way, thereby adding another level of complexity to the excitatory events happening at the somatic level. Moreover, neuronal inputs arising from multiple sites of the brain, and converging at the axonal level, might differentially impact DA release and ensure appropriate behavioral response to the information provided by an ever-changing environment in both physiological and pathological states (Floresco et al., 1998; Jones et al., 2010; Takahata and Moghaddam, 1998). Thus, mesocortical DA neurons may be engaged in cognitive processing underlying working memory or extinction processing, as well as acute stress reaction occurring during aversive experiences and so modulate alertness (Abraham et al., 2014). Meanwhile, mesoaccumbal DA neurons might be engaged in motivational functions like proper orientation of goal directed behaviors, in the emotional valence and attribution of saliency to the experience (Ungless, 2004).

Neuron

Review

Figure 3. Synaptic Plasticity within Specific Cellular Compartments of VTA Dopamine Neurons: Shaping and Tuning Terminal DA Release within Specific Target Regions in Response to Rewarding or Aversive Events DA release is regulated at the level of nerve terminals by glutamatergic afferents in the Nucleus accumbens (NAc) and in the medial prefrontal cortex (mPFC). Glutamatergic afferents arising from the basolateral amygdala (BLA) terminate in close apposition onto DA terminals in the NAc, while afferents emanated by the hippocampus (Hipp) localized in close apposition to DA terminals in the mPFC. DA release in both terminal regions is sensitive to AMPA receptors blockade, presumably because AMPA receptors are exerting a tonic control over DA release. We speculate that AMPA receptors may be actively transported to axonal terminals or incorporated there after de novo synthesis from pre-existing mRNA when DA neurons undergo plasticity processes. The latter condition could potentially confer a very sophisticated level of control over terminal DA release within specific target regions that are implicated in reward or aversion processing.

Particularly interesting to this matter is the notion that VTA dopaminergic neurons do not code, per se, for cognitive aspects of learned associations. As a matter of fact, DA neurons do not carry detailed aspects about the nature of the stimuli, but rather they have the ability to add ‘‘color or flavor’’ to experience-dependent processes implicated in memory formation, by virtue of their control over ‘‘saliency attribution’’ (Berridge and Robinson, 1998). The way DA neurons modulate saliency attribution to behavioral experiences is by controlling, via DA release, glutamatergic synaptic activity within the ventral striatum (O’Donnell and Grace, 1995; O’Donnell et al., 1999). Glutamatergic inputs reaching the NAc (from the ventral hippocampus, basolateral amygdala, PFC, and thalamus) provide information about contextual elements, cues, and even subtle descriptive features that belong to a specific environmental condition (Britt et al., 2012; Sesack and Grace, 2010). What DA does to these afferents is to control the signal-to-noise ratio and the resultant throughput of the ventral striatum, by maintaining a proper tuning between the drive and resultant behavioral responses promoted by limbic or cortical inputs, respectively (Goto and Grace, 2008).

While a lot of work has been done, solving all the biological questions discussed in this section will provide a much better understanding of how synaptic plasticity, intrinsic excitability, and concurrent DA release, occurring in specific circuits engaged in appetitive or aversive conditions, can produce maladaptive, pathological behaviors, such as substance use disorders.

Conclusions In conclusion, we have reviewed recent findings concerning the effects of both salient appetitive and aversive stimuli in DA neurons of the VTA. Over the past decade, the use of innovative techniques such as transgenic approaches and optogenetics has radically changed the view of the midbrain DA system. Science has come a long way in the description of the VTA that is now passing through a novel reclassification based on neuron subtypes displaying specific features at the molecular, anatomical, and electrophysiological level (Barrot, 2014; Margolis et al., 2008; Root et al., 2014; Tritsch et al., 2012). These groundbreaking scientific contributions are reshaping our knowledge about synaptic connectivity and molecular profiling of the VTA (Ekstrand et al., 2014). Neuron 86, June 3, 2015 ª2015 Elsevier Inc. 1153

Neuron

Review In this review, we speculate that, depending on the emotional content of the experiences, each individual projection-specific DA cell might make a unique contribution to inform the brain how to adapt to behavioral responses. Changes in synaptic configuration, and the resultant patterns of DA cell activity, might provide a mechanism by which synaptic neural adaptations occurring in the reward circuit are transferred to forebrain effectors. DA release, in its ‘‘conventional’’ or promiscuous co-release with other neurotransmitters, can promote or enhance the occurrence of spike-time-dependent plasticity at active corticolimbic-striatal synapses (Shiflett and Balleine, 2011). The occurrence of synaptic plasticity within the cortico-limbic-striatal circuits can affect and shape the function of each individual region that can now take the lead and impose its own specific function to orchestrate and guide future adaptive behaviors. Additionally, DA-mediated synaptic plasticity can support the maintenance and consolidation of learning, presumably by adding specific emotional weight, depending on the functional connectivity of the neural network in which neurons reside. REFERENCES Abraham, A.D., Neve, K.A., and Lattal, K.M. (2014). Dopamine and extinction: a convergence of theory with fear and reward circuitry. Neurobiol. Learn. Mem. 108, 65–77. Anggono, V., and Huganir, R.L. (2012). Regulation of AMPA receptor trafficking and synaptic plasticity. Curr. Opin. Neurobiol. 22, 461–469. Aragona, B.J., Day, J.J., Roitman, M.F., Cleaveland, N.A., Wightman, R.M., and Carelli, R.M. (2009). Regional specificity in the real-time development of phasic dopamine transmission patterns during acquisition of a cue-cocaine association in rats. Eur. J. Neurosci. 30, 1889–1899. Argilli, E., Sibley, D.R., Malenka, R.C., England, P.M., and Bonci, A. (2008). Mechanism and time course of cocaine-induced long-term potentiation in the ventral tegmental area. J. Neurosci. 28, 9092–9100. Avshalumov, M.V., and Rice, M.E. (2003). Activation of ATP-sensitive K+ (K(ATP)) channels by H2O2 underlies glutamate-dependent inhibition of striatal dopamine release. Proc. Natl. Acad. Sci. USA 100, 11729–11734. Avshalumov, M.V., Chen, B.T., and Rice, M.E. (2000). Mechanisms underlying H(2)O(2)-mediated inhibition of synaptic transmission in rat hippocampal slices. Brain Res. 882, 86–94. Avshalumov, M.V., Patel, J.C., and Rice, M.E. (2008). AMPA receptor-dependent H2O2 generation in striatal medium spiny neurons but not dopamine axons: one source of a retrograde signal that can inhibit dopamine release. J. Neurophysiol. 100, 1590–1601. Balleine, B., and Dickinson, A. (1992). Signalling and incentive processes in instrumental reinforcer devaluation. Q. J. Exp. Psychol. B 45, 285–301. Barrot, M. (2014). The ventral tegmentum and dopamine: a new wave of diversity. Neuroscience 282C, 243–247.

Bernard, V., and Bolam, J.P. (1998). Subcellular and subsynaptic distribution of the NR1 subunit of the NMDA receptor in the neostriatum and globus pallidus of the rat: co-localization at synapses with the GluR2/3 subunit of the AMPA receptor. Eur. J. Neurosci. 10, 3721–3736. Bernard, V., Somogyi, P., and Bolam, J.P. (1997). Cellular, subcellular, and subsynaptic distribution of AMPA-type glutamate receptor subunits in the neostriatum of the rat. J. Neurosci. 17, 819–833. Berridge, K.C. (2000). Measuring hedonic impact in animals and infants: microstructure of affective taste reactivity patterns. Neurosci. Biobehav. Rev. 24, 173–198. Berridge, K.C., and Robinson, T.E. (1998). What is the role of dopamine in reward: hedonic impact, reward learning, or incentive salience? Brain Res. Brain Res. Rev. 28, 309–369. Bliss, T.V., and Collingridge, G.L. (1993). A synaptic model of memory: longterm potentiation in the hippocampus. Nature 361, 31–39. Bliss, T.V., Collingridge, G.L., and Morris, R.G. (2014). Synaptic plasticity in health and disease: introduction and overview. Philos. Trans. R. Soc. Lond. B Biol. Sci. 369, 20130129. Bocklisch, C., Pascoli, V., Wong, J.C., House, D.R., Yvon, C., de Roo, M., Tan, K.R., and Lu¨scher, C. (2013). Cocaine disinhibits dopamine neurons by potentiation of GABA transmission in the ventral tegmental area. Science 341, 1521– 1525. Bonci, A., and Malenka, R.C. (1999). Properties and plasticity of excitatory synapses on dopaminergic and GABAergic cells in the ventral tegmental area. J. Neurosci. 19, 3723–3730. Branch, S.Y., Goertz, R.B., Sharpe, A.L., Pierce, J., Roy, S., Ko, D., Paladini, C.A., and Beckstead, M.J. (2013). Food restriction increases glutamate receptor-mediated burst firing of dopamine neurons. J. Neurosci. 33, 13861–13872. Britt, J.P., Benaliouad, F., McDevitt, R.A., Stuber, G.D., Wise, R.A., and Bonci, A. (2012). Synaptic and behavioral profile of multiple glutamatergic inputs to the nucleus accumbens. Neuron 76, 790–803. Brown, M.T., Tan, K.R., O’Connor, E.C., Nikonenko, I., Muller, D., and Lu¨scher, C. (2012). Ventral tegmental area GABA projections pause accumbal cholinergic interneurons to enhance associative learning. Nature 492, 452–456. Canavier, C.C., and Landry, R.S. (2006). An increase in AMPA and a decrease in SK conductance increase burst firing by different mechanisms in a model of a dopamine neuron in vivo. J. Neurophysiol. 96, 2549–2563. Carr, D.B., and Sesack, S.R. (1996). Hippocampal afferents to the rat prefrontal cortex: synaptic targets and relation to dopamine terminals. J. Comp. Neurol. 369, 1–15. Carroll, M.E., France, C.P., and Meisch, R.A. (1979). Food deprivation increases oral and intravenous drug intake in rats. Science 205, 319–321. Changizi, M.A., McGehee, R.M., and Hall, W.G. (2002). Evidence that appetitive responses for dehydration and food-deprivation are learned. Physiol. Behav. 75, 295–304. Chaudhury, D., Walsh, J.J., Friedman, A.K., Juarez, B., Ku, S.M., Koo, J.W., Ferguson, D., Tsai, H.C., Pomeranz, L., Christoffel, D.J., et al. (2013). Rapid regulation of depression-related behaviours by control of midbrain dopamine neurons. Nature 493, 532–536.

Beierholm, U., Guitart-Masip, M., Economides, M., Chowdhury, R., Du¨zel, E., Dolan, R., and Dayan, P. (2013). Dopamine modulates reward-related vigor. Neuropsychopharmacology 38, 1495–1503.

Chen, B.T., Bowers, M.S., Martin, M., Hopf, F.W., Guillory, A.M., Carelli, R.M., Chou, J.K., and Bonci, A. (2008). Cocaine but not natural reward self-administration nor passive cocaine infusion produces persistent LTP in the VTA. Neuron 59, 288–297.

Belin, D., and Everitt, B.J. (2008). Cocaine seeking habits depend upon dopamine-dependent serial connectivity linking the ventral with the dorsal striatum. Neuron 57, 432–441.

Cohen, J.Y., Haesler, S., Vong, L., Lowell, B.B., and Uchida, N. (2012). Neurontype-specific signals for reward and punishment in the ventral tegmental area. Nature 482, 85–88.

Bellone, C., and Lu¨scher, C. (2005). mGluRs induce a long-term depression in the ventral tegmental area that involves a switch of the subunit composition of AMPA receptors. Eur. J. Neurosci. 21, 1280–1288.

Collingridge, G.L., Peineau, S., Howland, J.G., and Wang, Y.T. (2010). Longterm depression in the CNS. Nat. Rev. Neurosci. 11, 459–473.

Bellone, C., and Luscher, C. (2006). Cocaine triggered AMPA receptor redistribution is reversed in vivo by mG luR-dependent long-term depression. Nat. Neurosci. 9, 636–641.

1154 Neuron 86, June 3, 2015 ª2015 Elsevier Inc.

Daberkow, D.P., Brown, H.D., Bunner, K.D., Kraniotis, S.A., Doellman, M.A., Ragozzino, M.E., Garris, P.A., and Roitman, M.F. (2013). Amphetamine paradoxically augments exocytotic dopamine release and phasic dopamine signals. J. Neurosci. 33, 452–463.

Neuron

Review Darvas, M., Wunsch, A.M., Gibbs, J.T., and Palmiter, R.D. (2014). Dopamine dependency for acquisition and performance of Pavlovian conditioned response. Proc. Natl. Acad. Sci. USA 111, 2764–2769.

Jackson, A., Mead, A.N., and Stephens, D.N. (2000). Behavioural effects of alpha-amino-3-hydroxy-5-methyl-4-isoxazolepropionate-receptor antagonists and their relevance to substance abuse. Pharmacol. Ther. 88, 59–76.

Ekstrand, M.I., Nectow, A.R., Knight, Z.A., Latcha, K.N., Pomeranz, L.E., and Friedman, J.M. (2014). Molecular profiling of neurons based on connectivity. Cell 157, 1230–1242.

Jennings, J.H., Sparta, D.R., Stamatakis, A.M., Ung, R.L., Pleil, K.E., Kash, T.L., and Stuber, G.D. (2013). Distinct extended amygdala circuits for divergent motivational states. Nature 496, 224–228.

Fadok, J.P., Dickerson, T.M., and Palmiter, R.D. (2009). Dopamine is necessary for cue-dependent fear conditioning. J. Neurosci. 29, 11089–11097.

Johanson, I.B., and Hall, W.G. (1979). Appetitive learning in 1-day-old rat pups. Science 205, 419–421.

Fadok, J.P., Darvas, M., Dickerson, T.M., and Palmiter, R.D. (2010). Long-term memory for pavlovian fear conditioning requires dopamine in the nucleus accumbens and basolateral amygdala. PLoS ONE 5, e12751.

Johnson, L.R., Aylward, R.L., Hussain, Z., and Totterdell, S. (1994). Input from the amygdala to the rat nucleus accumbens: its relationship with tyrosine hydroxylase immunoreactivity and identified neurons. Neuroscience 61, 851–865.

Feldman, D.E. (2012). The spike-timing dependence of plasticity. Neuron 75, 556–571. Floresco, S.B., Yang, C.R., Phillips, A.G., and Blaha, C.D. (1998). Basolateral amygdala stimulation evokes glutamate receptor-dependent dopamine efflux in the nucleus accumbens of the anaesthetized rat. Eur. J. Neurosci. 10, 1241– 1251. Friedman, A.K., Walsh, J.J., Juarez, B., Ku, S.M., Chaudhury, D., Wang, J., Li, X., Dietz, D.M., Pan, N., Vialou, V.F., et al. (2014). Enhancing depression mechanisms in midbrain dopamine neurons achieves homeostatic resilience. Science 344, 313–319. Gan, Q., Salussolia, C.L., and Wollmuth, L.P. (2015). Assembly of AMPA receptors: mechanisms and regulation. J. Physiol. 593, 39–48. Gore, B.B., Soden, M.E., and Zweifel, L.S. (2014). Visualization of plasticity in fear-evoked calcium signals in midbrain dopamine neurons. Learn. Mem. 21, 575–579. Goto, Y., and Grace, A.A. (2008). Limbic and cortical information processing in the nucleus accumbens. Trends Neurosci. 31, 552–558. Grace, A.A., and Bunney, B.S. (1984a). The control of firing pattern in nigral dopamine neurons: burst firing. J. Neurosci. 4, 2877–2890. Grace, A.A., and Bunney, B.S. (1984b). The control of firing pattern in nigral dopamine neurons: single spike firing. J. Neurosci. 4, 2866–2876. Grace, A.A., Floresco, S.B., Goto, Y., and Lodge, D.J. (2007). Regulation of firing of dopaminergic neurons and control of goal-directed behaviors. Trends Neurosci. 30, 220–227. Haber, S.N. (2014). The place of dopamine in the cortico-basal ganglia circuit. Neuroscience 282C, 248–257. Hall, M.E., and Mayer, M.A. (1975). Effects of alpha methyl-para-tyrosine on the recall of a passive avoidance response. Pharmacol. Biochem. Behav. 3, 579–582. Harnett, M.T., Bernier, B.E., Ahn, K.C., and Morikawa, H. (2009). Burst-timingdependent plasticity of NMDA receptor-mediated transmission in midbrain dopamine neurons. Neuron 62, 826–838. Harris, G.C., and Aston-Jones, G. (2003). Altered motivation and learning following opiate withdrawal: evidence for prolonged dysregulation of reward processing. Neuropsychopharmacology 28, 865–871. Hjelmstad, G.O., Xia, Y., Margolis, E.B., and Fields, H.L. (2013). Opioid modulation of ventral pallidal afferents to ventral tegmental area neurons. J. Neurosci. 33, 6454–6459. Hnasko, T.S., Hjelmstad, G.O., Fields, H.L., and Edwards, R.H. (2012). Ventral tegmental area glutamate neurons: electrophysiological properties and projections. J. Neurosci. 32, 15076–15085. Hollerman, J.R., and Schultz, W. (1998). Dopamine neurons report an error in the temporal prediction of reward during learning. Nat. Neurosci. 1, 304–309. Inglis, F.M., and Moghaddam, B. (1999). Dopaminergic innervation of the amygdala is highly responsive to stress. J. Neurochem. 72, 1088–1094. Isaac, J.T., Ashby, M.C., and McBain, C.J. (2007). The role of the GluR2 subunit in AMPA receptor function and synaptic plasticity. Neuron 54, 859–871.

Jones, J.L., Day, J.J., Aragona, B.J., Wheeler, R.A., Wightman, R.M., and Carelli, R.M. (2010). Basolateral amygdala modulates terminal dopamine release in the nucleus accumbens and conditioned responding. Biol. Psychiatry 67, 737–744. Kauer, J.A. (2004). Learning mechanisms in addiction: synaptic plasticity in the ventral tegmental area as a result of exposure to drugs of abuse. Annu. Rev. Physiol. 66, 447–475. Kelley, A.E., Andrzejewski, M.E., Baldwin, A.E., Hernandez, P.J., and Pratt, W.E. (2003). Glutamate-mediated plasticity in corticostriatal networks: role in adaptive motor learning. Ann. N Y Acad. Sci. 1003, 159–168. Kempadoo, K.A., Tourino, C., Cho, S.L., Magnani, F., Leinninger, G.M., Stuber, G.D., Zhang, F., Myers, M.G., Deisseroth, K., de Lecea, L., and Bonci, A. (2013). Hypothalamic neurotensin projections promote reward by enhancing glutamate transmission in the VTA. J. Neurosci. 33, 7618–7626. Komendantov, A.O., Komendantova, O.G., Johnson, S.W., and Canavier, C.C. (2004). A modeling study suggests complementary roles for GABAA and NMDA receptors and the SK channel in regulating the firing pattern in midbrain dopamine neurons. J. Neurophysiol. 91, 346–357. Kopp, C., Longordo, F., and Lu¨thi, A. (2007). Experience-dependent changes in NMDA receptor composition at mature central synapses. Neuropharmacology 53, 1–9. Koulchitsky, S., De Backer, B., Quertemont, E., Charlier, C., and Seutin, V. (2012). Differential effects of cocaine on dopamine neuron firing in awake and anesthetized rats. Neuropsychopharmacology 37, 1559–1571. Lammel, S., Ion, D.I., Roeper, J., and Malenka, R.C. (2011). Projection-specific modulation of dopamine neuron synapses by aversive and rewarding stimuli. Neuron 70, 855–862. Lammel, S., Lim, B.K., Ran, C., Huang, K.W., Betley, M.J., Tye, K.M., Deisseroth, K., and Malenka, R.C. (2012). Input-specific control of reward and aversion in the ventral tegmental area. Nature 491, 212–217. Lu¨scher, C., and Malenka, R.C. (2011). Drug-evoked synaptic plasticity in addiction: from molecular changes to circuit remodeling. Neuron 69, 650–663. Mameli, M., Balland, B., Luja´n, R., and Lu¨scher, C. (2007). Rapid synthesis and synaptic insertion of GluR2 for mGluR-LTD in the ventral tegmental area. Science 317, 530–533. Mameli, M., Halbout, B., Creton, C., Engblom, D., Parkitna, J.R., Spanagel, R., and Lu¨scher, C. (2009). Cocaine-evoked synaptic plasticity: persistence in the VTA triggers adaptations in the NAc. Nat. Neurosci. 12, 1036–1041. Mameli, M., Bellone, C., Brown, M.T., and Lu¨scher, C. (2011). Cocaine inverts rules for synaptic plasticity of glutamate transmission in the ventral tegmental area. Nat. Neurosci. 14, 414–416. Margolis, E.B., Mitchell, J.M., Ishikawa, J., Hjelmstad, G.O., and Fields, H.L. (2008). Midbrain dopamine neurons: projection target determines action potential duration and dopamine D(2) receptor inhibition. J. Neurosci. 28, 8908– 8913. McCutcheon, J.E., Beeler, J.A., and Roitman, M.F. (2012). Sucrose-predictive cues evoke greater phasic dopamine release than saccharin-predictive cues. Synapse 66, 346–351. Mendelson, J. (1966). The role of hunger in the T-maze learning for food by rats. J. Comp. Psychol. 62, 341–349.

Neuron 86, June 3, 2015 ª2015 Elsevier Inc. 1155

Neuron

Review Miller, N.E., and Hunt, J.M. (1944). Experimental Studies of Conflict. Personality and the Behavior Disorders (Oxford, England: Ronald Press), pp. 431–465.

Schultz, W., Tremblay, L., and Hollerman, J.R. (1998). Reward prediction in primate basal ganglia and frontal cortex. Neuropharmacology 37, 421–429.

Montague, P.R., Hyman, S.E., and Cohen, J.D. (2004). Computational roles for dopamine in behavioural control. Nature 431, 760–767.

Sesack, S.R., and Grace, A.A. (2010). Cortico-Basal Ganglia reward network: microcircuitry. Neuropsychopharmacology 35, 27–47.

Morales, M., and Root, D.H. (2014). Glutamate neurons within the midbrain dopamine regions. Neuroscience 282C, 60–68.

Shiflett, M.W., and Balleine, B.W. (2011). Molecular substrates of action control in cortico-striatal circuits. Prog. Neurobiol. 95, 1–13.

Morgan, M.J. (1974). Resistance to satiation. Anim. Behav. 22, 449–466.

Smiley, J.F., and Goldman-Rakic, P.S. (1993). Heterogeneous targets of dopamine synapses in monkey prefrontal cortex demonstrated by serial section electron microscopy: a laminar analysis using the silver-enhanced diaminobenzidine sulfide (SEDS) immunolabeling technique. Cereb. Cortex 3, 223–238.

O’Donnell, P., and Grace, A.A. (1995). Synaptic interactions among excitatory afferents to nucleus accumbens neurons: hippocampal gating of prefrontal cortical input. J. Neurosci. 15, 3622–3639. O’Donnell, P., Greene, J., Pabello, N., Lewis, B.L., and Grace, A.A. (1999). Modulation of cell firing in the nucleus accumbens. Ann. N Y Acad. Sci. 877, 157–175. Oleson, E.B., Gentry, R.N., Chioma, V.C., and Cheer, J.F. (2012). Subsecond dopamine release in the nucleus accumbens predicts conditioned punishment and its successful avoidance. J. Neurosci. 32, 14804–14808. Paladini, C.A., and Roeper, J. (2014). Generating bursts (and pauses) in the dopamine midbrain neurons. Neuroscience 282C, 109–121. Paquet, M., and Smith, Y. (2003). Group I metabotropic glutamate receptors in the monkey striatum: subsynaptic association with glutamatergic and dopaminergic afferents. J. Neurosci. 23, 7659–7669. Paquet, M., Tremblay, M., Soghomonian, J.J., and Smith, Y. (1997). AMPA and NMDA glutamate receptor subunits in midbrain dopaminergic neurons in the squirrel monkey: an immunohistochemical and in situ hybridization study. J. Neurosci. 17, 1377–1396. Phillips, P.E., Stuber, G.D., Heien, M.L., Wightman, R.M., and Carelli, R.M. (2003). Subsecond dopamine release promotes cocaine seeking. Nature 422, 614–618. Ranaldi, R. (2014). Dopamine and reward seeking: the role of ventral tegmental area. Rev. Neurosci. 25, 621–630. Roitman, M.F., Stuber, G.D., Phillips, P.E., Wightman, R.M., and Carelli, R.M. (2004). Dopamine operates as a subsecond modulator of food seeking. J. Neurosci. 24, 1265–1271. Roitman, M.F., Wheeler, R.A., Wightman, R.M., and Carelli, R.M. (2008). Realtime chemical responses in the nucleus accumbens differentiate rewarding and aversive stimuli. Nat. Neurosci. 11, 1376–1377. Romo, R., and Schultz, W. (1990). Dopamine neurons of the monkey midbrain: contingencies of responses to active touch during self-initiated arm movements. J. Neurophysiol. 63, 592–606. Root, D.H., Mejias-Aponte, C.A., Zhang, S., Wang, H.L., Hoffman, A.F., Lupica, C.R., and Morales, M. (2014). Single rodent mesohabenular axons release glutamate and GABA. Nat. Neurosci. 17, 1543–1551. Rosenkranz, J.A., and Grace, A.A. (2002). Dopamine-mediated modulation of odour-evoked amygdala potentials during pavlovian conditioning. Nature 417, 282–287. Saal, D., Dong, Y., Bonci, A., and Malenka, R.C. (2003). Drugs of abuse and stress trigger a common synaptic adaptation in dopamine neurons. Neuron 37, 577–582. Schultz, W. (1986). Responses of midbrain dopamine neurons to behavioral trigger stimuli in the monkey. J. Neurophysiol. 56, 1439–1461. Schultz, W. (2006). Behavioral theories and the neurophysiology of reward. Annu. Rev. Psychol. 57, 87–115. Schultz, W. (2011). Potential vulnerabilities of neuronal reward, risk, and decision mechanisms to addictive drugs. Neuron 69, 603–617. Schultz, W., Apicella, P., and Ljungberg, T. (1993). Responses of monkey dopamine neurons to reward and conditioned stimuli during successive steps of learning a delayed response task. J. Neurosci. 13, 900–913. Schultz, W., Dayan, P., and Montague, P.R. (1997). A neural substrate of prediction and reward. Science 275, 1593–1599.

1156 Neuron 86, June 3, 2015 ª2015 Elsevier Inc.

Smith, K.S., Berridge, K.C., and Aldridge, J.W. (2011). Disentangling pleasure from incentive salience and learning signals in brain reward circuitry. Proc. Natl. Acad. Sci. USA 108, E255–E264. Stamatakis, A.M., Jennings, J.H., Ung, R.L., Blair, G.A., Weinberg, R.J., Neve, R.L., Boyce, F., Mattis, J., Ramakrishnan, C., Deisseroth, K., and Stuber, G.D. (2013). A unique population of ventral tegmental area neurons inhibits the lateral habenula to promote reward. Neuron 80, 1039–1053. Steffensen, S.C., Svingos, A.L., Pickel, V.M., and Henriksen, S.J. (1998). Electrophysiological characterization of GABAergic neurons in the ventral tegmental area. J. Neurosci. 18, 8003–8015. Steinberg, E.E., Keiflin, R., Boivin, J.R., Witten, I.B., Deisseroth, K., and Janak, P.H. (2013). A causal link between prediction errors, dopamine neurons and learning. Nat. Neurosci. 16, 966–973. Stuber, G.D., Roitman, M.F., Phillips, P.E., Carelli, R.M., and Wightman, R.M. (2005a). Rapid dopamine signaling in the nucleus accumbens during contingent and noncontingent cocaine administration. Neuropsychopharmacology 30, 853–863. Stuber, G.D., Wightman, R.M., and Carelli, R.M. (2005b). Extinction of cocaine self-administration reveals functionally and temporally distinct dopaminergic signals in the nucleus accumbens. Neuron 46, 661–669. Stuber, G.D., Klanker, M., de Ridder, B., Bowers, M.S., Joosten, R.N., Feenstra, M.G., and Bonci, A. (2008). Reward-predictive cues enhance excitatory synaptic strength onto midbrain dopamine neurons. Science 321, 1690– 1692. Takahata, R., and Moghaddam, B. (1998). Glutamatergic regulation of basal and stimulus-activated dopamine release in the prefrontal cortex. J. Neurochem. 71, 1443–1449. Takahata, R., and Moghaddam, B. (2000). Target-specific glutamatergic regulation of dopamine neurons in the ventral tegmental area. J. Neurochem. 75, 1775–1778. Tan, K.R., Yvon, C., Turiault, M., Mirzabekov, J.J., Doehner, J., Laboue`be, G., Deisseroth, K., Tye, K.M., and Lu¨scher, C. (2012). GABA neurons of the VTA drive conditioned place aversion. Neuron 73, 1173–1183. Thomas, M.J., Malenka, R.C., and Bonci, A. (2000). Modulation of long-term depression by dopamine in the mesolimbic system. J. Neurosci. 20, 5581– 5586. Tong, Z.Y., Overton, P.G., and Clark, D. (1996). Antagonism of NMDA receptors but not AMPA/kainate receptors blocks bursting in dopaminergic neurons induced by electrical stimulation of the prefrontal cortex. J. Neural Transm. 103, 889–904. Tritsch, N.X., Ding, J.B., and Sabatini, B.L. (2012). Dopaminergic neurons inhibit striatal output through non-canonical release of GABA. Nature 490, 262–266. Tye, K.M., Mirzabekov, J.J., Warden, M.R., Ferenczi, E.A., Tsai, H.C., Finkelstein, J., Kim, S.Y., Adhikari, A., Thompson, K.R., Andalman, A.S., et al. (2013). Dopamine neurons modulate neural encoding and expression of depression-related behaviour. Nature 493, 537–541. Ungless, M.A. (2004). Dopamine: the salient issue. Trends Neurosci. 27, 702–706. Ungless, M.A., Whistler, J.L., Malenka, R.C., and Bonci, A. (2001). Single cocaine exposure in vivo induces long-term potentiation in dopamine neurons. Nature 411, 583–587.

Neuron

Review van Zessen, R., Phillips, J.L., Budygin, E.A., and Stuber, G.D. (2012). Activation of VTA GABA neurons disrupts reward consumption. Neuron 73, 1184–1194. Wise, R.A. (2004). Dopamine, learning and motivation. Nat. Rev. Neurosci. 5, 483–494.

Yamaguchi, T., Wang, H.L., Li, X., Ng, T.H., and Morales, M. (2011). Mesocorticolimbic glutamatergic pathway. J. Neurosci. 31, 8476–8490.

Wise, R.A., and Kiyatkin, E.A. (2011). Differentiating the rapid actions of cocaine. Nat. Rev. Neurosci. 12, 479–484.

Yuan, T., Mameli, M., O’ Connor, E.C., Dey, P.N., Verpelli, C., Sala, C., PerezOtano, I., Lu¨scher, C., and Bellone, C. (2013). Expression of cocaine-evoked synaptic plasticity by GluN3A-containing NMDA receptors. Neuron 80, 1025–1038.

Wise, R.A., Spindler, J., deWit, H., and Gerberg, G.J. (1978). Neurolepticinduced ‘‘anhedonia’’ in rats: pimozide blocks reward quality of food. Science 201, 262–264.

Zhang, H., and Sulzer, D. (2003). Glutamate spillover in the striatum depresses dopaminergic transmission by activating group I metabotropic glutamate receptors. J. Neurosci. 23, 10585–10592.

Wolf, M.E. (1998). The role of excitatory amino acids in behavioral sensitization to psychomotor stimulants. Prog. Neurobiol. 54, 679–720. Yamaguchi, T., Sheen, W., and Morales, M. (2007). Glutamatergic neurons are present in the rat ventral tegmental area. Eur. J. Neurosci. 25, 106–118.

Zweifel, L.S., Fadok, J.P., Argilli, E., Garelick, M.G., Jones, G.L., Dickerson, T.M., Allen, J.M., Mizumori, S.J., Bonci, A., and Palmiter, R.D. (2011). Activation of dopamine neurons is critical for aversive conditioning and prevention of generalized anxiety. Nat. Neurosci. 14, 620–626.

Neuron 86, June 3, 2015 ª2015 Elsevier Inc. 1157