- Email: [email protected]
Treatment with a muscarinic acetylcholine receptor antagonist impairs the acquisition of conditioned reward learning in rats
Neuroscience Letters 614 (2016) 95–98
Contents lists available at ScienceDirect
Neuroscience Letters journal homepage: www.elsevier.com/locate/neulet
Research paper
Treatment with a muscarinic acetylcholine receptor antagonist impairs the acquisition of conditioned reward learning in rats R. Nisanov a , E. Galaj a , R. Ranaldi a,b,∗ a b
Graduate Center of the City University of New York, United States Queens College of the City University of New York, Department of Psychology, United States
h i g h l i g h t s • The muscarinic antagonist, scopolamine, impairs the acquisition of conditioned reward learning. • The muscarinic antagonist, scopolamine, does not impair the expression of conditioned reward. • Muscarinic acetylcholine receptor stimulation is involved in the acquisition of conditioned reward learning.
a r t i c l e
i n f o
Article history: Received 15 November 2015 Received in revised form 19 December 2015 Accepted 31 December 2015 Available online 6 January 2016 Keywords: Conditioned reward Scopolamine Acetylcholine Muscarinic receptor Food Conditioned reinforcement
a b s t r a c t The neural mechanisms whereby a reward-associated stimulus gains reinforcing properties and comes to function as a conditioned reward (CR) are not understood. We propose that muscarinic acetylcholine (mACh) receptor stimulation is necessary for this type of learning. Here we tested the hypothesis that mACh receptor antagonism, with scopolamine, would attenuate the acquisition by a food-related stimulus of the capacity to function as a CR. Rats were exposed to 5 pre-exposure sessions during which two levers were present, one producing a light and the other a tone when pressed. This was followed by 3 conditioning sessions in which the levers were absent and the rats were presented with 30 light-food pairings delivered randomly. In the test session, the levers were present and presses on both levers were recorded. Different groups of rats received intraperitoneal injections of scopolamine (0, 0.375, 0.75 and 1 mg/kg) either prior to each conditioning session or prior to the test session. All groups showed significantly greater responding on the light lever in the test compared to the pre-exposure sessions, demonstrating the CR effect. In animals treated prior to conditioning the scopolamine groups pressed significantly less on the light lever than the vehicle group. In animals treated prior to the test the increased lever pressing for light was similar for all groups. These data suggest that scopolamine impaired the acquisition of CR but not its expression. The results support the hypothesis that mACh receptor stimulation is important for the acquisition by reward-associated stimuli of the ability to function as CRs. © 2016 Elsevier Ireland Ltd. All rights reserved.
1. Introduction Reward-related learning is a vital aspect of an organism’s ability to adapt to the environment. Learning about primary rewards (unconditioned stimuli; USs), such as food and water, also leads to learning about reward-related stimuli. A previously neutral stimulus repeatedly paired with a US can become a conditioned stimulus (CS) capable of eliciting behaviors that are similar or related to the US [20]. Furthermore, the CS is also capable of gaining reinforcing
∗ Corresponding author at: Psychology Department, Queens College, 65-30 Kissena Blvd, Flushing, NY 11367, United States. Fax: +1 718 997 3257. E-mail address: [email protected] (R. Ranaldi). http://dx.doi.org/10.1016/j.neulet.2015.12.064 0304-3940/© 2016 Elsevier Ireland Ltd. All rights reserved.
properties of its own, thereby, functioning as a conditioned reward (CR) [4,36]. The neural mechanisms whereby CSs gain reinforcing properties and become CRs are not fully understood and, therefore, constitute an aim of the current study. One possibility is that the neuronal pathways that transmit US signals converge with the neuronal pathways that mediate potential CS signals as inputs to the pathway that mediates the unconditioned response. The dual stimulation by the eventual CS and US of relevant neurons allows the CS to gain the capability of stimulating the same motivational neural circuits as the US and elicit the same motivational behavior, independently of the US [5,8,27,37]. If this is the case, then blocking one of these inputs to the reward-relevant pathway could prevent the acquisition of such learning.
96
R. Nisanov et al. / Neuroscience Letters 614 (2016) 95–98
Previous work suggests that the US signal mediated by food involves, at least, the release of acetylcholine (ACh) in the ventral tegmental area (VTA) [12,23], although ACh in other brain regions may also be important [22]. That ACh activity in the brain serves as a US signal for food is supported by studies showing that the blockade of muscarinic ACh (mACh) receptor stimulation in the VTA reduces eating [23,34] and impairs the acquisition, although not the expression, of food-related learning [35]. If this is so, then the blockade of mACh receptor stimulation should prevent the acquisition of reinforcing properties by a food-paired stimulus or CS. In this event the CS would also not function as a CR. In the present paper, we tested this hypothesis. Specifically, we hypothesized that systemic treatment with scopolamine, a mACh receptor antagonist, would attenuate or prevent the acquisition by a food-related stimulus of the capacity to function as a CR. 2. Methods 2.1. Subjects Subjects were 64 male, facility bred Long-Evans rats, individually housed in a climate controlled room on a 12 hour light/12 hour dark cycle (lights off at 6 AM). The rats, with initial free feeding weights of approximately 350 g (325–375 g), were maintained at 85% of their free-feeding weights throughout the experiment by daily rationed food portions. Water was freely accessible at all times except during behavioral test sessions.
Fig. 1. Mean number of presses on the light and tone levers during the pre-exposure and test phases for rats treated with scopolamine (0, 0.375, 0.75 and 1.5 mg/kg) prior to each conditioning session. Vertical lines represent the standard error of the mean (SEM). * represents a significant difference from vehicle in light lever presses.
The mACh receptor antagonist, scopolamine hydrobromide (Sigma–Aldrich, St. Louis, MO), was dissolved in 0.9% saline, at doses of 0, 0.375, 0.75 and 1.5 mg/kg.
0.375, 0.75 or 1.5 mg/kg). In this phase, both levers were removed from the operant chambers. During each conditioning session, rats were presented with a 3-sec light presentation followed by a food pellet delivery for a total of 30 light stimuli paired with 30 food pellets. After completion of this phase, there was a 2-day rest period. The test phase consisted of one 40 min session during which both levers were present and pressing one lever produced the tone and the other the light as in the pre-exposure phase. The number of presses on each lever was recorded.
2.3. Apparatus
2.6. Expression
Behavioral testing was conducted in 8 operant conditioning chambers housed in sound- and light-attenuating boxes. Each operant conditioning chamber, measuring 30.5 × 24.0 × 25.0 cm (l × w × h), had two aluminum walls, Plexiglas sidewalls and ceiling. The floor consisted of 0.60-cm diameter stainless steel rods spaced 2.0 cm apart. Each chamber was equipped with two levers, two white stimulus lights, a tone generator, and a food trough, all on the right wall. The food trough, measuring 3.0 × 3.6 × 2.0 cm (l × w × h), was centered between the two levers. Each lever was located 3.0 cm to the right and left of the food trough and 8.0 cm above the floor. Each white stimulus light was positioned 3 cm above each lever.
The conditioned reward paradigm was the same as in acquisition except that IP treatments of scopolamine (0, 0.375, 0.75 and 1.5 mg/kg) were administered prior to the test session and not prior to each conditioning session.
2.2. Drugs
2.4. Procedure
2.7. Data analysis The average number of responses made on each lever during the pre-exposure phase and the test phase was analyzed for each group. A three-way mixed design analyses of variance (ANOVA) was conducted with the following three factors; phase (pre-exposure and test), scopolamine dose (0, 0.375, 0.75 and 1.5 mg/kg) and lever (light and tone). Significant 3-way interactions were followed by interaction comparisons and tests of simple effects. Significant simple effects were further analyzed using Dunnett’s post hoc tests.
The conditioned reward paradigm consisted of three phases; pre-exposure, conditioning and test phases. This paradigm tested the acquisition and expression of reward related learning.
3. Results
2.5. Acquisition
3.1. Acquisition
During the pre-exposure phase, rats were placed in the operant conditioning chambers for 40 min sessions held once a day for five consecutive days. Pressing one lever produced a 3-s tone presentation while pressing the other lever resulted in a 3-s light presentation above that lever. The number of presses on each lever was recorded. The conditioning phase consisted of three 60 min sessions held on separate days with one day between sessions. Thirty minutes prior to being placed in the operant chambers, all rats were treated with an intraperitoneal (IP) injection of one dose of scopolamine (0,
In the pre-exposure phase, the numbers of lever presses on the light and tone levers were low with slightly more responding on the light lever. This pattern was similar in all groups (see Fig. 1). In the test phase, the vehicle group showed a large increase in light lever responding with little change in tone lever responding compared to the pre-exposure phase. The scopolamine groups showed no significant changes in responding on the tone lever but significantly large increases in responding on the light lever in the test phase compared to pre-exposure. However, the increases in light lever responding in the scopolamine groups were
R. Nisanov et al. / Neuroscience Letters 614 (2016) 95–98
Fig. 2. Mean number of presses on the light and tone levers during the pre-exposure and test phases for rats treated with scopolamine (0, 0.375, 0.75 and 1.5 mg/kg) prior to the test session. Vertical lines represent the standard error of the mean (SEM).
significantly smaller than in the vehicle group in a dose-related fashion (the higher the dose, the smaller the increase). A three-way ANOVA revealed a significant group × phase × lever interaction [F(3,28) = 5.676, p = 0.004]. Follow-up interaction comparisons at the level of each lever revealed a significant group × phase interaction for the light lever only, [F(3,28) = 8.427, p < 0.05]. Tests of simple effects of group at each level of phase revealed a significant group effect in the test phase [F(3,28) = 17.551, p < 0.05]. Dunnett’s post hoc tests for group differences on the lever presses for light revealed that all scopolamine groups were significantly different from the vehicle group, ps < .05. 3.2. Expression As in acquisition, the amount of lever presses for light and tone in the pre-exposure phase was low with slightly more responding on the light lever for all groups (Fig. 2). Two groups, the vehicle and 0.75 mg scopolamine groups, showed no significant changes in responding on the tone lever but all groups showed very large increases in responding on the light lever in the test compared to the pre-exposure phases. The increased lever pressing for light was similar for all groups. These observations were supported by statistical analyses. A three-way ANOVA revealed a significant phase × lever interaction, [F(1,28) = 155.32, p < 0.001] which did not interact with the group factor. Tests of simple effects of phase at each level of lever revealed a significant difference on the light lever presses between pre-exposure phase and test phase [F(1,28) = 325.86, p < 0.001]. 4. Discussion These experiments tested the hypothesis that antagonism of mACh receptors during acquisition of a stimulus-reward learning task would impair learning, which would be manifested as an attenuation of the capacity of the CS to function as a conditioned reward. Systemic administration of scopolamine prior to food-CS conditioning sessions resulted in a significant and dose-related reduction in the active lever presses (i.e., presses reinforced by a conditioned reward) when tested after conditioning. All animals consumed the same number of food pellets under scopolamine treatment, arguing against the possibility that the reduced CR effect occurred because of reduced food consumption or, perhaps, fewer stimulus-reward
97
pairings. Instead, these results suggest that mACh receptor stimulation is important for the acquisition by a food-associated stimulus of the capacity to function as a CR. In contrast, when scopolamine was administered during the expression phase—after conditioning already occurred—it produced no significant changes among groups on active lever presses. Thus, it seems from these data that mACh neurotransmission is necessary for the acquisition, but not the expression, of the CR effect. Muscarinic acetylcholine neurotransmission is believed to be an important component of reward-related learning. Systemic muscarinic receptor antagonism eliminated the acquisition of fructose conditioned flavor preference in rats [32] and cocaine conditioned place preference (CPP) in mice [18]. Antagonism of muscarinic receptors in the nucleus accumbens core or shell impaired acquisition and expression of lever pressing for sucrose [22]. Furthermore, scopolamine administration into the basolateral amygdala diminished the capability of a cocaine-associated stimulus to acquire conditioned rewarding properties when administered during acquisition but not when administered after the behavior was acquired [33]. Similarly, antagonism of mACh receptors in the ventral tegmental area (VTA) resulted in a significant dose-related impairment in the acquisition of food approach-related behaviors but not the expression of these behaviors when the treatment occurred after learning was already demonstrated [34,35]. The present results are the first to demonstrate that not only is mACh receptor stimulation necessary for the acquisition by rewardassociated stimuli of the ability to function as CSs but also important for such stimuli to function as CRs. Dopamine (DA) and ACh, both, have been directly implicated in reward-related learning. The presentation of a food-paired CS produced an increase in DA release in striatal regions [6,15,24] as well as ACh release in the VTA [12,23]. Blockade of DA neurotransmission during reward-stimulus conditioning resulted in diminished responding for CR as tested after conditioning [3,17,27]. Increased DA neurotransmission by indirect DA agonists, pripradol, amphetamine and cocaine, increased responding for CR [2,3,9–11,16,21,28–31] and a similar finding is seen with the direct DA agonists bromocriptine and quinpirole [4,27]. The DA agonistenhanced CR effect is inhibited with both D1 and D2 receptor selective antagonists [1,26]. Thus, it seems that dopaminergic neurotransmission is necessary for both the acquisition and expression of CR. We have proposed a model of reward-related learning that stipulates the prerequisite of contiguous cholinergic and glutamatergic signals converging on a common reward substrate for learning to occur [25]. In this model, we hypothesize that the US signal may consist of mACh receptor stimulation and the reward-associated stimulus (eventual CS) signal of NMDA receptor stimulation on a common reward substrate. Through contiguous US and reward-associated stimulus stimulation of the reward substrate, the reward-associated stimulus gains the ability to activate the same reward neural circuits as the US (i.e., becomes a CS) and elicit the same reward behavior, independently of the US [5,8,25,27,37]. According to this model, antagonism of mACh receptor stimulation, and therefore prevention of contiguous US and CS stimulation of the common reward substrate, would prevent the reward-associated stimulus from gaining the ability to activate the reward substrate on its own. In the current procedure, such antagonism would impair the acquisition by the food-associated stimulus of the capacity to function as a CS and therefore also as a CR. Furthermore, because the CS functions independently of the US after learning has occurred, it suggests that in the current procedure, the antagonism of the US signal after conditioning should have no effect. Our data are consistent with these predictions—scopolamine, which presumably reduces the US (i.e., food) signal, when administered during learning prevented the
98
R. Nisanov et al. / Neuroscience Letters 614 (2016) 95–98
learning but when administered after learning failed to affect the expression of the learning. Thus, these data add further support for this model and expand it to include conditioned reward. Future research will test this model directly by investigating the effects of local mACh blockade specifically in the VTA on acquisition and expression of CR learning. Scopolamine has been shown to inhibit both the formation and the retrieval of memory in a multitude of studies investigating the neural mechanisms of memory [7,13,14,19]. In light of this, it might be expected that scopolamine would impair the acquisition and expression of CR. Our results, however, demonstrate that scopolamine impaired only the acquisition of CR but the expression (memory retrieval) was generally unimpaired. In the present study the light CS, is both a memory of a stimulus associated with food and a conditioned reward with incentive motivational qualities. Responding here is a function of the reinforcing effects of the light CS. Perhaps, mACh receptor stimulation is not as important for the conditioned reinforcement effects of a CS as it is for its memory. This is an interesting phenomenon that will require more research to better understand. 5. Conclusion The findings of this study suggest that mACh receptor stimulation is important for the acquisition by a food-associated stimulus of the ability to function as a CR but not necessary for this function once the acquisition of learning has occurred. References [1] R.J. Beninger, Role of D 1 and D 2 Receptors in Learning Dopamine Receptor Interactions, Academic Press Limited, 1993, pp. 115–157. [2] R.J. Beninger, D.R. Hanson, A.G. Phillips, The acquisition of responding with conditioned reinforcement: effects of cocaine, (+)-amphetamine and pipradrol, Brit. J. Pharmacol. 74 (1981) 149–154. [3] R.J. Beninger, A.G. Phillips, The effects of pimozide on the establishment of conditioned reinforcement, Psychopharmacology 68 (1980) 147–153. [4] R.J. Beninger, R. Ranaldi, The effects of amphetamine, apomorphine SKF 38393, quinpirole and bromocriptine on responding for conditioned reward in rats, Behav. Pharmacol. 3 (1992) 155–163. [5] D. Bindra, A motivational view of learning, performance, and behavior modification, Psychol. Rev. 81 (1974) 199–213. [6] J.R. Blackburn, A.G. Phillips, A. Jakubovic, H.C. Fibiger, Dopamine and preparatory behavior: II. A neurochemical analysis, Behav. Neurosci. 103 (1989) 15. [7] A. Blokland, Acetylcholine: a neurotransmitter for learning and memory? Brain Res. Rev. 21 (1995) 285–300. [8] R.C. Bolles, Reinforcement, expectancy, and learning, Psychol. Rev. 79 (1972) 394–409. [9] S.L. Cohen, Effects of d-amphetamine on responding under second order schedules of reinforcement with paired and non-paired brief stimuli, J. Exp. Anal. Behav. 56 (1991) 289–302. [10] S.L. Cohen, M.N. Branch, Food-paired stimuli as conditioned reinforcers: effects of d-amphetamine, J. Exp. Anal. Behav. 56 (1991) 277–288. [11] F. Files, M. Branch, D. Clody, Effects of methylphenidate on responding under extinction in the presence and absence of conditioned reinforcement, Behav. Pharmacol. 1 (1989) 113–122. [12] M. Garzón, R.A. Vaughan, G.R. Uhl, M.J. Kuhar, V.M. Pickel, Cholinergic axon terminals in the ventral tegmental area target a subpopulation of neurons expressing low levels of the dopamine transporter, J. Comp. Neurol. 410 (1999) 197–210. [13] M.E. Hasselmo, Neuromodulation: acetylcholine and memory consolidation, Trends Cogn. Sci. 3 (1999) 351–359.
[14] M.E. Hasselmo, The Role of Acetylcholine in Learning and Memory, 16 (2009) 710-715. [15] L. Hernandez, B.G. Hoebel, Food reward and cocaine increase extracellular dopamine in the nucleus accumbens as measured by microdialysis, Life Sci. 42 (1988) 1705–1712. [16] R. Hill, Facilitation of conditioned reinforcement as a mechanism of psychomotor stimulation, in: Amphetamines and Related Compounds, Raven, New York, 1970, pp. 781–795. [17] D.C. Hoffman, R.J. Beninger, The effects of pimozide on the establishment of conditioned reinforcement as a function of the amount of conditioning, Psychopharmacology 87 (1985) 454–460. [18] Y. Itzhak, J.L. Martin, Scopolamine inhibits cocaine-conditioned but not unconditioned stimulant effects in mice, Psychopharmacology (Berl.) 152 (2000) 216–223. [19] I. Klinkenberg, A. Blokland, The validity of scopolamine as a pharmacological model for cognitive impairment: a review of animal behavioral studies, Neurosci. Biobehav. Rev. 34 (2010) 1307–1350. [20] N.J. Mackintosh, The Psychology of Animal Learning, Academic Press, London, 1974. [21] E.J. Mazurski, R.J. Beninger, The effects of (+)-amphetamine and apomorphine on responding for a conditioned reinforcer, Psychopharmacology 90 (1986) 239–243. [22] W.E. Pratt, A.E. Kelley, Nucleus accumbens acetylcholine regulates appetitive learning and motivation for food via activation of muscarinic receptors, Behav. Neurosci. 118 (2004) 730–739. [23] P.V. Rada, G.P. Mark, J.J. Yeomans, B.G. Hoebel, Acetylcholine release in ventral tegmental area by hypothalamic self-stimulation, eating, and drinking, Pharmacol. Biochem. Be 65 (2000) 375–379. [24] F.S. Radhakishun, J.M. van Ree, B.H. Westerink, Scheduled eating increases dopamine release in the nucleus accumbens of food-deprived rats as assessed with on-line brain dialysis, Neurosci. Lett. 85 (1988) 351–356. [25] R. Ranaldi, Dopamine and reward seeking: the role of ventral tegmental area, Rev. Neurosci. 25 (5) (2014) 1–10. [26] R. Ranaldi, R.J. Beninger, Dopamine D1 and D2 antagonists attenuate amphetamine-produced enhancement of responding for conditioned reward in rats, Psychopharmacology (Berl.) 113 (1993) 110–118. [27] R. Ranaldi, D. Pantalony, R.J. Beninger, The D1 agonist, SKF 38393, attenuates amphetamine-produced enhancement of responding for conditioned reward in rats, Psychopharmacology 113 (1994) 110–118. [28] T. Robbins, The potentiation of conditioned reinforcement by psychomotor stimulant drugs. A test of Hill’s hypothesis, Psychopharmacology 45 (1975) 103–114. [29] T. Robbins, G. Koob, Pipradrol enhances reinforcing properties of stimuli paired with brain stimulation, Pharmacol. Biochem. Be 8 (1978) 219–222. [30] T. Robbins, B. Watson, M. Gaskin, C. Ennis, Contrasting interactions of pipradrol, d-amphetamine, cocaine, cocaine analogues, apomorphine and other drugs with conditioned reinforcement, Psychopharmacology 80 (1983) 113–119. [31] T.W. Robbins, Relationship between reward-enhancing and stereotypical effects of psychomotor stimulant drugs, Nature 264 (1976) 57–59. [32] F.M. Rotella, K. Olsson, V. Vig, I. Yenko, J. Pagirsky, I. Kohen, A. Aminov, T. Dindyal, R.J. Bodnar, Muscarinic and nicotinic cholinergic receptor antagonists differentially mediate acquisition of fructose-conditioned flavor preference and quinine-conditioned flavor avoidance in rats, Neuorbiol. Learn. Mem. 123 (2015) 239–249. [33] R. See, J. McLaughlin, R. Fuchs, Muscarinic receptor antagonism in the basolateral amygdala blocks acquisition of cocaine-stimulus association in a model of relapse to cocaine-seeking behavior in rats, Neuroscience 117 (2003) 477–483. [34] R. Sharf, J. McKelvey, R. Ranaldi, Blockade of muscarinic acetylcholine receptors in the ventral tegmental area prevents acquisition of food-rewarded operant responding in rats, Psychopharmacology (Berl.) 186 (2006) 113–121. [35] R. Sharf, R. Ranaldi, Blockade of muscarinic acetylcholine receptors in the ventral tegmental area disrupts food-related learning in rats, Psychopharmacology 184 (2006) 87–94. [36] B.F. Skinner The behavior of organisms An experimental analysis 1938. [37] R.A. Wise, Dopamine, learning and motivation, Nat. Rev. Neurosci. 5 (2004) 483–494.
Contents lists available at ScienceDirect
Neuroscience Letters journal homepage: www.elsevier.com/locate/neulet
Research paper
Treatment with a muscarinic acetylcholine receptor antagonist impairs the acquisition of conditioned reward learning in rats R. Nisanov a , E. Galaj a , R. Ranaldi a,b,∗ a b
Graduate Center of the City University of New York, United States Queens College of the City University of New York, Department of Psychology, United States
h i g h l i g h t s • The muscarinic antagonist, scopolamine, impairs the acquisition of conditioned reward learning. • The muscarinic antagonist, scopolamine, does not impair the expression of conditioned reward. • Muscarinic acetylcholine receptor stimulation is involved in the acquisition of conditioned reward learning.
a r t i c l e
i n f o
Article history: Received 15 November 2015 Received in revised form 19 December 2015 Accepted 31 December 2015 Available online 6 January 2016 Keywords: Conditioned reward Scopolamine Acetylcholine Muscarinic receptor Food Conditioned reinforcement
a b s t r a c t The neural mechanisms whereby a reward-associated stimulus gains reinforcing properties and comes to function as a conditioned reward (CR) are not understood. We propose that muscarinic acetylcholine (mACh) receptor stimulation is necessary for this type of learning. Here we tested the hypothesis that mACh receptor antagonism, with scopolamine, would attenuate the acquisition by a food-related stimulus of the capacity to function as a CR. Rats were exposed to 5 pre-exposure sessions during which two levers were present, one producing a light and the other a tone when pressed. This was followed by 3 conditioning sessions in which the levers were absent and the rats were presented with 30 light-food pairings delivered randomly. In the test session, the levers were present and presses on both levers were recorded. Different groups of rats received intraperitoneal injections of scopolamine (0, 0.375, 0.75 and 1 mg/kg) either prior to each conditioning session or prior to the test session. All groups showed significantly greater responding on the light lever in the test compared to the pre-exposure sessions, demonstrating the CR effect. In animals treated prior to conditioning the scopolamine groups pressed significantly less on the light lever than the vehicle group. In animals treated prior to the test the increased lever pressing for light was similar for all groups. These data suggest that scopolamine impaired the acquisition of CR but not its expression. The results support the hypothesis that mACh receptor stimulation is important for the acquisition by reward-associated stimuli of the ability to function as CRs. © 2016 Elsevier Ireland Ltd. All rights reserved.
1. Introduction Reward-related learning is a vital aspect of an organism’s ability to adapt to the environment. Learning about primary rewards (unconditioned stimuli; USs), such as food and water, also leads to learning about reward-related stimuli. A previously neutral stimulus repeatedly paired with a US can become a conditioned stimulus (CS) capable of eliciting behaviors that are similar or related to the US [20]. Furthermore, the CS is also capable of gaining reinforcing
∗ Corresponding author at: Psychology Department, Queens College, 65-30 Kissena Blvd, Flushing, NY 11367, United States. Fax: +1 718 997 3257. E-mail address: [email protected] (R. Ranaldi). http://dx.doi.org/10.1016/j.neulet.2015.12.064 0304-3940/© 2016 Elsevier Ireland Ltd. All rights reserved.
properties of its own, thereby, functioning as a conditioned reward (CR) [4,36]. The neural mechanisms whereby CSs gain reinforcing properties and become CRs are not fully understood and, therefore, constitute an aim of the current study. One possibility is that the neuronal pathways that transmit US signals converge with the neuronal pathways that mediate potential CS signals as inputs to the pathway that mediates the unconditioned response. The dual stimulation by the eventual CS and US of relevant neurons allows the CS to gain the capability of stimulating the same motivational neural circuits as the US and elicit the same motivational behavior, independently of the US [5,8,27,37]. If this is the case, then blocking one of these inputs to the reward-relevant pathway could prevent the acquisition of such learning.
96
R. Nisanov et al. / Neuroscience Letters 614 (2016) 95–98
Previous work suggests that the US signal mediated by food involves, at least, the release of acetylcholine (ACh) in the ventral tegmental area (VTA) [12,23], although ACh in other brain regions may also be important [22]. That ACh activity in the brain serves as a US signal for food is supported by studies showing that the blockade of muscarinic ACh (mACh) receptor stimulation in the VTA reduces eating [23,34] and impairs the acquisition, although not the expression, of food-related learning [35]. If this is so, then the blockade of mACh receptor stimulation should prevent the acquisition of reinforcing properties by a food-paired stimulus or CS. In this event the CS would also not function as a CR. In the present paper, we tested this hypothesis. Specifically, we hypothesized that systemic treatment with scopolamine, a mACh receptor antagonist, would attenuate or prevent the acquisition by a food-related stimulus of the capacity to function as a CR. 2. Methods 2.1. Subjects Subjects were 64 male, facility bred Long-Evans rats, individually housed in a climate controlled room on a 12 hour light/12 hour dark cycle (lights off at 6 AM). The rats, with initial free feeding weights of approximately 350 g (325–375 g), were maintained at 85% of their free-feeding weights throughout the experiment by daily rationed food portions. Water was freely accessible at all times except during behavioral test sessions.
Fig. 1. Mean number of presses on the light and tone levers during the pre-exposure and test phases for rats treated with scopolamine (0, 0.375, 0.75 and 1.5 mg/kg) prior to each conditioning session. Vertical lines represent the standard error of the mean (SEM). * represents a significant difference from vehicle in light lever presses.
The mACh receptor antagonist, scopolamine hydrobromide (Sigma–Aldrich, St. Louis, MO), was dissolved in 0.9% saline, at doses of 0, 0.375, 0.75 and 1.5 mg/kg.
0.375, 0.75 or 1.5 mg/kg). In this phase, both levers were removed from the operant chambers. During each conditioning session, rats were presented with a 3-sec light presentation followed by a food pellet delivery for a total of 30 light stimuli paired with 30 food pellets. After completion of this phase, there was a 2-day rest period. The test phase consisted of one 40 min session during which both levers were present and pressing one lever produced the tone and the other the light as in the pre-exposure phase. The number of presses on each lever was recorded.
2.3. Apparatus
2.6. Expression
Behavioral testing was conducted in 8 operant conditioning chambers housed in sound- and light-attenuating boxes. Each operant conditioning chamber, measuring 30.5 × 24.0 × 25.0 cm (l × w × h), had two aluminum walls, Plexiglas sidewalls and ceiling. The floor consisted of 0.60-cm diameter stainless steel rods spaced 2.0 cm apart. Each chamber was equipped with two levers, two white stimulus lights, a tone generator, and a food trough, all on the right wall. The food trough, measuring 3.0 × 3.6 × 2.0 cm (l × w × h), was centered between the two levers. Each lever was located 3.0 cm to the right and left of the food trough and 8.0 cm above the floor. Each white stimulus light was positioned 3 cm above each lever.
The conditioned reward paradigm was the same as in acquisition except that IP treatments of scopolamine (0, 0.375, 0.75 and 1.5 mg/kg) were administered prior to the test session and not prior to each conditioning session.
2.2. Drugs
2.4. Procedure
2.7. Data analysis The average number of responses made on each lever during the pre-exposure phase and the test phase was analyzed for each group. A three-way mixed design analyses of variance (ANOVA) was conducted with the following three factors; phase (pre-exposure and test), scopolamine dose (0, 0.375, 0.75 and 1.5 mg/kg) and lever (light and tone). Significant 3-way interactions were followed by interaction comparisons and tests of simple effects. Significant simple effects were further analyzed using Dunnett’s post hoc tests.
The conditioned reward paradigm consisted of three phases; pre-exposure, conditioning and test phases. This paradigm tested the acquisition and expression of reward related learning.
3. Results
2.5. Acquisition
3.1. Acquisition
During the pre-exposure phase, rats were placed in the operant conditioning chambers for 40 min sessions held once a day for five consecutive days. Pressing one lever produced a 3-s tone presentation while pressing the other lever resulted in a 3-s light presentation above that lever. The number of presses on each lever was recorded. The conditioning phase consisted of three 60 min sessions held on separate days with one day between sessions. Thirty minutes prior to being placed in the operant chambers, all rats were treated with an intraperitoneal (IP) injection of one dose of scopolamine (0,
In the pre-exposure phase, the numbers of lever presses on the light and tone levers were low with slightly more responding on the light lever. This pattern was similar in all groups (see Fig. 1). In the test phase, the vehicle group showed a large increase in light lever responding with little change in tone lever responding compared to the pre-exposure phase. The scopolamine groups showed no significant changes in responding on the tone lever but significantly large increases in responding on the light lever in the test phase compared to pre-exposure. However, the increases in light lever responding in the scopolamine groups were
R. Nisanov et al. / Neuroscience Letters 614 (2016) 95–98
Fig. 2. Mean number of presses on the light and tone levers during the pre-exposure and test phases for rats treated with scopolamine (0, 0.375, 0.75 and 1.5 mg/kg) prior to the test session. Vertical lines represent the standard error of the mean (SEM).
significantly smaller than in the vehicle group in a dose-related fashion (the higher the dose, the smaller the increase). A three-way ANOVA revealed a significant group × phase × lever interaction [F(3,28) = 5.676, p = 0.004]. Follow-up interaction comparisons at the level of each lever revealed a significant group × phase interaction for the light lever only, [F(3,28) = 8.427, p < 0.05]. Tests of simple effects of group at each level of phase revealed a significant group effect in the test phase [F(3,28) = 17.551, p < 0.05]. Dunnett’s post hoc tests for group differences on the lever presses for light revealed that all scopolamine groups were significantly different from the vehicle group, ps < .05. 3.2. Expression As in acquisition, the amount of lever presses for light and tone in the pre-exposure phase was low with slightly more responding on the light lever for all groups (Fig. 2). Two groups, the vehicle and 0.75 mg scopolamine groups, showed no significant changes in responding on the tone lever but all groups showed very large increases in responding on the light lever in the test compared to the pre-exposure phases. The increased lever pressing for light was similar for all groups. These observations were supported by statistical analyses. A three-way ANOVA revealed a significant phase × lever interaction, [F(1,28) = 155.32, p < 0.001] which did not interact with the group factor. Tests of simple effects of phase at each level of lever revealed a significant difference on the light lever presses between pre-exposure phase and test phase [F(1,28) = 325.86, p < 0.001]. 4. Discussion These experiments tested the hypothesis that antagonism of mACh receptors during acquisition of a stimulus-reward learning task would impair learning, which would be manifested as an attenuation of the capacity of the CS to function as a conditioned reward. Systemic administration of scopolamine prior to food-CS conditioning sessions resulted in a significant and dose-related reduction in the active lever presses (i.e., presses reinforced by a conditioned reward) when tested after conditioning. All animals consumed the same number of food pellets under scopolamine treatment, arguing against the possibility that the reduced CR effect occurred because of reduced food consumption or, perhaps, fewer stimulus-reward
97
pairings. Instead, these results suggest that mACh receptor stimulation is important for the acquisition by a food-associated stimulus of the capacity to function as a CR. In contrast, when scopolamine was administered during the expression phase—after conditioning already occurred—it produced no significant changes among groups on active lever presses. Thus, it seems from these data that mACh neurotransmission is necessary for the acquisition, but not the expression, of the CR effect. Muscarinic acetylcholine neurotransmission is believed to be an important component of reward-related learning. Systemic muscarinic receptor antagonism eliminated the acquisition of fructose conditioned flavor preference in rats [32] and cocaine conditioned place preference (CPP) in mice [18]. Antagonism of muscarinic receptors in the nucleus accumbens core or shell impaired acquisition and expression of lever pressing for sucrose [22]. Furthermore, scopolamine administration into the basolateral amygdala diminished the capability of a cocaine-associated stimulus to acquire conditioned rewarding properties when administered during acquisition but not when administered after the behavior was acquired [33]. Similarly, antagonism of mACh receptors in the ventral tegmental area (VTA) resulted in a significant dose-related impairment in the acquisition of food approach-related behaviors but not the expression of these behaviors when the treatment occurred after learning was already demonstrated [34,35]. The present results are the first to demonstrate that not only is mACh receptor stimulation necessary for the acquisition by rewardassociated stimuli of the ability to function as CSs but also important for such stimuli to function as CRs. Dopamine (DA) and ACh, both, have been directly implicated in reward-related learning. The presentation of a food-paired CS produced an increase in DA release in striatal regions [6,15,24] as well as ACh release in the VTA [12,23]. Blockade of DA neurotransmission during reward-stimulus conditioning resulted in diminished responding for CR as tested after conditioning [3,17,27]. Increased DA neurotransmission by indirect DA agonists, pripradol, amphetamine and cocaine, increased responding for CR [2,3,9–11,16,21,28–31] and a similar finding is seen with the direct DA agonists bromocriptine and quinpirole [4,27]. The DA agonistenhanced CR effect is inhibited with both D1 and D2 receptor selective antagonists [1,26]. Thus, it seems that dopaminergic neurotransmission is necessary for both the acquisition and expression of CR. We have proposed a model of reward-related learning that stipulates the prerequisite of contiguous cholinergic and glutamatergic signals converging on a common reward substrate for learning to occur [25]. In this model, we hypothesize that the US signal may consist of mACh receptor stimulation and the reward-associated stimulus (eventual CS) signal of NMDA receptor stimulation on a common reward substrate. Through contiguous US and reward-associated stimulus stimulation of the reward substrate, the reward-associated stimulus gains the ability to activate the same reward neural circuits as the US (i.e., becomes a CS) and elicit the same reward behavior, independently of the US [5,8,25,27,37]. According to this model, antagonism of mACh receptor stimulation, and therefore prevention of contiguous US and CS stimulation of the common reward substrate, would prevent the reward-associated stimulus from gaining the ability to activate the reward substrate on its own. In the current procedure, such antagonism would impair the acquisition by the food-associated stimulus of the capacity to function as a CS and therefore also as a CR. Furthermore, because the CS functions independently of the US after learning has occurred, it suggests that in the current procedure, the antagonism of the US signal after conditioning should have no effect. Our data are consistent with these predictions—scopolamine, which presumably reduces the US (i.e., food) signal, when administered during learning prevented the
98
R. Nisanov et al. / Neuroscience Letters 614 (2016) 95–98
learning but when administered after learning failed to affect the expression of the learning. Thus, these data add further support for this model and expand it to include conditioned reward. Future research will test this model directly by investigating the effects of local mACh blockade specifically in the VTA on acquisition and expression of CR learning. Scopolamine has been shown to inhibit both the formation and the retrieval of memory in a multitude of studies investigating the neural mechanisms of memory [7,13,14,19]. In light of this, it might be expected that scopolamine would impair the acquisition and expression of CR. Our results, however, demonstrate that scopolamine impaired only the acquisition of CR but the expression (memory retrieval) was generally unimpaired. In the present study the light CS, is both a memory of a stimulus associated with food and a conditioned reward with incentive motivational qualities. Responding here is a function of the reinforcing effects of the light CS. Perhaps, mACh receptor stimulation is not as important for the conditioned reinforcement effects of a CS as it is for its memory. This is an interesting phenomenon that will require more research to better understand. 5. Conclusion The findings of this study suggest that mACh receptor stimulation is important for the acquisition by a food-associated stimulus of the ability to function as a CR but not necessary for this function once the acquisition of learning has occurred. References [1] R.J. Beninger, Role of D 1 and D 2 Receptors in Learning Dopamine Receptor Interactions, Academic Press Limited, 1993, pp. 115–157. [2] R.J. Beninger, D.R. Hanson, A.G. Phillips, The acquisition of responding with conditioned reinforcement: effects of cocaine, (+)-amphetamine and pipradrol, Brit. J. Pharmacol. 74 (1981) 149–154. [3] R.J. Beninger, A.G. Phillips, The effects of pimozide on the establishment of conditioned reinforcement, Psychopharmacology 68 (1980) 147–153. [4] R.J. Beninger, R. Ranaldi, The effects of amphetamine, apomorphine SKF 38393, quinpirole and bromocriptine on responding for conditioned reward in rats, Behav. Pharmacol. 3 (1992) 155–163. [5] D. Bindra, A motivational view of learning, performance, and behavior modification, Psychol. Rev. 81 (1974) 199–213. [6] J.R. Blackburn, A.G. Phillips, A. Jakubovic, H.C. Fibiger, Dopamine and preparatory behavior: II. A neurochemical analysis, Behav. Neurosci. 103 (1989) 15. [7] A. Blokland, Acetylcholine: a neurotransmitter for learning and memory? Brain Res. Rev. 21 (1995) 285–300. [8] R.C. Bolles, Reinforcement, expectancy, and learning, Psychol. Rev. 79 (1972) 394–409. [9] S.L. Cohen, Effects of d-amphetamine on responding under second order schedules of reinforcement with paired and non-paired brief stimuli, J. Exp. Anal. Behav. 56 (1991) 289–302. [10] S.L. Cohen, M.N. Branch, Food-paired stimuli as conditioned reinforcers: effects of d-amphetamine, J. Exp. Anal. Behav. 56 (1991) 277–288. [11] F. Files, M. Branch, D. Clody, Effects of methylphenidate on responding under extinction in the presence and absence of conditioned reinforcement, Behav. Pharmacol. 1 (1989) 113–122. [12] M. Garzón, R.A. Vaughan, G.R. Uhl, M.J. Kuhar, V.M. Pickel, Cholinergic axon terminals in the ventral tegmental area target a subpopulation of neurons expressing low levels of the dopamine transporter, J. Comp. Neurol. 410 (1999) 197–210. [13] M.E. Hasselmo, Neuromodulation: acetylcholine and memory consolidation, Trends Cogn. Sci. 3 (1999) 351–359.
[14] M.E. Hasselmo, The Role of Acetylcholine in Learning and Memory, 16 (2009) 710-715. [15] L. Hernandez, B.G. Hoebel, Food reward and cocaine increase extracellular dopamine in the nucleus accumbens as measured by microdialysis, Life Sci. 42 (1988) 1705–1712. [16] R. Hill, Facilitation of conditioned reinforcement as a mechanism of psychomotor stimulation, in: Amphetamines and Related Compounds, Raven, New York, 1970, pp. 781–795. [17] D.C. Hoffman, R.J. Beninger, The effects of pimozide on the establishment of conditioned reinforcement as a function of the amount of conditioning, Psychopharmacology 87 (1985) 454–460. [18] Y. Itzhak, J.L. Martin, Scopolamine inhibits cocaine-conditioned but not unconditioned stimulant effects in mice, Psychopharmacology (Berl.) 152 (2000) 216–223. [19] I. Klinkenberg, A. Blokland, The validity of scopolamine as a pharmacological model for cognitive impairment: a review of animal behavioral studies, Neurosci. Biobehav. Rev. 34 (2010) 1307–1350. [20] N.J. Mackintosh, The Psychology of Animal Learning, Academic Press, London, 1974. [21] E.J. Mazurski, R.J. Beninger, The effects of (+)-amphetamine and apomorphine on responding for a conditioned reinforcer, Psychopharmacology 90 (1986) 239–243. [22] W.E. Pratt, A.E. Kelley, Nucleus accumbens acetylcholine regulates appetitive learning and motivation for food via activation of muscarinic receptors, Behav. Neurosci. 118 (2004) 730–739. [23] P.V. Rada, G.P. Mark, J.J. Yeomans, B.G. Hoebel, Acetylcholine release in ventral tegmental area by hypothalamic self-stimulation, eating, and drinking, Pharmacol. Biochem. Be 65 (2000) 375–379. [24] F.S. Radhakishun, J.M. van Ree, B.H. Westerink, Scheduled eating increases dopamine release in the nucleus accumbens of food-deprived rats as assessed with on-line brain dialysis, Neurosci. Lett. 85 (1988) 351–356. [25] R. Ranaldi, Dopamine and reward seeking: the role of ventral tegmental area, Rev. Neurosci. 25 (5) (2014) 1–10. [26] R. Ranaldi, R.J. Beninger, Dopamine D1 and D2 antagonists attenuate amphetamine-produced enhancement of responding for conditioned reward in rats, Psychopharmacology (Berl.) 113 (1993) 110–118. [27] R. Ranaldi, D. Pantalony, R.J. Beninger, The D1 agonist, SKF 38393, attenuates amphetamine-produced enhancement of responding for conditioned reward in rats, Psychopharmacology 113 (1994) 110–118. [28] T. Robbins, The potentiation of conditioned reinforcement by psychomotor stimulant drugs. A test of Hill’s hypothesis, Psychopharmacology 45 (1975) 103–114. [29] T. Robbins, G. Koob, Pipradrol enhances reinforcing properties of stimuli paired with brain stimulation, Pharmacol. Biochem. Be 8 (1978) 219–222. [30] T. Robbins, B. Watson, M. Gaskin, C. Ennis, Contrasting interactions of pipradrol, d-amphetamine, cocaine, cocaine analogues, apomorphine and other drugs with conditioned reinforcement, Psychopharmacology 80 (1983) 113–119. [31] T.W. Robbins, Relationship between reward-enhancing and stereotypical effects of psychomotor stimulant drugs, Nature 264 (1976) 57–59. [32] F.M. Rotella, K. Olsson, V. Vig, I. Yenko, J. Pagirsky, I. Kohen, A. Aminov, T. Dindyal, R.J. Bodnar, Muscarinic and nicotinic cholinergic receptor antagonists differentially mediate acquisition of fructose-conditioned flavor preference and quinine-conditioned flavor avoidance in rats, Neuorbiol. Learn. Mem. 123 (2015) 239–249. [33] R. See, J. McLaughlin, R. Fuchs, Muscarinic receptor antagonism in the basolateral amygdala blocks acquisition of cocaine-stimulus association in a model of relapse to cocaine-seeking behavior in rats, Neuroscience 117 (2003) 477–483. [34] R. Sharf, J. McKelvey, R. Ranaldi, Blockade of muscarinic acetylcholine receptors in the ventral tegmental area prevents acquisition of food-rewarded operant responding in rats, Psychopharmacology (Berl.) 186 (2006) 113–121. [35] R. Sharf, R. Ranaldi, Blockade of muscarinic acetylcholine receptors in the ventral tegmental area disrupts food-related learning in rats, Psychopharmacology 184 (2006) 87–94. [36] B.F. Skinner The behavior of organisms An experimental analysis 1938. [37] R.A. Wise, Dopamine, learning and motivation, Nat. Rev. Neurosci. 5 (2004) 483–494.