- Email: [email protected]
Response acquisition with delayed reinforcement in Lewis and Fischer 344 rats
Behavioural Processes 74 (2007) 311–318
Response acquisition with delayed reinforcement in Lewis and Fischer 344 rats Karen G. Anderson ∗ , Mirari Elcoro West Virginia University, United States Received 15 May 2006; received in revised form 2 November 2006; accepted 28 November 2006
Abstract Previous work has shown neurochemical and behavioral differences between Lewis rats and Fischer 344 rats. Some of this work suggests that there might be differential sensitivity to delayed reinforcement between the two strains. To further explore this possibility, Lewis (n = 8) and Fischer 344 (n = 8) rats were exposed to a response–acquisition task with a non-resetting 20 s delay to reinforcement. A tandem fixed-ratio 1, fixed-time 20 s schedule of reinforcement was programmed for one of two levers; presses on the alternate lever had no programmed consequences. A greater number of Lewis rats (5/8) acquired lever pressing compared to the Fischer 344 rats (2/8). Future work with these strains may lead to a better understanding of the genetic and/or neurochemical factors involved in temporal control of behavior. © 2006 Elsevier B.V. All rights reserved. Keywords: Acquisition; Delay of reinforcement; Fischer 344; Lever press; Lewis; Rat
1. Introduction It is well established that the rate or frequency of an existing behavior may be weakened as a result of delaying presentation of the reinforcer that maintains it (cf. Anderson and Woolverton, 2003; Beardsley and Balster, 1993; Chung and Herrnstein, 1967; Lattal, 1984; Mazur, 1987; Myerson and Green, 1995). However, it also has been demonstrated that organisms can learn a new behavior when the consequence, i.e., reinforcement, is not immediate or signaled, and the response is not explicitly shaped (Lattal and Gleeson, 1990). Response acquisition with delayed reinforcement has been established with various species and response topographies (e.g., Critchfield and Lattal, 1993; Lattal and Gleeson, 1990; Lattal and Metzger, 1994; van Haaren, 1992). What is not well known, is how effects of delayed consequences on the acquisition and/or maintenance of behavior may differ as a function of individual differences, e.g., age, sex, species, behavioral history, genetics, and neurochemistry. One line of research that employs delayed reinforcement, with previously established behavior, is the area of self-
∗
Corresponding author at: West Virginia University, Department of Psychology, P.O. Box 6040, Morgantown, WV 26506, United States. E-mail address: [email protected] (K.G. Anderson). 0376-6357/$ – see front matter © 2006 Elsevier B.V. All rights reserved. doi:10.1016/j.beproc.2006.11.006
control/impulsive choice. In a typical paradigm (cf. Evenden and Ryan, 1996; Mazur, 1987), a choice is presented between a larger, more delayed reinforcer and a smaller, more immediate reinforcer. An individual is said to behave “impulsively” if the smaller, sooner reinforcer is chosen over the larger, later reinforcer and the outcome is a reduction in overall reinforcement. One suggested reason for the impulsive choice often seen in these paradigms is that the larger reinforcer loses some of its value, i.e., becomes less reinforcing, as the delay to its presentation is increased. Eventually, the reinforcing effectiveness of the larger, delayed reinforcer will fall below that of the smaller, immediate reinforcer and this is indicated by a switch in preference, such that the “impulsive” option is selected over the “self-controlled” option. This decrease in the larger reinforcer’s reinforcing effectiveness (or value) as a result of its delay to presentation is called delay (or temporal) discounting (Mazur, 1987) and individual differences in rates of delay discounting have been observed in human and non-human subjects. In particular, two strains of rats have been shown to differ in their choices of delayed reinforcers. Lewis and Fischer 344 rats were shown to differ in the number of large reinforcer (three food pellet) choices they emitted when a single, immediate food pellet was the alternative (Anderson and Woolverton, 2005). In particular, the Lewis rats emitted more small reinforcer (impulsive) choices than the Fischer 344 rats
312
K.G. Anderson, M. Elcoro / Behavioural Processes 74 (2007) 311–318
and had higher rates of delay discounting as indicated by steeper slopes in the delay-discounting function and by lower indifference points (the delay value where the larger reinforcer is equal to that of the smaller, immediate reinforcer). One interpretation of this outcome is that the Lewis rats’ lever pressing was more sensitive to the effects of delaying reinforcement, and this may be a result of neurochemical and/or genetic differences between the strains. Lewis rats have been shown to have differences (decreased activity) in dopaminergic (DA) and serotonergic (5-HT) systems in various brain regions compared to Fischer 344 rats (Burnet et al., 1996; Flores et al., 1998; Lindley et al., 1999; Martin et al., 2003; Selim and Bradberry, 1996). In particular, Lewis rats have fewer 5-HTIA binding sites and less 5-HT in some brain regions (Burnet et al., 1996; Selim and Bradberry, 1996) and fewer DA transporters and lower levels of DA in some areas of the brain (Flores et al., 1998; Lindley et al., 1999) compared to Fischer 344 rats. It is not known, however, what roles, if any, such differences play in acquisition or maintenance of behavior with delayed reinforcement. There is some evidence that 5-HT and/or DA in particular brain regions may influence effects of temporal variables (cf. Buhusi, 2003; Cardinal et al., 2001; Kheramin et al., 2004; Mobini et al., 2000; Wogar et al., 1992), but the findings have been mixed. In general, however, increased DA and 5-HT have each individually been linked to overestimations of the passage of time (e.g., Body et al., 2004; Matell et al., 2006) and lesions of these pathways have been shown to affect sensitivity to temporal events (e.g., Kheramin et al., 2004; Mobini et al., 2000; Morrissey et al., 1993; Wogar et al., 1992). A greater understanding of the variables that affect sensitivity to immediate versus delayed outcomes has much clinical and applied relevance, e.g., attention-deficit/hyperactivity disorder (ADHD), drug abuse, gambling. Studies of effects of drugs on response acquisition with immediate versus delayed reinforcement may also yield data with relevance for behavioral and/or neurochemical underpinnings for performance in such tasks (cf. Byrne et al., 1997; LeSage et al., 1996). To investigate strain differences in effects of delayed consequences on the acquisition of a response, Lewis and Fischer 344 rats with no experimental history were allowed to press two levers in a standard operant chamber. One lever was associated with the presentation of a food pellet after a non-resetting 20-s delay following the lever press. No food was delivered for pressing the alternate lever. Although other procedures may have been employed, the non-resetting delay was chosen to increase the likelihood that lever pressing would be acquired since it is possible that the obtained delay values may be shorter than the programmed value (cf. Wilkenfield et al., 1992). The delay duration was selected because it previously resulted in differential responding between the two strains when the choice was between one immediate food pellet and three food pellets following a 20-s delay (Anderson and Woolverton, 2005). It was hypothesized, based on the findings of Anderson and Woolverton (2005), that the Lewis rats would take longer than the Fischer 344 rats to acquire the response on the lever associated with delayed food presentation.
2. Methods 2.1. Subjects Eight experimentally naive, male Lewis (approximately 11 weeks old) and eight experimentally naive, male Fischer 344 rats (approximately 13 weeks old) (Harlan Sprague Dawley, IN) served as subjects. Each subject was housed individually and water was always available in each home cage. Illumination was maintained on a reversed 12:12 h dark/light cycle and humidity and temperature were kept constant. Subjects were fed 12–15 g of rat chow (Harlan, WI) daily, at approximately the same time of day unless otherwise specified. Animals were maintained in accordance with National Institutes of Health guidelines for Care and Use of Laboratory Animals. All experimental procedures were approved by the West Virginia University Animal Use and Care Committee and were conducted in facilities that were approved by the Association for Assessment and Accreditation of Laboratory Animal Care (AAALAC). 2.2. Apparatus Experimental sessions were conducted in eight standard operant chambers for rats, each enclosed in a melamine sound-attenuating cubicle (Med Associates, VT). Each chamber contained a working area of 30.5 cm by 24.1 cm by 21.0 cm, a grid floor, and a 45 mg pellet dispenser with a pellet receptacle centered between two standard response levers that were elevated 8 cm from the grid floor. Levers were positioned 11.5 cm apart from each other, were 4.8 cm wide, protruded 1.9 cm into the chamber and required at least 0.25 N for a response to be recorded. Two 28 V stimulus lights 2.5 cm in diameter were placed approximately 7 cm above each lever. Each chamber had a 28 V houselight on the wall opposite to the working wall, and a ventilation fan to circulate air and to mask extraneous noise. Equipment was interfaced to a computer and routines were programmed and conducted with MedPC-IV (Med Associates, VT). 2.3. Procedure 2.3.1. Feeder training All experimental sessions were conducted during the dark cycle, at approximately the same time. Before the start of the session, both levers were removed from each chamber and replaced with filler panels. Subjects were restricted from food for approximately 22 h before feeder training started. At the beginning of the session, each subject was placed in a darkened chamber with the ventilation fan on. Immediately after, the houselight was illuminated and 45 mg sucrose pellets (BioServ, NJ) were delivered according to a variable-time (VT) 60-s schedule (Fleshler and Hoffman, 1962). Pellet delivery was signaled by a 0.5 s flash of the houselight. Sessions ended after 60 pellets were delivered and were conducted on adjacent days for each group. Post-session feeding for all animals consisted of 10–12 g of rat chow and no additional food was available until the lever-press acquisition session.
K.G. Anderson, M. Elcoro / Behavioural Processes 74 (2007) 311–318
2.3.2. Lever-press acquisition This 8-h session was conducted approximately 48 h after feeder training (adjacent days for each group of rats). Sessions started at approximately 08:30 for both groups. At that time, subjects had been restricted from food for approximately 36 h. At the start of the session, each subject was placed in a darkened chamber with an activated ventilation fan. Responses on both levers were recorded but had no other programmed consequences. After 5 min, the houselight and both lever lights were illuminated. For half the subjects of each group, a left lever press initiated a tandem fixed-ratio (FR) 1, fixed-time (FT) 20-s schedule of reinforcement. The 20-s delay between the lever press and the pellet delivery was unsignaled and non-resetting. Lever presses during the delay were recorded but had no other programmed outcome. Right lever presses had no programmed consequences. These conditions were counterbalanced for the remaining half of the subjects in each group. The lever associated with the tandem FR 1 FT 20 s schedule of reinforcement was referred to as the operative lever, and the lever associated with no programmed consequences was referred to as the inoperative lever. Responses on each lever and the total number of pellets obtained were recorded during the session. 2.3.3. Data analysis Statistical analyses of the data (t-tests) were used to determine significant differences between groups in terms of body weight, response rate on each lever, and time of lever-press acquisition. Response rates were calculated by dividing the total number of responses on each lever by the total time for the session (responses per minute; rpm). Lever-press acquisition was characterized by the presence of a positively accelerated increase in cumulative responses as evidenced upon visual inspection of the data presented in a cumulative record. For quantitative analyses, lever-press acquisition was defined by the time when 25% of the total number of responses occurred. To do this, each session was divided into 1-min bins. Then the bin in which 25% of the total number of responses occurred was identified. As a measure of variability, standard errors around the mean (S.E.M.) were calculated for all the data presented. 3. Results At the start of the experiment, Lewis rats weighed an average of 240 g (S.E.M. = 6.87). Fischer 344 rats weighed an average of 311 g (S.E.M. = 2.93). This difference in weight was statistically significant [t(7) = 7.80, p < 0.001 (two-tailed)]. During feeder training, all subjects were observed to have consumed at least 60% of the pellets delivered. (Most rats consumed 100% of the pellets, but it was noted that some of the Lewis rats left a few pellets with one Lewis rat leaving 40% of the pellets.) During the acquisition session, subjects were exposed to the levers in the dark during a 5-min blackout. Lever-pressing rates during this time were low (range: 0.0–2.2 rpm). On average, the Lewis rats (M = 1.08 rpm) responded more during the 5-min blackout than the Fischer 344 rats (M = 0.3 rpm) and this difference was statistically significant [t(7) = 2.66, p = 0.032
313
(two-tailed)]. A t-test revealed no significant differences in lever pressing during the blackout and later acquisition. Table 1 shows the number of responses on each lever for each subject during lever-press acquisition. Overall, the eight Lewis rats emitted higher rates of responding (M = 0.31 rpm, S.E.M. = 0.08) than the eight Fischer 344 rats (M = 0.07 rpm, S.E.M. = 0.04) on the operative lever. This difference was statistically significant [t(7) = − 4.13, p = 0.004 (two-tailed)]. Also, Lewis rats responded at higher rates (M = 0.14 rpm, S.E.M. = 0.03) than Fischer 344 rats (M = 0.02 rpm, S.E.M. = 0.01) on the inoperative lever. This difference was also statistically significant [t(7) = − 3.51, p = 0.01 (two-tailed)]. For the rats that acquired lever pressing in both groups, the rate of lever pressing on the operative lever was greater than the rate on the inoperative lever. The five Lewis rats averaged 0.45 rpm (S.E.M. = 0.57) on the operative lever and 0.18 rpm (S.E.M. = 0.05) on the inoperative lever. The two Fischer 344 rats averaged 0.24 rpm (S.E.M. = 0.02) on the operative lever and 0.05 rpm (S.E.M. = 0.02) on the inoperative lever. These differences between operative-lever responses and inoperativelever responses in each group were statistically significant for the Fischer 344 rats [t(1) = 39, p = 0.02 (two-tailed)] and for the Lewis rats [t(4) = 2.75, p = 0.05 (two-tailed)]. It should be noted, however, that the statistical power in these analyses is reduced due to the small number of Fischer 344 rats that acquired the response. When comparing all eight rats within a strain, including those that acquired lever pressing and those that did not acquire lever pressing, more responding occurred on the operative lever than the inoperative lever, but the differences were not statistically significant in either group (Lewis: operative Table 1 Number of lever presses emitted by Lewis and Fischer 344 rats on the operative and inoperative levers during the 8 h lever-press acquisition session
Lewis L30-1 L30-2 L30-3 L30-4 L30-5 L30-6 L30-7 L30-8 M S.E.M. Fischer 344 F30-1 F30-2 F30-3 F30-4 F30-5 F30-6 F30-7 F30-8 M S.E.M.
Operative
Inoperative
263 130 252 32 39 27 177 262
27 124 77 36 35 32 150 51
147.75 37.36
66.50 16.53
124 6 2 106 0 9 18 21
31 1 4 13 2 2 23 15
35.75 17.56
11.38 3.95
314
K.G. Anderson, M. Elcoro / Behavioural Processes 74 (2007) 311–318
Table 2 Total number of pellets obtained per subject in each group during the lever-press acquisition session
Table 3 Time in the session (min) when 25% of the total number of lever presses on the operative lever were emitted for Lewis and Fischer 344 rats
Total pellets obtained
25% of the total responses (min)
Lewis L30-1 L30-2 L30-3 L30-4 L30-5 L30-6 L30-7 L30-8 M S.E.M. Fischer 344 F30-1 F30-2 F30-3 F30-4 F30-5 F30-6 F30-7 F30-8 M S.E.M.
163 91 127 15 27 16 120 175 91.75 23.09 63 4 1 70 0 5 11 13 20.88 10.10
M = 0.31 rpm, inoperative M = 0.14 rpm, p > 0.05; Fischer 344 operative M = 0.07 rpm, inoperative M = 0.02 rpm, p > 0.05). The total number of pellets obtained per subject is presented in Table 2. Lewis rats obtained an average of 91.75 pellets and Fischer 344 rats obtained an average of 20.88 pellets throughout the session. This difference was statistically significant [t(7) = −2.83, p = 0.03 (two-tailed)]. Figs. 1 and 2 depict cumulative responses for operative (solid line) and inoperative (dashed line) levers for Lewis and Fischer 344 rats, respectively. These figures show that five Lewis rats and two Fischer 344 rats had a positively accelerated pattern of responding once responding was initiated. Note also that two of the Lewis rats (L30-2 and L30-7) acquired lever pressing on the inoperative lever at approximately the same rate as the operative lever. To further characterize lever-press acquisition on the operative lever, Table 3 shows the 1-min bin in which each rat that acquired the response emitted 25% of the total number of responses. Five of eight Lewis rats emitted a quarter of the lever presses at an average of 216.40 min (S.E.M. = 43.58) into the session. Out of eight Fischer 344 rats, the two that acquired lever pressing emitted a quarter of the lever presses at an average of 334.50 min (S.E.M. = 48.50) into the session. Even though, on average, Lewis rats acquired the lever pressing earlier in the session than Fischer 344 rats, this difference was not statistically significant [t(7) = 1.53, p = 0.49 (two-tailed)]. 4. Discussion Strain differences in response acquisition with a 20-s (nonresetting) delay to reinforcement were observed between Lewis
Lewis L30-1 L30-2 L30-3 L30-7 L30-8 M S.E.M. Fischer 344 F30-1 F30-4 M S.E.M.
82 313 205 309 173 216.40 43.58 383 286 334.50 48.50
and Fischer 344 rats in the present study. A greater number of Lewis rats acquired lever pressing than the Fischer 344 rats. In addition, Lewis rats pressed the levers at higher rates and earned more food pellets than the Fischer 344 rats. Although strain differences were observed under the present conditions, they were in the opposite direction of what was hypothesized based on the findings in a delay-discounting paradigm. Lewis rats have steeper discounting functions (make more immediate, small reinforcer choices) than Fischer 344 rats. One possible explanation for such a finding is that Lewis rats are more sensitive to the effects of delaying reinforcement. It is noted, however, that other factors, e.g., other differences in rate of learning such as associative factors, cannot be ruled out at this point. Interestingly, Lewis rats have been shown to acquire an operant response faster than Fischer 344 rats when using non-delayed (immediate) food as a reinforcer on a fixed-ratio schedule of reinforcement (Martin et al., 2003) and in an autoshaping procedure (Kearns et al., 2006). Lewis rats also have been observed to self-administer alcohol, opiates, and cocaine more readily than Fischer 344 rats when their delivery was immediate (George and Goldberg, 1988; Martin et al., 2003). Thus, it seems likely that Lewis rats more readily acquire operant responding compared to Fischer 344 rats regardless of the nature of the reinforcer or delay to presentation. Such a strain difference is noteworthy and the limitations of this difference are worthy of continued investigation. One consideration when studying different strains of rats is differential genetic and neurochemical features. It is quite possible that some of the neurochemical differences between Lewis and Fischer 344 rats may underlie some of the behavioral differences that are apparent in delay-discounting and response acquisition tasks. For instance, Lewis rats have been shown to have decreased levels of 5-HT and DA in various brain regions compared to Fischer 344 rats (cf. Flores et al., 1998; Lindley et al., 1999; Selim and Bradberry, 1996; Sziraki et al., 2001). However, numerous inconsistencies have been reported and there are many unexplored brain regions and neurotransmitter subtypes. How neurochemical differences may affect behavior, particu-
K.G. Anderson, M. Elcoro / Behavioural Processes 74 (2007) 311–318
315
Fig. 1. Cumulative responses on operative (solid line) and inoperative (dashed line) levers by individual Lewis rats across the session.
larly behavior influenced by temporal events like those seen in response acquisition with delayed reinforcement or delaydiscounting tasks, is the subject of much research (e.g., Cardinal et al., 2001; Mobini et al., 2000). Previous work has shown
that Lewis rats had shorter delays for points of indifference between an immediate presentation of a single food pellet and a delayed presentation of three food pellets than Fischer 344 rats (Anderson and Woolverton, 2005). Differential temporal sensi-
316
K.G. Anderson, M. Elcoro / Behavioural Processes 74 (2007) 311–318
Fig. 2. Cumulative responses on operative (solid line) and inoperative (dashed line) levers by individual Fischer 344 rats across the session.
tivity may be one reason for the outcome in that study, but other factors, e.g., amount sensitivity, associative factors, cannot be immediately eliminated. There is some evidence, however, to support the involvement of DA and 5-HT systems in control by temporal variables (e.g., Body et al., 2002; Graham et al., 1994; Kheramin et al., 2004; Matell et al., 2006; Mobini et al., 2000;
Morrissey et al., 1993; Wogar et al., 1992), but clearly more research will need to be conducted. Differences in 5-HT and DA systems in the Lewis and Fischer 344 rats may have contributed to the observed strain differences in lever-press acquisition with a delayed reinforcer. However, it is also worth mentioning that there were differences between
K.G. Anderson, M. Elcoro / Behavioural Processes 74 (2007) 311–318
the two groups with respect to age (approximately 2 weeks) and weight (approximately 71 g). It is interesting that the 11-weekold Lewis rats acquired the response in greater numbers than the 13-week-old Fischer 344 rats. It is unclear how, or if, this age discrepancy affected conditioning. Future work could be aimed at identifying developmental variables and/or neurochemistry in sensitivity to temporal parameters and response acquisition. It seems unlikely, though, that the relatively small age difference in the present study affected the outcome. Likewise, the weight differences seem unlikely to have affected the results. Both groups were food restricted approximately 36 h before the start of the 8-h acquisition test. Thus, both groups were likely to be similarly motivated by the food reinforcer. Also, since the Lewis rats weighed less than the Fischer 344 rats, body weight was commensurate with age. This weight differential, however, may have impacted satiation during feeder training as some of the Lewis rats did not consume all of the pellets delivered. This outcome likely did not affect overall feeder training, however, since the Lewis rats acquired subsequent lever pressing faster than the Fischer 344 rats. An additional issue concerns the use of a non-resetting delay. Relative to a resetting delay, a non-resetting delay may allow for closer temporal contiguity between the response and the food delivery (Wilkenfield et al., 1992). That is, lever presses could occur after the 20-s delay was initiated and thus, be closer to reinforcer presentation. This may have resulted in adventitious reinforcement of lever pressing or shorter obtained delays to food presentation. Use of a resetting delay would eliminate such issues; however, if a resetting delay had been employed, response rates likely would have been lower than the ones obtained. Given that Fischer 344 rats produced significantly fewer responses than the Lewis rats, and that only two out of eight Fischer 344 rats acquired lever pressing, the use of a non-resetting delay proved to be more advantageous relative to the use of a resetting delay. Future work, however, may address effects of a resetting delay and different delay values. A final point concerns lever pressing on the inoperative lever for two of the seven rats that acquired the response. Because the delay was non-resetting, it is possible that lever presses on the inoperative lever were followed by food presentation. Thus, adventitious reinforcement of pressing on the inoperative lever was possible and may have contributed to the establishment of undifferentiated responding in the two rats. Such a result has been previously reported (see LeSage et al., 1996; Sutphin et al., 1998). In the present study, however, there were statistically significant differences between operative- and inoperative-lever responding within each of the two groups indicating that the reinforcement contingency differentially affected acquisition of lever pressing. In summary, strain differences between Lewis and Fischer 344 rats in the acquisition of lever pressing with a 20-s nonresetting delay were observed with greater acquisition seen in the Lewis rats. The differences between the groups may be attributed to differences in 5-HT and/or DA systems, as such systems have been implicated in temporal discrimination and delay discounting. A better understanding of how these neurochemical systems and particular brain regions are involved in
317
temporal control has much relevance for applied issues where delayed outcomes are likely, e.g., acquisition and maintenance of behavior, self-controlled versus impulsive choice. Acknowledgement The authors wish to acknowledge Tammy Wade-Galuska for writing the computer programs employed in this experiment. References Anderson, K.G., Woolverton, W.L., 2003. Effects of dose and infusion delay on cocaine self-administration choice in rhesus monkeys. Psychopharmacology 167, 424–430. Anderson, K.G., Woolverton, W.L., 2005. Effects of clomipramine on selfcontrol choice in Lewis and Fischer 344 rats. Pharmacol. Biochem. Behav. 80, 387–393. Beardsley, P.M., Balster, R.L., 1993. The effects of delay of reinforcement and dose on the self-administration of cocaine and procaine in rhesus monkeys. Drug Alcohol Depend. 34, 37–43. Body, S., Chiang, T.J., Mobini, S., Ho, M.Y., Bradshaw, C.M., Szabadi, E., 2002. Effect of 8-OH-DPAT on temporal discrimination following central 5-hydroxytryptamine depletion. Pharmacol. Biochem. Behav. 71, 787–793. Body, S., Kheramin, S., Ho, M.Y., Miranda Herrera, F., Bradshaw, C.M., Szabadi, E., 2004. Effects of fenfluramine on free-operant timing behaviour: evidence for involvement of 5-HT2A receptors. Psychopharmacology 176, 154–165. Buhusi, C.V., 2003. Dopaminergic mechanisms of interval timing and attention. In: Meek, W.H. (Ed.), Functional and Neural Mechanisms of Interval Timing. CRC, NY, pp. 317–338. Burnet, P.W., Mefford, I.N., Smith, C.C., Gold, P.W., Sternberg, E.M., 1996. Hippocampal 5-HT1A receptor binding site densities, 5-HT1A receptor messenger riboneucleic acid abundance and serotonin levels parallel the activity of the hypothalamo-pituitary-adrenal axis in rats. Behav. Brain Res. 73, 365–368. Byrne, T., LeSage, M.G., Poling, A., 1997. Effects of chlorpromazine on rats’ acquisition of lever-press responding with immediate and delayed reinforcement. Pharmacol. Biochem. Behav. 58, 31–35. Cardinal, R.N., Pennicott, D.R., Sugathapala, C.L., Robbins, T.W., Everitt, B.J., 2001. Impulsive choice induced by lesions of the nucleus accumbens core. Science 292, 2499–2501. Chung, S.H., Herrnstein, R.J., 1967. Choice and delay of reinforcement. J. Exp. Anal. Behav. 10, 67–74. Critchfield, T.S., Lattal, K.A., 1993. Acquisition of a spatially defined operant with delayed reinforcement. J. Exp. Anal. Behav. 59, 373–387. Evenden, J.L., Ryan, C.N., 1996. The pharmacology of impulsive behavior in rats: the effects of drugs on response choice with varying delays of reinforcement. Psychopharmacology 128, 161–170. Fleshler, M., Hoffman, H.S., 1962. A progression for generating variable–interval schedules. J. Exp. Anal. Behav. 5, 529–530. Flores, G., Wood, G.K., Barbeau, D., Quirion, R., Srivastava, L.K., 1998. Lewis and Fischer rats: a comparison of dopamine transporter and receptor levels. Brain Res. 814, 34–40. George, F.R., Goldberg, S.R., 1988. Genetic differences in responses to cocaine. In: Clouet, D., Ashgar, K., Brown, R. (Eds.), Mechanisms of Cocaine Abuse and Toxicity, NIDA Research Monograph, Vol. 88, U.S. Government Printing Office, pp. 239–249. Graham, S., Ho, M.Y., Bradshaw, C.M., Szabadi, E., 1994. Facilitated acquisition of a temporal discrimination following destruction of the ascending 5-hydroxytryptaminergic pathways. Psychopharmacology 116, 373–378. Kearns, D.N., Gomez-Serrano, M.A., Weiss, S.J., Riley, A.L., 2006. A comparison of Lewis and Fischer rat strains on autoshaping (sign-tracking), discrimination reversal learning and negative auto-maintenance. Behav. Brain Res. 169, 193–200. Kheramin, S., Body, S., Ho, M.Y., Velazquez-Martinex, D.N., Bradshaw, C.M., Szabadi, E., Deakin, J.F., Anderson, I.M., 2004. Effects of orbital prefrontal
318
K.G. Anderson, M. Elcoro / Behavioural Processes 74 (2007) 311–318
cortex dopamine depletion on inter-temporal choice: a quantitative analysis. Psychopharmacology 175, 206–214. Lattal, K.A., 1984. Signal functions in delayed reinforcement. J. Exp. Anal. Behav. 42, 239–253. Lattal, K.A., Gleeson, S., 1990. Response acquisition with delayed reinforcement. J. Exp. Psych. Animal Behav. Proc. 16, 27–39. Lattal, K.A., Metzger, B., 1994. Response acquisition by Siamese fighting fish (Betta splendens) with delayed visual reinforcement. J. Exp. Anal. Behav. 42, 239–253. LeSage, M.G., Byrne, T., Poling, A., 1996. Effects of d-amphetamine on response acquisition with immediate and delayed reinforcement. J. Exp. Anal. Behav. 66, 349–367. Lindley, S.E., Bengoechea, T.G., Wong, D.L., Schatzberg, A.F., 1999. Strain differences in mesotelencephalic dopaminergic neuronal regulation between Fischer 344 and Lewis rats. Brain Res. 832, 152–158. Martin, S., Lyupina, Y., Crespo, J.A., Gonzalez, B., Gracia-Lecumberri, C., Ambrosio, E., 2003. Genetic differences in NMD A and Dl receptor levels, and operant responding for food and morphine in Lewis and Fischer 344 rats. Brain Res. 973, 205–213. Matell, M.S., Bateson, M., Meek, W.H., 2006. Single-trials analyses demonstrate that increases in clock speed contribute to the methamphetamine-induced horizontal shifts in peak-interval timing functions. Psychopharmacology 188, 201–212. Mazur, J.E., 1987. An adjusting procedure for studying delayed reinforcement. In: Commons, M.L., Mazur, J.E., Nevin, J.A., Rachlin, H. (Eds.), Quantitative Analyses of Behavior: The Effect of Delay and Intervening Events on Reinforcement Value, vol. 5. Erlbaum, Hillsdale, NL, pp. 55–73.
Mobini, S., Chiang, T.J., Al-Ruwaitea, A.S., Ho, M.Y., Bradshaw, C.M., Szabadi, E., 2000. Effect of central 5-hydroxytryptamine depletion on inter-temporal choice: a quantitative analysis. Psychopharmacology 149, 313–318. Morrissey, G., Wogar, M.A., Bradshaw, C.M., Szabadi, E., 1993. Effect of lesions of the ascending 5-hydroxytryptaminergic pathways on timing behaviour investigated with an interval bisection task. Psychopharmacology 112, 80–85. Myerson, J., Green, L., 1995. Discounting of delayed rewards: models of individual choice. J. Exp. Anal. Behav. 64, 263–276. Selim, M., Bradberry, C.W., 1996. Effect of ethanol on extracellular 5-HT and glutamate in the nucleus accumbens and prefrontal cortex: comparison between the Lewis and Fischer 344 rat strains. Brain Res. 716, 157–164. Sutphin, G., Byrne, T., Poling, A., 1998. Response acquisition with delayed reinforcement: a comparison of two-lever procedures. J. Exp. Anal. Behav. 69, 17–28. Sziraki, I., Lipovac, M.N., Hashim, A., Sershen, H., Allen, D., Cooper, T., 2001. Differences in nicotine-induced dopamine release and nicotine pharmacokinetics between Lewis and Fischer 344 rats. Neurochem. Res. 26, 609–617. van Haaren, F., 1992. Response acquisition with fixed and variable resetting delays of reinforcement in male and female Wistar rats. Physiol. Behav. 52, 769–772. Wilkenfield, J., Nickel, M., Blakely, E., Poling, A., 1992. Acquisition of leverpress responding in rats with delayed reinforcement: a comparison of three procedures. J. Exp. Anal. Behav. 58, 431–433. Wogar, M.A., Bradshaw, C.M., Szabadi, E., 1992. Impaired acquisition of temporal differentiation performance following lesions of the ascending 5-hydroxytryptaminergic pathways. Psychopharmacology 107, 373–378.
Response acquisition with delayed reinforcement in Lewis and Fischer 344 rats Karen G. Anderson ∗ , Mirari Elcoro West Virginia University, United States Received 15 May 2006; received in revised form 2 November 2006; accepted 28 November 2006
Abstract Previous work has shown neurochemical and behavioral differences between Lewis rats and Fischer 344 rats. Some of this work suggests that there might be differential sensitivity to delayed reinforcement between the two strains. To further explore this possibility, Lewis (n = 8) and Fischer 344 (n = 8) rats were exposed to a response–acquisition task with a non-resetting 20 s delay to reinforcement. A tandem fixed-ratio 1, fixed-time 20 s schedule of reinforcement was programmed for one of two levers; presses on the alternate lever had no programmed consequences. A greater number of Lewis rats (5/8) acquired lever pressing compared to the Fischer 344 rats (2/8). Future work with these strains may lead to a better understanding of the genetic and/or neurochemical factors involved in temporal control of behavior. © 2006 Elsevier B.V. All rights reserved. Keywords: Acquisition; Delay of reinforcement; Fischer 344; Lever press; Lewis; Rat
1. Introduction It is well established that the rate or frequency of an existing behavior may be weakened as a result of delaying presentation of the reinforcer that maintains it (cf. Anderson and Woolverton, 2003; Beardsley and Balster, 1993; Chung and Herrnstein, 1967; Lattal, 1984; Mazur, 1987; Myerson and Green, 1995). However, it also has been demonstrated that organisms can learn a new behavior when the consequence, i.e., reinforcement, is not immediate or signaled, and the response is not explicitly shaped (Lattal and Gleeson, 1990). Response acquisition with delayed reinforcement has been established with various species and response topographies (e.g., Critchfield and Lattal, 1993; Lattal and Gleeson, 1990; Lattal and Metzger, 1994; van Haaren, 1992). What is not well known, is how effects of delayed consequences on the acquisition and/or maintenance of behavior may differ as a function of individual differences, e.g., age, sex, species, behavioral history, genetics, and neurochemistry. One line of research that employs delayed reinforcement, with previously established behavior, is the area of self-
∗
Corresponding author at: West Virginia University, Department of Psychology, P.O. Box 6040, Morgantown, WV 26506, United States. E-mail address: [email protected] (K.G. Anderson). 0376-6357/$ – see front matter © 2006 Elsevier B.V. All rights reserved. doi:10.1016/j.beproc.2006.11.006
control/impulsive choice. In a typical paradigm (cf. Evenden and Ryan, 1996; Mazur, 1987), a choice is presented between a larger, more delayed reinforcer and a smaller, more immediate reinforcer. An individual is said to behave “impulsively” if the smaller, sooner reinforcer is chosen over the larger, later reinforcer and the outcome is a reduction in overall reinforcement. One suggested reason for the impulsive choice often seen in these paradigms is that the larger reinforcer loses some of its value, i.e., becomes less reinforcing, as the delay to its presentation is increased. Eventually, the reinforcing effectiveness of the larger, delayed reinforcer will fall below that of the smaller, immediate reinforcer and this is indicated by a switch in preference, such that the “impulsive” option is selected over the “self-controlled” option. This decrease in the larger reinforcer’s reinforcing effectiveness (or value) as a result of its delay to presentation is called delay (or temporal) discounting (Mazur, 1987) and individual differences in rates of delay discounting have been observed in human and non-human subjects. In particular, two strains of rats have been shown to differ in their choices of delayed reinforcers. Lewis and Fischer 344 rats were shown to differ in the number of large reinforcer (three food pellet) choices they emitted when a single, immediate food pellet was the alternative (Anderson and Woolverton, 2005). In particular, the Lewis rats emitted more small reinforcer (impulsive) choices than the Fischer 344 rats
312
K.G. Anderson, M. Elcoro / Behavioural Processes 74 (2007) 311–318
and had higher rates of delay discounting as indicated by steeper slopes in the delay-discounting function and by lower indifference points (the delay value where the larger reinforcer is equal to that of the smaller, immediate reinforcer). One interpretation of this outcome is that the Lewis rats’ lever pressing was more sensitive to the effects of delaying reinforcement, and this may be a result of neurochemical and/or genetic differences between the strains. Lewis rats have been shown to have differences (decreased activity) in dopaminergic (DA) and serotonergic (5-HT) systems in various brain regions compared to Fischer 344 rats (Burnet et al., 1996; Flores et al., 1998; Lindley et al., 1999; Martin et al., 2003; Selim and Bradberry, 1996). In particular, Lewis rats have fewer 5-HTIA binding sites and less 5-HT in some brain regions (Burnet et al., 1996; Selim and Bradberry, 1996) and fewer DA transporters and lower levels of DA in some areas of the brain (Flores et al., 1998; Lindley et al., 1999) compared to Fischer 344 rats. It is not known, however, what roles, if any, such differences play in acquisition or maintenance of behavior with delayed reinforcement. There is some evidence that 5-HT and/or DA in particular brain regions may influence effects of temporal variables (cf. Buhusi, 2003; Cardinal et al., 2001; Kheramin et al., 2004; Mobini et al., 2000; Wogar et al., 1992), but the findings have been mixed. In general, however, increased DA and 5-HT have each individually been linked to overestimations of the passage of time (e.g., Body et al., 2004; Matell et al., 2006) and lesions of these pathways have been shown to affect sensitivity to temporal events (e.g., Kheramin et al., 2004; Mobini et al., 2000; Morrissey et al., 1993; Wogar et al., 1992). A greater understanding of the variables that affect sensitivity to immediate versus delayed outcomes has much clinical and applied relevance, e.g., attention-deficit/hyperactivity disorder (ADHD), drug abuse, gambling. Studies of effects of drugs on response acquisition with immediate versus delayed reinforcement may also yield data with relevance for behavioral and/or neurochemical underpinnings for performance in such tasks (cf. Byrne et al., 1997; LeSage et al., 1996). To investigate strain differences in effects of delayed consequences on the acquisition of a response, Lewis and Fischer 344 rats with no experimental history were allowed to press two levers in a standard operant chamber. One lever was associated with the presentation of a food pellet after a non-resetting 20-s delay following the lever press. No food was delivered for pressing the alternate lever. Although other procedures may have been employed, the non-resetting delay was chosen to increase the likelihood that lever pressing would be acquired since it is possible that the obtained delay values may be shorter than the programmed value (cf. Wilkenfield et al., 1992). The delay duration was selected because it previously resulted in differential responding between the two strains when the choice was between one immediate food pellet and three food pellets following a 20-s delay (Anderson and Woolverton, 2005). It was hypothesized, based on the findings of Anderson and Woolverton (2005), that the Lewis rats would take longer than the Fischer 344 rats to acquire the response on the lever associated with delayed food presentation.
2. Methods 2.1. Subjects Eight experimentally naive, male Lewis (approximately 11 weeks old) and eight experimentally naive, male Fischer 344 rats (approximately 13 weeks old) (Harlan Sprague Dawley, IN) served as subjects. Each subject was housed individually and water was always available in each home cage. Illumination was maintained on a reversed 12:12 h dark/light cycle and humidity and temperature were kept constant. Subjects were fed 12–15 g of rat chow (Harlan, WI) daily, at approximately the same time of day unless otherwise specified. Animals were maintained in accordance with National Institutes of Health guidelines for Care and Use of Laboratory Animals. All experimental procedures were approved by the West Virginia University Animal Use and Care Committee and were conducted in facilities that were approved by the Association for Assessment and Accreditation of Laboratory Animal Care (AAALAC). 2.2. Apparatus Experimental sessions were conducted in eight standard operant chambers for rats, each enclosed in a melamine sound-attenuating cubicle (Med Associates, VT). Each chamber contained a working area of 30.5 cm by 24.1 cm by 21.0 cm, a grid floor, and a 45 mg pellet dispenser with a pellet receptacle centered between two standard response levers that were elevated 8 cm from the grid floor. Levers were positioned 11.5 cm apart from each other, were 4.8 cm wide, protruded 1.9 cm into the chamber and required at least 0.25 N for a response to be recorded. Two 28 V stimulus lights 2.5 cm in diameter were placed approximately 7 cm above each lever. Each chamber had a 28 V houselight on the wall opposite to the working wall, and a ventilation fan to circulate air and to mask extraneous noise. Equipment was interfaced to a computer and routines were programmed and conducted with MedPC-IV (Med Associates, VT). 2.3. Procedure 2.3.1. Feeder training All experimental sessions were conducted during the dark cycle, at approximately the same time. Before the start of the session, both levers were removed from each chamber and replaced with filler panels. Subjects were restricted from food for approximately 22 h before feeder training started. At the beginning of the session, each subject was placed in a darkened chamber with the ventilation fan on. Immediately after, the houselight was illuminated and 45 mg sucrose pellets (BioServ, NJ) were delivered according to a variable-time (VT) 60-s schedule (Fleshler and Hoffman, 1962). Pellet delivery was signaled by a 0.5 s flash of the houselight. Sessions ended after 60 pellets were delivered and were conducted on adjacent days for each group. Post-session feeding for all animals consisted of 10–12 g of rat chow and no additional food was available until the lever-press acquisition session.
K.G. Anderson, M. Elcoro / Behavioural Processes 74 (2007) 311–318
2.3.2. Lever-press acquisition This 8-h session was conducted approximately 48 h after feeder training (adjacent days for each group of rats). Sessions started at approximately 08:30 for both groups. At that time, subjects had been restricted from food for approximately 36 h. At the start of the session, each subject was placed in a darkened chamber with an activated ventilation fan. Responses on both levers were recorded but had no other programmed consequences. After 5 min, the houselight and both lever lights were illuminated. For half the subjects of each group, a left lever press initiated a tandem fixed-ratio (FR) 1, fixed-time (FT) 20-s schedule of reinforcement. The 20-s delay between the lever press and the pellet delivery was unsignaled and non-resetting. Lever presses during the delay were recorded but had no other programmed outcome. Right lever presses had no programmed consequences. These conditions were counterbalanced for the remaining half of the subjects in each group. The lever associated with the tandem FR 1 FT 20 s schedule of reinforcement was referred to as the operative lever, and the lever associated with no programmed consequences was referred to as the inoperative lever. Responses on each lever and the total number of pellets obtained were recorded during the session. 2.3.3. Data analysis Statistical analyses of the data (t-tests) were used to determine significant differences between groups in terms of body weight, response rate on each lever, and time of lever-press acquisition. Response rates were calculated by dividing the total number of responses on each lever by the total time for the session (responses per minute; rpm). Lever-press acquisition was characterized by the presence of a positively accelerated increase in cumulative responses as evidenced upon visual inspection of the data presented in a cumulative record. For quantitative analyses, lever-press acquisition was defined by the time when 25% of the total number of responses occurred. To do this, each session was divided into 1-min bins. Then the bin in which 25% of the total number of responses occurred was identified. As a measure of variability, standard errors around the mean (S.E.M.) were calculated for all the data presented. 3. Results At the start of the experiment, Lewis rats weighed an average of 240 g (S.E.M. = 6.87). Fischer 344 rats weighed an average of 311 g (S.E.M. = 2.93). This difference in weight was statistically significant [t(7) = 7.80, p < 0.001 (two-tailed)]. During feeder training, all subjects were observed to have consumed at least 60% of the pellets delivered. (Most rats consumed 100% of the pellets, but it was noted that some of the Lewis rats left a few pellets with one Lewis rat leaving 40% of the pellets.) During the acquisition session, subjects were exposed to the levers in the dark during a 5-min blackout. Lever-pressing rates during this time were low (range: 0.0–2.2 rpm). On average, the Lewis rats (M = 1.08 rpm) responded more during the 5-min blackout than the Fischer 344 rats (M = 0.3 rpm) and this difference was statistically significant [t(7) = 2.66, p = 0.032
313
(two-tailed)]. A t-test revealed no significant differences in lever pressing during the blackout and later acquisition. Table 1 shows the number of responses on each lever for each subject during lever-press acquisition. Overall, the eight Lewis rats emitted higher rates of responding (M = 0.31 rpm, S.E.M. = 0.08) than the eight Fischer 344 rats (M = 0.07 rpm, S.E.M. = 0.04) on the operative lever. This difference was statistically significant [t(7) = − 4.13, p = 0.004 (two-tailed)]. Also, Lewis rats responded at higher rates (M = 0.14 rpm, S.E.M. = 0.03) than Fischer 344 rats (M = 0.02 rpm, S.E.M. = 0.01) on the inoperative lever. This difference was also statistically significant [t(7) = − 3.51, p = 0.01 (two-tailed)]. For the rats that acquired lever pressing in both groups, the rate of lever pressing on the operative lever was greater than the rate on the inoperative lever. The five Lewis rats averaged 0.45 rpm (S.E.M. = 0.57) on the operative lever and 0.18 rpm (S.E.M. = 0.05) on the inoperative lever. The two Fischer 344 rats averaged 0.24 rpm (S.E.M. = 0.02) on the operative lever and 0.05 rpm (S.E.M. = 0.02) on the inoperative lever. These differences between operative-lever responses and inoperativelever responses in each group were statistically significant for the Fischer 344 rats [t(1) = 39, p = 0.02 (two-tailed)] and for the Lewis rats [t(4) = 2.75, p = 0.05 (two-tailed)]. It should be noted, however, that the statistical power in these analyses is reduced due to the small number of Fischer 344 rats that acquired the response. When comparing all eight rats within a strain, including those that acquired lever pressing and those that did not acquire lever pressing, more responding occurred on the operative lever than the inoperative lever, but the differences were not statistically significant in either group (Lewis: operative Table 1 Number of lever presses emitted by Lewis and Fischer 344 rats on the operative and inoperative levers during the 8 h lever-press acquisition session
Lewis L30-1 L30-2 L30-3 L30-4 L30-5 L30-6 L30-7 L30-8 M S.E.M. Fischer 344 F30-1 F30-2 F30-3 F30-4 F30-5 F30-6 F30-7 F30-8 M S.E.M.
Operative
Inoperative
263 130 252 32 39 27 177 262
27 124 77 36 35 32 150 51
147.75 37.36
66.50 16.53
124 6 2 106 0 9 18 21
31 1 4 13 2 2 23 15
35.75 17.56
11.38 3.95
314
K.G. Anderson, M. Elcoro / Behavioural Processes 74 (2007) 311–318
Table 2 Total number of pellets obtained per subject in each group during the lever-press acquisition session
Table 3 Time in the session (min) when 25% of the total number of lever presses on the operative lever were emitted for Lewis and Fischer 344 rats
Total pellets obtained
25% of the total responses (min)
Lewis L30-1 L30-2 L30-3 L30-4 L30-5 L30-6 L30-7 L30-8 M S.E.M. Fischer 344 F30-1 F30-2 F30-3 F30-4 F30-5 F30-6 F30-7 F30-8 M S.E.M.
163 91 127 15 27 16 120 175 91.75 23.09 63 4 1 70 0 5 11 13 20.88 10.10
M = 0.31 rpm, inoperative M = 0.14 rpm, p > 0.05; Fischer 344 operative M = 0.07 rpm, inoperative M = 0.02 rpm, p > 0.05). The total number of pellets obtained per subject is presented in Table 2. Lewis rats obtained an average of 91.75 pellets and Fischer 344 rats obtained an average of 20.88 pellets throughout the session. This difference was statistically significant [t(7) = −2.83, p = 0.03 (two-tailed)]. Figs. 1 and 2 depict cumulative responses for operative (solid line) and inoperative (dashed line) levers for Lewis and Fischer 344 rats, respectively. These figures show that five Lewis rats and two Fischer 344 rats had a positively accelerated pattern of responding once responding was initiated. Note also that two of the Lewis rats (L30-2 and L30-7) acquired lever pressing on the inoperative lever at approximately the same rate as the operative lever. To further characterize lever-press acquisition on the operative lever, Table 3 shows the 1-min bin in which each rat that acquired the response emitted 25% of the total number of responses. Five of eight Lewis rats emitted a quarter of the lever presses at an average of 216.40 min (S.E.M. = 43.58) into the session. Out of eight Fischer 344 rats, the two that acquired lever pressing emitted a quarter of the lever presses at an average of 334.50 min (S.E.M. = 48.50) into the session. Even though, on average, Lewis rats acquired the lever pressing earlier in the session than Fischer 344 rats, this difference was not statistically significant [t(7) = 1.53, p = 0.49 (two-tailed)]. 4. Discussion Strain differences in response acquisition with a 20-s (nonresetting) delay to reinforcement were observed between Lewis
Lewis L30-1 L30-2 L30-3 L30-7 L30-8 M S.E.M. Fischer 344 F30-1 F30-4 M S.E.M.
82 313 205 309 173 216.40 43.58 383 286 334.50 48.50
and Fischer 344 rats in the present study. A greater number of Lewis rats acquired lever pressing than the Fischer 344 rats. In addition, Lewis rats pressed the levers at higher rates and earned more food pellets than the Fischer 344 rats. Although strain differences were observed under the present conditions, they were in the opposite direction of what was hypothesized based on the findings in a delay-discounting paradigm. Lewis rats have steeper discounting functions (make more immediate, small reinforcer choices) than Fischer 344 rats. One possible explanation for such a finding is that Lewis rats are more sensitive to the effects of delaying reinforcement. It is noted, however, that other factors, e.g., other differences in rate of learning such as associative factors, cannot be ruled out at this point. Interestingly, Lewis rats have been shown to acquire an operant response faster than Fischer 344 rats when using non-delayed (immediate) food as a reinforcer on a fixed-ratio schedule of reinforcement (Martin et al., 2003) and in an autoshaping procedure (Kearns et al., 2006). Lewis rats also have been observed to self-administer alcohol, opiates, and cocaine more readily than Fischer 344 rats when their delivery was immediate (George and Goldberg, 1988; Martin et al., 2003). Thus, it seems likely that Lewis rats more readily acquire operant responding compared to Fischer 344 rats regardless of the nature of the reinforcer or delay to presentation. Such a strain difference is noteworthy and the limitations of this difference are worthy of continued investigation. One consideration when studying different strains of rats is differential genetic and neurochemical features. It is quite possible that some of the neurochemical differences between Lewis and Fischer 344 rats may underlie some of the behavioral differences that are apparent in delay-discounting and response acquisition tasks. For instance, Lewis rats have been shown to have decreased levels of 5-HT and DA in various brain regions compared to Fischer 344 rats (cf. Flores et al., 1998; Lindley et al., 1999; Selim and Bradberry, 1996; Sziraki et al., 2001). However, numerous inconsistencies have been reported and there are many unexplored brain regions and neurotransmitter subtypes. How neurochemical differences may affect behavior, particu-
K.G. Anderson, M. Elcoro / Behavioural Processes 74 (2007) 311–318
315
Fig. 1. Cumulative responses on operative (solid line) and inoperative (dashed line) levers by individual Lewis rats across the session.
larly behavior influenced by temporal events like those seen in response acquisition with delayed reinforcement or delaydiscounting tasks, is the subject of much research (e.g., Cardinal et al., 2001; Mobini et al., 2000). Previous work has shown
that Lewis rats had shorter delays for points of indifference between an immediate presentation of a single food pellet and a delayed presentation of three food pellets than Fischer 344 rats (Anderson and Woolverton, 2005). Differential temporal sensi-
316
K.G. Anderson, M. Elcoro / Behavioural Processes 74 (2007) 311–318
Fig. 2. Cumulative responses on operative (solid line) and inoperative (dashed line) levers by individual Fischer 344 rats across the session.
tivity may be one reason for the outcome in that study, but other factors, e.g., amount sensitivity, associative factors, cannot be immediately eliminated. There is some evidence, however, to support the involvement of DA and 5-HT systems in control by temporal variables (e.g., Body et al., 2002; Graham et al., 1994; Kheramin et al., 2004; Matell et al., 2006; Mobini et al., 2000;
Morrissey et al., 1993; Wogar et al., 1992), but clearly more research will need to be conducted. Differences in 5-HT and DA systems in the Lewis and Fischer 344 rats may have contributed to the observed strain differences in lever-press acquisition with a delayed reinforcer. However, it is also worth mentioning that there were differences between
K.G. Anderson, M. Elcoro / Behavioural Processes 74 (2007) 311–318
the two groups with respect to age (approximately 2 weeks) and weight (approximately 71 g). It is interesting that the 11-weekold Lewis rats acquired the response in greater numbers than the 13-week-old Fischer 344 rats. It is unclear how, or if, this age discrepancy affected conditioning. Future work could be aimed at identifying developmental variables and/or neurochemistry in sensitivity to temporal parameters and response acquisition. It seems unlikely, though, that the relatively small age difference in the present study affected the outcome. Likewise, the weight differences seem unlikely to have affected the results. Both groups were food restricted approximately 36 h before the start of the 8-h acquisition test. Thus, both groups were likely to be similarly motivated by the food reinforcer. Also, since the Lewis rats weighed less than the Fischer 344 rats, body weight was commensurate with age. This weight differential, however, may have impacted satiation during feeder training as some of the Lewis rats did not consume all of the pellets delivered. This outcome likely did not affect overall feeder training, however, since the Lewis rats acquired subsequent lever pressing faster than the Fischer 344 rats. An additional issue concerns the use of a non-resetting delay. Relative to a resetting delay, a non-resetting delay may allow for closer temporal contiguity between the response and the food delivery (Wilkenfield et al., 1992). That is, lever presses could occur after the 20-s delay was initiated and thus, be closer to reinforcer presentation. This may have resulted in adventitious reinforcement of lever pressing or shorter obtained delays to food presentation. Use of a resetting delay would eliminate such issues; however, if a resetting delay had been employed, response rates likely would have been lower than the ones obtained. Given that Fischer 344 rats produced significantly fewer responses than the Lewis rats, and that only two out of eight Fischer 344 rats acquired lever pressing, the use of a non-resetting delay proved to be more advantageous relative to the use of a resetting delay. Future work, however, may address effects of a resetting delay and different delay values. A final point concerns lever pressing on the inoperative lever for two of the seven rats that acquired the response. Because the delay was non-resetting, it is possible that lever presses on the inoperative lever were followed by food presentation. Thus, adventitious reinforcement of pressing on the inoperative lever was possible and may have contributed to the establishment of undifferentiated responding in the two rats. Such a result has been previously reported (see LeSage et al., 1996; Sutphin et al., 1998). In the present study, however, there were statistically significant differences between operative- and inoperative-lever responding within each of the two groups indicating that the reinforcement contingency differentially affected acquisition of lever pressing. In summary, strain differences between Lewis and Fischer 344 rats in the acquisition of lever pressing with a 20-s nonresetting delay were observed with greater acquisition seen in the Lewis rats. The differences between the groups may be attributed to differences in 5-HT and/or DA systems, as such systems have been implicated in temporal discrimination and delay discounting. A better understanding of how these neurochemical systems and particular brain regions are involved in
317
temporal control has much relevance for applied issues where delayed outcomes are likely, e.g., acquisition and maintenance of behavior, self-controlled versus impulsive choice. Acknowledgement The authors wish to acknowledge Tammy Wade-Galuska for writing the computer programs employed in this experiment. References Anderson, K.G., Woolverton, W.L., 2003. Effects of dose and infusion delay on cocaine self-administration choice in rhesus monkeys. Psychopharmacology 167, 424–430. Anderson, K.G., Woolverton, W.L., 2005. Effects of clomipramine on selfcontrol choice in Lewis and Fischer 344 rats. Pharmacol. Biochem. Behav. 80, 387–393. Beardsley, P.M., Balster, R.L., 1993. The effects of delay of reinforcement and dose on the self-administration of cocaine and procaine in rhesus monkeys. Drug Alcohol Depend. 34, 37–43. Body, S., Chiang, T.J., Mobini, S., Ho, M.Y., Bradshaw, C.M., Szabadi, E., 2002. Effect of 8-OH-DPAT on temporal discrimination following central 5-hydroxytryptamine depletion. Pharmacol. Biochem. Behav. 71, 787–793. Body, S., Kheramin, S., Ho, M.Y., Miranda Herrera, F., Bradshaw, C.M., Szabadi, E., 2004. Effects of fenfluramine on free-operant timing behaviour: evidence for involvement of 5-HT2A receptors. Psychopharmacology 176, 154–165. Buhusi, C.V., 2003. Dopaminergic mechanisms of interval timing and attention. In: Meek, W.H. (Ed.), Functional and Neural Mechanisms of Interval Timing. CRC, NY, pp. 317–338. Burnet, P.W., Mefford, I.N., Smith, C.C., Gold, P.W., Sternberg, E.M., 1996. Hippocampal 5-HT1A receptor binding site densities, 5-HT1A receptor messenger riboneucleic acid abundance and serotonin levels parallel the activity of the hypothalamo-pituitary-adrenal axis in rats. Behav. Brain Res. 73, 365–368. Byrne, T., LeSage, M.G., Poling, A., 1997. Effects of chlorpromazine on rats’ acquisition of lever-press responding with immediate and delayed reinforcement. Pharmacol. Biochem. Behav. 58, 31–35. Cardinal, R.N., Pennicott, D.R., Sugathapala, C.L., Robbins, T.W., Everitt, B.J., 2001. Impulsive choice induced by lesions of the nucleus accumbens core. Science 292, 2499–2501. Chung, S.H., Herrnstein, R.J., 1967. Choice and delay of reinforcement. J. Exp. Anal. Behav. 10, 67–74. Critchfield, T.S., Lattal, K.A., 1993. Acquisition of a spatially defined operant with delayed reinforcement. J. Exp. Anal. Behav. 59, 373–387. Evenden, J.L., Ryan, C.N., 1996. The pharmacology of impulsive behavior in rats: the effects of drugs on response choice with varying delays of reinforcement. Psychopharmacology 128, 161–170. Fleshler, M., Hoffman, H.S., 1962. A progression for generating variable–interval schedules. J. Exp. Anal. Behav. 5, 529–530. Flores, G., Wood, G.K., Barbeau, D., Quirion, R., Srivastava, L.K., 1998. Lewis and Fischer rats: a comparison of dopamine transporter and receptor levels. Brain Res. 814, 34–40. George, F.R., Goldberg, S.R., 1988. Genetic differences in responses to cocaine. In: Clouet, D., Ashgar, K., Brown, R. (Eds.), Mechanisms of Cocaine Abuse and Toxicity, NIDA Research Monograph, Vol. 88, U.S. Government Printing Office, pp. 239–249. Graham, S., Ho, M.Y., Bradshaw, C.M., Szabadi, E., 1994. Facilitated acquisition of a temporal discrimination following destruction of the ascending 5-hydroxytryptaminergic pathways. Psychopharmacology 116, 373–378. Kearns, D.N., Gomez-Serrano, M.A., Weiss, S.J., Riley, A.L., 2006. A comparison of Lewis and Fischer rat strains on autoshaping (sign-tracking), discrimination reversal learning and negative auto-maintenance. Behav. Brain Res. 169, 193–200. Kheramin, S., Body, S., Ho, M.Y., Velazquez-Martinex, D.N., Bradshaw, C.M., Szabadi, E., Deakin, J.F., Anderson, I.M., 2004. Effects of orbital prefrontal
318
K.G. Anderson, M. Elcoro / Behavioural Processes 74 (2007) 311–318
cortex dopamine depletion on inter-temporal choice: a quantitative analysis. Psychopharmacology 175, 206–214. Lattal, K.A., 1984. Signal functions in delayed reinforcement. J. Exp. Anal. Behav. 42, 239–253. Lattal, K.A., Gleeson, S., 1990. Response acquisition with delayed reinforcement. J. Exp. Psych. Animal Behav. Proc. 16, 27–39. Lattal, K.A., Metzger, B., 1994. Response acquisition by Siamese fighting fish (Betta splendens) with delayed visual reinforcement. J. Exp. Anal. Behav. 42, 239–253. LeSage, M.G., Byrne, T., Poling, A., 1996. Effects of d-amphetamine on response acquisition with immediate and delayed reinforcement. J. Exp. Anal. Behav. 66, 349–367. Lindley, S.E., Bengoechea, T.G., Wong, D.L., Schatzberg, A.F., 1999. Strain differences in mesotelencephalic dopaminergic neuronal regulation between Fischer 344 and Lewis rats. Brain Res. 832, 152–158. Martin, S., Lyupina, Y., Crespo, J.A., Gonzalez, B., Gracia-Lecumberri, C., Ambrosio, E., 2003. Genetic differences in NMD A and Dl receptor levels, and operant responding for food and morphine in Lewis and Fischer 344 rats. Brain Res. 973, 205–213. Matell, M.S., Bateson, M., Meek, W.H., 2006. Single-trials analyses demonstrate that increases in clock speed contribute to the methamphetamine-induced horizontal shifts in peak-interval timing functions. Psychopharmacology 188, 201–212. Mazur, J.E., 1987. An adjusting procedure for studying delayed reinforcement. In: Commons, M.L., Mazur, J.E., Nevin, J.A., Rachlin, H. (Eds.), Quantitative Analyses of Behavior: The Effect of Delay and Intervening Events on Reinforcement Value, vol. 5. Erlbaum, Hillsdale, NL, pp. 55–73.
Mobini, S., Chiang, T.J., Al-Ruwaitea, A.S., Ho, M.Y., Bradshaw, C.M., Szabadi, E., 2000. Effect of central 5-hydroxytryptamine depletion on inter-temporal choice: a quantitative analysis. Psychopharmacology 149, 313–318. Morrissey, G., Wogar, M.A., Bradshaw, C.M., Szabadi, E., 1993. Effect of lesions of the ascending 5-hydroxytryptaminergic pathways on timing behaviour investigated with an interval bisection task. Psychopharmacology 112, 80–85. Myerson, J., Green, L., 1995. Discounting of delayed rewards: models of individual choice. J. Exp. Anal. Behav. 64, 263–276. Selim, M., Bradberry, C.W., 1996. Effect of ethanol on extracellular 5-HT and glutamate in the nucleus accumbens and prefrontal cortex: comparison between the Lewis and Fischer 344 rat strains. Brain Res. 716, 157–164. Sutphin, G., Byrne, T., Poling, A., 1998. Response acquisition with delayed reinforcement: a comparison of two-lever procedures. J. Exp. Anal. Behav. 69, 17–28. Sziraki, I., Lipovac, M.N., Hashim, A., Sershen, H., Allen, D., Cooper, T., 2001. Differences in nicotine-induced dopamine release and nicotine pharmacokinetics between Lewis and Fischer 344 rats. Neurochem. Res. 26, 609–617. van Haaren, F., 1992. Response acquisition with fixed and variable resetting delays of reinforcement in male and female Wistar rats. Physiol. Behav. 52, 769–772. Wilkenfield, J., Nickel, M., Blakely, E., Poling, A., 1992. Acquisition of leverpress responding in rats with delayed reinforcement: a comparison of three procedures. J. Exp. Anal. Behav. 58, 431–433. Wogar, M.A., Bradshaw, C.M., Szabadi, E., 1992. Impaired acquisition of temporal differentiation performance following lesions of the ascending 5-hydroxytryptaminergic pathways. Psychopharmacology 107, 373–378.