- Email: [email protected]
Effect of a single free food presentation on extinction responding in a multiple schedule
Accepted Manuscript Title: Effect of a Single Free Food Presentation on Extinction Responding in a Multiple Schedule Author: Adam J. Reiss Matthew C. Bell PII: DOI: Reference:
S0376-6357(16)30155-3 http://dx.doi.org/doi:10.1016/j.beproc.2016.07.002 BEPROC 3273
To appear in:
Behavioural Processes
Received date: Revised date: Accepted date:
9-4-2016 5-7-2016 6-7-2016
Please cite this article as: Reiss, Adam J., Bell, Matthew C., Effect of a Single Free Food Presentation on Extinction Responding in a Multiple Schedule.Behavioural Processes http://dx.doi.org/10.1016/j.beproc.2016.07.002 This is a PDF file of an unedited manuscript that has been accepted for publication. As a service to our customers we are providing this early version of the manuscript. The manuscript will undergo copyediting, typesetting, and review of the resulting proof before it is published in its final form. Please note that during the production process errors may be discovered which could affect the content, and all legal disclaimers that apply to the journal pertain.
Running head: FREE FOOD AND EXTINCTION
Effect of a Single Free Food Presentation on Extinction Responding in a Multiple Schedule Adam J. Reiss and Matthew C. Bell Santa Clara University
Author Note Adam J. Reiss Department of Psychology, Santa Clara University and Matthew C. Bell Department of Psychology, Santa Clara University. A portion of this work was presented at the 35th Annual Meeting of the Society for the Quantitative Analysis of Behavior, 2012, Seattle, WA. This paper was in partial fulfillment for the first author’s senior thesis. We thank Marin Avram and SCU undergraduate research assistants for assisting in data collection. Address correspondence to Matthew C. Bell, Department of Psychology, Santa Clara University, Santa Clara, CA 95053. Email: [email protected].
1
FREE FOOD AND EXTINCTION
Reiss & Bell, Effect of a single free food presentation on extinction responding in a multiple schedule
Highlights
Does a single response-independent food presentation increase or decrease persistence in extinction? Replicates previous research showing more persistence for rich compared to lean schedules. Found that a single pre-session food presentation lead to more persistent extinction responding. More persistent responding from pre-session food presentations is not predicted by behavioral momentum theory
2
FREE FOOD AND EXTINCTION
3 Abstract
The present study was designed to assess the effect of a single, response-independent food presentation on responding during extinction. Using a two-component multiple schedule, we examined differences in pigeons’ extinction responding resulting from a single responseindependent food presentation occurring at the beginning of the experimental session (30-s prior to the beginning of the first component). One component presented reinforcement according to a variable interval 45-s schedule and the second presented reinforcement according to a variable interval 180-s schedule. After establishing stable baseline responding we extinguished responding. We systematically manipulated the presence or absence of a single 3-s free food presentation using the food hopper that occurred 30-s prior to the presentation of the first component. We found the single free food presentation increased persistence of responding in extinction. This finding is inconsistent with behavioral momentum theory inasmuch as it assigns a response disruptive role to food presentations occurring outside of the context of the target operant.
KEYWORDS: arousal, resistance to change, extinction, multiple schedule, pigeons, key peck
FREE FOOD AND EXTINCTION
4
1 Introduction Reinforcement, by definition, functions to increase the likelihood that a behavior will occur in the future. The details of exactly how reinforcement works to affect change in behavior, however, has been a matter of much debate (see, e.g., Baum, 2012a, 2012b, 2015, for a recent account; see also Premack, 1959, Timberlake & Allison, 1974). Traditional theories generally suggest that reinforcement functions by increasing response strength (e.g., Skinner, 1938). Behavioral momentum theory (BMT) fits within the traditional approach. It uses Newtonian physics as a metaphor (Nevin, 1992; Nevin & Grace, 2000a), borrowing the concept of momentum which is a product of velocity and mass. With this framework, BMT describes both steady-state behavior and behavior that has been disrupted. The momentum of behavior is said to be the product of response rate and resistance to change, such that a behavior with more mass (i.e., more response strength) will be less affected (i.e., disrupted) compared to one with less mass when an external force is added to the situation. Nevin (1992; Nevin & Grace, 2000a) suggest that the two constituents of behavioral momentum are independent factors (but see Bell, 1999, and Grace, Schwendiman, & Nevin, 1998, for evidence challenging this claim). The first, corresponding to velocity, describes steadystate responding. It is comparable to Herrnstein’s (1970) quantitative model of the relationship between response rate and reinforcer rate. The second, corresponding to mass, describes how resistant behavior is to disruption. The typical experiment uses food reinforcers to establish steady-state responding that is then disrupted to evaluate resistance to change. The three disruption procedures most often used are extinction, prefeeding prior to experimental sessions, and response-independent food delivered outside of the context of the target operant.
FREE FOOD AND EXTINCTION
5
According to BMT, resistance to change is determined by the Pavlovian relationship between the stimulus and the reinforcement delivered in the context of that stimulus. Thus, the higher the rate of reinforcement during baseline training, the more resistant responding should be under conditions of disruption. Importantly, as the mechanism for increasing resistance to change is Pavlovian, the process of imbuing strength to the stimulus occurs independently from the operant response-reinforcer contingency. Nevin, Tota, Torquato, and Shull (1990) directly tested the independence of the operant contingency and resistance to change. In their study, pigeons were exposed to multiple schedules with equal VI schedules in both components. In one component, additional responseindependent food presentations occurred according to a variable time (VT) schedule, degrading the response-reinforcer contingency and reducing steady-state responding in that component. Despite the decrement in baseline response rate, they found greater resistance to extinction and satiation for the component where a VT schedule was paired with a VI schedule compared to a component with only a VI schedule (and no additional sources of reinforcement). Interestingly, the same basic procedure of presenting response-independent food but doing so outside of a target stimulus-reinforcer context (i.e., outside of the three-term contingency) serves to disrupt behavior. For example, Nevin, Mandell, and Atak (1983) trained six pigeons to peck a key on a two-component multiple schedule with VI schedules operating in each component. Response-independent food was presented during periods between components (in which keys were dark so that reinforcers received during this time were not associated with any key light stimulus) as a means of testing resistance to change. They found that behavior was disrupted by the dark-key food presentation as continued exposure to the dark-key food
FREE FOOD AND EXTINCTION
6
presentations often reduced response rates to the point that they significantly altered the rates at which they received reinforcement. Thus, BMT treats response-independent food presentations differently as a function of where they are presented. Within the context of a discriminated operant, those additional food presentations, through a Pavlovian conditioning mechanism, are associated with the discriminative stimulus and, as a result, increase the putative response strength and leads to higher resistance to change. When presented outside of the context of the operant three-term contingency those food presentations have been used as a tool to disrupt the operant behavior either by prefeeding subjects prior to the experimental session or by presenting responseindependent (free) food during inter-component intervals. Other perspectives suggest that response-independent food presentations not associated with a particular stimulus could serve a separate function. Killeen, Hansen, and Osborne (1978; also, Killeen 1981) predicted that once arousal had been generated, it would increase exponentially with subsequent presentations of an incentive before the arousal would eventually reach an asymptote. In terms of behavior, the model predicts that once arousal is generated by exposure to an incentive, response rates should increase. To test the model, Killeen et al. presented two pigeons with a non-contingent 2-s food presentation, after which the chamber was darkened and movement was recorded using six floor panels for 15 min. This movement was used as a quantification of arousal. They found that the rate of floor panel activation increased by about 10 responses per minute (the experimental chamber contained six movable floor panels with attached microswitches where each activation of the switch was counted as an activity responses) following the free presentation of food, supporting the prediction. It is worth noting
FREE FOOD AND EXTINCTION
7
that activity level increased and then decreased with increasing time from the free food presentation. There are meaningful differences between Killeen et al. (1978) and research conducted under the theoretical umbrella of BMT. One potentially important difference is that BMT-related work focuses on operant behavior whereas in Killeen et al. the target behavior was an increase in activity level and not a previously reinforced operant response. Note, however, Killeen’s Mathematical Principles of Reinforcement (MPR; Killeen, 1994; Killeen & Sitomer, 2003; see also Nevin, 2003, where the relationship between BMT and MPR is explored) includes arousal as one of the key components of the theory which can be and is applied to operant responding. The second, more critical difference, is that the typical behavioral momentum study programs rich free-food presentations to disrupt responding while Killeen et al. examined the effect of a single 2-s response-independent food presentation on increasing responding. The difference in the direction of the effect suggests that the function describing the effect of these free food presentations on behavior will be non-linear. The actual procedural difference is the number of free food presentations. Thus, fewer free food presentations appear to function to energize responding whereas more presentations decreases responding. The purpose of the present study was to examine the effect of a single responseindependent food presentation on key pecking during extinction, similar to Killeen et al. (1978), but using a procedure more typical for the behavioral momentum literature. Thus, baseline training presented pigeons with a two-component multiple schedule with one rich VI schedule component and one lean VI schedule component (e.g., Berry, Sweeney, & Odum, 2014;
FREE FOOD AND EXTINCTION
8
Kuroda, Cançado, & Podlesnik, 2016; Nevin, 1974; Nevin, Mandell, & Atak, 1983). Following baseline training we extinguished responding. In some conditions we also presented a single response-independent food presentation 30-s prior to the first response component. Thus, across a series of conditions, the free food presentation was present just in baseline training, just in extinction, both baseline and extinction, or neither. Our objective was to see if that one food presentation would have an incentive effect, a disruptive effect, or no effect on responding in a procedure typical of BMT research. 2 Method 2.1 Subjects Eight experienced White Carneaux pigeons (Palmetto Pigeon Plant, Sumter, SC) were maintained at 80% of their free-feeding weights, (M = 497.3 g, SD = 29.8). They were individually housed in a temperature-controlled colony with a 12-h day/night cycle with access to water and grit at all times. 2.2 Apparatus Four identical Coulbourn Instrument chambers were used which measured 29 cm wide by 25 cm deep by 29 cm high. Chambers were enclosed within a wooden enclosure. External sounds were masked by the ventilation fan. Within the chamber three response keys were located on the front panel 18 cm from the bottom and 6 cm apart. Three houselights, mounted 1 cm below the ceiling and 8 cm apart were located on the rear panel. The hopper was located 10 cm below the center response key and 3 cm above the floor. Experimental events were programmed and data was collected using Coulbourn’s Graphic State software, which recorded every event at every time point.
FREE FOOD AND EXTINCTION
9
2.3 Procedure Each session consisted of a two-component multiple schedule, where one component programmed reinforcement according to a VI 45-s schedule (the “rich” component) and the second component was programmed to reinforce responding according to a VI 180-s schedule (the “lean” component). All VI schedules were programmed using the Fleshler and Hoffman (1962) distribution. Reinforcement consisted of 3-s presentations of milo, during which time the hopper light was illuminated. Each component was presented four times during a session and lasted 5 min. The order of component presentations was pseudo-randomly alternated such that there were an equal number of rich and lean components within any given session. Sessions began with a 2-min blackout period, where the critical manipulation occurred. In control conditions (hereafter designated as “0”), no food was presented during the initial blackout. In free food conditions (hereafter “+1”), a single 3-s presentation of food occurred in the blackout, 30-s prior to the beginning of the first component. In free food conditions, the food presentation occurred equally often before a rich first component and a lean first component as the selection of the first component was random. Both baseline and extinction phases within a condition occurred with (+1) and without (0) free food presentations such that all possible combinations occurred. The number of sessions for each phase and the order of presentation of conditions was counterbalanced (see Table 1). The purpose of having baseline conditions with and without free food presentations and counterbalancing with subsequent extinction conditions with and without free food was to control for generalization effects that might occur as a function of a transition from a baseline
FREE FOOD AND EXTINCTION
10
condition and to allow assessment of the effects, if any, of the one free food presentation on baseline responding. 3. Results 3.1 Baseline data The mean response and reinforcement rates for the last five sessions from each condition were used for analysis. Data from the two +1 conditions were aggregated, as were data from the two 0 conditions, for analysis. Although reinforcement rates were below programmed rates, we did maintain a difference in baseline reinforcement rate (reinforcers per minute) for the rich reinforcement component (M = 67.8, SD = 2.6 reinforcers per hour) the lean component, M = 8.4 (SD = 1.0) reinforcers per hour. These data were analyzed with a 2 x 2 ANOVA with component (rich and lean) and pre-session food presentation (0 or +1) as factors showed a main effect of component with F(1, 7) = 6941.0, p < .001, p2 = .999 and no main effect of pre-food presentation, F(1, 7) = 0.009, p = .93 and no interaction, F(1, 7) = 0.3, p = .61. The mean baseline response rate (responses per minute, averaged across the last five sessions of each baseline) for the rich reinforcement component was M = 76.7 (SD = 49.1) responses per minute and for the lean reinforcement component, the mean baseline reinforcement rate was M = 46.2 (SD = 48.3) responses per minute. These results were analyzed using a 2 x 2 within subjects ANOVA, again with component (rich and lean) and pre-session food presentation (0 or +1) as factors, showed a main effect of component with F(1, 7) = 27.9, p = .001, p2 = .80. There was no main effect of pre-food presentation, F(1, 7) = 0.3, p = .62 and no interaction, F(1, 7) = 0.4, p = .55. 3.2 Resistance to change
FREE FOOD AND EXTINCTION
11
Given there was no differences in responding during baseline as a function of the additional food and the fact that a preliminary analysis suggested there was no effect of a transition from +1 to 0 or from 0 to +1 on responding during extinction, we collapsed across those conditions for analysis of extinction data. The first eight sessions of extinction were used for consistency in analysis for all subjects and because responding in the later extinction sessions was typically near zero responses. Using the mean of the last five baseline sessions preceding each extinction test, log proportion of baseline was calculated for each extinction session and collapsed into two-session blocks (shown in Figure 1). Following previous BMT research (see, e.g., Nevin & Grace, 1999, 2000b; Nevin, McLean, & Grace, 2001; Nevin, Grace, Holland, & McLean, 2001) these data were used to calculate slope for analysis (response rate data are presented in Table 2). The resulting slopes were analyzed using a 2 x 2 within-subjects ANOVA with component (rich and lean) and pre-session food presentation (0 or +1) as factors. There was a significant main effect of component, F(1, 7) = 9.4, p =.02, p2 = .57 and a significant main effect of pre-session food presentation, F(1, 7) = 18.4, p =.004, p2 = .72. The interaction was not significant, F(1, 7) = 0.24, p = .64. As Figure 2 illustrates, the rich component was more resistant to change compared to the lean component, as evidenced by smaller negative slopes. Importantly, the figure also shows the main effect of the pre-session food presentation which shows more resistance to change for the conditions where one food presentation was present compared to the conditions where it was absent. 4 Discussion Using a two-component multiple schedule procedure with a rich and lean VI component,
FREE FOOD AND EXTINCTION
12
we replicated previous work showing rate of reinforcement predicts resistance to change in a multiple schedule (e.g. Nevin, 1984; for reviews, see Nevin 1992; Nevin & Grace, 2000a). More importantly, we found that a single pre-session food presentation can have a measurable impact on responding during extinction (but not on baseline responding), particularly in the lean component. Examination of Figure 1 suggests that the additional free food presentation appears to have had a lesser effect on responding in the rich component, which may simply be a reflection of higher rates of responding in the rich component and thus more change required to show an effect. Our finding is consistent with the notion that response-independent food presented outside of a stimulus-reinforcer context can function to arouse responding during extinction performance. This finding is inconsistent with BMT inasmuch as extra-session food presentations are considered to function as behavior disruptors. It is consistent with Killeen et al. (1978) who showed that a single food presentation at the beginning of a daily experimental session functioned to increase overall activity level. Thus, food presentations presented outside of the context of the three-term contingency can function to arouse behavior. While it is possible that BMT could be configured to address this issue, it has not yet been formally incorporated into the model. BMT would have to have some mechanism for determining when these events function as disruptors and when they function to arouse responding. Currently, when response-independent food is presented in the presence of a discriminative stimulus and its associated operant, it is said to enhance response strength (e.g., Nevin et al., 1990) and when it occurs outside of that context, it functions to disrupt responding (e.g., Nevin et al., 1983). The relationship between response-independent food presentations and its effect on responding and the dynamics of response change during extinction are complex, as
FREE FOOD AND EXTINCTION
13
demonstrated in recent research investigating related factors and their effect on responding during extinction (e.g., Podlesnik & Bai, 2015; Podlesnik & Fleet, 2014) and recent modeling of extinction in the context of behavioral momentum theory (e.g., Nevin & Shahan, 2011). Recent research applying BMT to an animal model of drug relapse demonstrated that extinction of a food contingency can function to increase previously-extinguished cocaine-seeking behavior in rats (Quick, Pyszczynski, Colston, & Shahan, 2011). One potential concern is that the present results are simply due to a generalization decrement effect. Given the extinction data were collapsed across the preceding baseline conditions (i.e., those with and without a free food presentation occurring at the beginning of the session), this seems unlikely. Any generalization decrement effects resulting from the presence or absence of the single food presentation should have been minimized or eliminated because of our counterbalanced design (Dulaney, & Bell, 2008; Nevin, McLean, & Grace, 2001 explicitly investigated generalization decrement and resistance to extinction). Our results are related to recent studies examining the effect of additional responseindependent food presentations on the reinstatement of extinguished operant responding. Reinstatement studies present the reinforcer used to maintain responding during baseline training following extinction, typically within the presence of the discriminative stimulus, resulting in the extinguished response reappearing (e.g., Reid, 1958). This paradigm has served as an animal model of relapse in research on substance abuse (for a review, see Shaham, Shalev, Lu, de Wit, & Stewart, 2003). Podlesnik and Shahan (2009; see also Podlesnik & Shahan, 2010), for example, describe a study quite similar to the present experiment. In their baseline condition pigeons were
FREE FOOD AND EXTINCTION
14
presented with a two-component multiple schedule with a rich and a lean component. Following extinction, they reinstated responding by providing two food presentations at the beginning of each component of reinstatement session. In other words, the additional food presentations occurred in the presence of the stimuli that had been correlated with food during baseline training. They found more responding in the rich component compared to the lean component, consistent with BMT. Miranda-Dukoski, Bensemann, and Podlesnik (2016) replicated the basic procedure and finding of Podlesnik and Shahan (2009) using a four-component multiple schedule. During baseline training, two target components, one rich and one lean, reinforced responding while two similar components were associated with extinction. Two reinstatement tests followed extinction where three 2-s food presentations occurred. In one reinstatement test, those food presentations occurred in the presence of the target component stimuli (i.e., where responding was reinforced). In the other test, the food presentations occurred in the presence of the stimuli that had always been correlated with extinction. They found generally larger recovery of responding (relapse) when the food presentations occurred in the presence of the target component stimuli compared to the presentations in the presence of the extinction-related stimuli and, moreover, more recovery of responding for the rich component compared to the lean. When the food presentations occurred in the presence of the extinction-related stimuli, no differences in recovery of responding were seen. This finding contrasts with our findings but this is not particularly surprising, given the numerous procedural differences. It is difficult to know from the present study what variables are responsible for the different result. One issue warranting further investigation is predicting when free food presentations
FREE FOOD AND EXTINCTION
15
function to arouse responding and when they disrupt responding, as this has important implications for substance abuse relapse. Related to that is determining the function of responseindependent food presentations when they are in the presence or absence of discriminative stimuli. In a related study, Podlesnik and Jimenez-Gomez (2016) reported a disruption test where free food was presented in a cup within the operant condition apparatus (contrafreeloading; see Osborne, 1977, for a review) and compared it to the more typical disruption by pre-session feeding. In both conditions the food functioned to disrupt responding in a fashion consistent with BMT – i.e., more persistent responding in the presence of the stimulus correlated with a richer rate of reinforcement compared to a stimulus correlated with a leaner reinforcement rate regardless of the context in which the food appeared (i.e., in contrafreeloading it was available within the context of the discriminated operant and in prefeeding it was presented prior to the experimental session). This study represents a similar attempt to investigate the effect of response-independent food on responding when it is presented within and outside of the context of the discriminated operant. 5 Conclusion We demonstrated that a single response-independent food presentation occurring 30-s prior to the presentation of the first component in a two-component multiple schedule can have a measurable effect on responding during extinction. Our findings are consistent with the food presentation having an arousal function, similar to what was reported by Killeen et al. (1978). They are inconsistent with the disruption role BMT assigns food presentations occurring outside of the context of the stimulus being assessed for resistance to change.
FREE FOOD AND EXTINCTION
16 References
Baum, W. M. (2012a). Extinction as discrimination: The molar view. Behavioural Processes, 90, 101-110. doi:10.1016/j.beproc.2012.02.011 Baum, W. M. (2012b). Rethinking reinforcement: Allocation, induction, and contingency. Journal of the Experimental Analysis of Behavior, 97, 101-124. doi:10.1901/jeab.2012.97-101 Baum, W. M. (2015). The role of induction in operant schedule performance. Behavioural Processes, 114, 26-33. doi:10.1016/j.beproc.2015.01.006 Bell, M. C. (1999). Pavlovian contingencies and resistance to change in a multiple schedule. Journal of the Experimental Analysis of Behavior, 72, 81-96. Berry, M. S., Sweeney, M. M., & Odum, A. L. (2014). Effects of baseline reinforcement rate on operant ABA and ABC renewal. Behavioural Processes, 108, 87-93. doi:10.1016/j.beproc.2014.09.009 Dulaney, A. E., & Bell, M. C. (2008). Resistance to extinction, generalization decrement, and conditioned reinforcement. Behavioural Processes, 78, 253-258. doi:10.1016/j.beproc.2007.12.006 Fleshler, M., & Hoffman, H. S. (1962). A progression for generating variable-interval schedules. Journal of the Experimental Analysis of Behavior, 5, 529-530. doi:10.1901/jeab.1962.5-529 Grace, R. C., Schwendiman, J. W., & Nevin, J. A. (1998). Effects of unsignaled delay of reinforcement on preference and resistance to change. Journal of the Experimental Analysis of Behavior, 69, 247–261.
FREE FOOD AND EXTINCTION
17
Herrnstein, R. J. (1970). On the law of effect. Journal of the Experimental Analysis of Behavior, 13, 243-266. doi:10.1901/jeab.1970.13-243 Killeen, P. R. (1981). Incentive theory. Nebraska Symposium on Motivation, 29, 169-216. Killeen, P. R. (1994). Mathematical principles of reinforcement. Behavioral and Brain Sciences, 17, 105–172. Killeen, P. R., & Sitomer, M. T. (2003). MPR. Behavioural Processes, 62, 49-64. doi:10.1016/S0376-6357(03)00017-2 Killeen, P. R., Hanson, S. J., & Osborne, S. R. (1978). Arousal: Its genesis and manifestation as response rate. Psychological Review, 85, 571-581. Kuroda, T., Cançado, C. X., & Podlesnik, C. A. (2016). Resistance to change and resurgence in humans engaging in a computer task. Behavioural Processes, 125, 1-5. doi:10.1016/j.beproc.2016.01.010 Miranda-Dukoski, L., Bensemann, J., & Podlesnik, C. A. (2016). Training reinforcement rates, resistance to extinction, and the role of context in reinstatement. Learning & Behavior, 44, 29-48. doi:10.3758/s13420-015-0188-8 Nevin, J. A. (1974). Response strength in multiple schedules. Journal of the Experimental Analysis of Behavior, 21, 389-408. doi:10.1901/jeab.1974.21-389 Nevin, J. A. (1984). Pavlovian determiners of behavioral momentum. Animal Learning & Behavior, 12, 363-370. Nevin, J. A. (1992). An integrative model for the study of behavioral momentum. Journal of the Experimental Analysis of Behavior, 57, 301-316.
FREE FOOD AND EXTINCTION
18
Nevin, J. A. (2003). Mathematical principles of reinforcement and resistance to change. Behavioural Processes, 62, 65-73. doi:10.1016/S0376-6357(03)00018-4 Nevin, J. A., & Grace, R. C. (1999). Does the context of reinforcement affect resistance to change?. Journal of Experimental Psychology: Animal Behavior Processes, 25, 256-268. doi:10.1037/0097-7403.25.2.256 Nevin, J. A., & Grace, R. C. (2000a). Behavioral momentum and the Law of Effect. Behavioral and Brain Sciences, 23, 73-130. Nevin, J. A., & Grace, R. C. (2000b). Preference and resistance to change with constant-duration schedule components. Journal of the Experimental Analysis of Behavior, 74, 79-100. doi:10.1901/jeab.2000.74-79 Nevin, J. A., & Shahan, T. A. (2011). Behavioral momentum theory: Equations and applications. Journal of Applied Behavior Analysis, 44, 877-895. doi:10.1901/jaba.2011.44-877 Nevin, J. A., Grace, R. C., Holland, S., & McLean, A. P. (2001). Variable-ratio versus variableinterval schedules: Response rate, resistance to change, and preference. Journal of the Experimental Analysis of Behavior, 76, 43-74. doi:10.1901/jeab.2001.76-43 Nevin, J. A., Mandell, C., & Atak, J. R. (1983). The analysis of behavioral momentum. Journal of the Experimental Analysis of Behavior, 39, 49-59. Nevin, J. A., McLean, A. P., & Grace, R. C. (2001). Resistance to extinction: Contingency termination and generalization decrement. Animal Learning & Behavior, 29, 176-191. doi:10.3758/BF03192826
FREE FOOD AND EXTINCTION
19
Nevin, J. A., Tota, M. E., Torquato, R. D., & Shull, R. L. (1990). Alternative reinforcement increases resistance to change: Pavlovian or operant contingencies?. Journal of the Experimental Analysis of Behavior, 53, 359-379. doi:10.1901/jeab.1990.53-359 Osborne, S. R. (1977). The free food (contrafreeloading) phenomenon: A review and analysis. Animal Learning & Behavior, 5, 221-235. doi:10.3758/BF03209232 Podlesnik, C. A., & Bai, J. H. (2015). Method of stimulus combination impacts resistance to extinction. Journal of the Experimental Analysis of Behavior, 104, 30-47. doi:10.1002/jeab.155 Podlesnik, C. A., & Fleet, J. D. (2014). Signaling added response‐independent reinforcement to assess Pavlovian processes in resistance to change and relapse. Journal of the Experimental Analysis of Behavior, 102, 179-197. doi:10.1002/jeab.96 Podlesnik, C. A., & Jimenez-Gomez, C. (2016). Contrafreeloading, reinforcement rate, and behavioral momentum. Behavioural Processes, 128, 24-28. http://dx.doi.org/10.1016/j.beproc.2016.03.022 Podlesnik, C. A., & Shahan, T. A. (2009). Behavioral momentum and relapse of extinguished operant responding. Learning & Behavior, 37, 357-364. doi:10.3758/LB.37.4.357 Podlesnik, C. A., & Shahan, T. A. (2010). Extinction, relapse, and behavioral momentum. Behavioural Processes, 84, 400-411. doi:10.1016/j.beproc.2010.02.001 Premack, D. (1959). Toward empirical behavior laws: I. Positive reinforcement. Psychological Review, 66, 219-233. doi:10.1037/h0040891
FREE FOOD AND EXTINCTION
20
Quick, S. L., Pyszczynski, A. D., Colston, K. A., & Shahan, T. A. (2011). Loss of alternative non-drug reinforcement induces relapse of cocaine-seeking in rats: Role of dopamine D₁ receptors.Neuropsychopharmacology, 36, 1015-1020. doi:10.1038/npp.2010.239 Reid, R. L. (1958). The role of the reinforcer as a stimulus. British Journal of Psychology, 49, 202-209. doi:10.1111/j.2044-8295.1958.tb00658.x Shaham Y., Shalev U., Lu L., de Wit H., & Stewart J. (2003). The reinstatement model of drug relapse: History, methodology, and major findings. Psychopharmacology, 168, 3–20. Skinner, B. F. (1938). The behavior of organisms: An experimental analysis. Cambridge, MA: Appleton-Century-Crofts. Timberlake, W., & Allison, J. (1974). Response deprivation: An empirical approach to instrumental performance. Psychological Review, 81, 146-164. doi:10.1037/h0036101
FREE FOOD AND EXTINCTION
21
Table 1. Number of sessions for each condition and order of conditions for the one food presentations. The labels represent which baseline and extinction conditions were in effect, with “0” indicating no free-food presentation occurred for that part of the study and “+1” indicating that one food presentation did occur at the beginning of each session. For example, 0/0 indicates that both the baseline and extinction had no pre-session food presentations whereas +1/0 indicates that the baseline included a single food presentation at the beginning of the session and the subsequent extinction condition did not.
Bird G260, G264
G263, G265
G251, G255
G253, G268
One Food Baseline Extinction
0/0 37 10
0/+1 37 10
+1/+1 29 8
+1/0 29 8
One Food Baseline Extinction
0/+1 13 8
0/0 13 8
+1/0 10 8
+1/+1 10 8
One Food Baseline Extinction
+1/+1 10 10
+1/0 10 10
0/0 22 11
0/+1 22 11
One Food Baseline Extinction
+1/0 10 8
+1/+1 10 8
0/+1 10 9
0/0 10 9
Condition 1
Condition 2
Condition 3
Condition 4
FREE FOOD AND EXTINCTION
22
Table 2. Response rates (responses per min) for each block, including baseline (B) M (SD) of the last five sessions and extinction responding represented as blocks of two-session for the rich (VI 45-s) and lean (VI 180-s) components. Condition labels indicate which baseline and extinction conditions were in effect. A “0” indicates no free-food presentation occurred for that part of the study and “+1” indicates that one food presentation did occur at the beginning of each session. For example, 0/0 indicates that both the baseline and extinction had no pre-session food presentations and +1/0 indicates that the baseline included a single food presentation at the beginning of the session and the subsequent extinction condition did not.
NOTE THAT THE TABLE WAS UPLOADED SEPARATELY
FREE FOOD AND EXTINCTION
23 Figure Captions
Figure 1. Resistance to change data during extinction across session blocks as log proportion of baseline response rates for individual subjects. Mean responding is also presented in the bottom right graph. Data are shown as a function of rich and lean components and the presence (+1) or absence (0) of a single food presentation occurring 30-s prior to the beginning of the first component. Figure 2. Mean slope during extinction for the rich (left panel) and lean (right panel) components as a function of whether food was never presented during extinction (0) or if one food presentation occurred 30-s prior to the first component (+1). Error bars show the standard error of the mean.
Table 2. Response rates (responses per min) for each block, including baseline (B) M (SD) of the last five sessions and extinction responding represented as blocks of two-session for the rich (VI 45-s) and lean (VI 180-s) components. Condition labels indicate which baseline and extinction conditions were in effect. A “0” indicates no free-food presentation occurred for that part of the study and “+1” indicates that one food presentation did occur at the beginning of each session. For example, 0/0 indicates that both the baseline and extinction had no pre-session food presentations and +1/0 indicates that the baseline included a single food presentation at the beginning of the session and the subsequent extinction condition did not.
G260 Condition Block VI 45-s
VI 180-s
G264 VI 45-s
74.5 (5.9) 81.9 75.6 78.4 17.5
VI 180-s
G263 VI 45-s
VI 180-s
68 (7.5) 31.0 15.0 0.3 3.9
177.6 (9.3) 172.1 47.7 135.2 2.8
162.5 (42) 51.5 14.3 38.9 2.4
0/0 `
B 1 2 3 4
46.7 (4.4) 26.3 (16.2) 58.0 12.2 28.7 8.9 6.0 0.0 18.5 0.1
0/+1 B 1 2 3 4
29.8 (8) 26.1 12.5 4.5 9.2
12.2 (5.3) 3.7 4.7 6.3 0.0
67.9 (3.7) 60.1 (22.4) 62.9 39.7 34.4 21.0 31.4 8.9 22.2 13.3
186.4 (9.4) 121.9 148.3 112.6 19.1
123.4 (32) 56.5 22.4 9.7 4.7
B 1 2 3 4
43.5 (5.6) 22 (10.4) 16.7 9.8 4.4 1.1 0.4 0.0 1.3 0.2
68 (3.5) 55.3 (15.9) 29.7 19.1 29.7 12.1 31.4 4.1 4.8 4.3
183.7 (7.1) 67.3 68.7 13.1 36.4
206 (10.8) 37.6 86.9 1.5 5.6
B 1 2 3 4
49.3 (5.3) 25.3 (5.6) 20.1 9.0 10.3 6.0 1.8 3.7 9.1 0.0
70.9 (3.3) 31.6 (19.3) 51.9 16.2 21.7 10.5 34.3 6.0 9.2 3.9
+1/0
+1/+1 174.1 (10.6) 147.8 (46.8) 61.5 27.2 22.7 28.1 29.3 31.5 23.7 3.4
G265 VI 45-s
VI 180-s
G251 VI 45-s
VI 180-s
G255 VI 45-s
VI 180-s
G253 VI 45-s
104.1 (7.6) 47.7 (10.1) 85.2 14.9 21.9 3.5 2.6 3.1 6.3 5.2
68.1 (3.4) 27.1 (6.3) 53.5 11.5 18.7 10.7 11.4 4.3 17.5 2.8
31.7 (2.9) 11.4 (4.3) 22.0 6.8 10.3 3.3 3.8 0.7 7.5 1.0
32.7 (3.1) 32.8 1.5 12.9 0.6
107.5 (25.9) 55.6 (20) 77.6 43.8 92.5 24.9 63.3 19.4 20.6 4.1
63.6 (7.1) 29.4 (4.7) 46.4 15.4 19.8 7.3 13.6 4.3 17.9 6.2
33.8 (2.2) 13.5 (3.4) 25.8 6.3 14.7 5.4 10.0 2.9 7.4 1.4
38.7 (25) 24.5 1.8 0.1 4.7
33.2 (7.8) 4.9 3.1 2.5 3.4
30.3 (4) 16.4 (2.3) 22.0 8.8 8.3 1.7 3.6 0.3 6.9 0.3
32.9 (4) 34.3 4.4 1.7 6.0
78.8 (8.8) 28.7 (10.9) 47.2 14.9 31.4 6.5 13.0 2.0 3.0 0.6
28.4 (3) 18.6 (4.9) 28.9 11.9 15.3 8.0 9.1 0.6 10.3 1.5
45.7 (5.1) 33.9 5.0 0.3 16.5
118.2 (6) 60.2 36.9 42.0 17.1
34.6 (16.3) 24.5 9.3 6.2 6.1
115.3 (1.3) 54.4 (15.1) 55.8 10.3 14.7 14.5 26.1 1.2 11.1 3.5
71.9 (6) 49.4 12.5 11.2 10.2
VI 180-s
G268 VI 45-s
VI 180-s
19 (9.7) 1.5 0.3 1.1 0.6
63.1 (4.2) 44.7 (12.1) 71.6 13.9 81.4 2.2 30.9 1.7 22.7 0.2
4.7 (3.6) 0.5 0.6 0.1 0.2
71.3 (8.2) 84.3 38.4 52.8 27.9
39.6 (9.9) 5.6 1.9 4.7 2.2
10.7 (5.9) 6.5 0.1 0.0 0.3
71.9 (6.2) 77.7 82.6 66.1 20.2
12.2 (7) 1.0 0.4 0.4 0.4
8.2 (4.7) 4.0 0.9 0.5 0.7
74.9 (11.8) 28 (11.5) 80.8 5.6 71.8 0.4 46.2 1.6 39.0 2.0
S0376-6357(16)30155-3 http://dx.doi.org/doi:10.1016/j.beproc.2016.07.002 BEPROC 3273
To appear in:
Behavioural Processes
Received date: Revised date: Accepted date:
9-4-2016 5-7-2016 6-7-2016
Please cite this article as: Reiss, Adam J., Bell, Matthew C., Effect of a Single Free Food Presentation on Extinction Responding in a Multiple Schedule.Behavioural Processes http://dx.doi.org/10.1016/j.beproc.2016.07.002 This is a PDF file of an unedited manuscript that has been accepted for publication. As a service to our customers we are providing this early version of the manuscript. The manuscript will undergo copyediting, typesetting, and review of the resulting proof before it is published in its final form. Please note that during the production process errors may be discovered which could affect the content, and all legal disclaimers that apply to the journal pertain.
Running head: FREE FOOD AND EXTINCTION
Effect of a Single Free Food Presentation on Extinction Responding in a Multiple Schedule Adam J. Reiss and Matthew C. Bell Santa Clara University
Author Note Adam J. Reiss Department of Psychology, Santa Clara University and Matthew C. Bell Department of Psychology, Santa Clara University. A portion of this work was presented at the 35th Annual Meeting of the Society for the Quantitative Analysis of Behavior, 2012, Seattle, WA. This paper was in partial fulfillment for the first author’s senior thesis. We thank Marin Avram and SCU undergraduate research assistants for assisting in data collection. Address correspondence to Matthew C. Bell, Department of Psychology, Santa Clara University, Santa Clara, CA 95053. Email: [email protected].
1
FREE FOOD AND EXTINCTION
Reiss & Bell, Effect of a single free food presentation on extinction responding in a multiple schedule
Highlights
Does a single response-independent food presentation increase or decrease persistence in extinction? Replicates previous research showing more persistence for rich compared to lean schedules. Found that a single pre-session food presentation lead to more persistent extinction responding. More persistent responding from pre-session food presentations is not predicted by behavioral momentum theory
2
FREE FOOD AND EXTINCTION
3 Abstract
The present study was designed to assess the effect of a single, response-independent food presentation on responding during extinction. Using a two-component multiple schedule, we examined differences in pigeons’ extinction responding resulting from a single responseindependent food presentation occurring at the beginning of the experimental session (30-s prior to the beginning of the first component). One component presented reinforcement according to a variable interval 45-s schedule and the second presented reinforcement according to a variable interval 180-s schedule. After establishing stable baseline responding we extinguished responding. We systematically manipulated the presence or absence of a single 3-s free food presentation using the food hopper that occurred 30-s prior to the presentation of the first component. We found the single free food presentation increased persistence of responding in extinction. This finding is inconsistent with behavioral momentum theory inasmuch as it assigns a response disruptive role to food presentations occurring outside of the context of the target operant.
KEYWORDS: arousal, resistance to change, extinction, multiple schedule, pigeons, key peck
FREE FOOD AND EXTINCTION
4
1 Introduction Reinforcement, by definition, functions to increase the likelihood that a behavior will occur in the future. The details of exactly how reinforcement works to affect change in behavior, however, has been a matter of much debate (see, e.g., Baum, 2012a, 2012b, 2015, for a recent account; see also Premack, 1959, Timberlake & Allison, 1974). Traditional theories generally suggest that reinforcement functions by increasing response strength (e.g., Skinner, 1938). Behavioral momentum theory (BMT) fits within the traditional approach. It uses Newtonian physics as a metaphor (Nevin, 1992; Nevin & Grace, 2000a), borrowing the concept of momentum which is a product of velocity and mass. With this framework, BMT describes both steady-state behavior and behavior that has been disrupted. The momentum of behavior is said to be the product of response rate and resistance to change, such that a behavior with more mass (i.e., more response strength) will be less affected (i.e., disrupted) compared to one with less mass when an external force is added to the situation. Nevin (1992; Nevin & Grace, 2000a) suggest that the two constituents of behavioral momentum are independent factors (but see Bell, 1999, and Grace, Schwendiman, & Nevin, 1998, for evidence challenging this claim). The first, corresponding to velocity, describes steadystate responding. It is comparable to Herrnstein’s (1970) quantitative model of the relationship between response rate and reinforcer rate. The second, corresponding to mass, describes how resistant behavior is to disruption. The typical experiment uses food reinforcers to establish steady-state responding that is then disrupted to evaluate resistance to change. The three disruption procedures most often used are extinction, prefeeding prior to experimental sessions, and response-independent food delivered outside of the context of the target operant.
FREE FOOD AND EXTINCTION
5
According to BMT, resistance to change is determined by the Pavlovian relationship between the stimulus and the reinforcement delivered in the context of that stimulus. Thus, the higher the rate of reinforcement during baseline training, the more resistant responding should be under conditions of disruption. Importantly, as the mechanism for increasing resistance to change is Pavlovian, the process of imbuing strength to the stimulus occurs independently from the operant response-reinforcer contingency. Nevin, Tota, Torquato, and Shull (1990) directly tested the independence of the operant contingency and resistance to change. In their study, pigeons were exposed to multiple schedules with equal VI schedules in both components. In one component, additional responseindependent food presentations occurred according to a variable time (VT) schedule, degrading the response-reinforcer contingency and reducing steady-state responding in that component. Despite the decrement in baseline response rate, they found greater resistance to extinction and satiation for the component where a VT schedule was paired with a VI schedule compared to a component with only a VI schedule (and no additional sources of reinforcement). Interestingly, the same basic procedure of presenting response-independent food but doing so outside of a target stimulus-reinforcer context (i.e., outside of the three-term contingency) serves to disrupt behavior. For example, Nevin, Mandell, and Atak (1983) trained six pigeons to peck a key on a two-component multiple schedule with VI schedules operating in each component. Response-independent food was presented during periods between components (in which keys were dark so that reinforcers received during this time were not associated with any key light stimulus) as a means of testing resistance to change. They found that behavior was disrupted by the dark-key food presentation as continued exposure to the dark-key food
FREE FOOD AND EXTINCTION
6
presentations often reduced response rates to the point that they significantly altered the rates at which they received reinforcement. Thus, BMT treats response-independent food presentations differently as a function of where they are presented. Within the context of a discriminated operant, those additional food presentations, through a Pavlovian conditioning mechanism, are associated with the discriminative stimulus and, as a result, increase the putative response strength and leads to higher resistance to change. When presented outside of the context of the operant three-term contingency those food presentations have been used as a tool to disrupt the operant behavior either by prefeeding subjects prior to the experimental session or by presenting responseindependent (free) food during inter-component intervals. Other perspectives suggest that response-independent food presentations not associated with a particular stimulus could serve a separate function. Killeen, Hansen, and Osborne (1978; also, Killeen 1981) predicted that once arousal had been generated, it would increase exponentially with subsequent presentations of an incentive before the arousal would eventually reach an asymptote. In terms of behavior, the model predicts that once arousal is generated by exposure to an incentive, response rates should increase. To test the model, Killeen et al. presented two pigeons with a non-contingent 2-s food presentation, after which the chamber was darkened and movement was recorded using six floor panels for 15 min. This movement was used as a quantification of arousal. They found that the rate of floor panel activation increased by about 10 responses per minute (the experimental chamber contained six movable floor panels with attached microswitches where each activation of the switch was counted as an activity responses) following the free presentation of food, supporting the prediction. It is worth noting
FREE FOOD AND EXTINCTION
7
that activity level increased and then decreased with increasing time from the free food presentation. There are meaningful differences between Killeen et al. (1978) and research conducted under the theoretical umbrella of BMT. One potentially important difference is that BMT-related work focuses on operant behavior whereas in Killeen et al. the target behavior was an increase in activity level and not a previously reinforced operant response. Note, however, Killeen’s Mathematical Principles of Reinforcement (MPR; Killeen, 1994; Killeen & Sitomer, 2003; see also Nevin, 2003, where the relationship between BMT and MPR is explored) includes arousal as one of the key components of the theory which can be and is applied to operant responding. The second, more critical difference, is that the typical behavioral momentum study programs rich free-food presentations to disrupt responding while Killeen et al. examined the effect of a single 2-s response-independent food presentation on increasing responding. The difference in the direction of the effect suggests that the function describing the effect of these free food presentations on behavior will be non-linear. The actual procedural difference is the number of free food presentations. Thus, fewer free food presentations appear to function to energize responding whereas more presentations decreases responding. The purpose of the present study was to examine the effect of a single responseindependent food presentation on key pecking during extinction, similar to Killeen et al. (1978), but using a procedure more typical for the behavioral momentum literature. Thus, baseline training presented pigeons with a two-component multiple schedule with one rich VI schedule component and one lean VI schedule component (e.g., Berry, Sweeney, & Odum, 2014;
FREE FOOD AND EXTINCTION
8
Kuroda, Cançado, & Podlesnik, 2016; Nevin, 1974; Nevin, Mandell, & Atak, 1983). Following baseline training we extinguished responding. In some conditions we also presented a single response-independent food presentation 30-s prior to the first response component. Thus, across a series of conditions, the free food presentation was present just in baseline training, just in extinction, both baseline and extinction, or neither. Our objective was to see if that one food presentation would have an incentive effect, a disruptive effect, or no effect on responding in a procedure typical of BMT research. 2 Method 2.1 Subjects Eight experienced White Carneaux pigeons (Palmetto Pigeon Plant, Sumter, SC) were maintained at 80% of their free-feeding weights, (M = 497.3 g, SD = 29.8). They were individually housed in a temperature-controlled colony with a 12-h day/night cycle with access to water and grit at all times. 2.2 Apparatus Four identical Coulbourn Instrument chambers were used which measured 29 cm wide by 25 cm deep by 29 cm high. Chambers were enclosed within a wooden enclosure. External sounds were masked by the ventilation fan. Within the chamber three response keys were located on the front panel 18 cm from the bottom and 6 cm apart. Three houselights, mounted 1 cm below the ceiling and 8 cm apart were located on the rear panel. The hopper was located 10 cm below the center response key and 3 cm above the floor. Experimental events were programmed and data was collected using Coulbourn’s Graphic State software, which recorded every event at every time point.
FREE FOOD AND EXTINCTION
9
2.3 Procedure Each session consisted of a two-component multiple schedule, where one component programmed reinforcement according to a VI 45-s schedule (the “rich” component) and the second component was programmed to reinforce responding according to a VI 180-s schedule (the “lean” component). All VI schedules were programmed using the Fleshler and Hoffman (1962) distribution. Reinforcement consisted of 3-s presentations of milo, during which time the hopper light was illuminated. Each component was presented four times during a session and lasted 5 min. The order of component presentations was pseudo-randomly alternated such that there were an equal number of rich and lean components within any given session. Sessions began with a 2-min blackout period, where the critical manipulation occurred. In control conditions (hereafter designated as “0”), no food was presented during the initial blackout. In free food conditions (hereafter “+1”), a single 3-s presentation of food occurred in the blackout, 30-s prior to the beginning of the first component. In free food conditions, the food presentation occurred equally often before a rich first component and a lean first component as the selection of the first component was random. Both baseline and extinction phases within a condition occurred with (+1) and without (0) free food presentations such that all possible combinations occurred. The number of sessions for each phase and the order of presentation of conditions was counterbalanced (see Table 1). The purpose of having baseline conditions with and without free food presentations and counterbalancing with subsequent extinction conditions with and without free food was to control for generalization effects that might occur as a function of a transition from a baseline
FREE FOOD AND EXTINCTION
10
condition and to allow assessment of the effects, if any, of the one free food presentation on baseline responding. 3. Results 3.1 Baseline data The mean response and reinforcement rates for the last five sessions from each condition were used for analysis. Data from the two +1 conditions were aggregated, as were data from the two 0 conditions, for analysis. Although reinforcement rates were below programmed rates, we did maintain a difference in baseline reinforcement rate (reinforcers per minute) for the rich reinforcement component (M = 67.8, SD = 2.6 reinforcers per hour) the lean component, M = 8.4 (SD = 1.0) reinforcers per hour. These data were analyzed with a 2 x 2 ANOVA with component (rich and lean) and pre-session food presentation (0 or +1) as factors showed a main effect of component with F(1, 7) = 6941.0, p < .001, p2 = .999 and no main effect of pre-food presentation, F(1, 7) = 0.009, p = .93 and no interaction, F(1, 7) = 0.3, p = .61. The mean baseline response rate (responses per minute, averaged across the last five sessions of each baseline) for the rich reinforcement component was M = 76.7 (SD = 49.1) responses per minute and for the lean reinforcement component, the mean baseline reinforcement rate was M = 46.2 (SD = 48.3) responses per minute. These results were analyzed using a 2 x 2 within subjects ANOVA, again with component (rich and lean) and pre-session food presentation (0 or +1) as factors, showed a main effect of component with F(1, 7) = 27.9, p = .001, p2 = .80. There was no main effect of pre-food presentation, F(1, 7) = 0.3, p = .62 and no interaction, F(1, 7) = 0.4, p = .55. 3.2 Resistance to change
FREE FOOD AND EXTINCTION
11
Given there was no differences in responding during baseline as a function of the additional food and the fact that a preliminary analysis suggested there was no effect of a transition from +1 to 0 or from 0 to +1 on responding during extinction, we collapsed across those conditions for analysis of extinction data. The first eight sessions of extinction were used for consistency in analysis for all subjects and because responding in the later extinction sessions was typically near zero responses. Using the mean of the last five baseline sessions preceding each extinction test, log proportion of baseline was calculated for each extinction session and collapsed into two-session blocks (shown in Figure 1). Following previous BMT research (see, e.g., Nevin & Grace, 1999, 2000b; Nevin, McLean, & Grace, 2001; Nevin, Grace, Holland, & McLean, 2001) these data were used to calculate slope for analysis (response rate data are presented in Table 2). The resulting slopes were analyzed using a 2 x 2 within-subjects ANOVA with component (rich and lean) and pre-session food presentation (0 or +1) as factors. There was a significant main effect of component, F(1, 7) = 9.4, p =.02, p2 = .57 and a significant main effect of pre-session food presentation, F(1, 7) = 18.4, p =.004, p2 = .72. The interaction was not significant, F(1, 7) = 0.24, p = .64. As Figure 2 illustrates, the rich component was more resistant to change compared to the lean component, as evidenced by smaller negative slopes. Importantly, the figure also shows the main effect of the pre-session food presentation which shows more resistance to change for the conditions where one food presentation was present compared to the conditions where it was absent. 4 Discussion Using a two-component multiple schedule procedure with a rich and lean VI component,
FREE FOOD AND EXTINCTION
12
we replicated previous work showing rate of reinforcement predicts resistance to change in a multiple schedule (e.g. Nevin, 1984; for reviews, see Nevin 1992; Nevin & Grace, 2000a). More importantly, we found that a single pre-session food presentation can have a measurable impact on responding during extinction (but not on baseline responding), particularly in the lean component. Examination of Figure 1 suggests that the additional free food presentation appears to have had a lesser effect on responding in the rich component, which may simply be a reflection of higher rates of responding in the rich component and thus more change required to show an effect. Our finding is consistent with the notion that response-independent food presented outside of a stimulus-reinforcer context can function to arouse responding during extinction performance. This finding is inconsistent with BMT inasmuch as extra-session food presentations are considered to function as behavior disruptors. It is consistent with Killeen et al. (1978) who showed that a single food presentation at the beginning of a daily experimental session functioned to increase overall activity level. Thus, food presentations presented outside of the context of the three-term contingency can function to arouse behavior. While it is possible that BMT could be configured to address this issue, it has not yet been formally incorporated into the model. BMT would have to have some mechanism for determining when these events function as disruptors and when they function to arouse responding. Currently, when response-independent food is presented in the presence of a discriminative stimulus and its associated operant, it is said to enhance response strength (e.g., Nevin et al., 1990) and when it occurs outside of that context, it functions to disrupt responding (e.g., Nevin et al., 1983). The relationship between response-independent food presentations and its effect on responding and the dynamics of response change during extinction are complex, as
FREE FOOD AND EXTINCTION
13
demonstrated in recent research investigating related factors and their effect on responding during extinction (e.g., Podlesnik & Bai, 2015; Podlesnik & Fleet, 2014) and recent modeling of extinction in the context of behavioral momentum theory (e.g., Nevin & Shahan, 2011). Recent research applying BMT to an animal model of drug relapse demonstrated that extinction of a food contingency can function to increase previously-extinguished cocaine-seeking behavior in rats (Quick, Pyszczynski, Colston, & Shahan, 2011). One potential concern is that the present results are simply due to a generalization decrement effect. Given the extinction data were collapsed across the preceding baseline conditions (i.e., those with and without a free food presentation occurring at the beginning of the session), this seems unlikely. Any generalization decrement effects resulting from the presence or absence of the single food presentation should have been minimized or eliminated because of our counterbalanced design (Dulaney, & Bell, 2008; Nevin, McLean, & Grace, 2001 explicitly investigated generalization decrement and resistance to extinction). Our results are related to recent studies examining the effect of additional responseindependent food presentations on the reinstatement of extinguished operant responding. Reinstatement studies present the reinforcer used to maintain responding during baseline training following extinction, typically within the presence of the discriminative stimulus, resulting in the extinguished response reappearing (e.g., Reid, 1958). This paradigm has served as an animal model of relapse in research on substance abuse (for a review, see Shaham, Shalev, Lu, de Wit, & Stewart, 2003). Podlesnik and Shahan (2009; see also Podlesnik & Shahan, 2010), for example, describe a study quite similar to the present experiment. In their baseline condition pigeons were
FREE FOOD AND EXTINCTION
14
presented with a two-component multiple schedule with a rich and a lean component. Following extinction, they reinstated responding by providing two food presentations at the beginning of each component of reinstatement session. In other words, the additional food presentations occurred in the presence of the stimuli that had been correlated with food during baseline training. They found more responding in the rich component compared to the lean component, consistent with BMT. Miranda-Dukoski, Bensemann, and Podlesnik (2016) replicated the basic procedure and finding of Podlesnik and Shahan (2009) using a four-component multiple schedule. During baseline training, two target components, one rich and one lean, reinforced responding while two similar components were associated with extinction. Two reinstatement tests followed extinction where three 2-s food presentations occurred. In one reinstatement test, those food presentations occurred in the presence of the target component stimuli (i.e., where responding was reinforced). In the other test, the food presentations occurred in the presence of the stimuli that had always been correlated with extinction. They found generally larger recovery of responding (relapse) when the food presentations occurred in the presence of the target component stimuli compared to the presentations in the presence of the extinction-related stimuli and, moreover, more recovery of responding for the rich component compared to the lean. When the food presentations occurred in the presence of the extinction-related stimuli, no differences in recovery of responding were seen. This finding contrasts with our findings but this is not particularly surprising, given the numerous procedural differences. It is difficult to know from the present study what variables are responsible for the different result. One issue warranting further investigation is predicting when free food presentations
FREE FOOD AND EXTINCTION
15
function to arouse responding and when they disrupt responding, as this has important implications for substance abuse relapse. Related to that is determining the function of responseindependent food presentations when they are in the presence or absence of discriminative stimuli. In a related study, Podlesnik and Jimenez-Gomez (2016) reported a disruption test where free food was presented in a cup within the operant condition apparatus (contrafreeloading; see Osborne, 1977, for a review) and compared it to the more typical disruption by pre-session feeding. In both conditions the food functioned to disrupt responding in a fashion consistent with BMT – i.e., more persistent responding in the presence of the stimulus correlated with a richer rate of reinforcement compared to a stimulus correlated with a leaner reinforcement rate regardless of the context in which the food appeared (i.e., in contrafreeloading it was available within the context of the discriminated operant and in prefeeding it was presented prior to the experimental session). This study represents a similar attempt to investigate the effect of response-independent food on responding when it is presented within and outside of the context of the discriminated operant. 5 Conclusion We demonstrated that a single response-independent food presentation occurring 30-s prior to the presentation of the first component in a two-component multiple schedule can have a measurable effect on responding during extinction. Our findings are consistent with the food presentation having an arousal function, similar to what was reported by Killeen et al. (1978). They are inconsistent with the disruption role BMT assigns food presentations occurring outside of the context of the stimulus being assessed for resistance to change.
FREE FOOD AND EXTINCTION
16 References
Baum, W. M. (2012a). Extinction as discrimination: The molar view. Behavioural Processes, 90, 101-110. doi:10.1016/j.beproc.2012.02.011 Baum, W. M. (2012b). Rethinking reinforcement: Allocation, induction, and contingency. Journal of the Experimental Analysis of Behavior, 97, 101-124. doi:10.1901/jeab.2012.97-101 Baum, W. M. (2015). The role of induction in operant schedule performance. Behavioural Processes, 114, 26-33. doi:10.1016/j.beproc.2015.01.006 Bell, M. C. (1999). Pavlovian contingencies and resistance to change in a multiple schedule. Journal of the Experimental Analysis of Behavior, 72, 81-96. Berry, M. S., Sweeney, M. M., & Odum, A. L. (2014). Effects of baseline reinforcement rate on operant ABA and ABC renewal. Behavioural Processes, 108, 87-93. doi:10.1016/j.beproc.2014.09.009 Dulaney, A. E., & Bell, M. C. (2008). Resistance to extinction, generalization decrement, and conditioned reinforcement. Behavioural Processes, 78, 253-258. doi:10.1016/j.beproc.2007.12.006 Fleshler, M., & Hoffman, H. S. (1962). A progression for generating variable-interval schedules. Journal of the Experimental Analysis of Behavior, 5, 529-530. doi:10.1901/jeab.1962.5-529 Grace, R. C., Schwendiman, J. W., & Nevin, J. A. (1998). Effects of unsignaled delay of reinforcement on preference and resistance to change. Journal of the Experimental Analysis of Behavior, 69, 247–261.
FREE FOOD AND EXTINCTION
17
Herrnstein, R. J. (1970). On the law of effect. Journal of the Experimental Analysis of Behavior, 13, 243-266. doi:10.1901/jeab.1970.13-243 Killeen, P. R. (1981). Incentive theory. Nebraska Symposium on Motivation, 29, 169-216. Killeen, P. R. (1994). Mathematical principles of reinforcement. Behavioral and Brain Sciences, 17, 105–172. Killeen, P. R., & Sitomer, M. T. (2003). MPR. Behavioural Processes, 62, 49-64. doi:10.1016/S0376-6357(03)00017-2 Killeen, P. R., Hanson, S. J., & Osborne, S. R. (1978). Arousal: Its genesis and manifestation as response rate. Psychological Review, 85, 571-581. Kuroda, T., Cançado, C. X., & Podlesnik, C. A. (2016). Resistance to change and resurgence in humans engaging in a computer task. Behavioural Processes, 125, 1-5. doi:10.1016/j.beproc.2016.01.010 Miranda-Dukoski, L., Bensemann, J., & Podlesnik, C. A. (2016). Training reinforcement rates, resistance to extinction, and the role of context in reinstatement. Learning & Behavior, 44, 29-48. doi:10.3758/s13420-015-0188-8 Nevin, J. A. (1974). Response strength in multiple schedules. Journal of the Experimental Analysis of Behavior, 21, 389-408. doi:10.1901/jeab.1974.21-389 Nevin, J. A. (1984). Pavlovian determiners of behavioral momentum. Animal Learning & Behavior, 12, 363-370. Nevin, J. A. (1992). An integrative model for the study of behavioral momentum. Journal of the Experimental Analysis of Behavior, 57, 301-316.
FREE FOOD AND EXTINCTION
18
Nevin, J. A. (2003). Mathematical principles of reinforcement and resistance to change. Behavioural Processes, 62, 65-73. doi:10.1016/S0376-6357(03)00018-4 Nevin, J. A., & Grace, R. C. (1999). Does the context of reinforcement affect resistance to change?. Journal of Experimental Psychology: Animal Behavior Processes, 25, 256-268. doi:10.1037/0097-7403.25.2.256 Nevin, J. A., & Grace, R. C. (2000a). Behavioral momentum and the Law of Effect. Behavioral and Brain Sciences, 23, 73-130. Nevin, J. A., & Grace, R. C. (2000b). Preference and resistance to change with constant-duration schedule components. Journal of the Experimental Analysis of Behavior, 74, 79-100. doi:10.1901/jeab.2000.74-79 Nevin, J. A., & Shahan, T. A. (2011). Behavioral momentum theory: Equations and applications. Journal of Applied Behavior Analysis, 44, 877-895. doi:10.1901/jaba.2011.44-877 Nevin, J. A., Grace, R. C., Holland, S., & McLean, A. P. (2001). Variable-ratio versus variableinterval schedules: Response rate, resistance to change, and preference. Journal of the Experimental Analysis of Behavior, 76, 43-74. doi:10.1901/jeab.2001.76-43 Nevin, J. A., Mandell, C., & Atak, J. R. (1983). The analysis of behavioral momentum. Journal of the Experimental Analysis of Behavior, 39, 49-59. Nevin, J. A., McLean, A. P., & Grace, R. C. (2001). Resistance to extinction: Contingency termination and generalization decrement. Animal Learning & Behavior, 29, 176-191. doi:10.3758/BF03192826
FREE FOOD AND EXTINCTION
19
Nevin, J. A., Tota, M. E., Torquato, R. D., & Shull, R. L. (1990). Alternative reinforcement increases resistance to change: Pavlovian or operant contingencies?. Journal of the Experimental Analysis of Behavior, 53, 359-379. doi:10.1901/jeab.1990.53-359 Osborne, S. R. (1977). The free food (contrafreeloading) phenomenon: A review and analysis. Animal Learning & Behavior, 5, 221-235. doi:10.3758/BF03209232 Podlesnik, C. A., & Bai, J. H. (2015). Method of stimulus combination impacts resistance to extinction. Journal of the Experimental Analysis of Behavior, 104, 30-47. doi:10.1002/jeab.155 Podlesnik, C. A., & Fleet, J. D. (2014). Signaling added response‐independent reinforcement to assess Pavlovian processes in resistance to change and relapse. Journal of the Experimental Analysis of Behavior, 102, 179-197. doi:10.1002/jeab.96 Podlesnik, C. A., & Jimenez-Gomez, C. (2016). Contrafreeloading, reinforcement rate, and behavioral momentum. Behavioural Processes, 128, 24-28. http://dx.doi.org/10.1016/j.beproc.2016.03.022 Podlesnik, C. A., & Shahan, T. A. (2009). Behavioral momentum and relapse of extinguished operant responding. Learning & Behavior, 37, 357-364. doi:10.3758/LB.37.4.357 Podlesnik, C. A., & Shahan, T. A. (2010). Extinction, relapse, and behavioral momentum. Behavioural Processes, 84, 400-411. doi:10.1016/j.beproc.2010.02.001 Premack, D. (1959). Toward empirical behavior laws: I. Positive reinforcement. Psychological Review, 66, 219-233. doi:10.1037/h0040891
FREE FOOD AND EXTINCTION
20
Quick, S. L., Pyszczynski, A. D., Colston, K. A., & Shahan, T. A. (2011). Loss of alternative non-drug reinforcement induces relapse of cocaine-seeking in rats: Role of dopamine D₁ receptors.Neuropsychopharmacology, 36, 1015-1020. doi:10.1038/npp.2010.239 Reid, R. L. (1958). The role of the reinforcer as a stimulus. British Journal of Psychology, 49, 202-209. doi:10.1111/j.2044-8295.1958.tb00658.x Shaham Y., Shalev U., Lu L., de Wit H., & Stewart J. (2003). The reinstatement model of drug relapse: History, methodology, and major findings. Psychopharmacology, 168, 3–20. Skinner, B. F. (1938). The behavior of organisms: An experimental analysis. Cambridge, MA: Appleton-Century-Crofts. Timberlake, W., & Allison, J. (1974). Response deprivation: An empirical approach to instrumental performance. Psychological Review, 81, 146-164. doi:10.1037/h0036101
FREE FOOD AND EXTINCTION
21
Table 1. Number of sessions for each condition and order of conditions for the one food presentations. The labels represent which baseline and extinction conditions were in effect, with “0” indicating no free-food presentation occurred for that part of the study and “+1” indicating that one food presentation did occur at the beginning of each session. For example, 0/0 indicates that both the baseline and extinction had no pre-session food presentations whereas +1/0 indicates that the baseline included a single food presentation at the beginning of the session and the subsequent extinction condition did not.
Bird G260, G264
G263, G265
G251, G255
G253, G268
One Food Baseline Extinction
0/0 37 10
0/+1 37 10
+1/+1 29 8
+1/0 29 8
One Food Baseline Extinction
0/+1 13 8
0/0 13 8
+1/0 10 8
+1/+1 10 8
One Food Baseline Extinction
+1/+1 10 10
+1/0 10 10
0/0 22 11
0/+1 22 11
One Food Baseline Extinction
+1/0 10 8
+1/+1 10 8
0/+1 10 9
0/0 10 9
Condition 1
Condition 2
Condition 3
Condition 4
FREE FOOD AND EXTINCTION
22
Table 2. Response rates (responses per min) for each block, including baseline (B) M (SD) of the last five sessions and extinction responding represented as blocks of two-session for the rich (VI 45-s) and lean (VI 180-s) components. Condition labels indicate which baseline and extinction conditions were in effect. A “0” indicates no free-food presentation occurred for that part of the study and “+1” indicates that one food presentation did occur at the beginning of each session. For example, 0/0 indicates that both the baseline and extinction had no pre-session food presentations and +1/0 indicates that the baseline included a single food presentation at the beginning of the session and the subsequent extinction condition did not.
NOTE THAT THE TABLE WAS UPLOADED SEPARATELY
FREE FOOD AND EXTINCTION
23 Figure Captions
Figure 1. Resistance to change data during extinction across session blocks as log proportion of baseline response rates for individual subjects. Mean responding is also presented in the bottom right graph. Data are shown as a function of rich and lean components and the presence (+1) or absence (0) of a single food presentation occurring 30-s prior to the beginning of the first component. Figure 2. Mean slope during extinction for the rich (left panel) and lean (right panel) components as a function of whether food was never presented during extinction (0) or if one food presentation occurred 30-s prior to the first component (+1). Error bars show the standard error of the mean.
Table 2. Response rates (responses per min) for each block, including baseline (B) M (SD) of the last five sessions and extinction responding represented as blocks of two-session for the rich (VI 45-s) and lean (VI 180-s) components. Condition labels indicate which baseline and extinction conditions were in effect. A “0” indicates no free-food presentation occurred for that part of the study and “+1” indicates that one food presentation did occur at the beginning of each session. For example, 0/0 indicates that both the baseline and extinction had no pre-session food presentations and +1/0 indicates that the baseline included a single food presentation at the beginning of the session and the subsequent extinction condition did not.
G260 Condition Block VI 45-s
VI 180-s
G264 VI 45-s
74.5 (5.9) 81.9 75.6 78.4 17.5
VI 180-s
G263 VI 45-s
VI 180-s
68 (7.5) 31.0 15.0 0.3 3.9
177.6 (9.3) 172.1 47.7 135.2 2.8
162.5 (42) 51.5 14.3 38.9 2.4
0/0 `
B 1 2 3 4
46.7 (4.4) 26.3 (16.2) 58.0 12.2 28.7 8.9 6.0 0.0 18.5 0.1
0/+1 B 1 2 3 4
29.8 (8) 26.1 12.5 4.5 9.2
12.2 (5.3) 3.7 4.7 6.3 0.0
67.9 (3.7) 60.1 (22.4) 62.9 39.7 34.4 21.0 31.4 8.9 22.2 13.3
186.4 (9.4) 121.9 148.3 112.6 19.1
123.4 (32) 56.5 22.4 9.7 4.7
B 1 2 3 4
43.5 (5.6) 22 (10.4) 16.7 9.8 4.4 1.1 0.4 0.0 1.3 0.2
68 (3.5) 55.3 (15.9) 29.7 19.1 29.7 12.1 31.4 4.1 4.8 4.3
183.7 (7.1) 67.3 68.7 13.1 36.4
206 (10.8) 37.6 86.9 1.5 5.6
B 1 2 3 4
49.3 (5.3) 25.3 (5.6) 20.1 9.0 10.3 6.0 1.8 3.7 9.1 0.0
70.9 (3.3) 31.6 (19.3) 51.9 16.2 21.7 10.5 34.3 6.0 9.2 3.9
+1/0
+1/+1 174.1 (10.6) 147.8 (46.8) 61.5 27.2 22.7 28.1 29.3 31.5 23.7 3.4
G265 VI 45-s
VI 180-s
G251 VI 45-s
VI 180-s
G255 VI 45-s
VI 180-s
G253 VI 45-s
104.1 (7.6) 47.7 (10.1) 85.2 14.9 21.9 3.5 2.6 3.1 6.3 5.2
68.1 (3.4) 27.1 (6.3) 53.5 11.5 18.7 10.7 11.4 4.3 17.5 2.8
31.7 (2.9) 11.4 (4.3) 22.0 6.8 10.3 3.3 3.8 0.7 7.5 1.0
32.7 (3.1) 32.8 1.5 12.9 0.6
107.5 (25.9) 55.6 (20) 77.6 43.8 92.5 24.9 63.3 19.4 20.6 4.1
63.6 (7.1) 29.4 (4.7) 46.4 15.4 19.8 7.3 13.6 4.3 17.9 6.2
33.8 (2.2) 13.5 (3.4) 25.8 6.3 14.7 5.4 10.0 2.9 7.4 1.4
38.7 (25) 24.5 1.8 0.1 4.7
33.2 (7.8) 4.9 3.1 2.5 3.4
30.3 (4) 16.4 (2.3) 22.0 8.8 8.3 1.7 3.6 0.3 6.9 0.3
32.9 (4) 34.3 4.4 1.7 6.0
78.8 (8.8) 28.7 (10.9) 47.2 14.9 31.4 6.5 13.0 2.0 3.0 0.6
28.4 (3) 18.6 (4.9) 28.9 11.9 15.3 8.0 9.1 0.6 10.3 1.5
45.7 (5.1) 33.9 5.0 0.3 16.5
118.2 (6) 60.2 36.9 42.0 17.1
34.6 (16.3) 24.5 9.3 6.2 6.1
115.3 (1.3) 54.4 (15.1) 55.8 10.3 14.7 14.5 26.1 1.2 11.1 3.5
71.9 (6) 49.4 12.5 11.2 10.2
VI 180-s
G268 VI 45-s
VI 180-s
19 (9.7) 1.5 0.3 1.1 0.6
63.1 (4.2) 44.7 (12.1) 71.6 13.9 81.4 2.2 30.9 1.7 22.7 0.2
4.7 (3.6) 0.5 0.6 0.1 0.2
71.3 (8.2) 84.3 38.4 52.8 27.9
39.6 (9.9) 5.6 1.9 4.7 2.2
10.7 (5.9) 6.5 0.1 0.0 0.3
71.9 (6.2) 77.7 82.6 66.1 20.2
12.2 (7) 1.0 0.4 0.4 0.4
8.2 (4.7) 4.0 0.9 0.5 0.7
74.9 (11.8) 28 (11.5) 80.8 5.6 71.8 0.4 46.2 1.6 39.0 2.0