- Email: [email protected]
Investigations of timing during the schedule and reinforcement intervals with wheel-running reinforcement
Behavioural Processes 73 (2006) 240–247
Investigations of timing during the schedule and reinforcement intervals with wheel-running reinforcement Terry W. Belke ∗ , Melissa M. Christie-Fougere Mount Allison University, Sackville, New Brunswick, Canada E4L 1C7 Received 25 May 2006; accepted 2 June 2006
Abstract Across two experiments, a peak procedure was used to assess the timing of the onset and offset of an opportunity to run as a reinforcer. The first experiment investigated the effect of reinforcer duration on temporal discrimination of the onset of the reinforcement interval. Three male Wistar rats were exposed to fixed-interval (FI) 30-s schedules of wheel-running reinforcement and the duration of the opportunity to run was varied across values of 15, 30, and 60 s. Each session consisted of 50 reinforcers and 10 probe trials. Results showed that as reinforcer duration increased, the percentage of postreinforcement pauses longer than the 30-s schedule interval increased. On probe trials, peak response rates occurred near the time of reinforcer delivery and peak times varied with reinforcer duration. In a second experiment, seven female Long-Evans rats were exposed to FI 30-s schedules leading to 30-s opportunities to run. Timing of the onset and offset of the reinforcement period was assessed by probe trials during the schedule interval and during the reinforcement interval in separate conditions. The results provided evidence of timing of the onset, but not the offset of the wheel-running reinforcement period. Further research is required to assess if timing occurs during a wheel-running reinforcement period. © 2006 Elsevier B.V. All rights reserved. Keywords: Wheel-running reinforcement; Timing; Peak procedure; Fixed interval; Lever pressing; Rat
With wheel-running reinforcement, completion of a schedule requirement provides access to the opportunity to run for a period of time (e.g., Belke and Heyman, 1994; Collier and Hirsch, 1971; Kagan and Berkun, 1954; Iversen, 1993; Pierce et al., 1986; Premack et al., 1964). When the requirement is a fixed-interval (FI) schedule and the period of access to the opportunity to run is constant, the procedure provides an opportunity to assess temporal discrimination of the onset and the offset of a wheel-running reinforcement period. The present study reports on two experiments that investigated the role of timing with wheel-running reinforcement. 1. Experiment 1 In the first experiment, the effect of wheel-running reinforcer duration on temporal discrimination of the onset of the wheel-running period was investigated. Following the termination of a wheel-running reinforcer, an animal typically pauses ∗
Corresponding author. E-mail address: [email protected] (T.W. Belke).
0376-6357/$ – see front matter © 2006 Elsevier B.V. All rights reserved. doi:10.1016/j.beproc.2006.06.004
for a period of time before pressing a lever. This postreinforcement pause (PRP) varies with the duration of the wheel-running reinforcement period (Belke, 1997; Belke and Dunbar, 1998; Belke and Hancock, 2003) and is longer than typically observed with more conventional reinforcers such as a small bit of food or water (Belke, 2000). Furthermore, with conventional reinforcers, pause duration shortens appreciably when the schedule requirement, either ratio or interval, is changed from fixed to variable. This suggests that the pause duration is largely a function of the reinforcement schedule. In contrast, with wheelrunning reinforcement the PRP duration remains long even when the schedule requirement is variable (Belke, 1996, 1997). PRP duration with wheel-running reinforcement appears to be less a function of the schedule than of the reinforcer (Belke and Dunbar, 1998). One implication of this difference is that if the duration of the pause is typically longer than the duration of the interval of the schedule (IS), then the IS will not be experienced as programmed. At the extreme, the schedule will be experienced as a continuous reinforcement schedule rather than as an intermittent schedule of reinforcement (Belke and Dunbar, 1998). For this reason, response-initiated schedules (e.g., tandem fixed ratio
T.W. Belke, M.M. Christie-Fougere / Behavioural Processes 73 (2006) 240–247
[FR] 1 variable interval [VI] 30-s schedule) have been used to ensure that the animal experiences the programmed intervals (Belke, 1996, 1997; Belke and Heyman, 1994). On standard FI schedules, as the duration of the PRP following wheel running increases or the duration of the schedule value decreases, the percentage of intervals experienced as terminating at a time after the schedule value has elapsed should increase (Belke and Dunbar, 1998). For example, on a FI schedule with a 60-s wheelrunning reinforcement interval, 88% of PRPs were longer than the IS when the IS was 15 s and 73% were longer when the IS was 30 s. To the extent that fewer intervals are experienced as terminating when the IS elapses, discrimination of the time when the reinforcer becomes available may be affected. That is, if the majority of reinforcers occur following postreinforcement pauses of variable durations, all longer than the IS, then the animal will not be experiencing the periodicity of delivery of the reinforcer that underlies discrimination of IS. Consequently, the objective of the first experiment was to assess temporal discrimination of the onset of the wheel-running reinforcement period with different reinforcer durations using a peak procedure. The peak procedure, developed by Catania (1970) and extensively utilized by Roberts (1981) to investigate temporal discrimination in animals, involves an FI schedule with reinforced and non-reinforced trials. On FI schedules, responding is low early and accelerates as the IS elapses so that the highest rate of responding occurs just prior to the onset of the reinforcer. On reinforced trials, the first response after the IS requirement has lapsed produces the reinforcer. On non-reinforced trials, also known as probe trials, the interval extends beyond the end of the IS and does not terminate with a reinforcer. On probe trials, responding generally increases up to the time when reinforcement would have occurred and decreases as the interval progresses beyond the time when reinforcement would have occurred. The result, when averaged over many probe trials, is a Gaussian curve centered on the time, at or near, when the reinforcer would typically occur. 1.1. Method 1.1.1. Subjects Four male Wistar rats obtained from Charles River Breeding Laboratories, St. Constant, Quebec served as subjects. At the start of the experiment, the rats were approximately 7 months old and experimentally naive. They were individually housed in standard polycarbonate cages (48 cm × 27 cm × 22 cm) in a colony room maintained at 20 ◦ C with a 12-h light/dark cycle (lights on at 07:00). For the duration of the experiment, animals were maintained at a target weight of 335 ± 10 g. Distilled water was freely available in their home cages. One rat fell ill shortly after completing the first experimental condition and was removed from the study. Data from this rat were not included in the analyses in Section 1.2. 1.1.2. Apparatus Experimental sessions of responding on levers for the opportunity to run occurred in activity wheels (1 Wahmann and 3 Lafayette Instruments Model #86041 A) without side cages. The
241
diameters of the wheels were 35.5 cm. Each wheel was located in a sound-proof shell equipped with a fan for ventilation and to mask extraneous noise. A retractable lever (Med Associates ENV-112) was mounted directly at the opening of each wheel. The lever extended 1.8 cm into the wheel through an opening (7 cm × 9 cm) in the center at the base of the wheel frame. A microswitch attached to the wheel frame recorded wheel revolutions. The force required to close the lever microswitches ranged between 18 and 27 g. Bulbs (24-V dc) mounted on the sides of the wheel frame served to illuminate the inside of the wheel. A solenoid-operated brake was attached to the base of each wheel. When the solenoid was operated, a rubber tip attached to a metal shaft contacted the outer rim of the wheel and brought the wheel to a stop. Control of experimental events and recording of data were handled by IBM® personal computers interfaced to the wheels. 1.1.3. Procedure Rats used in this experiment were selected from a larger group of animals that were initially trained to lever press for the opportunity to run and then distributed to several experimental procedures. This training began by providing the animals with the opportunity to run freely in a running wheel for 30 min each day. Sessions occurred at the same time of day for 15 days. The number of wheel revolutions was recorded for each rat on each day. After 15 days, the highest rate runners were selected for further training. In the next phase the rats continued to receive 30-min access to the free-moving running wheel at the same time each day. Following the session of wheel running each rat was placed in a standard operant conditioning chamber and lever pressing was shaped by reinforcing successive approximations. Each lever press produced 0.1 ml of a 15% sucrose solution. When subjects reliably pressed the lever, the schedule of reinforcement was shifted from requiring only a single response per reinforcer (FR 1) to one requiring a variable number of responses averaging 3 (i.e., a variable-ratio [VR] schedule). This schedule remained in effect for four sessions, with each session terminating when 50 sucrose reinforcers were obtained. After four sessions on the VR 3 schedule, sessions in the operant conditioning chamber were discontinued. At this point, the retractable lever in each wheel chamber was extended during the wheel-running sessions and the opportunity to run for 60 s was contingent upon a single lever press. Retraction of the lever and movement of the wheel with the release of the brake signaled access to the running period. Each session consisted of 30 opportunities to run. During each session, the lights at the side of the wheel were illuminated. The schedule of reinforcement was changed in the following sequence: FR 1, VR 3, VR 5, and VR 9. Subjects remained on each schedule for four sessions before advancing to the next schedule. Following this initial training of a larger group of rats, four rats were assigned to the current study, one of which subsequently died due to health problems. The rats were placed on a standard FI 30-s schedule. Each reinforcer consisted of a 30s opportunity to run. Each session ended when 60 reinforcers were completed. Sessions were run 7 days a week. After 60 sessions, reinforcer duration was changed and probe sessions began.
242
T.W. Belke, M.M. Christie-Fougere / Behavioural Processes 73 (2006) 240–247
Reinforcer durations were 15, 30, and 60 s and each animal experienced a different order of durations. The duration orders for IB13, 16, and 17 were 30–60–15, 60–30–15, 15–30–60 s, respectively. Each reinforcer duration was in effect for 40 probe sessions. After this fixed number of sessions, reinforcer duration was changed. During a probe session, probe trials did not occur during the first 10 reinforcers. Following the first 10 reinforcers, probe trials occurred during 2 out of each successive set of 10 reinforcers that occurred. In total, probe trials occurred in 10 of the last 50 reinforcers of a session. The occurrence of a probe trial was determined by random selection from a list of numbers. During a probe trial, the duration of the interval was extended to 90 s plus a variable duration varying between 5 and 25 s. The addition of this variable time period ensured that the probe duration was variable. Following the termination of a probe, the lights located at the side of the wheel were extinguished and the retractable lever was retracted. After 5 s, the lights were illuminated and the retractable lever was extended. Since a probe trial did not terminate with an opportunity to run, the 5-s blackout period and retraction of the lever were necessary to signal the initiation of the next schedule interval. The present procedure differed from the typical peak procedure in that no intertrial interval (ITI) was programmed to occur following the occurrence of a reinforcer during FI trials. With respect to this particular experiment, inclusion of an ITI following the termination of a wheel-running reinforcer would have negated the effect of the long PRP that follows a wheel-running reinforcer on responding during the subsequent fixed interval. The objective of this experiment was to assess the effect of this long PRP on temporal discrimination of the IS. Lever presses, time spent lever pressing, and PRP duration were recorded for each reinforcement and collectively for the entire session. Within the 30-s schedule interval, lever presses were recorded cumulatively in successive 5-s segments across the schedule intervals leading to reinforcement within a session. During probes, lever presses were recorded in successive 5-s intervals cumulatively throughout 90-s period that was common to all probes to produce one set of probe values for each session. Data were averaged over the last 10 sessions in each condition for analysis. Standard errors were calculated over values from these 10 sessions. Peak time was calculated for each response rate function for the probes from each session based on iterative calculations of medians over decreasing intervals (Roberts, 1981). First, the median was determined over the range from 0 to 60-s. This is the range over which responding would be expected to be centered if the animals had discriminated the 30s IS. The value of this median determines the range over which the next median was calculated. If the median was less than 30 s, the lower limit of the new range was 0 s and the upper limit was twice the value of the median. If the median was greater than 30 s, then the upper limit was 60 s and the lower limit was twice the difference between the median greater than 30 s and 30 s. This procedure was repeated across successive determinations of medians until it produced a median that was within 0.5 s of the previous median. The peak time for each reinforcer dura-
Table 1 Median postreinforcement pause durations (s) and percentage of postreinforcement pauses (PRPs) greater than the schedule interval (30 s) for each animal Rat
Median
Percent of PRPs > 30 s
15
30
60
15
30
60
IB13 IB16 IB17
24.8 23.6 25.7
26.1 29.2 24.5
29.4 34.8 30.5
35.9 32.4 38.0
37.1 48.8 32.1
48.3 57.3 51.3
Mean
24.7
26.6
31.6
35.4
39.3
52.3
tion condition was the average of the peak times across the 10 sessions. 1.2. Results Table 1 shows the median PRPs and percentage of PRPs greater than the schedule value for each animal. Median PRPs were used because the median is less susceptible to outlier values than is the mean. As reinforcer duration increased, the median PRP duration and the percentage of pauses greater than the IS increased. A repeated measures ANOVA showed a significant effect of reinforcer duration on median PRPs, F(2, 4) = 8.07, p = 0.04. Paired t-test comparisons showed that the percentage of intervals with pauses greater than the IS was significantly higher when the reinforcer duration was 60 s than when it was 15 s (t(2) = −3.99, p = 0.03, one tailed) or 30 s (t(2) = −3.86, p = 0.03, one tailed). Fig. 1 shows mean response rates (presses/min) in successive 5-s segments of the 90-s probes for each wheel-running
Fig. 1. Response rates (presses/min) in successive 5-s intervals within the 90-s probe interval for the 15, 30, and 60-s wheel-running reinforcer duration conditions for each rat and the group. Standard error bars are provided for each mean. The stippled line indicates the time of reinforcement delivery on the FI 30-s schedule.
T.W. Belke, M.M. Christie-Fougere / Behavioural Processes 73 (2006) 240–247
243
Table 2 Average peak times (s) and standard errors for each reinforcer duration condition for each animal Rat
Duration 15
30
60
IB13 IB16 IB17
34.9 (1.1) 35.6 (0.6) 37.6 (0.8)
38.2 (0.8) 38.4 (1.3) 37.3 (1.0)
38.7 (1.7) 40.2 (1.3) 41.1 (1.6)
Mean
36.0 (0.8)
38.0 (0.3)
40.0 (0.7)
reinforcer duration for each animal and the group. Visual inspection of this figure suggests that lever-pressing rates increased toward the time when the wheel-running reinforcer was scheduled to occur and then decreased at a time near or past when the reinforcer was to occur. For the 15-s reinforcer condition, the highest lever-pressing rates occurred in the 25–30, 30–35, and 35–40s segments for IB13, 16, and 17, respectively. For the 30-s reinforcer condition, the highest lever-pressing rates occurred in the 30–35, 35–40, and 35–40s segments. For the 60-s reinforcer condition, the highest lever-pressing rates occurred in the 30–35, and 35–40s segments for IB13 and 17. For IB16, the highest lever-pressing rate occurred in the 35–40 and 45–50s segments. On average, the highest lever-pressing rates occurred in the 31–35s segment of the 90-s probe in the 15-s reinforcer duration condition and the 36–40s segment in the 30 and 60-s conditions. Table 2 shows average peak times across probes for each reinforcer duration for each animal. In general peak times corresponded with the intervals with the highest average leverpressing rates, but were higher in some instances due to variability across probes generated within each session. As suggested by Fig. 1, as wheel-running reinforcer duration increased, average peak time increased. Due to the lack of power of non-parametric analyses to detect differences with such a small number of observations, peak times were analyzed using a parametric analysis. A repeated measures ANOVA revealed a significant effect of reinforcer duration, F(2, 4) = 10.28, p = 0.03. Fig. 2 shows lever-pressing rates during the IS for the nonprobe trials (i.e., reinforcer occurred) in the left panel and wheel-running rates during the interval of reinforcement (IR) over all trials in the right panel for each duration condition for each animal and the group. The left panels show that, on average, as reinforcer and PRP duration increased, responding increased less rapidly as the interval progressed. As a result, the highest rate of responding in the interval closest to reinforcement decreased as the IR increased. Mean lever-pressing rates for the 15, 30, and 60-s reinforcer durations were 19.71, 15.96, and 11.68 responses/min. This decrease in responding as a function of reinforcer duration reflects a shift in the initiation of responding to a later time as PRP duration increased. It is also consistent with the decrease in responding observed in the probes. The right panels show that within the IR, wheelrunning rates were higher at the onset and declined as the IR elapsed toward termination. The potential role of temporal discrimination of the offset of the IR with respect to producing this
Fig. 2. Lever-pressing rates (presses/min) in successive 5-s intervals within the FI 30-s schedule in the left panels and wheel-running rates (rpm) in successive 5-s intervals within the reinforcement period in the right panels for the 15, 30, and 60-s wheel-running reinforcer duration conditions for each rat and the group. Standard error bars are provided for each mean.
pattern of running within the IR will be the focus of the next experiment. 1.3. Discussion Consistent with previous investigations using other reinforcers (Roberts, 1981, 1982), rats in the present study appeared to time the onset of an opportunity to run. Responding on probe trials peaked at or near the time when the opportunity to run would have occurred. As the duration of the wheel-running reinforcer period increased, postreinforcement pause durations increased. At the longest duration animals experienced just less than half of the schedule intervals as terminating with a reinforcer 30 s after the end of the previous reinforcer. Changes in
244
T.W. Belke, M.M. Christie-Fougere / Behavioural Processes 73 (2006) 240–247
response rate functions coupled with the analysis of peak times suggest that as reinforcer duration increased, there was a small but systematic shift in peak times during the probes. Thus, perception of the onset of reinforcement may have been affected by reinforcer duration; however, due to the small number of observations, this conclusion should be regarded as tentative. It is worth noting that this effect of lengthening PRPs on responding during a subsequent IS can be attenuated or eliminated through the use of an intertrial interval that is typically incorporated into the peak procedure. 2. Experiment 2 Experiment 1 demonstrated that, as with other reinforcers, rats discriminate the onset of an opportunity to run and that the long PRP that follows running may affect discrimination of the time of onset. Experiment 2 investigated the role of timing during the IR period. During the IS, discrimination of the time when a reinforcer occurs governs the pattern of responding. Immediately after a reinforcer, responding is absent or low. As time elapses toward when reinforcement becomes available, responding accelerates and is highest in the interval immediately preceding the time when the interval elapses. During the IR, there is also a pattern of running. Following the onset of an opportunity to run, wheel running is highest and decreases as the interval progresses. Decreases in running are greater earlier in the interval (see Fig. 2). This pattern of running within the reinforcement interval (IR, interval of reinforcement) raises the possibility that the rats are timing the IR and that their temporal discrimination of the IR may influence running during the IR (Belke and McLaughlin, 2005). Experiment 1 provided evidence that rats discriminate the time of onset of an opportunity to run. The objective of the current experiment was to determine if rats also discriminate the offset of a brief opportunity to run. To do this, rats responding on a FI 30-s schedule of wheel-running reinforcement with a 30-s IR were exposed to extended probe intervals during the IS and IR to assess temporal discrimination of the onset and offset of an opportunity to run. 2.1. Method 2.1.1. Subjects Seven female Long-Evans rats obtained from Charles River Breeding Laboratories, St. Constant, Quebec served as subjects. In the interim between Experiment 1 and the current study, the laboratory switched to using female rats to study the reinforcing properties of wheel running. Female rats run at higher rates than do male rats and, presumably, find wheel running a more efficacious reinforcer. This difference negates the need to select rats to be trained to lever press for the opportunity to run based on wheel running rates during the initial training phase. At the start of the experiment, the rats were approximately 18 months old and had participated in previous wheel-running reinforcement studies. For the duration of the experiment, the animals were maintained at a target weight of 260 ± 10 g. All other conditions were as stated in Section 1.1.1.
2.1.2. Apparatus The apparatus used in Experiment 1 was used in Experiment 2. 2.1.3. Procedure Training the rats to press a lever for the opportunity to run followed the same steps described in Experiment 1 with the exception that the rats were not trained to press a lever in a standard operant conditioning chamber using sucrose solution as a reinforcer. After being exposed to the opportunity to run continuously for 30 min each day, the lever was extended and the opportunity to run for 60 s was contingent upon a single lever press. All rats learned to lever press within one or two sessions. Prior to participating in the current study, the subjects participated in other operant investigations of wheel-running reinforcement, the most recent of which involved an FI 30-s schedule with a 45-s reinforcer. In preparation for the current procedure, all subjects were exposed to a baseline condition for 20 sessions. In the baseline condition, the schedule of wheel-running reinforcement was a FI 30-s schedule and the consequence was the opportunity to run for 30 s. Each session terminated when 50 reinforcers were completed. Following the baseline condition, timing during the IS was assessed in three rats and during the IR in four rats. After 20 sessions, all rats were returned to the baseline condition. Following the completion of 20 baseline sessions, timing during the IR was assessed in the 3 rats that had timing assessed during the IS and timing during the IS was assessed in the 4 rats that had timing assessed during the IR. These procedures remained in effect for 20 sessions. Finally, all animals were once again returned to the baseline condition for 20 sessions. Across this set of conditions, timing in both the IS and IR was assessed in all seven subjects. The peak procedure was used to assess timing during the IS. During an IS probe session, probe trials did not occur during the first 10 reinforcers. Following the first 10 reinforcers, probe trials occurred during 2 out of each successive set of 10 reinforcers. In total, probe trials occurred in 8 of the last 40 trials of a session. The occurrence of a probe trial was determined by random selection from a list of numbers. During an IS probe trial, the duration of the interval was extended to 90 s plus a variable duration varying between 5 and 25 s. As in Experiment 1, following the termination of a probe, the lever was retracted and a 5-s blackout period occurred. After 5 s, the lights at the side of the wheel were illuminated and the lever extended to signal the start of the next schedule interval. No ITI was programmed to occur. During sessions with IS probes, wheel-running reinforcer duration remained constant at 30 s. Assessment of timing during the IR involved a similar procedure with the exception that during the last 40 reinforcers of a session 8 probe trials occurred. During an IR probe trial, the duration of the wheel-running reinforcement period was extended to 60 s plus a variable duration varying between 5 and 25 s. The termination of the probe trial and the onset of the next schedule value were signaled by the onset of the brake and the extension of the lever. Again, no ITI was programmed to occur. During sessions with IR probes, the schedule interval remained constant at 30 s.
T.W. Belke, M.M. Christie-Fougere / Behavioural Processes 73 (2006) 240–247
245
After the completion of final set of 20 baseline sessions, all rats were placed on a FI 30-s schedule with a 60-s opportunity to run as a reinforcing consequence. Sessions terminated after 25 reinforcers were completed to equate the total duration of opportunity to run with the time available to run during the baseline condition in which there were 50 reinforcement periods 30 s in duration. This condition remained in effect for 20 sessions. The purpose of this condition was to serve as a control against which wheel running during the 60-s probe could be compared. If wheel running during the 60-s IR probes with a 30-s wheelrunning reinforcer showed evidence of timing, then the pattern of wheel running should differ from that which occurs during a 60-s wheel-running reinforcer. Lever presses, time spent lever pressing, and postreinforcement pause duration were recorded for each reinforcement and collectively for the entire session. During schedule probes, lever presses were recorded in successive 5-s intervals throughout the 90-s period. During reinforcer probes, wheel revolutions were recorded in successive 5-s intervals throughout the 60-s period. Data from the last five sessions in each condition were analyzed. Mean peak times were calculated as previously described in Experiment 1. 2.2. Results Fig. 3 shows mean response rates during successive 5-s segments of the 90-s probe interval for each animal and the group. With the exception of TK 2, the rats show a pattern of responding suggesting the development of temporal discrimination. Responding rises toward a peak and then declines. However, the location of the peak rate was quite variable. The highest rate was in the 30–35 s segment for TK 16, the 35–40 s segment for TK 9 and 15, the 40–45 s segment for TK 1 and 11, and the 45–50 s segment for TK 6. For TK2, responding rose toward the 30–35 s segment, declined in the 35–40 s segment and then continued to rise to a peak rate in the 55–60 s segment. For the group, the highest rate occurred in the 40–45 s segment. With respect to peak times, the mean peak time across all rats was 42.4 s. Fig. 4 shows wheel-running rates (rpm) in successive 5s segments of the 60-s probe interval and the 60-s IR for each animal and the group. In general, running during the 60s probes rises from the first 5 s to a peak during the second 5-s segment and then declines as the interval progresses, leveling out in the latter part of the interval. With the exception of TK 15, there is no evidence of an increase in running after 30 s as would be expected if the decline in running was due to the discrimination that the running period would terminate. For TK 15, running decreased from the second to the fifth 5s segment and then increased from the sixth to the eighth 5-s segment. A repeated measures ANOVA with source (probe, reinforcer) and 5-s interval (1–12) as within-subject variables revealed a significant effect of interval, F(11, 66) = 45.21, p < 0.001, but no effect of source, F(1, 6) = 2.80, p = 0.15, and no interaction, F(11, 66) = 1.69, p = 0.10. Thus, running during the 60-s probe did not differ from running during the 60-s reinforcer. The similarity of
Fig. 3. Response rates (presses/min) in successive 5-s intervals within the 90s probe interval for each rat and the group. Standard error bars are provided for each mean. Mean peak time and standard errors are noted below the rat designation. The stippled line indicates the time of reinforcement delivery on the FI 30-s schedule.
the two patterns does not support the assertion that timing plays a role in the decline in running that occurs within a reinforcement period. 2.3. Discussion As observed in Experiment 1, behavior during the IS leading up to the onset of an opportunity to run showed evidence of temporal discrimination. Responding was lowest following the termination of a reinforcement period and increased as the interval progressed toward the elapse of the schedule when a response would produce the onset of an opportunity to run. Peak response rates were more varied and less well defined than in
246
T.W. Belke, M.M. Christie-Fougere / Behavioural Processes 73 (2006) 240–247
A review of the literature failed to reveal an instance of the peak procedure being used to assess the temporal discrimination of the offset of an interval during which behavior could occur. Consequently, the validity of using the procedure in this manner is open to question. Also, if there is a difference in the motivational value of the onset and the offset of a reinforcer then this may make the procedure more suitable to timing the onset as opposed to the offset of a reinforcement period. Satiation would be one reason for this difference in motivational significance. Consumption of the reinforcer prior to the offset of a reinforcer may make the offset of the reinforcer less salient. Finally, if factors other than timing such as fatigue, satiation, and/or habituation (Belke and McLaughlin, 2005) play a role in determining the pattern of running during the IR, then discerning an effect of temporal discrimination on running may be difficult. Fatigue due to prior running seems a likely explanation. Satiation provides another possible explanation; however, because running, unlike eating or drinking, does not involve the ingestion of a substance or an identified physiological mechanism for satiety, satiation seems a possible, but less probable alternative. In contrast, there is evidence to suggest that habituation plays a role. Belke and McLaughlin (2005) showed that the presentation of a tone during a wheel-running reinforcement period leads to a recovery of running consistent with dishabituation. This effect supports a role for habituation as opposed to fatigue or satiation. Regardless, control of responding by one or more of these other factors might prevent changes in running related to timing, if timing influenced responding. 3. General discussion
Fig. 4. Wheel-running rates (rpm) in successive 5-s intervals within the 60-s probe interval and the 60-s wheel-running reinforcement period for each rat and the group. Standard error bars are provided for each mean. The stippled line indicates the time of termination of the 30-s wheel-running reinforcement for sessions with 60-s probes.
Experiment 1. Exposing the animals to fewer sessions containing probes may account for this difference. In contrast, the peak procedure, perhaps, better labeled the “trough” procedure, did not provide any evidence to support the assertion that the decline in running that occurs during a wheelrunning reinforcement period results from timing the interval. If the animals were timing the IR, then as the termination of the IR approached, discrimination of the time of termination might lead the animal to reduce its running rate. Extending the running period during IR probes failed to produce an increase in running in the period extending beyond the time when the brake would have been asserted.
In both experiments, lever pressing reinforced by the opportunity for wheel running on a fixed-interval schedule showed evidence of temporal discrimination of the schedule interval. Responding was low following the termination of a reinforcer and increased as the interval elapsed toward the time when the reinforcer would become available. Extending the interval beyond the time when the reinforcer was typically delivered produced a peak in responding near the time when the reinforcer would have occurred followed by a decline in responding. These data clearly show that the rats are timing the onset of the reinforcer. In contrast, a similar extension of the reinforcer interval failed to produce a change in wheel-running behavior that would suggest that the rats were timing the offset of the interval. The absence of evidence of timing during the wheel-running reinforcement period may be relevant to recent findings from a study of choice between opportunities to run of different durations (Belke, 2006). In this study, rats were exposed to concurrent schedules of wheel-running reinforcement with equal schedule values. The reinforcing consequences were opportunities to run that were of equal (i.e., 30 s versus 30 s) or unequal duration (10 and 50 s). Choice between the alternatives did not differ with differences in duration between the alternatives until the duration of one of the alternatives became very short. A systematic preference for the longer duration opportunity to run occurred when the shorter duration was 2.5 s and the longer duration was 57.5 s. Belke (2006) hypothesized that the lack of an effect of
T.W. Belke, M.M. Christie-Fougere / Behavioural Processes 73 (2006) 240–247
the differences in duration could arise if either the rats were not timing the difference in durations or the rats were not associating the difference in duration with the response alternatives. This latter explanation was favored based on a study by Killeen and Smith (1984) that showed that pigeons’ accuracy in attributing the occurrence of a reinforcer to their behavior as opposed to occurring independent of their behavior diminished as the duration of the reinforcer (i.e., access to food) increased from 1 to 4 s. Killeen and Smith argued that engaging in consummatory behavior during the reinforcer interval adversely affected an animal’s ability to accurately attribute the source of the consequence. Extrapolation to Belke (2006) study suggested that the wheel running that occurred during the first 10 s, that was common to both a 10 and 50-s reinforcer, diminished the association of the difference in duration with the respective response alternative. The other reason this explanation was favored was that previous research on the ability of animals to discriminate intervals of different durations (Church et al., 1976; Crystal, 1999; Roberts and Church, 1978) suggested that rats can accurately discriminate between stimulus events of different durations for intervals as long and as short as those used by Belke (2006). Although not conclusive, the absence of evidence for timing of the wheel-running reinforcer interval questions the assumption that rats are timing the wheel-running intervals. If this is the case, then the indifference in choice between wheelrunning reinforcer durations observed by Belke (2006) would occur as a result of the former rather than the latter explanation proposed by Belke (2006). Further investigation of timing during wheel-running reinforcement periods is required, preferably using procedures other than the peak procedure, such as a duration discrimination procedure (Church et al., 1976; Crystal, 1999; Roberts and Church, 1978), to more conclusively determine if rats are timing wheel-running reinforcer intervals. Acknowledgements Part of this report is related to an undergraduate thesis submitted by the second author in partial fulfillment of a B.Sc. degree at Mount Allison University, Sackville, Canada. This research was supported by Grant 0GP0170022 from the Natural Sciences and Engineering Research Council of Canada. Correspondence regarding this article should be sent to Terry W. Belke, Department of Psychology, Mount Allison University, Sackville, New Brunswick, Canada E4L 1C7 or via e-mail to [email protected].
247
References Belke, T.W., 1996. The effect of a change in body weight on running and responding reinforced by the opportunity to run. Psychol. Rec. 46, 421– 433. Belke, T.W., 1997. Running and responding reinforced by the opportunity to run: effect of reinforcer duration. J. Exp. Anal. Behav. 67, 337–351. Belke, T.W., 2000. Differences in responding for sucrose and wheel-running reinforcement: excitatory stimulus effects or inhibitory after-effects? Anim. Learn. Behav. 28, 332–343. Belke, T.W., 2006. Concurrent-schedules of wheel-running reinforcement: choice between different durations of opportunity to run in rats. Learn. Behav. 34, 61–70. Belke, T.W., Dunbar, M., 1998. Effects of fixed-interval schedule and reinforcer duration on responding reinforced by the opportunity to run. J. Exp. Anal. Behav. 70, 69–78. Belke, T.W., Hancock, S.D., 2003. Responding for sucrose and wheel-running reinforcement: effects of sucrose concentration and wheel-running reinforcer duration. J. Exp. Anal. Behav. 79, 243–265. Belke, T.W., Heyman, G.M., 1994. A matching law analysis of the reinforcing efficacy of wheel running in rats. Anim. Learn. Behav. 22, 267– 274. Belke, T.W., McLaughlin, R.J., 2005. Habituation contributes to the decline in wheel running within wheel-running reinforcement periods. Behav. Process. 68, 107–115. Catania, A.C., 1970. Reinforcement schedules and psychophysical judgments: a study of some temporal properties of behavior. In: Schoenfeld, W.N. (Ed.), The Theory of Reinforcement Schedules. Appleton-Century-Crofts, New York, pp. 1–42. Church, R.M., Getty, D.J., Lerner, N.D., 1976. Duration discrimination by rats. J. Exp. Psychol.: Anim. Behav. Process. 2, 303–312. Collier, G., Hirsch, E., 1971. Reinforcing properties of spontaneous activity in the rat. J. Compar. Physiol. Psychol. 77, 155–160. Crystal, J.D., 1999. Systematic nonlinearities in the perception of temporal intervals. J. Exp. Psychol.: Anim. Behav. Process. 25, 3–17. Iversen, I.H., 1993. Techniques for establishing schedules with wheel running as reinforcement in rats. J. Exp. Anal. Behav. 60, 219–238. Kagan, J., Berkun, M., 1954. The reward value of running activity. J. Compar. Physiol. Psychol. 47, 108. Killeen, P.R., Smith, J.P., 1984. Perception of contingency in conditioning: scalar timing, response bias, and erasure of memory by reinforcement. J. Exp. Psychol.: Anim. Behav. Process 10, 333–345. Pierce, W.D., Epling, W.F., Boer, D.P., 1986. Deprivation and satiation: the interrelations between food and wheel running. J. Exp. Anal. Behav. 46, 199–210. Premack, D., Schaeffer, R.W., Hundt, A., 1964. Reinforcement of drinking by running: effect of fixed ratio and reinforcement time. J. Exp. Anal. Behav. 7, 91–96. Roberts, S., 1981. Isolation of an internal clock. J. Exp. Psychol.: Anim. Behav. Process. 7, 242–268. Roberts, S., 1982. Cross-modal use of an internal clock. J. Exp. Psychol.: Anim. Behav. Process. 8, 2–22. Roberts, S., Church, R.M., 1978. Control of an internal clock. J. Exp. Psychol.: Anim. Behav. Process. 4, 318–337.
Investigations of timing during the schedule and reinforcement intervals with wheel-running reinforcement Terry W. Belke ∗ , Melissa M. Christie-Fougere Mount Allison University, Sackville, New Brunswick, Canada E4L 1C7 Received 25 May 2006; accepted 2 June 2006
Abstract Across two experiments, a peak procedure was used to assess the timing of the onset and offset of an opportunity to run as a reinforcer. The first experiment investigated the effect of reinforcer duration on temporal discrimination of the onset of the reinforcement interval. Three male Wistar rats were exposed to fixed-interval (FI) 30-s schedules of wheel-running reinforcement and the duration of the opportunity to run was varied across values of 15, 30, and 60 s. Each session consisted of 50 reinforcers and 10 probe trials. Results showed that as reinforcer duration increased, the percentage of postreinforcement pauses longer than the 30-s schedule interval increased. On probe trials, peak response rates occurred near the time of reinforcer delivery and peak times varied with reinforcer duration. In a second experiment, seven female Long-Evans rats were exposed to FI 30-s schedules leading to 30-s opportunities to run. Timing of the onset and offset of the reinforcement period was assessed by probe trials during the schedule interval and during the reinforcement interval in separate conditions. The results provided evidence of timing of the onset, but not the offset of the wheel-running reinforcement period. Further research is required to assess if timing occurs during a wheel-running reinforcement period. © 2006 Elsevier B.V. All rights reserved. Keywords: Wheel-running reinforcement; Timing; Peak procedure; Fixed interval; Lever pressing; Rat
With wheel-running reinforcement, completion of a schedule requirement provides access to the opportunity to run for a period of time (e.g., Belke and Heyman, 1994; Collier and Hirsch, 1971; Kagan and Berkun, 1954; Iversen, 1993; Pierce et al., 1986; Premack et al., 1964). When the requirement is a fixed-interval (FI) schedule and the period of access to the opportunity to run is constant, the procedure provides an opportunity to assess temporal discrimination of the onset and the offset of a wheel-running reinforcement period. The present study reports on two experiments that investigated the role of timing with wheel-running reinforcement. 1. Experiment 1 In the first experiment, the effect of wheel-running reinforcer duration on temporal discrimination of the onset of the wheel-running period was investigated. Following the termination of a wheel-running reinforcer, an animal typically pauses ∗
Corresponding author. E-mail address: [email protected] (T.W. Belke).
0376-6357/$ – see front matter © 2006 Elsevier B.V. All rights reserved. doi:10.1016/j.beproc.2006.06.004
for a period of time before pressing a lever. This postreinforcement pause (PRP) varies with the duration of the wheel-running reinforcement period (Belke, 1997; Belke and Dunbar, 1998; Belke and Hancock, 2003) and is longer than typically observed with more conventional reinforcers such as a small bit of food or water (Belke, 2000). Furthermore, with conventional reinforcers, pause duration shortens appreciably when the schedule requirement, either ratio or interval, is changed from fixed to variable. This suggests that the pause duration is largely a function of the reinforcement schedule. In contrast, with wheelrunning reinforcement the PRP duration remains long even when the schedule requirement is variable (Belke, 1996, 1997). PRP duration with wheel-running reinforcement appears to be less a function of the schedule than of the reinforcer (Belke and Dunbar, 1998). One implication of this difference is that if the duration of the pause is typically longer than the duration of the interval of the schedule (IS), then the IS will not be experienced as programmed. At the extreme, the schedule will be experienced as a continuous reinforcement schedule rather than as an intermittent schedule of reinforcement (Belke and Dunbar, 1998). For this reason, response-initiated schedules (e.g., tandem fixed ratio
T.W. Belke, M.M. Christie-Fougere / Behavioural Processes 73 (2006) 240–247
[FR] 1 variable interval [VI] 30-s schedule) have been used to ensure that the animal experiences the programmed intervals (Belke, 1996, 1997; Belke and Heyman, 1994). On standard FI schedules, as the duration of the PRP following wheel running increases or the duration of the schedule value decreases, the percentage of intervals experienced as terminating at a time after the schedule value has elapsed should increase (Belke and Dunbar, 1998). For example, on a FI schedule with a 60-s wheelrunning reinforcement interval, 88% of PRPs were longer than the IS when the IS was 15 s and 73% were longer when the IS was 30 s. To the extent that fewer intervals are experienced as terminating when the IS elapses, discrimination of the time when the reinforcer becomes available may be affected. That is, if the majority of reinforcers occur following postreinforcement pauses of variable durations, all longer than the IS, then the animal will not be experiencing the periodicity of delivery of the reinforcer that underlies discrimination of IS. Consequently, the objective of the first experiment was to assess temporal discrimination of the onset of the wheel-running reinforcement period with different reinforcer durations using a peak procedure. The peak procedure, developed by Catania (1970) and extensively utilized by Roberts (1981) to investigate temporal discrimination in animals, involves an FI schedule with reinforced and non-reinforced trials. On FI schedules, responding is low early and accelerates as the IS elapses so that the highest rate of responding occurs just prior to the onset of the reinforcer. On reinforced trials, the first response after the IS requirement has lapsed produces the reinforcer. On non-reinforced trials, also known as probe trials, the interval extends beyond the end of the IS and does not terminate with a reinforcer. On probe trials, responding generally increases up to the time when reinforcement would have occurred and decreases as the interval progresses beyond the time when reinforcement would have occurred. The result, when averaged over many probe trials, is a Gaussian curve centered on the time, at or near, when the reinforcer would typically occur. 1.1. Method 1.1.1. Subjects Four male Wistar rats obtained from Charles River Breeding Laboratories, St. Constant, Quebec served as subjects. At the start of the experiment, the rats were approximately 7 months old and experimentally naive. They were individually housed in standard polycarbonate cages (48 cm × 27 cm × 22 cm) in a colony room maintained at 20 ◦ C with a 12-h light/dark cycle (lights on at 07:00). For the duration of the experiment, animals were maintained at a target weight of 335 ± 10 g. Distilled water was freely available in their home cages. One rat fell ill shortly after completing the first experimental condition and was removed from the study. Data from this rat were not included in the analyses in Section 1.2. 1.1.2. Apparatus Experimental sessions of responding on levers for the opportunity to run occurred in activity wheels (1 Wahmann and 3 Lafayette Instruments Model #86041 A) without side cages. The
241
diameters of the wheels were 35.5 cm. Each wheel was located in a sound-proof shell equipped with a fan for ventilation and to mask extraneous noise. A retractable lever (Med Associates ENV-112) was mounted directly at the opening of each wheel. The lever extended 1.8 cm into the wheel through an opening (7 cm × 9 cm) in the center at the base of the wheel frame. A microswitch attached to the wheel frame recorded wheel revolutions. The force required to close the lever microswitches ranged between 18 and 27 g. Bulbs (24-V dc) mounted on the sides of the wheel frame served to illuminate the inside of the wheel. A solenoid-operated brake was attached to the base of each wheel. When the solenoid was operated, a rubber tip attached to a metal shaft contacted the outer rim of the wheel and brought the wheel to a stop. Control of experimental events and recording of data were handled by IBM® personal computers interfaced to the wheels. 1.1.3. Procedure Rats used in this experiment were selected from a larger group of animals that were initially trained to lever press for the opportunity to run and then distributed to several experimental procedures. This training began by providing the animals with the opportunity to run freely in a running wheel for 30 min each day. Sessions occurred at the same time of day for 15 days. The number of wheel revolutions was recorded for each rat on each day. After 15 days, the highest rate runners were selected for further training. In the next phase the rats continued to receive 30-min access to the free-moving running wheel at the same time each day. Following the session of wheel running each rat was placed in a standard operant conditioning chamber and lever pressing was shaped by reinforcing successive approximations. Each lever press produced 0.1 ml of a 15% sucrose solution. When subjects reliably pressed the lever, the schedule of reinforcement was shifted from requiring only a single response per reinforcer (FR 1) to one requiring a variable number of responses averaging 3 (i.e., a variable-ratio [VR] schedule). This schedule remained in effect for four sessions, with each session terminating when 50 sucrose reinforcers were obtained. After four sessions on the VR 3 schedule, sessions in the operant conditioning chamber were discontinued. At this point, the retractable lever in each wheel chamber was extended during the wheel-running sessions and the opportunity to run for 60 s was contingent upon a single lever press. Retraction of the lever and movement of the wheel with the release of the brake signaled access to the running period. Each session consisted of 30 opportunities to run. During each session, the lights at the side of the wheel were illuminated. The schedule of reinforcement was changed in the following sequence: FR 1, VR 3, VR 5, and VR 9. Subjects remained on each schedule for four sessions before advancing to the next schedule. Following this initial training of a larger group of rats, four rats were assigned to the current study, one of which subsequently died due to health problems. The rats were placed on a standard FI 30-s schedule. Each reinforcer consisted of a 30s opportunity to run. Each session ended when 60 reinforcers were completed. Sessions were run 7 days a week. After 60 sessions, reinforcer duration was changed and probe sessions began.
242
T.W. Belke, M.M. Christie-Fougere / Behavioural Processes 73 (2006) 240–247
Reinforcer durations were 15, 30, and 60 s and each animal experienced a different order of durations. The duration orders for IB13, 16, and 17 were 30–60–15, 60–30–15, 15–30–60 s, respectively. Each reinforcer duration was in effect for 40 probe sessions. After this fixed number of sessions, reinforcer duration was changed. During a probe session, probe trials did not occur during the first 10 reinforcers. Following the first 10 reinforcers, probe trials occurred during 2 out of each successive set of 10 reinforcers that occurred. In total, probe trials occurred in 10 of the last 50 reinforcers of a session. The occurrence of a probe trial was determined by random selection from a list of numbers. During a probe trial, the duration of the interval was extended to 90 s plus a variable duration varying between 5 and 25 s. The addition of this variable time period ensured that the probe duration was variable. Following the termination of a probe, the lights located at the side of the wheel were extinguished and the retractable lever was retracted. After 5 s, the lights were illuminated and the retractable lever was extended. Since a probe trial did not terminate with an opportunity to run, the 5-s blackout period and retraction of the lever were necessary to signal the initiation of the next schedule interval. The present procedure differed from the typical peak procedure in that no intertrial interval (ITI) was programmed to occur following the occurrence of a reinforcer during FI trials. With respect to this particular experiment, inclusion of an ITI following the termination of a wheel-running reinforcer would have negated the effect of the long PRP that follows a wheel-running reinforcer on responding during the subsequent fixed interval. The objective of this experiment was to assess the effect of this long PRP on temporal discrimination of the IS. Lever presses, time spent lever pressing, and PRP duration were recorded for each reinforcement and collectively for the entire session. Within the 30-s schedule interval, lever presses were recorded cumulatively in successive 5-s segments across the schedule intervals leading to reinforcement within a session. During probes, lever presses were recorded in successive 5-s intervals cumulatively throughout 90-s period that was common to all probes to produce one set of probe values for each session. Data were averaged over the last 10 sessions in each condition for analysis. Standard errors were calculated over values from these 10 sessions. Peak time was calculated for each response rate function for the probes from each session based on iterative calculations of medians over decreasing intervals (Roberts, 1981). First, the median was determined over the range from 0 to 60-s. This is the range over which responding would be expected to be centered if the animals had discriminated the 30s IS. The value of this median determines the range over which the next median was calculated. If the median was less than 30 s, the lower limit of the new range was 0 s and the upper limit was twice the value of the median. If the median was greater than 30 s, then the upper limit was 60 s and the lower limit was twice the difference between the median greater than 30 s and 30 s. This procedure was repeated across successive determinations of medians until it produced a median that was within 0.5 s of the previous median. The peak time for each reinforcer dura-
Table 1 Median postreinforcement pause durations (s) and percentage of postreinforcement pauses (PRPs) greater than the schedule interval (30 s) for each animal Rat
Median
Percent of PRPs > 30 s
15
30
60
15
30
60
IB13 IB16 IB17
24.8 23.6 25.7
26.1 29.2 24.5
29.4 34.8 30.5
35.9 32.4 38.0
37.1 48.8 32.1
48.3 57.3 51.3
Mean
24.7
26.6
31.6
35.4
39.3
52.3
tion condition was the average of the peak times across the 10 sessions. 1.2. Results Table 1 shows the median PRPs and percentage of PRPs greater than the schedule value for each animal. Median PRPs were used because the median is less susceptible to outlier values than is the mean. As reinforcer duration increased, the median PRP duration and the percentage of pauses greater than the IS increased. A repeated measures ANOVA showed a significant effect of reinforcer duration on median PRPs, F(2, 4) = 8.07, p = 0.04. Paired t-test comparisons showed that the percentage of intervals with pauses greater than the IS was significantly higher when the reinforcer duration was 60 s than when it was 15 s (t(2) = −3.99, p = 0.03, one tailed) or 30 s (t(2) = −3.86, p = 0.03, one tailed). Fig. 1 shows mean response rates (presses/min) in successive 5-s segments of the 90-s probes for each wheel-running
Fig. 1. Response rates (presses/min) in successive 5-s intervals within the 90-s probe interval for the 15, 30, and 60-s wheel-running reinforcer duration conditions for each rat and the group. Standard error bars are provided for each mean. The stippled line indicates the time of reinforcement delivery on the FI 30-s schedule.
T.W. Belke, M.M. Christie-Fougere / Behavioural Processes 73 (2006) 240–247
243
Table 2 Average peak times (s) and standard errors for each reinforcer duration condition for each animal Rat
Duration 15
30
60
IB13 IB16 IB17
34.9 (1.1) 35.6 (0.6) 37.6 (0.8)
38.2 (0.8) 38.4 (1.3) 37.3 (1.0)
38.7 (1.7) 40.2 (1.3) 41.1 (1.6)
Mean
36.0 (0.8)
38.0 (0.3)
40.0 (0.7)
reinforcer duration for each animal and the group. Visual inspection of this figure suggests that lever-pressing rates increased toward the time when the wheel-running reinforcer was scheduled to occur and then decreased at a time near or past when the reinforcer was to occur. For the 15-s reinforcer condition, the highest lever-pressing rates occurred in the 25–30, 30–35, and 35–40s segments for IB13, 16, and 17, respectively. For the 30-s reinforcer condition, the highest lever-pressing rates occurred in the 30–35, 35–40, and 35–40s segments. For the 60-s reinforcer condition, the highest lever-pressing rates occurred in the 30–35, and 35–40s segments for IB13 and 17. For IB16, the highest lever-pressing rate occurred in the 35–40 and 45–50s segments. On average, the highest lever-pressing rates occurred in the 31–35s segment of the 90-s probe in the 15-s reinforcer duration condition and the 36–40s segment in the 30 and 60-s conditions. Table 2 shows average peak times across probes for each reinforcer duration for each animal. In general peak times corresponded with the intervals with the highest average leverpressing rates, but were higher in some instances due to variability across probes generated within each session. As suggested by Fig. 1, as wheel-running reinforcer duration increased, average peak time increased. Due to the lack of power of non-parametric analyses to detect differences with such a small number of observations, peak times were analyzed using a parametric analysis. A repeated measures ANOVA revealed a significant effect of reinforcer duration, F(2, 4) = 10.28, p = 0.03. Fig. 2 shows lever-pressing rates during the IS for the nonprobe trials (i.e., reinforcer occurred) in the left panel and wheel-running rates during the interval of reinforcement (IR) over all trials in the right panel for each duration condition for each animal and the group. The left panels show that, on average, as reinforcer and PRP duration increased, responding increased less rapidly as the interval progressed. As a result, the highest rate of responding in the interval closest to reinforcement decreased as the IR increased. Mean lever-pressing rates for the 15, 30, and 60-s reinforcer durations were 19.71, 15.96, and 11.68 responses/min. This decrease in responding as a function of reinforcer duration reflects a shift in the initiation of responding to a later time as PRP duration increased. It is also consistent with the decrease in responding observed in the probes. The right panels show that within the IR, wheelrunning rates were higher at the onset and declined as the IR elapsed toward termination. The potential role of temporal discrimination of the offset of the IR with respect to producing this
Fig. 2. Lever-pressing rates (presses/min) in successive 5-s intervals within the FI 30-s schedule in the left panels and wheel-running rates (rpm) in successive 5-s intervals within the reinforcement period in the right panels for the 15, 30, and 60-s wheel-running reinforcer duration conditions for each rat and the group. Standard error bars are provided for each mean.
pattern of running within the IR will be the focus of the next experiment. 1.3. Discussion Consistent with previous investigations using other reinforcers (Roberts, 1981, 1982), rats in the present study appeared to time the onset of an opportunity to run. Responding on probe trials peaked at or near the time when the opportunity to run would have occurred. As the duration of the wheel-running reinforcer period increased, postreinforcement pause durations increased. At the longest duration animals experienced just less than half of the schedule intervals as terminating with a reinforcer 30 s after the end of the previous reinforcer. Changes in
244
T.W. Belke, M.M. Christie-Fougere / Behavioural Processes 73 (2006) 240–247
response rate functions coupled with the analysis of peak times suggest that as reinforcer duration increased, there was a small but systematic shift in peak times during the probes. Thus, perception of the onset of reinforcement may have been affected by reinforcer duration; however, due to the small number of observations, this conclusion should be regarded as tentative. It is worth noting that this effect of lengthening PRPs on responding during a subsequent IS can be attenuated or eliminated through the use of an intertrial interval that is typically incorporated into the peak procedure. 2. Experiment 2 Experiment 1 demonstrated that, as with other reinforcers, rats discriminate the onset of an opportunity to run and that the long PRP that follows running may affect discrimination of the time of onset. Experiment 2 investigated the role of timing during the IR period. During the IS, discrimination of the time when a reinforcer occurs governs the pattern of responding. Immediately after a reinforcer, responding is absent or low. As time elapses toward when reinforcement becomes available, responding accelerates and is highest in the interval immediately preceding the time when the interval elapses. During the IR, there is also a pattern of running. Following the onset of an opportunity to run, wheel running is highest and decreases as the interval progresses. Decreases in running are greater earlier in the interval (see Fig. 2). This pattern of running within the reinforcement interval (IR, interval of reinforcement) raises the possibility that the rats are timing the IR and that their temporal discrimination of the IR may influence running during the IR (Belke and McLaughlin, 2005). Experiment 1 provided evidence that rats discriminate the time of onset of an opportunity to run. The objective of the current experiment was to determine if rats also discriminate the offset of a brief opportunity to run. To do this, rats responding on a FI 30-s schedule of wheel-running reinforcement with a 30-s IR were exposed to extended probe intervals during the IS and IR to assess temporal discrimination of the onset and offset of an opportunity to run. 2.1. Method 2.1.1. Subjects Seven female Long-Evans rats obtained from Charles River Breeding Laboratories, St. Constant, Quebec served as subjects. In the interim between Experiment 1 and the current study, the laboratory switched to using female rats to study the reinforcing properties of wheel running. Female rats run at higher rates than do male rats and, presumably, find wheel running a more efficacious reinforcer. This difference negates the need to select rats to be trained to lever press for the opportunity to run based on wheel running rates during the initial training phase. At the start of the experiment, the rats were approximately 18 months old and had participated in previous wheel-running reinforcement studies. For the duration of the experiment, the animals were maintained at a target weight of 260 ± 10 g. All other conditions were as stated in Section 1.1.1.
2.1.2. Apparatus The apparatus used in Experiment 1 was used in Experiment 2. 2.1.3. Procedure Training the rats to press a lever for the opportunity to run followed the same steps described in Experiment 1 with the exception that the rats were not trained to press a lever in a standard operant conditioning chamber using sucrose solution as a reinforcer. After being exposed to the opportunity to run continuously for 30 min each day, the lever was extended and the opportunity to run for 60 s was contingent upon a single lever press. All rats learned to lever press within one or two sessions. Prior to participating in the current study, the subjects participated in other operant investigations of wheel-running reinforcement, the most recent of which involved an FI 30-s schedule with a 45-s reinforcer. In preparation for the current procedure, all subjects were exposed to a baseline condition for 20 sessions. In the baseline condition, the schedule of wheel-running reinforcement was a FI 30-s schedule and the consequence was the opportunity to run for 30 s. Each session terminated when 50 reinforcers were completed. Following the baseline condition, timing during the IS was assessed in three rats and during the IR in four rats. After 20 sessions, all rats were returned to the baseline condition. Following the completion of 20 baseline sessions, timing during the IR was assessed in the 3 rats that had timing assessed during the IS and timing during the IS was assessed in the 4 rats that had timing assessed during the IR. These procedures remained in effect for 20 sessions. Finally, all animals were once again returned to the baseline condition for 20 sessions. Across this set of conditions, timing in both the IS and IR was assessed in all seven subjects. The peak procedure was used to assess timing during the IS. During an IS probe session, probe trials did not occur during the first 10 reinforcers. Following the first 10 reinforcers, probe trials occurred during 2 out of each successive set of 10 reinforcers. In total, probe trials occurred in 8 of the last 40 trials of a session. The occurrence of a probe trial was determined by random selection from a list of numbers. During an IS probe trial, the duration of the interval was extended to 90 s plus a variable duration varying between 5 and 25 s. As in Experiment 1, following the termination of a probe, the lever was retracted and a 5-s blackout period occurred. After 5 s, the lights at the side of the wheel were illuminated and the lever extended to signal the start of the next schedule interval. No ITI was programmed to occur. During sessions with IS probes, wheel-running reinforcer duration remained constant at 30 s. Assessment of timing during the IR involved a similar procedure with the exception that during the last 40 reinforcers of a session 8 probe trials occurred. During an IR probe trial, the duration of the wheel-running reinforcement period was extended to 60 s plus a variable duration varying between 5 and 25 s. The termination of the probe trial and the onset of the next schedule value were signaled by the onset of the brake and the extension of the lever. Again, no ITI was programmed to occur. During sessions with IR probes, the schedule interval remained constant at 30 s.
T.W. Belke, M.M. Christie-Fougere / Behavioural Processes 73 (2006) 240–247
245
After the completion of final set of 20 baseline sessions, all rats were placed on a FI 30-s schedule with a 60-s opportunity to run as a reinforcing consequence. Sessions terminated after 25 reinforcers were completed to equate the total duration of opportunity to run with the time available to run during the baseline condition in which there were 50 reinforcement periods 30 s in duration. This condition remained in effect for 20 sessions. The purpose of this condition was to serve as a control against which wheel running during the 60-s probe could be compared. If wheel running during the 60-s IR probes with a 30-s wheelrunning reinforcer showed evidence of timing, then the pattern of wheel running should differ from that which occurs during a 60-s wheel-running reinforcer. Lever presses, time spent lever pressing, and postreinforcement pause duration were recorded for each reinforcement and collectively for the entire session. During schedule probes, lever presses were recorded in successive 5-s intervals throughout the 90-s period. During reinforcer probes, wheel revolutions were recorded in successive 5-s intervals throughout the 60-s period. Data from the last five sessions in each condition were analyzed. Mean peak times were calculated as previously described in Experiment 1. 2.2. Results Fig. 3 shows mean response rates during successive 5-s segments of the 90-s probe interval for each animal and the group. With the exception of TK 2, the rats show a pattern of responding suggesting the development of temporal discrimination. Responding rises toward a peak and then declines. However, the location of the peak rate was quite variable. The highest rate was in the 30–35 s segment for TK 16, the 35–40 s segment for TK 9 and 15, the 40–45 s segment for TK 1 and 11, and the 45–50 s segment for TK 6. For TK2, responding rose toward the 30–35 s segment, declined in the 35–40 s segment and then continued to rise to a peak rate in the 55–60 s segment. For the group, the highest rate occurred in the 40–45 s segment. With respect to peak times, the mean peak time across all rats was 42.4 s. Fig. 4 shows wheel-running rates (rpm) in successive 5s segments of the 60-s probe interval and the 60-s IR for each animal and the group. In general, running during the 60s probes rises from the first 5 s to a peak during the second 5-s segment and then declines as the interval progresses, leveling out in the latter part of the interval. With the exception of TK 15, there is no evidence of an increase in running after 30 s as would be expected if the decline in running was due to the discrimination that the running period would terminate. For TK 15, running decreased from the second to the fifth 5s segment and then increased from the sixth to the eighth 5-s segment. A repeated measures ANOVA with source (probe, reinforcer) and 5-s interval (1–12) as within-subject variables revealed a significant effect of interval, F(11, 66) = 45.21, p < 0.001, but no effect of source, F(1, 6) = 2.80, p = 0.15, and no interaction, F(11, 66) = 1.69, p = 0.10. Thus, running during the 60-s probe did not differ from running during the 60-s reinforcer. The similarity of
Fig. 3. Response rates (presses/min) in successive 5-s intervals within the 90s probe interval for each rat and the group. Standard error bars are provided for each mean. Mean peak time and standard errors are noted below the rat designation. The stippled line indicates the time of reinforcement delivery on the FI 30-s schedule.
the two patterns does not support the assertion that timing plays a role in the decline in running that occurs within a reinforcement period. 2.3. Discussion As observed in Experiment 1, behavior during the IS leading up to the onset of an opportunity to run showed evidence of temporal discrimination. Responding was lowest following the termination of a reinforcement period and increased as the interval progressed toward the elapse of the schedule when a response would produce the onset of an opportunity to run. Peak response rates were more varied and less well defined than in
246
T.W. Belke, M.M. Christie-Fougere / Behavioural Processes 73 (2006) 240–247
A review of the literature failed to reveal an instance of the peak procedure being used to assess the temporal discrimination of the offset of an interval during which behavior could occur. Consequently, the validity of using the procedure in this manner is open to question. Also, if there is a difference in the motivational value of the onset and the offset of a reinforcer then this may make the procedure more suitable to timing the onset as opposed to the offset of a reinforcement period. Satiation would be one reason for this difference in motivational significance. Consumption of the reinforcer prior to the offset of a reinforcer may make the offset of the reinforcer less salient. Finally, if factors other than timing such as fatigue, satiation, and/or habituation (Belke and McLaughlin, 2005) play a role in determining the pattern of running during the IR, then discerning an effect of temporal discrimination on running may be difficult. Fatigue due to prior running seems a likely explanation. Satiation provides another possible explanation; however, because running, unlike eating or drinking, does not involve the ingestion of a substance or an identified physiological mechanism for satiety, satiation seems a possible, but less probable alternative. In contrast, there is evidence to suggest that habituation plays a role. Belke and McLaughlin (2005) showed that the presentation of a tone during a wheel-running reinforcement period leads to a recovery of running consistent with dishabituation. This effect supports a role for habituation as opposed to fatigue or satiation. Regardless, control of responding by one or more of these other factors might prevent changes in running related to timing, if timing influenced responding. 3. General discussion
Fig. 4. Wheel-running rates (rpm) in successive 5-s intervals within the 60-s probe interval and the 60-s wheel-running reinforcement period for each rat and the group. Standard error bars are provided for each mean. The stippled line indicates the time of termination of the 30-s wheel-running reinforcement for sessions with 60-s probes.
Experiment 1. Exposing the animals to fewer sessions containing probes may account for this difference. In contrast, the peak procedure, perhaps, better labeled the “trough” procedure, did not provide any evidence to support the assertion that the decline in running that occurs during a wheelrunning reinforcement period results from timing the interval. If the animals were timing the IR, then as the termination of the IR approached, discrimination of the time of termination might lead the animal to reduce its running rate. Extending the running period during IR probes failed to produce an increase in running in the period extending beyond the time when the brake would have been asserted.
In both experiments, lever pressing reinforced by the opportunity for wheel running on a fixed-interval schedule showed evidence of temporal discrimination of the schedule interval. Responding was low following the termination of a reinforcer and increased as the interval elapsed toward the time when the reinforcer would become available. Extending the interval beyond the time when the reinforcer was typically delivered produced a peak in responding near the time when the reinforcer would have occurred followed by a decline in responding. These data clearly show that the rats are timing the onset of the reinforcer. In contrast, a similar extension of the reinforcer interval failed to produce a change in wheel-running behavior that would suggest that the rats were timing the offset of the interval. The absence of evidence of timing during the wheel-running reinforcement period may be relevant to recent findings from a study of choice between opportunities to run of different durations (Belke, 2006). In this study, rats were exposed to concurrent schedules of wheel-running reinforcement with equal schedule values. The reinforcing consequences were opportunities to run that were of equal (i.e., 30 s versus 30 s) or unequal duration (10 and 50 s). Choice between the alternatives did not differ with differences in duration between the alternatives until the duration of one of the alternatives became very short. A systematic preference for the longer duration opportunity to run occurred when the shorter duration was 2.5 s and the longer duration was 57.5 s. Belke (2006) hypothesized that the lack of an effect of
T.W. Belke, M.M. Christie-Fougere / Behavioural Processes 73 (2006) 240–247
the differences in duration could arise if either the rats were not timing the difference in durations or the rats were not associating the difference in duration with the response alternatives. This latter explanation was favored based on a study by Killeen and Smith (1984) that showed that pigeons’ accuracy in attributing the occurrence of a reinforcer to their behavior as opposed to occurring independent of their behavior diminished as the duration of the reinforcer (i.e., access to food) increased from 1 to 4 s. Killeen and Smith argued that engaging in consummatory behavior during the reinforcer interval adversely affected an animal’s ability to accurately attribute the source of the consequence. Extrapolation to Belke (2006) study suggested that the wheel running that occurred during the first 10 s, that was common to both a 10 and 50-s reinforcer, diminished the association of the difference in duration with the respective response alternative. The other reason this explanation was favored was that previous research on the ability of animals to discriminate intervals of different durations (Church et al., 1976; Crystal, 1999; Roberts and Church, 1978) suggested that rats can accurately discriminate between stimulus events of different durations for intervals as long and as short as those used by Belke (2006). Although not conclusive, the absence of evidence for timing of the wheel-running reinforcer interval questions the assumption that rats are timing the wheel-running intervals. If this is the case, then the indifference in choice between wheelrunning reinforcer durations observed by Belke (2006) would occur as a result of the former rather than the latter explanation proposed by Belke (2006). Further investigation of timing during wheel-running reinforcement periods is required, preferably using procedures other than the peak procedure, such as a duration discrimination procedure (Church et al., 1976; Crystal, 1999; Roberts and Church, 1978), to more conclusively determine if rats are timing wheel-running reinforcer intervals. Acknowledgements Part of this report is related to an undergraduate thesis submitted by the second author in partial fulfillment of a B.Sc. degree at Mount Allison University, Sackville, Canada. This research was supported by Grant 0GP0170022 from the Natural Sciences and Engineering Research Council of Canada. Correspondence regarding this article should be sent to Terry W. Belke, Department of Psychology, Mount Allison University, Sackville, New Brunswick, Canada E4L 1C7 or via e-mail to [email protected].
247
References Belke, T.W., 1996. The effect of a change in body weight on running and responding reinforced by the opportunity to run. Psychol. Rec. 46, 421– 433. Belke, T.W., 1997. Running and responding reinforced by the opportunity to run: effect of reinforcer duration. J. Exp. Anal. Behav. 67, 337–351. Belke, T.W., 2000. Differences in responding for sucrose and wheel-running reinforcement: excitatory stimulus effects or inhibitory after-effects? Anim. Learn. Behav. 28, 332–343. Belke, T.W., 2006. Concurrent-schedules of wheel-running reinforcement: choice between different durations of opportunity to run in rats. Learn. Behav. 34, 61–70. Belke, T.W., Dunbar, M., 1998. Effects of fixed-interval schedule and reinforcer duration on responding reinforced by the opportunity to run. J. Exp. Anal. Behav. 70, 69–78. Belke, T.W., Hancock, S.D., 2003. Responding for sucrose and wheel-running reinforcement: effects of sucrose concentration and wheel-running reinforcer duration. J. Exp. Anal. Behav. 79, 243–265. Belke, T.W., Heyman, G.M., 1994. A matching law analysis of the reinforcing efficacy of wheel running in rats. Anim. Learn. Behav. 22, 267– 274. Belke, T.W., McLaughlin, R.J., 2005. Habituation contributes to the decline in wheel running within wheel-running reinforcement periods. Behav. Process. 68, 107–115. Catania, A.C., 1970. Reinforcement schedules and psychophysical judgments: a study of some temporal properties of behavior. In: Schoenfeld, W.N. (Ed.), The Theory of Reinforcement Schedules. Appleton-Century-Crofts, New York, pp. 1–42. Church, R.M., Getty, D.J., Lerner, N.D., 1976. Duration discrimination by rats. J. Exp. Psychol.: Anim. Behav. Process. 2, 303–312. Collier, G., Hirsch, E., 1971. Reinforcing properties of spontaneous activity in the rat. J. Compar. Physiol. Psychol. 77, 155–160. Crystal, J.D., 1999. Systematic nonlinearities in the perception of temporal intervals. J. Exp. Psychol.: Anim. Behav. Process. 25, 3–17. Iversen, I.H., 1993. Techniques for establishing schedules with wheel running as reinforcement in rats. J. Exp. Anal. Behav. 60, 219–238. Kagan, J., Berkun, M., 1954. The reward value of running activity. J. Compar. Physiol. Psychol. 47, 108. Killeen, P.R., Smith, J.P., 1984. Perception of contingency in conditioning: scalar timing, response bias, and erasure of memory by reinforcement. J. Exp. Psychol.: Anim. Behav. Process 10, 333–345. Pierce, W.D., Epling, W.F., Boer, D.P., 1986. Deprivation and satiation: the interrelations between food and wheel running. J. Exp. Anal. Behav. 46, 199–210. Premack, D., Schaeffer, R.W., Hundt, A., 1964. Reinforcement of drinking by running: effect of fixed ratio and reinforcement time. J. Exp. Anal. Behav. 7, 91–96. Roberts, S., 1981. Isolation of an internal clock. J. Exp. Psychol.: Anim. Behav. Process. 7, 242–268. Roberts, S., 1982. Cross-modal use of an internal clock. J. Exp. Psychol.: Anim. Behav. Process. 8, 2–22. Roberts, S., Church, R.M., 1978. Control of an internal clock. J. Exp. Psychol.: Anim. Behav. Process. 4, 318–337.