Response factors in delay discounting: Evidence for Pavlovian influences on delay discounting in pigeons

Behavioural Processes 98 (2013) 37–43

Contents lists available at SciVerse ScienceDirect

Behavioural Processes journal homepage: www.elsevier.com/locate/behavproc

Response factors in delay discounting: Evidence for Pavlovian influences on delay discounting in pigeons Daniel D. Holt ∗ , Joseph D. Carlson, Victoria L. Follett, Nicole J. Jerdee, David P. Kelley III, Kaija M. Muhich, Andrew M. Tiry, Nicholas K. Reetz University of Wisconsin-Eau Claire, Department of Psychology, Hibbard Humanities Hall 277, Eau Claire, WI 54702, United States

a r t i c l e

i n f o

Article history: Received 9 November 2012 Received in revised form 6 April 2013 Accepted 11 April 2013 Keywords: Adjusting-amount Delay Discounting Key peck Pigeons Treadle press

a b s t r a c t Pigeons completed a delay-discounting task where in different conditions the required response was either key pecking or treadle pressing. Because of stimulus–reinforcer relations that are known to form between localized visual cues and the delivery of food (e.g., autoshaping), we predicted that there would be steeper rates of discounting with key pecking than treadle pressing. To account for possible effort differences between key pecking and treadle pressing, pigeons also completed a discounting task where multiple key pecks were required to gain access to the food. The rates of discounting for the key peck and effort-equivalence discounting procedures were similar, and both were steeper than the rate of discounting for the treadle-pressing procedure. While it is tacitly assumed that behavior in choice situations is largely under the control of operant contingencies, the present results suggest that when developing animal analogs to study discounting in a discrete-trial choice procedure, the stimulus–reinforcer relations (Pavlovian conditioning) may need to be taken into consideration. © 2013 Elsevier B.V. All rights reserved.

1. Introduction Choice is often studied in terms of delay discounting, which refers to the decrease in the present subjective value of a reinforcer with an increase in the delay to receiving that reinforcer (e.g., Green and Myerson, 2004; Rachlin, 2006). A growing body of literature demonstrates that delay discounting is a cross-species phenomenon (e.g., Freeman et al., 2009; Green et al., 1994, 2004; Mazur, 1987; Rodriguez and Logue, 1988). The pattern of delay discounting across species is well described by the following hyperbolic function: V=

A 1 + kD

where V represents the present subjective value of a reinforcer, A refers to the amount of the reward, D represents the delay to receipt of the reinforcer, and the parameter k is a free parameter and quantifies the steepness of discounting (Mazur, 1987). Although cross-species similarities in discounting are present, some species differences exist. For instance, past research has shown that pigeons tend to discount delayed reinforcers more

∗ Corresponding author at: Department of Psychology, MSC 7704, James Madison University, Harrisonburg, VA 22807, United States. Tel.: +1 540 568 5051; fax: +1 540 568 3322. E-mail address: [email protected] (D.D. Holt). 0376-6357/$ – see front matter © 2013 Elsevier B.V. All rights reserved. http://dx.doi.org/10.1016/j.beproc.2013.04.009

steeply than other species (Mazur, 1987; Tobin and Logue, 1994; Green et al., 2004; Mazur, 2005). Green et al. (2004), for example, used the same temporal-discounting procedure with rats and pigeons and found that though both species discounted delayed reinforcers in a way well described by a hyperbolic function, pigeons, consistently showed steeper rates of discounting than rats. It is likely that multiple factors contribute to the relatively steep rate of discounting demonstrated by pigeons. For instance, pigeons may have evolved under environmental food source pressures where consistently choosing the immediate smaller option led to an increase in the likelihood of survival. As a result of variable environmental pressures and the resulting evolutionary adaptations, there may be large metabolic differences across species (e.g., Jetz et al., 2008) that could contribute to the heightened valuation of the smaller, more immediate reinforcer. It is also possible that the design of the discounting-task may contribute to the high rates of discounting observed in pigeons (i.e., it is a procedural artifact). For example, a key peck to the smaller-sooner alternative typically results in darkening of the key-light, a very brief (∼0.5 s) illumination of a stimulus light, followed immediately by the presentation of food, whereas a key peck to the larger-later alternative results in darkening of the key-light, an extended illumination of a stimulus light (e.g., 20 s), which is then followed by the presentation of food. Because the smaller-sooner reinforcer is received almost immediately following a response in a delay-discounting paradigm, the stimulus–reinforcer relation of responses to the smaller-sooner option is stronger than that of responses to the larger-later option.

38

D.D. Holt et al. / Behavioural Processes 98 (2013) 37–43

When considering the temporal relation between the stimulus (i.e., illuminated response key) and reinforcer delivery, there is a relatively strong contiguous relation associated with the smallersooner alternative. Using a trace-conditioning procedure, past research has demonstrated that a stimulus–reinforcer relation is strongest when the delay to the reinforcer is minimal (Lucas et al., 1981). In a discounting paradigm, a strong contiguous relation between the key-light and the smaller-sooner reinforcer may lead the pigeon to reflexively peck the key associated with the immediate reinforcer whereas; the relationship between the larger-later reinforcer and the key-light is weaker. Because of the close temporal relationship between the key-light and the smaller-sooner reinforcer, the stimulus associated with the smaller sooner reinforcer may be more likely to elicit a response. The close temporal relationship may conflict with the purpose of a free-choice trial because of the way the stimulus associated with the smaller-sooner reinforcer has a greater likelihood of eliciting a pecking response than the stimulus associated with the larger-later reinforcer. There is research suggesting that the behavior of pecking in particular may be sensitive to Pavlovian conditioning (i.e., the strength of the stimulus–reinforcer relationship). Lapatto and Lewis (1985) and Poling et al. (1985) found evidence to indicate that Pavlovian conditioning influenced responding on a single-key self-control procedure. For instance, Lapatto and Lewis found evidence that under an omission procedure, response-independent food delivery maintained responding when the choice signal was the key light, but not when the choice signal was a tone. These results suggest the behavior of pecking, initially thought to be under the control of operant contingencies, may be influenced by Pavlovian conditioning. The differential categorization of response type is further supported in the literature investigating behavior under the control of multiple schedules (e.g., Brown and Jenkins, 1968; Schwartz and Gamzu, 1977; Westbrook, 1973). For example Green and Rachlin (1975) studied pigeons on a multiple schedule where the components were a VI 2-min schedule of reinforcement and a VI 2-min + VT 15-s schedule of reinforcement. They found that at points of transition to higher rates of food delivery (e.g., when the VT 15-s was added to one of the components of the multiple schedule), there was a marked and transient increase in responding. Green and Rachlin suggested that through several pairings of key light presentation (the conditioned stimulus) with food delivery (the unconditioned stimulus), the illumination of the key light began to elicit a pecking response (a conditioned response) toward the illuminated key. That is to say, after several transitions from lower to higher rates of food delivery, pigeons’ key pecking was elicited in a direction toward the localized visual cue associated with the higher rate of food delivery (i.e., autoshaping). Since then, researchers have been able to replicate the results with key pecking but not with an alternative response. For example, Green and Holt (2003) expanded on the study by Green and Rachlin by comparing rates of key pecking to rates of treadle pressing under multiple schedules of reinforcement. Green and Holt were able to replicate the results with key pecking, but not with treadle pressing. Under an otherwise identical critical phase of the experiment (i.e., the addition of response-independent food deliveries to one of the components), pigeons treadled less in the presence of the signal associated with the response-independent food deliveries but rates of key pecking increased under the same schedule of reinforcement. The pattern of results observed with treadle pressing are consistent with what would be predicted from operant conditioning, while the results observed with the pecking response are consistent with Pavlovian conditioned response. That is, the key pecking response is influenced by the stimulus–reinforcer relationship while the treadle pressing response is influenced by the response–reinforcer relationship.

The present study investigated whether the influences of Pavlovian conditioning found when using multiple schedules of reinforcement could be extended to a discrete-trial arrangement used to investigate delay discounting with pigeons. To address this question, pigeons completed an adjusting-amount procedure (Green et al., 2004) where the required response type (key pecking and treadle pressing) varied across conditions. We predicted that, regardless of the required response type, increases in delay to food delivery on the larger-later alternative would be associated with an increase in responding on the smaller-sooner alternative (i.e., pigeons would discount the value of a larger-later outcome). We also predicted steeper rates of discounting with key pecking than with treadle pressing based on the hypothesis that the Pavlovian conditioning would increase the likelihood of pigeons pecking the choice alternative with the strongest stimulus–reinforcer relation (i.e., the smaller-sooner alternative). Pigeons also completed an effort-equivalence procedure where completion of a fixed ratio (FR) schedule was required to gain access to food. The FR requirement was established individually for each pigeon using a procedure that produced an estimate of the number of key pecks equivalent to a single treadle press. This was done in an attempt to rule out the potential confound of effort differences between the response requirements (i.e., it was assumed that key pecking requires less effort than treadle pressing for the pigeon). The FR requirement was an important manipulation because any differences in the degree of discounting between key pecking and treadle pressing could also be explained in terms of the effort required to emit the response (e.g., Chelonis et al., 1998; Floresco et al., 2008; Grossbard and Mazur, 1986). If the type of the response requirement (and not effort) was at least partially responsible for differences in discounting, then we would expect to find steeper discounting with key pecking than with treadle pressing, even with the additional response requirement. 2. Method 2.1. Subjects Seven white Carneaux pigeons were housed individually and maintained at 80–85% of their free-feeding body weights by means of supplemental feedings immediately after each daily session. Water and grit were continuously available in their home cages, which were housed in a room on a 12:12 h light/dark cycle. The sex of the pigeons was unknown. At the beginning of the study, pigeons were over 5 years old, and each had some history with a choice procedure and were previously trained to respond to illuminated response keys. None of the pigeons had previous experience with treadle pressing. 2.2. Apparatus Three different test chamber arrangements were used in the present study; key-peck chamber, treadle-press chamber, and effort-discounting chamber. Each chamber (30 cm long by 25 cm wide by 30 cm high) was placed within a light- and soundattenuating enclosure with a ventilating fan running continuously. Chamber floors were stainless steel grids. Each chamber was equipped with a closed-camera video system, which permitted every session to be viewed on a monitor in an adjacent control room. This allowed for ongoing viewing of environmental events and pigeon behavior. Key-peck chambers contained one houselight centered on the rear wall and 2 cm from the top of the chamber. Three response keys, each 2.5 cm in diameter, were spaced equidistantly and located on the front wall 8.5 cm from the top of the chamber. The left

D.D. Holt et al. / Behavioural Processes 98 (2013) 37–43

response key was illuminated red, the middle was illuminated yellow, and the right was illuminated green. A force of approximately 0.15 N was necessary to activate the microswitch, which produced an audible feedback click. Cue lights were located 7 cm directly above the left and right keys and were illuminated red or green respectively. These chambers also contained two 20 mg pellet dispensers (Bio-Serve Dustless Precision Pellets® ); the left dispenser magazine was located 1.2 cm from the left wall, and the right dispenser magazine was located 1.2 cm from the right wall. The distance between the magazines was 11.5 cm and each was located beneath their respective response key. The 20 mg food pellets were used to more precisely control the amount of food delivered for responding on both the adjusting smaller-sooner and the standard larger-later alternatives. The delivery of food pellets was important because the number of pellets delivered for smaller-sooner responding was the dependent variable. Food pellets were delivered at a rate of one every 0.3 s. During pellet delivery, the respective food magazine was illuminated with white light. Two separate pellet dispensers were used to increase discriminability between response alternatives (i.e., smaller-sooner and larger-later). Infrared photodetectors were used to detect whether there were pellets in the magazine. The treadle-press chambers were arranged similarly to the keypeck chamber, but the left and right keys were replaced with treadles. Treadles were modified Med-Associates response levers (ENV-110 M). The treadle itself was an aluminum rectangle (8 cm long by 4 cm wide) attached via a hinge located 2.5 cm above the lever. This arrangement resulted in the treadle being positioned at a 45◦ angle to the floor and extending 6.5 cm into the operant chamber. Each treadle was positioned near the bottom of the panel with the front edge 2 cm from the floor. A minimum force of 0.10 N was required to activate the microswitch, which produced an audible feedback click. In addition, to be counted as an effective response, the pigeons were required to hold the treadle down for a duration of at least 0.5 s. The response duration criterion was used to extinguish pecking of the treadle. Actual treadle-press durations tended to be longer than the response criterion though. As such, it is likely that the only effect of this response requirement was to eliminate pecking of the treadle to access the food. Red and green cue lights were located 7 cm above the left and right treadles, respectively, and feeders were located directly above each of the treadles. Also, above each feeder was a directional light that illuminated each treadle when it was active. This directional light was intended to be an analog to the transilumination of the response keys, signaling the opportunity to make an effective response. The third type of test chamber was an effort-discounting chamber used to determine the number of key pecks equivalent to a single treadle press. The houselight was located in the same location as in the discounting chambers. A single treadle was located on the left side of the front panel with a downward-facing directional light located directly above it. A single response key was located on the right side of the front panel. A cue light was located directly

39

above the response key and could be illuminated red, yellow, or green. The feeder was located directly between the treadle and the response key, and was 9.5 cm from the left wall, 9.5 cm from the right wall, and 3 cm from the floor. 2.3. Procedure The present study used an adjusting-amount procedure (Green et al., 2004) with six different conditions. The order of conditions (and number of sessions to stability) for each pigeon is presented in Table 1. In separate conditions, and for both key-peck and treadle-press response requirements, the standard alternative consisted of 20 pellets delivered after fixed delays of 2 s or 20 s while the adjusting amount always started out at 10 pellets delivered almost immediately (≈0.05 s) but was either increased or decreased depending on how responding was distributed. 2.3.1. Key-peck and treadle-press discounting procedures Pigeons were initially trained to key peck and treadle press using autoshaping and hand-shaping procedures, respectively. The number of training sessions required for each pigeon varied from 2 to 6 sessions for the key-peck response and from 5 to 25 sessions for the treadle-press response. While hand shaping of the treadle-press response was designed to establish stepping on the treadle, a few pigeons learned to peck the treadle instead. Pecking of the treadle was extinguished, as evidenced by video surveillance of each session, by including a response-duration requirement of 0.5 s. Following training, pigeons began the choice task. Sessions were conducted daily, and consisted of 10 blocks of four trials. In the event that all 10 blocks were not completed, the session terminated after 60 min. Each block consisted of two forced-choice trials followed by two free-choice trials. By random determination, the order of the forced-choice trials was either initial presentation of the smaller-sooner option (adjusting amount) followed by the larger-later, option (20 pellets, standard amount), or the reverse. Following the completion of the two forced trials, pigeons were presented with both choice alternatives concurrently (i.e., a free choice trial). The start of a trial was signaled by the illumination of the middle key and houselight. On forced-choice trials, a response to the middle key darkened the middle key and illuminated either the left or right key (or, the left or right treadle was illuminated via a directional light). During free-choice trials, a response to the middle key darkened the middle key and illuminated both the right and left key or the right and left treadles. The right response alternative (green) was associated with the standard amount reinforcement option (i.e., larger-later). Following a response to the larger-later option, the key/treadle was darkened and a signaled delay (i.e., illumination of the green cue light) was initiated (2 s or 20 s). Following the delay, the green cue light was extinguished, the food magazine light was illuminated, and the standard number of pellets was delivered. The food magazine stayed lit for at least 10 s or until all

Table 1 Number of sessions to stability and order of conditions in parenthesis. Pigeon

Key peck discounting 2-s delay

8 10 57 73 156 171 187 Mean a

No data.

84 (1) 92 (3) 25 (1) 50 (2) 56 (1) 102 (4) 46 (4) 65

Treadle discounting

Effort discounting

20-s delay

2-s delay

20-s delay

2-s delay

20-s delay

20 (4) 21 (4) 20 (6) 25 (1) 20 (2) 41 (1) 21 (3) 24

44 (2) 73 (1) 27 (2) 32 (4) 24 (4) 21 (2) 29 (1) 35.7

37 (3) 30 (2) 36 (3) 66 (3) 25 (3) 20 (3) 72 (2) 40.8

30 (6) 20 (5) 26 (4) 89 (6) 21 (6) 56 (5) 29 (6) 38.7

37 (5) 24 (6) 21 (5) a (5) 24 (5) 20 (6) 27 (5) 25.5

40

D.D. Holt et al. / Behavioural Processes 98 (2013) 37–43

the food pellets were consumed (determined by the photo-beam detector). Food presentations were followed by an inter-trial interval (ITI), the duration of which was adjusted so that the total time of each trial was 70 s from the response on either the right or left key. The arrangement for the adjusting-amount option (the left key/treadle paired with a red stimulus light) was identical to the procedure for the standard-amount option, except that pellet delivery was nearly immediate (≈0.05 s) and the number of pellets delivered was determined by the distribution of choices made on the previous block of trials. If the standard amount was chosen on both free-choice trials in the preceding block, then the adjusting amount increased by 1 pellet for the subsequent block. If the adjusting amount option was chosen on both free-choice trials in the preceding block, the adjusting amount was decreased by 1 pellet for the subsequent block. If each alternative was chosen once in the preceding two free-choice trials, the adjusting amount remained unchanged for the subsequent block. At the start of all subsequent sessions of the same condition, the adjusting amount began at what it had been during the last block of the preceding session. Each condition was in effect for a minimum of 20 sessions and terminated when the data from five consecutive sessions had met stability criteria. To evaluate stability, each session was divided into two half-sessions of five blocks each, and the mean number of pellets delivered for a response on the adjusting-amount key/treadle was calculated for each half session. A condition was terminated when the means for all 10 half-sessions were within plus or minus 2 pellets of the grand mean and there was no visible trend. Next, an indifference point was calculated for each condition. The indifference point served as the dependent variable, and was defined as the average number of pellets delivered on the smaller-sooner alternative over the last five sessions. 2.3.2. Treadle press and key peck effort-equivalence assessment procedure Both treadle-press and key-peck response devices were made concurrently available during sessions designed to determine effort-equivalence. Using a discrete-trial choice procedure, each trial began with the simultaneous illumination of the directional light above the treadle and transilumination of the response key. A single treadle-press response constituted a choice for the treadle alternative and always resulted in the immediate delivery of 3 food pellets and the illumination of the pellet magazine for 10 s. The first response to the key light constituted a choice for the keypeck alternative and always resulted in the immediate termination of the availability of the treadle press as an effective response. The response key remained transiluminated until the FR requirement was satisfied, followed immediately by the delivery of 3 pellets of food and the illumination of the pellet magazine for 10 s. A new trial started immediately following the termination of the magazine light. Daily sessions were terminated after the completion of 50 trials or 5 min without a response. The key-peck response requirement for food delivery was set to an FR-1 for the first session and adjusted between sessions in an attempt to find the number of responses that were approximately equivalent to a single treadle press. The allocation of behavior within a session was used as the estimate of preference between the response alternatives. If the majority of the responses (>60%) were key pecks, then the FR requirement for the key-peck alternative was increased by five responses in the next session. The same rule applied when the majority of responses (>60%) were treadle presses. That is, when the treadle was chosen the majority of the time, the FR requirement for the key peck alternative was decreased by five responses in the next session. By titrating the FR requirement on the key peck alternative, we were able to obtain an estimate of effort equivalence between the response alternatives. The effort-equivalence procedure

Table 2 Number of key peck responses equivalent to a single treadle press for each pigeon. Pigeon

# of key-pecks

8 10 57 73 156 171 187

25 10 15 5 15 10 10a

a Responding by Pigeon 187 stabilized at 15 responses, but was not able to complete the task at this response requirement. When the key-peck response requirement was reduced to an FR-10, however, it was able to complete the task.

continued until the allocation of behavior between the two response types within a session was approximately equal (<10% difference in allocation) over two consecutive sessions. For each pigeon, the number of key pecks equivalent to the effort required to make one treadle press (i.e., the indifference point) was set to the terminal FR value. Table 2 shows the FR-equivalence point for each pigeon. At this point, the effort-equivalence discounting task began. 2.3.3. Effort-equivalence discounting procedure The apparatus, procedure, and test parameters for the effortequivalence discounting task were identical to the standard discounting task with the exception that the number of key-peck responses required to obtain food pellets was set to coincide with the treadle/key peck FR-equivalence point. Once a pigeon pecked one of the choice keys, the other choice key was darkened immediately, and it was required to continue pecking that key until the FR schedule had been completed. The 70 s ITI began immediately following the completion of the FR requirement. After the completion of the FR requirement the programmed delay began followed by the delivery of food pellets. In the case that a pigeon made a choice but stopped pecking before completing the FR requirement, the program would remain operating until either the pigeon began pecking on that key once again or 10 min had passed without a response. If 10 min passed with no responding on the selected key or a total of 60 min elapsed, the session was terminated. 3. Results In the key-peck, treadle-press, and effort-equivalence discounting tasks, increases in the delay to the 20-pellet outcome from 2 s to 20 s reliably resulted in decreases in indifference points. This pattern of results is consistent with the devaluation of outcomes associated with delay discounting, and is evident across all three discounting tasks. Fig. 1 shows the relation between indifference points in all three conditions for each pigeon. Individual data are presented in separate panels, with the upper left panel showing the overall mean across pigeons for each condition. The exception being Pigeon 73, who failed to complete the effort equivalence procedure in the 20-s delay condition. The indifference points in the 2-s and 20-s delay conditions of the key-peck discounting task (8.43 and 1.02 pellets, respectively) was less than the corresponding discounting rates in the treadlepress condition (19.66 and 3.24 pellets, respectively). This pattern of results is evident in each pigeon in almost every condition. The one exception was the 20-s condition for Pigeon 8, where slightly lower rates of discounting were observed for key pecking than treadle pressing. The relative differences between key pecking and treadle pressing at the 2-s and 20-s delays were not equivalent. The relative difference at the 2-s delay was, on average, 11.23 pellets greater with treadle pressing than the key pecking, whereas, at the 20-s delay the indifference point was, on average, 2.24 pellets greater with treadle pressing than key pecking.

Indifference Point (number of immediate reward pellets equal to the 20 pellets)

D.D. Holt et al. / Behavioural Processes 98 (2013) 37–43

20 18 16 14 12 10 8 6 4 2 0 20 18 16 14 12 10 8 6 4 2 0 20 18 16 14 12 10 8 6 4 2 0 20 18 16 14 12 10 8 6 4 2 0

Mean

41

P73

Key Peck Treadle Press Key Peck - Effort

2 seconds

20 seconds

P8

P156

P10

P171

P57

P187

2 seconds

20 seconds

Delay (in s) to 20-pellet alternative Fig. 1. Mean number of immediate food pellets equal in value to the 20-pellet reward (i.e., the indifference point) when it was delivered after 2 s or 20 s. Indifference points for key pecking (gray bar), treadle pressing (black bar), and effort-equivalent key pecking responses (diagonal stripe) are shown for each pigeon. Aggregate data are presented in the upper-left panel.

Table 2 shows the number of responses at which each pigeon was indifferent on the effort-equivalence task. The number of key pecks equivalent to a single treadle press ranged from 5 to 25 pecks. These values were then used as the FR value in the effort-equivalence discounting task. For instance, when Pigeon 8 completed the effort-discounting task, 25 key pecks were required to gain access to the food. The exception was Pigeon 187 who was indifferent at 15 key pecks but was tested using an FR-10 schedule. This was done because Pigeon 187 failed to complete a full session of the effort-discounting task within the time constraints (less than 10 min between responses) when the response requirement was set at 15 pecks. As a result, we reduced the ratio requirement for Pigeon 187 to a FR-10 schedule where the pigeon was able to complete daily sessions. As seen in Fig. 1, with increased effort required on the key peck response alternative, the difference between key pecking and

treadle pressing remained. Again, smaller indifference points were obtained in the effort-equivalence-discounting task than in the treadle-press discounting task. There was little variation between the indifference points obtained with the effort-equivalence and original key-peck discounting tasks at the individual level, as illustrated in the nearly equivalent indifference points observed at the aggregate level. 4. Discussion The present study investigated whether the form of the required response influences discounting behavior on a typical adjustingamount discounting task. We predicted that: (a) discounting would occur for both response types; (b) the key-peck response would show smaller indifference points (i.e., less discounting) compared to the treadle-press response; and (c) when we attempted to equate

42

D.D. Holt et al. / Behavioural Processes 98 (2013) 37–43

the effort involved in treadle pressing and key pecking, the indifference points observed with the two responses would remain. We based our first prediction on previous research showing that pigeons discount delayed reinforcers (e.g., Green et al., 2004; Mazur, 1987). Our second prediction was based on specific stimulus–reinforcer relations between key-light illumination and the presentation of food pellets (e.g., Gamzu and Williams, 1971; Gamzu and Schwartz, 1973). Because the delay between the response (key peck) and reinforcer (food pellet) is shorter for the smaller-sooner reinforcer than the larger-later reinforcer, the stimulus–reinforcer relation is stronger for the smaller-sooner option, which potentially elicits responding to the signal for that option. Important here is that while the same contiguous arrangements exist for key pecking and treadle pressing, the elicited response of pecking would have direct influence when the pigeon is key pecking but not when treadle pressing. Because the Pavlovian effect is on key pecking, and these key pecks would be directed toward the key with the strongest stimulus–reinforcer relation (i.e., the smaller-sooner alternative), thus making the pigeon appear to discount more steeply when key pecking than treadle pressing. The third prediction was also based on the idea that stimulus–reinforcer relations elicit pecking on the smaller-sooner response key. If elicited pecks, and not effort, are responsible for the differences between key pecking and treadle pressing, then equating the effort of key pecking to treadle pressing should have little effect. The results confirmed each of our predictions. First, although not enough delay conditions were run to establish discounting functions, it is clear from the current data that the present value of the delayed outcome decreased as the delay to the larger outcome increased from 2 s to 20 s. Next, the indifference points for the keypeck response were smaller than the indifference points for the treadle-press response for all pigeons in both the 2-s and 20-s conditions. Finally, when the effort required to obtain reinforcement was increased for key pecking, the obtained indifference points remained much smaller when compared to treadle pressing. The present results suggest that the steeper delay discounting demonstrated in the key-peck conditions of the current study can be explained, in part, by the nature of the key-peck response. Given this unintended influence and the danger of tacitly assuming that discounting is purely the result of basic operant procedures, our results suggest that researchers should consider the potential interaction of the required response and the delay-discounting task. This becomes particularly important when attempting to establish cross-species generality of delay discounting and its subsequent mathematical description where free parameters are intended to capture psychological processes. That is, the strength of the stimulus–reinforcer relation may also be well described by a hyperbolic function, which, if true, would effectively mask the Pavlovian influences on behavior. In effect, the present data suggest that we might be quantifying behavior that is well described by the same mathematical function but under some circumstances, caused by separate processes. Alternatively, it could be argued that differences between the key-peck and treadle-press conditions are the result of increasing the delay to both reinforcers. For example, in the second experiment by Calvert et al. (2011) the researchers found shallower rates of discounting when both outcomes were delayed. Thus, one explanation as to why the pigeons appeared to discount less steeply in the effort-equivalence condition may be due to the extra delay when treadle pressing. Although we do not have a measure of response latency on each trial, one might assume that the overall time to food access was longer when treadle pressing than key pecking. Therefore, the shallower rates of discounting in the treadle-press condition may be due to differences in overall amount of time until the pigeon receives the reinforcer. Alternatively, it could be argued that Calvert et al. found shallower rates of

discounting in the second experiment than in the first experiment because they broke the contiguous relationship between the stimulus light (CS) and the reinforcer (US). The same explanation may apply to the shallower rates of discounting in the effort-equivalence procedure as well. By increasing the number of responses required to receive the reinforcer, we inevitably added a delay to both conditions. However, if one were to consider the effort-equivalence discounting task an analog to the treadle-press condition in that it takes longer to complete the FR requirement in the effort-discounting task, it appears that delaying both alternatives may not have been a factor in this case. We observed similar rates of discounting on the standard-discounting and effort-discounting tasks. However, it is possible that the time to complete the FR schedule may be closer to the FR-1 schedule than the overall time required for a treadle press. That is, even though more effort was required to complete the effort-equivalence procedure (anywhere from 5 to 25 key pecks), pigeon may have been able to emit 5–25 key-pecks much faster than the time needed to step on a treadle. It is also possible that the effective delay to reinforcement may be the delay following the final response in the FR requirement rather than the total time to reinforcer delivery. But because we do not have measures of either response latency or the time it took to complete the FR requirement, the handling of these alternative explanations are speculative and should be addressed in future research. The effects of Pavlovian conditioning on pigeons’ key-peck response in choice situations resulting in a bias toward the smallersooner alternative has been established in the literature, but under different procedural arrangements (e.g., Lapatto and Lewis, 1985; Poling et al., 1985). A handful of studies have found that, under otherwise identical situations, Pavlovian conditioning does not directly influence treadle pressing. For example, Green and Holt (2003), found increases in rates of key pecking and decreases in rates of treadle pressing at points of transition to higher rates of food delivery on multiple schedules. This suggests that the form of the required response, in combination with stimulus–reinforcer relations, may influence behavior in choice situations. Indeed, previous findings involve responding under a two-component multiple schedule, but stimulus arrangements remained similar across procedures. In the two alternative discrete-trial arrangements, there is still a signal (CS) associated with the delivery of food, and the strength of the behavior (CR) is likely influenced by the temporal arrangement with food delivery (US). Therefore, if one response has a stronger association with the signal than an alternative response, we could predict the response (CR) that would be directed toward the signal. If the signal were also located on the response key, we would predict elicited responding toward that response key. With treadle pressing, the effect of Pavlovian conditioning does not directly influence the behavior. As such, we would expect key pecking, but not treadle pressing, to be influenced in a systematic way under our testing arrangement. Despite procedural differences, we would still expect to see a strong response bias toward the stronger CS, which in the present case is the smaller-sooner response alternative. Important to note here is that, under a concurrent nonindependent VI–VI schedule arrangement, Chelonis and Logue (1996) found that pigeons were not differentially sensitive to delays or reinforcer magnitude between key pecking and treadle pressing. While their finding suggests no effect of response type in a choice situation, one notable difference between the present study and the Chelonis and Logue study is that, in the latter case, response rates for key pecking and treadle pressing were assessed at steady-state. Because the Pavlovian effects on responding appear to be transient and relatively brief under continuous CS presentation (the effect appears to last less than 10 s; Green and Rachlin, 1975; Green and Holt, 2003), the Chelonis and Logue

D.D. Holt et al. / Behavioural Processes 98 (2013) 37–43

findings may represent responding that is under the influence of operant contingencies, and any effect of Pavlovian conditioning may not be captured during steady-state performance. When Mazur (2012) investigated whether a pre-trial requirement of 1 or 40 responses before each trial would affect the choice of rats and pigeons, he found that the two species were quite different. The pre-trial response requirement influenced the choice of rats but not pigeons. Mazur attributed the results from pigeons as evidence for a molecular approach to choice. Because the rats’ choice was influenced by pre-trial response requirements, and simulations seemed to rule out a purely molecular approach, Mazur suggested that the rats’ behavior might be under joint control of molecular and molar variables. However, more empirical tests are needed to determine whether molar variables are necessary to fully account for the results obtained for rats. While the data from the current study do not address the important questions regarding molar/molecular influences, they do indicate that Pavlovian effects may have influenced pigeons’ responding, which could have masked any effects of the pre-trial response requirement. In addition to the manipulations suggested by Mazur, we suggest the use of a treadle-press response requirement to explore this possibility. In addition to the research needed to more fully rule out effort and time differences as explanations for the observed results, we would suggest first, extending the current study to include a fuller range of delays for each response requirement and second, exploring the efficacy of the effort-equivalence procedure used. Additional delays would allow for a comparison of the mathematical form of the discount functions for both response types. It may be that there is one mathematical formula that predicts discounting for a Pavlovian response better than discounting for an operant response. Next, when examining the effort-equivalence procedure, future researchers could examine the number of responses equal to one treadle press under steady state responding. While in the present study, the researchers did wait until it appeared that the pigeons reached a steady state, future researchers may want to set a minimum number of sessions similar to what the discounting procedure uses. For example, the present study set a minimum of 20 sessions before responding was considered stable; the researchers could also set the same 20-session criterion for the effort-equivalence procedure. Also, after the completing the FR-schedule or the treadle press, pigeons received three food pellets. While this amount is similar to the average number of pellets received at the 20-s delay, the indifference point at the 2-s delay was much larger (19.66 pellets). Future researchers may want to determine if the same indifference point holds across varying reinforcer magnitudes. As a final note with the effort equivalence procedure, because the pecking response was similar to the response required in the discounting task (i.e., the same stimulus relation between the key light and the food pellet), the preferences observed in the effort equivalence procedure may also be influenced by the Pavlovian nature of the pigeon’s pecking response. In conclusion, while this study represents the first demonstration of systematic differences in discounting as a function of response requirement in pigeons, it is clear that additional research is needed to more fully understand the observed differences. That is, the observed differences could be due to differences in effort between the required responses and/or due to differences in added delay to both the immediate and delayed rewards. The results also suggest, however, that rather than assuming the degree to which organisms discount delayed outcomes reflects a characteristic like impulsivity, researchers may also need to consider the putative influence of Pavlovian processes associated with response type on choice. Again, while the influences of Pavlovian processes may not be the only additional factor to consider here, the integration

43

of behavioral ecology and behavioral economics may be required for a fuller understanding of discounting and behavior in choice situations. Acknowledgments This study was supported by the University of Wisconsin-Eau Claire Office of Research and Sponsored Programs. The authors would like to thank the members of the pigeon laboratory group at the University of Wisconsin-Eau Claire for their assistance in running the experiment. Thanks are also due to Bryan K. Saville and Joel Myerson for helpful comments and suggestions regarding the manuscript. Portions of the work were presented at the Behavioral Economics: From Demand Curves to Public Policy Conference, Chicago, IL, March, 2011 and at the Mid-American Association for Behavior Analysis Convention, Bloomingdale, IL, October, 2011. References Brown, P.L., Jenkins, H.M., 1968. Auto-shaping of the pigeon’s key-peck. J. Exp. Anal. Behav. 11, 1–8. Calvert, A.L., Green, L., Myerson, J., 2011. Discounting in Pigeons when the choice is between two delayed rewards: implications for species comparisons. Front. Neurosci. 5, 96. Chelonis, J.A., Logue, A.W., 1996. Effects of response type on pigeons’ sensitivity to variation in reinforcer amount and reinforcer delay. J. Exp. Anal. Behav. 66, 297–309. Chelonis, J.A., Logue, A.W., Sheehy, R., Mao, J., 1998. Effects of response effort on self-control in rats. Anim. Learn. Behav. 26, 408–415. Floresco, S.B., Tse, M.T.L., Ghods-Sharifi, S., 2008. Dopaminergic and glutamatergic regulation of effort- and delay-based decision making. Neuropsychopharmacology 33, 1966–1979. Freeman, K., Green, L., Myerson, J., Woolverton, W., 2009. Delay discounting of saccharin in rhesus monkeys. Behav. Process. 82, 214–218. Gamzu, E., Schwartz, B., 1973. The maintenance of key pecking by stimulus response-independent food presentation. J. Exp. Anal. Behav. 19, 65–72. Gamzu, E., Williams, D.R., 1971. Classical conditioning of a complex skeletal response. Science 171, 923–925. Green, L., Fry, A.F., Myerson, J., 1994. Discounting of delayed rewards: a life-span comparison. Psychol. Sci. 5, 33–36. Green, L., Holt, D.D., 2003. Economical and biological influences on key pecking and treadle pressing in pigeons. J. Exp. Anal. Behav. 80, 43–58. Green, L., Myerson, J., 2004. A discounting framework for choice with delayed and probabilistic rewards. Psychol. Bull. 130, 769–792. Green, L., Myerson, J., Holt, D.D., Slevin, J.R., Estle, S.J., 2004. Discounting of delayed food rewards in pigeons and rats: is there a magnitude effect? J. Exp. Anal. Behav. 81, 39–50. Green, L., Rachlin, H., 1975. Economic and biological influences on a pigeon’s key peck. J. Exp. Anal. Behav. 23, 55–62. Grossbard, C.L., Mazur, J.E., 1986. A comparison of delays and ratio requirements in self-control choice. J. Exp. Anal. Behav. 45, 305–315. Jetz, W., Freckleton, R.P., McKechnie, A.E., 2008. Environment, migratory tendency, phylogeny and basal metabolic rate in birds. PLoS ONE 3, e3261. Lapatto, D., Lewis, P., 1985. Contributions of elicitation to measures of self-control. J. Exp. Anal. Behav. 44, 69–77. Lucas, G.A., Deich, J.D., Wasserman, E.A., 1981. Trace autoshaping: acquisition, maintenance, and path dependence at long trace intervals. J. Exp. Anal. Behav. 36, 61–74. Mazur, J.E., 2012. Effects of pre-trial response requirements on self-control choices by rats and pigeons. J. Exp. Anal. Behav. 97, 215–230. Mazur, J.E., 2005. Effects of reinforcement probability, delay, and response requirements on the choice of rats and pigeons: possible species differences. J. Exp. Anal. Behav. 83, 263–279. Mazur, J.E., 1987. An adjusting procedure for studying delayed reinforcement. In: Commons, M.L., Mazur, J.E., Nevin, J.A., Rachlin, H. (Eds.), Quantitative Analysis of Behavior: The Effect of Delay and of Intervening Events on Reinforcement Value. Erlbaum, Hillsdale, NJ, pp. 55–73. Poling, A., Thomas, J., Hall-Johnson, E., Picker, M., 1985. Self-control revisited: some factors that affect autoshaped responding. Behav. Process. 10, 77–85. Rachlin, H., 2006. Notes on discounting. J. Exp. Anal. Behav. 85, 425–435. Rodriguez, M.L., Logue, A.W., 1988. Adjusting delay to reinforcement: comparing choice in pigeons and humans. J. Exp. Psychol. Anim. Behav. Process. 14, 105–117. Schwartz, B., Gamzu, E., 1977. Pavlovian control of operant behavior: an analysis of autoshaping and its implications for operant conditioning. In: Honig, W.K., Staddon, J.E.R. (Eds.), Handbook of Operant Behavior. Prentice-Hall, Englewood Cliffs, NJ, pp. 53–97. Tobin, H., Logue, A.W., 1994. Self-control across species (Columbia livia, Homo sapiens, and Rattusnorvegicus). J. Comp. Psychol. 108, 126–133. Westbrook, R.F., 1973. Failure to obtain positive contrast when pigeons press a bar. J. Exp. Anal. Behav. 20, 499–510.