Within-session discrimination between times of reinforcers in the minutes range by pigeons

Behavioural Processes, 30 (1993) 0 1993

Elsevier Science

BEPROC

283

283-290

Publishers

B.V. All rights reserved

0376-6357/93/$06.00

00501

Within-session

discrimination

between times

of reinforcers in the minutes range by pigeons C.M.S. Plowright University

of Ottawa,

Ottawa,

Ontario,

Canada

(Accepted 16 June 1993)

Abstract This experiment

tested whether pigeons could discriminate

between a large reinforcer

delivered at 6 min (early) vs. 16 min (late) within a 20 min session. was used in which an early reinforcer was delivered in one context, another. Rate of responding on the first link of a chain schedule reinforcer condition. Furthermore, in probe sessions during which delivered, the discrimination

Key words:

Time;

A within subject design and a late reinforcer in was higher in the early no large reinforcer was

was maintained. Implications for foraging are discussed.

Pigeon; Foraging; Discrimination

Introduction In foraging situations, animals are known to be sensitive to several kinds of temporal intervals occuring sequentially within a foraging session. For instance, in a patch choice situation, they respond to differences in travel time to a patch (Cuthill et al., 1990) as well as delay to food within a patch (Ydenberg, 1984). In diet selection, where animals search for food and then must decide whether to accept or reject encountered prey items, foragers respond to manipulations in search time (Shettleworth and Plowright, 1992) and food handling time (see Shettleworth, 1988). In natural conditions, it may also be advantageous for animals to be able to time simultaneous events independently (Meek and Church, 1984). Recent research has suggested that foragers may indeed be sensitive to

Correspondence Kl N 6N5.

to: C.M.S.

Canada.

Plowright,

School of Psychology,

University

of Ottawa,

Ottawa,

Ontario,

284

temporal variables over and above the individual delays that make up a foraging session. For instance, in diet selection, choice depends not only on search time etc., but also on session length itself, in accordance with the predictions of foraging theory (Lucas, 1985, 1990). The effect of session length, or ‘time horizon’, has been observed with sessions not only on the order of seconds (Lucas, 1987) but also several minutes (Barnard and Hurst, 1987; Yoerg and Kamil, 1988; Plowright and Shettleworth, 1991). In these time horizon studies, session length is confounded with total amount of reinforcement. However the results of the studies do raise the general question, addressed here, of whether animals can discriminate between ‘higher order’ intervals in the range of minutes when these occur within the context of a foraging session. Unfortunately the literature in operant psychology cannot provide a definitive answer to this question, since it has been almost solely concerned with discrimination of very short time intervals, in the order of a few seconds (e.g. Fetterman and Killeen, 1992) and unusually up to 1 or 2 min (Dreyfus et al., 1988). Exceptions include operant studies of patterns of responding on very long FI schedules: for example, Dews (1978) used a FI 1000 s schedule with rhesus monkeys. Mellgren et al. (I 983) reported control of running speed of rats by intertrial intervals in the range of 1 to 20 min when the intertrial interval predicted trial outcome of reinforcement or nonreinforcement. Also, work on suppression of saccharin intake by rats depending on delay of future food (Lucas et al., 1988, 1990) has shown anticipation at a delay of 32 min. These studies of long delays, however, do not typically place any other concurrent demands of temporal processing upon the animals. Although Meek and Church (1984) have provided evidence that rats can time not only individual time intervals within a trial, but also trial length itself, their trial length was only 50 s. In order to relate the results of this experiment to previous research on time horizon in diet selection (Plowright and Shettleworth, 1991), a schedule of reinforcement was given to pigeons which was a simplified version of the diet selection task in which time horizon effects were observed. No choice was given, and so the situation was a chain schedule of reinforcement for 20 min: VI 5 s on one key followed by FI IO s on the other. Within the session, a large reinforcer was delivered either early or late (6 min vs. 16 min into the session). The dependent measure was the rate of responding on the first link of the chain schedule. If the birds could discriminate between the times of delivery of the reinforcers, then a difference in the patterns of responding throughout the session would be expected. Since session length was kept constant, any difference in the pattern of responding could not be attributed to a difference in total amount of food. Toward the end of the experiment, probe sessions were administered during which no large reinforcer was delivered at the usual time. In this way, it could be determined whether the birds expected the reinforcer or merely reacted to its presence.

Materials

and Methods

Subjects Eight White King pigeons were used. Three were naive and five had previous experience in an experiment on diet selection. Their weight was restricted to 85 + 2% free feeding body weight. One bird began the experiment 64 days after the rest: it replaced one subject

285

which consistently failed to respond, even though it had been successfully Water and grit were continuously available except during testing.

pretrained.

Apparatus Four operant chambers measuring 32.5 X 32.5 X 32 cm were used. Each was enclosed in a separate sound- and light-attenuating chamber. White noise masked extraneous noises. A house light was mounted in the upper left corner of each chamber. Three operant keys measuring 2.5 cm in diameter were located 5.5 cm from the top of one of the two side panels. Only the left key and the center key were used. The center key could be transilluminated with white light and the side key with green. Each key could be triggered by a force of 2.5 N. A feeder opening measuring 5 x 5 x 3 cm deep was located 20 cm below each side key. Pigeon pellets could be dispensed into the feeder opening. The experimental contingencies were controlled and the subjects’ responses were recorded by computer using MED-PC language for operant research (Tatham and Zurn, 1989).

Pretraining The birds were trained to keypeck on each key using a combination of autoshaping and handshaping. Pretraining consisted of 2 phases. Sessions lasted 20 min. The first phase consisted of 4 sessions. Subjects were reinforced on a FI 2 s schedule on the side key. After the first response following 2 s, four 20 mg BioServ pellets were delivered and the feeder light was illuminated for 4 s. The FI requirement was increased on the days following the first to FI 4, FI 6 and FI 10 s. In the second phase, which lasted between 4 and 7 sessions, responding to the center key on a VI 5 s was required before the side key could be illuminated so that the FI requirement could be fulfilled. The 12 time intervals making up the VI 5 s were generated according to the method of Fleschler and Hoffman (1962). One interval was randomly selected on each trial. Once the side key was illuminated, the center key was turned off.

Procedure The experimental sessions were the same as those in the second phase, except that part way into the session, a large reinforcer (12 pellets) was delivered after completion of the first VI 5 s on the center key following a certain time interval. In ‘early reinforcer’ sessions, this time interval was 6 min, and ‘late reinforcer’ sessions, it was 16 min. The feeder light duration for the large reinforcer was 7 s. No FI on the side key was required after delivery of the large reinforcer, but a new trial began with a VI on the center key. Each of the four chambers was distinctively marked. The sides were lined with colored construction paper and the house light was covered with a piece of acetate of matching color. One chamber was colored yellow, one blue, one red and one green. Each day the birds were tested twice: once around IO:00 am and once around 3:00 pm. Half the birds were tested in the yellow and blue chambers with 2 birds receiving early reinforcer in the blue chamber and late reinforcer in the yellow, and 2 receiving early in

286

the yellow and late in the blue. The other half were similarly tested in the green and red chambers. Every second day early sessions were first in the day and late sessions were second. On alternate days, late reinforcer sessions were first and early reinforcer sessions were second. On any one day 4 birds started with late and 4 started with early. The experiment consisted of 4 blocks of 16 days each. After that, the experiment continued for 20 days during which probe sessions were interspersed with regular sessions. During these that no extra administered sessions, and

probe sessions, the procedure was the same as in the regular sessions, except large reinforcer was delivered at the usual time. Eight probe sessions were in the chamber in which the birds usually received the early reinforcer eight in which they usually received the late reinforcer sessions.

Design and statistics The data were analyzed using CLIM (for details of the model fitting procedure, see Baker and Nelder, 1978). The two independent variables were reinforcer condition (early or late) and time within a session which was entered into the model as a continuous variate. The effect of individual birds was removed before testing for the effects of interest. The results for the four blocks were first analyzed separately in order to track the development of any effects over the course of the experiment. In order to examine the rate of responding to the centre key, the responses to the centre key during the VI 5 s were totalled in 2 min time bins for each session and each bird. Since number of keypecks is a count, the data were not normally distributed. Rather, they were positively skewed, with a mode near or at zero. Hence a Poisson error term was specified in the linear model.

Results

Regular sessions Figure 1 shows the average number of responses to the centre key during the VI 5 in each of the ten 2-min time bins for the early and late reinforcer conditions. In the statistics reported below, a level of significance of 0.05 was adopted. The pattern of results for the first two blocks resembled each other and were grouped together, as were the results for the last two blocks. For Blocks 1 and 2, which are combined in Fig. la, the effect of reinforcer condition depended on time within the session (xc,) 2 = 103). This interaction was primarily due to the large increase in rate of responding following the large reinforcer in each reinforcer condition. A quadratic term for time within the session was significant (xfi, = 237), showing that the effect of time within session was non-linear, and this term also interacted with reinforcer condition <$, = 74). In Blocks 3 and 4 (Fig. 1 b), the birds responded at a higher rate in the early reinforcer condition (main effect of reinforcer condition xC1) 2 = 2711, and this effect was more pronounced for the first half of the session (interaction of reinforcer condition by time within the session $,, = 50). A quadratic term for time within the session was again

287

‘61a

I6

b

14

y

“-

0

‘*-

f

lo-

z y

a-

'

6-

EARLY .I. 1.1 ,' ,....i....J..1 ...I. ,v

B

m

4-

r

2:BLOCKS3gl

BLOCKS l&2

t

:

111l181111111111111S 4 6 8 10 12 14 16 18 20 ' 2

0

TIME (MIN)

TIME (MIN)

Fig. 1. Average number of keypecks to the centre key for early and late reinforcer conditions throughout the session. Standard error bars are given. (a) Blocks 1 and 2; (b) Blocks 3 and 4.

significant (xg, = 308), showing that the effect of time within session was non-linear, this term interacted with reinforcer condition (x:1, = 11).

Probe

and

sessions

The differences reported for Fig. 1 do not necessarily show that the birds anticipate the large reinforcers delivered at 6 vs. 16 min, but may be entirely due to the birds’ reactions to the large reinforcers. During probe sessions, no large reinforcers were delivered at the usual time, and so any differences between reinforcer conditions can not be due to a reaction to the reinforcers themselves. The results for the probe sessions are shown in Figure 2. The discrimination evident in the regular sessions is maintained: the rate of responding was higher in the early reinforcer condition (main effect of reinforcer condition

PROBE ’

2

4

6

6

SESSIONS

10 12 14 16 16 20

TIME

(MIN)

Fig. 2. Average number of keypecks to the centre key for probe sessions in the boxes where pigeons usually received early and late reinforcers.

Standard error bars are given.

288

,Y;, = 105>, and this effect was more pronounced for the first half of the session (interaction of reinforcer condition by time within the session xh, = 17). The quadratic term for time within the session was again significant (xc,, * = 81) and this term again interacted with reinforcer condition (,y(,, 2 = 20). For 6 of the 8 birds, the overall average number of responses was greater in the context in which the early reinforcer was usually delivered. For one of the other two birds, the effect was in the opposite direction, and for the other the rate of responding was at floor level. No difference between the early and late sessions in the total number of reinforcers was evident: there were 44.7 reinforcers (S.E. = 1.31) in the early sessions and 46.3 (S.E. = 1.22) in the late sessions. Therefore the difference in the pattern of responding could not be attributed to a difference in total amount of food. The difference in rate of responding between the early and late sessions is also unlikely to be due to satiation: the overall rate of responding was higher in the early condition than in the late condition when these were administered first in the day (I 1.6 vs. 9.4 keypecks, respectively) as well as when they were administered second in the day (12.1 vs. 10.5 keypecks, respectively).

Discussion In this experiment, the pattern of responding on the VI 5 s on the centre key depended on whether a large reinforcer was delivered at 6 min or 16 min. Furthermore, the difference persisted in the absence of these large reinforcers showing that the birds did not merely react to the presence of the large reinforcer. Since session length was kept constant, the difference could not be attributed merely to differences in total amount of reinforcers or responses. These results show that animals are capable of discrimination between events separated by a time interval in the minutes range occuring within a schedule of reinforcement. They suggest that in our previous study (Plowright and Shettleworth, 1991) on the effect of time horizon on choice in a diet selection task (where the task was in many respects similar to the situation in this experiment), pigeons may be able to discriminate between ‘higher order variables’ such as session length and do not necessarily rely on total amount of reinforcer as a cue to session length. Although a discrimination between early and late reinforcers was evident, the difference in rate of responding is only clear at the beginning of the session. In the probe sessions, no peaks in responding were evident at the times where the birds usually received the large reinforcers. Changes in experimental design might lead to response profiles which are more strongly related to the temporal positioning of the large reinforcers. A successive-conditions design in which the birds were only exposed to one condition at a time for a block of days would eliminate the need for a conditional discrimination (e.g. green box - early, red box _ late). More accurate timing has also found when other responses (e.g. perching, Jasselette et al., 1990) are used instead of keypecking. One possible explanation for the results obtained is that the large reinforcer presented late in the session was less discriminable from other reinforcers delivered in the session compared to the large reinforcer presented early. This seems unlikely in view of the increase in rate of responding following the large reinforcer late in the session at the beginning of the experiment (Fig. 1 a>: if anything, the response is greater than the response to the large reinforcer presented early, which would suggest that the late reinforcer was at least equally discriminable from the other reinforcers delivered in the session.

289

Assuming

that the pigeons could time the components of the chained schedule, the

present results are consistent with Meek and Church’s (1984) contention that animals can process temporal information from two internal clocks simultaneously. However, another possible way in which the pigeons could discriminate between the two times would be by keeping record of their own responses or the food items they obtain: in one environment the large reinforcer follows a few responses/food items, and in the other it follows more. Certainly, satiation cues have been shown to serve as discriminative stimuli (Davidson et al., 1992). However, a conditional discrimination would be required here, where a certain number of food items or responses predicts early reward in one context, and a different number predicts late reward in a different context. Further research might be aimed at determining whether the mechanism underlying the temporal discrimination is one requiring an internal clock (Church and Gibbon, 1982) or not. The discrimination reported here could perhaps be taken as an instance of time discounting: rewards in the near future are valued more than rewards in the distant future. One functional explanation of time discounting is that future rewards are more uncertain, i.e. the animal’s foraging is more likely to be interrupted before those rewards can be collected (Kagel et al., 1986; see McFarland and Houston, 1981, for an alternative explanation). Even though interruptions were not part of this experiment, discrimination of long time intervals as reported here may enable a response to important temporal variables, such as possible interruptions, in other foraging situations.

Acknowledgements This research was supported by an operating grant from the Natural Sciences and Engineering Research Council of Canada (NSERC). I thank Heather Duggan and Liz Parkin for technical assistance and Sara Shettleworth for comments.

References Baker, R.J. and Nelder, R.A., Modeling. Oxford:

1978.

The GLIM

Numerical Algorithms

Barnard, C.M. and Hurst,

J.L.,

1987.

araneus L. Animal Behaviour,

System,

Release 3: Generalised

Linear

Interactive

Group.

Time constraints

and prey selection in common shrews

Sorex

35, 1827-1837.

Church, R.M. and Gibbon, J., 1982.

Temporal generalization.

J. Exp. Psychol.:

Anim. Behav. Process.,

8: 165-186. Cuthill,

l.c.,

Kacelnik, A., Krebs, J.R., Haccou, P. and Iwasa, Y., 1990.

The effect of recent experience on foraging decisions. Anim. Davidson,

T.L,

Flynn,

discriminative Dews,

P.B.,

F.W.

stimuli.

1978.

and Jarrard, L.E.,

J. Exp. Psychol.:

Studies

1992.

Anim.

of responding

Starlings exploiting

Potency of food deprivation

Behav. Process.,

under fixed-interval

patches:

Behav., 40: 625-640. intensity

cues as

18: 174-l 81.

schedules

of reinforcement:

II. The

scalloped pattern of the cumulative record. J. Exp. Anal. Behav., 29: 67-75. Dreyfus,

L.R.,

Fetterman,

J.C.,

Smith,

L.D.

relations by pigeons. J. Exp. Psychol.: Fetterman, J.C. and Killeen, Exp. Psychol.: Fleschler,

Anim.

P.R.,

1992.

Behav. Process.,

M. and Hoffman,

Anal. Behav., 5: 529-530.

H., 1962.

and Stubbs,

Anim.

D.A.,

Behav. Process.,

Time discrimination

1988.

Discrimination

of temporal

14: 349-367.

in Columba

livia and Homo sapiens. J.

18: 80-94.

A progression for generating variable-interval

schedules. J. Exp.

290 Jasselette, P., Lejeune,

H. and Wearden,

timing. J. Exp. Psychol.: Kagel, J.H.,

Green,

horizons Lucas,

J.H.,

1990.

Behav. Process.,

L. and Caraco, T.,

adaptation? Anim. Lucas, G.A.,

Anim.

1986.

The perching response and the laws of animal

16: 150-I

When

Cawley,

G.A.,

D.J. and Timberlake,

Timberlake, Anim.

1985.

W.,

Cawley,

W.,

Lucas, J.R., 1987.

constraints

126:

1988.

Anticipatory

or

and Drew,

contrast as a measure of time

Anim. Learn. Behav., 16: 377-382. J.,

1990.

Anticipation

of future

food:

predictions from different constraints.

680-705.

1990.

Anim.

548.

Time

scale and diet choice decisions.

Mechanisms of Food Selection, pp. 165-l

16: 169-177.

on diet choice: different

The influence of time constraints on diet choice of the great tit, Parus major.

Behav., 35: 1538-l

McFarland,

D.J.

Behav. Process.,

Time

Am. Naturalist,

Lucas, J.R.,

constraint

and facilitation of saccharin intake depending on the delay and type of future food. J.

Exp. Psychol.: Lucas, J.R.,

the future:

Behav., 34: 271-283.

in the rat: Some methodological determinants.

Suppression

61.

foragers discount

NATO

In R.N.

Hughes

(Editor),

ASI Series, Series A: Life Sciences, Berlin,

Behavioural

Springer-Verlag,

85. D.J. and Houston,

A.I., 1981.

Quantitative Ethology: The State Space Approach. Pitman,

London. Meek, W.H.

and Church,

Behav. Process., Mellgren,

R.L.,

Mays, M.Z.

temporal stimuli Plowright,

C.M.S.

tion task. Anim. Shettleworth,

R.M.,

1984.

Simultaneous

temporal

processing. J. Exp. Psychol.:

Anim.

IO: 1-29. and Haddad, N.F.,

lasting for minutes.

and Shettleworth,

S.J., 1991.

Learn. Behav., 19: 103-I

S.J., 1988.

1983.

Learn. Motiv.,

Discrimination

and generalization

by rats of

14: 75-91.

Time horizon

and choice by pigeons in a prey-seiec-

12.

Foraging as operant behavior and operant behavior as foraging: What have

we learned? Psychol. Learn. Motiv., 22: l-49. Shettleworth, Psychol.: Tatham,

T.A.

S.J. and Plowright, Anim.

C.M.S.,

Behav. Process.,

and Zurn,

Behav. Res. Methods, Ydenberg, R.C., 1984.

K.R.,

1989.

Instruments

1992.

How pigeons estimate rates of prey encounter. J. Exp.

18: 219-235. The

MED-PC

Computers,

experimental

apparatus programming

system.

21: 294-302.

Great tits and giving-up times:

Decision rules for leaving patches. Behaviour,

90: l-24. Yoerg, S.I. and Kamil, A., 1988.

Diet choices of blue jays (Cyanocitta cristata)

spent foraging. J. Comp. Psychol.,

102:

230-235.

as a function of time