Effect of intertrial unconditioned stimulus (US) presentations upon responding to a conditioned stimulus predictive of either the same or a different appetitive US

LEARNING

AND

MOTIVATION

21, 266-286 (1990)

Effect of Intertrial Unconditioned Stimulus (US) Presentations upon Responding to a Conditioned Stimulus Predictive of either the Same or a Different Appetitive US KELLY

J. STANHOPE

University

of Cambridge

The effects of intertrial unconditioned stimulus (US) presentations on the acquisition of keypecking to a conditioned stimulus (CS) paired with either the same or a different appetitive US was examined in two experiments with pigeons. In the first experiment it was found that intertrial grain presentations prevented acquisition of conditioned responding to both a CS predictive of water and a CS predictive of grain. In contrast, intertrial water presentations did not prevent the acquisition of responding to a CS predictive of grain, although the intertrial water presentations did prevent the acquisition of responding to a CS predictive of water. Experiment 2 replicates the results of Experiment 1 and also shows that in the absence of any intertrial events the pigeons acquired responding to both the CS for grain and the CS for water. The results of these experiments are discussed in terms of context blocking. Q 1990 Academic PMS. IIK.

One of the necessary conditions for learning a conditioned stimulus (CS)-unconditioned stimulus (US) association is that the US is not already predicted by other stimuli. Evidence for this comes from Kamin’s (1969) blocking effect in which following repeated pairings of a CS (CSl) and a US, a second CS (CS2) is compounded with the first CS and the compound is followed by the same US. In these circumstances, the animal demonstrates little behavioral evidence of having learned the relationship between CS2 and the US. Learning the CS2-US association is said to be blocked by prior learning of the CSl-US association. A widely accepted account of the blocking effect is based on the notion I thank my supervisor, Dr. Anthony Dickinson, and also Dr. P. Durlach and Professor N. J. Mackintosh for helpful advice throughout the course of this work. Thanks are also due to V. M. LoLordo, D. Williams, A. Delamater. and A. Droungas for helpful comments on an earlier version of this manuscript. Reprint requests and correspondence should be addressed to the author at the Laboratory of Experimental Psychology, University of Sussex, Falmer, Brighton, England, BNl 9QG. 266 0023-%%I% Copyright All rights

$3.00

8 1’930 by Academic Press, Inc. of reproduction in any form reserved.

EFFECT

OF

INTERTRIAL

US PRESENTATIONS

267

that learning a CS-US association will only occur when the US is unexpected or surprising. In order to determine what constitutes surprise, researchers have conducted transreinforcer blocking experiments. In a typical transreinforcer blocking experiment, CSl is paried with one US and the CSICS2 compound is paired with a different US. The question of interest is whether unblocking (i.e., learning about the CS2-US association) occurs when the compounded stimuli predict a US that in some way differs from that predicted by CSl . There are numerous experiments that report unblocking when the two USs differ in both sensory and affective properties. For example, Dickinson and Mackintosh (1979) found that either increases or decreases in the number of food pellets or the number of shocks delivered attenuated blocking of the added CS (see also, Holland, 1984; Dickinson, Hall, & Mackintosh, 1976). Similarly, Bakal, Johnson, and Rescorla (1974) found that changing from a weakly aversive US (loud noise) during single element training to a more strongly aversive US (shock) during the compound conditioning stage resulted in conditioning to the added cue. Moreover, some researchers have shown that changing from an appetitive US during stage 1 training to an aversive US during stage 2 can actually potentiate conditioning to the added cue (e.g., Dickinson, 1977; Goodman & Fowler, 1983). Rescorla (1971) has referred to this effect as superconditioning. It might be taken to suggest that a shift between an appetitive and an aversive US can produce sufficient surprise to actually accelerate conditioning to the added cue. Although numerous experiments have shown that receiving a US that has unexpected affective properties results in unblocking, very few experiments have addressed the question of whether a US must be of a surprising affective value for learning to occur. Furthermore, the experiments that have looked at this issue have produced mixed results. On the one hand, Ganesan and Pearce (1988) found that changing the US from food to water or vice versa did not attenuate blocking. This outcome is consistent with the view that shifting between two USs with different sensory properties is not sufficient for unblocking to occur. On the other hand, however, a rabbit eyelid conditioning experiment conducted by Stickney and Donahoe (1983) found that changing the location of interorbital shock from the left to the right eye did attenuate blocking. This latter outcome is consistent with the conclusion that receiving a US of an expected affective value but with unexpected sensory properties does produce sufficient surprise for unblocking to occur. The purpose of the present experiments was to investigate further whether unblocking would occur when the expected US and the actual US had similar affective properties but dissimilar sensory properties. The experiments employed pigeons that were trained while both hungry

268

KELLY

J. STANHOPE

and thirsty and used grain and water as the unconditioned stimuli. As there is typically a large amount of between-subjects variability in pigeon experiments, a within-subjects design was used. Also, in order to maximize the chances of observing unblocking, an intermixed, as opposed to a two stage, blocking procedure was used (see Wagner, 1969). With an intermixed training procedure the potential blocking stimulus is not pretrained. Instead, the single element training trials are intermixed with the compound conditioning trials. The advantage of using an intermixed procedure is that the pigeon would have recently experienced the pairings of the single element CS, and the US, and therefore might be less likely to “forget” the specific properties of the reinforcer paired with the element. The present experiments differed from traditional transreinforcer experiments in one other important aspect. In the transreinforcer experiments cited above, the blocking stimulus was a discrete cue. As it is notoriously difficult to find blocking using discrete cues in pigeons (e.g., Jenkins, Barnes, & Barrera, 1981; Stanhope, 1988; Tomie, 1981; but see Rescorla, 1981; Schreurs & Westbrook, 1982), the blocking stimulus in the present experiment was the training context. It is a well-established finding that the occurrence of intertrial US presentations will interfere with the acquisition of conditioned responding that otherwise would have proceeded normally (e.g., Gamzu & Williams, 1973; Rescorla, 1968; Tomie, 1976a,b). Although competing accounts exist (e.g., Gibbon & Balsam, 1981; Jenkins et al., 1981), a popular explanation of contingency effects is context blocking. The intertrial US presentations result in the context becoming a good predictor of the US and, like any other excitatory stimulus presented in compound with the target stimulus, the contextual cues are thought to block learning of the discrete cue-US association. The question addressed in the present paper is whether a context-US association will interfere with the acquisition of conditioned responding to a discrete cue paired with either the same or a different appetitive US. There is some evidence from an instrumental paradigm that contingency effects do respect the specific sensory properties of the reinforcer. Colwill and Rescorla (1986), for example, reported an experiment in which rats were trained to make two different operant responses: one response led to the delivery of Noyes pellets and the other response led to the delivery of sucrose solution. Once responding was established, response-independent presentations of either pellets or sucrose solution were added. These unearned reinforcer presentations resulted in a decline in responding for both reinforcers but the effect was more marked for the response that produced the same reinforcer than for the response that produced the different reinforcer (see also Dickinson & Mulatero, 1989; Williams, 1989).

EFFECT

OF

INTERTRIAL

US PRESENTATIONS

269

It is not clear, however, how relevant Colwill and Rescorla’s results are to the issue addressed in the present paper. First, Colwill and Rescorla used an instrumental as opposed to a Pavlovian conditioning procedure. Some researchers have argued that the way in which reinforcers are encoded differs between these two conditioning paradigms (Dickinson, 1989; Dickinson & Dawson, 1987). Also, Colwill and Rescorla looked at the effect of adding response-independent presentations of reinforcers upon previously trained conditioned behaviors. The issue addressed in the present paper is the way in which noncontigent reinforcers affect the initial acquisition of conditioned responding. EXPERIMENT

1

Experiment 1 includes three groups of pigeons. All the pigeons were trained in an experimental context where one color keylight was followed with a .25 probability of access to grain and another color keylight was followed with a .25 probability of water. The occurrence of food and/or water in the absence of the keylight varied among groups. In Group Unsignaled food (Uf), food was delivered with a .25 probability in the absence of the keylights; in Group Unsignaled Water (Uw), water was delivered with a .25 probability in the absence of the keylights; and in Group Unsignaled Food and Water (Ufw) food and water each occurred with a .125 probability when the keylights were not on. If transreinforcer blocking is observed, keypecking should not develop in any of these groups. The context should come to predict the delivery of a reinforcer and thereby block conditioning to both discrete cues. On the other hand, if unblocking occurs when the CS predicts a reinforcer with sensory properties that differ from the US predicted by the context, a very different pattern of results should emerge. The pigeons in Group Uf should only acquire conditioned responding to the CS predictive of water. In contrast, the pigeons in Group Uw should only acquire conditioned responding to the CS predictive of food. Finally, the pigeons in Groups Ufw should acquire pecking to both CSs since these stimuli are better predictors of food and water than the training context. Method Subjects

Eighteen experimentally naive feral pigeons were used. The pigeons were individually housed in a colony room on a natural light-dark cycle. Grit was freely available in the home cage. Apparatus Two Campden of 33.5 X 30.5

X

Instruments pigeon chambers with internal dimensions 31.0 cm (length x width x height) were used. The

270

KELLY

J. STANHOPE

chamber floor was a stainless-steel grid and the front wall was a threekey intelligence panel. A house light (24 V, 2.8 W) was centered on the front wall 2.0 cm below the ceiling. Only the right-hand key was used. This key was 10.8 cm to the right of the house light and 22.8 cm above the floor. The key was 2 cm in diameter and could be transilluminated with white, green, or red light. Black electrical tape covered the center and the left-hand keys. A recessed magazine (6.5 x 5.0 x 11 cm) was centered on the front wall 7.0 cm above the floor. A light (28 V, 1.1 W) was in the ceiling of the magazine and a 1.O x l.O-cm grain aperture was centered in the magazine floor. Grain was presented at this opening by operating a solenoid which raised up a grain tray. A 3.2 x 1.0 x 0.5cm water tray was fixed to the back wall of the magazine 3.0 cm above the floor. Water was delivered into this tray when a 24-V Honeywell Skinner Valve was operated. The water was stored in a reservoir located approximately 1.5 m above the chamber. The pigeon chambers were enclosed within sound and light attenuating shells. A ventilation fan that was attached to these shells was constantly operated. The events in the pigeon chambers were controlled by an Acorn Atom microcomputer which was located in another room. Procedure Preliminary training. Prior to magazine training for grain the pigeons were food deprived to 85% of their free-feeding weight. The pigeons were given restricted access to grain but water was freely available in the home cage. Once grain magazine training began, the bird’s water intake was also restricted. Water was provided in an amount (in milliliters) that was 80% of the bird’s daily grain intake (in grams). On Days l-3 the pigeons were given hand-magazine training sessions for grain alone, but on Days 4-9, sessions of hand-magazine training for water were intermixed with sessions of magazine training for grain. Typically, the birds were given only one magazine training session on each of these days. However, on some of the days, birds that were not eating and/or drinking reliably received an additional session with the US that had not been consumed. All hand-magazine training sessions were 20 min long. On Day 10 the birds began automated magazine training sessions in which 27 deliveries of grain and 27 deliveries of water were scheduled to occur in a random sequence with a mean intertrial interval of 30 s. Food deliveries were 4 s long and water deliveries were 0.1 ml, with the magazine light being operated for 4 s. On alternate days, birds that did not eat and/or drink reliably received a hand-magazine training session

EFFECT

OF INTERTRIAL

US PRESENTATIONS

271

for the US(s) that had not been consumed. The birds that were readily consuming both grain and water throughout the automated sessions were run approximately every third day. During magazine training, many of the birds stopped consuming one of the two reinforcers. This was probably due to the lack of sufficient motivation for that reinforcer. Therefore, birds that would not eat or that were slow to eat had their daily water allotment increased by 5% and, in addition, received 10 ml of extra water on that day. Birds that were not drinking or were slow to drink had their daily water allotment decreased by 5% and were not given any supplemental water on that day. At the end of Day 18 the birds were consuming both reinforcers with what casual observation would suggest to be approximately equal latencies. Experimental training, Six pigeons were randomly assigned to each of three groups: Group Uf, Group Uw, and Group Ufw. The pigeons in each of these groups received 30 daily training sessions with two different colored keylight CSs. One CS, the CSf, had a .25 probability of being followed by a 4-s food presentation and the second CS, the CSw, had a .25 probability of being followed by a 0. l-ml water delivery. Both CSs were 5 s long and the colors of the keylights (either red or green) that were used as the CSf and the CSw were counterbalanced within each group. Twelve presentations of each CS were scheduled in a random order in each session and a random three CSf and three CSw presentations were followed by the delivery of the appropriate US. The key was dark during the intertrial interval. The mean intertrial interval was 40 s. Events that occurred in the absence of the CSs differed among groups. Group Uf received 24 unsignaled presentations of grain, Group Uw received 24 unsignaled deliveries of water, and Group Ufw received 12 unsignaled presentations of food and 12 unsignaled presentations of water. The number of unsignaled USs was chosen so as to equate the probability of a US given that no CS had occurred [p(US/no CS)] with the probability of a US given that a CS had occurred [p(US/CS)]. These probabilities were calculated using a method similar to that described by Durlach (1983). The l&min experimental session was divided up into 120 9-s intervals. The first 5 s of a random 12 of these intervals contained a CSf presentation and the first 5 s of another randomly selected 12 intervals contained a CSw presentation. The last 4 s of a random three CSf (CSw) intervals contained a delivery of food (water). Of the 96 9-s CS-free intervals, 24 were randomly selected to contain an unsignaled US presentation. With these parameters both the probability of a US in the presence of a CS and the probability of a US in the absence of a CS are equal to .25. A new random sequence was selected each day. Although the overall relationship between a CS and a US presentation

272

KELLY

J. STANHOPE

in the three groups was random, if the probability of food in the presence and the absence of the CSf and the CSw are considered separately, positive contingencies emerge. In all the groups the probability of food or water given that the CSf or the CSw had occurred was .25. In group Uf, the probability of water in the absence of the CSw was 0 whereas the probability of food in the absence of the CSf was .25. There was a positive contingency between the CSw and water and a random contingency between the CSf and food. Conversely, in Group Uw, there was a positive contingency between the CSf and food and a random contingency between the CSw and water. Finally, there was a .I25 probability of food and a .125 probability of water in the absence of a CS in group Ufw. In this group then, both the CSf and the CSw were positively correlated with their respective USs. Throughout the experiment the grain hopper was weighed before and after each session and the water tray was checked to see if it was empty. If a bird failed to eat or did not consume all of its water during two consecutive sessions, the proportion of the bird’s daily water intake was adjusted either up (if it did not eat) or down (if it did not drink) by 5%. Data collection and analysis. Each day the percentage of CSf and CSw trials with a response were recorded by an Acorn Atom microcomputer. Depending on the group assignment, many of the birds pecked on either none or all of the trials. Consequently, the data were analyzed using nonparametric statistics. The level of significance was set at p < .05. One pigeon in Group Uw died during the course of the experiment. The data for that bird were excluded from any analyses. Results and Discussion The percentages of CSf trials with a response for the pigeons in Groups Uf, Uw, and Ufw are plotted in the top panel of Fig. 1. Acquisition of responding to the CSf appeared in Group Uw and, by the end of training, the pigeons in Group Uw were responding on more of the CSf trials than the birds in the other two groups. A Mann-Whitney I/ test conducted using the mean percentage of trials with a response on the last two sessions found that the percentage of CSf trials with a response for the pigeons in Group Uw was greater than for Group Ufw, U = 3. There was no overlap between the percentage of CSf trials with a response in Group Uw and Group Uf. Although there appears to be some evidence of acquisition of responding to the CSf in Group Ufw, on the last two sessions of training the percentage of CSf trials with a response did not differ between Groups Ufw and Uf, U = 21. The percentages of CSw trials with a response are shown in the lower panel of Fig. 1. The pigeons in Group Uw displayed a transitory increase in the percentage of CSw trials with a response. By the end of training,

EFFECT

OF INTERTRIAL

5

10

US PRESENTATIONS

15

20

25

30

25

30

273

Successive Sessions

r; $ $ a

0 0

5

10

15

20

Successive Sessions

FIG. 1. The percentage of CSf (upper) and CSw trials (lower) with at least one response for the three groups of pigeons during successive sessions of Experiment 1.

however, there was little responding in any of the groups. On the last two sessions of training the mean percentage of trials with a response for the pigeons in all three groups was less than 5%. The pigeons in this experiment failed to acquire responding to a CS that was associated with the same probability of water delivery as the experimental context. However, intertrial water presentations did not prevent the acquisition of key pecking to a CS paired with food. The pigeons in Group Uw acquired conditioned responding to the CSf. This result suggests that contingency effects respect the specific nature of the reinforcer. In contrast, the pigeons that received the intertrial food presentations failed to acquire conditioned responding to either the CS predictive of food or the CS predictive of water. Thus, it appears that intertrial food presentations block the acquisition of conditioned responding to cues predictive of either food or water. This result is consistent with the view that context blocking effects do not respect the specific sensory properties of the reinforcer.

274

KELLY

.I. STANHOPE

These asymmetrical results might be easily accounted for. Perhaps, even in the absence of any intertrial events, pigeons will not acquire conditioned responding to a CS predictive of only a .25 probability of water. In that case, the failure to find unblocking to the CSw in Group Uf was due to the fact that the present parameters did not result in a CS-water association. The plausibility of this suggestion is supported by the fact that the pigeons in Group Ufw failed to acquire responding to the CSw even though the CSw predicted a higher probability of a US than did the training context. In order to test this hypothesis, a control group that receives a CS predictive of a .25 probability of water and no intertrial events is required. This group is included in the second experiment. EXPERIMENT

2

This experiment was similar in design to the first experiment with the exception that Group Ufw was replaced by a control group, Group Uo, that did not experience any unsignaled USs. The pigeons in all three groups received a CSf and a CSw which, on 25% of the trials, were followed by the appropriate US. During the intertrial interval the pigeons in groups Uf and Uw received presentations of food or water, respectively. The pigeons in Group Uo did not receive any events during the intertrial interval. A second aspect of Experiment 2 that differed from the first experiment was that the force of the pigeons’ keypecks was recorded. A number of researchers (e.g., Jenkins & Moore, 1973; Skelton, Spetch, & Wilkie, 1980; Spetch, Wilkie, & Skelton, 1981; Stanhope, 1989) have found that pigeons make more forceful pecks to stimuli predictive of food than to stimuli predictive of water. This result has been interpreted (e.g., Stanhope, 1989) as showing that the CS has come to excite a representation of the reinforcer that encodes some information about the specific sensory properties of that reinforcer. If the pigeons in Group Uo acquire conditioned responding to both the CSf and the CSw, and if that responding is reinforcer specific, it might be concluded that the pigeons in Group Uo were able to discriminate between the food and water USs. Thus, a finding of transreinforcer blocking in Group Uf and/or Grup Uw could not easily be attributed to the possibility that the subjects were unable to detect any differences between the two USs. Method

Subjects Twenty-four experimentally naive feral pigeons with a mean free-feeding weight of 335 g were used. Housing conditions were the same in this experiment as in Experiment 1.

EFFECT OF INTERTRIAL

US PRESENTATIONS

275

Apparatus Four training chambers were used. Two of the chambers were those described in Experiment 1. The two additional chambers were identical to one another but differed in a number of aspects from the chambers previously used. Each of the new chambers had the internal dimensions of 37.2 x 23.2 x 31.1 cm. A two-key intelligence panel made up the front wall. A houselight (24 V, 2.8 W) was centered 2.5 cm below the ceiling on the front wall. Two translucent Perspex keys (2.5 cm in diameter) were located 24.8 cm above the floor. One was 6.5 cm to the left of the houselight and the other was 6.5 cm to the right of the houselight. Only the right-hand key was used, and the left-hand key was covered with black electrical tape. The food magazine (6.5 x 5.0 x 11.O cm) was recessed into the center of the intelligence panel 8.8 cm above the floor. Operation of a solenoid raised a grain tray up to a 1.0 x l.O-cm aperture in the magazine floor. Illumination of the magazine could be provided by a small light (24 V, 1.1 W) located in the ceiling of the magazine. Water could be delivered to a water tray (3.2 x 1.0 x 0.5 cm) fixed to the back wall of the food magazine 3.0 cm above the floor. The method of water delivery was the same as that described for the boxes used in Experiment 1. In all four chambers the right-hand key consisted of a Perspex flap (4.5 x 5.5 cm) mounted on the back of the intelligence panel. The flap was hinged at the top so that when force was applied to the front of the key it was pushed back. Two RS strain gauges (No. 308-102) were glued, one on each side, on a curved brass spring (1.9 cm x 0.7 cm x 0.03 mm) that was mounted behind the key. The gauges, with a gauge resistance of 120 ohm and a 2: 1 gauge factor, were wired in series, center taped. When the key was in its resting position the spring did not quite touch the back of the key, but when the key was depressed, the spring was deflected backward. The resulting resistance changes were fed into a strain gauge amplifier and then into an analog to digital port in an Acorn System 4 microcomputer. The output from the analog to digital converter (ADC) in response to a known amount of weight applied to each key was measured before the experiment and the results of a regression analysis on these data are shown in Table 1. The slopes and intercepts were comparable among the four boxes and the linear functions accounted for greater than 97% of the variability. Each key was equipped with an in-line projector which could shine onto the back of the key and illuminate the left-hand side of the key with either red or green light. The right-hand side of the key remained dark. The peck data were collected by the System 4 microcomputer and an Acorn Atom controlled the events in each pair of boxes.

276

KELLY

J. STANHOPE

TABLE 1 Regression Equations and R' Values for the Relationship between the ADC Output and the Amount of Weight Placed on the Response Key Box

Slope

Intercept

R2

1

,671 .680 ,680 .540

14.3 16.1 14.0 20.8

,991 .975 ,984 ,987

2 3 4

Procedure Magazine training. For ease of training, the pigeons in this experiment were deprived of, and magazine trained for, only one reinforcer at a time. The pigeons were initially reduced to 85% of their free-feeding weight by restricting their food intake. Once this criterion was achieved, food magazine training began. On Days 1 and 2, the pigeons were simply placed into the illuminated chambers in which the magazine light was on, the grain tray was heaped full of wheat, and the water tray was filled with water. The pigeons were left undisturbed in the chamber for 30 min. On the next 4 days (Day 3-6) the pigeons were given daily magazine training sessions for food that were identical to those described in Experiment 1. Any pigeon that was not eating efficiently received additional daily food magazine training sessions on Days 7-9, by which time all the pigeons were reliably eating. On Day 10 the pigeons were shifted from food deprivation to water deprivation. The shift was accomplished by removing water from the home cage and providing the pigeons with free access to grain. Beginning on Day 11, the pigeons were given enough water to maintain them at their 85% weight. Also on Day 11, the pigeons were placed into the chambers for 60 min with the house light on, the magazine light illuminated, and the water tray filled. For the pigeons that did not drink the water on this day, this procedure was repeated on Day 12. Handmagazine training for water was conducted across Days 13-16 and was identical to the water magazine training sessions in Experiment 1. At the end of these sessions all the pigeons were reliably consuming each 0.1 -ml water delivery. At the end of Day 16, the grain was removed from the pigeon’s home cage and each pigeon was given access to three times the normal allotment of water. Beginning on Day 17, each pigeon was provided with water (in milliliters) in an amount that was 80% of the daily food intake (in grams). Grain was provided in an amount sufficient to maintain the pigeon at 85% of its free-feeding weight. Eighteen daily automated magazine training sessions began on Day 18. During each session, 27 presentations of grain (4-s long) and 27 presentations of water (0.1 ml in

EFFECT

OF INTERTRIAL

US PRESENTATIONS

277

volume) occurred in a random order with a mean intertrial interval of 30 s. As in the previous experiment, the grain hopper was weighed before and after each session and the water tray was checked at the end of each session. Pigeons that failed to eat or drink had their food and water allotments adjusted according to the method previously described. By the end of the 18th automated session all the pigeons were reliably consuming both reinforcers. Experimental training. The pigeons were randomly assigned to three groups; Group Uo, Group Uf, and Group Uw. The pigeons in Groups Uf and Uw were given exactly the same training as those in Groups Uf and Uw in the first experiment. The training for the pigeons in Group Uo differed from that of the other two groups only in that the intertrial US deliveries were omitted. The pigeons were given 41 daily training sessions. Data collection and analysis. For ease of programming, data were collected from only half of the animals that were run on each day. Four of the animals in each group were started 1 day out of phase with the other subjects. All of the animals had their data collected on the first training session and on every alternate day thereafter. Results and Discussion The mean percentage of CSf trials with a response is displayed in the upper panel of Fig. 2. As shown in the figure, the pigeons in Groups Uo and Uw acquired responding to the CSf. By the last two sessions of training the pigeons in Groups Uo and Uw were responding on 76 and 85% of the trials, respectively. The pigeons in Group Uf responded on less than 5% of the CSf trials. A Mann-Whitney U test, which compared the percentage of CSf trials with a response among the three groups on sessions 39 and 41, found that the pigeons in Groups Uo and Uw made a response on an equivalent percentage of CSf trials, U = 18. Furthermore, the pigeons in both Group Uo and Uw responded on a higher percentage of trials than did the pigeons in Group Uf, U = 3.5 and U = 6.5, respectively. Replicating the results of Experiment 1, intertrial food presentations appeared to block the acquisition of responding to the stimulus predictive of food whereas intertrial water presentations did not. The percentage of CSw trials with a response is displayed in the lower panel of Fig. 2. Across the training sessions the pigeons in Group Uo acquired responding to the CSw. By the last two sessions of training the pigeons in Group Uo were responding on about 47% of the CSw trials. Although there appeared to be a transitory increase in responding in Group Uw, by the last two sessions of training the pigeons in both Group Uf and Group Uw made a response on fewer than 5% of the CSw trials. A Mann-Whitney U test on the number of trials with a response during

278

KELLY

J. STANHOPE

80 -

01

5

9

13

17

21

25

29

33

37

41

29

33

37

41

Sessions

B 100 z a

m XI .e 3 22 .m c 3 5

80

E8

0

k

d

80 40 20 1

5

9

13

17

21

25

Sessions

FIG. 2. The percentage of CSP (upper) and CSw trials (lower) with a response for the three groups of pigeons during successive sessions of Experiment 2.

Sessions 39 and 41 found that the pigeons in Group Uo responded on more CSw trials than did the pigeons in either Group Uf (V = II) or Group Uw (V = 11.5). The percentage of CSw trials with a response in Group Uf and Group Uw did not differ (U = 31 S). Although the pigeons in Group Uo acquired responding to the CSw, it should be noted that the asymptotic percentage of CSw trials with a response was only 47 whereas the asymptote to the CSf (77% trials with a response) was much higher. In fact, in Group Uo every pigeon but one responded on a higher percentage of the CSf trials than on the CSw trials. One pigeon responded on 100% of both types of trials. On Sessions 39 and 41 all eight of the pigeons in Group Uo responded on the CSf trials but only six of these pigeons made a response on the CSw trials. For these six pigeons, the mean force (in arbitrary units) of the CSf pecks was 76.5 and the mean force of the pecks made on the CSw trials was 47.9 arbitrary units. All of the six pigeons responded with a higher force on the CSf trials than on the CSw trials.

EFFECT

OF

INTERTRIAL

US PRESENTATIONS

279

The results of this experiment replicate the asymmetrical effect found in Experiment 1; intertrial food presentations interfered with the acquisition of conditioned responding to stimuli predictive of either food or water but intertrial water presentations interfered only with the acquisition of responding to a stimulus predictive of water. At asymptote, responding to the stimulus predictive of food appeared to be unaffected by the intertrial water presentations. The present findings also rule out the potentially trivial possibility that the failure to see responding to the CSw in Group Uf was due to the fact that the parameters used in this experiment would not generate conditioned responding on the CSw trials even if no intertrial events occurred. The pigeons that did not receive intertrial US presentations (i.e., the pigeons in Group Uo) acquired conditioned responding to both the stimulus predictive of food and the stimulus predictive of water. Finally, the results from Experiment 2 go some way toward ruling out the possibility that the pigeons were unable to discriminate between the sensory properties of food and water. The pigeons in Group Uo made more forceful pecks to the CSf than to the CSw. This difference in force suggests that the pigeons were pecking the conditioned stimuli with the species-specific behavior appropriate to the US. Reinforcer-specific conditioned behaviors would not be expected unless the pigeons were able to discriminate between the two reinforcers. GENERAL

DISCUSSION

The results of these experiments are difficult to interpret. On one hand, it was found that uncorrelated water presentations interfered with the acquisition of responding to a stimulus predictive of water but not to a stimulus predictive of food. This outcome is consistent with the conclusion that a difference between the sensory properties of the US predicted by the contextual cues and the US delivered on the CSf trials is sufficient to disrupt blocking. On the other hand, however, unsignaled food presentations were found to interfere with the acquisition of conditioned responding to both a CS paired with food and to a CS paired with water. This latter finding is consistent with the conclusion that a change in only the sensory properties of a US does not attenuate blocking. The favored explanation of the present results is based on the assumption that, with the present deprivation procedure, food was a more valuable reinforcer than water. There is some evidence from the present experiments to justify this claim. In Experiment 2, the pigeons in Group Uo responded on fewer of the CSw trials than on the CSf trials. Furthermore, in an experiment that employed the same deprivation regime as used in the present studies, Stanhope (1989, Experiment 1) found that pigeons were slower to acquire responding to a stimulus predictive of water than to a stimulus predictive of food.

280

KELLY

J. STANHOPE

If food was a more valuable reinforcer than water, and if unblocking occurs whenever there is a difference between the affective value of the expected and the received US, then unblocking should have been observed in both Group Uf and Group Uw. In Group Uw the stimulus paired with food predicted a more valuable reinforcer than that associated with the context, and in Group Uf, the stimulus paired with water predicted a less valuable reinforcer than that associated with the context. Unfortunately, the latter prediction is at odds with the present results. The pigeons in Group Uf did not acquire responding to the CSw. Instead, the results from the present experiments could be interpreted as showing that an upshift, but not a downshift, in the value of the reinforcer resulted in the attenuation of blocking. Although some researchers have reported that downshifts in reinforcement can produce excitatory conditioning to the added cue (e.g., Dickinson & Mackintosh, 1979; Dickinson, Hall, & Mackintosh, 1976; Feldman, 1971; Holland, 1984; Neely & Wagner, 1974), other researchers have found that downshifts in the magnitude of the reinforcer result in the acquisition of conditioned inhibition (e.g., Cotton, Goodall, & Mackintosh, 1982; Wagner, Mazur, Donegan, & Pfautz, 1980) or the acquisition of both conditioned inhibition and conditioned excitation to an added cue (Holland, 1988; Mackintosh & Cotton, 198.5; Nelson, 1987). Mackintosh and Cotton (1985) and Nelson (1987), for example, found that stimuli predictive of an incomplete reduction in reinforcement acted as conditioned inhibitors in the presence of strong excitatory cues and as conditioned excitors in the presence of weak excitatory cues. In the present experiments, for the pigeons in Group Uf, the CSw signaled a US that was less valuable than the US predicted by the context. As a consequence the CSw may have acquired inhibitory properties. It is not clear how an inhibitory association would effect the acquisition of excitatory conditioned responding, but it seems reasonable to conclude that it may have interfered with either (1) the acquisition of an excitatory CS-water association or (2) the performance of conditioned behaviors that might have occurred as a result of an excitatory CS-water association. Another explanation for why the pigeons in Group Uf failed to acquire to the CSw is based on the notion that intertrial food and water presentations result in different sorts of competing behaviors. It is possible that food-related behaviors are more likely to interfere with autoshaping to a discrete cue than are water-related behaviors. Given that Durlach (1983) and Goddard and Jenkins (1987) have shown that signaling intertrial food presentations reduces their detrimental effects on conditioned responding, it is unlikely that these competing behaviors are generated by a nonassociative mechanism. However, intertrial US presentations might result in the context evoking conditioned behaviors that could interfere with the acquisition of responding to the keylight stimuli. If

EFFECT

OF

INTERTRIAL

US PRESENTATIONS

281

context-food associations result in the context evoking behaviors that are more likely to interfere with conditioning to a keylight stimulus than behaviors evoked as a consequence of context-water associations, the present findings could be accounted for. In partial support of the competing response account, Reberg, Innis, Mann, and Eizenga (1978) noted that periodic food presentations frequently result in interim behaviors that take pigeons away from the food magazine. It seems likely that such behaviors would interfere with conditioning to a discrete keylight CS presented on the magazine wall. However, Reberg et al. (1978) also reported that pigeons which were given periodic water presentations spent a lot of time pecking in and around the water magazine. Presumably, the behaviors of looking down and pecking in the magazine would be even more likely to interfere with conditioning to the keylight. There is little reason, therefore, to assume that unsignaled food presentations result in the context-evoking behaviors that are more likely to interfere with autoshaping than behaviors evoked by the context as a result of inter-trial water presentations. Given that the food and water reinforcers that were used in the present experiments may have had different affective values, it is not known whether the unblocking observed to the CSf in Group Uw was due to the grain having different sensory properties and/or different affective properties than the otherwise expected water. The results from a series of experiments that were recently reported by Holland (1988) suggest that both of these factors may have been important. In one experiment (Experiment 2), for example, Holland trained rats with a CS predictive of either food or sucrose. In the second stage of that experiment, the original CS was presented in compound with a novel CS and was followed by the delivery of two USs in sequence. For the rats in one group, Group Up-Same, the compound stimulus was followed by two presentations of the same reinforcer that was used during the initial stage of training. For the rats in another group, Group Up-Diff, the compound stimulus was also followed by two USs, but only the first US in the sequence was the same US that had been presented during single element training. If sucrose was used during the single element training, the compound was followed in sequence by sucrose and food, and if food was the US used during the single element training, the compound was followed by food and sucrose. In a later test, Holland found that the rats in Group Up-Diff displayed more conditioned responding to the novel element of the compound cue than did the rats in Group Up-Same. This outcome seems to indicate that upshifts in the affective value of the US that are also accompanied by changes in the sensory properties of the US produce greater surprise than do upshifts in the value of the reinforcer that are not accompanied by changes in the sensory properties of the US. Although Holland’s (1988) results imply that changes in the sensory

282

KELLY

J. STANHOPE

properties of the US can produce surprise, the results do not address the question of whether the surprise produced by receiving a US that has unexpected sensory properties, but an expected affective value, is sufficient to disrupt blocking. In all of Holland’s experiments, changes in the sensory properties of the US were confounded with changes in its affective value. It seems that the only experiments that address the issue of whether unblocking occurs when the expected and the received USs differ only in sensory properties are those reported by Ganesan and Pearce (1988) and Stickney and Donahoe (1983). Unfortunately, the results of their reports lead to quite different conclusions. On the one hand, in a series of appetitive conditioning experiments with rats, Ganesan and Pearce found that changing the US from food during single element training to water during compound conditioning (or vice versa) did not disrupt blocking to the added cue. This finding is consistent with the view that the occurrence of sensory surprise is not a sufficient condition for disrupting blocking. In contrast, the results from the rabbit eyelid conditioning experiment conducted by Stickney and Donahoe suggest that sensory surprise that is unaccompanied by affective surprise will attenuate blocking. In the first stage of their experiment, Stickney and Donahoe paired a CS with a I-mA, 50-ms shock to the left paraorbital region. During the second stage of the experiment, for one group of animals, the pretrained element was compounded with a novel cue and was followed by a I-mA, 50-ms shock to the contralateral paraorbital region. Relative to controls, the animals that experienced a shift in the location of the shock immediately prior to compound conditioning displayed more responding to the novel cue. The outcome of Stickney and Donahoe’s experiment could be taken to suggest that a change in only the sensory aspects of a US is sufficient to disrupt blocking. However, Ganesan and Pearce (1988) have suggested that the shift in the location of the shock in Stickney and Donahoe’s experiment was accompanied by a change in the perceived affective value of the US. Site-specific habituation may have occurred to the shock delivered during the first stage of the experiment. As a consequence, the shock delivered to the new location during the compound-conditioning stage of the experiment might have been perceived as more intense, and therefore more aversive, than the shock that was received at the end of the first stage of training. Thus, the attenuation of blocking observed by Stickney and Donahoe may have been due to affective surprise. In light of this possibility, and given the results of their own experiments, Gasesan and Pearce conclude that affective surprise may be a necessary condition for unblocking to occur. The results reported in the present paper are not inconsistent with that view. It should be noted that the present results in no way contradict the findings from experiments that have looked at the effect of superimposing

EFFECT

OF INTERTRIAL

US PRESENTATIONS

283

presentations of either the same or different noncontingent reinforcers upon an instrumentally reinforced response baseline (Colwill & Rescorla, 1986; Dickinson & Mulatero, 1989; Williams, 1989). As was outlined in the introduction, these experiments found that noncontingent presentations of the same reinforcer had a more detrimental effect upon the performance of instrumentally conditioned behaviors than did noncontingent presentations of a different reinforcer. Thus, when an instrumental conditioning procedure is used, contingency effects respect the specific sensory properties of the reinforcer. One explanation for the discrepancy between the results of the instrumental conditioning experiments and the results of the Pavlovian conditioning experiments reported here is that there is a fundamental difference in the associative structure that mediates responding on these two different types of contingencies (e.g., Dickinson, 1989). More parsimonious explanations, however, may be based upon two other procedural differences. First, the present experiments looked at the effect of noncontingent reinforcer presentations upon the acquisition of conditioned responding, whereas the instrumental studies looked at the effect of noncontingent reinforcer presentations upon the performance of previously trained conditioned behaviors. It may be that previously established Pavlovian-conditioned behaviors would also be more suppressed by the addition of intertrial presentations of the same, as opposed to a different, appetitive US. Second, in the instrumental conditioning studies reported by Dickinson and Mulatero (1989) and Colwill and Rescorla (1986), the two reinforcers that were used might have been of equal value. In the Dickinson and Mulatero (1989) experiment, for example, the animals responded for both reinforcers with equivalent rates. In contrast, I have argued that in the present studies the reinforcers was not of equivalent affective value. If reinforcers of unequal value are used, asymmetrical effects similar to those reported in the present study might be observed regardless of whether an instrumental or a Pavlovian conditioning procedure is employed. Of relevance to this second point, in the instrumental conditioning experiment reported by Williams (1989), the two reinforcers were of unequal value. On baseline, the rats responded with a much higher rate for one of the reinforcers than for the other. Due to a floor effect, however, the effect of noncontingent reinforcer presentations upon the rate of responding for the less valuable reinforcer is difficult to interpret. Therefore, it remains to be determined whether asymmetrical effects similar to those observed in the present experiments would also be observed in an instrumental conditioning experiment in which different valued reinforcers are used.

284

KELLY

J. STANHOPE

REFERENCES Bakal, C. W., Johnson, R. D., & Rescorla, R. A. (1974). The effect of change in US quantity on the blocking effect. Pavlovian Journal of Biological Sciences, 9, 97-103. Colwill, R. M., & Rescorla, R. A. (1986). Associative structures in instrumental learning. In Gordon H. Bower (Ed.), The psychology of /earning and motivation (Vol. 20. pp. 55-104). London: Academic Press. Cotton, M. M., Goodall, G., & Mackintosh, N. J. (1982). Inhibitory conditioning resulting from a reduction in the magnitude of reinforcement. Quarter/y Journal of Experimental Psychology, 34B, 163-180. Dickinson, A. (1989). Expectancy theory in animal conditioning. In S. B. Klein and R. R. Mowrer (Eds.), Contemporary learning theories: Pavlovian conditioning and the status of traditional learning theory (pp. 279-308). Hillsdale, NJ: Lawrence Erlbaum Associates. Dickinson, A. (1977). Appetitive-aversive interaction: Superconditioning of fear by an appetitive CS. Quarterly Journal of Experimental Psychology, 29, 71-83. Dickinson, A., & Dawson, G. R. (1987). Pavlovian processes in the motivational control of instrumental performance. Quarterly Journal of Experimental Psychology, 39B, 201-213. Dickinson, A., & Dearing, M. F. (1979). Appetitive-aversive interactions and inhibitory processes. In A. Dickinson and R. A. Boakes (Eds.). Mechanisms of /earning and motivation: A memorial volume to Jerzey Konorski (pp. 203-232). Hillsdale, NJ: Lawrence Erlbaum Associates. Dickinson, A., Hall, G., & Mackintosh, N. J. (1976). Journul of Experimental Psychology: Animal Behavior Processes, 2, 313-322. Dickinson, A., & Mackintosh, N. J. (1979). Reinforcer specificity in the enhancement of conditioning by posttrial surprise. Journal of Experimentul Psychology; Animul Behavior Processes, 5, 162-177. Dickinson, A., & Mulatero, C. W. (1989). Reinforcer specificity of the suppression of instrumental performance on a non-contingent schedule. Behavioural Processes, 19. 167-180. Durlach, P. J. (1983). Effect of signaling intertrial unconditioned stimuli in autoshaping. Journal

of Experimental

Psychology:

Animal

Behavior

Processes,

9, 374-389.

Feldman, J. M. (1971). Added cue control as a function of reinforcer predictability. of Experimental

Psychology.

Journal

91, 318-325.

Gamzu, E. R., & Williams, D. R. (1973). Associative factors underlying the pigeon’s key pecking in autoshaping procedures. Journal of the Experimental Analysis of Behavior, 19, 225-232.

Ganesan, R.. & Pearce, J. M. (1988). Effect of changing the unconditioned stimulus on appetitive blocking. Journal of Experimental Psychology: Animal Behavior Processes, 14, 280-291. Gibbon, J., & Balsam, P. (1981). Spreading associations in time. In C. M. Locurto, H. S. Terrace, & John Gibbon (Eds.), Autoshaping und conditioning theory (pp. 219253). New York: Academic Press. Goddard, M. J.. & Jenkins, H. M. (1987). Effect of signaling extra unconditioned stimuli on autoshaping. Animal Lenrning und Behavior, 11, 40-46. Goodman, J. H., & Fowler, H. (1983). Blocking and enhancement of fear conditioned by appetitive CSs. Animal Learning and Behavior, 11, 75-82. Holland, P. C. (1984). Unblocking in Pavlovian appetitive conditioning. Journal of Experimental

Psychology:

Animal

Behavior

Processes,

Holland, P. C. (1988). Excitation and inhibition Psychology:

Animal

Behavior

Processes,

10, 476-497.

in unblocking. Journal

14, 261-279.

of Experimental

EFFECT

OF INTERTRIAL

US PRESENTATIONS

285

Jenkins, H. M., and Moore, B. R. (1973). The form of the autoshaped response with food or water reinforcers. Journal of the Experimental Analysis of Behavior, 20, 163-181. Jenkins, H. M., Barnes, R. A., & Barrera, F. J. (1981). Why autoshaping depends on trial spacing. In C. M. Locurto, H. S. Terrace, & John Gibbon (Eds.), Autoshaping and conditioning theory (pp. 255-284). New York: Academic Press. Kamin, L. J. (1969). Predictability, surprise, attention and conditioning. In B. A. Campbell & R. M. Church (Eds.), Punishment and aversive behavior (pp. 279-296). New York: Appleton-Century-Crofts. Mackintosh, N. J., & Cotton, M. M. (1985). Conditioned inhibition from reinforcement reduction. In Ralph R. Miller & Norman E. Spear (Eds.), Information processing in animals: Conditioned inhibition (pp. 89-l 11). Hillsdale. NJ: Lawrence Erlbaum Associates. Neely, J. H., & Wagner, A. R. (1974). Attenuation of blocking with shifts in reward: The involvement of schedule-generated contextual cues. Journal of Experimental Psychology,

102, 751-763.

Nelson, K. (1987). Conditioned inhibition from incomplete reductions in the probability of reinforcement. Quarterly Journal of Experimental Psychology, 39B, 365-391. Reberg, D., Innis, N. K., Mann, B., & Eizenga. C. (1978). “Superstitious” behaviour resulting from periodic response-independent presentation of food or water. Animal Behaviour,

26,

507-5

19.

Rescorla, R. A. (1968). Probability of shock in the presence and absence of CS in fear conditioning. Journal of Comparative and Physiological Psychology, 66, 1-5. Rescorla, R. A. (1971). Variation in the effectiveness of reinforcement and nonreinforcement following prior inhibitory conditioning. Learning and Motivation, 2, 113-123. Rescorla, R. A. (1981). Within-signal learning in autoshaping. Animal Learning and Behavior.

9, 245-252.

Schreurs, B. G.. & Westbrook, R. F. (1982). The effects of changes in the CS-US interval during compound conditioning upon an otherwise blocked element. Quarterly Journal of Experimental

34B,

Psychology,

19-30.

Skelton, R. W., Spetch, M. L., & Wilkie, D. M. (1980). A method for automatically recording topographical differences in pigeons’ keypecking for food and water reinforcers. Behavior Research Method and Instrumentation, 12, 349-352. Spetch, M. L., Wilkie, D. M., & Skelton, R. W. (1981). Control of the pigeons’ keypecking topography by a schedule of alternating food and water reward. Animal Learning and Behavior,

9, 223-229.

Stanhope, K. J. (1989). Dissociation between the strength and the force of a conditioned response. Animal Learning and Behavior, 17, 311-321. Stanhope, K. J. (1988). Reinforcer representations in Pavlovian conditioning. Unpublished Ph.D. dissertation, University of Cambridge. Stickney, K. J., & Donahoe. J. W. (1983). Attenuation of blocking by a change in US locus.

Animal

Learning

and

Behavior,

11, 60-66.

Tomie, A. (1976a). Retardation of autoshaping: Control by prior contextual stimuli. Science, 192, 1244-1246. Tomie, A. (1976b). Interference with autoshaping by prior context conditioning. Journal of Experimental

Psychology:

Animal

Behavior

Processes,

2, 323-334.

Tomie, A. (1981). Effect of unpredicted food on the subsequent acquisition of autoshaping: Analysis of the context-blocking hypothesis. In C. M. Locurto, H. S. Terrace, & John Gibbon (Eds.), Autoshaping and conditioning theory (pp. 181-215). New York: Academic Press. Wagner, A. R. (1969). Stimulus validity and stimulus selection in associative learning. In N. J. Mackintosh & W. K. Honig (Eds.), Fundamental issues in associative learning (pp. 90-122). Halifax, Canada: Dalhousie University Press.

286

KELLY

J. STANHOPE

Wagner, A. R., Mazur, J. E., Donegan, N. H., & Pfautz, P. L. (1980). Evaluation of blocking and conditioned inhibition to a CS signaling a decrease in US intensity. Journal

of Experimental

Psychology:

Animal

Behavior

Processes,

6, 376-385.

Williams, B. A. (1989). The effect of response contingency and reinforcement identity on response suppression by alternative reinforcement. Learning and Motivation, 20,204224.

Received July 10, 1989 Revised January 8, 1990