- Email: [email protected]
Facilitation of responding in a filled-delay trace autoshaping procedure: An occasion-setting hypothesis
.EARNING
AND
20, 225-241 (1989)
MOTIVATION
Facilitation of Responding in a Filled-Delay Trace Autoshaping Procedure: An Occasion-Setting Hypothesis G. V. THOMAS, D. ROBERTSON, G. CUNNIFFE University
of Birmingham,
Birmingham,
England
AND
D. A. LIEBERMAN University
of Stirling,
Stirling,
Scotland
Rescorla (1982) has reported that pigeons’ responding to a trace key-light conditioned stimulus (CS) is facilitated if another stimulus fills the interval between that trace CS and the subsequent food unconditioned stimulus (US). This effect cannot be explained solely in terms of second-order conditioning of the CS by the interval stimulus (IS) because responding was facilitated more to a trace CS (CSl) followed on half its trials by both the IS and food than to a second CS (CS2) that was equally often followed by the same IS and food, but on different trials. To explain this result, Rescorla proposed that the gap-bridging IS catalyzed the association of CSI with food, perhaps by enhancing the memory of that CS when food occurred. Experiment 1 replicated the original finding and, additionally, showed that rate of responding to a key-light IS was greater following CSI than CS2. Experiment 2 showed that facilitated responding to the IS following CSI did not depend on spatial contiguity of CSI and the IS and thus could not simply be a consequence of higher rates of responding to CSl carrying over to an immediately following stimulus on the same key. This result suggests instead that higher rates to the IS following CSl were a consequence of CSs acting as occasion setters, signaling the trials on which the IS would be followed by food. In Experiment 3 we investigated whether such occasion-setting could itself account for the facilitation of responding to CSI: Both CSs were followed by an IS on every trial, but the IS-food contingency was varied. There Experiments 1 and 2 were based on a dissertation by G. Cunniffe submitted to Birmingham University in partial fulfillment of the requirements for a baccalaureate degree. Experiment 3 was supported by a grant from the United Kingdom Science and Engineering Research Council. We thank David Booth for his comments on an earlier version of this article. Address correspondence and reprint requests to Glyn V. Thomas, School of Psychology, University of Birmingham, P.O. Box 363, Birmingham, B15 2TT. England. 225
0023-%90/89 $3.00 Copyright 0 1989 by Academic Press, Inc. All rights of reproduction in any form reserved.
226
THOMAS
ET AL.
was more responding to a CS which provided information about the IS-food contingency than to a CS which did not. These results provide no evidence for catalysis of a CSl-food association by the IS but suggest that facilitation of responding to CSl could be a consequence of subjects learning the occasionsetting contingency inherent in the design of experiments used to investigate gap-filling effects. Q 1989 Academic Press, Inc.
Trace conditioning (Pavlov, 1927) involves presentation of a conditioned stimulus (CS) followed after a temporal gap by presentation of an unconditioned stimulus (US). It has long been known that trace conditioning is fess effective than delay conditioning in which the CS continues till the onset of the US (Pavlov, 1927). It has also been found that the longer the temporal gap in the trace procedure, the slower the acquisition of a conditioned response (CR) to the CS (see, for example, Kamin, 1965). These facts have often been taken to suggest that conditioning to the trace CS is mediated by a memorial representation of that CS which decays over time (but see Balsam & Gibbon, 1982). It has also been established that acquisition of responding to a trace CS is facilitated if another stimulus intervenes between the end of the CS and the onset of the US (Kaplan & Hearst, 1982; Pearce, Nicholas, & Dickinson, 1981; Rescorfa, 1982). Most of this facilitation can be explained in terms of second-order conditioning (Pavlov, 1927). The IS has close temporal contiguity with the US, becomes strongly associated with it, and in turn supports second-order conditioning to the immediately preceding trace CS. Thus responding to the trace CS is presumed to reflect the addition of the first-order conditioning of the CS to the US over the delay to the second-order conditioning from the immediately following interval stimulus (IS). In support of this interpretation, Pearce ef al. (1981) have found that facilitation of responding to the trace CS depends critically on the level of conditioning to the IS (see also Kaplan & Hearst, 1982). In one experiment (Experiment 4), however, Rescorfa (1982) observed a facilitation of responding to a trace CS by an IS in circumstances which rule out an explanation in terms of the summation of first- and secondorder conditioning. In this experiment, two CSs were followed by food and an IS on half of their occurrences. The CSs differed only in that one CS (CSI) was followed by an IS on the same trials as food, whereas food and the IS occurred on different trials for the other CS (CS2). With this “dissociation” design, presented schematically in Table 1, Rescorfa found that pigeons responded more to CSI , the CS for which the IS and food occurred on the same trial. Rescorfa (1982) argued that these results could not be explained in terms of simple first- and second-order conditioning because both CSs were followed by the IS and by (delayed) food with equal frequency. Indeed, according to most contemporary theories (e.g. Wagner. 1981)
FACILITATION
OF RESPONDING
227
TABLE 1 The Four Types of Trials in Rescorla (1982), Experiment 4 Trial type
CS
Interval condition
us
A B C D
CSl CSl cs2 cs2
Interval stimulus
Food No food Food No food
Interval stimulus
the signaling of the food following CSI by the IS should have actually overshadowed conditioning of an association between CSl and food. Furthermore, Rescorla assumed that second-order conditioning to CS 1 and CS2 would be equal because they were both followed by the same IS on half the trials. Insofar as the pairings of the IS with food (on A trials, Table 1) had established it as a CS, it should have contributed to second-order conditioning of CS2 (on D trials) just as much as to the conditioning of CSl (on A trials). Rescorla concluded that the most likely explanation was that the IS somehow acted as a “catalyst” to facilitate the learning of the association between the CS and the delayed food US. As to mechanisms, Rescorla noted that the catalytic effect could be perceptual in origin: “. . . events which are bridged in time appear to go together” (Rescorla, 1982, p. 140). Alternatively, he suggested that the IS might “better preserve it [the CS] in short-term memory so as to make it more available for association with the US” (Rescorla, 1982, p. 140). While neither of these proposals was articulated in detail, they both seem to invoke explanatory principles which might have considerable generality, applying to effects which have hitherto been interpreted solely in terms of the familiar principles of secondary reinforcement and second-order conditioning (e.g., the effects of chained schedules, see Kelleher & Gollub, 1962). We should note that only the experiments using the dissociation design produced results which clearly support a memorial or perceptual interpretation of the facilitation effect. Experiments 1, 2, and 3 (Rescorla, 1982), for example, used a simple trace conditioning procedure and obtained effects of an IS which could all be accounted for in terms of simple first- and second-order conditioning, as Rescorla himself conceded. Therefore, given the potential theoretical and empirical importance of the effect found in Rescorla’s Experiment 4, we planned a series of experiments to investigate it further. EXPERIMENT
1
The first experiment was designed as a simple replication of the procedure used in Experiment 4 of Rescorla’s 1982 paper. Pigeons were trained on a discrimination with a trace autoshaping procedure in which
228
THOMAS
ET AL.
two different key-light CSs were followed on half of their occurrences by food after a 5-s gap. For one CS (CSl), another key light stimulus intervened between the end of the trace CS and food, completely filling the temporal gap. The other CS (CS2) was also followed by the identical additional key light stimulus on half the trials, but not on those terminating in food. As in Rescorla’s study the subjects had been trained previously, but on different cues from those in the present experiment. Also as in Rescorla’s study, the subjects were preexposed to the stimuli subsequently used in this training. Method Subjects. The subjects were 16 Rock pigeons, approximately 18 months old and maintained at 80% of their free-feeding weights. All birds had participated in conditioning experiments but had never previously encountered the key light stimuli used in the present experiment. Apparatus. Three identical Lehigh Valley Electronics operant chambers for pigeons were used. The chambers measured 35 x 30 x 36 cm. The gray metal front panel contained a horizontal row of three response keys, approximately 25 cm above the floor and approximately 8.5 cm apart. Each key was 2.5 cm in diameter, and a 6 x 5-cm food hopper was located directly below the center key. Only the center key was used in this experiment, and on it could be back-projected a variety of colors and patterns. The operant chambers were placed in light- and soundattenuating shells, with ventilation fans providing masking noise. The experiments were controlled and data recorded by three PET microcomputers located in an adjoining room. Procedure. No magazine training was necessary because all the subjects had been trained to eat from the food hopper in their previous experiments. AH subjects were first given one session of extinction to the key stimuli used in their previous experiments (red, green, and white key lights) in which each of the stimuli was presented 12 times for 5 s without consequence. The stimuli were presented in a random order with an interstimulus interval of 60 s. All subjects then received one session of preexposure to the stimuli to be used in the present experiment, 12 presentations each of a vertical bar and a cross ( x ), and 24 presentations of a triangle. The stimuli were presented for 5 s in a random order and with an interval of 60 s between successive presentations. All subjects were then assigned at random to two groups of eight subjects each and trained for 12 days on a Pavlovian trace conditioning procedure derived from Rescorla (1982, Experiment 4). For subjects in the first group, each session contained 12 trials in which 5-s presentations of a vertical bar were followed immediately by 5 s of a triangle, followed by food: 12 trials in which the vertical bar was presented without con-
FACILITATION
229
OF RESPONDING
sequence; 12 trials in which 5-s presentations of a cross were followed by a 5-s gap which ended in food; and 12 trials in which the cross was followed by 5 s of the triangle but food was not given. Thus the triangle and food followed both the cross and the vertical bar equally often; but occurred on the same trials following the vertical bar, and on different trials following the cross. The second group of subjects was trained on exactly the same procedure except that the roles of vertical bar and cross stimuli were reversed. For both groups the different kinds of trials were presented in a different random order each session, with the restriction that each of the four kinds of trials occurred three times in each quarter of the session. The mean intertrial interval was 60 s and the chamber was illuminated with a house-light throughout each session, except during food presentations. Results The principal data from this experiment were the rates of responding to the two trace CSs. Figure 1 presents the mean rates of responding to CSl and CS2 (collapsed across stimulus identities), and also the rates of responding to the triangle when it followed CSl (on food trials) and CS2 (on nonfood trials). Inspection of Fig. 1 indicates a low level of responding to both CSI and CS2 prior to conditioning, and that rates of responding to both CSs increased over trials.
2
4
6 SESSIONS
8
10
12
FIG. I. Mean rates of responding to CSI and CSZ, and to the IS (IS) when it followed CSI and CS2 in each session of Experiment 1. The data have been collapsed across stimulus identities.
230
THOMAS
ET AL.
The data on CS responding were first submitted to a three-way ANOVA (subgroup x sessions x CS identity). There was no significant effect of subgroup (stimulus identities of CSl and CS2). There was a significant effect of trials (F(11, 154) = 6.35, p < .0.5), but the difference between rates of responding to CSl and CS2 failed to reach significance at the .05 probability level as did all the interaction effects. It seemed possible that the session-by-session use of Wilcoxon tests (Rescorla, 1982) might be more sensitive than the ANOVA at detecting within-subject differences in responding to CSl and CS2. This proved to be the case. Wilcoxon tests applied to the data from each session indicated that, by Session IO, subjects responded significantly more to CSl than to CS2 (~(14) = 2.34, two-tailed p < .05). A Wilcoxon test performed on the averaged data for the final six sessions also confirmed that in the final performance there was significantly more responding to CSl than to CS2 (~$14) = 2.35, two-tailed p < .05). Figure 1 also shows that differences in responding emerged to the IS (triangle), depending on whether it followed CSI or CS2: Wilcoxon tests applied to rates of responding to the IS indicated that, over the final six sessions, responding to this stimulus was significantly higher when it followed CSl and led to food than when it followed CS2 and did not (z( 14) = 1.99, two-tailed p < .05). The correlation over sessions between the difference in group mean rates to the CSs (CSl-CS2) and the difference in group mean rates to the IS when it followed each of the CSs (IS following CS 1 - IS following CS2) was calculated using the Spearman rank correlation coefficient. This correlation was found to be positive and statistically significant (v = .59, t(l0) = 2.33, p < .05). Only 1 of the 16 subjects made more than two responses in total to the blank response key during the unfilled 5-s period following CS2 on food trials and CSl on nonfood trials. This subject responded on average at 18 responses per minute in the gap following CSl and on average at two responses per minute in that following CS2.
Perhaps the most important outcome of Experiment 1 is that it confirms Rescorla’s finding that a stimulus filling the interval between a trace CS and food can facilitate responding to that trace CS in ways which cannot be accounted for by simple first- and second-order conditioning. The present results add to those reported by Rescorla by indicating that higher rates of responding develop also to the IS when it follows CSl. There are at least two possible explanations for this result. The first is that the higher rates to the IS following CSl were merely a consequence of the higher rates to CSl carrying over to the immediately following IS on the same response key. Such a carry-over effect could
FACILITATION
OF RESPONDING
231
occur, for example, if the transition from CS to IS occurred in the middle of a burst of responding. A second, potentially more interesting, explanation for higher response rates to the IS following CSl is that subjects had learned that it would then terminate in food, whereas the same IS was unreinforced when it followed CS2. The contingency that the IS would lead to food when it followed CSl, but not when it followed CS2, is inherent in Rescorla’s dissociation design which was used in the present experiment. Such contingencies, in which one stimulus signals whether a second stimulus will be reinforced, have been described as occasionsetting (see Ross & Holland, 1981), and there is considerable evidence that rats and pigeons can be sensitive to them. The possibility of subjects learning this occasion-setting contingency is of particular interest here because such learning could provide the basis of an alternative to Rescorla’s proposed explanations of the facilitation of responding to CSl. There are several ways in which learning the occasion-setting contingency could facilitate CSl responding, none of them involving an enhanced CS-US association as Rescorla proposed. One possibility is that subjects attended more to CSl because it signaled when the subsequent IS would terminate with food, and this enhanced attention to a food-predictive and localized CS led the pigeons to peck more at that CS (Collins & Pearce, 1985). Learning the occasion-setting relation is assumed here not to have affected direct first-order conditioning of the CSs with food (cf. Holland, 1985). This attentional account also requires the assumption that pigeons should attend (and respond) more to cues (such as CSI) that signal reinforcement of the IS than those (such as CS2) that signal its nonreinforcement. In support of this assumption, more orientation and approach to cues that predict reinforcement than to those that predict nonreinforcement (even when both are equally informative) has been reported previously for a variety of experimental paradigms (see, for example, Wasserman, Franklin, & Hearst, 1974). This occasion-setting account is applicable both to the present experiment and to Rescorla’s original demonstration of the facilitation effect (Rescorla, 1982, Experiment 4). EXPERIMENT
2
The viability of an occasion-setting account of the facilitation of CSl responding depends crucially on the demonstration that subjects do in fact learn the occasion-setting contingency involving the CSs and the interval stimuli. The purpose of Experiment 2 was to investigate whether the differences in rates of responding to the IS found in Experiment 1 could be safely taken to mean that subjects had learned the occasionsetting contingencies on the IS or were merely a consequence of higher rates of responding from CSl carrying over to an immediately following IS on the same key.
232
THOMAS
ET AL.
The general procedure was exactly the same as in Experiment 1 except that the trace CSs were now presented on the right-hand response key and the IS was presented on the center response key. Any carryover of higher rates of responding to CSl should now not increase (and might even reduce) responding to the immediately following IS, now located on a different key to that on which the CSs were presented. Any differences in rates of responding to the IS observed with this arrangement would then be strong evidence that subjects had learned the occasionsetting contingency. Method Subjects. The same 16 birds used in Experiment 1 were maintained at 80% of their free-feeding weights. Apparatus. The apparatus was the same as that used in Experiment 1 except that the response keys were changed so that an illuminated cross (+) or square could be projected onto the right-hand key, and an orange disc onto the center key. Procedure. All subjects were first given three extinction sessions each consisting of 12 nonreinforced presentations of each of the stimuli used in Experiment I. Each stimulus was presented for 5 s, and the mean interval between presentations was 60 s. The subjects were then given one session of preexposure to the stimuli to be used in Experiment 2; 12 presentations each of the cross and square on the right-hand key and 24 presentations of the orange disc on the center key. Timing of these presentations was the same as in the preexposure phase of Experiment 1. There then followed 12 days of training on exactly the same trace conditioning procedure used in Experiment 1. In each session there were 24 trials each of CSl and CS2. An IS (the orange key light) and food followed both CSs on half their occurrences, on the same trials following CSl but on different trials following CS2. The eight subjects who had received the cross (x) as CSl in Experiment 1 now received the square as CSl and the cross (+ ) as CS2. For the remaining subjects the roles of cross and square were reversed. Whether as CSl or as CS2, the cross and square were always presented on the right-hand key. For all subjects the orange IS was always presented on the center key. Results
and Discussion
Figure stimulus CSl and sponding
2 presents rates of responding to CSl and CS2 collapsed over identities, and rates of responding to the IS when it followed CS2. Inspection of Figure 2 suggests that mean rates of reto CSl and CS2 were very similar at the start of training, but
FACILITATION
233
OF RESPONDING
160 1 0-a c--o 6-Q c--a
I
CSl cs2 ISAFTERCSI ISAFTERCSZ
I
I
2
4 ' 6 SESSIONS
b
1'
8
I
I
10
12
FIG. 2. Mean rates of responding to CSI and CS2, and to the IS (IS) when it followed CSl and CS2 in each session of Experiment 2. The data have been collapsed across stimulus identities.
diverged substantially over successive sessions of training. A Wilcoxon test indicated that the rates to the two CSs were significantly different over the final block of six sessions (~(16) = 2.04, two-tailed p < -05). Figure 2 also shows that early in training rates of responding to the IS following CSl were similar to those when it followed CS2; but that from Session 4 mean rates to the IS began to differ substantially depending on the previous CS. A Wilcoxon test performed on the averaged data for the final six sessions confirmed that, in the final performance, rates of responding to the IS were significantly higher when it followed CSl than when it followed CS2 (~(16) = 3.04, two-tailed p < .05). The ranked correlation between the group difference in rates to the CSs (CSICS2) and the group difference in rates to the IS when it followed each of the CSs (IS following CSl - IS following CS2) was calculated over all sessions and found to be positive and statistically significant (r = .89, t(l0) = 6.11, p < .05). The main outcome of Experiment 2, therefore, was that locating the CSs and the IS on different keys did not prevent the development of significant facilitation of responding to the IS when it followed CSl. Indeed, the magnitude of the facilitation effect observed in Experiment 2 was if anything greater than that observed in Experiment 1, suggesting that the carry over of CS responding played no role in the development of differential responding to the IS. The development of differential responding to the IS, therefore, must
234
THOMAS
ET AL.
be considered as strong evidence that subjects learned the sequential conditional discrimination that the IS would be followed by food only when preceded by CSI and not when preceded by CS2. It should also be noted that, as in Experiment 1, conditional discrimination of the IS developed concurrently with the facilitation of responding to CS 1, suggesting that the two effects might be related. The results of the present experiment suggest that facilitation of CSl responding cannot be the cause of rate differences to the IS and must now be considered instead as a possible consequence of them. EXPERIMENT 3 The third experiment was designed to assess whether occasion-setting by the CSs could produce a facilitation of CS 1 responding, independently of possible memory enhancement or perceptual integration produced by the IS. The experimental design was based on that used in Experiments 1 and 2 but now involved two different interval stimuli (see Table 2). As in the previous experiments, two trace CSs (CSl and CS2) were followed by interval stimulus and delayed food with similar overall frequency. Consequently, consideration of simple first- and second-order conditioning effects predicted equal responding to CSl and CS2. However, now CSl was immediately followed by one IS (P) on food trials and by a different IS (Q) on nonfood trials. CS2 was also followed equally often by interval stimuli P and Q, but there was no correlation between the interval stimuli and reinforcement with food on CS2 trials. Thus CSl but not CS2 would signal occasions on which stimulus P would terminate in food and on which stimulus Q would not. If learning the inherent occasion-setting contingencies enhances attention, and thus responding, to a predictive cue, then there should eventually be more responding to CSl than to CS2. On the other hand, accounts which attribute to an IS properties independent of its predictiveness of reinforcement (e.g., enhancing memory for the preceding CS) predict no difference in responding between CSl and CS2 with this procedure because both CSs were followed by an IS on food trials.
TABLE 2 The Four Types of Trials in Experiment Trial type A B C D
cs CSI CSI cs2 cs2
Interval stimulus
P or Q P or Q
3 US Food No food Food No food .~-
FACILITATION
OF
RESPONDING
235
Method Subjects. The subjects were 16 Rock pigeons, approximately 24 months old and maintained at 80% of their free-feeding weights. All birds had previously participated in conditioning experiments but had not participated in Experiments 1 or 2 and had never before encountered the key light stimuli used in the present experiment. Apparatus. The apparatus was exactly the same as that used in the previous experiments, but only the right-hand key was operative in this experiment. Procedure. No magazine training was necessary because all the subjects had been trained to eat from the food hopper in their previous experiments. All subjects were first given one session of extinction to the key stimuli used in their previous experiments (white vertical and horizontal lines on a dark background) in which each of the stimuli was presented 12 times for 5 s without consequence. The stimuli were presented in a random order with an interstimulus interval of 60 s. All subjects then received one session of preexposure to the stimuli to be used in the present experiment, i.e., 12 presentations each of a vertical bar, a cross (X ), and green and red key illumination. The stimuli were presented for 5 s in a random order and with an interval of 60 s between successive presentations. All subjects were then trained for 20 days on a trace conditioning procedure with two trace CSs (CSI and CS2). In each session there were 24 5-s presentations of each CS, 12 of which were followed by food after a 5-s delay. On food trials CSl was followed by 5 s of one IS (P) and on nonfood trials by 5 s of a different stimulus (Q). On both food and nonfood trials CS2 was sometimes followed by stimulus P and sometimes by stimulus Q with equal probability. The result of this procedure was that CSI signaled when stimulus P would be followed by food but CS2 did not. The subjects were randomly assigned to two groups and for 10 subjects in the first group CSl was a vertical bar and CS2 was a cross ( x ). For the subjects in the second group the identities of CSl and CS2 were reversed. For half the subjects in each group IS P was a red key light and stimulus Q a green key light; for the remaining subjects in each group the colors of P and Q were reversed. For all subjects the mean intertrial interval was 60 s and the chamber was illuminated with a houselight throughout each session, except during food presentations. All other aspects of the procedure were the same as in Experiment 1. Results
The principal data from this experiment were the rates of responding to the two trace CSs. Figure 3 presents the mean rates of responding to
236
THOMAS
ET AL,
180. 160-
o-0 CSl
O-0
cs2
L--A
PAFTERCSl
6-6
PAFTER
CS2
m-m
QAFTER
IX.1
D-O
(1AFTERCS2
'
c
2
4
6
8 sEs:PclNs
12
14
16
18
20
FIG. 3. Mean rates of responding to CSI and CS2 and to interval stimuli P and Q when they followed CSl and CS2 respectively for each session of Experiment 3. The data have been collapsed across stimulus identities.
CSl and CS2 (collapsed across stimulus identities), and also the rates of responding to interval stimuli P and Q (also collapsed across stimulus identities). Inspection of Fig. 3 indicates that there was a low initial level of responding to both CSI and CS2, and that rates of responding to both CSs increased over the first four sessions, but then changed much more slowly. Figure 3 also shows that mean rates of responding to CSl became higher than those to CS2 after the initial rapid rise in mean response rates. A Wilcoxon test on the averaged data from the final block of five sessions indicated that the difference between the two CSs was statistically significant (~(16) = 2.99, two-tailed p < .05). Figure 3 also shows the development of conditional discriminations of the interval stimuli P and Q, depending on whether they followed CSl or CS2. A Wilcoxon test on the averaged data from the final block of five sessions indicated that response rates to stimulus P when it followed CSI (and always led to food) were not significantly higher than when it followed CS2 (and led to food on a random half of its trials). However, over the same final five sessions, response rates to stimulus Q were significantly lower (~(16) = 2.79, two-tailed p < .05) when it followed CSl (and never led to food) than when it followed CS2 (and led to food on a random half of its trials). The correlation over all sessions between the difference in group mean rates to the CSs (CSI-CS2) and the difference in group mean rates to
FACILITATION
OF
RESPONDING
231
the interval stimuli (IS following CSI - IS following CS2) were calculated separately for stimuli P and Q. The ranked correlation between the difference in response rates to the CSs and the difference in response rates to stimulus P when it followed CSl and CS2 was significantly positive (r = .66, t(l8) = 3.72, p < .05). The correlation between the difference in rates to the CSs and the difference in response rates to stimulus Q when it followed CSl and CS2 was significantly negative (r = -.77, t(H) = 5.04, p < .05). Finally, Fig. 3 also shows that there were overall higher rates to stimulus P than to stimulus Q (regardless of the identity of the preceding CS). This difference in response rates is consistent with the higher overall frequency of food reinforcement following P than Q. Discussion The results of Experiment 3 clearly indicate that higher rates of responding develop to a CS which signals whether a subsequent stimulus will terminate in food or not. Yet, as in Experiments 1 and 2, the design controls against the effects of simple first- and second-order conditioning. Since the present study was carried out we have learned that essentially similar results to those obtained in Experiment 3 were reported by Udell and Rescorla (1979) in a paper concerned with conditioning of simultaneous and successive common elements in discrimination procedures. The facilitation of responding to CSl observed in both these experiments cannot be accounted for in terms of an enhanced learning of the trace CSl-food association catalyzed by any stimulus in the CSl-food interval, because CSl and CS2 were both always followed by stimuli that completely filled the temporal gap between them and food. It follows, therefore, that some other mechanism must be involved. We suggest that the facilitation of CSl responding was mediated by subjects’ learning the occasion-setting contingency inherent in the dissociation design. Another aspect of the results of Experiment 3 which is also consistent with this suggestion is that significant differences in responding to CSl and CS2 did not develop until differences in responding to the interval stimuli had developed (indicating that subjects had learned the relevant conditional discrimination). The data also indicate that the contingencies on interval stimuli were not perfectly discriminated by the subjects because there were consistent differences in mean rates of responding to interval stimuli P and Q when they followed CS2 and had identical outcomes in terms of the probability of reinforcement. Generalization from CSl trials is the most likely reason for this effect. Nevertheless, the conditional discrimination of the interval stimuli P and Q depending on the preceding CS was clearly correlated with the emergence of significant facilitation of responding to CS 1.
238
THOMAS
ET AL.
Altogether, these results give strong support for the conclusion that responding is enhanced to a stimulus which sets the occasion for a following stimulus to be reinforced. GENERAL
DISCUSSION
The aim of the series of experiments reported here was to explore the mechanisms involved in the facilitation of responding to a trace CS by a stimulus completely filling the interval between that CS and food. Gapfilling effects obtained in experiments not employing the dissociation design (e.g., Pearce et al., 1981; Rescorla, 1982, Experiments l-3) can all be accounted for in terms of the summation of first- and second-order conditioning. We are concerned here only with demonstrations of this effect which have employed the dissociation design developed by Rescorla (1982) to control for the effects of first- and second-order conditioning. It is now apparent that the facilitation of CSl responding found with the dissociation design could be explained in at least two ways: Higher rates of responding to CSl could be a consequence of an enhanced CSUS association, catalyzed by perceptual or memorial effects of a gapfilling IS as proposed by Rescorla (1982). Alternatively, the higher rates of CSl responding could be a by-product of subjects learning the occasion-setting contingencies inherent in the dissociation design. The present study has demonstrated that subjects do learn the occasion-setting contingency involving the CSs, the IS, and food (Experiment 2), and that occasion-setting effects alone can enhance rates of responding to CSl, in circumstances which preclude any catalytic effects of the IS (Experiment 3). In contrast, there is no currently available evidence for catalytic effects of a gap-filling stimulus on a trace CS-US association. Although the present results do not refute the possibility that gap-filling stimuli can serve a catalytic function, they invalidate the current basis for postulating such a function. There are several possible mechanisms by which occasion-setting could facilitate responding to CSI. As already noted, learning the occasionsetting contingency could engender increased attention to a predictive CSl. Yet another mechanism by which learning the occasion-setting contingency could lead to enhanced attention and pecking to CSl can be derived from the Pearce-Hall model of Pavlovian conditioning (Pearce & Hall, 1980). A rather different mechanism whereby learning the occasion-setting contingency could facilitate CS 1 responding involves enhanced second-order conditioning of CSl . In Experiment 1, for example, the IS was preceded by CSl on food trials and by CS2 on nonfood trials. Given that subjects were able to discriminate the IS when it followed CSl from when it followed CS2 (supported by the differences in responding to the IS), then CSl would receive greater second-order con-
FACILITATION
OF RESPONDING
239
ditioning than CS2 because it was followed by a more powerful firstorder stimulus (the conditionally discriminated IS). Greater second-order conditioning of CSl could then produce the observed facilitation of responding to that CS. All these hypotheses of how exactly occasion-setting produced extra responding to CSl are consistent with the available data and cannot yet be separated. All share a common implication, however, that the facilitation of CSl responding need not reflect an enhanced association between CSI and food, but could be a by-product of the occasion-setting relations among the trace CSs, the interval stimuli, and food. The present account can also accommodate the effect of devaluing the reinforcer after trace conditioning with an interval stimulus (Rescorla, 1982, Experiment 5). That experiment found that a trace CS followed by an IS and then a reinforcer (a first-order CS) was more sensitive to subsequent devaluation of the reinforcer than a comparable trace CS established with an unfilled gap. Rescorla concluded that this result supported his contention that the interval stimulus had a catalytic effect, facilitating the association of the trace CS and its reinforcer. This conclusion rests on the presumption that first- and second-order autoshaping in this experiment involved associations between representations of stimuli, with the consequence that subjects could integrate learning across different phases of the experiment (see, for example, Leyland, 1977). If we assume, however, that reinforcer devaluation can also be integrated with the conditional discriminations that subjects learned among CSs, interval stimuli, and reinforcement, then the reported effects of reinforcer devaluation could also be reconciled with an occasion-setting account of the original facilitation effect. It should be noted, however, that the present occasion-setting hypothesis cannot readily account for facilitation of responding to a trace CS produced by presentation of a marking stimulus (see Thomas, Robertson, & Lieberman, 1987). In the marking procedure a brief salient stimulus (a bright flash of light) immediately followed a trace CS, and thus intervened between the CS and US. Unlike the IS in the present study, however, the marking stimulus did not completely fill the delay interval. Thomas et al. (1987) suggested that the brief flash of light enhanced responding to the preceding CS by marking it in memory, so that it was more likely to be recalled and associated with reinforcement at the end of the delay (see also Lieberman, McIntosh, & Thomas, 1979). There are several reasons why an occasion-setting reinterpretation of the marking results is implausible. First, it seems to us that it would not be easy for subjects to learn an occasion-setting relation involving a brief stimulus presented only at the start of the delay interval (and therefore not contiguous with reinforcement). Second, in their Experiment 4 using the dissociation design, Thomas et al. found facilitation of responding
240
THOMAS
ET AL.
to CSl at an early stage in training, before the stage at which subjects in the present experiments had learned apparently easier occasion-setting contingencies. Similarly, in discrimination experiments not using Rescorla’s dissociation design (Experiments 1, 2, and 3 of Thomas et al., 1987), acquisition of differential responding to a reinforced trace CS (CSl) was facilitated by an immediately subsequent brief stimulus right from the start of training. To explain these latter results in terms of occasionsetting would require the unlikely assumption that subjects found it easier to learn the conditional discrimination that the brief stimulus would be followed by food only after CSl than the simpler direct relation between CSl and food. Finally, occasion-setting cannot readily explain why Thomas et al. (1987) found that a brief IS facilitated responding to CSl during acquisition but not at asymptote, the exact opposite of the pattern of facilitation obtained in the present experiments with an IS completely bridging the delay interval. At present, therefore, we do not consider that occasion-setting played a significant role in the facilitation effects obtained with briefly presented salient stimuli (markers). Equally, it also seems to us unlikely that marking in memory played a significant role in the facilitation of CSl responding investigated in the present experiments. In the procedure used in Experiment 3, for example, CSl and CS2 were both followed by an interval stimulus on reinforced trials. If the interval stimulus had marked the preceding CS in memory then it should have marked CSl and CS2 equally, so that consideration of marking effects cannot explain the facilitation of responding to CSl which was observed in that experiment. In addition to overshadowing and simple second-order conditioning, there appear to be at least two other mechanisms which may operate when a stimulus intervenes between a trace CS and food, namely marking and occasion-setting. Thomas et al. (1987) found that marking may occur if the intervening stimulus is brief, salient, and follows immediately after the preceding trace CS. The present study found that occasion-setting may occur if the trace CS signals when the IS will lead to reinforcement. Learning this occasion-setting contingency in an autoshaping paradigm could produce either or both of enhanced second-order conditioning of the CS or enhanced attention, and therefore pecking, to the CS. The present results clearly favor an occasion-setting account of the effects of an IS completely filling the delay interval between a trace CS and food in facilitating responding to that CS. There is currently no evidence for a perceptual or memorial interpretation of the effects of such an IS. REFERENCES Balsam. P. D., & Gibbon, J. (1982). Factors underlying trace decrements in autoshaping. Behaviour Analysis Letters, 2, 197-204. Collins, L., 62 Pearce. J. M. (1985). Predictive accuracy and the effects of partial rein-
FACILITATION
241
OF RESPONDING
forcement on serial autoshaping. Journal of Experimental Psychology: Animal Behavior Processes, 11, 548-564. Holland, P. C. (1985). The nature of conditioned inhibition in serial and simultaneous feature negative discriminations. In R. R. Miller & N. E. Spear (Eds.), Information processing in animals: Conditioned inhibition (pp. 267-297). Hillsdale, NJ: Erlbaum. Kamin, L. J. (1965). Temporal and intensity characteristics of the conditioned stimulus. In W. F. Prokasy (Ed.), Classical conditioning: A symposium (pp. 118-147). New York: Appleton-Century-Crofts. Kaplan, P., &Hearst, E. (1982). Bridging temporal gaps between CS and US in autoshaping: Insertion of other stimuli before, during, and after CS. Journal of Experimental Psychology: Animal Behavior Processes, 8, 187-203. Kelleher, R. T., & Gollub, L. R. (1962). A review of positive conditioned reinforcement. Journal
of the Experimental
Analysis
Leyland, C. M. (1977). Higher-order chology,
of Behavior,
5. 543-597. Journal
autoshaping. Quarterly
of Experimental
Psy-
29, 607-619.
Lieberman. D. A., McIntosh, D. C.. & Thomas, G. V. (1979). Learning when reward is delayed: A marking hypothesis. Journal of Experimental Psychology: Animal Behavior Processes,
5, 224-242.
Pavlov, 1. P. (1927). Conditioned reflexes. Oxford, England: Oxford Univ. Press. Pearce, J. M., & Hall. G. (1980). A model for Pavlovian learning: Variations in the effectiveness of conditioned but not of unconditioned stimuli. Psychological Review. 87, 532-552. Pearce, J. M., Nicholas, D. J., & Dickinson, A. (1981). The potentiation effect during serial conditioning. Quarterly Journal of Experimental Psychology, 33B, 159-179. Ross, R. T., & Holland, P. C. (1981). Conditioning of simultaneous and serial featurepositive discriminations. Animal Learning & Behavior, 9, 293-303. Rescorla, R. A. (1982). Effect of a stimulus intervening between CS and US in autoshaping. Journal of Experimental Psychology: Animal Behavior Processes, 8, 131-141. Thomas. G. V., Robertson, D., & Lieberman, D. A. (1987). Marking effects in Pavlovian trace conditioning. Journal of Experimental Psychology: Animal Behavior Processes, 13, 126-135. Udell, H., & Rescorla, R. A. (1979). Conditioning of simultaneous and successive common elements in a discrimination and pseudodiscrimination. Bulletin of the Psychonomic Society, 14, 453-456. Wagner, A. R. (1981). SOP: A model of automatic memory processing in animal behavior. In N. E. Spear & R. R. Miller (Eds.). Information processing in animals: Memory mechanisms (pp. 5-47). Hillsdale, NJ: Erlbaum. Wasserman, E. A., Franklin, S. R., & Hearst, E. (1974). Pavlovian appetitive contingencies and approach versus withdrawal to conditioned stimuli in pigeons. Journal of Comparative and Physiological Psychology, 86, 616-627. Received November 14, 1988 Revised March 10, 1989
AND
20, 225-241 (1989)
MOTIVATION
Facilitation of Responding in a Filled-Delay Trace Autoshaping Procedure: An Occasion-Setting Hypothesis G. V. THOMAS, D. ROBERTSON, G. CUNNIFFE University
of Birmingham,
Birmingham,
England
AND
D. A. LIEBERMAN University
of Stirling,
Stirling,
Scotland
Rescorla (1982) has reported that pigeons’ responding to a trace key-light conditioned stimulus (CS) is facilitated if another stimulus fills the interval between that trace CS and the subsequent food unconditioned stimulus (US). This effect cannot be explained solely in terms of second-order conditioning of the CS by the interval stimulus (IS) because responding was facilitated more to a trace CS (CSl) followed on half its trials by both the IS and food than to a second CS (CS2) that was equally often followed by the same IS and food, but on different trials. To explain this result, Rescorla proposed that the gap-bridging IS catalyzed the association of CSI with food, perhaps by enhancing the memory of that CS when food occurred. Experiment 1 replicated the original finding and, additionally, showed that rate of responding to a key-light IS was greater following CSI than CS2. Experiment 2 showed that facilitated responding to the IS following CSI did not depend on spatial contiguity of CSI and the IS and thus could not simply be a consequence of higher rates of responding to CSl carrying over to an immediately following stimulus on the same key. This result suggests instead that higher rates to the IS following CSl were a consequence of CSs acting as occasion setters, signaling the trials on which the IS would be followed by food. In Experiment 3 we investigated whether such occasion-setting could itself account for the facilitation of responding to CSI: Both CSs were followed by an IS on every trial, but the IS-food contingency was varied. There Experiments 1 and 2 were based on a dissertation by G. Cunniffe submitted to Birmingham University in partial fulfillment of the requirements for a baccalaureate degree. Experiment 3 was supported by a grant from the United Kingdom Science and Engineering Research Council. We thank David Booth for his comments on an earlier version of this article. Address correspondence and reprint requests to Glyn V. Thomas, School of Psychology, University of Birmingham, P.O. Box 363, Birmingham, B15 2TT. England. 225
0023-%90/89 $3.00 Copyright 0 1989 by Academic Press, Inc. All rights of reproduction in any form reserved.
226
THOMAS
ET AL.
was more responding to a CS which provided information about the IS-food contingency than to a CS which did not. These results provide no evidence for catalysis of a CSl-food association by the IS but suggest that facilitation of responding to CSl could be a consequence of subjects learning the occasionsetting contingency inherent in the design of experiments used to investigate gap-filling effects. Q 1989 Academic Press, Inc.
Trace conditioning (Pavlov, 1927) involves presentation of a conditioned stimulus (CS) followed after a temporal gap by presentation of an unconditioned stimulus (US). It has long been known that trace conditioning is fess effective than delay conditioning in which the CS continues till the onset of the US (Pavlov, 1927). It has also been found that the longer the temporal gap in the trace procedure, the slower the acquisition of a conditioned response (CR) to the CS (see, for example, Kamin, 1965). These facts have often been taken to suggest that conditioning to the trace CS is mediated by a memorial representation of that CS which decays over time (but see Balsam & Gibbon, 1982). It has also been established that acquisition of responding to a trace CS is facilitated if another stimulus intervenes between the end of the CS and the onset of the US (Kaplan & Hearst, 1982; Pearce, Nicholas, & Dickinson, 1981; Rescorfa, 1982). Most of this facilitation can be explained in terms of second-order conditioning (Pavlov, 1927). The IS has close temporal contiguity with the US, becomes strongly associated with it, and in turn supports second-order conditioning to the immediately preceding trace CS. Thus responding to the trace CS is presumed to reflect the addition of the first-order conditioning of the CS to the US over the delay to the second-order conditioning from the immediately following interval stimulus (IS). In support of this interpretation, Pearce ef al. (1981) have found that facilitation of responding to the trace CS depends critically on the level of conditioning to the IS (see also Kaplan & Hearst, 1982). In one experiment (Experiment 4), however, Rescorfa (1982) observed a facilitation of responding to a trace CS by an IS in circumstances which rule out an explanation in terms of the summation of first- and secondorder conditioning. In this experiment, two CSs were followed by food and an IS on half of their occurrences. The CSs differed only in that one CS (CSI) was followed by an IS on the same trials as food, whereas food and the IS occurred on different trials for the other CS (CS2). With this “dissociation” design, presented schematically in Table 1, Rescorfa found that pigeons responded more to CSI , the CS for which the IS and food occurred on the same trial. Rescorfa (1982) argued that these results could not be explained in terms of simple first- and second-order conditioning because both CSs were followed by the IS and by (delayed) food with equal frequency. Indeed, according to most contemporary theories (e.g. Wagner. 1981)
FACILITATION
OF RESPONDING
227
TABLE 1 The Four Types of Trials in Rescorla (1982), Experiment 4 Trial type
CS
Interval condition
us
A B C D
CSl CSl cs2 cs2
Interval stimulus
Food No food Food No food
Interval stimulus
the signaling of the food following CSI by the IS should have actually overshadowed conditioning of an association between CSl and food. Furthermore, Rescorla assumed that second-order conditioning to CS 1 and CS2 would be equal because they were both followed by the same IS on half the trials. Insofar as the pairings of the IS with food (on A trials, Table 1) had established it as a CS, it should have contributed to second-order conditioning of CS2 (on D trials) just as much as to the conditioning of CSl (on A trials). Rescorla concluded that the most likely explanation was that the IS somehow acted as a “catalyst” to facilitate the learning of the association between the CS and the delayed food US. As to mechanisms, Rescorla noted that the catalytic effect could be perceptual in origin: “. . . events which are bridged in time appear to go together” (Rescorla, 1982, p. 140). Alternatively, he suggested that the IS might “better preserve it [the CS] in short-term memory so as to make it more available for association with the US” (Rescorla, 1982, p. 140). While neither of these proposals was articulated in detail, they both seem to invoke explanatory principles which might have considerable generality, applying to effects which have hitherto been interpreted solely in terms of the familiar principles of secondary reinforcement and second-order conditioning (e.g., the effects of chained schedules, see Kelleher & Gollub, 1962). We should note that only the experiments using the dissociation design produced results which clearly support a memorial or perceptual interpretation of the facilitation effect. Experiments 1, 2, and 3 (Rescorla, 1982), for example, used a simple trace conditioning procedure and obtained effects of an IS which could all be accounted for in terms of simple first- and second-order conditioning, as Rescorla himself conceded. Therefore, given the potential theoretical and empirical importance of the effect found in Rescorla’s Experiment 4, we planned a series of experiments to investigate it further. EXPERIMENT
1
The first experiment was designed as a simple replication of the procedure used in Experiment 4 of Rescorla’s 1982 paper. Pigeons were trained on a discrimination with a trace autoshaping procedure in which
228
THOMAS
ET AL.
two different key-light CSs were followed on half of their occurrences by food after a 5-s gap. For one CS (CSl), another key light stimulus intervened between the end of the trace CS and food, completely filling the temporal gap. The other CS (CS2) was also followed by the identical additional key light stimulus on half the trials, but not on those terminating in food. As in Rescorla’s study the subjects had been trained previously, but on different cues from those in the present experiment. Also as in Rescorla’s study, the subjects were preexposed to the stimuli subsequently used in this training. Method Subjects. The subjects were 16 Rock pigeons, approximately 18 months old and maintained at 80% of their free-feeding weights. All birds had participated in conditioning experiments but had never previously encountered the key light stimuli used in the present experiment. Apparatus. Three identical Lehigh Valley Electronics operant chambers for pigeons were used. The chambers measured 35 x 30 x 36 cm. The gray metal front panel contained a horizontal row of three response keys, approximately 25 cm above the floor and approximately 8.5 cm apart. Each key was 2.5 cm in diameter, and a 6 x 5-cm food hopper was located directly below the center key. Only the center key was used in this experiment, and on it could be back-projected a variety of colors and patterns. The operant chambers were placed in light- and soundattenuating shells, with ventilation fans providing masking noise. The experiments were controlled and data recorded by three PET microcomputers located in an adjoining room. Procedure. No magazine training was necessary because all the subjects had been trained to eat from the food hopper in their previous experiments. AH subjects were first given one session of extinction to the key stimuli used in their previous experiments (red, green, and white key lights) in which each of the stimuli was presented 12 times for 5 s without consequence. The stimuli were presented in a random order with an interstimulus interval of 60 s. All subjects then received one session of preexposure to the stimuli to be used in the present experiment, 12 presentations each of a vertical bar and a cross ( x ), and 24 presentations of a triangle. The stimuli were presented for 5 s in a random order and with an interval of 60 s between successive presentations. All subjects were then assigned at random to two groups of eight subjects each and trained for 12 days on a Pavlovian trace conditioning procedure derived from Rescorla (1982, Experiment 4). For subjects in the first group, each session contained 12 trials in which 5-s presentations of a vertical bar were followed immediately by 5 s of a triangle, followed by food: 12 trials in which the vertical bar was presented without con-
FACILITATION
229
OF RESPONDING
sequence; 12 trials in which 5-s presentations of a cross were followed by a 5-s gap which ended in food; and 12 trials in which the cross was followed by 5 s of the triangle but food was not given. Thus the triangle and food followed both the cross and the vertical bar equally often; but occurred on the same trials following the vertical bar, and on different trials following the cross. The second group of subjects was trained on exactly the same procedure except that the roles of vertical bar and cross stimuli were reversed. For both groups the different kinds of trials were presented in a different random order each session, with the restriction that each of the four kinds of trials occurred three times in each quarter of the session. The mean intertrial interval was 60 s and the chamber was illuminated with a house-light throughout each session, except during food presentations. Results The principal data from this experiment were the rates of responding to the two trace CSs. Figure 1 presents the mean rates of responding to CSl and CS2 (collapsed across stimulus identities), and also the rates of responding to the triangle when it followed CSl (on food trials) and CS2 (on nonfood trials). Inspection of Fig. 1 indicates a low level of responding to both CSI and CS2 prior to conditioning, and that rates of responding to both CSs increased over trials.
2
4
6 SESSIONS
8
10
12
FIG. I. Mean rates of responding to CSI and CSZ, and to the IS (IS) when it followed CSI and CS2 in each session of Experiment 1. The data have been collapsed across stimulus identities.
230
THOMAS
ET AL.
The data on CS responding were first submitted to a three-way ANOVA (subgroup x sessions x CS identity). There was no significant effect of subgroup (stimulus identities of CSl and CS2). There was a significant effect of trials (F(11, 154) = 6.35, p < .0.5), but the difference between rates of responding to CSl and CS2 failed to reach significance at the .05 probability level as did all the interaction effects. It seemed possible that the session-by-session use of Wilcoxon tests (Rescorla, 1982) might be more sensitive than the ANOVA at detecting within-subject differences in responding to CSl and CS2. This proved to be the case. Wilcoxon tests applied to the data from each session indicated that, by Session IO, subjects responded significantly more to CSl than to CS2 (~(14) = 2.34, two-tailed p < .05). A Wilcoxon test performed on the averaged data for the final six sessions also confirmed that in the final performance there was significantly more responding to CSl than to CS2 (~$14) = 2.35, two-tailed p < .05). Figure 1 also shows that differences in responding emerged to the IS (triangle), depending on whether it followed CSI or CS2: Wilcoxon tests applied to rates of responding to the IS indicated that, over the final six sessions, responding to this stimulus was significantly higher when it followed CSl and led to food than when it followed CS2 and did not (z( 14) = 1.99, two-tailed p < .05). The correlation over sessions between the difference in group mean rates to the CSs (CSl-CS2) and the difference in group mean rates to the IS when it followed each of the CSs (IS following CS 1 - IS following CS2) was calculated using the Spearman rank correlation coefficient. This correlation was found to be positive and statistically significant (v = .59, t(l0) = 2.33, p < .05). Only 1 of the 16 subjects made more than two responses in total to the blank response key during the unfilled 5-s period following CS2 on food trials and CSl on nonfood trials. This subject responded on average at 18 responses per minute in the gap following CSl and on average at two responses per minute in that following CS2.
Perhaps the most important outcome of Experiment 1 is that it confirms Rescorla’s finding that a stimulus filling the interval between a trace CS and food can facilitate responding to that trace CS in ways which cannot be accounted for by simple first- and second-order conditioning. The present results add to those reported by Rescorla by indicating that higher rates of responding develop also to the IS when it follows CSl. There are at least two possible explanations for this result. The first is that the higher rates to the IS following CSl were merely a consequence of the higher rates to CSl carrying over to the immediately following IS on the same response key. Such a carry-over effect could
FACILITATION
OF RESPONDING
231
occur, for example, if the transition from CS to IS occurred in the middle of a burst of responding. A second, potentially more interesting, explanation for higher response rates to the IS following CSl is that subjects had learned that it would then terminate in food, whereas the same IS was unreinforced when it followed CS2. The contingency that the IS would lead to food when it followed CSl, but not when it followed CS2, is inherent in Rescorla’s dissociation design which was used in the present experiment. Such contingencies, in which one stimulus signals whether a second stimulus will be reinforced, have been described as occasionsetting (see Ross & Holland, 1981), and there is considerable evidence that rats and pigeons can be sensitive to them. The possibility of subjects learning this occasion-setting contingency is of particular interest here because such learning could provide the basis of an alternative to Rescorla’s proposed explanations of the facilitation of responding to CSl. There are several ways in which learning the occasion-setting contingency could facilitate CSl responding, none of them involving an enhanced CS-US association as Rescorla proposed. One possibility is that subjects attended more to CSl because it signaled when the subsequent IS would terminate with food, and this enhanced attention to a food-predictive and localized CS led the pigeons to peck more at that CS (Collins & Pearce, 1985). Learning the occasion-setting relation is assumed here not to have affected direct first-order conditioning of the CSs with food (cf. Holland, 1985). This attentional account also requires the assumption that pigeons should attend (and respond) more to cues (such as CSI) that signal reinforcement of the IS than those (such as CS2) that signal its nonreinforcement. In support of this assumption, more orientation and approach to cues that predict reinforcement than to those that predict nonreinforcement (even when both are equally informative) has been reported previously for a variety of experimental paradigms (see, for example, Wasserman, Franklin, & Hearst, 1974). This occasion-setting account is applicable both to the present experiment and to Rescorla’s original demonstration of the facilitation effect (Rescorla, 1982, Experiment 4). EXPERIMENT
2
The viability of an occasion-setting account of the facilitation of CSl responding depends crucially on the demonstration that subjects do in fact learn the occasion-setting contingency involving the CSs and the interval stimuli. The purpose of Experiment 2 was to investigate whether the differences in rates of responding to the IS found in Experiment 1 could be safely taken to mean that subjects had learned the occasionsetting contingencies on the IS or were merely a consequence of higher rates of responding from CSl carrying over to an immediately following IS on the same key.
232
THOMAS
ET AL.
The general procedure was exactly the same as in Experiment 1 except that the trace CSs were now presented on the right-hand response key and the IS was presented on the center response key. Any carryover of higher rates of responding to CSl should now not increase (and might even reduce) responding to the immediately following IS, now located on a different key to that on which the CSs were presented. Any differences in rates of responding to the IS observed with this arrangement would then be strong evidence that subjects had learned the occasionsetting contingency. Method Subjects. The same 16 birds used in Experiment 1 were maintained at 80% of their free-feeding weights. Apparatus. The apparatus was the same as that used in Experiment 1 except that the response keys were changed so that an illuminated cross (+) or square could be projected onto the right-hand key, and an orange disc onto the center key. Procedure. All subjects were first given three extinction sessions each consisting of 12 nonreinforced presentations of each of the stimuli used in Experiment I. Each stimulus was presented for 5 s, and the mean interval between presentations was 60 s. The subjects were then given one session of preexposure to the stimuli to be used in Experiment 2; 12 presentations each of the cross and square on the right-hand key and 24 presentations of the orange disc on the center key. Timing of these presentations was the same as in the preexposure phase of Experiment 1. There then followed 12 days of training on exactly the same trace conditioning procedure used in Experiment 1. In each session there were 24 trials each of CSl and CS2. An IS (the orange key light) and food followed both CSs on half their occurrences, on the same trials following CSl but on different trials following CS2. The eight subjects who had received the cross (x) as CSl in Experiment 1 now received the square as CSl and the cross (+ ) as CS2. For the remaining subjects the roles of cross and square were reversed. Whether as CSl or as CS2, the cross and square were always presented on the right-hand key. For all subjects the orange IS was always presented on the center key. Results
and Discussion
Figure stimulus CSl and sponding
2 presents rates of responding to CSl and CS2 collapsed over identities, and rates of responding to the IS when it followed CS2. Inspection of Figure 2 suggests that mean rates of reto CSl and CS2 were very similar at the start of training, but
FACILITATION
233
OF RESPONDING
160 1 0-a c--o 6-Q c--a
I
CSl cs2 ISAFTERCSI ISAFTERCSZ
I
I
2
4 ' 6 SESSIONS
b
1'
8
I
I
10
12
FIG. 2. Mean rates of responding to CSI and CS2, and to the IS (IS) when it followed CSl and CS2 in each session of Experiment 2. The data have been collapsed across stimulus identities.
diverged substantially over successive sessions of training. A Wilcoxon test indicated that the rates to the two CSs were significantly different over the final block of six sessions (~(16) = 2.04, two-tailed p < -05). Figure 2 also shows that early in training rates of responding to the IS following CSl were similar to those when it followed CS2; but that from Session 4 mean rates to the IS began to differ substantially depending on the previous CS. A Wilcoxon test performed on the averaged data for the final six sessions confirmed that, in the final performance, rates of responding to the IS were significantly higher when it followed CSl than when it followed CS2 (~(16) = 3.04, two-tailed p < .05). The ranked correlation between the group difference in rates to the CSs (CSICS2) and the group difference in rates to the IS when it followed each of the CSs (IS following CSl - IS following CS2) was calculated over all sessions and found to be positive and statistically significant (r = .89, t(l0) = 6.11, p < .05). The main outcome of Experiment 2, therefore, was that locating the CSs and the IS on different keys did not prevent the development of significant facilitation of responding to the IS when it followed CSl. Indeed, the magnitude of the facilitation effect observed in Experiment 2 was if anything greater than that observed in Experiment 1, suggesting that the carry over of CS responding played no role in the development of differential responding to the IS. The development of differential responding to the IS, therefore, must
234
THOMAS
ET AL.
be considered as strong evidence that subjects learned the sequential conditional discrimination that the IS would be followed by food only when preceded by CSI and not when preceded by CS2. It should also be noted that, as in Experiment 1, conditional discrimination of the IS developed concurrently with the facilitation of responding to CS 1, suggesting that the two effects might be related. The results of the present experiment suggest that facilitation of CSl responding cannot be the cause of rate differences to the IS and must now be considered instead as a possible consequence of them. EXPERIMENT 3 The third experiment was designed to assess whether occasion-setting by the CSs could produce a facilitation of CS 1 responding, independently of possible memory enhancement or perceptual integration produced by the IS. The experimental design was based on that used in Experiments 1 and 2 but now involved two different interval stimuli (see Table 2). As in the previous experiments, two trace CSs (CSl and CS2) were followed by interval stimulus and delayed food with similar overall frequency. Consequently, consideration of simple first- and second-order conditioning effects predicted equal responding to CSl and CS2. However, now CSl was immediately followed by one IS (P) on food trials and by a different IS (Q) on nonfood trials. CS2 was also followed equally often by interval stimuli P and Q, but there was no correlation between the interval stimuli and reinforcement with food on CS2 trials. Thus CSl but not CS2 would signal occasions on which stimulus P would terminate in food and on which stimulus Q would not. If learning the inherent occasion-setting contingencies enhances attention, and thus responding, to a predictive cue, then there should eventually be more responding to CSl than to CS2. On the other hand, accounts which attribute to an IS properties independent of its predictiveness of reinforcement (e.g., enhancing memory for the preceding CS) predict no difference in responding between CSl and CS2 with this procedure because both CSs were followed by an IS on food trials.
TABLE 2 The Four Types of Trials in Experiment Trial type A B C D
cs CSI CSI cs2 cs2
Interval stimulus
P or Q P or Q
3 US Food No food Food No food .~-
FACILITATION
OF
RESPONDING
235
Method Subjects. The subjects were 16 Rock pigeons, approximately 24 months old and maintained at 80% of their free-feeding weights. All birds had previously participated in conditioning experiments but had not participated in Experiments 1 or 2 and had never before encountered the key light stimuli used in the present experiment. Apparatus. The apparatus was exactly the same as that used in the previous experiments, but only the right-hand key was operative in this experiment. Procedure. No magazine training was necessary because all the subjects had been trained to eat from the food hopper in their previous experiments. All subjects were first given one session of extinction to the key stimuli used in their previous experiments (white vertical and horizontal lines on a dark background) in which each of the stimuli was presented 12 times for 5 s without consequence. The stimuli were presented in a random order with an interstimulus interval of 60 s. All subjects then received one session of preexposure to the stimuli to be used in the present experiment, i.e., 12 presentations each of a vertical bar, a cross (X ), and green and red key illumination. The stimuli were presented for 5 s in a random order and with an interval of 60 s between successive presentations. All subjects were then trained for 20 days on a trace conditioning procedure with two trace CSs (CSI and CS2). In each session there were 24 5-s presentations of each CS, 12 of which were followed by food after a 5-s delay. On food trials CSl was followed by 5 s of one IS (P) and on nonfood trials by 5 s of a different stimulus (Q). On both food and nonfood trials CS2 was sometimes followed by stimulus P and sometimes by stimulus Q with equal probability. The result of this procedure was that CSI signaled when stimulus P would be followed by food but CS2 did not. The subjects were randomly assigned to two groups and for 10 subjects in the first group CSl was a vertical bar and CS2 was a cross ( x ). For the subjects in the second group the identities of CSl and CS2 were reversed. For half the subjects in each group IS P was a red key light and stimulus Q a green key light; for the remaining subjects in each group the colors of P and Q were reversed. For all subjects the mean intertrial interval was 60 s and the chamber was illuminated with a houselight throughout each session, except during food presentations. All other aspects of the procedure were the same as in Experiment 1. Results
The principal data from this experiment were the rates of responding to the two trace CSs. Figure 3 presents the mean rates of responding to
236
THOMAS
ET AL,
180. 160-
o-0 CSl
O-0
cs2
L--A
PAFTERCSl
6-6
PAFTER
CS2
m-m
QAFTER
IX.1
D-O
(1AFTERCS2
'
c
2
4
6
8 sEs:PclNs
12
14
16
18
20
FIG. 3. Mean rates of responding to CSI and CS2 and to interval stimuli P and Q when they followed CSl and CS2 respectively for each session of Experiment 3. The data have been collapsed across stimulus identities.
CSl and CS2 (collapsed across stimulus identities), and also the rates of responding to interval stimuli P and Q (also collapsed across stimulus identities). Inspection of Fig. 3 indicates that there was a low initial level of responding to both CSI and CS2, and that rates of responding to both CSs increased over the first four sessions, but then changed much more slowly. Figure 3 also shows that mean rates of responding to CSl became higher than those to CS2 after the initial rapid rise in mean response rates. A Wilcoxon test on the averaged data from the final block of five sessions indicated that the difference between the two CSs was statistically significant (~(16) = 2.99, two-tailed p < .05). Figure 3 also shows the development of conditional discriminations of the interval stimuli P and Q, depending on whether they followed CSl or CS2. A Wilcoxon test on the averaged data from the final block of five sessions indicated that response rates to stimulus P when it followed CSI (and always led to food) were not significantly higher than when it followed CS2 (and led to food on a random half of its trials). However, over the same final five sessions, response rates to stimulus Q were significantly lower (~(16) = 2.79, two-tailed p < .05) when it followed CSl (and never led to food) than when it followed CS2 (and led to food on a random half of its trials). The correlation over all sessions between the difference in group mean rates to the CSs (CSI-CS2) and the difference in group mean rates to
FACILITATION
OF
RESPONDING
231
the interval stimuli (IS following CSI - IS following CS2) were calculated separately for stimuli P and Q. The ranked correlation between the difference in response rates to the CSs and the difference in response rates to stimulus P when it followed CSl and CS2 was significantly positive (r = .66, t(l8) = 3.72, p < .05). The correlation between the difference in rates to the CSs and the difference in response rates to stimulus Q when it followed CSl and CS2 was significantly negative (r = -.77, t(H) = 5.04, p < .05). Finally, Fig. 3 also shows that there were overall higher rates to stimulus P than to stimulus Q (regardless of the identity of the preceding CS). This difference in response rates is consistent with the higher overall frequency of food reinforcement following P than Q. Discussion The results of Experiment 3 clearly indicate that higher rates of responding develop to a CS which signals whether a subsequent stimulus will terminate in food or not. Yet, as in Experiments 1 and 2, the design controls against the effects of simple first- and second-order conditioning. Since the present study was carried out we have learned that essentially similar results to those obtained in Experiment 3 were reported by Udell and Rescorla (1979) in a paper concerned with conditioning of simultaneous and successive common elements in discrimination procedures. The facilitation of responding to CSl observed in both these experiments cannot be accounted for in terms of an enhanced learning of the trace CSl-food association catalyzed by any stimulus in the CSl-food interval, because CSl and CS2 were both always followed by stimuli that completely filled the temporal gap between them and food. It follows, therefore, that some other mechanism must be involved. We suggest that the facilitation of CSl responding was mediated by subjects’ learning the occasion-setting contingency inherent in the dissociation design. Another aspect of the results of Experiment 3 which is also consistent with this suggestion is that significant differences in responding to CSl and CS2 did not develop until differences in responding to the interval stimuli had developed (indicating that subjects had learned the relevant conditional discrimination). The data also indicate that the contingencies on interval stimuli were not perfectly discriminated by the subjects because there were consistent differences in mean rates of responding to interval stimuli P and Q when they followed CS2 and had identical outcomes in terms of the probability of reinforcement. Generalization from CSl trials is the most likely reason for this effect. Nevertheless, the conditional discrimination of the interval stimuli P and Q depending on the preceding CS was clearly correlated with the emergence of significant facilitation of responding to CS 1.
238
THOMAS
ET AL.
Altogether, these results give strong support for the conclusion that responding is enhanced to a stimulus which sets the occasion for a following stimulus to be reinforced. GENERAL
DISCUSSION
The aim of the series of experiments reported here was to explore the mechanisms involved in the facilitation of responding to a trace CS by a stimulus completely filling the interval between that CS and food. Gapfilling effects obtained in experiments not employing the dissociation design (e.g., Pearce et al., 1981; Rescorla, 1982, Experiments l-3) can all be accounted for in terms of the summation of first- and second-order conditioning. We are concerned here only with demonstrations of this effect which have employed the dissociation design developed by Rescorla (1982) to control for the effects of first- and second-order conditioning. It is now apparent that the facilitation of CSl responding found with the dissociation design could be explained in at least two ways: Higher rates of responding to CSl could be a consequence of an enhanced CSUS association, catalyzed by perceptual or memorial effects of a gapfilling IS as proposed by Rescorla (1982). Alternatively, the higher rates of CSl responding could be a by-product of subjects learning the occasion-setting contingencies inherent in the dissociation design. The present study has demonstrated that subjects do learn the occasion-setting contingency involving the CSs, the IS, and food (Experiment 2), and that occasion-setting effects alone can enhance rates of responding to CSl, in circumstances which preclude any catalytic effects of the IS (Experiment 3). In contrast, there is no currently available evidence for catalytic effects of a gap-filling stimulus on a trace CS-US association. Although the present results do not refute the possibility that gap-filling stimuli can serve a catalytic function, they invalidate the current basis for postulating such a function. There are several possible mechanisms by which occasion-setting could facilitate responding to CSI. As already noted, learning the occasionsetting contingency could engender increased attention to a predictive CSl. Yet another mechanism by which learning the occasion-setting contingency could lead to enhanced attention and pecking to CSl can be derived from the Pearce-Hall model of Pavlovian conditioning (Pearce & Hall, 1980). A rather different mechanism whereby learning the occasion-setting contingency could facilitate CS 1 responding involves enhanced second-order conditioning of CSl . In Experiment 1, for example, the IS was preceded by CSl on food trials and by CS2 on nonfood trials. Given that subjects were able to discriminate the IS when it followed CSl from when it followed CS2 (supported by the differences in responding to the IS), then CSl would receive greater second-order con-
FACILITATION
OF RESPONDING
239
ditioning than CS2 because it was followed by a more powerful firstorder stimulus (the conditionally discriminated IS). Greater second-order conditioning of CSl could then produce the observed facilitation of responding to that CS. All these hypotheses of how exactly occasion-setting produced extra responding to CSl are consistent with the available data and cannot yet be separated. All share a common implication, however, that the facilitation of CSl responding need not reflect an enhanced association between CSI and food, but could be a by-product of the occasion-setting relations among the trace CSs, the interval stimuli, and food. The present account can also accommodate the effect of devaluing the reinforcer after trace conditioning with an interval stimulus (Rescorla, 1982, Experiment 5). That experiment found that a trace CS followed by an IS and then a reinforcer (a first-order CS) was more sensitive to subsequent devaluation of the reinforcer than a comparable trace CS established with an unfilled gap. Rescorla concluded that this result supported his contention that the interval stimulus had a catalytic effect, facilitating the association of the trace CS and its reinforcer. This conclusion rests on the presumption that first- and second-order autoshaping in this experiment involved associations between representations of stimuli, with the consequence that subjects could integrate learning across different phases of the experiment (see, for example, Leyland, 1977). If we assume, however, that reinforcer devaluation can also be integrated with the conditional discriminations that subjects learned among CSs, interval stimuli, and reinforcement, then the reported effects of reinforcer devaluation could also be reconciled with an occasion-setting account of the original facilitation effect. It should be noted, however, that the present occasion-setting hypothesis cannot readily account for facilitation of responding to a trace CS produced by presentation of a marking stimulus (see Thomas, Robertson, & Lieberman, 1987). In the marking procedure a brief salient stimulus (a bright flash of light) immediately followed a trace CS, and thus intervened between the CS and US. Unlike the IS in the present study, however, the marking stimulus did not completely fill the delay interval. Thomas et al. (1987) suggested that the brief flash of light enhanced responding to the preceding CS by marking it in memory, so that it was more likely to be recalled and associated with reinforcement at the end of the delay (see also Lieberman, McIntosh, & Thomas, 1979). There are several reasons why an occasion-setting reinterpretation of the marking results is implausible. First, it seems to us that it would not be easy for subjects to learn an occasion-setting relation involving a brief stimulus presented only at the start of the delay interval (and therefore not contiguous with reinforcement). Second, in their Experiment 4 using the dissociation design, Thomas et al. found facilitation of responding
240
THOMAS
ET AL.
to CSl at an early stage in training, before the stage at which subjects in the present experiments had learned apparently easier occasion-setting contingencies. Similarly, in discrimination experiments not using Rescorla’s dissociation design (Experiments 1, 2, and 3 of Thomas et al., 1987), acquisition of differential responding to a reinforced trace CS (CSl) was facilitated by an immediately subsequent brief stimulus right from the start of training. To explain these latter results in terms of occasionsetting would require the unlikely assumption that subjects found it easier to learn the conditional discrimination that the brief stimulus would be followed by food only after CSl than the simpler direct relation between CSl and food. Finally, occasion-setting cannot readily explain why Thomas et al. (1987) found that a brief IS facilitated responding to CSl during acquisition but not at asymptote, the exact opposite of the pattern of facilitation obtained in the present experiments with an IS completely bridging the delay interval. At present, therefore, we do not consider that occasion-setting played a significant role in the facilitation effects obtained with briefly presented salient stimuli (markers). Equally, it also seems to us unlikely that marking in memory played a significant role in the facilitation of CSl responding investigated in the present experiments. In the procedure used in Experiment 3, for example, CSl and CS2 were both followed by an interval stimulus on reinforced trials. If the interval stimulus had marked the preceding CS in memory then it should have marked CSl and CS2 equally, so that consideration of marking effects cannot explain the facilitation of responding to CSl which was observed in that experiment. In addition to overshadowing and simple second-order conditioning, there appear to be at least two other mechanisms which may operate when a stimulus intervenes between a trace CS and food, namely marking and occasion-setting. Thomas et al. (1987) found that marking may occur if the intervening stimulus is brief, salient, and follows immediately after the preceding trace CS. The present study found that occasion-setting may occur if the trace CS signals when the IS will lead to reinforcement. Learning this occasion-setting contingency in an autoshaping paradigm could produce either or both of enhanced second-order conditioning of the CS or enhanced attention, and therefore pecking, to the CS. The present results clearly favor an occasion-setting account of the effects of an IS completely filling the delay interval between a trace CS and food in facilitating responding to that CS. There is currently no evidence for a perceptual or memorial interpretation of the effects of such an IS. REFERENCES Balsam. P. D., & Gibbon, J. (1982). Factors underlying trace decrements in autoshaping. Behaviour Analysis Letters, 2, 197-204. Collins, L., 62 Pearce. J. M. (1985). Predictive accuracy and the effects of partial rein-
FACILITATION
241
OF RESPONDING
forcement on serial autoshaping. Journal of Experimental Psychology: Animal Behavior Processes, 11, 548-564. Holland, P. C. (1985). The nature of conditioned inhibition in serial and simultaneous feature negative discriminations. In R. R. Miller & N. E. Spear (Eds.), Information processing in animals: Conditioned inhibition (pp. 267-297). Hillsdale, NJ: Erlbaum. Kamin, L. J. (1965). Temporal and intensity characteristics of the conditioned stimulus. In W. F. Prokasy (Ed.), Classical conditioning: A symposium (pp. 118-147). New York: Appleton-Century-Crofts. Kaplan, P., &Hearst, E. (1982). Bridging temporal gaps between CS and US in autoshaping: Insertion of other stimuli before, during, and after CS. Journal of Experimental Psychology: Animal Behavior Processes, 8, 187-203. Kelleher, R. T., & Gollub, L. R. (1962). A review of positive conditioned reinforcement. Journal
of the Experimental
Analysis
Leyland, C. M. (1977). Higher-order chology,
of Behavior,
5. 543-597. Journal
autoshaping. Quarterly
of Experimental
Psy-
29, 607-619.
Lieberman. D. A., McIntosh, D. C.. & Thomas, G. V. (1979). Learning when reward is delayed: A marking hypothesis. Journal of Experimental Psychology: Animal Behavior Processes,
5, 224-242.
Pavlov, 1. P. (1927). Conditioned reflexes. Oxford, England: Oxford Univ. Press. Pearce, J. M., & Hall. G. (1980). A model for Pavlovian learning: Variations in the effectiveness of conditioned but not of unconditioned stimuli. Psychological Review. 87, 532-552. Pearce, J. M., Nicholas, D. J., & Dickinson, A. (1981). The potentiation effect during serial conditioning. Quarterly Journal of Experimental Psychology, 33B, 159-179. Ross, R. T., & Holland, P. C. (1981). Conditioning of simultaneous and serial featurepositive discriminations. Animal Learning & Behavior, 9, 293-303. Rescorla, R. A. (1982). Effect of a stimulus intervening between CS and US in autoshaping. Journal of Experimental Psychology: Animal Behavior Processes, 8, 131-141. Thomas. G. V., Robertson, D., & Lieberman, D. A. (1987). Marking effects in Pavlovian trace conditioning. Journal of Experimental Psychology: Animal Behavior Processes, 13, 126-135. Udell, H., & Rescorla, R. A. (1979). Conditioning of simultaneous and successive common elements in a discrimination and pseudodiscrimination. Bulletin of the Psychonomic Society, 14, 453-456. Wagner, A. R. (1981). SOP: A model of automatic memory processing in animal behavior. In N. E. Spear & R. R. Miller (Eds.). Information processing in animals: Memory mechanisms (pp. 5-47). Hillsdale, NJ: Erlbaum. Wasserman, E. A., Franklin, S. R., & Hearst, E. (1974). Pavlovian appetitive contingencies and approach versus withdrawal to conditioned stimuli in pigeons. Journal of Comparative and Physiological Psychology, 86, 616-627. Received November 14, 1988 Revised March 10, 1989