Acquisition and transfer of occasion setting in operant feature positive and feature negative discriminations

LEARNING

AND

MOTIVATION

22, 366-387 (191)

Acquisition and Transfer of Occasion Setting in Operant Feature Positive and Feature Negative Discriminations PETER C. HOLLAND Duke

University

The acquisition and transfer of stimulus control in discrete-trial operant feature positive and feature negative discriminations was examined in two experiments with rats. When the feature and target cues were presented simultaneously on compound training trials, the feature appeared to acquire simple excitatory or inhibitory control, which readily transferred to other target cues and which was substantially reduced by post-training counterconditioning. Conversely, when the feature preceded the target during training, the feature’s control was relatively more specific to responding in the presence of its original target and less influenced counterconditioning. These results are comparable to outcomes of analogous experiments (previously reported) that used Pavlovian conditioning procedures. 8 1991 Academic

Press. Inc.

The temporal arrangement of elements within a compound stimulus (XA) can affect the way rats solve Pavlovian appetitive feature positive (XA+, A-) and feature negative (A+, XA-) discriminations. Holland (1983, 1985) claimed that when X and A are presented simultaneously on compound trials, excitatory associations are formed between the feature X and the unconditioned stimulus (US) in feature positive discriminations, and inhibitory associations are formed between X and the US in feature negative discriminations. However, I further claimed that if the onset of X precedes the onset of A on compound trials in these discriminations, X acquires the ability to modulate the action of the association between A and the US, “setting the occasion” (Moore, Newman, & Glasgow, 1969; Skinner, 1938) for either responding or nonresponding to A. Considerable experimentation showed that this occasion setting or modThis research was supported by a grant from the National Science Foundation. I thank Marie Crock and Stephanie Nevels for their assistance in collecting and analyzing these data. Reprint requests should be sent to Peter Holland, Psychology Department, Duke University, Durham, NC 27706. 366 0023-9690191 $3.00 Copyright B 1991 by Academic Press. Inc. All rights of reproduction in any form resewed.

OCCASION

SETTING

367

ulation differs from simple excitation and inhibition. For example, the range of transfer of X’s occasion setting powers is often more limited than that of its simple excitatory or inhibitory powers. Although X’s simple associative strength sums with that of any target cue, its occasion setting powers often act only on cues explicitly trained as targets of occasion setters and sometimes are quite specific to X’s original target, A (e.g., Holland, 1986; 1989a, 1989~). Similarly, although X’s excitatory or inhibitory powers are masked by counterconditioning, its occasion setting powers are often substantially unaffected. Holland (1989b) found that after feature positive discrimination training, nonreinforced presentations of X eliminated x’s ability to evoke a conditioned response (CR), but had no effect on x’s ability to facilitate responding to A. Likewise, Holland (1989~) found that after feature negative discrimination training, reinforced presentations of X eliminated X’s simple inhibitory powers, but had no effect on X’s ability to suppress responding to A. The purpose of the experiments reported here was to extend research in Pavlovian occasion setting to an appetitive, discrete trial operant conditioning procedure with rats. It is of interest to determine whether variations in the temporal relations among the elements of a compound discriminative stimulus (Sd) alter the nature of operant discriminative control in the same way that those variations among the elements of compound conditioned stimuli ((2%) affect the nature of Pavlovian stimulus control. It might be argued, for example, that even a simple operant discrimination involves stimulus control akin to occasion setting, because the Sd sets the occasion for responding based on the response-reinforcer relation. A simple operant Sd may be functionally like a feature in a Pavlovian positive patterning (XA+, A -, X-) discrimination: responses (A) are reinforced (+) in the presence of the Sd (X), but reinforcement is not delivered in the presence of X unless A (the response) occurs, and A is not reinforced unless X is present. Recent data in fact suggest that operant Sds have much in common with Pavlovian occasion setters. For example, Davidson, Aparicio, and Rescorla (1988) found that a cue trained as an operant Sd effectively set the occasion for responding to a cue that had been trained as a target in Pavlovian occasion setting, and a Pavlovian occasion setter effectively controlled an operant that had previously been brought under the control of an explicit Sd. Consequently, the serial arrangement of compound cues in operant X + A+, A-, and A+, X 4 A- discriminations might be analogous to arranging “higher order” occasion setting relations in Pavlovian conditioning procedures (e.g., Arnold, Grahame, & Miller, 1991; Sidman, 1986), in which the higher order occasion setter indicates when a second stimulus will set the occasion for responding to a third cue. Thus, it is of some interest to compare the effects of temporal arrangement of the elements of compound cues on transfer and counterconditioning in operant

368 discriminations criminations. In Experiment on discrete trial (X --, A +, A -) examined those + A -) feature

PETER C. HOLLAND

with

the effects found

previously

in Pavlovian

dis-

1, I examined transfer and counterconditioning effects lever-pressing after simultaneous (XA + A -) and serial feature positive discriminations, and in Experiment 2, I effects after simultaneous (A+, XA-) and serial (A+, X negative discriminations. EXPERIMENT

1

In the acquisition phase of Experiment 1, lever presses during a tone were reinforced with sucrose delivery if the tone was accompanied by a light, but not if the tone had been presented alone. In Group Ser, an empty trace interval intervened between the light and the tone on compound trials, and in Group Sim, the light and tone were presented simultaneously on compound trials. In addition, in both groups, lever presses during a clicker were reinforced early in the acquisition phase, but nonreinforced later in training. This latter procedure was designed to train a potential target for assessment of transfer of the light’s occasion setting powers. The light’s ability to modulate responding to the clicker was then assessed in transfer tests. Finally, the light was presented alone repeatedly, with no reinforcement available, and its ability to modulate responding to the tone and clicker was reassessed. Method Subjects and apparatus. The subjects were 16 male albino rats bred in the Psychology Department of Duke University. They were about 140 days old at the beginning of the experiment and had not been involved in previous research. The rats were maintained at 80% of their ad lib body weights throughout the experiment by limiting their access to food. Water was available at all times in their individual home cages. There were eight identical experimental chambers, each 22.9 x 20.3 x 20.3 cm. The front and back walls of each chamber were aluminum; the side walls and top were clear acrylic. A food cup was recessed behind a 5 x 5 cm opening in the front wall; the bottom of the opening was 2 cm from the floor, and its center was 2 cm to the right of the center of the front wall. A 6-W jewelled lamp (“panel light”) was centered on the front wall, 4 cm above the top of the food cup opening. Four cm left of the food cup opening was a 2 x 2 cm lever, mounted 3 cm above the floor. During the first four sessions of lever press training, the lever was enlarged by clipping a 3.3 x 1.5 cm metal paper clamp over it. The floor of the chamber was composed of 0.48~cm stainless steel rods spaced 1.9 cm apart. Each of the chambers was enclosed in a sound-attenuating shell. A normally-off 6-W house light was mounted on one wall of the shell, 10 cm above the top of the chamber and 2 cm in front of and 10 cm to

OCCASION

SE’ITING

369

the left of the front wall of the chamber; 10 cm below the house light there was a speaker for delivering the auditory Css. Thus, facing the food cup, the house light was to the left and above the front panel of the chamber and the panel light was in the center, near the top of that wall panel. Finally, each shell was enclosed in another sound-attenuating box. Constant background noise (62.5 dB-A) was provided by a ventilating fan on each box. Procedure. First, the rats were trained to press the lever. In the first, 60-min session, the rats received twenty 0.3-ml deliveries of 0.2 M sucrose (the reinforcer used throughout these experiments) on a variable-time 2min schedule. Lever presses were reinforced during that period, and during the remaining 20 min of the session. In the second and third sessions, each rat was allowed to remain in its chamber until it had made about 50 lever presses. Lever presses were reinforced, but there were no response-independent sucrose presentations in these sessions. The next two sessions were designed to place lever pressing under the control of a noise stimulus, in order to both strengthen the lever press tendency and minimize intertrial lever pressing. Each of these sessions was 60 min long and included 60 presentations of a noise (72 dB-A), during which each lever press was reinforced; lever presses in the absence of the noise were not reinforced. In the first of these sessions each noise presentation was 30 s in duration; in the second session, each noise presentation was 15 s long. Then, discrimination training was started. In each of the first ten 60min sesions, there were three kinds of trials. C+ and A- trials were the same in both groups. C+ trials comprised 5-s presentations of a clicker (8 Hz, 64 dB), during which each response was reinforced. A- trials comprised 5-s presentations of a 1500-Hz tone (72 dB), during which responding had no programmed consequence. Compound trials differed between the groups. In Group Sim (n = 8), each of those XA + trials comprised a 5-s presentation of a compound of the house light and the tone, during which each response was reinforced. In Group Ser (n = 8) each X -+ A + compound trial consisted of a 5-s house light presentation followed, after a 5-s empty trace interval, by a 5-s tone presentation. Each response during the tone on compound trials was reinforced. In sessions l-5, there were 20 of each trial type, and in sessions 6-10 there were 12 of each of the two types of reinforced trials, and 36 of the nonreinforced A cue. Sessions 11-14 were similar to Sessions 6-10, except the 12 C’ trials were replaced by 12 C- trials, during which responding was not reinforced. Next, Transfer Test 1 was given, in order to examine the effects of X on responding to the trained-and-extinguished C cue. In a single 60-min session, 10 of each of five kinds of trials were administered. Both groups received 5-s nonreinforced tone (A -), clicker (C-), and house light (X-)

370

PETER C. HOLLAND

presentations. In Group Sim, those trials were intermixed with 5-s presentations of house light + clicker (XC-) and house light + tone (XA -) test compounds, and house light + tone trials (XT+) during which responding during the tone was reinforced (in order to maintain responding in the session). In Group Ser, A -, C, and X- trials were intermixed with serial 5-s house light -+ 5-s trace -+ 5-s tone (X -+ A-) and 5-s house light + 5-s trace + 5-s clicker (H -+ Cm) test trials, and 5-s house light --$ 5-s trace + 5-s tone refresher trials, on which responding during the tone was reinforced. The remaining phases of the experiment examined the effects of extinguishing the house light cue, X. First, in each of four 60-min sessions, both groups of rats received 40 nonreinforced 5-s presentations of the house light, and 20 reinforced 5-s presentations of the noise cue that had been presented in preliminary training (in order to maintain lever pressing and keep the rats awake during house light presentations). Then, Transfer Test 2, identical to Transfer Test 1, was administered. Unfortunately, responding to the trained and extinguished C cue alone was quite frequent in Test 2, perhaps as a result of generalization from noise training. Consequently, five additional extinction sessions were given, in order to reextinguish responding evoked by the clicker. In each of those sessions there were 10 reinforced presentations of the 5-s noise, 25 nonreinforced presentations of the house light (X), and 25 nonreinforced 5-s presentations of the clicker (C). Finally, a third Transfer Test session, identical to the first two, was administered. Results and Discussion Data analysis. The rate of lever pressing, the percentage of trials on which at least one response occurred, and the latency to the first response, during each 5-s interval of the CS presentations and the 5-s pre-CS interval were all recorded. Response rates did not provide an appropriate measure on reinforced trials because the rats did not press for several seconds after sucrose was delivered, artificially depressing the rates during reinforced trials. Consequently, the percentage of trials on which at least one response occurred was used as the primary measure in the various acquisition phases of these experiments. However, in test sessions, in which sucrose was not available on test trials, response rate appeared to be the most sensitive measure (mostly because it was less susceptible to ceiling effects), and I present those rates. The general pattern of test data was similar for all three measures, however. The measure of central tendency reported here is the median. Distribution free inferential statistics were used throughout, with the level of significance defined as 0.05, two-sided. The number of scores used in each statistical test is given parenthetically; values less than the number of

OCCASION

t;l

371

SETTING

loo-

z E-J

'

80

-

60.

1

2

3

4

5

6

7

8

9

1011121314

SESSIONS

‘r GROUP SER

0-O O---O l --•* o..o

GROUP SIY

A+ Ac+ c-

A-A Q ~3 v-v Tj v

FIG. 1. Acquisition of feature positive discrimination performance (percentage of trials on which at least one response occurred) in Experiment 1. A’ refers to reinforced responding during the tone target on serial compound trials in Group Ser (Serial), or during the house light + tone simultaneous compound trials in Group Sim (simultaneous). A- refers to nonreinforced responding during tone-alone trials. C’ refers to reinforced responding during the clicker in sessions l-10, and 17 refers to nonreinforced responding during the clicker during sessions 11-14 (indicated by arrow).

subjects in a condition (that is, less than 8) indicate tied observations in within-subjects comparisons. Acquisition. Acquisition of the serial and simultaneous feature positive discriminations proceeded at comparable rates (Fig. 1). Although over the first half of training responding was more likely during the reinforced A cue in Group Ser than during the reinforced XA cue in Group Sim, Mann-Whitney U(8, 8) = 10, discrimination difference scores of the two groups (XA minus A in Group Sim and A on X+ A trials minus A alone in Group Ser) did not differ during those sessions, U (8,8) = 22. Similarly, over the first half of training, the likelihood of responding on reinforced clicker trials was greater in Group Ser than in Group Sim, U(8, 8) = 11.5. There were no reliable between-groups differences in performance during any of the cues in the second half of training. Transfer tests. The top panel of Fig. 2 shows reponse rates during the various cues in Transfer Test 1. In Group Ser (left side of panel), X facilitated responding during both the original target cue (A) and the trained and extinguished cue (C): The rates of responding during A and C were greater when those cues were presented after X (the shaded bars of the pairs labeled A or C) than when they were presented alone (blank

372

PETER C. HOLLAND

40 I

GROUP

GROUP SIM

SER I

30..

TARGET IN COMPOUND 0 TARGET ALONE

20..

A 40 Y 3 z

30

I

GROUP

0

A

C

GROUP

30

0 SIM

TARGET IN COUPOUND 0 TARGET ALONE

A

0

C

GROUP

SER I

C

GROUP

SER I

A 40

C

0

SIM

TARGET IN COMPOUND 0 TARGET ALONE

i

A

C

A

0

C

0

TARGET

FIG. 2. Response rates during Transfer Tests 1 (top panel), 2 (middle), and 3 (bottom) of Experiment 1. The unshaded bars show responding during the original tone target presented alone (A), the transfer target presented alone (C), or the pre-CS interval (0). The shaded bars show responding on trials that included the feature (x) and the original tone target (A), the feature and the transfer clicker target (C), or the feature alone. In Group Sim the compounds were simultaneous; in Group Ser, the compounds were serial, with a 5-s empty interval between the house light feature and the target cue. In Group Sim, featurealone responding was recorded during the 5-s feature cue, in Group Ser it was recorded during the second 5-s period after the feature (when targets would normally be administered).

OCCASION

SETI’ING

373

bars), Wilcoxon Ts (8) = 0. However, transfer of X’s powers to the C target was not complete: discrimination difference scores were reliably greater for the X + A versus A discrimination that for the X + C versus C discrimination, T(8) = 4. Neither responding during the original A target nor that during the transfer C target was attributable solely to responding controlled directly by X in Group Ser. Responding during C on X -+ C trials, and during A on X + A trials, was more frequent than responding on X + 0 trials (shaded bar labeled 0) during the 5-s empty interval that corresponded to the time when reinforcement was available on X * A trials in training (i.e., the interval that began 5 s after the termination of X), Ts(8) G 1.5, and during the house light X cue itself, 1.8 responses/min, Ts(8) = 0. In Group Sim, responding was apparently controlled entirely by X. Responding did not differ significantly among X alone, XA, and XC trials, Ts (7 or 8) 3 7, and the XA versus A and XC versus C difference scores did not differ from each other, T(7) = 11. Thus, transfer of X’s power to a new compound was complete. There were no reliable between-group differences in responding controlled by the X feature, or in the size of the XA versus A or XC versus C discrimination difference scores, Us(8, 8) 2 19. However, there was some evidence for greater transfer in Group Sim than in Group Ser. A measure of transfer was constructed in each group by subtracting the XC versus C difference score from the XA versus A difference score. These differential transfer scores were marginally greater in Group Sim, U(8, 8) = 15, p < .lO. Over the four days of extinction of responding to the house light feature, the percentages of trials on which a response occurred decreased from 77.9% to 17.5% during the house light in Group Sim, and from 41.9% to 4.7% during the 5-s empty interval that began 5 s after termination of the house light in Group Ser. Those losses persisted in Transfer Test 2 (shaded bars labeled 0 in the center panel of Fig. 2): response rates during X in Group Sim and during the appropriate empty interval after X in Group Ser were significantly lower than in Transfer Test 1, Ts(8) = 0. The extinction sessions administered between Transfer Tests 1 and 2 affected discrimination performance differently in the two groups. In Group Ser (left side of center panel of Fig. 2), the X + C versus C discriminative transfer performance was destroyed, but performance on the original X --i* A versus A discrimination was not reliably affected. These data suggest that at least a portion of the apparent transfer observed in Group Ser on X + C trials in Test 1 may have been due to simple control by X alone, but performance observed on the original, X --i, A, training trials was not dependent on X’s ability to exert simple control over responding. Conversely, in Group Sim, performance on both discriminations was considerably disrupted by nonreinforced presentations

374

PETER C. HOLLAND

of X, as would be anticipated if responding during the compound cues reflected solely the strength of X. In Group Ser, the X + C versus C difference scores were reliably lower in Test 2 than in Test 1, T(8) = 0; indeed, X no longer enhanced responding to C on those trials, relative to C-alone trials, T(7) = 14. It is difficult to conclude, however, that extinction of responding to X completely eliminated its ability to modulate responding to a new target, because part of the reduced discrimination performance was due to an increase in responding to C alone, T(7) = 2, perhaps as a consequence of the reinforcement of responding during another auditory cue, the noise, during the intervening extinction procedure. On the other hand, the X + A versus A difference scores were not reliably lower in Test 2, T(6) = 5, in Group Ser. Furthermore, the change in difference scores between Tests 1 and 2 was reliably greater for the X + C versus C than the X + A versus A discrimination, T(8) = 1. Thus, in Group Ser, extinction of X had a greater decremental effect on transfer responding than on original discrimination performance. In Group Sim, both the original and transfer discrimination difference scores were reliably lower in Test 2 than in Test 1, E(8) = 0. Although X maintained some ability to add to the responding controlled by A alone, T(7) = 3, p < .lO, whereas responding during XC was no greater than responding during C alone, the XA versus A and XC versus C difference scores did not differ from each other, T(7) = 12. Furthermore, the change in difference scores between Tests 1 and 2 did not differ for the two discriminations, T(7) = 11. In Test 2, there were no reliable between-groups differences in responding to the X feature or in XC versus C transfer discrimination performance, Us@, 8) 2 21. However, both the XA versus A original discrimination difference scores and the XA versus A minus XC versus C differential transfer scores (see above) were significantly greater in Group Ser, Ts(8) 6 4. Furthermore, the change in both of those measures between Tests 1 and 2 was greater in Group Ser than in Group Sim, Ts(8) 6 2.5. Thus, extinction of X had considerably less effect on original discrimination performance in Group Ser, but equivalent effects on transfer performance in the two groups. It is worth noting that the equivalent effects of extinction on transfer performance in the two groups is consistent with my claim (above) that at least some of the apparent transfer performance in Group Ser is more properly described as the consequence of simple control of responding by the X feature, as in Group Sim. To minimize the influence of the inflated responding observed during C presentations in Test 2, nonreinforced presentations of both X and C were presented between Tests 2 and 3. Response rates during Test 3 are shown in the bottom panel of Fig. 2. The nonreinforced C presentations were effective in returning responding during C alone to low levels in

OCCASION

SETI-ING

375

both groups (unshaded bars labeled C). Despite the additional nonreinforced presentations of X, in Group Ser, X retained some of its ability to enhance responding to A, and regained some of its ability to enhance responding to C, Ts(8) = 0. Conversely, in Group Sim, response rates were no higher during either the XA or XC compounds than during A or C alone, R(7) 5 8. Between-groups comparisons showed performance on the original discrimination in Group Ser to be superior to that in Group Sim, U(8, 8) = 5, but no reliable difference in performance on the transfer discrimination, U(8, 8) = 17. Overall, these data indicate that the nature of control exerted by the feature X in feature positive discriminations depended on the arrangement of the elements of the reinforced compound cue. In Group Sim, which received training with simultaneous compounds, XA+, A-, lever pressing was controlled primarily by the house light feature (X) cue: response rates during the original XA compound, the transfer XC compound, and to X alone were similar, and extinction of responding during X alone produced large decrements in responding during the compound cues as well. Conversely, in Group Ser, which received training with serial compounds, X + A+, A-, control of lever pressing was not easily attributable to X alone: responding during X alone or in the trace intervals after it occurred at relatively low rates and transfer to the X + C compound was incomplete. More important, nonreinforcement of lever pressing during X alone had relatively little effect on lever press rates on X + A compound trials. It could be argued that the use of before-after comparisons and the lack of no-extinction controls weakens Experiment l’s assessments of the effects of feature extinction. Other changes besides those engendered by the nonreinforcement of responding during the feature-alone phase may have contributed to the observed response decrements. But it seems unlikely that those unspecified processes would act differentially on responding established by serial versus simultaneous compound procedures. Thus, the major conclusion from these comparisons, that of a differential effect of extinction after serial versus after simultaneous training, remains on relatively firm ground. The observation of less transfer and more immunity to feature extinction after serial than after simultaneous feature positive discrimination training is consistent with my previous findings in Pavlovian appetitive conditioning experiments (Holland, 1986, 1989a, 1989b). The data patterns were not identical, however. In Group Ser in this experiment, Xreliably augmented responding during the trained and extinguished transfer target (C) in both Tests 1 and 3, whereas in the Pavlovian experiments, some of which used the same physical cues and intratrial temporal intervals used in the present experiments, I observed no transfer of X’s facilitatory powers to trained and extinguished target cues. Unfortunately, the present experiments do not permit completely dis-

376

PETER C. HOLLAND

entangling a true transfer effect from a variety of other sources for the facilitation of responding to C. For example, in Test 1, some of the apparent transfer to C might be attributable to simple summation of responding controlled by X and C: responding controlled by X alone was relatively frequent, and extinction of that responding was observed to reduce X’s ability to modulate responding to C (Test 2). Or, X may have enhanced the response rate during C through some other, unspecified disinhibitory process. Furthermore, generalization and response mediation effects cannot be ruled out (e.g., generalization between the original tone and transfer clicker targets may have been enhanced, relative to that observed in earlier Pavlovian experiments, by their both controlling the same lever-press + food contingency). Although the evidence in Experiment 1 for transfer to trained and extinguished cues is not conclusive, it should be noted that such transfer is not implausible. Other investigations of occasion-setting-like phenomena have reported transfer to trained and extinguished cues. For example, Bouton and Swartzentruber (1986) found that contextual cues associated with the reinforcement of one explicit CS also potentiate conditioned responses elicited by other trained and extinguished cues, but not cues that were partially reinforced. And Rescorla (e.g., 1985) reported that in pigeon autoshaping a serial feature cue facilitated responding to a trained and extinguished cue, but not to weakly trained consistently reinforced cues. It is not clear why trained and extinguished cues serve as acceptable targets of transfer in these preparations, but not the Pavlovian preparation I examined earlier (e.g., Holland, 1986, 1989a). EXPERIMENT 2

Experiment 2 examined transfer and counterconditioning effects after serial and simultaneous feature negative discrimination training. Three groups of rats received feature negative discrimination training, in which responding during a tone was reinforced when it was presented alone, but not when the tone was accompanied by the house light. In Group Set-, an empty trace interval occurred between house light and tone presentations on compound trials (A +, X * A - ). Because the serial discrimination was much more difficult than the simultaneous discrimination, two groups of rats received training with simultaneous presentation of house light and tone on compound trials (A+, XA-), in order to separately equate for both performance level (Group Sim-E) and number of discrimination trials (Group Sim-L). In addition, a fourth group of rats (Group Con) received a simple discrimination between a reinforced tone and a nonreinforced house light (A +, X-), to provide a baseline against which to assess the effects of serial and simultaneous feature negative discrimination training. Next, transfer of X’s suppressive powers to a separately-trained Sd, C,

OCCASION

SETTING

377

was examined in a transfer test. In order to separate the effects of the temporal arrangement of the cues within the compounds on learning and on performance, the transfer test included both serial and simultaneous compound test trials for all groups. Finally, X was counterconditioned by reinforcing responding during X, and X’s ability to suppress responding was reassessed. Methods Subjects and apparatus. The subjects were 24 male and 4 female albino rats, whose origins, ages, history, and maintenance was similar to that described in Experiment 1. The apparatus comprised seven of the eight chambers used in Experiment 1. Procedure. First, the rats were trained to press the lever. In the first 60-min session, the rats received twenty 0.3-ml deliveries of 0.2 M sucrose on a variable-time 1-min schedule. Lever presses were reinforced during that period, and during the remaining 40 min of the session. The next four sessions were designed to place lever pressing under the control of a clicker stimulus, in order to both strengthen the lever press tendency and minimize intertrial lever pressing. Each of these sessions was 60 min long, and included sixty 30-s presentations of the clicker, during which each lever press was reinforced; lever presses in the absence of the clicker were not reinforced. Then, the rats were randomly assigned to one of four groups (ns = 7) and training of the A cue to be used in discrimination training was given. In each of two 60-min sessions, every response during each of sixty 5-s presentations of the tone were reinforced. Next, all subjects received five 60-min sessions of discrimination training. In each session, subjects received thirty 5-s presentation of the tone (A+), during which each lever press was reinforced, and 30 nonreinforced presentations of either a 5-s house light + 5-s empty trace + 5-s tone (X + A-) compound (Group Ser), a 5-s house light + tone (XA -) compound (Groups Sim-E and SimL), or the 5-s house light (X-) alone (Group Con). Subjects in Group Sim-E then proceeded immediately to the next phase, while the rats in the remaining groups continued discrimination training. Each of the 23 remaining sessions was similar to the previous five discrimination training sessions, except that there were 20 reinforced tone trials and 40 nonreinforced compound or house light alone trials in each session. All rats then received a single 60-min session in which lever presses during each of sixty 5-s clicker (C’) presentations were reinforced, followed by Transfer Test 1. In each of two 60-min sessions during which no sucrose reinforcement was given, all subjects received ten XA-, ten X+ A-, and ten A- presentations, in order to assess performance on the original discrimination, and ten C-, ten 5-s house light + 5-s trace

378

PETER

C. HOLLAND

+ 5-s clicker, and ten 5-s house light + clicker presentations to assess transfer of X’s powers to a separately trained Sd. Next, the rats in Groups Ser, Sim-L, and Con received four 60-min counterconditioning sessions in which responding on house light alone (X+) trials was reinforced (the rats in Group Sim-E did not participate in the remainder of the procedures of Experiment 2). In each session there were sixty 5-s house light presentations. In the first two sessions, sucrose was delivered independent of responding 5 s after the onset of the house light on half of the trials, and 15 s after the onset of the house light on the other half. Each response that occurred during the 5-s house light or during the second 5-s empty trace interval after house light termination was reinforced on both kinds of trials. In each of the next two sessions, the response-reinforcement contingencies were the same as in the first two sessions, but there was no response-independent reinforcement. The timing of reinforcement availability was intended to establish excitatory control both during the house light and during the interval after it in all three groups, so that comparable excitatory control would be evident during both serial and simultaneous test trials in Transfer Test 2. Then, responding during the clicker and tone Sds was refreshed by reinforcing lever presses during each of thirty 5-s presentations of each of those cues in one 60-min session. Finally, Transfer Test 2, indentical to the first session of Transfer Test 1, was administered. Results Figure 3 shows acquisition of the discriminations in Experiment 2. The A+, X-, control discrimination (Group Con), and the simultaneous A+, XA - discrimination (Groups Sim-E and Sim-L) were learned rapidly, but performance on the serial A+, X + A - discrimination was slow to develop. Even on the final 4-session block, performance was poorer in Group Ser: responding during A on nonreinforced X --, A trials in that group was reliably greater than responding during nonreinforced XA trials in Group Sim-L, U(7, 7) = 1, and during nonreinforced X trials in Group Con, U(7, 7) = 2. However, performance of Group Sim-E in its last 4session block (at least one response on 90.8% of A+ trials and on 24.3% of XA- trials) did not differ reliably from that of Group Ser, U(7, 7) = 18.5 for XA responding, so the effort to equate performance levels of those two groups was moderately successful. Figure 4 shows the results of Transfer Test 1. Consider first the performance of Group Ser (top left panel of Fig. 4). The original A+, X + A- discrimination was maintained: response rates were higher during A alone trials than during either serial X + A or simultaneous XA test trials, 7’s(7) = 0. Furthermore, both the A versus X + A and A versus XA discrimination difference scores were greater in Group Ser than in the unpaired control, Group Con, Us(7, 7) c 5.

OCCASION

W

SETTING

379

012345678

FOUR-SESSION BLOCKS

A-A

A---..A m-m

GROUP SAY *+

GROUP SIY MGROUP CON *+

FIG. 3. Acquisition of feature negative discrimination performance (percentage of trials on which at least one response occurred) in Experiment 2. A + refers to reinforced responding during the tone alone. A- refers to responding during the tone on serial compound trials in Group Ser (Serial); XA - refers to responding during the house light + tone simultaneous compound trials in Group Sim-L (simultaneous). X- refers to nonreinforced responding during house light-alone trials in Group Con XI. The performance of Group Sim-E is not shown in this Figure.

In addition, X’s suppressive power transferred to the separately trained excitor, C. Responding was greater on C-alone trials than on either X --f C, T(7) = 1, or XC, T(6) = 0, trials. However, transfer in Group Ser was less complete when assessed with serial test trials. First, the C versus X+ C difference scores were smaller than the A versus X + A difference scores, T(7) = 2, whereas no such difference was found on simultaneous test trials, T(7) = 10.5. Second, although the C versus XC difference scores were greater in Group Ser than in Group Con, U(7, 7) = 3, the C versus X + C difference scores did not differ reliably between the two groups, U(7, 7) = 14.5. Third, the difference between the original and transfer discrimination difference scores, a measure of the relative magnitude of transfer, was greater for the serial tests than for the simultaneous tests in Group Ser, T(7) = 2, but not in Group Con. Fourth, the relative magnitude of transfer (as just defined) on serial tests was greater in Group Ser than in Group Con, T(7) = 0, but no such between-group difference was observed on simultaneous test trials, T(7) = 14. Finally, the difference in magnitude of transfer on serial than on simultaneous test trials was larger in Group Ser than in Group Con, U(7, 7) = 11. Next, consider the performance of Groups Sim-L (top right panel) and

380

PETER C. HOLLAND

ORIGINAL

TRANSFER

ORIGINAL

TRANSFER

TARGET

-

ORIGINAL

SIMULTANEOUS

TRANSFER

TARGET FIG. 4. Response rates during Transfer Test 1 in the four groups (see text) of Experiment 2. The unshaded bars show responding during either the original tone target (left side of each graph) or the transfer clicker (C) target (right side), presented alone (A). The hatched bars show responding during the original A or transfer (C) targets on serial test trials, and the shaded bars show responding on simultaneous compound test trials that included the feature (x) and the original tone target (XA), or the feature and the transfer clicker target (Xc).

Sim-E (bottom right), which were trained with simultaneous, A+, XAprocedures. With serial test compounds (crosshatched bars), X had no reliable effect on responding during either A or C, Ts(7, 7) 2 10 in either group. This lack of an inhibitory effect in serial testing is consistent with data from Pavlovian conditioning experiments in my laboratory (e.g., Holland, 1989~; Holland & Lamarre, 1984); indeed, Pavlov (1927) reported that the termination of inhibitory cues provoked excitatory responding. With simultaneous test compounds (shaded bars), however, inhibition and transfer was essentially complete in Groups Sim-E and Sim-L. X suppressed responding during both the original (A) and transfer (C) excitors, R(7) = 0, and there were no reliable differences between the A versus XA and C versus XC difference scores in either group, Ts(7) 2 12. Furthermore, both the X versus XA and X versus XC difference scores were greater in Groups Sim-E and Sim-L than in Group Con, Us(7, 7) s 8.5. After Test 1, the subjects in Groups Ser, Sim-L, and Con received four

OCCASION

SETTING

381

...~.....I. a”..-‘/. I.

0

SESSIONS FIG. 5. Acquisition of lever pressing (percent trials with at least one response) during the house light cue in the counterconditioning phase of Experiment 2. The house light had been trained as X in an X-, A +, A - discrimination (Group Ser), an XA’, A- discrimination (Goup Sim), or a simple A*, Xm discrimination (Group Con).

sessions in which responding during X presentations was reinforced. The results of that counterconditioning phase are shown in Fig. 5. Responding during the house light itself and during the second empty trace interval after it were comparable throughout and were combined in Fig. 5 (mean of the medians). Over all four sessions, responding was reliably more frequent in Group Con than in either of the other two groups, Us(7, 7) < 11. Thus, both Group Sim-L and Group Ser passed a retardation or resistance to reinforcement test (Hearst, 1972; Rescorla, 1969) for identifying X as possessing conditioned inhibitory control. Although Fig. 5 hints of greater responding in Group Ser than in Group Sim, that difference was not reliable, U(7, 7) = 16, p = 0.32. Figure 6 shows responding during Transfer Test 2, which was administered after counterconditioning of X. Consider first the performance of Group Ser (top left panel). Despite controlling substantial responding, X maintained its ability to suppress responding to its original target, A. Test 2 responding during A on serial X + A trials was less frequent than responding on A-alone trials, T(7) = 0. In fact, responding during A on X+ A trials was also less frequent than responding during the comparable empty trace interval after X on X-alone trials, T(7) = 1.5. Furthermore, performance on the A versus X -+ A discrimination in Test 2 did not differ reliably from that in Test 1, T(7) = 9.5, indicating that counterconditioning had little effect on X’s ability to suppress responding to A on X + A trials. On the other hand, although Test 2 responding during the simultaneous XA compound was lower than responding on A-alone trials, T(6) = 1, the A versus XA difference scores were reliably lower in Test 2 than in

382

PETER C. HOLLAND

ORIGINAL

TRANSFER

ORIGINAL

TRANSFER

GROUP SIM-E 10

5

0

ORIGINAL

TRANSFER

ORIGINAL

TRANSFER

TARGET

TARGET 0

EXCITOR

m

SERIAL

m

SIMULTANEOUS

El 6. Response rates during Transfer Test 2 in the three remaining groups (see text) of Experiment 2. The unshaded bars show responding during either the original tone target (left side of each graph) or the transfer clicker (C) target (right side of each graph), presented alone (A). The hatched bars show responding during the original A or transfer (C) targets on serial test trials. The shaded bars show responding on simultaneous compound test trials that included the feature (x) and the original tone target (XA), or the feature and the transfer clicker target (XC). The lined bars show responding during the mean of the response rates during the house light and during the second 5-s interval after that cue. FIG.

Test 1, T(7) = 0. Thus, counterconditioning had some decremental effect on the display of X’s suppressive powers in simultaneous compound tests in Group Ser (but not enough to eliminate those powers). There was no evidence for maintenance of transfer performance in Group Ser, regardless of the type of test trials. Responding on C-alone trials in Test 2 was not reliably greater than responding on either XC or X --, C trials, Ts(7) 3 8. Furthermore, the C versus XC discrimination difference scores were reliably lower in Test 2 than in Test 1, T(7) = 0. X showed no evidence of suppressive powers with either the original A or transfer C target, on either serial or simultaneous test trials, in either Group Sim-L or Group Con, Ts(7) 2 10. Thus, the maintenance of discriminative responding within the original target discrimination in

OCCASION

SETTING

383

Group Ser was unique: indeed, only the original discrimination difference scores were greater in that group than in either of the others, Us(7, 7) s 10, other Us 2 19. Discussion On the whole, the results of Experiment 2 are consistent with the data from comparable Pavlovian appetitive conditioning experiments from this laboratory (Holland, 1989~). If one considers only the results of test compounds that used the same temporal sequence as was used in training, that is, serial compounds in Group Ser and simultaneous compounds in Groups Sim-E and Sim-L, transfer was greater after simultaneous training than after serial training, and counterconditioning abolished the simultaneously trained X’s suppressive powers, but had little effect on the serially trained X’s suppressive powers. This set of data is consistent with the data that led to my earlier claims that Pavlovian feature negative procedures generate simple conditioned inhibition when the elements of the compounds are presented simultaneously in training, but generate occasion setting, with quite different properties, when the elements are arranged serially. However, testing of X’s powers within simultaneous compounds revealed quite a different pattern in Group Ser: transfer was essentially complete in Test 1, and counterconditioning eliminated X’s suppressive effects. That is, performance of Group Ser when tested with simultaneous compounds resembled that of Group Sim, when that group was tested with simultaneous compounds. It might be argued that these data suggest that the effects of the temporal arrangement of the feature and target cues are on performance, rather than on learning (e.g., Rescorla, 1989). That is, there might be essentially no difference between the nature of control established with serial and simultaneous procedures, but rather transfer and counterconditioning effects are only revealed in simultaneous tests. However, both the absence of any suppressive effect of X on serial test trials in the simultaneously trained groups, and the maintenance of X’s suppressive powers on serial, but not simultaneous, trials after counterconditioning in Group Ser (despite equivalent control of responding by X alone at both test intervals), make that account less plausible. Another possible account for the results of the simultaneous tests of responding in Group Ser recognizes that it is unlikely that the serial compounds used in Experiment 2 would generate only negative occasion setting. The interval between the onset of X and the termination of A was very brief, 15 s. Evidence from Pavlovian conditioning experiments (Holland, 1989~) indicated that these temporal parameters established conditioned inhibition as well as occasion setting. Similarly, in Pavlovian feature positive experiments (e.g., Holland, 1989a, 1989b), and in Experiment 1 of this report, these same parameters endowed X with both

384

PETER C. HOLLAND

modulatory and simple excitatory control powers. Simple inhibitory powers of X in Group Ser might have been responsible for the substantial transfer observed with simultaneous test compounds in Test 1. (Note that these simple inhibitory powers would not generate transfer on serial test trials in Group Ser, because simple inhibition is apparently not revealed in serial tests: in Group Sim, X had no inhibitory effect on either A or X when it preceded those targets.) The results of Test 2 support the claim that the transfer observed with simultaneous compounds in Group Ser in Test 1 was due to simple inhibitory powers of X. Because counterconditioning would be anticipated to eliminate simple inhibition (which transfers readily), but leave negative occasion setting (which transfers less readily) relatively unaffected, one would expect counterconditioning to destroy evidence of transfer, but not evidence of suppression of responding to the original target (A). Similar support for the view that both serial and simultaneous procedures generated simple inhibition is given by the results of the counterconditioning phase itself: the acquisition of responding was retarded (relative to Group Con’s performance) in both Groups Sim-L and Ser, suggesting inhibitory control by X in both groups. Thus, taken together, the results of Tests 1 and 2 and the counterconditioning phase support a distinction between two kinds of inhibitory control in these operant conditioning experiments analogous to the distinction derived previously from Pavlovian conditioning experiments: simple inhibition, which transfers readily to new target cues and is masked by counterconditioning, and negative occasion setting, which transfers less readily, and is less affected by counterconditioning. However, it is worth pointing out that, unlike in the Pavlovian experiments, after serial training there was SoMe transfer of control to a separately trained target cue in serial compound testing (Test 1). That transfer is not easily attributable to simple inhibitory properties of X in Group Ser, because other features of these data (just discussed) suggest that simple inhibition is not revealed in serial tests. GENERAL DISCUSSION Discrete-trial operant feature positive and feature negative discrimination procedures yielded results comparable to those reported earlier from Pavlovian procedures (e.g., Holland, 1989a, 1989b, 1989~). When the feature and target cues were presented simultaneously on compound training trials, the feature appeared to acquire simple excitatory or inhibitory control, which readily transferred to other target cues and which was substantially reduced by post-training counterconditioning. Conversely, when the feature preceded the target during training, the feature’s control was relatively more specific to responding in the presence of its original target, and less influenced by counterconditioning. Elsewhere

OCCASION

SETTING

385

(e.g., Holland, 1983,1985), I have argued that this latter pattern of results suggests that with serial arrangements of stimulus elements within compounds, the feature cue acquires hierarchical control over responding that is elicited by another, target cue (occasion setting). The application of this notion to simultaneous and serial operant discrimination performance is less straightforward. First, it might be argued that the present data reflect the influence of Pavlovian, stimulus-stimulus contingencies only, rather than any direct modulation of the action of operant response-reinforcer contingencies. For example, one might claim that the serial feature cues in these experiments acted only by modulating the Pavlovian value of the target cues, which in turn influenced the expression of responding engendered by the operant response-reinforcer contingencies. Limited support for that argument is provided by some of the observations of Holland and Coldwell (1990) who found that, at least in some circumstances within this discrete trial operant procedure, the Pavlovian stimulus-stimulus relations were more critical than operant response-reinforcer or stimulus-response relations in the occurrence of serial transfer or counterconditioning phenomena. Second, it has frequently been argued that even simple discriminative control over operants (in the present experiments, that exerted by the simultaneously trained features, or by the targets alone) may be hierarchical. That is, operant responding may be based on a three-term relation, with the discriminative stimulus modulating the effectiveness of a response-reinforcer association (e.g., Colwill & Rescorla, 1990; Jenkins, 1985; Skinner, 1938). If even simple discriminative stimuli are thus comparable to occasion setters (e.g., Davidson et al., 1988), then the present experiments are not functionally equivalent to my previous studies of occasion setting in Pavlovian conditioning. Instead, they may be more analogous to investigations of higher order Pavlovian occasion setting (e.g., Arnold ef al., 1991), in which an occasion setter’s control is itself modulated by another cue. Consequently, the data reported here may support Thomas and Schmidt’s (1989) suggestion that arranging serial (but not simultaneous) relations among discriminative stimuli encourage animals to abstract four-term relations (Sidman, 1986) among the associative elements. If simple discriminative stimuli exert hierarchical control over responding generated by response-reinforcer relations, then the present experiments do not address the question of whether simple operant discriminative stimuli control responding in the same way that Pavlovian occasion setters control responding elicited by their targets. A more appropriate comparison to those Pavlovian experiments would contrast the nature of discriminative control when the opportunity to perform the response (and hence experience the response-reinforcer relation) is available in the presence of the discriminative cue, with the nature of control when the

386

PETER C. HOLLAND

opportunity to perform the response is available only after the termination of the discriminative cue. The analogous operant transfer question would be, does an Sd trained to modulate action of one response-reinforcer relation also modulate action of an association of another response and the reinforcer (e.g., Gutman & Maier, 1978; Hearst & Peterson, 1973)? Similarly, the operant analogue of extinguishing the feature in the absence of its target or reinforcer would be the presentation of the Sd at a time when the response could not be performed, as in “latent extinction” experiments (e.g., Seward & Levy, 1949). Regardless of the resolution of these issues, the present data suggest that temporal variables thought to affect the way rats solve Pavlovian conditional discriminations affect rats’ solution of discrete-trial operant discriminations in comparable manners. Future research will address the relation of these different solution mechanisms to the Pavlovian or operant control exerted by the cues. REFERENCES Arnold,

H. M., Grahame, N. J., & Miller,

Animal

Learning

& Behavior.

R. R. (1991). Higher-order

occasion setting.

19, 58-64.

Bouton, M. E., & Swartzentruber, D. (1986). Analysis of the associative and occasionsetting properties of contexts participating in a Pavlovian discrimination. Journal of Experimental

Psychology:

Animal

Behavior

Processes,

12, 333-350.

Colwill, R. M., & Rescorla, R. A. (1990). Evidence for the hierarchical structure of instrumental learning. Animal Learning & Behavior, 18, 71-82. Davidson, T. L., Aparicio, J., & Rescorla, R. A. (1988). Transfer between Pavlovian facilitators and instrumental discriminative stimuli. Animal Learning & Behavior, 16, 285-291. Gutman, A., & Maier, S. F. (1978). Operant and Pavlovian factors in cross-response transfer of inhibitory stimulus control. Learning and Motivation, 9, 231-254. Hearst, E. (1972). Some persistent problems in the analysis of conditioned inhibition. In R. A. Boakes & M. S. Halliday (Eds.), Inhibition and Learning (pp. 5-39). London: Academic Press. Hearst, E., & Peterson, G. B. (1973). Transfer of conditioned excitation and inhibition from one operant response to another. Journal of Experimental Psychology, 99, 360368. Holland, P. C. (1983). Occasion-setting in Pavlovian feature positive discriminations. In M. L. Commons, R. J. Herrnstein, & A. R. Wagner (Eds.), Quantitative analyses of behavior: Discrimination processes, Vol. 4 (pp. 183-206). New York: Ballinger. Holland, P. C. (1985). The nature of conditioned inhibition in serial and simultaneous feature negative discriminations. In R. R. Miller and N. E. Spear (Eds.) Information processing in animals: Conditioned inhibition (pp. 267-297). Hillsdale, NJ: Erlbaum. Holland, P. C. (1986). Transfer after serial feature positive discrimination training. Learning and Motivation,

17, 243-268.

Holland, P. C. (1989a). Acquisition and transfer of conditional discrimination performance. Journal

of Experimental

Psychology:

Animal

Behavior

Processes,

15, 154-165.

Holland, P. C. (1989b). Feature extinction enhances transfer of occasion setting. Animal Learning

& Behavior,

17, 269-279.

Holland, P. C. (1989~). Transfer of negative occasion setting and conditioned

inhibition

OCCASION

387

SETTING

across conditioned

and unconditioned stimuli. Journal of Experimental Psychology: 15, 311-328. Holland, P. C., & Coldwell, S. E. (1990). [Transfer of inhibitory stimulus control across stimuli and responses after serial and simultaneous feature negative discrimination training]. Unpublished manuscript. Holland, P. C., & Lamarre, J. (1984). Transfer of inhibition after serial and simultaneous feature negative discrimination training. Learning and Motivation, 25, 219-243. Jenkins, H. M. (1985). Conditioned inhibition of key pecking in the pigeon. In R. R. Miller and N. E. Spear (Eds.), Information processing in animals: Conditioned inhibition (pp. 327-353). Hillsdale, NJ: Erlbaum. Moore, J. W., Newman, F. L., & Glasgow, B. (1969). Intertrial cues as discriminative stimuli in human eyelid conditioning. Journal of Experimental Psychology, 79, 319326. Pavlov, I. P. (1927). Conditioned reflexes, London: Oxford University Press. Rescorla, R. A. (1969). Pavlovian conditioned inhibition. Psychological Bulletin, 72, 7794. Rescorla, R. A. (1985). Conditioned inhibition and facilitation. In R. R. Miller and N. E. Spear (Eds.) Information processing in animals: Conditioned inhibition (pp. 299-326). Hillsdale, NJ: Erlbaum. Rescorla, R. A. (1989). Simultaneous and sequential conditioned inhibition in autoshaping. Animal

Quarterly

Behavior

Journal

Processes,

of Experimental

Psychology,

41B,

275-286.

Sidman, M. (1986). Functional analysis of emergent verbal classes. In T. Thompson and M. D. Zeiler (Eds.), Analysis and integration of behavioral units. Hillsdale, NJ: Erlbaum. Skinner, B. F. (1938). The behavior of organisms. New York: Appleton-Century. Thomas, D. R., & Schmidt, E. (1989). Does conditional discrimination learning by pigeons necessarily involve hierarchical relationships? Journal of the Experimental Analysis of Behavior,

52, 249-260.

Received June 29, 1990 Revised September 24, 1990