Associative interference between cues and between outcomes presented together and presented apart: an integration

Behavioural Processes 57 (2002) 163– 185 www.elsevier.com/locate/behavproc

Associative interference between cues and between outcomes presented together and presented apart: an integration Ralph R. Miller *, Martha Escobar Department of Psychology, State Uni6ersity of New York – Binghamton, Binghamton, NY 13902 -6000, USA Accepted 16 November 2001

Abstract In recent years, ‘stimulus competition’ in the study of acquired behavior has referred exclusively to (a) associative competition between cues trained in compound (e.g. overshadowing and blocking). Rarely cited are older experiments cast in the verbal learning tradition, now complemented with data from humans and rats in Pavlovian preparations, that demonstrate (b) competition between cues separately trained with a common outcome (i.e. proactive and retroactive interference). Similarly neglected are numerous examples of (c) competition between outcomes separately trained with a common cue within the verbal learning literature (also proactive and retroactive interference) as well as within the Pavlovian literature (i.e. counterconditioning). Recent data demonstrate (d) competition between outcomes trained in compound, thereby completing the four cells of a 2 × 2 matrix (competing stimuli trained together vs. trained apart and the competing stimuli being cues or outcomes) which highlights the ubiquitous nature of associative stimulus interference/competition. Most contemporary theories of acquired behavior can account for the phenomena in one or at most two cells of this matrix. Whether a common mechanism underlies the phenomena in all four cells of the matrix is currently unclear, but until such time as data preclude a common mechanism, parsimony encourages efforts to develop a model that encompasses all four cells. Here we offer a tentative model that addresses all four cells, albeit with two processes. © 2002 Elsevier Science B.V. All rights reserved. Keywords: Associative interference; Learning theories; Stimulus competition

1. Introduction The capacity to account for cue competition is a benchmark that currently must be met by any theory of acquired behavior (i.e. learning). Cue competition has held this central position ever since Kamin (e.g. 1968) argued that acquisition of * Corresponding author. Tel.: +1-607-777-2291; fax: + 1607-777-4890. E-mail address: [email protected] (R.R. Miller).

an association between a cue and an outcome requires pairing the cue with a ‘surprising’ outcome, (i.e. the occurrence of the outcome must constitute new information about the environment). For example, if a conditioned stimulus (CS) A is paired with (immediately followed by) an unconditioned stimulus (US) prior to a compound of CSs X and A being paired with the same US (i.e. A–US followed by AX –US), X will accrue less control of behavior than if the A–US pairings had not occurred. This deficit in respond-

0376-6357/02/$ - see front matter © 2002 Elsevier Science B.V. All rights reserved. PII: S 0 3 7 6 - 6 3 5 7 ( 0 2 ) 0 0 0 1 2 - 8

164

R.R. Miller, M. Escobar / Beha6ioural Processes 57 (2002) 163–185

Fig. 1. A 2 ×2 matrix of different types of associative interference effects. In each cell, some representative examples of interference procedurally appropriate for that cell are listed. ‘X’ represents the target conditioned stimulus, ‘A’ represents the interfering cue, ‘US’ represents an unconditioned stimulus, ‘O’ represents an outcome, which might be a biologically significant US or an innocuous stimulus that is later paired with a US, and O1 and O2 (US1 and US2 if they are biologically significant) represent two distinctly different outcomes. The larger font of the overshadowing cue in cell 1 (denoted here as ‘A’) reflects the finding that overshadowing of a target cue is greatest when the overshadowing cue is of considerably higher salience than the target cue. Contemporary models of acquired behavior have focused almost exclusively on phenomena of cell 1 to the exclusion of phenomena in cells 2, 3 and 4.

ing to CS X is called ‘blocking’ (Kamin, 1968; Lashley, 1942). In Kamin’s framework, blocking occurs because the US is not surprising when it is paired with CS X due to CS A’s already providing information that the US is about to occur. Since Kamin’s influential writings, this principle has been incorporated into almost every new model of learning in one or another form (e.g. McLaren and Mackintosh, 2000; Pearce, 1987; Pearce and Hall, 1980; Rescorla and Wagner, 1972). One reason that this view has been so widely accepted is that it is convergent with the most apparent function of learning, which is to anticipate events in the environment so that an organism can prepare for and perhaps even modify them. If one cue already predicts an outcome, there is seem-

ingly little utility in learning that a second cue that accompanies the first cue also predicts the same outcome. This ‘sensible’ view of learning at the functional level nicely explains other examples of cue competition, such as overshadowing (Pavlov, 1927), the relative stimulus validity effect (Wagner et al., 1968), and the overexpectation effect (Rescorla, 1970). As a result of this acceptance of Kamin’s view, researchers in recent years have focused on phenomena consistent with this principle (i.e. competition between cues trained together [in compound]), and have tended to ignore other types of stimulus competition. However, other forms of stimulus competition have been reported in the literature on learning over much of the last century.

R.R. Miller, M. Escobar / Beha6ioural Processes 57 (2002) 163–185

Fig. 1 depicts four different types of potential stimulus competition, distinguished from one another by (1) whether the competing stimuli are cues (i.e. antecedent events, which might serve as signals for events which immediately follow them) or outcomes (subsequent events), and (2) whether the competing elements are trained in compound (simultaneous or serial) or in widely separated trials. Competition between cues presented together during training, which has received so much attention since Kamin’s papers were published in the late 1960s, constitutes what is identified here as cell 1 of the matrix in Fig. 1. The purpose of this paper is to summarize old and new data documenting the occurrence of types of stimulus interference (i.e. competition) that constitute examples of cells 2, 3, and 4. Recognition that these other types of interference occur would suggest reassessment of models of learning and/or acquired responding that address only phenomena in cell 1 and fail to speak to phenomena in cells 2, 3, and 4.

2. Cell 2—interference between outcomes trained together Cell 2 refers to competition between outcomes that are trained together (i.e. a common cue leads to a compound of two [or more] outcomes). As such, it is in some sense the mirror image of cell 1, which depicts competition between cues (i.e. antecedent events in the training trial dyad) trained together. The fundamental problem in assessing the occurrence of this sort of interference (at least with nonhuman animals) is that the most straightforward experimental designs call for presenting two unconditioned stimuli (USs) at the same time (i.e. CS“US1 + US2). If the two different USs support the same response (e.g. approach), competition is difficult to assess because there would be ambiguity concerning whether CS– US1 or CS – US2 associations underlie the observed response. And if the two USs support different responses (e.g. approach and withdrawal), diminution of the (target) conditioned response appropriate for one US due to the presence during conditioning of the other US could be due to

165

response competition, which is interesting in its own right, but is likely quite different from the associative interference in which we here are centrally interested. Associative interference is assumed to be less peripheral than response competition, and to reflect competition between associations (in this case, CS– US1 and CS – US2) either for retrieval (i.e. performance deficit) or for storage in memory (i.e. acquisition deficit or loss of information). One way to circumvent the response competition problem is to examine situations in which there are two outcomes, but neither is biologically significant (i.e. at the presumed time of interference, they do not elicit high levels of responding) during training (when they are presented in compound). If neither outcome supports responding, response competition would be an implausible source of interference between X– O1 and X–O2, where CS X is the target cue and O1 and O2 are biologically innocuous outcomes. Of course, to assess whether the presence of O2 during training of X–O1 degraded the X–O1 association in stored memory strength or in retrievability, O1 must be made biologically significant prior to testing. Hence, we performed several studies that embedded outcome competition in a sensory preconditioning procedure. In sensory preconditioning (Brogden, 1939), two innocuous stimuli with no consequence for behavior are paired. Then one of these stimuli is paired with a biologically significant stimulus (US), and responding to the other stimulus is typically observed (see the top half of Fig. 2).

2.1. Standard preparation Before we turn to data, we here describe our standard preparation which was used in all of the experiments that we describe below. Our subjects were naive, adult, water-deprived rats. The cues and innocuous outcomes were moderate intensity lights, tones, clicks, and white noise. The US was a brief, mild, footshock. Subjects were first allowed to locate a water tube in the experimental apparatus. Then they were exposed to the experimental treatments. Prior to testing, they were again acclimated to the experimental context in

166

R.R. Miller, M. Escobar / Beha6ioural Processes 57 (2002) 163–185

Fig. 2. Competition between outcomes trained together. The top of this figure represents the basic procedure and results of a sensory preconditioning procedure (after Brogden, 1939). ‘X’ represents the conditioned stimulus, ‘O’ represents an innocuous stimulus that serves as an outcome, a ‘/’ separates explicitly unpaired stimuli. ‘CR’ represents appreciable conditioned responding, ‘---’ represents an absence of conditioned responding. The bottom half of this figure depicts blocking of outcomes trained in compound (replotted from Miller and Matute, 1998). This is an instance of stimulus competition between outcomes trained in compound (cell 2 of Fig. 1). ‘X’ represents the conditioned stimulus, O1 and O2 represent two innocuous stimuli that served as outcomes, and US represents the footshock unconditioned stimulus with which O2 was paired following training of X (see text for details on the need to use innocuous stimuli as outcomes in order to observe competition). The graph presents times to complete five cumulative seconds of drinking in the presence of CS X. A higher score indicates more control of behavior.

order to restabilize baseline drinking from the water tube, which might have been disrupted by the footshock US during training. Finally, on the test day, we waited until each individual subject completed five cumulative seconds of drinking

and then immediately presented the test CS, leaving it on until the subject completed five more cumulative seconds of drinking in the presence of this CS. Thus, each subject was drinking at the moment that the test CS was presented. Our

R.R. Miller, M. Escobar / Beha6ioural Processes 57 (2002) 163–185

dependent variable was the time that it took subjects to complete five cumulative seconds of drinking in the presence of the test CS. Scores were converted to log seconds in order to better normalize the distribution of scores within groups. Longer times to complete five cumulative seconds of drinking presumably represent greater fear of the test CS due to its ability to better activate the expectation of the footshock US. All of our CSs were auditory or visual cues, which were appropriately counterbalanced in each experimental design.

2.2. E6idence of interference between outcomes trained together We first used sensory preconditioning to examine competition between outcomes in Esmoriz-Arranz et al. (1997) and followed this paper with Miller and Matute (1998), which made the same point using somewhat different control groups. The treatment and mean test scores of some of the experimental groups from the Miller and Matute study are depicted in the lower half of Fig. 2. In phase 1, we first established an association between CS X and outcome 1. Then in phase 2 we paired CS X with a simultaneous compound of outcome 1 and outcome 2 (outcomes 1 and 2 were innocuous stimuli). Next, we made outcome 2 biologically significant by pairing it with footshock. Finally, we measured fear induced by CS X. Presumably, suppression of drinking by CS X was a direct function of the strength of the X– O2 association because outcome 1 had no biological significance either during training or testing. As can be seen in Fig. 2, prior training of the CS X –O1 association decreased behavioral control by CS X (group blocking), with such control necessarily being mediated by the X– O2 association as only O2 was paired with the footshock. These studies provide clear demonstrations of blocking of outcomes presented in compound during training. Competition between outcomes such as this is beyond the domain of Kamin’s (1968) view that the absence of surprise was the basis of stimulus competition, as well as for most of the more formal models that are de-

167

scended from it (e.g. Rescorla and Wagner, 1972) Interestingly, Rescorla (1980) (pp. 90–97) observed similar results in a second-order conditioning design. The fact that he used second-order conditioning allowed response competition as a potential interpretation of his observations. However, his central interest in these experiments was second-order conditioning, not competition between outcomes. Although he noted that his data suggested competition between outcomes trained in compound, he never pursued nor publicized the finding, probably because it did not conform to the informational hypothesis that was and still is commonly used to account for stimulus competition (prior knowledge based on a CS X–O1 association does not make the occurrence of O2 after CS X unsurprising or redundant) (but see Rescorla, 1991). Other studies have been published demonstrating that subsequent stimuli paired with a common antecedent stimulus on the same training trial(s) yield the greatest competition when the subsequent stimuli have the same temporal relationship to the antecedent stimulus. The role of this variable can be observed in at least two different situations. First, the competing subsequent stimuli can be presented in serial compound (e.g. X“ O1 “ O2) as opposed to simultaneous compound (e.g. X“ O1 + O2). Second, the subsequent stimuli (O1 and O2) can be presented simultaneously during target training but on later trials the temporal relationship between them and the common antecedent stimulus (X) is shifted so that O1 and O2 come to have a different temporal relationship to X. For example, we could train X“ O1 + O2 with a Pavlovian delay procedure and then give further X“ O2 training with a trace procedure (i.e. with an interval between X and O2). The usual outcome of this procedure is that interference with the X –O1 association is reduced, as compared with a group that lacks the further X“ O2 training (Blaisdell et al., 1997; Burger et al., 2000). We return to the implications of this observation later.

168

R.R. Miller, M. Escobar / Beha6ioural Processes 57 (2002) 163–185

3. Cell 3— interference between cues trained apart Cell 3 of Fig. 1 refers to competition between cues that are trained apart, but have a common outcome. The pairings of the interfering cue with the common outcome can come before the target pairings (ordinarily called proactive interference), after the target pairings (ordinarily called retroactive interference), or interspersed with the target pairings. Although the interspersed situation has been little examined,1 there is a considerable literature concerning proactive and retroactive interference, most of it dating back to the vast verbal learning literature published during the middle of the 20th century (for a review see Slamecka and Ceraso, 1960). Typically, the basic preparation consisted of human participants being presented with a list of paired associates (words, nonsense syllables, or trigrams) of the form A“ O1 (e.g. ‘dog-chair’), then a second pair of the form B“ O2 (e.g. ‘cat-couch’). At test, participants were presented with a cue (A if the concern was retroactive interference by B, or B if the concern was proactive interference by A) and asked what outcome had been paired with it. Although people were clearly able to learn these paired associates over repeated trials, the likelihood of them recalling that A had been paired with O1 sometimes deteriorated if they had also been exposed to B“ O1 (i.e. if a common outcome served as O1 and O2). These results appear to be clear examples of cell-3 type interference. However, this procedure was used exclusively with human participants, it was usually embedded in a long list of paired associates, and it was basically a

verbal task which possibly depended upon cognitive faculties that did not engage underlying basic learning processes (e.g. Pavlovian conditioning). For these reasons, some people have wondered whether these observations should be used to evaluate modern learning theories, in the same way that overshadowing and blocking (i.e. competition between cues presented together) are used because verbal learning effects could reflect processes other than associative learning. Indeed, verbal learning studies often lacked control groups to allow assessment of underlying processes, such as the influence of degrading of the A–O1 correlation by presenting O1 in the absence of A (during the B “ O1 pairings) as opposed to learning a second association to the same outcome. Moreover, the verbal learning literature concerning this sort of interference is ambiguous. Although there were many reports of such interference (e.g. Postman, 1962), there were also a number of failures to obtain it (e.g. Jung, 1963). Unfortunately, interest in such interference effects decreased before the variables determining this ambiguity were thoroughly studied.

3.1. Interference between cues trained apart in non6erbal preparations Recently, Matute and Pinen˜ o (1998) demonstrated interference between cues trained apart, specifically retroactive interference. Their studies used human participants playing a video game in which they had to anticipate a specific outcome. Verbal instructions were used in their task, but the actual training experiences (i.e. Pavlovian contingencies between colors presented on a com-

Fig. 3. Competition between conditioned stimuli trained apart with a common outcome in first-order conditioning and in sensory preconditioning. The top panel summarizes the treatments of some groups from a first-order conditioning experiment that failed to obtain retroactive interference, whereas the bottom panel summarizes an experiment that did yield retroactive interference when the target and competing training occurred embedded within the first part of a sensory preconditioning procedure (both replotted from Escobar et al., 2001). The target conditioned stimulus (CS) is represented by ‘X,’ the interfering CS is represented by ‘A,’ and the initially innocuous stimulus outcome is represented by ‘O.’ After the target training (phase 1) and the interfering training (phase 2) the outcome was paired with a footshock unconditioned stimulus (US). The graphs present times to complete five cumulative seconds of drinking in the presence of CSs X and A. A higher score indicates more control of behavior. For details, see text and Escobar et al. (2001). 1 If anything, intermixed training should encourage stimulus generalization and facilitate learning.

R.R. Miller, M. Escobar / Beha6ioural Processes 57 (2002) 163–185

Fig. 3.

169

170

R.R. Miller, M. Escobar / Beha6ioural Processes 57 (2002) 163–185

puter monitor) and responses used to assess [retroactive] inference (i.e. rate of button pressing) were nonverbal. Our laboratory has recently performed analogous studies using rats as subjects, first replicating their basic findings and then adding studies to further illuminate the underlying processes. Because Matute and Pinen˜ o reported retroactive interference, and the verbal learning experiments often found retroactive interference easier to obtain than proactive interference, we initially focused on retroactive interference. In phase 1, rats were exposed to pairings of the target cue and outcome in which the outcome was a biologically significant unconditioned stimulus (footshock; X“US). Then, in phase 2, group competition was exposed to A“US pairings. As can be seen in the top of Fig. 3, subsequent responding to cue X was not attenuated relative to a control group that received A-alone presentations in Phase 2 (see top panel of Fig. 3). That is, no retroactive interference of X“ US training by A “ US training was observed. However, one must note that in this experiment we first endowed cue X with a response potential (in phase 1) and then in phase 2 we tried to attenuate (interfere with) that response potential. Considerable prior work in our laboratory has determined that, all other factors being equal, it is distinctly more difficult to attenuate a response potential that is already established than to prevent the initial establishment of that response potential (Denniston et al., 1996; Miller and Matute, 1998; Oberling et al., 2000). Thus, by using first-order conditioning in phase 1, we may have been using a situation in which retroactive interference is particularly difficult to obtain. In contrast to our study with a footshock outcome, in Matute and Pinen˜ o’s (1998) studies, for ethical and practical reasons having to do with the use of human subjects, the outcome was not biologically significant. To better approximate that condition in our laboratory with rats as subjects, we recast the initial experiment within the first half of a sensory preconditioning procedure. By our avoiding first-order conditioning, in phase 2 we were effectively trying to induce retroactive interference with an association that did not support responding at the time of phase 2. The

essential features of this study are depicted in the bottom panel of Fig. 3. As in the preceding experiment, in phase 1 rats were given target pairings, but now the US was replaced by an innocuous outcome (i.e. X“ O). In phase 2, different treatments were given to the various groups with the critical group (competition) receiving a potentially interfering treatment that consisted of pairing cue A with the same innocuous outcome that had been used in phase 1 (i.e. A“ O). Only after completion of the interference treatment (phase 2) was the Outcome made biologically significant through O“ US (footshock) pairings so that there would be some motivation for behavior with which to assess the current efficacy of the X“ O association. As can be seen in the lower part of Fig. 3, group competition showed substantially less responding to cue X than did any of the control groups, which were exposed to either Aalone (group Cx– A), O-alone (group Cx–O), or no event (group Cx-alone) during phase 2. That is, retroactive interference as a result of A“ O training was evident. Moreover, the lack of retroactive interference in group Cx–O relative to group Cx – alone indicates that the source of the interference in group comp was not due to degraded contingency as a result of phase 2 exposure to the outcome. (Possibly, more phase 2 exposure to O would have reduced behavioral control by X as a result of reducing the contingency between X and O.) The details of this study are described in Escobar et al. (2001).

3.2. Context dependency Toward trying to better understand the basis of interference in cell 3 of Fig. 1, we sought to identify factors that influence when one will observe proactive as opposed to retroactive interference. The two phases of interference training in some sense create ambiguity with respect to the common element, which is the outcome for cell 3. In the basic learning literature, there is strong evidence that simple associations are relatively independent of the contexts in which they were trained. That is, transfer from a training context to a different test context usually results in little loss of behavioral control. However, if there is ambiguity, such as two conflicting associations,

R.R. Miller, M. Escobar / Beha6ioural Processes 57 (2002) 163–185

171

Fig. 4. Competition between cues trained apart in different contexts. Conditioned stimuli X and A were paired with innocuous stimulus outcome O. Subsequently, O was paired with a footshock US. Subscripts indicate whether the treatment took place in context 1 or 2. The group names refer first to the phase 2 treatment and then, after the hyphen, to the context in which the two tests occurred. The graph presents times to complete five cumulative seconds of drinking in the presence of CSs X and A. A higher score indicates more control of behavior. Test X assessed retroactive interference and test A assessed proactive interference. For details, see text and Escobar et al. (2001).

then organisms appear to use the context to disambiguate the situation, that is, determine which of two conflicting associations immediately obtains (e.g. Bouton, 1993). Here, we sought to see if the inherent ambiguity created by the two phases of interference training (see phases 1 and 2 for group comp in the lower half of Fig. 3) also created a context dependency. This was done in two experiments, one in which we manipulated

the background cues as the contexts of learning and testing, and the other in which we used a punctate (i.e. brief) stimulus to define the immediate context. In the first experiment, all rats were given X“ O pairings in context 1 followed by various treatments, most notably A“ O pairings (see Fig. 4 for the specific treatments received by each group) in context 2, which was distinctly different from

172

R.R. Miller, M. Escobar / Beha6ioural Processes 57 (2002) 163–185

context 1. Then, the initially innocuous outcome O was paired with a footshock US in both contexts 1 and 2. Finally subjects in group comp-1 were tested in context 1 with CS X and CS A, whereas all other subjects were similarly tested in context 2. In group comp-1, behavioral control was strong for CS X and weak for CS A (i.e. proactive interference with the A“O association by the X “ O association), whereas in group comp-2, the results were reversed (i.e. retroactive interference with the X“O association by the A “O association). The remaining three groups exhibited strong responding to X and weak responding to A (which had not been paired with O for these groups). Thus, the specific context used for testing appears to have influenced (i.e. primed) which association was most readily retrieved when a test cue (X or A) was presented. Note that, as a function of test conditions, this experiment obtained evidence of both retroactive and proactive interference; specifically, the type of interference observed was influenced by which training context was used for testing. Importantly, the potential of the specific test context background cues to modulate which association was behaviorally evidenced indicates that at the time of testing both associations were still at some level encoded in the comp subjects; phase 2 training did not produce a loss of the association formed during phase 1. In the second experiment, we used a single context for all phases of the experiment and attempted to prime either the phase 1 or phase 2 association by presenting at the start of the testing sessions a stimulus that was intermittently presented either during phase 1 (B) or during phase 2 (C) training (see Fig. 5). During training, both B and C were presented sufficiently far away in time from the X “ O and A“ O training trials that B and C acquired little behavioral control. As can be seen in Fig. 5, group none, which had no priming stimulus presented during the test trials exhibited stronger responding to A than X, indicating retroactive interference. In contrast, priming the test trials with stimulus B enhanced behavioral control by X and attenuated it by A. Priming the test trials with stimulus C was seen to have the opposite effect, although response levels did not differ appreciably from those of group

none, consistent with the fact that, without priming, this situation yielded retroactive interference. The notable finding is that a punctate priming stimulus transformed a situation in which retroactive interference was evident (group none) into a situation in which proactive interference was evident (group B). As in the preceding experiment, the effectiveness of the priming cues to modulate which association was behaviorally evidenced demonstrates that the observed deficits in responding to X and A were retrieval failures rather than a consequence of irrevocably lost memories. (Although we emphasize priming here, surely retention intervals and amount of training are also potent variables.) These studies in conjunction with the several examples of this sort of interference within the verbal learning literature give credence to the occurrence of competition between cues trained apart with a common outcome, and begin to suggest some of the factors that influence such competition. (Note that we are assuming that there are strong parallels between paired-associate learning and interference phenomena within the Pavlovian tradition. See Donahoe and Palmer (1994) for a review of the similarities and differences between paired-associate and Pavlovian learning.)

4. Cell 4— interference between outcomes trained apart Interference between outcomes trained apart with a common cue is evident in a number of different situations. Among these are the vast number of demonstrations of A“ B, A“ C interference in the old verbal learning literature (e.g. Postman, 1962). However, as discussed before, verbal learning experiments have sometimes been discounted for possibly engaging higher-order cognitive processes unique to humans processing verbal information. Additional examples of cell 4-type interference include counterconditioning and extinction. In counterconditioning (e.g. Dearing and Dickinson, 1979; Pavlov, 1927), a CS is first paired with one outcome (US1) and then with a second outcome (US2). The more obvious effect is that the X“ US2 pairings interfere with

R.R. Miller, M. Escobar / Beha6ioural Processes 57 (2002) 163–185

173

Fig. 5. Competition between cues trained apart with different punctate cues being presented during phase 1 and phase 2. For each animal, training and testing occurred in the same context. Conditioned stimuli X and A were paired with innocuous stimulus outcome O. Interspersed with the X “ O trials of phase 1 were presentations of stimulus B and interspersed with the A “O trials of phase 2 were presentations of stimulus C. Subsequently O was paired with a footshock US. During test sessions, B or C was presented some minutes before X or A was presented. The group names refer to the priming cue presented at the beginning of each test trial except for group unpaired, the name of which reflects its phase 2 treatment. The graph presents times to complete five cumulative seconds of drinking in the presence of CSs X and A. A higher score indicates more control of behavior. Test X assessed retroactive interference and test A assessed proactive interference. For details, see text and Escobar et al. (2001).

behavior based on the X“ US1 association, and less obviously, but clearly occurring, behavior based on the X“US2 association is subject to interference by the X“ US1 experience. In extinction, a CS is first paired with an outcome (X“ US) and then the CS is presented alone (X“ no US). That the extinction treatment decreases conditioned responding to the CS can also be viewed as a form of interference. Moreover, whether this interference is retroactive or proactive can be modulated by contextual or punctate stimuli. For

example, Bouton and his associates have extensively examined the consequences of acquisition training in one context and extinction treatment in a second context. Although the extinction treatment eliminated conditioned responding to X in the extinction context, vigorous responding was observed in both the training context and in a third neutral context (e.g. Bouton and Bolles, 1979; for a punctate priming stimulus, see Brooks and Bouton, 1993). This stimulus-induced recovery from extinction is called renewal and it has

174

R.R. Miller, M. Escobar / Beha6ioural Processes 57 (2002) 163–185

now been observed in many different species and preparations.

4.1. Response competition and temporal factors As mentioned above, counterconditioning is subject to interpretation in terms of response competition, which is potentially important but is not to be confused with associative interference, the topic at issue here. Moreover, acquisition followed by extinction uses a US and the absence of the US as the two outcomes, which may introduce some factors not inherent to competition between two different outcomes, each of which consists of a stimulus presentation. Hence, we (Escobar et al., 2002) conducted some experiments to see if we could obtain competition, specifically retroactive interference, between two distinctly different outcomes trained apart with a common cue. In addition to this demonstration, these studies had two additional goals. One was to compare the magnitude of competition between outcomes trained apart with the magnitude of competition between cues trained apart; and the other was to examine if the temporal relationship between the cue and the two outcomes influenced the degree of interference. To obviate a role of response competition, as we did in our examination of cell 2, the two phases of interference training were embedded in the first part of a sensory preconditioning procedure. In the first experiment, during phase 1 all rats were exposed to A“ B pairings. In phase 2, group A-same received A“ C pairings (potentially creating competition between outcomes B and C); group A-diff received C“ A pairings which should have produced associative interference similar to group A-same only if the temporal order of A and C made no difference; and group A-unpaired received equivalent exposure to A and C unpaired, thereby serving as a control for responding based on the A “ B pairings being attenuated by nonassociative exposure to A or C during phase 2 (see the top of Fig. 6). Groups B-same, B-diff and B-unpaired were treated the same as their A counterparts, except that in phase 2 A was replaced by B; thus, these three groups assessed competition between cues A and C rather than outcomes B

and C. In phase 3, all subjects were exposed to B-footshock US pairings and finally subjects were tested for lick suppression in the presence of cue A. Inspection of the mean suppression scores of experiment 1 in Fig. 6 reveals that behavioral control by cue A was lower in group A-same than in group A-unpaired. This difference documents competition between outcomes trained apart and further that the observed interference was not due to degraded contingency as a result of the phase 2 exposures to A (i.e. A-noB trials) that were received by group A-same. Additionally, the lower score of group A-same relative to group A-diff indicates that the A–C pairings of phase 2 yielded more retroactive interference when stimulus A was maintained in the same temporal position in phase 2 that it had in phase 1 (i.e. changing A to an outcome in phase 2 reduced its potential to interfere with the phase 1 memory of it serving as a cue). Parallel comparisons of the three B groups in experiment 1 encourage the same conclusions about competition between cues. Moreover, the difference observed between groups A-same and A-unpaired seems quite similar to that seen between groups B-same and B-unpaired, indicating that, at least in this situation, competition between outcomes was comparable in magnitude to competition between cues. Similar conclusions concerning the comparable magnitude of interference between cues trained apart, provided they have a common outcome, and between outcomes trained apart, provided they have a common cue, were reached by Pinen˜ o and Matute (2000) based on their study of human participants in a behavioral (nonverbal) task. The above comparison of competition between outcomes and between cues might be faulted because B was consistently paired with the US and testing was consistently performed with A. Thus, the B groups in experiment 1 were exposed to B in all three phases of treatment, whereas the A groups were not. Additionally, the A groups were tested on a stimulus that was presented in phase 2, whereas the B groups were not. To determine if either of these confounds might have contributed to our observations, we replicated the study, but this time in Phase 3 paired A with the footshock

R.R. Miller, M. Escobar / Beha6ioural Processes 57 (2002) 163–185

175

Fig. 6. Competition between outcomes trained apart and between cues trained apart. For each subject, all training and testing occurred in the same context. Stimuli A, B and C were all innocuous during phases 1 and 2. ‘/’ separates stimuli that were presented explicitly unpaired, and US represents the footshock unconditioned stimulus. The graph presents times to complete five cumulative seconds of drinking in the presence of CS A (experiment 1) and B (experiment 2). A higher score indicates more control of behavior. For details, see text and Escobar et al. (2002).

US and then we tested on stimulus B. Assessing the A“B association by reinforcing A and then testing on B may seem like grounds for pause because it constitutes backward conditioning; however, there is a large and growing literature

that under some conditions backward pairings support excitatory responding (e.g. Arcediano et al., 2002; Matzel et al., 1988; Pavlov, 1927; Savastano and Miller, 1998). Thus, experiment 2 inverted the confounds of experiment 1. If the

176

R.R. Miller, M. Escobar / Beha6ioural Processes 57 (2002) 163–185

results of experiment 1 depended on either of these confounds, different results should have come from experiment 2. However, inspection of the group mean scores of experiment 2 in Fig. 6 indicates that these changes had no appreciable effect. Consequently, we may conclude that these potential confounds did not influence our original observations, and competition between outcomes occurs and is similar in magnitude to competition between cues.

5. Why have modern models of learning ignored cells 2, 3 and 4? Collectively, the experiments reviewed above demonstrate competition between cues presented together, cues presented apart, outcomes presented together and outcomes presented apart. Moreover, much of these data are not new, but date back the better part of 40 years. How and why did students of learning forget such a large and relevant literature? Here we can only speculate. But we would suggest that there were at least five reasons beyond the recurring human problem of failing to remember history. One reason was that the controls used in the older studies were not always up to modern standards. As we have demonstrated, these interference effects can be obtained even with appropriate control groups. A second reason is that the materials were verbal items with human subjects, and consequently might be explained with higher-order mental processes rather than basic associative processes that transcend species. However, the present studies suggest that these effects can also be obtained in Pavlovian situations with nonhuman subjects. A third reason for this neglect, applicable to at least cells 3 and 4, was a sense that different mechanisms accounted for these types of interference, relative to cell 1 (cell 2 was not well recognized). But recent observations discussed below (Section 6) challenge the notion of different mechanisms. A fourth reason that cell 3 has been neglected may be based on the view that cue competition is of greater magnitude when the competing cues are trained together than when they are trained apart. However, rarely do re-

searchers include appropriate controls to properly compare cell 1-type interference with cell 3-type interference. Moreover, researchers ordinarily report only studies with tuned parameters that maximize the type of interference in which they are immediately interested; there is no reason to think that the conditions that maximize one type of interference will also maximize the other type of interference. Because of the focus in recent decades on cell 1-type interference, we better understand the optimal parameters to obtain this type of interference in the animal laboratory than we do to obtain the other types of interference. A fifth reason that cells 3 and 4 were ignored likely had to do with the functionally compelling nature of the informational account provided by Kamin (1968) and Rescorla and Wagner (1972). That is, the central value of the ability to learn environmental relationships seemingly is to anticipate impending events so one might either prepare for or even try to alter the nature of these events. In this framework, redundant information about impending events is viewed as being of little functional value and deleterious with respect to the putative limited capacity of memory. Hence, it is appealing to hypothesize a learning mechanism that is sensitive to redundancy and precludes learning associations about events that, during training, were expected based on simultaneously presented cues. Cell 1-type stimulus competition is entirely consistent with this view, whereas the stimulus competition depicted in the other three cells of the matrix in Fig. 1 are not. As a consequence of the appeal of the functional viewpoint, cells 3 and 4 came to be ignored and were eventually forgotten by many researchers. However, it is important to note that, as its name indicates, the functional viewpoint speaks to the function of learning, not necessarily the underlying process. As biologists have repeatedly found, process does not always reflect function. Process is severely limited by a species’ evolutionary history and random factors in sexual recombination and mutation. Several phenomena (e.g. priming effects) suggest that subjects encode great amounts of, mostly redundant, information (process); cue competition effects reflect the tendency of organisms to use only part of this information (func-

R.R. Miller, M. Escobar / Beha6ioural Processes 57 (2002) 163–185

tion). In summary, learning theory appears to have allowed an implicit concern for the function of learning to overly influence the empirical examination of basic phenomena. We have erroneously assumed that function dictates process.

6. Common or different mechanisms? Traditionally, cell 1-type stimulus competition was regarded as a learning deficit. That is, subjects were assumed unable to exhibit the deficient behavior without new training trials with the target cue (i.e. further training; Rescorla and Wagner, 1972). In contrast, cell 3- and 4-type competition has been viewed largely as a performance deficit that could be reversed without further training with the target cue. For example, retroactive interference was known to wane over increasing retention intervals (e.g. Postman et al., 1968). The corresponding return of behavior based on phase 1 training without further training suggested that the original deficit (i.e. retroactive interference) was due to a failure to access an association that was still encoded within the subject, rather than an irrevocable loss of the phase 1 association. The previously described studies by Escobar et al. (2001) demonstrating priming by either a specific stimulus or a training context also speak to this sort of competition being at least in part a performance deficit. However, this distinction between the mechanisms underlying cell 1type interference and mechanisms underlying cells 3 and 4-type interference does not stand up to contemporary scrutiny. Cell 1-type interference (e.g. overshadowing and blocking) in recent years has been found to be reversible without further training. Manipulations that are often found to undo this type of stimulus competition include, (1) lengthening the retention interval (i.e. spontaneous recovery, e.g. Kraemer et al., 1988), (2) administration of socalled ‘reminder treatments’ which consist of presentation of either the outcome alone, the cue alone, or the training context (e.g. Balaz et al., 1982), and (3) posttraining massive extinction of the overshadowing or blocking stimulus (e.g. Blaisdell et al., 1999b; Kaufman and Bolles, 1981;

177

Matzel et al., 1985). Although these demonstrations of recovery from competition between cues trained together (particularly, extinction of the competing cue) were initially viewed as clear demonstrations of the original response deficit being a performance failure (e.g. Miller and Matzel, 1988, Miller and Schachtman, 1985), newer learning models have been formulated that view recovery from cell 1-type stimulus competition resulting from reminder treatments or extinction of the competing cue as new learning that occurs in the absence of the target cue (Dickinson and Burke, 1996; Van Hamme and Wasserman, 1994); however, these models are less able to account for spontaneous recovery in terms of new learning. But how one accounts for recovery from cell 1-type cue competition without further training with the target cue is a digression from the main point here, which is that the response deficits constituting cells 1–4 in Fig. 1 have now all been seen to be reversible without further training with the target stimulus (see Miller et al., 1986; Spear, 1976). Although researchers may still be arguing over theories, the basic recovery from interference phenomenon appears to be obtainable in each cell of the matrix. Of course, there are many other phenomena besides recovery from interference on which the behavioral deficits represented in each cell could and should be compared. A challenge for the future is that researchers make further comparisons between these different types of stimulus competition to see if the same phenomena can be demonstrated for each cell. If they can, this would argue for a common account of interference in all cells; if they cannot, this would encourage the development of different models for different cells of the matrix. But until such time as clear differences in the phenomena obtainable for each cell of the matrix is documented, the rule of parsimony calls for learning models to attempt to explain all four types of competition with a single mechanism.

7. A larger perspective Above we have focused on stimulus interaction to the extent that it interferes with behavioral

178

R.R. Miller, M. Escobar / Beha6ioural Processes 57 (2002) 163–185

control by a target stimulus (i.e. an inverse relationship exists between the behavioral control of the target stimulus and the interacting [competing] stimulus). However, stimulus interaction also includes the potential for facilitation of behavioral control by the interacting stimulus (i.e. a direct relationship exists between behavioral control of the target stimulus and the interacting [facilitating] stimulus). Examples of stimulus facilitation between cues trained in compound are provided by potentiation (Rusiniak et al., 1979) and augmentation (Batson and Batsell, 2000), which are [arguably] the same treatments as overshadowing and blocking, respectively, but result in enhanced rather than attenuated responding to the target cue. Second-order conditioning (Pavlov, 1927), sensory preconditioning (Brogden, 1939), and mediated extinction (Holland and Forbes, 1982) provide examples of stimulus facilitation between stimuli trained apart. We suggest that interference and facilitation together might best be called stimulus interaction, which would refer collectively to all situations in which the associative status of one stimulus, C, influences responding based upon an association between two other stimuli (A– B). Stimulus facilitation has received much less attention in recent years than has stimulus interference (including competition), and correspondingly far fewer models have been proposed to account for stimulus facilitation (but see Holland, 1990). Surely, there must be principled ways of determining whether interference or facilitation will be observed, but currently few models make clear predictions concerning the conditions under which one or the other will be observed (but see McLaren and Mackintosh, 2000). Suggested critical variables include stimulus similarity (Osgood, 1949), number of training trials (e.g. Yin et al., 1994), whether the elements in training are presented simultaneously or serially (e.g. Dwyer, 2001), and whether the training situation favors within-compound associations being formed between the interacting stimuli (e.g. Aitken et al., 2001). However, there is much research left to be done before we fully understand this issue. A detailed discussion of facilitation is beyond the domain of this article. We

mention facilitation here only because it would be misleading to imply that interference is the only type of stimulus interaction. Facilitation is an important facet of stimulus interaction that ideally some day will be explained by a model capable of accounting for both interference and facilitation including the conditions under which each is observed. To obtain any of these stimulus interaction effects, interference or facilitation, one needs appropriate parameters. These interaction effects are not as ubiquitous as many textbooks imply. For example, in our initial attempts to obtain blocking, we had to run many pilot studies until we identified parameters that consistently resulted in blocking. Research toward better identifying these parameters surely would help us understand the mechanisms underlying stimulus interaction. For example, [forward] blocking has been found to wane with large numbers of phase 2 trials (i.e. AX“ US; Azorlosa and Cicala, 1988). This rarely cited observation is damaging to models of learning that account for learning in terms of the cue added in phase 2 (the blocked CS) being redundant with the one originally trained during phase 1 (the blocking CS); that is, in an informational framework there is no basis for expecting the added cue to gain behavioral control if phase 1 training was asymptotic.

8. A tentative model First, let us summarize the empirical phenomenon that we wish to explain. Organisms learn many relationships between stimuli during their lives which seemingly do not interfere with each other. Thus, it is not surprising that associative interference, to the extent that it occurs at all, appears to be limited largely to situations in which the two competing associations have a common element, specifically a common outcome in the cases of cells 1 and 3 and a common cue in the cases of cells 2 and 4 (see Fig. 1). In cells 1 and 2 (i.e. competition between stimuli trained together), the requirement of a common element is largely met automatically by the competing

R.R. Miller, M. Escobar / Beha6ioural Processes 57 (2002) 163–185

elements being trained together.2 This is not the case with cells 3 and 4 (i.e. competition between stimuli presented apart), in which there is a greater probability of differences between instances of the potentially common element that are paired with each of the two competing stimuli. Here, the necessity of an element common to the two associations leads to the expectation that any change in the ‘common’ element between the two phases of training in cells 3 and 4 will attenuate interference. Consistent with this, stimulus interference appears to be greatest when two associations have one element in common (not two which makes phase 2 equivalent to more training trials with the same elements), with ‘common’ here including the temporal (and likely spatial) positions of the competing stimuli relative to the common element (Escobar et al., 2002; but see Keppel et al., 1971). Contemporary models (e.g. Gallistel and Gibbon, 2000; Mackintosh, 1975; McLaren and Mackintosh, 2000; Miller and Matzel, 1988; Pearce, 1987; Pearce and Hall, 1980; Rescorla and Wagner, 1972) were designed largely to account for competition between cues predicting a common occurrence of a single outcome (cell 1). Most of these models view competition between cues trained together as a failure to acquire an association between the target cue and the outcome (e.g. Rescorla and Wagner, 1972), but some of them regard such competition as a failure to retrieve the target cue–outcome association (e.g. Denniston et al., 2001). Regardless of their orientation, these models fail to account for other types of associative competition, such as situations in which there 2 Even in the case of stimuli trained together, however, the common element may have a different relationship with the competing stimuli. One such case are situations in which there are discrepancies in the temporal relationship of the common element to the two competing elements. For example, Barnet et al. (1993) and Blaisdell et al. (1997) observed that blocking and overshadowing, respectively, decreased when the temporal relationships of the two competing cues with respect to the outcome differed. Barnet et al. changed the temporal relationship of the blocking CS with respect to the outcome between phases 1 and 2 of the blocking procedure, whereas Blaisdell et al. changed the temporal relationship of the overshadowing CS to the outcome between overshadowing treatment and testing.

179

are competing outcomes or the competing stimuli are not presented in compound. Addressing cell 4 alone, Bouton (e.g. 1993) proposed a retrieval-focused model which posits that, in situations in which a cue is associated to multiple outcomes, contextual or punctate stimuli present at the time of testing permit subjects to select which one of the competing outcomes will be retrieved to immediately influence behavior. In Bouton’s framework, retrieval of a specific association is facilitated by those cues that were present at the time in which that association was trained. This model has proven efficient in accounting for many forms of cell 4 interference, but it is unable to account for competition between outcomes that are trained in compound (cell 2), much less any sort of competition between cues (cells 1 and 3). For example, competition between cues trained apart (cell 3-type interference) is problematic to this model because the competing stimuli in this case are cues, each of which should be able to retrieve representations of the common outcome without interacting with one another because each cue is part of but one association. More generally, exclusive of contingency effects (which have been demonstrated to be insufficient to account for many apparent interference effects), the existing models are all fundamentally unable to explain associative interference represented by more than one cell of the matrix in Fig. 1. The failure of contemporary models of learning to explain more than one cell of the interference matrix depicted in Fig. 1 suggests that new models require either different or additional mechanism(s) to address all four cells of the matrix. In seeking such a model, we were greatly influenced by the apparent reversibility of the interference deficits in each of the four cells of the matrix (see the preceding discussion within Section 6), which suggests that interference effects have an impact mostly at the retrieval level. We propose that interference between cues and outcomes trained together and apart could be accounted for by a dual-process ‘retrieval’ model that combines a variation of Denniston et al.’s (2001) comparator hypothesis (also see Miller and Matzel, 1988), with an extrapolation of Bouton’s (1993) retrieval model by Escobar et al. (2001, 2002) and Pinen˜ o

180

R.R. Miller, M. Escobar / Beha6ioural Processes 57 (2002) 163–185

and Matute (2000). Importantly, the mechanisms of each of these two processes apply to all four cells of the matrix but, because of the procedures defining each cell, one of the two processes is moot under the procedures in which the other is engaged: the particular associative structures make one of the two processes inefficient in each cell.

8.1. Process 1: comparator process We assume that this process, an extension of the comparator hypothesis (Denniston et al., 2001; Miller and Matzel, 1988), prevails in cell 1- and cell 2-type interference. The comparator hypothesis’ account of cell 1-type interference (i.e. ‘cue competition’) has been described many times previously. To summarize, it asserts that retrieval of the target CS– outcome association is a positive function of the strength of that association and, importantly, also a negative function of the product of (1) the strength of the association(s) between the target cue and other cues present during training (i.e. comparator [competing] stimuli) and (2) the strength of the association(s) between each of these ‘comparator’ stimuli and the outcome. Consistent with previously mentioned observations, this latter (indirectly) accessed representation of the outcome is assumed to attenuate retrieval of the target association most effectively when the two representations of the common outcome (one directly addressed by the target CS and the other indirectly addressed by the target CS through comparator stimuli) are identical, and less effectively as they are made more dissimilar either in conventional physical attributes (e.g. Blaisdell et al., 1997; but see e.g. Ganesan and Pearce, 1988) or the temporal relationship of the common outcome to the competing cues (e.g. Blaisdell et al., 1999a). Hence, the effectiveness of the target cue in retrieving a representation of the outcome is relative to that of other cues that were present during training of the target cueoutcome association. We now assert that this account can explain cell 2-type interference (i.e. competition between outcomes trained together), as well as cell 1-type

interference, if we treat the nontarget outcome as a potential comparator stimulus for the target cue. Note that the distinction between cues and outcomes is more likely in the mind of the experimenter than the subject. As Goddard (1999), among others, has demonstrated, outcomes can effectively signal impending events (i.e. other outcomes). The comparator hypothesis assumes that effective comparator stimuli are all salient stimuli, other than the target CS and the US, that are present during training of the target CS. During cell 2 treatment, the target cue is not only being paired with the target outcome, it is being paired with the nontarget outcome, which in turn is also being paired with the target outcome. Thus, at test, the cue not only has access to its direct association to the target outcome, but also the cue-nontarget outcome association and hence indirectly to the nontarget outcome-target outcome association. This latter (indirect) associative pathway presumably attenuates activation based on the [direct] target association. To the subject, the nontarget outcome of cell 2 plays a role almost identical to the nontarget cue of cell 1. However, the conventional temporal relationships are slightly different in that in cell 2 there is typically a serial relationship between the cue and the competing element (i.e. nontarget outcome), whereas in cell 1 there is ordinarily a simultaneous relationship between the target cue and the competing element (i.e. nontarget cue). Notably, for each cell, when these two temporal relationships are integrated to yield a temporal relationship for the indirect (mediated) association between the target cue and the target outcome, the target cue has the same net temporal relationship to the outcome as obtains for the direct association between the target cue and target outcome. The proposed comparator process in principle applies to situations depicted by all four cells of the matrix, but its action is minimal with respect to cells 3 and 4 because the association between the target cue and interfering cue in cell 3, and the association between the target outcome and the nontarget outcome in cell 4 should be weak or nonexistent.

R.R. Miller, M. Escobar / Beha6ioural Processes 57 (2002) 163–185

8.2. Process 2: priming process We assume that this process prevails in cell 3and cell 4-type interference. We propose that behavior indicative of interference will be observed when two associations have a common element (cue or outcome) and the nontarget (i.e. interfering) association is better primed for retrieval than is the target association. In this framework, a priming stimulus does not itself activate the representation of a stimulus stored in memory (unless it independently has excitatory potential, which is often the case); rather it increases the ease with which one physical stimulus (a cue) can activate the representation of another stimulus (an outcome), after pairings of the cue and outcome in the presence of the priming stimulus (i.e. positive priming; for a similar view, see Neely, 1977). An example of this priming function is contextual modulation of conditioned responding. Bringing a subject back to the context in which a cue– outcome association was previously learned does not result in the immediate elicitation of a conditioned response. However, if the cue is presented in that context, conditioned responding often will more likely be observed than if the cue was presented in another context. Importantly, in our view, a priming stimulus not only enhances the retrievability of representations of stimuli trained in its presence but, at the same time, it decreases the retrievability of (i.e. it negatively primes) any other stimulus representation common to either of the two stimuli constituting the positively primed association. A typical example of this sort of priming is the so-called renewal effect (Bouton and Bolles, 1979), in which testing an extinguished cue in the context of acquisition results in renewed responding (i.e. it primes the cue-outcome experience) and decreases the likelihood of retrieval of the cue-no outcome 3 These assumptions give rise to the Jabberwockian statement that, in addition to the need for a retrieval cue for the target association, retrieval is a function of the strength of the target association modulated by its prime (if present), minus the strength of any competing association modulated by its prime (if present), minus the degree to which the target prime facilitates retrieval of any competing association, plus the degree to which any competing prime facilitates retrieval of the target association.

181

experience.3 It is the addition of this priming mechanism that accounts for the stimulus modulation of interference (proactive vs. retroactive) in cells 3 and 4 of the interference matrix. This mechanism has much in common with the cognitive construct of retrieval inhibition (e.g. Bjork, 1989), which refers to decreased accessibility of certain items that are in fact stored in memory. (The concept of retrieval inhibition is not to be confused with the concept of conditioned inhibition, which refers to the presumed associative value of a stimulus that signals the omission of an expected outcome.) The role of priming cues for interference within cells 1 and 2 is minimized because the two competing associations in these cases are trained together (i.e. simultaneously) and consequently are ordinarily subject to the same priming cues. Thus, although priming cues may influence retrieval of associations, they should not tend to differentially favor one association over the other when the two associations were trained together as is the case in cells 1 and 2. As previously discussed, Bouton (1993) insightfully used the priming construct, albeit without emphasizing that name, to account for the contextual dependency of extinction (i.e. renewal), counterconditioning, and the CS-preexposure effect (i.e. cell 4). In addition to Bouton’s work with priming by contexts, Holland (1992; also see Miller and Oberling, 1998) has demonstrated in great detail how discrete cues, which he calls ‘occasion setters,’ can play a similar role as priming stimuli. However, Bouton’s and Holland’s accounts are limited to cell 4 of the Fig. 1 matrix (i.e. priming stimuli would disambiguate the value of a cue trained with different outcomes). Here we make that account more general by proposing that in any situation in which a retrieval cue (be it discrete or contextual) is presented, the degree to which the primed association will control behavior will depend on the degree to which that retrieval cue primes any other association that has a common element with the target association. Alternatively stated, a stimulus (discrete or contextual) that [positively] primes one association automatically negatively primes all other associations that share a stimulus representation (i.e. cue or outcome) with the target association.

182

R.R. Miller, M. Escobar / Beha6ioural Processes 57 (2002) 163–185

In order to explain the effects upon cells 3 and 4-type interference of switching the temporal order of stimuli between training phases (Escobar et al., 2002), as previously proposed for the comparator mechanism, we continue here to assume that a stimulus representation encodes not only the traditional physical attributes of the stimulus, but also the spatiotemporal location of the stimulus with respect to its associate (see Savastano and Miller, 1998, for the rationale of this assumption). Thus, we consider that the reduced interference observed when the position of the common element between is switched between the two phases of training is a consequence of a decrease in the similarity of the so-called common element between phases, that is, the common element is less common. In our view, positively priming one association negatively primes a second association only if the two associations share a common cue or outcome. Temporal shifting of the order of elements decreases the commonality of the common element, thereby reducing interference. Toward placing this priming mechanism in historical context, we note that McGeoch (1932) suggested that the most strongly activated association at the moment of retrieval would be expressed at the expense of other associations sharing some degree of similarity with the activated association. Moreover, he suggested that background (contextual) stimuli are responsible for the activation of one or another association (i.e. priming). Congruent with this view, Slamecka and Ceraso (1960) reviewed the literature as of 1960 and concluded that greater training on one association increased the likelihood that it would interfere with other associations and would not be subject to interference itself. We (Escobar et al., 2001) have suggested that this observation can account for failures to observe interference with stimuli that have inherent or acquired biological significance, provided one assumes that degree of training and biological significance increase the strength of the association. We can incorporate these observations with the highly plausible assumption that, orthogonal to priming and comparator processes, strength of an association increases its retrievability.

The model presented here lacks parsimony in that it involves separate mechanisms to explain interference between stimuli trained together and interference between stimuli trained apart. However, this is presumably just a first step toward an integration of the different types of interference within a common framework. At least, here we have provided an account of all four types of interference with only two mechanisms. Moreover, these two mechanisms both apply to all four types of interference; only the procedures in two of the four situations render one process ineffective and the procedures in the other two situations render the other process ineffective.

9. Conclusions Associative competition between events is not limited to antecedent events trained together. It also occurs between antecedent events trained separately, as well as between subsequent events trained together and apart. Competition between subsequent events challenges associative models that emphasize predictive value. Associative competition between events trained apart challenges associative models that account for associative competition in a framework that requires simultaneous activation of the representations of the competing stimuli. All four types of associative interference (cells 1–4 of the matrix in Fig. 1) are parameter dependent, and all are likely mirrored by associative facilitation under appropriate conditions. Stimulus competition/interference appears to be a much more general phenomenon than is commonly recognized. Whether the different forms of stimulus interference arise from a common underlying mechanism is not yet clear. But herein we propose the outline of a model that addresses all four types of interference. This model attributes interference to a combination of (1) a comparator mechanism in which the retrieval of associations is attenuated by the existence of other associations to the elements of the target association (an extension of Miller and Matzel, 1988; also see Denniston et al., 2001), and (2) a priming mechanism in which priming stimuli not only positively prime associations acquired in

R.R. Miller, M. Escobar / Beha6ioural Processes 57 (2002) 163–185

their presence, but negatively prime all other associations (an extension of Bouton, 1993 and McGeoch, 1932). Due to procedural differences between the different cells of the interference matrix, the comparator mechanism has its greatest influence on competition between stimuli (cues or outcomes) trained together (i.e. cells 1 and 2), and the priming mechanism has its greatest influence on competition between elements (cues or outcomes) that are trained apart (i.e. cells 3 and 4). Acknowledgements Support for the preparation of this manuscript was provided by NIMH grant 33881. We thank Jeffrey Amundson, Francisco Arcediano, Raymond Chang, Peter Killeen and Steven Stout for their comments on an earlier version of the manuscript.

References Aitken, M.R.F., Larkin, M.J.W., Dickinson, A., 2001. Re-examination of the role of within-compound associations in the retrospective revaluation of causal judgments. Q. J. Exp. Psychol. 54B, 27–51. Arcediano, F., Escobar, M., Miller, R.R., 2002. Temporal integration and temporal backward associations in human and nonhuman subjects, submitted for publication. Azorlosa, J.L., Cicala, G.A., 1988. Increased conditioning in rats to a blocked CS after the first compound trial. Bull. Psychon. Soc. 26, 254 –257. Balaz, M.A., Gutsin, P., Cacheiro, H., Miller, R.R., 1982. Blocking as a retrieval failure: reactivation of associations to a blocked stimulus. Q. J. Exp. Psychol. 34B, 99 –113. Barnet, R.C., Grahame, N.J., Miller, R.R., 1993. Temporal encoding as a determinant of blocking. J. Exp. Psychol. Anim. Behav. Process. 19, 327 –341. Batson, J.D., Batsell, W.R. Jr, 2000. Augmentation, not blocking, in an A + /AX+ flavor-conditioning procedure. Psychon. Bull. Rev. 7, 466 –471. Bjork, R.A., 1989. Retrieval inhibition as an adaptive mechanism in human memory. In: Roediger, H.L. III, Craik, F.I.M. (Eds.), Varieties of Memory and Consciousness: Essays in Honour of Endel Tulving. Lawrence Erlbaum, Hillsdale, NJ, pp. 309 –330. Blaisdell, A.P., Denniston, J.C., Miller, R.R., 1997. Unblocking with qualitative change of unconditioned stimulus. Learn. Motiv. 28, 268 –279. Blaisdell, A.P., Denniston, J.C., Miller, R.R., 1999a. Posttraining shifting of the overshadowing stimulus-US interval

183

alleviates the overshadowing deficit. J. Exp. Psychol. Anim. Behav. Process. 25, 18 – 27. Blaisdell, A.P., Gunther, L.M., Miller, R.R., 1999b. Recovery from blocking achieved by extinguishing the blocking CS. Anim. Learn. Behav. 27, 63 – 76. Bouton, M.E., 1993. Context, time, and memory retrieval in the interference paradigms of Pavlovian learning. Psychol. Bull. 114, 80 – 99. Bouton, M.E., Bolles, R.C., 1979. Contextual control of the extinction of conditioned fear. Learn. Motiv. 10, 445 – 466. Brogden, W.J., 1939. Sensory pre-conditioning. J. Exp. Psychol. 25, 323 – 332. Brooks, D.C., Bouton, M.E., 1993. A retrieval cue for extinction attenuates spontaneous recovery. J. Exp. Psychol. Anim. Behav. Process. 19, 77 – 89. Burger, D.C., Mallemat, H., Miller, R.R., 2000. Overshadowing of subsequent events and recovery thereafter. Q. J. Exp. Psychol. 53B, 149 – 171. Dearing, M.F., Dickinson, A., 1979. Counterconditioning of shock by a water reinforcer in rabbits. Anim. Learn. Behav. 7, 360 – 366. Denniston, J.C., Miller, R.R., Matute, H., 1996. Biological significance as a determinant of cue competition. Psychol. Sci. 7, 325 – 331. Denniston, J.C., Savastano, H.I., Miller, R.R., 2001. The extended comparator hypothesis: learning by contiguity, responding by relative strength. In: Mowrer, R.R., Klein, S.B. (Eds.), Handbook of Contemporary Learning Theories. Erlbaum, Hillsdale, NJ, pp. 65 – 117. Dickinson, A., Burke, J., 1996. Within-compound associations mediate the retrospective revaluation of causality judgments. Q. J. Exp. Psychol. 49B, 60 – 80. Donahoe, J.W., Palmer, D.C., 1994. Learning and Complex Behavior. Allyn and Bacon, Needham Heights, MA. Dwyer, D.M., 2001. Mediated conditioning and retrospective revaluation with LiCl then flavour pairings. Q. J. Exp. Psychol. 54B, 145 – 165. Escobar, M., Arcediano, F., Miller, R.R., 2002. Conditions favoring retroactive interference between antecedent events and between subsequent events. Psychon. Bull. Rev., 8, 691 – 697. Escobar, M., Matute, H., Miller, R.R., 2001. Cues trained apart compete for behavioral control in rats: convergence with the associative interference literature. J. Exp. Psychol. Gen. 130, 97 – 115. Esmoriz-Arranz, F.J., Miller, R.R., Matute, H., 1997. Blocking of subsequent and antecedent events. J. Exp. Psychol. Anim. Behav. Process. 23, 145 – 156. Gallistel, C.R., Gibbon, J., 2000. Time, rate and conditioning. Psychol. Rev. 107, 219 – 275. Ganesan, R., Pearce, J.M., 1988. Effect of changing the unconditioned stimulus on appetitive blocking. J. Exp. Psychol. Anim. Behav. Process. 14, 280 – 291. Goddard, M.J., 1999. The role of US signal value in contingency, drug conditioning, and learned helplessness. Psychon. Bull. Rev. 6, 412 – 423.

184

R.R. Miller, M. Escobar / Beha6ioural Processes 57 (2002) 163–185

Holland, P.C., 1992. Occasion setting in Pavlovian conditioning. In: Medin, D.L. (Ed.), The Psychology of Learning and Motivation, vol. 28. Academic Press, San Diego, CA, pp. 69– 125. Holland, P.C., Forbes, D.T., 1982. Representation-mediated extinction of conditioned flavor aversions. Learn. Motiv. 13, 454– 471. Holland, P.C., 1990. Forms of memory in Pavlovian conditioning. In: McGaugh, J.L., Weinberger, N.M., Lynch, G. (Eds.), Brain Organization and Memory: Cells, Systems and Circuits. Oxford University Press, New York, pp. 78 –105. Jung, J., 1963. Effects of response meaningfulness (m) on transfer of training under two different paradigms. J. Exp. Psychol. 65, 377 – 384. Kamin, L.J., 1968. Attention-like’ processes in classical conditioning. In: Jones, M.R. (Ed.), Miami Symposium on the Prediction of Behavior: Aversive Stimulation. University of Miami Press, Miami, FL, pp. 9 – 33. Kaufman, M.A., Bolles, R.C., 1981. A nonassociative aspect of overshadowing. Bull. Psychon. Soc. 18, 318 – 320. Keppel, G., Bonge, D., Strand, B.Z., Paker, J., 1971. Direct and indirect interference in the recall of paired associates. J. Exp. Psychol. 88, 414 – 422. Kraemer, P.J., Lariviere, N.A., Spear, N.E., 1988. Expression of a taste aversion conditioned with an odor –taste compound: overshadowing is relatively weak in weanlings and decreases over a retention interval in adults. Anim. Learn. Behav. 16, 164 – 168. Lashley, K.S., 1942. An examination of the continuity theory as applied to discriminative learning. J. Gen. Psychol. 26, 241 – 265. Mackintosh, N.J., 1975. A theory of attention: variations in the associability of stimuli with reinforcement. Psychol. Rev. 82, 276– 298. Matute, H., Pinen˜ o, O., 1998. Stimulus competition in the absence of compound conditioning. Anim. Learn. Behav. 26, 3–14. Matzel, L.D., Schachtman, T.R., Miller, R.R., 1985. Recovery of an overshadowed association achieved by extinction of the overshadowed stimulus. Learn. Motiv. 16, 398 –412. Matzel, L.D., Held, F.P., Miller, R.R., 1988. Information and expression of simultaneous and backward associations: implications for contiguity theory. Learn. Motiv. 19, 317 – 344. McGeoch, J.A., 1932. Forgetting and the law of disuse. Psychol. Rev. 39, 352 – 370. McLaren, I.P.L., Mackintosh, N.J., 2000. An elemental model of associative learning. I. Latent inhibition and perceptual learning. Anim. Learn. Behav. 28, 211 –246. Miller, R.R., Matute, H., 1998. Biological significance in forward and backward blocking: resolution of a discrepancy between animal conditioning and human causal judgment. J. Exp. Psychol. Gen. 125, 370 –386. Miller, R.R., Matzel, L.D., 1988. The comparator hypothesis: A response rule for the expression of associations. In: Bower, G.H. (Ed.), The Psychology of Learning and Motivation, vol. 22. Academic Press, San Diego, CA, pp. 51 –92. Miller, R.R., Oberling, P., 1998. Analogies between occasion

setting and Pavlovian conditioning. In: Schmajuk, N.A., Holland, P.C. (Eds.), Occasion Setting: Associative Learning and Cognition in Animals. American Psychological Association, Washington, DC, pp. 3 – 35. Miller, R.R., Schachtman, T.R., 1985. Conditioning context as an associative baseline: implications for response generation and the nature of conditioned inhibition. In: Miller, R.R., Spear, N.E. (Eds.), Information Processing in Animals: Conditioned Inhibition. Lawrence Erlbaum, Hillsdale, NJ, pp. 51– 88. Miller, R.R., Kasprow, W.J., Schachtman, T.R., 1986. Retrieval variability: sources and consequences. Am. J. Psychol. 99, 145 – 218. Neely, J.H., 1977. Semantic priming and retrieval from lexical memory: roles of inhibitionless spreading activation and limited-capacity attention. J. Exp. Psychol. Gen. 106, 226 – 254. Oberling, P., Bristol, A.S., Matute, H., Miller, R.R., 2000. Biological significance attenuates overshadowing, relative validity, and degraded contingency effects. Anim. Learn. Behav. 28, 172 – 186. Osgood, C.E., 1949. The similarity paradox in human learning: a resolution. Psychol. Rev. 56, 132 – 143. Pavlov, I.P., 1927. Conditioned Reflexes. Oxford University Press, London. Pearce, J.M., 1987. A model for stimulus generalization in Pavlovian conditioning. Psychol. Rev. 94, 61 – 73. Pearce, J.M., Hall, G., 1980. A model for Pavlovian learning: variations in the effectiveness of conditioned but not of unconditioned stimuli. Psychol. Rev. 87, 532 – 552. Pinen˜ o, O., Matute, H., 2000. Interference in human predictive learning when associations share a common element. Int. J. Comp. Psychol. 13, 16 – 33. Postman, L., 1962. Transfer of training as a function of experimental paradigm and degree of first-list learning. J. Verbal Learn. Verbal Behav. 1, 109 – 118. Postman, L., Stark, K., Fraser, J., 1968. Temporal changes in interference. J. Verbal Learn. Verbal Behav. 7, 672 – 694. Rescorla, R.A., 1970. Reduction in the effectiveness of reinforcement after prior excitatory conditioning. Learn. Motiv. 1, 372 – 381. Rescorla, R.A., 1980. Pavlovian Second-Order Conditioning. Lawrence Erlbaum, Hillsdale, NJ. Rescorla, R.A., 1991. Associations of multiple outcomes with an instrumental response. J. Exp. Psychol. Anim. Behav. Process. 17, 465 – 474. Rescorla, R.A., Wagner, A.R., 1972. A theory of Pavlovian conditioning: variations in the effectiveness of reinforcement and non-reinforcement. In: Black, A.H., Prokasy, W.F. (Eds.), Classical Conditioning II: Current Theory and Research. Appleton-Century Crofts, New York, pp. 64 – 99. Rusiniak, K.W., Hankins, W.G., Garcia, J., Brett, L.P., 1979. Flavor-illness aversions: potentiation of odor by taste in rats. Behav. Neural Biol. 25, 1 – 17. Savastano, H.I., Miller, R.R., 1998. Time as content in Pavlovian conditioning. Behav. Process. 44, 147 – 162.

R.R. Miller, M. Escobar / Beha6ioural Processes 57 (2002) 163–185 Slamecka, N.J., Ceraso, J., 1960. Retroactive and proactive inhibition of verbal learning. Psychol. Bull. 57, 449 – 475. Spear, N.E., 1976. Retrieval of memories: a psychobiological approach. In: Estes, W.K. (Ed.), Handbook of Learning and Cognitive Processes: Attention and Memory, vol. 4. Erlbaum, Hillsdale, NJ, pp. 17 –90. Van Hamme, L.J., Wasserman, E.A., 1994. Cue competition in causality judgments: the role of nonpresentation of com-

185

pound stimulus elements. Learn. Motiv. 25, 127 – 151. Wagner, A.R., Logan, F.A., Haberlandt, K., Price, T., 1968. Stimulus selection in animal discrimination learning. J. Exp. Psychol. 76, 171 – 180. Yin, H., Barnet, R.C., Miller, R.R., 1994. Second-order conditioning and Pavlovian conditioned inhibition: operational similarities and differences. J. Exp. Psychol. Anim. Behav. Process. 20, 419 – 428.