- Email: [email protected]
Guessing Strategies in Perceptual Identification: A Reply to McKoon and Ratcliff
CONSCIOUSNESS AND COGNITION ARTICLE NO.
5, 512–524 (1996)
0030
Guessing Strategies in Perceptual Identification: A Reply to McKoon and Ratcliff LEAH L. LIGHT1 Department of Psychology, Pitzer College, Claremont, California 91711-6110 AND
ROBERT F. KENNISON Claremont Graduate School, Claremont, California 91711-6163 Light and Kennison (this issue) found that bias effects in the forced-choice perceptual identification of words occurred only in a subset of participants, those who claimed on a strategy questionnaire to be deliberately guessing words they had studied previously. McKoon and Ratcliff (this issue) raised a number of objections to the proposal that bias effects are due to guessing strategies, citing difficulties in our statistical treatment of data, our use of subjective reports to classify participants, and our approach to the general problem of separating implicit from explicit influences on performance. This article responds to these objections. 1996 Academic Press
Prior experience can have both positive and negative effects on performance in a variety of indirect memory tasks, including perceptual identification. Ratcliff and McKoon (in press; Ratcliff, McKoon, & Verwoerd, 1989) presented evidence that in single stimulus perceptual identification tasks (which they call naming tasks), prior study increases the likelihood of correctly identifying words, while at the same time also increasing the likelihood that studied words will be incorrectly given as intrusions when words similar in orthography are flashed for identification. Ratcliff and McKoon have also developed a forced-choice version of the perceptual identification task. In this task, each test trial consists of a flashed word followed by two alternatives, one of which is the flashed word. When the two alternatives consist of the flashed word and a word orthographically similar to it, perceptual identification is better for previously studied words than for new words (a benefit) but poorer for words orthographically similar to studied words than for unstudied words (a cost). Because costs and benefits are approximately equal in magnitude, Ratcliff and McKoon have argued that these are bias effects, and they have proposed a counter model to account not only for the pattern of findings in the forced-choice task but also in the naming task and in a yes/no task which they have investigated. Detailed models of performance on indirect measures of memory are rare. Thus, the appearance of the counter model is an important event. Reply to commentary on L. L. Light and R. F. Kennison (1996). Guessing strategies, aging, and bias effects in perceptual identification. Consciousness and Cognition, 5(4), 463–499. 1 To whom reprint requests should be addressed at Department of Psychology, Pitzer College, 1050 North Mills Avenue, Claremont, CA 91711-6110. 512 1053-8100/96 $18.00 Copyright 1996 by Academic Press All rights of reproduction in any form reserved.
REPLY TO COMMENTARY
513
We have recently suggested that in the forced-choice task with similar alternatives (the only variant of the task we have investigated), some participants may be deliberately guessing studied words, thereby converting what is nominally an indirect measure of memory into a direct measure (Light & Kennison, this issue). In our Experiments 2 and 5, participants who claimed to be guessing previously seen words showed both costs and benefits, whereas participants who denied using deliberate guessing strategies showed neither. If bias effects arise from strategic guessing, there is no need to model them as implicit memory phenomena. McKoon and Ratcliff (this issue) take issue with our conclusions, as well as with several aspects of our methodology. In this article, we address concerns that they raised, separating those that we believe to be well-founded from those we believe are not. Separating Participants by Effect Size In our article, we ended by noting that a sizable proportion of our participants showed neither costs nor benefits in the forced-choice task. We suggested that this was noteworthy in the context of models that postulated universal automatic mechanisms. The assumption underlying our interest in the dispersion of costs and benefits was that automatic processes might be hard-wired and therefore subject to relatively little variation across individuals or across populations of individuals (cf. Hasher & Zacks, 1979). McKoon and Ratcliff propose that, on the contrary, the amount of variation seen in our data is about what would be expected. They assumed the magnitude of priming effects to be .10 and, using the normal approximation to the binomial, estimated the proportion of people who should show costs or benefits $0 to be .70. We should note that this estimate is based on the difference between two binomial distributions, one with p 5 .7 and one with p 5 .8 for N 5 10, the number of items per condition in Experiment 2. In Experiment 5 (combining across levels of processing), there were 20 items per condition and the expected proportion of individuals showing costs (or benefits) is .77 using this method. The observed proportions of individuals who showed benefits were .55 in Experiment 2 and .54 in Experiment 5; the observed proportions of individuals who showed costs were .53 and .48. The observed values are lower than those expected. This is not altogether suprising because our observed costs and benefits were actually less than .10, so the estimated proportions would be smaller if the actual values for costs and benefits were used in the calculations. On the other hand, the method of estimation assumes independent binomial distributions, whereas in priming experiments the measures are correlated, so that the standard error of the difference between proportions is exaggerated in McKoon and Ratcliff’s computations. McKoon and Ratcliff’s (this issue) computations are based on the assumption that we are dealing with a single distribution for all participants. For us, the real question is whether we are dealing with one distribution or with two distributions, one for guessers and one for nonguessers. Figures 1 and 2 give frequency distributions for costs and benefits in Experiments 2 and 5. Examination of these figures suggests that the assumption of a single distribution may not be warranted. (The distributions appear more markedly different for guessers and nonguessers for costs than for benefits, but this may be due to problems with ceiling effects for benefits.) We classified all participants in Experiment 2 and participants in Experiment 5
514
LIGHT AND KENNISON
FIG. 1. Frequency distributions of benefits and costs for guessers and nonguessers in Experiment 2. Note that the signs for costs have been reversed so that the positive values represent costs.
who received standard guessing instructions into those who showed benefits and those who did not (or into those who showed costs and those who did not) and asked how they divided themselves into guessers and nonguessers. In both experiments we found that costs, though not benefits, predicted responding on the strategy questionnaire. In Experiment 2, 34 people (collapsed across age) showed a cost and 30 did not; of those showing a cost, 25 said they guessed, whereas of those who did not show a cost, only 9 said they guessed, χ 2 5 12.14. In Experiment 5, 23 people given standard instructions demonstrated a cost and 25 did not; of those showing a cost, 13 said they guessed, whereas of those who did not show a cost, only 5 said they guessed, χ 2 5 6.83. In Experiment 2, 35 showed a benefit, whereas 29 did not; 21 of those showing a benefit and 13 showing no benefit said they guessed, χ2 5 1.47. In Experiment 5, among those given standard instructions, 26 showed a benefit and 22 did not; 12 of those with benefits and 6 of those with no benefits said they guessed, χ 2 5 1.81. To the extent that costs and benefits are equivalent in magnitude and arise from the same automatic processes, as postulated by the counter model, we would expect that the breakdowns would be similar for costs and benefits. They were not, but, as noted above, this may be due to the fact that ceiling effects on new items constrain the magnitude of observable benefits. When we excluded from analysis
FIG. 2. Frequency distributions of benefits and costs for guessers and nonguessers in Experiment 5. Note that the signs for costs have been reversed so that the positive values represent costs.
REPLY TO COMMENTARY
515
those individuals with high baselines ($.85), the value of the benefit χ2 for Experiment 2 increased to 2.99, p , .10, though that for Experiment 5 remained nonsignificant. Thus, there is reason to believe that we are in fact dealing with different distributions of costs and benefits for guessers and nonguessers in the forced-choice task. This is just what we claimed in Light and Kennison (this issue). Selecting Participants out of Counterbalanced Designs The utility of the analysis of guessers and nonguessers presented above depends on the legitimacy of our grouping participants based on their responses to our strategy questionnaire. Both Alan Richardson-Klavehn (in a review) and McKoon and Ratcliff (this issue) correctly noted that selecting participants out of a counterbalanced design runs the risk of subject selection artifacts. We did not report in Light and Kennison (this issue) whether this potential risk was actualized in our experiments. We do not think that subject selection artifacts compromise our conclusions. We did two sorts of analyses of the data to address this issue. In all cases we ignored age as a variable to make the designs of the analyses manageable. (a) We computed χ 2 results to see if the different lists of items involved in the counterbalancing scheme were disproportionately represented among guessers and nonguessers in any of the experiments. These were uniformly nonsignificant. (b) We did analyses of variance using list as a factor. Treating list as a factor left the conclusions from Experiments 1–3 and 5 unchanged. In Experiment 4, when nonguessers only were considered there was both a reliable benefit and a benefit by list interaction; there appear numerically to be benefits for three of the four study lists. There was no overall cost, as reported, though there was a cost by list interaction, such that one list (numerically) showed a cost and three showed a benefit of study for dissimilar items. The sample sizes here were very similar across lists, ranging only from 6 to 8, and we don’t think the interactions are too meaningful at this juncture, especially since they occurred in only one of five studies. (Nevertheless, we cannot resist noting that the cost analysis in Experiment 4, with three lists showing a benefit of study for dissimilar items, is consistent with our finding in Experiment 1 that studying words had positive effects on identification of words orthographically similar to them, a result not predicted by the counter model.) McKoon and Ratcliff (this issue) provide an important reminder that selecting participants out of counterbalanced designs can produce subject–item interactions. Such interactions are also a risk with another procedure that we used, namely, examination of costs and benefits for the fastest two-thirds of responses in Experiment 5. With this technique, a design may appear to be fully counterbalanced, but in fact there is no guarantee that all items occur equally often in each cell, and it is the slowest (hardest?) items for each participant that are discarded. Thus, conclusions based on analyses of responses selected on the basis of their speed should be interpreted cautiously. The Guessing Questionnaire The strongest evidence for our claim that costs and benefits in the forced-choice task arise from deliberate guessing is based on classifying participants into guessers and nonguessers according to their responses to our strategy questionnaire. McKoon
516
LIGHT AND KENNISON
and Ratcliff (this issue) argue that participants’ responses to our questionnaire are ‘‘uninformative.’’ They state that ‘‘a subject’s report that he or she responded by guessing cannot be taken as conclusive, or even persuasive, evidence that responses actually were determined by guessing.’’ They offered two reasons for their skepticism. First, they raised the possibility that bias effects arise from implicit processes but that people notice that their responses tend to be alternatives from the study list and attribute this (incorrectly) to guessing. This, of course, turns our argument precisely on its head. Second, they noted that people ‘‘didn’t even have to think up the guessing explanation themselves’’ because our guessing questionnaire explicitly mentions the guessing strategy. The implication here is that we may have managed to induce unwarranted inferences or illusions of memory in a subset of our participants, namely, those who show large costs and benefits on forced-choice perceptual identification tests. We will address the second point first. It is an important point because the structure of our strategy questionnaire is both typical and problematic. It is typical in that it begins with the most general question and ends with the most specific question with intermediate questions increasing in pointedness. As a reminder, the questions we asked in Experiments 1–4 were: (1) What do you think was the purpose of the word identification task that you just finished? (2) Did you use a particular strategy to identify the words? (3) Did you notice any relation between the sentences you read and the words on the identification test? (4) While doing the identification test did you notice whether some of the words you identified were from the sentences? (5) While doing the identification task did you deliberately guess words from the sentences you read when you were not sure of a word? These questions were intended to permit people to report their strategies with the least amount of prompting possible. They do, however, move from the least leading to the most leading, with the experimenter’s hypothesis possibly implied by the last question. Moreover, people who say yes to the last question will most likely have answered positively to the two preceding questions about what they noticed. This means that they may be ‘‘set up’’ to answer positively to the last question if it mentions a plausible strategy. Use of strategy questionnaires is quite common in some laboratories and, to the best of our knowledge, these questionnaires never begin by asking the most explicit questions about strategies first (see e.g., Bowers & Schacter, 1990). Nor do they ask questions about other possible strategies people might have used, e.g., about perceptual strategies such as attending to word shape. Nor do they inquire as to whether some words seemed easier to see than others and why this might be. For that matter they don’t ask about the possibility that people might deliberately not guess words they think were on the study list in order to defeat the experimenter’s purpose if they surmise it was to test memory for list words. Thus, there is a real possibility that all of us who query people about strategies may inadvertently be framing our questions in a way that suggests to people that they should answer positively to questions about guessing. We noted earlier that people whom we tested rarely referred to guessing strategies in answering the second question. From this fact we argued that the second question was more often interpreted as referring to perceptual strategies. However, in keeping with McKoon and Ratcliff ’s critique, it might just as easily be claimed that participants understood that guessing strategies were included here but denied using them. There is no way to know whether this is the case or not given our data.
REPLY TO COMMENTARY
517
We think it nonetheless crucial to note here that people are not always susceptible to experimenters’ suggestions about the causes of their behavior or their beliefs. Nesbitt and Wilson (1977) offer several examples in which researchers suggested specific hypotheses about why people performed particular actions or held particular opinions but participants vigorously denied their suggestions. In some cases, Nesbitt and Wilson reported that participants went so far as to agree that other folks might behave in the ways suggested by experimenters but nevertheless denied that they themselves did. There is also evidence from studies of context effects in surveys that people who are aware of the relationships between questions may resist extraneous influences from previous questions, if these influences are blatant enough (e.g., Strack, 1992). We turn now to McKoon and Ratcliff ’s first reason for discounting participants’ reports of guessing. In his review of an earlier version of our paper, Ratcliff suggested that, unlike the single-stimulus procedure, the forced-choice procedure provides people with two alternatives, one of which was actually studied, and therefore allows for comparison of responses that are made with words that are remembered from the study list. This, he argued, could make people especially susceptible to saying that they guessed if they responded relatively often with words from the study lists (i.e., showed larger than average costs and benefits). This would, of course, produce the association we found between responses to our questionnaire and magnitude of costs and benefits in forced-choice. As we pointed out in Light and Kennison (this issue), it would also explain why we found that more people said they were guessing in Experiment 2 (forced-choice procedure) than in Experiment 1 (single-stimulus procedure). McKoon and Ratcliff (this issue) go further and state that ‘‘for the typical subject, no other explanation would come to mind.’’ One lesson here might be that if we are going to suggest one strategy to people we ought to suggest alternatives as well, if for no other reason than to jog their memories in the (here unlikely) event that they have forgotten what they did (cf. Jobe, Keller, & Smith, 1996). We have no evidence about what sorts of attributions people are or are not likely to make if they notice that they are responding with studied words in either single-stimulus or forced-choice perceptual identification (and McKoon and Ratcliff don’t offer any evidence either). However, we can propose at least one alternative to adopting a guessing hypothesis. Witherspoon and Allan (1985) found that after studying words people judged their durations on a second presentation as longer than those of new words tested at similar durations. Thus in tasks such as we used, people might mistakenly come to believe that words from the list were presented for a longer duration than new words. Such attributions would be consistent with reports of ‘‘popouts.’’ We did not ask questions relevant to this possibility and cannot assess whether it would be more likely to be embraced by participants than an attribution about guessing strategies. We offer it merely to illustrate that there is at least one alternative attribution that might be made here. We certainly agree that subjective reports by participants that they guessed are not conclusive evidence that they did in fact engage in deliberate recollection on the forced-choice task. There is abundant evidence for the fallibility of subjective reports of mental processes (see Bargh, 1994, and Nisbett and Wilson, 1977, for reviews.) We believe, nonetheless, that reasonable people may disagree about whether such reports are ‘‘persuasive evidence’’ (see Ericsson and Simon, 1994). At a minimum,
518
LIGHT AND KENNISON
we think it not implausible to suppose that people can accurately report their strategies. One major concern is that retrospective reports about details of cognitive processing may be less accurate than think-aloud protocols obtained while participants perform a particular task (e.g., Russo, Johnson, & Stephens, 1989). Obtaining concurrent reports about guessing strategies during a perceptual identification task is not feasible inasmuch as doing so could result in a change of participants’ strategies. We did not, however, ask for detailed retrospective accounts of how people identified individual words that were flashed. We simply asked about their general strategies, so the risk of memory failure seems low here. We also provided some corroborative evidence that we believe strengthens our claims. In Experiment 5, people who reported themselves to be guessers in the standard instructions condition had better recognition memory. They also had somewhat longer response latencies in the perceptual identification task. These two findings are consistent with the notions that people who have better episodic memory are more likely to engage in guessing strategies and that guessing leads to longer latencies in perceptual identification. Similar vs. Dissimilar Alternatives in Forced-Choice Perceptual Identification One of the strengths of the counter model is that it explains why costs and benefits are obtained in the forced-choice task for similar but not for dissimilar alternatives. McKoon and Ratcliff (this issue) took us to task for not explaining why participants would decide to consciously recall studied words when the alternatives in the perceptual identification task are similar but not when they are dissimilar. They presented the results of a new experiment in which participants were instructed to guess, using instructions like those of Light and Kennison (this issue, Experiment 5). With instructions to guess, participants did indeed show costs and benefits for both similar and dissimilar response alternatives. McKoon and Ratcliff argue that ‘‘instructions to choose words from the study list removed the difference between similar and dissimilar data because the instructions induced explicit retrieval, suggesting that explicit retrieval ordinarily plays little or no part in perceptual identification.’’ We would certainly predict the outcome obtained by McKoon and Ratcliff with deliberate guessing instructions. It remains to be seen whether people who claim to be guessing when standard instructions (that do not ask people to guess items from the study list when unsure) are used would also show costs and benefits for dissimilar alternatives. McKoon and Ratcliff (this issue) repeatedly assert that neither costs nor benefits are obtained when alternatives in the forced-choice task are dissimilar. If these effects for dissimilar alternatives are truly 0 in standard instruction conditions, when computed without classifying people as guessers and nonguessers, we could not reasonably expect that guessers would show costs and benefits. Thus, it behooves us to take a look at the actual values of costs and benefits for studies using dissimilar alternatives. Ratcliff, McKoon, and Verwoerd (1989, Experiment 5) reported a benefit of .05 and a cost of 2.03 (i.e., a benefit) when comparing their predicting explicit and predicitng control conditions. In this experiment, then, there was a small (nonsignificant) benefit and no cost of prior exposure. Ratcliff and McKoon (in press) compared word identification with similar and dissimilar alternatives in two experiments. There was little evidence for costs or benefits in these experiments (see their Tables
REPLY TO COMMENTARY
519
1 and 4), though in one case (Experiment 1, .8 similar alternatives) baseline was sufficiently high that ceiling effects may preclude demonstration of large benefits. Nonetheless, it remains an empirical question as to whether, when baseline is not excessively high, guessers and nonguessers would show different patterns of results with dissimilar alternatives in a standard instructions condition. If they did not, this would constitute strong evidence against our arguments. Separating Implicit Influences on Memory from Intentional Retrieval McKoon and Ratcliff (this issue) take strenuous exception to our use of the functional independence criterion in our research. We plead guilty to this charge. We did indeed argue that if guessers were engaging in deliberate recollection in the forcedchoice task, their performance would be sensitive to variables known to affect direct measures of memory such as recall and recognition. McKoon and Ratcliff suggest that by doing so we have turned the functional independence criterion into a functional dependence criterion. According to the functional independence criterion, it can be concluded that two memory tasks involve different processes, systems, or sources of information if it can be shown that there is a variable that affects performance on one of them in one way and performance on the other in some other way (either not at all or in the opposite direction) and another variable that has the opposite pattern of effects. A functional dependence criterion would then permit the inference that the same mechanism underlies performance on two tasks if they are similarly affected by a range of variables. Difficulties in this reasoning have been pointed out elsewhere (e.g., Hintzman, 1990; Ratcliff & McKoon, 1995) as well as in McKoon and Ratcliff (this issue) and we readily acknowledge the fallibility of the logic here. It may simply not be possible to carve memory cleanly at its joints into different types because more than one memory system, process, or source of information is involved in performance of any given task. As noted by McKoon and Ratcliff (this issue), among others, techniques such as process dissociation (Jacoby, 1991) similarly fail to cleanly separate intentional from automatic retrieval. Comparing individuals in different populations, such as amnesic patients and age-matched controls or young and old, is also problematic because cognitive deficits rarely exist in isolation and impairment in a particular domain is typically not total. (See Ostergaard & Jernigan, 1993, for further discussion.) Comparing young and older adults may not provide the most sensitive test of effects of deliberate guessing on perceptual identification because normal older adults, though they usually show impaired recall and recognition, are by no means densely amnesic.2 More sensitive tests would be obtained by comparing patients with severe anterograde amnesic to normal controls. McKoon and Ratcliff (this issue) suggest that the best way to separate automatic 2 Although we have treated the aging variable as a diagnostic for deliberate recollection, age comparisons are of interest in themselves. Our results confirm prior reports that age differences in priming in perceptual identification are small or nonexistent. Bias effects in the counter model arise from a form of inhibition, the stealing of counts from counters for words similar to studied words. If the counter model is correct, our results suggest that there are no age differences in the inhibitory processes underlying perceptual identification.
520
LIGHT AND KENNISON
and intentional retrieval processes is examination of within task retrieval processing. The techniques that they suggest, however, while useful, do not unambiguously warrant conclusions that a particular task is process pure. For instance, they advocate use of response time deadlines and memory loads ‘‘to eliminate (or sharply curtail) the slower processes that are associated with the retrieval of explicit information.’’ The words in parenthesis here are telling. Examination of performance with fast responses cannot guarantee that intentional retrieval processes have been eliminated. The definition of fast or automatic responses also varies across experimental paradigms or even within paradigms. For instance, Ratcliff and McKoon (1995, Experiment 1) divided observed response times in an object decision task into fast and slow halves (above or below 900 ms). Ratcliff and McKoon (in press, Experiment 1) eliminated the slowest third of their responses. It is not obvious, absent a model of response times for implicit and explicit processes in retrieval for particular tasks, how decisions about cutting points should be made. Arbitrary choices of cutting points run the risk of capitalizing on chance factors. Excluding slow responses from consideration may also result in discarding more difficult items (either items more difficult for all or many people or items difficult for particular participants) and, when latencies vary across conditions, can lead to differential loss of items across conditions, with possible loss of power. Using response deadline procedures avoids exclusion of large numbers of responses, but choice of deadlines is still arbitrary. Without a model to specify the distributions of implicit and explicit process latencies in a particular situation there is just no way of knowing whether all or most responses in a condition reflect intentional retrieval. Using a memory load or other divided attention task may not provide a surefire way to eliminate or reduce the influence of intentional retrieval in indirect memory tasks. Ratcliff and McKoon (1995) imposed a memory load by giving participants a list of digits to remember while they performed an object decision task. They suggested that this would inhibit use of an intentional retrieval strategy and their data showed different patterns of responses for load and no-load conditions. Craik, Govoni, Naveh-Benjamin, and Anderson (1996), however, found that dividing attention during retrieval had relatively little effect on accuracy of recall or recognition (compared to dividing attention during encoding), though response times in the divided attention task were elevated when people allocated more attention to the retrieval task. Thus, retrieval efforts may take priority over divided attention tasks. Alternatively, participants may be able to do some time sharing between episodic retrieval and a concurrent divided attention task, even a very demanding one. In either case, use of a divided attention or memory load task at retrieval may not eliminate deliberate retrieval during priming tasks. Failure to find an effect of divided attention may also be uninformative unless a range of secondary tasks varying in difficulty is tried. For any given secondary task, null results cannot be taken as evidence that only automatic processes are involved because the task selected may not tax resources enough to have deleterious effects on the primary task. (Mulligan (in press) have made an analogous point about effects of divided attention during encoding on priming in indirect memory tasks.) Thus, we agree with McKoon and Ratcliff (this issue) that looking at within-task processing is a useful approach to separating implicit and explicit retrieval. Where
REPLY TO COMMENTARY
521
we differ is that McKoon and Ratcliff accord a special status to this approach, whereas we do not. Rather, we believe that, like other approaches, it has strengths and weaknesses. We believe the field is better served by using a variety of approaches that will, if we are successful, provide converging evidence for implicit and explicit processes (see also Schacter & Cooper, 1995; Schacter & Tulving, 1994; Roediger & McDermott, 1993). For these reasons, we continue to believe that a functional independence (or dependence) strategy is not unreasonable in exploratory studies. One of our goals in the research reported in Light and Kennison (this issue) was to determine whether performance of people who said they were guessing on the forced-choice task was similar to that of performance on direct measures of memory. This was a quite limited goal inasmuch as we made no strong claims about mechanisms underlying performance on direct and indirect measures of memory, i.e., whether they are based on the same or different memory systems, processes, or representations. In this context, it seems reasonable to see whether performance on a task whose characteristics are poorly known resembles performance on one that is better known (i.e., is sensitive to the same variables in the same way). Knowing whether tasks behave in the same way is important for theory development. Determining whether two tasks indeed respond similarly to different variables is therefore a common research strategy. It appears to be one used by Ratcliff and McKoon (in press) themselves in the context of developing a process model: If performance on naming and performance on forced choice are to be explained with the same processing mechanisms, then it would be expected that many variables would affect them in parallel ways. We tested one such variable, amount of forgetting. In general, a notable characteristic of implicit memories is that their effects on performance are not reduced as much by delay between study and test as the effects of explicit memories.
Thus, we believe our case would have been strengthened had costs and benefits in guessers been influenced by levels of processing at encoding or by adult age, two variables known to affect recall and recognition more than they affect performance on repetition priming. As it turned out, costs and benefits in people given standard guessing instructions did not show effects of either levels of processing or age, although instructed guessers in Experiment 5 did, a set of outcomes contrary to our predictions. As we noted in Light and Kennison (this issue), our studies did not always permit clean inferences about similarities or differences in the effects of variables across tasks. For instance, in Experiment 5, we included a recognition test as well as as a forced-choice perceptual identification test. However, the recognition test differed from the perceptual identification task in at least two ways over and above differences in retrieval requirements. The distractors were not orthographically similar to studied words and the test was single stimulus rather than forced-choice. We suggested that guessers given standard instructions might treat the forced-choice task like a recognition test and that a recognition test structured more like the perceptual identification task might not be sensitive to effects of age or levels of processing. We have recently completed an experiment in which 8 young adults (mean age 5 19.00 years, range 5 18–22) and 8 older adults (mean age 5 71.25 years, range 5 67–75) studied two
522
LIGHT AND KENNISON
lists of words under the same conditions as those in Experiment 5, rating the words in one list for pleasantness and counting the vowels in the other. All participants then received a two-alternative forced-choice recognition test on which distractors were orthographically similar to the studied words paired with them. We found that recognition accuracy was near ceiling for words rated for pleasantness (.93 for both young and older adults) but lower for words in the vowel counting condition (.74 for the young and .77 for the old), F(1, 14) 5 34.03, MSE 5 .007. Our original conjecture was clearly wrong for the levels of processing manipulation, though ceiling effects preclude conclusions about age, at least for the pleasantness condition. Even this version of our recognition condition did not fully mimic the conditions obtaining for forced-choice perceptual identification because only the alternatives were given— there was no flash of a word that was studied, a word similar to one studied, or a new word prior to presentation of the alternatives. Flashing one of these types of words at near threshold conditions prior to testing for recognition might produce illusions of memory that we would not observe with our procedure (see e.g., Whittlesea, Jacoby, & Girard, 1990). Conclusions Are bias effects in forced-choice perceptual identification dependent on guessing strategies? Light and Kennison (this issue) argued that they might be, based primarily on comparison of people who responded that they guessed on a strategy questionnaire and people who responded that they did not guess.3 McKoon and Ratcliff (this issue) objected that selecting participants out of counterbalanced designs can produce subject selection artifacts. We have argued that this is unlikely in our research. McKoon and Ratcliff also suggested that the strategy questionnaire may have induced people who showed large costs and benefits to incorrectly attribute their production of words from the study lists to guessing. We have acknowledged this possibility. We have also noted that two of our findings are consonant with our claim that people are indeed accurate in reporting that they guessed—guessers in Experiment 5 had higher recognition scores and longer perceptual identification latencies than people who claimed not to guess. We also pointed out that there is evidence from research on survey questionnaires that people may resist blatant suggestions from previous questions, a situation that may exist with our strategy questionnaire. McKoon and Ratcliff argued that the only plausible attribution that people might make in the forced-choice task is that they were guessing previously studied items; we have proposed an alternative attribution, namely that people could come to believe that words that popped out were presented for longer durations. McKoon and Ratcliff note in passing that two of our predictions did not pan out. First, we failed to observe age differences in the size of costs or benefits among 3 Light and Kennison (this issue) also suggested that the processes involved in single stimulus word identification were different from those involved in forced-choice word identification. Contrary to predictions of the counter model, they found facilitation for words that were orthographically similar to studied items when the single stimulus task was used (Experiment 1). Masson and MacLeod (1996) have also presented evidence suggesting that the bases for word identification differ in the single stimulus and forced-choice tasks.
REPLY TO COMMENTARY
523
guessers in Experiments 2 and 5. Second, we failed to observe effects of levels of processing on costs and benefits in Experiment 5. These findings are potentially damaging to our hypothesis. We have argued here that although older adults do have impaired episodic memory, they are not totally (or even severely) amnestic. For instance, in Experiment 5, corrected recognition scores were .78 for the young and .64 for the old. The new forced-choice recognition experiment reported in this article closely approximated the conditions of the forced-choice perceptual identification task in Light and Kennison (this issue, Experiment 5) by presenting orthographically similar distractors for studied words. Under these conditions, recognition was virtually identical in young and old, so age differences in perceptual identification might not be expected. A more stringent test of the hypothesis that populations with impaired episodic memory should show diminished costs and benefits would be to compare normal populations with more severely amnestic populations. Ostergaard and Jernigan (1993) have provided evidence that more impaired populations may have reduced priming, though there have as yet been no studies of amnesic patients using the forced-choice task. Light and Kennison (this issue) suggested that levels of processing effects might not be found for forced-choice recognition with orthographically similar lures. In this article, we reported a test of this hypothesis. The results did not conform to our expectations—judging pleasantness of studied words conferred a large benefit in recognition with similar lures. Thus, failure to find levels of processing effects in costs and benefits for guessers constitutes negative evidence for our claim. Where do we go from here? McKoon and Ratcliff (this issue) argue that use of a functional independence strategy to study what they refer to as implicit and explicit processes is wrongheaded in the face of evidence that it fails on both logical and empirical grounds. They recommend instead within-task manipulation of retrieval, such as deadline procedures or imposition of memory loads during testing. Our take on this is that such methods also fail to cleanly separate deliberate recollection from other, possibly more automatic, processes. Thus, we believe that use of a variety of techniques that will hopefully provide converging evidence is warranted. ACKNOWLEDGMENTS Preparation of the manuscript was supported by National Institute on Aging Grant R37 AG02452. We are grateful to Dale Berger for statistical advice, to Marisa Collett and Greg Frigo for help in data collection, and to Roger Ratcliff for his helpful responses to numerous queries about points raised in his critique of our work.
REFERENCES Bargh, J. A. (1994). The four horsemen of automaticity: Awareness, intention, efficiency, and control in social cognition. In R. S. Wyer, Jr., & T. K. Srull (Eds.), Handbook of social cognition (2nd ed., Vol. 1, pp. 1–40). Hillsdale, NJ: Erlbaum. Bowers, J. S., & Schacter, D. L. (1990). Implicit memory and test awareness. Journal of Experimental Psychology: Learning, Memory, and Cognition, 16, 404–416. Craik, F. I. M., Govoni, R., Naveh-Benjamin, M., & Anderson, N. D. (1996). The effects of divided attention on encoding and retrieval processes in human memory. Journal of Experimental Psychology: General, 125, 159–180.
524
LIGHT AND KENNISON
Ericsson, K. A., & Simon, H. A. (1993). Protocol analysis: Verbal reports as data (Rev. ed.). Cambridge, MA: MIT Press. Hasher, L., & Zacks, R. T. (1979). Automatic and effortful processes in memory. Journal of Experimental Psychology: General, 108, 356–388. Hintzman, D. L. (1990). Human learning and memory: Connections and dissociations. Annual Review of Psychology, 41, 109–139. Jacoby, L. L. (1991). A process dissociation framework: Separating automatic from intentional uses of memory. Journal of Memory and Language, 30, 513–541. Jobe, J. B., Keller, D. M., & Smith, A. F. (1996). Cognitive techniques in interviewing older people. In N. Schwarz & S. Sudman (Eds.), Answering questions: Methodology for determining cognitive and communicative processes in survey research (pp. 197–219). San Francisco: Jossey–Bass. Light, L. L., & Kennison, R. F. (1996). Guessing strategies, aging, and bias effects in perceptual identification. Consciousness and Cognition, 5(4), 463–499. Masson, M. E. J., & Macleod, C. M. (1996). Contributions of processing fluency to repetition effects in masked word identification. Canadian Journal of Experimental Psychology, 50, 9–21. McKoon, G., & Ratcliff, R. (1996). Separating implicit from explicit retrieval processes in perceptual identification. Consciousness and Cognition, 5(4), 500–511. Mulligan, N. W. (in press). Attention and implicit memory tests: The effects of varying attentional load on conceptual priming. Memory & Cognition. Nisbett, R. E., & Wilson, T. D. (1977). Telling more than we can know: Verbal reports on mental processes. Psychological Review, 84, 231–259. Ostergaard, A. L., & Jernigan, T. L. (1993). Are word priming and explicit memory mediated by different brain structures? In P. Graf & M. E. J. Masson (Eds.), Implicit memory: New directions in cognition, development, and neuropsychology (pp. 327–349). Hillsdale, NJ: Erlbaum. Ratcliff, R., & McKoon, G. (1995). Bias in the priming of object decisions. Journal of Experimental Psychology: Learning, Memory, and Cognition, 21, 754–767. Ratcliff, R., & McKoon, G. (in press). A counter model for implicit priming in perceptual word identification. Psychological Review. Ratcliff, R., McKoon, G., & Verwoerd, M. (1989). A bias interpretation of facilitation in perceptual identification. Journal of Experimental Psychology: Learning, Memory, and Cognition, 15, 378– 387. Roediger, H. L. III, & McDermott, K. B. (1993). Implicit memory in normal human subjects. In F. Boller & J. Grafman (Eds.), Handbook of neuropsychology (Vol. 8, pp. 63–131). New York: Elsevier. Russo, J. E., Johnson, E. J., & Stephens, D. L. (1989). The validity of verbal protocols. Memory & Cognition, 17, 759–769. Schacter, D. L., & Cooper, L. A. (1995). Bias in the priming of object decisions: Logic, assumptions, and data. Journal of Experimental Psychology: Learning, Memory, and Cognition, 21, 768–776. Schacter, D. L., & Tulving, E. (1994). What are the memory systems of 1994? In D. L. Schacter & E. Tulving (Eds.), Memory systems 1994 (pp. 1–38). Cambridge, MA: MIT Press. Strack, F. (1992). ‘‘Order effects’’ in survey research: Activation and information functions of preceding questions. In N. Schwarz & S. Sudman (Eds.), Context effects in social and psychological research (pp. 23–34). New York: Springer-Verlag. Whittlesea, B. W. A., Jacoby, L. L., & Girard, K. (1990). Illusions of immediate memory: Evidence of an attributional basis for feelings of familiarity and perceptual quality. Journal of Memory and Language, 29, 716–732. Witherspoon, D., & Allan, L. G. (1985). The effect of a prior presentation on temporal judgments in a perceptual identification task. Memory & Cognition, 13, 101–111.
5, 512–524 (1996)
0030
Guessing Strategies in Perceptual Identification: A Reply to McKoon and Ratcliff LEAH L. LIGHT1 Department of Psychology, Pitzer College, Claremont, California 91711-6110 AND
ROBERT F. KENNISON Claremont Graduate School, Claremont, California 91711-6163 Light and Kennison (this issue) found that bias effects in the forced-choice perceptual identification of words occurred only in a subset of participants, those who claimed on a strategy questionnaire to be deliberately guessing words they had studied previously. McKoon and Ratcliff (this issue) raised a number of objections to the proposal that bias effects are due to guessing strategies, citing difficulties in our statistical treatment of data, our use of subjective reports to classify participants, and our approach to the general problem of separating implicit from explicit influences on performance. This article responds to these objections. 1996 Academic Press
Prior experience can have both positive and negative effects on performance in a variety of indirect memory tasks, including perceptual identification. Ratcliff and McKoon (in press; Ratcliff, McKoon, & Verwoerd, 1989) presented evidence that in single stimulus perceptual identification tasks (which they call naming tasks), prior study increases the likelihood of correctly identifying words, while at the same time also increasing the likelihood that studied words will be incorrectly given as intrusions when words similar in orthography are flashed for identification. Ratcliff and McKoon have also developed a forced-choice version of the perceptual identification task. In this task, each test trial consists of a flashed word followed by two alternatives, one of which is the flashed word. When the two alternatives consist of the flashed word and a word orthographically similar to it, perceptual identification is better for previously studied words than for new words (a benefit) but poorer for words orthographically similar to studied words than for unstudied words (a cost). Because costs and benefits are approximately equal in magnitude, Ratcliff and McKoon have argued that these are bias effects, and they have proposed a counter model to account not only for the pattern of findings in the forced-choice task but also in the naming task and in a yes/no task which they have investigated. Detailed models of performance on indirect measures of memory are rare. Thus, the appearance of the counter model is an important event. Reply to commentary on L. L. Light and R. F. Kennison (1996). Guessing strategies, aging, and bias effects in perceptual identification. Consciousness and Cognition, 5(4), 463–499. 1 To whom reprint requests should be addressed at Department of Psychology, Pitzer College, 1050 North Mills Avenue, Claremont, CA 91711-6110. 512 1053-8100/96 $18.00 Copyright 1996 by Academic Press All rights of reproduction in any form reserved.
REPLY TO COMMENTARY
513
We have recently suggested that in the forced-choice task with similar alternatives (the only variant of the task we have investigated), some participants may be deliberately guessing studied words, thereby converting what is nominally an indirect measure of memory into a direct measure (Light & Kennison, this issue). In our Experiments 2 and 5, participants who claimed to be guessing previously seen words showed both costs and benefits, whereas participants who denied using deliberate guessing strategies showed neither. If bias effects arise from strategic guessing, there is no need to model them as implicit memory phenomena. McKoon and Ratcliff (this issue) take issue with our conclusions, as well as with several aspects of our methodology. In this article, we address concerns that they raised, separating those that we believe to be well-founded from those we believe are not. Separating Participants by Effect Size In our article, we ended by noting that a sizable proportion of our participants showed neither costs nor benefits in the forced-choice task. We suggested that this was noteworthy in the context of models that postulated universal automatic mechanisms. The assumption underlying our interest in the dispersion of costs and benefits was that automatic processes might be hard-wired and therefore subject to relatively little variation across individuals or across populations of individuals (cf. Hasher & Zacks, 1979). McKoon and Ratcliff propose that, on the contrary, the amount of variation seen in our data is about what would be expected. They assumed the magnitude of priming effects to be .10 and, using the normal approximation to the binomial, estimated the proportion of people who should show costs or benefits $0 to be .70. We should note that this estimate is based on the difference between two binomial distributions, one with p 5 .7 and one with p 5 .8 for N 5 10, the number of items per condition in Experiment 2. In Experiment 5 (combining across levels of processing), there were 20 items per condition and the expected proportion of individuals showing costs (or benefits) is .77 using this method. The observed proportions of individuals who showed benefits were .55 in Experiment 2 and .54 in Experiment 5; the observed proportions of individuals who showed costs were .53 and .48. The observed values are lower than those expected. This is not altogether suprising because our observed costs and benefits were actually less than .10, so the estimated proportions would be smaller if the actual values for costs and benefits were used in the calculations. On the other hand, the method of estimation assumes independent binomial distributions, whereas in priming experiments the measures are correlated, so that the standard error of the difference between proportions is exaggerated in McKoon and Ratcliff’s computations. McKoon and Ratcliff’s (this issue) computations are based on the assumption that we are dealing with a single distribution for all participants. For us, the real question is whether we are dealing with one distribution or with two distributions, one for guessers and one for nonguessers. Figures 1 and 2 give frequency distributions for costs and benefits in Experiments 2 and 5. Examination of these figures suggests that the assumption of a single distribution may not be warranted. (The distributions appear more markedly different for guessers and nonguessers for costs than for benefits, but this may be due to problems with ceiling effects for benefits.) We classified all participants in Experiment 2 and participants in Experiment 5
514
LIGHT AND KENNISON
FIG. 1. Frequency distributions of benefits and costs for guessers and nonguessers in Experiment 2. Note that the signs for costs have been reversed so that the positive values represent costs.
who received standard guessing instructions into those who showed benefits and those who did not (or into those who showed costs and those who did not) and asked how they divided themselves into guessers and nonguessers. In both experiments we found that costs, though not benefits, predicted responding on the strategy questionnaire. In Experiment 2, 34 people (collapsed across age) showed a cost and 30 did not; of those showing a cost, 25 said they guessed, whereas of those who did not show a cost, only 9 said they guessed, χ 2 5 12.14. In Experiment 5, 23 people given standard instructions demonstrated a cost and 25 did not; of those showing a cost, 13 said they guessed, whereas of those who did not show a cost, only 5 said they guessed, χ 2 5 6.83. In Experiment 2, 35 showed a benefit, whereas 29 did not; 21 of those showing a benefit and 13 showing no benefit said they guessed, χ2 5 1.47. In Experiment 5, among those given standard instructions, 26 showed a benefit and 22 did not; 12 of those with benefits and 6 of those with no benefits said they guessed, χ 2 5 1.81. To the extent that costs and benefits are equivalent in magnitude and arise from the same automatic processes, as postulated by the counter model, we would expect that the breakdowns would be similar for costs and benefits. They were not, but, as noted above, this may be due to the fact that ceiling effects on new items constrain the magnitude of observable benefits. When we excluded from analysis
FIG. 2. Frequency distributions of benefits and costs for guessers and nonguessers in Experiment 5. Note that the signs for costs have been reversed so that the positive values represent costs.
REPLY TO COMMENTARY
515
those individuals with high baselines ($.85), the value of the benefit χ2 for Experiment 2 increased to 2.99, p , .10, though that for Experiment 5 remained nonsignificant. Thus, there is reason to believe that we are in fact dealing with different distributions of costs and benefits for guessers and nonguessers in the forced-choice task. This is just what we claimed in Light and Kennison (this issue). Selecting Participants out of Counterbalanced Designs The utility of the analysis of guessers and nonguessers presented above depends on the legitimacy of our grouping participants based on their responses to our strategy questionnaire. Both Alan Richardson-Klavehn (in a review) and McKoon and Ratcliff (this issue) correctly noted that selecting participants out of a counterbalanced design runs the risk of subject selection artifacts. We did not report in Light and Kennison (this issue) whether this potential risk was actualized in our experiments. We do not think that subject selection artifacts compromise our conclusions. We did two sorts of analyses of the data to address this issue. In all cases we ignored age as a variable to make the designs of the analyses manageable. (a) We computed χ 2 results to see if the different lists of items involved in the counterbalancing scheme were disproportionately represented among guessers and nonguessers in any of the experiments. These were uniformly nonsignificant. (b) We did analyses of variance using list as a factor. Treating list as a factor left the conclusions from Experiments 1–3 and 5 unchanged. In Experiment 4, when nonguessers only were considered there was both a reliable benefit and a benefit by list interaction; there appear numerically to be benefits for three of the four study lists. There was no overall cost, as reported, though there was a cost by list interaction, such that one list (numerically) showed a cost and three showed a benefit of study for dissimilar items. The sample sizes here were very similar across lists, ranging only from 6 to 8, and we don’t think the interactions are too meaningful at this juncture, especially since they occurred in only one of five studies. (Nevertheless, we cannot resist noting that the cost analysis in Experiment 4, with three lists showing a benefit of study for dissimilar items, is consistent with our finding in Experiment 1 that studying words had positive effects on identification of words orthographically similar to them, a result not predicted by the counter model.) McKoon and Ratcliff (this issue) provide an important reminder that selecting participants out of counterbalanced designs can produce subject–item interactions. Such interactions are also a risk with another procedure that we used, namely, examination of costs and benefits for the fastest two-thirds of responses in Experiment 5. With this technique, a design may appear to be fully counterbalanced, but in fact there is no guarantee that all items occur equally often in each cell, and it is the slowest (hardest?) items for each participant that are discarded. Thus, conclusions based on analyses of responses selected on the basis of their speed should be interpreted cautiously. The Guessing Questionnaire The strongest evidence for our claim that costs and benefits in the forced-choice task arise from deliberate guessing is based on classifying participants into guessers and nonguessers according to their responses to our strategy questionnaire. McKoon
516
LIGHT AND KENNISON
and Ratcliff (this issue) argue that participants’ responses to our questionnaire are ‘‘uninformative.’’ They state that ‘‘a subject’s report that he or she responded by guessing cannot be taken as conclusive, or even persuasive, evidence that responses actually were determined by guessing.’’ They offered two reasons for their skepticism. First, they raised the possibility that bias effects arise from implicit processes but that people notice that their responses tend to be alternatives from the study list and attribute this (incorrectly) to guessing. This, of course, turns our argument precisely on its head. Second, they noted that people ‘‘didn’t even have to think up the guessing explanation themselves’’ because our guessing questionnaire explicitly mentions the guessing strategy. The implication here is that we may have managed to induce unwarranted inferences or illusions of memory in a subset of our participants, namely, those who show large costs and benefits on forced-choice perceptual identification tests. We will address the second point first. It is an important point because the structure of our strategy questionnaire is both typical and problematic. It is typical in that it begins with the most general question and ends with the most specific question with intermediate questions increasing in pointedness. As a reminder, the questions we asked in Experiments 1–4 were: (1) What do you think was the purpose of the word identification task that you just finished? (2) Did you use a particular strategy to identify the words? (3) Did you notice any relation between the sentences you read and the words on the identification test? (4) While doing the identification test did you notice whether some of the words you identified were from the sentences? (5) While doing the identification task did you deliberately guess words from the sentences you read when you were not sure of a word? These questions were intended to permit people to report their strategies with the least amount of prompting possible. They do, however, move from the least leading to the most leading, with the experimenter’s hypothesis possibly implied by the last question. Moreover, people who say yes to the last question will most likely have answered positively to the two preceding questions about what they noticed. This means that they may be ‘‘set up’’ to answer positively to the last question if it mentions a plausible strategy. Use of strategy questionnaires is quite common in some laboratories and, to the best of our knowledge, these questionnaires never begin by asking the most explicit questions about strategies first (see e.g., Bowers & Schacter, 1990). Nor do they ask questions about other possible strategies people might have used, e.g., about perceptual strategies such as attending to word shape. Nor do they inquire as to whether some words seemed easier to see than others and why this might be. For that matter they don’t ask about the possibility that people might deliberately not guess words they think were on the study list in order to defeat the experimenter’s purpose if they surmise it was to test memory for list words. Thus, there is a real possibility that all of us who query people about strategies may inadvertently be framing our questions in a way that suggests to people that they should answer positively to questions about guessing. We noted earlier that people whom we tested rarely referred to guessing strategies in answering the second question. From this fact we argued that the second question was more often interpreted as referring to perceptual strategies. However, in keeping with McKoon and Ratcliff ’s critique, it might just as easily be claimed that participants understood that guessing strategies were included here but denied using them. There is no way to know whether this is the case or not given our data.
REPLY TO COMMENTARY
517
We think it nonetheless crucial to note here that people are not always susceptible to experimenters’ suggestions about the causes of their behavior or their beliefs. Nesbitt and Wilson (1977) offer several examples in which researchers suggested specific hypotheses about why people performed particular actions or held particular opinions but participants vigorously denied their suggestions. In some cases, Nesbitt and Wilson reported that participants went so far as to agree that other folks might behave in the ways suggested by experimenters but nevertheless denied that they themselves did. There is also evidence from studies of context effects in surveys that people who are aware of the relationships between questions may resist extraneous influences from previous questions, if these influences are blatant enough (e.g., Strack, 1992). We turn now to McKoon and Ratcliff ’s first reason for discounting participants’ reports of guessing. In his review of an earlier version of our paper, Ratcliff suggested that, unlike the single-stimulus procedure, the forced-choice procedure provides people with two alternatives, one of which was actually studied, and therefore allows for comparison of responses that are made with words that are remembered from the study list. This, he argued, could make people especially susceptible to saying that they guessed if they responded relatively often with words from the study lists (i.e., showed larger than average costs and benefits). This would, of course, produce the association we found between responses to our questionnaire and magnitude of costs and benefits in forced-choice. As we pointed out in Light and Kennison (this issue), it would also explain why we found that more people said they were guessing in Experiment 2 (forced-choice procedure) than in Experiment 1 (single-stimulus procedure). McKoon and Ratcliff (this issue) go further and state that ‘‘for the typical subject, no other explanation would come to mind.’’ One lesson here might be that if we are going to suggest one strategy to people we ought to suggest alternatives as well, if for no other reason than to jog their memories in the (here unlikely) event that they have forgotten what they did (cf. Jobe, Keller, & Smith, 1996). We have no evidence about what sorts of attributions people are or are not likely to make if they notice that they are responding with studied words in either single-stimulus or forced-choice perceptual identification (and McKoon and Ratcliff don’t offer any evidence either). However, we can propose at least one alternative to adopting a guessing hypothesis. Witherspoon and Allan (1985) found that after studying words people judged their durations on a second presentation as longer than those of new words tested at similar durations. Thus in tasks such as we used, people might mistakenly come to believe that words from the list were presented for a longer duration than new words. Such attributions would be consistent with reports of ‘‘popouts.’’ We did not ask questions relevant to this possibility and cannot assess whether it would be more likely to be embraced by participants than an attribution about guessing strategies. We offer it merely to illustrate that there is at least one alternative attribution that might be made here. We certainly agree that subjective reports by participants that they guessed are not conclusive evidence that they did in fact engage in deliberate recollection on the forced-choice task. There is abundant evidence for the fallibility of subjective reports of mental processes (see Bargh, 1994, and Nisbett and Wilson, 1977, for reviews.) We believe, nonetheless, that reasonable people may disagree about whether such reports are ‘‘persuasive evidence’’ (see Ericsson and Simon, 1994). At a minimum,
518
LIGHT AND KENNISON
we think it not implausible to suppose that people can accurately report their strategies. One major concern is that retrospective reports about details of cognitive processing may be less accurate than think-aloud protocols obtained while participants perform a particular task (e.g., Russo, Johnson, & Stephens, 1989). Obtaining concurrent reports about guessing strategies during a perceptual identification task is not feasible inasmuch as doing so could result in a change of participants’ strategies. We did not, however, ask for detailed retrospective accounts of how people identified individual words that were flashed. We simply asked about their general strategies, so the risk of memory failure seems low here. We also provided some corroborative evidence that we believe strengthens our claims. In Experiment 5, people who reported themselves to be guessers in the standard instructions condition had better recognition memory. They also had somewhat longer response latencies in the perceptual identification task. These two findings are consistent with the notions that people who have better episodic memory are more likely to engage in guessing strategies and that guessing leads to longer latencies in perceptual identification. Similar vs. Dissimilar Alternatives in Forced-Choice Perceptual Identification One of the strengths of the counter model is that it explains why costs and benefits are obtained in the forced-choice task for similar but not for dissimilar alternatives. McKoon and Ratcliff (this issue) took us to task for not explaining why participants would decide to consciously recall studied words when the alternatives in the perceptual identification task are similar but not when they are dissimilar. They presented the results of a new experiment in which participants were instructed to guess, using instructions like those of Light and Kennison (this issue, Experiment 5). With instructions to guess, participants did indeed show costs and benefits for both similar and dissimilar response alternatives. McKoon and Ratcliff argue that ‘‘instructions to choose words from the study list removed the difference between similar and dissimilar data because the instructions induced explicit retrieval, suggesting that explicit retrieval ordinarily plays little or no part in perceptual identification.’’ We would certainly predict the outcome obtained by McKoon and Ratcliff with deliberate guessing instructions. It remains to be seen whether people who claim to be guessing when standard instructions (that do not ask people to guess items from the study list when unsure) are used would also show costs and benefits for dissimilar alternatives. McKoon and Ratcliff (this issue) repeatedly assert that neither costs nor benefits are obtained when alternatives in the forced-choice task are dissimilar. If these effects for dissimilar alternatives are truly 0 in standard instruction conditions, when computed without classifying people as guessers and nonguessers, we could not reasonably expect that guessers would show costs and benefits. Thus, it behooves us to take a look at the actual values of costs and benefits for studies using dissimilar alternatives. Ratcliff, McKoon, and Verwoerd (1989, Experiment 5) reported a benefit of .05 and a cost of 2.03 (i.e., a benefit) when comparing their predicting explicit and predicitng control conditions. In this experiment, then, there was a small (nonsignificant) benefit and no cost of prior exposure. Ratcliff and McKoon (in press) compared word identification with similar and dissimilar alternatives in two experiments. There was little evidence for costs or benefits in these experiments (see their Tables
REPLY TO COMMENTARY
519
1 and 4), though in one case (Experiment 1, .8 similar alternatives) baseline was sufficiently high that ceiling effects may preclude demonstration of large benefits. Nonetheless, it remains an empirical question as to whether, when baseline is not excessively high, guessers and nonguessers would show different patterns of results with dissimilar alternatives in a standard instructions condition. If they did not, this would constitute strong evidence against our arguments. Separating Implicit Influences on Memory from Intentional Retrieval McKoon and Ratcliff (this issue) take strenuous exception to our use of the functional independence criterion in our research. We plead guilty to this charge. We did indeed argue that if guessers were engaging in deliberate recollection in the forcedchoice task, their performance would be sensitive to variables known to affect direct measures of memory such as recall and recognition. McKoon and Ratcliff suggest that by doing so we have turned the functional independence criterion into a functional dependence criterion. According to the functional independence criterion, it can be concluded that two memory tasks involve different processes, systems, or sources of information if it can be shown that there is a variable that affects performance on one of them in one way and performance on the other in some other way (either not at all or in the opposite direction) and another variable that has the opposite pattern of effects. A functional dependence criterion would then permit the inference that the same mechanism underlies performance on two tasks if they are similarly affected by a range of variables. Difficulties in this reasoning have been pointed out elsewhere (e.g., Hintzman, 1990; Ratcliff & McKoon, 1995) as well as in McKoon and Ratcliff (this issue) and we readily acknowledge the fallibility of the logic here. It may simply not be possible to carve memory cleanly at its joints into different types because more than one memory system, process, or source of information is involved in performance of any given task. As noted by McKoon and Ratcliff (this issue), among others, techniques such as process dissociation (Jacoby, 1991) similarly fail to cleanly separate intentional from automatic retrieval. Comparing individuals in different populations, such as amnesic patients and age-matched controls or young and old, is also problematic because cognitive deficits rarely exist in isolation and impairment in a particular domain is typically not total. (See Ostergaard & Jernigan, 1993, for further discussion.) Comparing young and older adults may not provide the most sensitive test of effects of deliberate guessing on perceptual identification because normal older adults, though they usually show impaired recall and recognition, are by no means densely amnesic.2 More sensitive tests would be obtained by comparing patients with severe anterograde amnesic to normal controls. McKoon and Ratcliff (this issue) suggest that the best way to separate automatic 2 Although we have treated the aging variable as a diagnostic for deliberate recollection, age comparisons are of interest in themselves. Our results confirm prior reports that age differences in priming in perceptual identification are small or nonexistent. Bias effects in the counter model arise from a form of inhibition, the stealing of counts from counters for words similar to studied words. If the counter model is correct, our results suggest that there are no age differences in the inhibitory processes underlying perceptual identification.
520
LIGHT AND KENNISON
and intentional retrieval processes is examination of within task retrieval processing. The techniques that they suggest, however, while useful, do not unambiguously warrant conclusions that a particular task is process pure. For instance, they advocate use of response time deadlines and memory loads ‘‘to eliminate (or sharply curtail) the slower processes that are associated with the retrieval of explicit information.’’ The words in parenthesis here are telling. Examination of performance with fast responses cannot guarantee that intentional retrieval processes have been eliminated. The definition of fast or automatic responses also varies across experimental paradigms or even within paradigms. For instance, Ratcliff and McKoon (1995, Experiment 1) divided observed response times in an object decision task into fast and slow halves (above or below 900 ms). Ratcliff and McKoon (in press, Experiment 1) eliminated the slowest third of their responses. It is not obvious, absent a model of response times for implicit and explicit processes in retrieval for particular tasks, how decisions about cutting points should be made. Arbitrary choices of cutting points run the risk of capitalizing on chance factors. Excluding slow responses from consideration may also result in discarding more difficult items (either items more difficult for all or many people or items difficult for particular participants) and, when latencies vary across conditions, can lead to differential loss of items across conditions, with possible loss of power. Using response deadline procedures avoids exclusion of large numbers of responses, but choice of deadlines is still arbitrary. Without a model to specify the distributions of implicit and explicit process latencies in a particular situation there is just no way of knowing whether all or most responses in a condition reflect intentional retrieval. Using a memory load or other divided attention task may not provide a surefire way to eliminate or reduce the influence of intentional retrieval in indirect memory tasks. Ratcliff and McKoon (1995) imposed a memory load by giving participants a list of digits to remember while they performed an object decision task. They suggested that this would inhibit use of an intentional retrieval strategy and their data showed different patterns of responses for load and no-load conditions. Craik, Govoni, Naveh-Benjamin, and Anderson (1996), however, found that dividing attention during retrieval had relatively little effect on accuracy of recall or recognition (compared to dividing attention during encoding), though response times in the divided attention task were elevated when people allocated more attention to the retrieval task. Thus, retrieval efforts may take priority over divided attention tasks. Alternatively, participants may be able to do some time sharing between episodic retrieval and a concurrent divided attention task, even a very demanding one. In either case, use of a divided attention or memory load task at retrieval may not eliminate deliberate retrieval during priming tasks. Failure to find an effect of divided attention may also be uninformative unless a range of secondary tasks varying in difficulty is tried. For any given secondary task, null results cannot be taken as evidence that only automatic processes are involved because the task selected may not tax resources enough to have deleterious effects on the primary task. (Mulligan (in press) have made an analogous point about effects of divided attention during encoding on priming in indirect memory tasks.) Thus, we agree with McKoon and Ratcliff (this issue) that looking at within-task processing is a useful approach to separating implicit and explicit retrieval. Where
REPLY TO COMMENTARY
521
we differ is that McKoon and Ratcliff accord a special status to this approach, whereas we do not. Rather, we believe that, like other approaches, it has strengths and weaknesses. We believe the field is better served by using a variety of approaches that will, if we are successful, provide converging evidence for implicit and explicit processes (see also Schacter & Cooper, 1995; Schacter & Tulving, 1994; Roediger & McDermott, 1993). For these reasons, we continue to believe that a functional independence (or dependence) strategy is not unreasonable in exploratory studies. One of our goals in the research reported in Light and Kennison (this issue) was to determine whether performance of people who said they were guessing on the forced-choice task was similar to that of performance on direct measures of memory. This was a quite limited goal inasmuch as we made no strong claims about mechanisms underlying performance on direct and indirect measures of memory, i.e., whether they are based on the same or different memory systems, processes, or representations. In this context, it seems reasonable to see whether performance on a task whose characteristics are poorly known resembles performance on one that is better known (i.e., is sensitive to the same variables in the same way). Knowing whether tasks behave in the same way is important for theory development. Determining whether two tasks indeed respond similarly to different variables is therefore a common research strategy. It appears to be one used by Ratcliff and McKoon (in press) themselves in the context of developing a process model: If performance on naming and performance on forced choice are to be explained with the same processing mechanisms, then it would be expected that many variables would affect them in parallel ways. We tested one such variable, amount of forgetting. In general, a notable characteristic of implicit memories is that their effects on performance are not reduced as much by delay between study and test as the effects of explicit memories.
Thus, we believe our case would have been strengthened had costs and benefits in guessers been influenced by levels of processing at encoding or by adult age, two variables known to affect recall and recognition more than they affect performance on repetition priming. As it turned out, costs and benefits in people given standard guessing instructions did not show effects of either levels of processing or age, although instructed guessers in Experiment 5 did, a set of outcomes contrary to our predictions. As we noted in Light and Kennison (this issue), our studies did not always permit clean inferences about similarities or differences in the effects of variables across tasks. For instance, in Experiment 5, we included a recognition test as well as as a forced-choice perceptual identification test. However, the recognition test differed from the perceptual identification task in at least two ways over and above differences in retrieval requirements. The distractors were not orthographically similar to studied words and the test was single stimulus rather than forced-choice. We suggested that guessers given standard instructions might treat the forced-choice task like a recognition test and that a recognition test structured more like the perceptual identification task might not be sensitive to effects of age or levels of processing. We have recently completed an experiment in which 8 young adults (mean age 5 19.00 years, range 5 18–22) and 8 older adults (mean age 5 71.25 years, range 5 67–75) studied two
522
LIGHT AND KENNISON
lists of words under the same conditions as those in Experiment 5, rating the words in one list for pleasantness and counting the vowels in the other. All participants then received a two-alternative forced-choice recognition test on which distractors were orthographically similar to the studied words paired with them. We found that recognition accuracy was near ceiling for words rated for pleasantness (.93 for both young and older adults) but lower for words in the vowel counting condition (.74 for the young and .77 for the old), F(1, 14) 5 34.03, MSE 5 .007. Our original conjecture was clearly wrong for the levels of processing manipulation, though ceiling effects preclude conclusions about age, at least for the pleasantness condition. Even this version of our recognition condition did not fully mimic the conditions obtaining for forced-choice perceptual identification because only the alternatives were given— there was no flash of a word that was studied, a word similar to one studied, or a new word prior to presentation of the alternatives. Flashing one of these types of words at near threshold conditions prior to testing for recognition might produce illusions of memory that we would not observe with our procedure (see e.g., Whittlesea, Jacoby, & Girard, 1990). Conclusions Are bias effects in forced-choice perceptual identification dependent on guessing strategies? Light and Kennison (this issue) argued that they might be, based primarily on comparison of people who responded that they guessed on a strategy questionnaire and people who responded that they did not guess.3 McKoon and Ratcliff (this issue) objected that selecting participants out of counterbalanced designs can produce subject selection artifacts. We have argued that this is unlikely in our research. McKoon and Ratcliff also suggested that the strategy questionnaire may have induced people who showed large costs and benefits to incorrectly attribute their production of words from the study lists to guessing. We have acknowledged this possibility. We have also noted that two of our findings are consonant with our claim that people are indeed accurate in reporting that they guessed—guessers in Experiment 5 had higher recognition scores and longer perceptual identification latencies than people who claimed not to guess. We also pointed out that there is evidence from research on survey questionnaires that people may resist blatant suggestions from previous questions, a situation that may exist with our strategy questionnaire. McKoon and Ratcliff argued that the only plausible attribution that people might make in the forced-choice task is that they were guessing previously studied items; we have proposed an alternative attribution, namely that people could come to believe that words that popped out were presented for longer durations. McKoon and Ratcliff note in passing that two of our predictions did not pan out. First, we failed to observe age differences in the size of costs or benefits among 3 Light and Kennison (this issue) also suggested that the processes involved in single stimulus word identification were different from those involved in forced-choice word identification. Contrary to predictions of the counter model, they found facilitation for words that were orthographically similar to studied items when the single stimulus task was used (Experiment 1). Masson and MacLeod (1996) have also presented evidence suggesting that the bases for word identification differ in the single stimulus and forced-choice tasks.
REPLY TO COMMENTARY
523
guessers in Experiments 2 and 5. Second, we failed to observe effects of levels of processing on costs and benefits in Experiment 5. These findings are potentially damaging to our hypothesis. We have argued here that although older adults do have impaired episodic memory, they are not totally (or even severely) amnestic. For instance, in Experiment 5, corrected recognition scores were .78 for the young and .64 for the old. The new forced-choice recognition experiment reported in this article closely approximated the conditions of the forced-choice perceptual identification task in Light and Kennison (this issue, Experiment 5) by presenting orthographically similar distractors for studied words. Under these conditions, recognition was virtually identical in young and old, so age differences in perceptual identification might not be expected. A more stringent test of the hypothesis that populations with impaired episodic memory should show diminished costs and benefits would be to compare normal populations with more severely amnestic populations. Ostergaard and Jernigan (1993) have provided evidence that more impaired populations may have reduced priming, though there have as yet been no studies of amnesic patients using the forced-choice task. Light and Kennison (this issue) suggested that levels of processing effects might not be found for forced-choice recognition with orthographically similar lures. In this article, we reported a test of this hypothesis. The results did not conform to our expectations—judging pleasantness of studied words conferred a large benefit in recognition with similar lures. Thus, failure to find levels of processing effects in costs and benefits for guessers constitutes negative evidence for our claim. Where do we go from here? McKoon and Ratcliff (this issue) argue that use of a functional independence strategy to study what they refer to as implicit and explicit processes is wrongheaded in the face of evidence that it fails on both logical and empirical grounds. They recommend instead within-task manipulation of retrieval, such as deadline procedures or imposition of memory loads during testing. Our take on this is that such methods also fail to cleanly separate deliberate recollection from other, possibly more automatic, processes. Thus, we believe that use of a variety of techniques that will hopefully provide converging evidence is warranted. ACKNOWLEDGMENTS Preparation of the manuscript was supported by National Institute on Aging Grant R37 AG02452. We are grateful to Dale Berger for statistical advice, to Marisa Collett and Greg Frigo for help in data collection, and to Roger Ratcliff for his helpful responses to numerous queries about points raised in his critique of our work.
REFERENCES Bargh, J. A. (1994). The four horsemen of automaticity: Awareness, intention, efficiency, and control in social cognition. In R. S. Wyer, Jr., & T. K. Srull (Eds.), Handbook of social cognition (2nd ed., Vol. 1, pp. 1–40). Hillsdale, NJ: Erlbaum. Bowers, J. S., & Schacter, D. L. (1990). Implicit memory and test awareness. Journal of Experimental Psychology: Learning, Memory, and Cognition, 16, 404–416. Craik, F. I. M., Govoni, R., Naveh-Benjamin, M., & Anderson, N. D. (1996). The effects of divided attention on encoding and retrieval processes in human memory. Journal of Experimental Psychology: General, 125, 159–180.
524
LIGHT AND KENNISON
Ericsson, K. A., & Simon, H. A. (1993). Protocol analysis: Verbal reports as data (Rev. ed.). Cambridge, MA: MIT Press. Hasher, L., & Zacks, R. T. (1979). Automatic and effortful processes in memory. Journal of Experimental Psychology: General, 108, 356–388. Hintzman, D. L. (1990). Human learning and memory: Connections and dissociations. Annual Review of Psychology, 41, 109–139. Jacoby, L. L. (1991). A process dissociation framework: Separating automatic from intentional uses of memory. Journal of Memory and Language, 30, 513–541. Jobe, J. B., Keller, D. M., & Smith, A. F. (1996). Cognitive techniques in interviewing older people. In N. Schwarz & S. Sudman (Eds.), Answering questions: Methodology for determining cognitive and communicative processes in survey research (pp. 197–219). San Francisco: Jossey–Bass. Light, L. L., & Kennison, R. F. (1996). Guessing strategies, aging, and bias effects in perceptual identification. Consciousness and Cognition, 5(4), 463–499. Masson, M. E. J., & Macleod, C. M. (1996). Contributions of processing fluency to repetition effects in masked word identification. Canadian Journal of Experimental Psychology, 50, 9–21. McKoon, G., & Ratcliff, R. (1996). Separating implicit from explicit retrieval processes in perceptual identification. Consciousness and Cognition, 5(4), 500–511. Mulligan, N. W. (in press). Attention and implicit memory tests: The effects of varying attentional load on conceptual priming. Memory & Cognition. Nisbett, R. E., & Wilson, T. D. (1977). Telling more than we can know: Verbal reports on mental processes. Psychological Review, 84, 231–259. Ostergaard, A. L., & Jernigan, T. L. (1993). Are word priming and explicit memory mediated by different brain structures? In P. Graf & M. E. J. Masson (Eds.), Implicit memory: New directions in cognition, development, and neuropsychology (pp. 327–349). Hillsdale, NJ: Erlbaum. Ratcliff, R., & McKoon, G. (1995). Bias in the priming of object decisions. Journal of Experimental Psychology: Learning, Memory, and Cognition, 21, 754–767. Ratcliff, R., & McKoon, G. (in press). A counter model for implicit priming in perceptual word identification. Psychological Review. Ratcliff, R., McKoon, G., & Verwoerd, M. (1989). A bias interpretation of facilitation in perceptual identification. Journal of Experimental Psychology: Learning, Memory, and Cognition, 15, 378– 387. Roediger, H. L. III, & McDermott, K. B. (1993). Implicit memory in normal human subjects. In F. Boller & J. Grafman (Eds.), Handbook of neuropsychology (Vol. 8, pp. 63–131). New York: Elsevier. Russo, J. E., Johnson, E. J., & Stephens, D. L. (1989). The validity of verbal protocols. Memory & Cognition, 17, 759–769. Schacter, D. L., & Cooper, L. A. (1995). Bias in the priming of object decisions: Logic, assumptions, and data. Journal of Experimental Psychology: Learning, Memory, and Cognition, 21, 768–776. Schacter, D. L., & Tulving, E. (1994). What are the memory systems of 1994? In D. L. Schacter & E. Tulving (Eds.), Memory systems 1994 (pp. 1–38). Cambridge, MA: MIT Press. Strack, F. (1992). ‘‘Order effects’’ in survey research: Activation and information functions of preceding questions. In N. Schwarz & S. Sudman (Eds.), Context effects in social and psychological research (pp. 23–34). New York: Springer-Verlag. Whittlesea, B. W. A., Jacoby, L. L., & Girard, K. (1990). Illusions of immediate memory: Evidence of an attributional basis for feelings of familiarity and perceptual quality. Journal of Memory and Language, 29, 716–732. Witherspoon, D., & Allan, L. G. (1985). The effect of a prior presentation on temporal judgments in a perceptual identification task. Memory & Cognition, 13, 101–111.