- Email: [email protected]
Stimulated verbal recall and analysis of sources of recall
~OURNAL OF VERBAL LEARNING AND VERBAL BEHAVIOR 2, 422-428 (1963)
Stimulated Verbal Recall and Analysis of Sources of Recall EDWARD
A. BILODEAU, PAUL W. Fox, AND KENNETH
A. BLICK
Department o] Psychology, Tulane University, New Orleans, Louislana~
Bilodeau, Sulzer, and Levy (1962) have urged the increased use of reminders in the study of retention. Reminders are stimuli which serve to cue or aid S to recall previous events. The basic events for which reminders are required have been identified as (a) responses previously made, (b) responses which might have been made but were not, and (c) informative feedbacks. Though aided recall is a common condition, retention research has traditionally emphasized variables of training. Methods of modified free recall in verbal ]earning have received some increased attention lately (Barnes and Underwood, 1959; Postman, 1961a). Bilodeau, Levy, and Sulzer (1963) have discussed how their analysis of the recall environment for motor memory can be generalized to studies employing verbal material and how recall may be induced by the regulation of related events. The present study follows from their analysis. The first objective was to demonstrate that verbal reminders do serve to evoke otherwise dormant verbal responses. The second objective was to vary recall by varying the reminders. The third objective was to assign each and every response to one of seven component sources of recall, instead of merely counting the number correct and incorrect. Russell and Jenkins (1954) published cultural norms produced by Kent-Rosanoff stimulus words. These display all associated re1 The principal p a r t of this experiment was accomplished while the senior author was on research leave granted by the Tulane University Council on Research.
sponses arranged in descending order of frequency. We have calculated the probability of evoking R given S. Thus R1, or cultural primary, represents the single most probable response to S~, R2 represents the second most probable response to $1, etc. For example, chair is an R~ to the stimulus table, and its probability(p) of occurrence is .84. The p of R2 (Jood) in discrete-free association is .04, etc. It was hypothesized that the presentation of an S~ such as table during a memory test (that is, using table as a reminder) would tend to arouse all the responses in the hierarchy in proportion to the values of p of R1. Thus, during recall the techniques of the free association experiment were to be blended with the techniques of the standard memory experiment. It was decided to train S with several R2 words (for example, food) and test for recall in the presence of the corresponding Sx words (for example, table). The training experience with R2 words should raise the values of p (R2) and thereby increase the probability of a correct recall over the cultural incidence of .04 of the example. The application of $1 should augment the probabilities of both R2 and R~ over those of a group of Ss not so stimulated. Therefore, the stimulated S, when correct, should respond with food, and when incorrect, chair might well obtrude as a response. Using Kent-Rosanoff stimuli during recall should produce response distributions which can be meaningfully compared with the norms of Russell-Jenkins. The norms were examined with respect to 422
423
STIMULATED VERBAL RECALL
the relationship between p(RI) and p(R2). A good curvilinear relation can be seen in the 100 dots of the scatter plot in Fig. 1. When .9
~, .7
n.-
"~'",,,,'
rr
- =p(R~..)
~,,," "
.5
.=p(R z)
":'
'
0
**
b'._.I
% .~
.
.°'~
..
• w
o_
• %%
"." ' ,
I
.2
,
I
:
n
.4 PROBABILITY OF
.,.
I
m
.6
"
I
.8
R I
The 100 words represented in Fig. 1 seemed fairly divisible into three categories: Category A: Low p(RI), low p(R2), and high p (Rs-~) Category B: Medium p(R1), maximum p(R2), and medium p ( R s - n ) Category C: High p(R1), low p(R2), and low p ( R s - n ) . It was hypothesized that stimulus words from category A would maximize the incidence of R s - , responses, category B would maximize the occurrence of R2 responses, and category C would maximize R1 responses. Fifteen stimulus words were found to suit the definitions of categories A, B, and C. Their mean probabilities of evoking R1, R2, and Rs-n are shown in Table 1. Since the three TABLE 1 RVSSELL-JENxlXS PROBABILITIES OT RESPONSEBY LIST CATEGORY
FIO. 1. Scatter plot showing the relationship between various terms within the hierarchy of associative responses of the Russell-Jenkins cultural norms. T h e eta between p ( R 2) and p ( R 1) equals .65.
p(R1) is very low or very high, p(R2) is low; when p(R1) is of moderate strength, p(R2) reaches its maximum. As a crude approximation, p(R2) is negatively related to p(R1)s greater than .30. Including all values of p(R1) produces an eta of .65, which confirms the suggestion of a substantial relationship between the cultural primaries and secondaries. The 100 dots with tails in Fig. 1 show a strong negative relationship between the probability of all other responses (summed over Rs to Rn) and p(R1). This amounts to p ( R s - n ) - - - - 1 . 0 0 - - p ( R 1 ) - - p ( R 2 ) . The relation between p ( R s - n ) and p(R1) is necessarily negative, but it is not necessarily as handsome as it turned out to be. It should be readily observed that the two relationships shown in Fig. 1 can be used to enhance the prediction of the incidence of correct responses and to foretell the kinds of responses produced when recall is incorrect•
AND
POSITION Ok" RESPONSE IN
HIERARCHY
Position in hierarchy Category A B C Da Grand Mean
P(R1) .14 .32 .73 .36 .38
P(R2) .12 .24 .05 .14 .14
p(Rs_n) .74 .44 .22 .50 .48
a Based on 55 stimulus words.
categories make use of 45 (3 X 15) stimulus words, the remaining Kent-Rosanoff words are called category D. A careful examination of the pattern of probabilities in Table 1 will best serve to reveal the purpose of this study, for we have hypothesized that recall and misrecall will vary with the values entered there. METHOD Subjects. A total of 166 Tulane University freshm e n a n d 34 Air Force reservists from the N e w Orleans area were unsystematically divided into two groups of 100 Ss each. The reservists were equally represented in both groups. Materials and Procedure. The basic i n s t r u m e n t was a 5-page booklet measuring 8~ X 7 inches, arranged from first to fifth page in the following sequence: cover page, 1st Training page, 2nd Training page,
424
BILODEAU, FOX, AND BLICK
vowel cancellation page, and Recall page. The cover page provided a line on which S wrote his name. The Recall page presented reminders (Group R) or it did not (Group NR). The freshmen were run in one intact group during freshman orientation week; the reservists were run during a regular reserve meeting. All page turning was paced by E who instructed Ss when and what to do with each page. In addition, instructions were printed at the top of each page and all Ss were explicitly told to follow the written instructions. In selecting the training material, ten R 2 words were repeatedly chosen at random (though no word was allowed to repeat) and typed five per page of Training. The Ss were asked for the recall of the first or second page. The booklet of any one reminded S was matched in material, sequence of training pages, and page called for by the booklet of a not-reminded S. The design made possible the examination of the effects of page, list, and sequence. Twenty sec. per page was allowed during Training. The instructions at the top of the first and second pages read: "Study the words shown below. Do not turn the page until you are told to do so." The retention interval was 120 sec. and was filled with a vowel-cancelling activity. Finally, an additional 120 sec. was allowed for the Recall page. Groups and Design. For Group R, the stimulus words corresponding to the R 2 words of one page of Training were typed alongside five empty lines on the Recall page. Group R received instructions as follows: "A short while ago you studied some words on page _ _ . The words below may help you remember those words. Write down the words that appeared on page _ _ . You must fill in every line. If you cannot remember, write in the first word that the printed word makes you think of. Do not look back to the previous pages." The Recall page for Group NR contained no stimulus words and similar instructions: " . . . If you cannot remember, write in the first word that you think of . . .-2 It is important to note that both groups were instructed to free associate if they were unable to remember the training words. A total of 1000 spaces was provided for responses (5 X 100 X 2) and examined according to one of seven classifications. The seven included position in 2 The two blanks appearing in each instruction were filled by hand with a large, red, printed A or B. The first and second pages of Training were labeled A and B, respectively. The printing was executed in a bold manner and contrasted sharply with the remainder of the page which was dittoed. These procedures were followed in order to enhance the differentiation of pages or lists and would tend to hold down the frequency of Other List intrusions.
Kent-Rosanoff Hierarchy (R l, R2, or Rs_n), Other List intrusion (an error arising from the other training page), or Same List intrusion (an error arising from the required page), Blank (where S failed to respond), or Remainder (none of the previous six). RESULTS Differences in n u m b e r of correct responses w i t h i n groups as a f u n c t i o n of p a g e effect, list order, or list effect were null. G r o u p R p r o d u c e d 487 responses and G r o u p N R 441 ( F z 1 4 . 3 , d]=1/198, p ~ . 0 1 ) . T h e totals per S are less v a r i a b l e for G r o u p R w h e n e x a m i n e d b y the p e r c e n t a g e of Ss failing to respond, or giving f r o m one to five responses. F o r G r o u p R, less t h a n 10% fail to p r o d u c e five responses, while 2 9 % f r o m G r o u p N R give less t h a n five. I n relation to total responses, R and N R Groups correctly recall 62 a n d 6 0 % of the items, respectively. W h e n a n a l y z e d in this way, it is seen t h a t reminders serve to evoke m o r e responses, but n o t a higher p e r c e n t a g e of correct ones. T h e p a t t e r n of misrecall differs g r e a t l y from group to group. F o r G r o u p N R , 3 0 % of a total of 237 errors are intrusions f r o m the O t h e r List, and 6 3 % of errors h a v e their source either in S's failure to m a k e a response ( 2 5 % ) or the emission of a u n i q u e response ( 3 8 % ) . I n contrast, o n l y 1 1 % of a total of 198 errors m a d e b y G r o u p R come f r o m the O t h e r List and o n l y 18% result f r o m the combined sources of Blanks (7 % ) and R e m a i n d e r ( 1 1 % ) . I n short, while 9 3 % of the errors m a d e b y G r o u p N R h a v e their source in other list intrusions, blanks or the r e m a i n d e r class, o n l y 2 9 % of the errors b y G r o u p R are att r i b u t a b l e to this c o m b i n a t i o n of factors. M o s t i m p o r t a n t of all, responses falling into R1 and R 3 - n a c c o u n t for 6 5 % of all errors m a d e b y G r o u p R while a p p e a r i n g b u t 7 % of the time for G r o u p N R . T o s u m m a r i z e the a b o v e findings, the rem i n d e r s h a v e e v o k e d a significantly higher level of response, a n d led to m a j o r differences in the source a n d c h a r a c t e r of errors produced. P r o v i d i n g r e m i n d e r s stimulates n o t o n l y R2
STIMULATED
425
VERBAL RECALL
TABLE 2 OBTAINED RESPONSE PROBABILITIES CLASSEFIEDBY LIST CATEGORY AND SOURCE OF RECALL
List
Position in K-R hierarchy Category A B C D Group R Group NR
P(R1)
P(R2)
.03 .13 .22 .12 .13 .01
.57 .67 .51 .62 .60 .22
p(Ra_n) .25 .10 .09 .12 .13 ~3
responses and a large proportion of R1 and R 3 - n responses but also leads to a corresponding decrease in the proportion of errors under Other List intrusions, Blanks, and Remainder. I n effect, the reminders have (a) aroused latent responses and (b) served to assign all Rs to one page or the other and to one line or another. Recall as a Function o] the Cultural Probabilities
Table 2 contains a two-way breakdown of the obtained probabilities of Group R with n ~ 500 as a base. The rows show the effect of the four categories of stimulus words or levels of p(R1), p(R2), and P ( R 3 - n ) . The columns show the results of assigning the data to seven component sources of recall. Only the responses tabled under p(R2) were Correct in the sense of being the right words written in the right places. The major purpose of the experiment was to reveal coVariation of the obtained response probabilities with probabilities of the KentRosanoff words given in Table 1. The two variables are obviously related; all 12 subgroups (4 categories by 3 positions in the Hierarchy) follow the pattern predicted. I n other words, the normative probabilities account for the trends in the data. For example, when the stimulus words are maximum for p(R1), intrusions of p(R1) words are also maximum; when the stimulus words are maximum for p(R2), correct responses are also at a maximum; and so forth2 3 The results of the present study were good, yet
Unresolved
Other
Same
.04 .06 .04 .03 .04 .14
.04 .00 .03 .03 .03 .30
R~mainder .03 .03 .06 .04 .04 .18
Blank .03 .00 .04 .03 .03 .12
The remaining information in Table 2 helps to place the major findings above in perspective. 1. Between 410 and 450 of the 500 responses of categories A, B, C, and D were found tabled in the Russell-Jenkins norms, indicating both the relevance and generalizability of the cultural norms# The narrow range of these figures suggests an over-all uniformity of production or difficulty. 2. The 500 responses of Group N R at the bottom of Table 2 show an entirely different profile with fewer instances of Rls, R2s and R3_~,s and higher incidences under List and Unresolved. The highest p (.30) is found under Same List, meaning that 150 of the recalls were words of the required list, but written in the wrong place on the Recall page. Also, another 150 of the recalls of Group N R were not resolved at all. 3. The last four sources of recall for G r o u p R produce scant data since each source amounts to approximately p ~ .03. There is no decisive way of telling why there is such uniformity, but it would seem that the remindleft some doubt about whether they could be repeated or generalized. So a second experiment was designed; it was simpler than its predecessor in that the 55 category D words were omitted entirely and other more minor technical simplifications effected. Furthermore, the experiment was run on an~ffther campus. The results were at least equally good, perhaps even better. 4 For category A, for example, add .03, .57, and .25. Also, an examination of the mean total number of different words evoked in the Russell-Jenkins norms showed 128, 96, and 80 for categories Aj-B, and C, respectively.
426
BILODEAU~ FOX, AND BLICK
ers worked well enough to leave explained by the four traditional error. 4. The distributions of Other sions are summarized in Table 3.
little to be sources of List intruGroup N R
TABLE 3 ANALYSIS OF OTI-IER LIST INTRUSIONS No. of
Percentage of Ss
Intrusions
Group R
Group N R
0 1 2 3 4 5
82 16 1 1 0 0
56 23 17 2 2 0
produces .14 recalls from the Other List, an especially noteworthy finding. This is low on an absolute basis, but high compared to the values ordinarily obtained in retention research employing two lists and more orthodox training procedures. Group R produces only .04 recalls from the Other List, and provides an excellent base from which to begin research on the deliberate stimulation of Other List intrusions.
it is to work. Normative data based upon continued associations should improve the power to predict from the three stimulus categories. Such norms are not available but have been discussed by Russell (1961). A basic feature of the present experiment was the use of extraexperimental information about response arousal as the major independent variable. By relating R1, R2, and R8-11 to a common stimulus term in the norms it was possible to employ that stimulus term independently of the material of Training and to examine the positive and negative effects of stimulated recall over associative probabilities ranging from .1i to .84. This approach seems in keeping with the aims of the unit-sequence hypothesis of Underwood and Postman (1960) in many ways, and our control over intrusions of R~ and Ra-,1 arising from preexperimental sources bears importantly on this hypothesis.
Since Same List and Other List have led to such low and uniform entries, we now know that the present training materials and techniques of cuing recall can be used as a standard of comparison in evaluating strategies of DISCUSSION retrieval calculated to produce Same and Other List intrusions. The objectives are simiOne thousand spaces for responses were prolar to those of Deese (1959a, 1959b, 1961), vided in Recall and each was examined and though he has concentrated his attention on assigned to one of seven classifications of inter-item relationships of the training marecall and misrecall. This analysis showed that the cultural norms of Russell and Jenkins have terial and tested for retention by the method considerable power in accounting for what of ]ree recall. We have, on the contrary, centered our attention on modifying the recall other individuals will say in a recall context. It must be noted that every response tabled environment and paid no special attention to by Russell and Jenkins is a primary response the inter-item associative strengths. Since the or R1, the result of a single free association. training material consisted exclusively of variFor example, when p ( R 2 ) - - . 2 0 , one of ous R2s, we are confident that the inter-item every five Ss responded first with R2. Yet the relations are fairly low. It would seem, then, present experiment treated R2 as if it were that the studies of Deese are similar in intent second in strength to R1, i.e., an S asked to and complementary in procedure. respond two times to the stimulus word would The present data argue for the exploitation respond first with R1 and then with R2. Rosen of cultural norms for the purpose of analyzing and Russell (1957) and Corer (1958) have interference of the extraexperimentat variety shown this to be true to a degree and it is (Underwood and Postman, 1960; Postman, what the present study presumes to be true if 1961b). Accordingly, one important conse-
STIMULATED VERBAL RECALL
427
quence will be an increased emphasis on the most frequent responses (R2) to Kent-Rosaconstruction of training materials and varia- noff stimulus words, and were tested 2 rain. tions of reminders that will permit still other later for recall in the presence (Reminded) or assessments of the influence of preexisting absence (Not Reminded) of the stimuli correlanguage habits on retention. Given the prob- sponding to the R2 words. The 1000 pieces of ability relationships between stimuli and their data were assigned to one of seven classes of response associates, it should not be too diffi- recall. Stimulated recall produced major difcult to demonstrate interference of a proactive ferences in the amount and source of errors. sort far beyond the level produced here. Blick The Russell-Jenkins cultural norms were (1963), for example, has successfully shown analyzed to separate the stimulus words into that the intrusion of primaries is strongly three categories. The categories varied the related to the value of p(R1). Such experi- probability of producing R1 responses (culments, in effect, are designed to produce mis- tural primary), R2 (cultural secondary), and recalls of specific varieties by taking advantage R~-n (sum of cultural responses 3 to n). of cultural S-R habits. Correct and incorrect recalls varied directly Finally, the objectives of this experiment, with the cultural probabilities in the norms. developed in extending motor-memory meth- It was concluded that the present technique ods to verbal recall, demanded major proce- reveals the powerful effects of extraexperidural innovations which need to be restated. mental, proacting interference and facilitation. (a) A general-purpose booklet technique for It was also concluded that the methodologies training and testing intact classes was devel- of memory and association testing can be oped. The booklet took the classical form of gainfully blended. Training A, Training B, and Test A or B. REFERENCES (b) The verbal items were treated as a scalar variable by determining their strengths of BARNES, J. M., AND UNDERWOOD, B. J. "Fate" of first-list associatiops in transfer theory. ]. exp. association in Kent-Rosanoff word hierarchies. Psychol., 1959, 58, 97-105. (c) A reanalysis of the cultural norms was BILODEAU, E. A., LEVY, C. M., AND SI/LZER, J. L. made. By establishing the quantity p(R1) to Long-term retention under conditions of artifiS as the critical independent variable, stimcially induced recall of related events. Percept. ulus categories with quantitatively different mot. Skills, 1963, 16, 895-910. predictor patterns were found. (d) Words BILODEAU, E. A., SULZER, J. L., AND LEVY, C. M. Theory and data on the interrelationships of (R2) which were relatively weakly related to three factors of memory. Psychol. Monogr., 1962, the normative stimuli were selected as train76, No. 20 (Whole No. 539). ing materials. (e) The recall environment was BLICK, K. A. Dominant responses as a source of manipulated by introducing cues experiinterference in verbal retention. Unpublished doctoral dissertation, Tulane Univer., 1963. mentally independent of training, yet bearing known pre-experimental relations to the ma- COEER, C. N. Comparison of word associations obtained by the methods of discrete single word terial of training, and then combining the and continued association. Psychol. Rep., 1958, techniques of free association and memory 4, 507-510. testing. (f) An analytic scoring system was DEESE, J. On the prediction of occurrence of particdeveloped to permit the evaluation of six ular verbal intrusions in immediate recall. ]. exp. Psychol., 1959, 58, 17-22. (a) operationally different sources of error. DEESE, J. Influence of inter-item associative strength upon immediate free recall. Psychol. Rep., 1959, SUMMARY 5, 305-312. (b) Two groups of 100 Ss each were trained DEESE, J. From the isolated verbal unit to connected with lists of five items consisting of the second discourse. In C. N. Corer (Ed.), Verbal learning
428
BILODEAU~ FOX~ AND BLICK
and verbal behavior. New York: McGraw-Hill, 1961. Pp. 11-31. POSTMAN, L. The present status of interference theory. In C. N. Cofer (Ed.), Verbal learning and verbal behavior. New York: McGraw-Hill, 1961. Pp. 152-196. (a) POSTlVmN, L. Extra-experimental interference and the retention of words. ]. exp. Psychol., 1961, 61, 97110. (b) ROSEI~, E., AND RUSSELL, W. A. Frequency-characteristics of successive word-association. Amer. ]. Psychol., 1957, 70, 120-122. RUSSELL, W. A. Assessment versus experimental
acquisition of verbal habits. In C. N. Cofer (Ed.), Verbal learning and verbal behavior. New York: McGraw-Hill, 1961. Pp. 110-123. RUSSELL, W. A., AND JENKINS, J. J. The complete Minnesota norms for responses to 100 words from the Kent-Rosanoff Word Association Test. Tech. Rep. No. 11, 1954, Contract No. NSonr66216, Office of Naval Research and University of Minnesota. U1WDERWOOD, B. J., AND POSTiM~,N, L. Extraexperimental sources of interference in forgetting. Psychol. Rev., 1960, 67, 73-95. (Received April 11, 1963)
Stimulated Verbal Recall and Analysis of Sources of Recall EDWARD
A. BILODEAU, PAUL W. Fox, AND KENNETH
A. BLICK
Department o] Psychology, Tulane University, New Orleans, Louislana~
Bilodeau, Sulzer, and Levy (1962) have urged the increased use of reminders in the study of retention. Reminders are stimuli which serve to cue or aid S to recall previous events. The basic events for which reminders are required have been identified as (a) responses previously made, (b) responses which might have been made but were not, and (c) informative feedbacks. Though aided recall is a common condition, retention research has traditionally emphasized variables of training. Methods of modified free recall in verbal ]earning have received some increased attention lately (Barnes and Underwood, 1959; Postman, 1961a). Bilodeau, Levy, and Sulzer (1963) have discussed how their analysis of the recall environment for motor memory can be generalized to studies employing verbal material and how recall may be induced by the regulation of related events. The present study follows from their analysis. The first objective was to demonstrate that verbal reminders do serve to evoke otherwise dormant verbal responses. The second objective was to vary recall by varying the reminders. The third objective was to assign each and every response to one of seven component sources of recall, instead of merely counting the number correct and incorrect. Russell and Jenkins (1954) published cultural norms produced by Kent-Rosanoff stimulus words. These display all associated re1 The principal p a r t of this experiment was accomplished while the senior author was on research leave granted by the Tulane University Council on Research.
sponses arranged in descending order of frequency. We have calculated the probability of evoking R given S. Thus R1, or cultural primary, represents the single most probable response to S~, R2 represents the second most probable response to $1, etc. For example, chair is an R~ to the stimulus table, and its probability(p) of occurrence is .84. The p of R2 (Jood) in discrete-free association is .04, etc. It was hypothesized that the presentation of an S~ such as table during a memory test (that is, using table as a reminder) would tend to arouse all the responses in the hierarchy in proportion to the values of p of R1. Thus, during recall the techniques of the free association experiment were to be blended with the techniques of the standard memory experiment. It was decided to train S with several R2 words (for example, food) and test for recall in the presence of the corresponding Sx words (for example, table). The training experience with R2 words should raise the values of p (R2) and thereby increase the probability of a correct recall over the cultural incidence of .04 of the example. The application of $1 should augment the probabilities of both R2 and R~ over those of a group of Ss not so stimulated. Therefore, the stimulated S, when correct, should respond with food, and when incorrect, chair might well obtrude as a response. Using Kent-Rosanoff stimuli during recall should produce response distributions which can be meaningfully compared with the norms of Russell-Jenkins. The norms were examined with respect to 422
423
STIMULATED VERBAL RECALL
the relationship between p(RI) and p(R2). A good curvilinear relation can be seen in the 100 dots of the scatter plot in Fig. 1. When .9
~, .7
n.-
"~'",,,,'
rr
- =p(R~..)
~,,," "
.5
.=p(R z)
":'
'
0
**
b'._.I
% .~
.
.°'~
..
• w
o_
• %%
"." ' ,
I
.2
,
I
:
n
.4 PROBABILITY OF
.,.
I
m
.6
"
I
.8
R I
The 100 words represented in Fig. 1 seemed fairly divisible into three categories: Category A: Low p(RI), low p(R2), and high p (Rs-~) Category B: Medium p(R1), maximum p(R2), and medium p ( R s - n ) Category C: High p(R1), low p(R2), and low p ( R s - n ) . It was hypothesized that stimulus words from category A would maximize the incidence of R s - , responses, category B would maximize the occurrence of R2 responses, and category C would maximize R1 responses. Fifteen stimulus words were found to suit the definitions of categories A, B, and C. Their mean probabilities of evoking R1, R2, and Rs-n are shown in Table 1. Since the three TABLE 1 RVSSELL-JENxlXS PROBABILITIES OT RESPONSEBY LIST CATEGORY
FIO. 1. Scatter plot showing the relationship between various terms within the hierarchy of associative responses of the Russell-Jenkins cultural norms. T h e eta between p ( R 2) and p ( R 1) equals .65.
p(R1) is very low or very high, p(R2) is low; when p(R1) is of moderate strength, p(R2) reaches its maximum. As a crude approximation, p(R2) is negatively related to p(R1)s greater than .30. Including all values of p(R1) produces an eta of .65, which confirms the suggestion of a substantial relationship between the cultural primaries and secondaries. The 100 dots with tails in Fig. 1 show a strong negative relationship between the probability of all other responses (summed over Rs to Rn) and p(R1). This amounts to p ( R s - n ) - - - - 1 . 0 0 - - p ( R 1 ) - - p ( R 2 ) . The relation between p ( R s - n ) and p(R1) is necessarily negative, but it is not necessarily as handsome as it turned out to be. It should be readily observed that the two relationships shown in Fig. 1 can be used to enhance the prediction of the incidence of correct responses and to foretell the kinds of responses produced when recall is incorrect•
AND
POSITION Ok" RESPONSE IN
HIERARCHY
Position in hierarchy Category A B C Da Grand Mean
P(R1) .14 .32 .73 .36 .38
P(R2) .12 .24 .05 .14 .14
p(Rs_n) .74 .44 .22 .50 .48
a Based on 55 stimulus words.
categories make use of 45 (3 X 15) stimulus words, the remaining Kent-Rosanoff words are called category D. A careful examination of the pattern of probabilities in Table 1 will best serve to reveal the purpose of this study, for we have hypothesized that recall and misrecall will vary with the values entered there. METHOD Subjects. A total of 166 Tulane University freshm e n a n d 34 Air Force reservists from the N e w Orleans area were unsystematically divided into two groups of 100 Ss each. The reservists were equally represented in both groups. Materials and Procedure. The basic i n s t r u m e n t was a 5-page booklet measuring 8~ X 7 inches, arranged from first to fifth page in the following sequence: cover page, 1st Training page, 2nd Training page,
424
BILODEAU, FOX, AND BLICK
vowel cancellation page, and Recall page. The cover page provided a line on which S wrote his name. The Recall page presented reminders (Group R) or it did not (Group NR). The freshmen were run in one intact group during freshman orientation week; the reservists were run during a regular reserve meeting. All page turning was paced by E who instructed Ss when and what to do with each page. In addition, instructions were printed at the top of each page and all Ss were explicitly told to follow the written instructions. In selecting the training material, ten R 2 words were repeatedly chosen at random (though no word was allowed to repeat) and typed five per page of Training. The Ss were asked for the recall of the first or second page. The booklet of any one reminded S was matched in material, sequence of training pages, and page called for by the booklet of a not-reminded S. The design made possible the examination of the effects of page, list, and sequence. Twenty sec. per page was allowed during Training. The instructions at the top of the first and second pages read: "Study the words shown below. Do not turn the page until you are told to do so." The retention interval was 120 sec. and was filled with a vowel-cancelling activity. Finally, an additional 120 sec. was allowed for the Recall page. Groups and Design. For Group R, the stimulus words corresponding to the R 2 words of one page of Training were typed alongside five empty lines on the Recall page. Group R received instructions as follows: "A short while ago you studied some words on page _ _ . The words below may help you remember those words. Write down the words that appeared on page _ _ . You must fill in every line. If you cannot remember, write in the first word that the printed word makes you think of. Do not look back to the previous pages." The Recall page for Group NR contained no stimulus words and similar instructions: " . . . If you cannot remember, write in the first word that you think of . . .-2 It is important to note that both groups were instructed to free associate if they were unable to remember the training words. A total of 1000 spaces was provided for responses (5 X 100 X 2) and examined according to one of seven classifications. The seven included position in 2 The two blanks appearing in each instruction were filled by hand with a large, red, printed A or B. The first and second pages of Training were labeled A and B, respectively. The printing was executed in a bold manner and contrasted sharply with the remainder of the page which was dittoed. These procedures were followed in order to enhance the differentiation of pages or lists and would tend to hold down the frequency of Other List intrusions.
Kent-Rosanoff Hierarchy (R l, R2, or Rs_n), Other List intrusion (an error arising from the other training page), or Same List intrusion (an error arising from the required page), Blank (where S failed to respond), or Remainder (none of the previous six). RESULTS Differences in n u m b e r of correct responses w i t h i n groups as a f u n c t i o n of p a g e effect, list order, or list effect were null. G r o u p R p r o d u c e d 487 responses and G r o u p N R 441 ( F z 1 4 . 3 , d]=1/198, p ~ . 0 1 ) . T h e totals per S are less v a r i a b l e for G r o u p R w h e n e x a m i n e d b y the p e r c e n t a g e of Ss failing to respond, or giving f r o m one to five responses. F o r G r o u p R, less t h a n 10% fail to p r o d u c e five responses, while 2 9 % f r o m G r o u p N R give less t h a n five. I n relation to total responses, R and N R Groups correctly recall 62 a n d 6 0 % of the items, respectively. W h e n a n a l y z e d in this way, it is seen t h a t reminders serve to evoke m o r e responses, but n o t a higher p e r c e n t a g e of correct ones. T h e p a t t e r n of misrecall differs g r e a t l y from group to group. F o r G r o u p N R , 3 0 % of a total of 237 errors are intrusions f r o m the O t h e r List, and 6 3 % of errors h a v e their source either in S's failure to m a k e a response ( 2 5 % ) or the emission of a u n i q u e response ( 3 8 % ) . I n contrast, o n l y 1 1 % of a total of 198 errors m a d e b y G r o u p R come f r o m the O t h e r List and o n l y 18% result f r o m the combined sources of Blanks (7 % ) and R e m a i n d e r ( 1 1 % ) . I n short, while 9 3 % of the errors m a d e b y G r o u p N R h a v e their source in other list intrusions, blanks or the r e m a i n d e r class, o n l y 2 9 % of the errors b y G r o u p R are att r i b u t a b l e to this c o m b i n a t i o n of factors. M o s t i m p o r t a n t of all, responses falling into R1 and R 3 - n a c c o u n t for 6 5 % of all errors m a d e b y G r o u p R while a p p e a r i n g b u t 7 % of the time for G r o u p N R . T o s u m m a r i z e the a b o v e findings, the rem i n d e r s h a v e e v o k e d a significantly higher level of response, a n d led to m a j o r differences in the source a n d c h a r a c t e r of errors produced. P r o v i d i n g r e m i n d e r s stimulates n o t o n l y R2
STIMULATED
425
VERBAL RECALL
TABLE 2 OBTAINED RESPONSE PROBABILITIES CLASSEFIEDBY LIST CATEGORY AND SOURCE OF RECALL
List
Position in K-R hierarchy Category A B C D Group R Group NR
P(R1)
P(R2)
.03 .13 .22 .12 .13 .01
.57 .67 .51 .62 .60 .22
p(Ra_n) .25 .10 .09 .12 .13 ~3
responses and a large proportion of R1 and R 3 - n responses but also leads to a corresponding decrease in the proportion of errors under Other List intrusions, Blanks, and Remainder. I n effect, the reminders have (a) aroused latent responses and (b) served to assign all Rs to one page or the other and to one line or another. Recall as a Function o] the Cultural Probabilities
Table 2 contains a two-way breakdown of the obtained probabilities of Group R with n ~ 500 as a base. The rows show the effect of the four categories of stimulus words or levels of p(R1), p(R2), and P ( R 3 - n ) . The columns show the results of assigning the data to seven component sources of recall. Only the responses tabled under p(R2) were Correct in the sense of being the right words written in the right places. The major purpose of the experiment was to reveal coVariation of the obtained response probabilities with probabilities of the KentRosanoff words given in Table 1. The two variables are obviously related; all 12 subgroups (4 categories by 3 positions in the Hierarchy) follow the pattern predicted. I n other words, the normative probabilities account for the trends in the data. For example, when the stimulus words are maximum for p(R1), intrusions of p(R1) words are also maximum; when the stimulus words are maximum for p(R2), correct responses are also at a maximum; and so forth2 3 The results of the present study were good, yet
Unresolved
Other
Same
.04 .06 .04 .03 .04 .14
.04 .00 .03 .03 .03 .30
R~mainder .03 .03 .06 .04 .04 .18
Blank .03 .00 .04 .03 .03 .12
The remaining information in Table 2 helps to place the major findings above in perspective. 1. Between 410 and 450 of the 500 responses of categories A, B, C, and D were found tabled in the Russell-Jenkins norms, indicating both the relevance and generalizability of the cultural norms# The narrow range of these figures suggests an over-all uniformity of production or difficulty. 2. The 500 responses of Group N R at the bottom of Table 2 show an entirely different profile with fewer instances of Rls, R2s and R3_~,s and higher incidences under List and Unresolved. The highest p (.30) is found under Same List, meaning that 150 of the recalls were words of the required list, but written in the wrong place on the Recall page. Also, another 150 of the recalls of Group N R were not resolved at all. 3. The last four sources of recall for G r o u p R produce scant data since each source amounts to approximately p ~ .03. There is no decisive way of telling why there is such uniformity, but it would seem that the remindleft some doubt about whether they could be repeated or generalized. So a second experiment was designed; it was simpler than its predecessor in that the 55 category D words were omitted entirely and other more minor technical simplifications effected. Furthermore, the experiment was run on an~ffther campus. The results were at least equally good, perhaps even better. 4 For category A, for example, add .03, .57, and .25. Also, an examination of the mean total number of different words evoked in the Russell-Jenkins norms showed 128, 96, and 80 for categories Aj-B, and C, respectively.
426
BILODEAU~ FOX, AND BLICK
ers worked well enough to leave explained by the four traditional error. 4. The distributions of Other sions are summarized in Table 3.
little to be sources of List intruGroup N R
TABLE 3 ANALYSIS OF OTI-IER LIST INTRUSIONS No. of
Percentage of Ss
Intrusions
Group R
Group N R
0 1 2 3 4 5
82 16 1 1 0 0
56 23 17 2 2 0
produces .14 recalls from the Other List, an especially noteworthy finding. This is low on an absolute basis, but high compared to the values ordinarily obtained in retention research employing two lists and more orthodox training procedures. Group R produces only .04 recalls from the Other List, and provides an excellent base from which to begin research on the deliberate stimulation of Other List intrusions.
it is to work. Normative data based upon continued associations should improve the power to predict from the three stimulus categories. Such norms are not available but have been discussed by Russell (1961). A basic feature of the present experiment was the use of extraexperimental information about response arousal as the major independent variable. By relating R1, R2, and R8-11 to a common stimulus term in the norms it was possible to employ that stimulus term independently of the material of Training and to examine the positive and negative effects of stimulated recall over associative probabilities ranging from .1i to .84. This approach seems in keeping with the aims of the unit-sequence hypothesis of Underwood and Postman (1960) in many ways, and our control over intrusions of R~ and Ra-,1 arising from preexperimental sources bears importantly on this hypothesis.
Since Same List and Other List have led to such low and uniform entries, we now know that the present training materials and techniques of cuing recall can be used as a standard of comparison in evaluating strategies of DISCUSSION retrieval calculated to produce Same and Other List intrusions. The objectives are simiOne thousand spaces for responses were prolar to those of Deese (1959a, 1959b, 1961), vided in Recall and each was examined and though he has concentrated his attention on assigned to one of seven classifications of inter-item relationships of the training marecall and misrecall. This analysis showed that the cultural norms of Russell and Jenkins have terial and tested for retention by the method considerable power in accounting for what of ]ree recall. We have, on the contrary, centered our attention on modifying the recall other individuals will say in a recall context. It must be noted that every response tabled environment and paid no special attention to by Russell and Jenkins is a primary response the inter-item associative strengths. Since the or R1, the result of a single free association. training material consisted exclusively of variFor example, when p ( R 2 ) - - . 2 0 , one of ous R2s, we are confident that the inter-item every five Ss responded first with R2. Yet the relations are fairly low. It would seem, then, present experiment treated R2 as if it were that the studies of Deese are similar in intent second in strength to R1, i.e., an S asked to and complementary in procedure. respond two times to the stimulus word would The present data argue for the exploitation respond first with R1 and then with R2. Rosen of cultural norms for the purpose of analyzing and Russell (1957) and Corer (1958) have interference of the extraexperimentat variety shown this to be true to a degree and it is (Underwood and Postman, 1960; Postman, what the present study presumes to be true if 1961b). Accordingly, one important conse-
STIMULATED VERBAL RECALL
427
quence will be an increased emphasis on the most frequent responses (R2) to Kent-Rosaconstruction of training materials and varia- noff stimulus words, and were tested 2 rain. tions of reminders that will permit still other later for recall in the presence (Reminded) or assessments of the influence of preexisting absence (Not Reminded) of the stimuli correlanguage habits on retention. Given the prob- sponding to the R2 words. The 1000 pieces of ability relationships between stimuli and their data were assigned to one of seven classes of response associates, it should not be too diffi- recall. Stimulated recall produced major difcult to demonstrate interference of a proactive ferences in the amount and source of errors. sort far beyond the level produced here. Blick The Russell-Jenkins cultural norms were (1963), for example, has successfully shown analyzed to separate the stimulus words into that the intrusion of primaries is strongly three categories. The categories varied the related to the value of p(R1). Such experi- probability of producing R1 responses (culments, in effect, are designed to produce mis- tural primary), R2 (cultural secondary), and recalls of specific varieties by taking advantage R~-n (sum of cultural responses 3 to n). of cultural S-R habits. Correct and incorrect recalls varied directly Finally, the objectives of this experiment, with the cultural probabilities in the norms. developed in extending motor-memory meth- It was concluded that the present technique ods to verbal recall, demanded major proce- reveals the powerful effects of extraexperidural innovations which need to be restated. mental, proacting interference and facilitation. (a) A general-purpose booklet technique for It was also concluded that the methodologies training and testing intact classes was devel- of memory and association testing can be oped. The booklet took the classical form of gainfully blended. Training A, Training B, and Test A or B. REFERENCES (b) The verbal items were treated as a scalar variable by determining their strengths of BARNES, J. M., AND UNDERWOOD, B. J. "Fate" of first-list associatiops in transfer theory. ]. exp. association in Kent-Rosanoff word hierarchies. Psychol., 1959, 58, 97-105. (c) A reanalysis of the cultural norms was BILODEAU, E. A., LEVY, C. M., AND SI/LZER, J. L. made. By establishing the quantity p(R1) to Long-term retention under conditions of artifiS as the critical independent variable, stimcially induced recall of related events. Percept. ulus categories with quantitatively different mot. Skills, 1963, 16, 895-910. predictor patterns were found. (d) Words BILODEAU, E. A., SULZER, J. L., AND LEVY, C. M. Theory and data on the interrelationships of (R2) which were relatively weakly related to three factors of memory. Psychol. Monogr., 1962, the normative stimuli were selected as train76, No. 20 (Whole No. 539). ing materials. (e) The recall environment was BLICK, K. A. Dominant responses as a source of manipulated by introducing cues experiinterference in verbal retention. Unpublished doctoral dissertation, Tulane Univer., 1963. mentally independent of training, yet bearing known pre-experimental relations to the ma- COEER, C. N. Comparison of word associations obtained by the methods of discrete single word terial of training, and then combining the and continued association. Psychol. Rep., 1958, techniques of free association and memory 4, 507-510. testing. (f) An analytic scoring system was DEESE, J. On the prediction of occurrence of particdeveloped to permit the evaluation of six ular verbal intrusions in immediate recall. ]. exp. Psychol., 1959, 58, 17-22. (a) operationally different sources of error. DEESE, J. Influence of inter-item associative strength upon immediate free recall. Psychol. Rep., 1959, SUMMARY 5, 305-312. (b) Two groups of 100 Ss each were trained DEESE, J. From the isolated verbal unit to connected with lists of five items consisting of the second discourse. In C. N. Corer (Ed.), Verbal learning
428
BILODEAU~ FOX~ AND BLICK
and verbal behavior. New York: McGraw-Hill, 1961. Pp. 11-31. POSTMAN, L. The present status of interference theory. In C. N. Cofer (Ed.), Verbal learning and verbal behavior. New York: McGraw-Hill, 1961. Pp. 152-196. (a) POSTlVmN, L. Extra-experimental interference and the retention of words. ]. exp. Psychol., 1961, 61, 97110. (b) ROSEI~, E., AND RUSSELL, W. A. Frequency-characteristics of successive word-association. Amer. ]. Psychol., 1957, 70, 120-122. RUSSELL, W. A. Assessment versus experimental
acquisition of verbal habits. In C. N. Cofer (Ed.), Verbal learning and verbal behavior. New York: McGraw-Hill, 1961. Pp. 110-123. RUSSELL, W. A., AND JENKINS, J. J. The complete Minnesota norms for responses to 100 words from the Kent-Rosanoff Word Association Test. Tech. Rep. No. 11, 1954, Contract No. NSonr66216, Office of Naval Research and University of Minnesota. U1WDERWOOD, B. J., AND POSTiM~,N, L. Extraexperimental sources of interference in forgetting. Psychol. Rev., 1960, 67, 73-95. (Received April 11, 1963)