Cognitive and Affective Outcomes of Learning from Knowledge Maps

JOBNAME: Vol 21#1 PAGE: 1 SESS: 63 OUTPUT: Mon Apr 29 10:53:29 1996 /xypage/worksmart/tsp000/66511f/3

CONTEMPORARY EDUCATIONAL PSYCHOLOGY ARTICLE NO. 0008

21, 94–101 (1996)

BRIEF RESEARCH REPORT Cognitive and Affective Outcomes of Learning from Knowledge Maps RICHARD H. HALL University of Missouri, Rolla AND

ANGELA O’DONNELL Rutgers University The purpose of the present experiment was to replicate and extend research on knowledge maps by examining both cognitive and affective outcomes associated with knowledge map processing. Students studied a 1500 word passage on the autonomic nervous system in the form of a knowledge map or traditional text and completed a free recall test on the information 2 days later. On both days of the experiment, students completed subjective graphs on which they were required to rate their motivation, anxiety, and concentration over the course of the experiment. Significant differences were found between the map and text groups on measures of recall, subjective concentration, and motivation, with those in the map group scoring significantly higher. In addition, correlational analyses indicated that recall measures and the subjective measures were not significantly related. The results indicate that the positive impact of knowledge maps goes beyond objective cognitive outcomes to include subjective ratings of concentration and motivation as well. © 1996 Academic Press, Inc.

Knowledge maps are a method of displaying text material in a two dimensional, spatial, node link network. A number of experiments indicate that this method is an effective vehicle for the delivery of academic text (e.g., Hall, Dansereau, & Skaggs, 1992; Patterson, Dansereau, & Newborn, 1992; Lambiotte & Dansereau, 1992). This research has been couched within Danserea’s spatial/ verbal processing model (Dansereau, 1989; Lambiotte, Dansereau, Cross, & Reynolds, 1989). One of the basic assumptions of the model is that the structural properties of the map format activate spatial processing channels which enhance superordinate/main idea concepts. Presumably during subsequent retrieval, this structural information, stored within spatial schemas, can act to cross cue detailed information during recall (Dansereau, 1989; Dean & Kulhavy, 1981; Kulhavy, Lee, & Caterino, 1985; Lambiotte et al., 1989). One purpose of this experiment was to replicate previous research on knowledge maps by assessing the effect of Address reprint requests to Richard H. Hall, Department of Psychology, University of Missouri, Rolla, MO 65401-0249. 94 0361-476X/96 $18.00 Copyright © 1996 by Academic Press, Inc. All rights of reproduction in any form reserved.

JOBNAME: Vol 21#1 PAGE: 2 SESS: 63 OUTPUT: Mon Apr 29 10:53:29 1996 /xypage/worksmart/tsp000/66511f/3

KNOWLEDGE MAPS

95

studying from knowledge maps on recall of superordinate and subordinate information. The principal purpose of this experiment was to extend research on knowledge maps by examining affective outcomes of studying from the maps. Through the examination of these additional outcomes, a more complete picture of knowledge map processing will emerge, and the model cited above can be extended to include affective factors. This is particularly important since informal observations of students studying from knowledge maps indicates that initial exposure to the maps creates a significant affective response sometimes termed “map shock.” While this term may imply a negative consequence of the maps, the effectiveness of the maps may be partially due to some type of “metacognitive triggering effect” that is often associated with such a “boost” in affect (Baker & Brown, 1984). If various subjective-affective outcomes are found to be related to knowledge map processing, a second important question follows. Are these outcomes related to the objective-cognitive advantages of the mapping method that have been found in previous experiments? Although intuitively it would seem that low levels of subjective anxiety, high levels of subjective motivation, and high levels of subjective concentration would be associated with increased recall, research indicates that this is not necessarily the case e.g., (Gross, 1990). In fact, in one experiment that used a subjective graphing technique such as the present experiment, measures of motivation were not significantly predictive of recall (Hall, Dansereau, O’Donnell, & Skaggs, 1989). In another experiment utilizing this technique in conjunction with cooperative and individual learning, students’ subjective ratings of concentration were significantly predictive of recall on some recall tests but not others (Hall, Dansereau, & O’Donnell, 1989). In the present experiment, the ongoing subjective states of the learner were measured via a subjective graphing instrument (Hall, Dansereau, & O’Donnell, 1989; O’Donnell, Dansereau, Hall, & Rocklin, 1987). When completing subjective graphs, students are asked to subjectively rate their anxiety/nervousness, motivation, and concentration over the learning episode. The principal advantage of this method is that it allows students to represent their processing in a continuous, rather than discrete, fashion. This is almost surely more consistent with students’ own subjective view of their ongoing affect. Initial investigations indicate that the measure is psychometrically sound (Hall, Dansereau, & O’Donnell, 1989), and that the graphs are an effective tool for measuring affect within the context of an experiment on alternative learning techniques (Hall, Dansereau, & O’Donnell, 1989; Hall et al., 1989; O’Donnell et al., 1987; 1988). The present experiment will address the following experimental questions: (1) What are the differences in subjective perceptions of anxiety, motivation, and concentration between those who study from knowledge maps vs those who study from traditional text, during studying and testing; (2) What are the differences between those same groups in objective cognitive outcomes (i.e., recall of

JOBNAME: Vol 21#1 PAGE: 3 SESS: 63 OUTPUT: Mon Apr 29 10:53:29 1996 /xypage/worksmart/tsp000/66511f/3

96

HALL AND O’DONNELL

superordinate vs subordinate information they studied); and 3) What is the relationship between subjective and objective outcomes? METHOD Participants Forty-three participants recruited from general psychology classes at a small science and technology-oriented midwestern public university participated. They received class credit for their participation.

Materials During the training phase of the experiment students used a 500-word knowledge map that represented a comparison of football and baseball. During the experimental treatment phase, students studied information consisting of approximately 1500 words that described the relationship between the sympathetic and parasympathetic divisions of the autonomic nervous system, in the form of a knowledge map or traditional text. The text and maps both included the same number of words. In order to assess students’ subjective ratings, three subjective graphs were used on both days of the experiment. On the Y axis of the graphs is a scale from 1 to 10 representing the degree of a given processing state. In this experiment concentration, motivation, and anxiety were used. The X axis represents time, and different activities that occurred during the experiment are marked on this axis to give students a framework for rating their reactions. Students also completed the Delta Reading Vocabulary test (Deignan, 1973) and the Map Planning test of spatial scanning ability (Ekstrom, French, & Harman, 1976) to be used as potential covariates in the analysis of recall. Both the Delta Reading Vocabulary test (Hall, Dansereau, & Skaggs, 1992; Hall & Sidio-Hall, 1994a) and the map planning test (Hall & Sidio-Hall, 1994b) have been found to be significant predictors of recall of information studied in the form of knowledge maps in previous research. Copies of any of the materials used can be obtained from the authors.

Procedure On the first day of the study, the experimenter began by giving participants a brief overview of the experiment. Students were then given a few minutes to become familiar with the training map, after which the experimenter briefly went over the map with the students. (In order to eliminate differential training effects, all students received this introductory knowledge map exposure, whether they subsequently studied knowledge maps or text.) Following this introductory phase, the participants studied the information on the autonomic nervous system for 30 min according to their randomlyassigned condition. Twenty-two of the students studied from knowledge maps and 21 studied from traditional text. Before the students began studying, the experimenter emphasized the importance of studying “conscientiously” during the time allotted. Participants were told to study as if they were “studying for a final exam.” Informal observation by the experimenter indicated that students followed these directions. After studying, students completed the Map Planning Test (Ekstrom, French, & Harman, 1976). Following this they completed three subjective graphs on which they rated their concentration, motivation, and anxiety. During the second session of the experiment, which took place 2 days later, students began by completing a free recall test on the materials. They were allowed 15 min to complete the test. Students were asked to recall all of the information they could on the autonomic nervous system materials they studied the first day in whatever order and whatever form they chose. Following this, students completed the Delta Reading Vocabulary test (Deignan, 1973) and then completed the same three

JOBNAME: Vol 21#1 PAGE: 4 SESS: 63 OUTPUT: Mon Apr 29 10:53:29 1996 /xypage/worksmart/tsp000/66511f/3

KNOWLEDGE MAPS

97

subjective graphs as the first day with the X axis representing the activities of the second day of the experiment.

RESULTS Scoring of the Dependent Variables Subjective graphs. Scoring of the subjective graphs began by converting students’ graphs to data points. For each graph students received a series of scores (from 1 to 10) representing the points where their graphs passed over the vertical lines representing time periods. Free recall. Scoring of the free-recall tests was based on a procedure developed by Meyer (1975) and Holley, Dansereau, McDonald, Garland, and Collins (1979). The recall key was constructed by dividing the original material into an inclusive set of propositions. Each proposition contained one fact, stated in the form of a simple declarative sentence. An experienced scorer matched these propositions with the set of propositions contained in a participant’s free-recall test. The participant received from one to four points, depending on the accuracy of the match. It is important to note that the accuracy score is not based on the exact wording of a students recall, but the ideas expressed. For example, one of the propositions on the scoring key taken from the information studied reads: “Something frightening happens in the environment.” If a student’s recall contained the sentence, “A scary thing happens around you,” the student would receive a 4 for that proposition. The sum of these scores constituted participants’ recall scores. Reliability was established by having a second experienced rater score a randomly selected subset of ten recall tests for both passages. A reliability coefficient of r 4 .97 was achieved. Recall scores were converted to percentage of possible recall. That is, a recall score of 100% would indicate that a student had accurately recalled every proposition of a given type on the recall key. Recall scores were further divided into superordinate and subordinate recall. Propositions that were associated with the main structure of the knowledge maps were summed to create a superordinate score for each recall, and the remaining propositions were summed to create subordinate recall scores. Superordinate propositions are those that provide information about the general framework of the domain; for example, “Nerves travel out of the spinal cord.” Subordinate propositions provide more specific information with respect to the framework; for example, “Nerves that leave the spinal cord are called ‘preganglionic.’ ” Group Comparisons Subjective graphs. Analysis of the subjective graphs began with a profile analysis (Tabachnick & Fidell, 1989) to assess the degree of heterogeneity of data points across studying and across testing between the two groups for each graph. That is, the analysis tested the assumption that the two groups differed in the “profile” or pattern of data points for each session. In all cases the two groups

JOBNAME: Vol 21#1 PAGE: 5 SESS: 63 OUTPUT: Mon Apr 29 10:53:29 1996 /xypage/worksmart/tsp000/66511f/3

98

HALL AND O’DONNELL

did not significantly differ. Therefore the data points were collapsed to create mean scores for both studying and testing. A series of three, two-way analyses of variance were then computed with Format (map vs text) as a between-subject factor and stage (studying vs testing) as a within-subject factor. In the first, motivation, analysis of variance, a significant main effect was found for format F(1,41) 4 22.97, p < .0001. Those in the map groups (M 4 6.04) reported significantly higher levels of motivation than those in the text groups (M 4 3.96). In addition, a main effect for stage was found F(1,41) 4 4.23, p < .05, with students reporting higher levels of motivation while studying (M 4 5.30) than while testing (M 4 4.76). A significant interaction was not found. The cell statistics for studying and testing respectively were: (M 4 6.30, SD 4 1.69 and M 4 5.80, SD 4 1.56) for those in the map group and (M 4 4.27, SD 4 1.91 and M 4 3.66 and SD 4 1.45) for those in the text group. A significant main effect was also found for Format in the second, concentration, ANOVA F(1,41) 4 5.74, p < .05, with those in the map groups (M 4 6.70) reporting higher levels of concentration than those in the text groups (M 4 5.49). As with the motivation ANOVA, a significant main effect was also found for Stage F(5.89), p < .05, with reported concentration dropping significantly from studying (M 4 6.59) to testing (M 4 5.63). The cell statistics for studying and testing respectively were: (M 4 7.20, SD 4 2.86 and M 4 6.20, SD 4 1.73) for those in the map group and (M 4 5.95, SD 4 2.00 and M 4 5.03 and SD 4 1.55) for those in the text group. No significant effects were found in the ANOVA in which subjective anxiety served as the dependent variable. Free recall. Before conducting the analysis, correlations of Map Planning and Delta scores with recall were computed. Delta proved to be a significant predictor of recall but Map Planning did not. Tests of homogeneity of regression slopes for the recall/Delta relationship indicated that Delta did not interact with experimental group in predicting recall. For these reasons, Delta Reading Vocabulary scores were used as a covariate in the recall analysis. A two way mixed analysis of covariance was then performed with Format (map vs text) serving as a between subject factor and Level (superordinate vs subordinate propositions) serving as a within subject factor. A significant main effect was found for Format F(1,40) 4 4.47, p < .05 with those in the map group (adjusted M 4 17) scoring significantly higher than those in the text group (adjusted M 4 11). In addition, a significant main effect was found for level F(1,41) 4 29.48, p < .0001, with students recalling significantly more superordinate (adjusted M 4 18) than subordinate (adjusted M 4 10) propositions. No other effects were significant. The adjusted means and standard deviations for superordinate and subordinate recall respectively were: (M 4 21, SD 4 15 and M 4 13, SD 4 8) for those in the map group and (M 4 15, SD 4 12 and M 4 7 and SD 4 5) for those in the text group.

JOBNAME: Vol 21#1 PAGE: 6 SESS: 63 OUTPUT: Mon Apr 29 10:53:29 1996 /xypage/worksmart/tsp000/66511f/3

KNOWLEDGE MAPS

99

Although the percentages recalled appear small, they are not considering the nature of the recall task. In order for a student to recall 100% he or she would have to recall accurately all of the propositions included in a 2500-word text passage that they studied for 30 min 2 days ago. These scores are consistent with previous research that has used the same type of free recall task and propositional scoring (e.g., Hall et al., 1992; Hall & Sidio-Hall, 1994a; Lambiotte & Dansereau, 1992). Relationship between outcomes. The analysis of the relationship between cognitive and affective outcome consisted of a series of zero-order correlations. The three subjective graph scores associated with both studying and testing were correlated with both detail and main ideal recall. None of these correlations were significant at the .05 level. The correlations between motivation and superordinate and subordinate recall for studying and testing respectively were: r 4 .00, r 4 .08, r 4 .02, and r 4 .16. For concentration they were: r 4 .11, r 4 .21, r 4 .08, and r 4 .23. Last, for anxiety they were: r 4 .11, r 4 .17, r 4 −.06, and r 4 .02. DISCUSSION Affective Outcomes The analysis of subjective concentration indicated that concentration dropped significantly from studying to testing. This is not surprising and is probably due to experimental fatigue. More importantly, with respect to text format, the results mirrored the objective cognitive outcomes, in that students in the map groups reported significantly higher levels of concentration across both studying and testing. This may be due to additional effort that is required to assimilate the structural information presented in the maps, and/or it may be a function of increased levels of motivation as indicated by other subjective ratings. This increase in concentration may then transfer to subsequent testing. With respect to subjective ratings of motivation, as with concentration, a significant drop occurred from studying to testing, again, probably due to experimental fatigue. With regard to the experimental question, a significant effect for Format was found with those in the knowledge map groups reporting significantly higher levels of motivation across both studying and testing. This demonstrates that the positive impact of knowledge maps goes beyond cognitive outcomes, and affects student motivation levels as well. Although students showed higher levels of motivation and concentration when studying from knowledge maps, the groups did not differ significantly as a function of perceived anxiety. This may be due to an increase in arousal/anxiety associated with exposure to the novelty of the knowledge map format. This increase in anxiety may then be partially alleviated by the positive affect generated by the maps, as represented by heightened levels of motivation and perceived concentration. This effect in turn may carry over from the study to testing situation.

JOBNAME: Vol 21#1 PAGE: 7 SESS: 63 OUTPUT: Mon Apr 29 10:53:29 1996 /xypage/worksmart/tsp000/66511f/3

100

HALL AND O’DONNELL

Cognitive Outcomes With respect to the first experimental question, the results are consistent with past research cited in the introduction, in that those in the maps groups significantly outscored their counterparts, who studied traditional text, on the recall of that information. This effect is due presumably to a number of factors. The spatial nature of the maps simplify complex concepts that would normally require difficult “metadiscourse skills” (Lambiotte & Dansereau, 1992). Further the structural information provides a scaffolding or structural framework which aids in the organization of ideas for later recall (Dansereau, 1989). The maps also appear to allow for a more flexible route in processing (Hall et al., 1992; Lambiotte & Dansereau, 1992). The lack of and format × level interaction indicated that the maps were more effective across both superordinate and subordinate propositions. While Dansereau’s model (Dansereau, 1989; Lambiotte et al., 1989), discussed in the introduction, is partly based on the notion that the structural properties of the maps emphasize the superordinate structure of the concepts to be learned, this does not imply that the maps will not also aid in acquisition of subordinate information as well. In fact, the accessible macrostructure may make it easier to pick up that information. Relationship between Cognitive and Affective Outcomes The present results indicated that, while knowledge maps tended to enhance subjective motivation/concentration and recall, the subjective and objective outcomes were independent. The correlational analyses indicated that there was no significant relationship between recall and the subjective graph measures. While this finding seems counterintuitive, it is not unprecedented, as evidenced by research cited in the introduction. Moreover, the results must be interpreted with caution. The small number of subjects leads to significance tests for the correlations that have very low power. Also, the subjects were in two distinct groups and combining them for the purposes of correlation may not be appropriate. Whichever is the case, future research using a larger number of subjects is necessary before any strong conclusions can be drawn with regard to this relationship. Conclusions These results serve to replicate previous experiments that indicate that knowledge maps can be an effective tool for enhancing acquisition of text materials relative to a more traditional format. This experiment extended these results by demonstrating that this positive effect can occur in the retention of both superordinate and subordinate propositions. In addition, objective-cognitive outcomes are mirrored in students subjective ratings of concentration and motivation. It appears, then, the positive outcomes of the knowledge maps goes beyond just

JOBNAME: Vol 21#1 PAGE: 8 SESS: 63 OUTPUT: Mon Apr 29 10:53:29 1996 /xypage/worksmart/tsp000/66511f/3

KNOWLEDGE MAPS

101

objective-cognitive outcomes to include subjective and affective outcomes as well. REFERENCES BAKER & BROWN (1984). Metacognitive skills and reading. In P. D. Pearson, (ed.) Handbook of reading research (pp. 353–394). New York, Longman. CLARK, M. S., & FISKE, S. T. (1982). Affect and cognition. Hillsdale, NJ: Erlbaum. DANSEREAU, D. F. (1989, March). Knowledge maps: An analysis of spatial verbal processing. Paper presented at the annual meeting of the American Educational Research Association, San Francisco, CA. DEAN, R. S., & KULHAVY, R. W. (1981). Influence of spatial organization in prose learning. Journal of Educational Psychology, 73, 57–64. DEIGNAN, G. M. (1973). The delta reading vocabulary test. Air Force Human Resources Laboratory, Lowry Air Force Base, CO. EKSTROM, R. B., FRENCH, J. W., & HARMAN, H. H. (1976). Manual for kit of factor referenced cognitive tests. Princeton, NJ: Educational Testing Service. GROSS, T. F. (1990). General test and state anxiety in real examinations: State is not test anxiety. Educational Research Quarterly, 14, 11–20. HALL, R. H., DANSEREAU, D. F., & SKAGGS, L. P. (1992). Knowledge Maps and the Presentation of Related Information Domains. Journal of Experimental Education, 61, 5–18. HALL, R. H., DANSEREAU, D. F., & O’DONNELL, A. M. (1989). Subjective graphing of metacognitive, affective and social processing: A psychometric analysis. Journal of Experimental Education, 57, 271–284. HALL, R. H., DANSEREAU, D. F., O’DONNELL, A. M., & SKAGGS, L. P. (1989). The effect of textual errors on dyadic and individual learning. Journal of Reading Behavior, 21, 127–139. HALL, R. H., & SIDIO-HALL, M. A. (1994a). The effect of color enhancement on knowledge map processing. Journal of Experimental Education, 62, 209–217. HALL, R. H., & SIDIO-HALL, M. A. (1994b). The effect of student color coding of knowledge maps and test anxiety on student learning. Journal of Experimental Education, 62, 291–302. HOLLEY, C. D., DANSEREAU, D. F., MCDONALD, B. A., GARLAND, J. C., & COLLINS, K. W. (1979). Evaluation of a hierarchical mapping technique as an aid to prose processing. Contemporary Educational Psychology, 4, 227–237. KULHAVY, R. W., LEE, J. B., & CATERINO, L. C. (1985). Conjoint retention of maps and related discourse. Contemporary Educational Psychology, 10, 28–37. LAMBIOTTE, J. G., & DANSEREAU, D. F. (1992). Effects of knowledge maps and prior knowledge on recall of science lecture content. Journal of Experimental Education, 60, 189–201. LAMBIOTTE, J. G., DANSEREAU, D. F., CROSS, D. R., & REYNOLDS, S. B. (1989). Multirelational semantic maps. Educational Psychology Review, 1, 331–367. MEYER, B. J. F. (1975). The organization of prose and its effects on memory. Amsterdam: NorthHolland. O’DONNELL, A. M., DANSEREAU, D. F., HYTHECKER, V. I., HALL, R. H., SKAGGS, L. P., LAMBIOTTE, J. G., & YOUNG, M. D. (1988). Cooperative procedural learning: The effects of prompting and pre-versus distributed planning activities. Journal of Educational Psychology, 80, 161–171. O’DONNELL, A. M., DANSEREAU, D. F., HALL, R. H., & ROCKLIN, T. R. (1987). Cognitive, social/ affective, and metacognitive outcomes of cooperative learning. Journal of Educational Psychology, 79, 431–437. PATTERSON, M. E., DANSEREAU, D. F., & NEWBORN, D. (1992). Effects of communication aids and strategies on cooperative teaching. Journal of Educational Psychology, 84(4), 453–461. TABACHNICK, B. G., & FIDELL, L. S. (1989). Using multivariate statistics (2nd ed). Harper and Row, New York.