Chapter 10 Reference Maps as a Framework for Remembering Text

Comprehension of Graphics W. Schnotz and R. W. Kulhavy (Editors) e 1994 Elsevier Science B.V. All rights reserved.

Chapter 10

Reference Maps as a Framework for Remembering Text Raymond W. Kulhavy & William A. Stock Arizona State University, Tempe, USA

Linda C. Caterino Walker Research Institute, Tempe, USA

ABSTRACT This paper develops a theory based in dual coding, that explains why learning a map improves memory for associated text information. The basic premise is that visual displays such as maps are encoded as intact images which retain the spatial properties of the objective stimulus. Such images possess a computational advantage in working memory because the information within them is simultaneousy available to cue retrieval of related text. The paper describes eight experiments all of which support both the image coding and working memory assumptions. There is evidence that reference maps improve memory for related text content (Dean & Kulhavy, 1981). Abel and Kulhavy (1986) suggested that maps improve memory for text by serving either an instantiation or mnemonic function. Maps perform an instantiation function when they help learners assign unique meaning to portions of a related text. This position derives from a schema-theoretic point of view, where maps "activate" a relevant schema that can be used to interpret a text (Anderson & Pearson, 1984; Royer & Cable, 1976). Schema explanations are based on the premise that instantiating stimuli (here maps) can be directly related to the prior knowledge base of the

154

R.M. Kulhavy, W.A. Stock & LC. Caterino

learner, and that the activation of prior knowledge facilitates learning of associated stimuli (here text). On the surface, instantiation is an attractive candidate for explaining map-text facilitation. However, an inspection of relevant research is at odds with this position. Virtually all of the research has used maps that are fictitious, or unfamiliar to the subject population. Such maps have doubtful value in terms of activating prior knowledge. Further, the texts used in such research are typically concrete, easily understandable, and normed to insure comprehensibility. These conditions are distinctly different from those in research where pictures are used to instantiate convoluted prose [see Bransford and Johnson (1972) for the classic example]. Hence, we conclude that the instantiation explanation does not apply to the research available with maps and text. The second possible explanation advanced by Abel and Kulhavy (1986) is that maps serve a mnemonic function for text recall. In this case, map information is used to directly cue retrieval of associated text events. There is no requirement that the map serve any purpose other than furnishing information that facilitates text memory. Most map-text research appears to fall within the mnemonic category, and we have framed the remainder of our discussion in terms of this function. Three reliable outcomes appear in map-text research. First, maps increase text recall only when the maps are presented as organized stimulus units. When map features are presented as lists, or in a fragmented manner, memory for associated text decreases (e.g., Reynolds, 1968; Schwartz & Kulhavy, 1981). Second, maps improve memory for text only when learners expend considerable effort to encode the map as an organized or intact unit. Virtually every study demonstrating map-text facilitation has used specific techniques to insure that the map is learned as an organized entity (Dean & Kulhavy, 1981). Third, maps increase text recall only when the information in the text is directly related to map content. When text information is not specifi cally associated with features on the map, recall is about the same for map-present and map-absent groups (e.g., Mastropieri & Peters, 1987). Our explanation of map-text facilitation is based on a version of dual coding theory (e.g., Paivio, 1986). In this case, maps are coded as images in a non-verbal memory store, while text is coded sequentially as propositions into verbal memory. Also, there exist associative connections between non-verbal and verbal stores, so that information

Reference Maps as a Frameworkfor Remembering Text

155

in one code can be used to activate information in the other code. Maps improve text memory simply because the information in the map image can be used to cue retrieval of related events in the verbal store (Kulhavy, Lee, & Caterino, 1985). Further, we believe that maps can be encoded as intact representations in memory. Intact map images retain the spatial characteristics of the objective stimuli, and the information within them becomes simultaneously available when they are brought into working memory. Such images have an advantage as far as the cuing of text retrieval is concerned. The advantage derives from the fact that attention can be shifted from location to location across the map image, without exceeding the limits of working memory. This same computational advantage has been discussed by Larkin and Simon (1987) with respect to how people use the information in diagrams for problem solving. Hence, the more intact the representation of the map, the more likely that information in it can be located efficiently and used for retrieving associated text. If maps facilitate text recall in the manner described above, then an important topic for research is to determine what attributes of maps themselves are responsible for improved text memory. It is our position that maps present two types of information. First, they contain information about individual features, including landmark and point markers such as labels, drawings and topographic symbols. These features may be further differentiated by the addition of what Bertin (1983) calls "retinal variables", such as shape, size and color. We group all visual characteristics of individual map features under the heading "feature information". The second type of information on a map is the spatial relations that exist among features. We group such relations under the heading of "structural information". Structural information denotes the network of direction and distance relations that exist among individual features within a map space, and boundary designations that can be used as reference points for features.

It is an empirical question whether feature or structural information, or both, are responsible for improving text memory. For example, consider the general elaboration position that has developed from research on pictures and prose (e.g., Bradshaw & Anderson, 1982; Willows and Houghton, 1987). When individual visual stimuli, such as pictures or drawings, are studied in association with words, sentences or segments of text, recall of the verbal material is generally facilitated. Elaboration explanations hold that processing the two types of stimuli together increases their accessibility in memory - in other words, "two codes are better than one". When elaboration explanations are applied to

156

R.M. Kulhavy, W.A. Stock & LiC. Caterino

map-text research, the focus is on the effects of individual feature information. That is, each feature provides an elaborative context for the specific portion of the text passage with which it is associated. This explanation handily accounts for the consistent result that text event recall increases only when events are directly related to specific map features. However, note that this position does not predict a difference between situations where features are presented alone and where they are distributed in a complex spatial display - such as a map. In fact, there is little in the elaboration explanation to suggest that structural information in maps has any effect on the recall of text. We have explored the implications of such a position in a recent paper (Kulhavy, Stock, Woodard & Haygood, in press). The dual coding position outlined above emphasizes the structural information that appears in maps. We assume that maps are encoded as intact entities in which structural information provides a framework for individual features. Because structure is included in the image, feature information becomes simultaneously available during working memory operations at retrieval. Such reasoning leads to the straightforward prediction that structural information will improve memory for associated text. This explanation accounts for the remaining two consistent outcomes described earlier. First, encoding structural relations probably requires considerable effort on the part of the learner, which accounts for the attention control procedures previously mentioned. Second, when maps are presented as organized entities, people are able to encode the structural relations (e.g., Schwartz & Kulhavy, 1981). Under these conditions, map images can be scanned efficiently, and structural information used to improve retrieval of text. We have conducted a series of experiments designed to test the assumptions discussed above. The remainder of this paper is devoted to a discussion of this evidence.

Feature attributes and structure In three separate experiments we tested the relative contribution of feature and structural information to text recall (Kulhavy, Stock, Woodard, & Haygood, in press; Kulhavy & Stock, 1992). All three studies used undergraduate volunteers, a map of an Italian town, and a text of about 600-words that contained events associated with features on the map. In Experiments I and 2 features were presented as labels on

Reference Maps as a Frameworkfor Remembering Text

157

the map, or as labels-plus-icons, where the icons were line drawings of the feature in question (e.g., temple). In Experiment I, structure was defined as the number of times subjects placed map features in the correct location prior to hearing the text. In Experiment 2, structure was calculated using a goodness of fit statistic based on the accuracy with which feature interrelations were reproduced during recall of the text. Presenting feature information at retrieval (labels or icons) had no effect on recall of text. In both experiments, memory for map structure was a significant predictor of associated text recall. Embellishing individual features with icons had no effect on text memory in Experiment I, but did produce facilitation in Experiment 2, where icon drawings were treated as a within-subjects variable. In Experiment 3, undergraduates again saw the Italian town map and heard the text (Werner-Bellman, Klein, Brooks, Kulhavy & Stock 1992). In this study all features were represented by both labels and icon drawings, but half of the icons in each group were colored and half were not colored. Subjects received instruction either to pay attention to properties of individual features, or to encode the map as an intact image. Structural encoding was measured by the goodness of fit index based on the reconstructed feature interrelations. Again, goodness of structural encoding directly predicted memory for text events. Neither icon color nor encoding instructions were related to text recall in an important manner. Proportion event recall for the low and high structure groups for each of the three experiments is depicted in Figure 1.

Map structure and retrieval In a two experiment series, we investigated the effects of disrupting the organization structure of the map itself during text retrieval (Kulhavy, Stock, Peterson, Pridemore, & Klein, 1992). Undergraduates studied a city map while hearing a text containing both feature names and related events. In the first study, subjects retrieved text events while viewing a second map that, (a) was a duplicate of the original (intact map), (b) had the features rearranged within the map space (changed map), or (c) displayed only the land border of the original map with no feature information (border only). Event retrieval was highest for the intact map group, with the groups ranked: intact map > border-only > changed map. In the second experiment, subjects again studied the map

158

H.M. Kulhavy, W.A. Stock & L.C. Caterino

Dlfferences In esch Experlment p

.06

Figure 1. Mean proportion of text events reculled as a function of low and high encoded map structure for each of three experiments. and text, and received either the intact map or individual features as retrieval cues. The intact map yielded significantly higher text recall. Because we used a retrieval methodology in the two experiments just described, several colleagues suggested that the results could be best explained by the Principle of Encoding Specificity (Tulving & Thomson, 1972). To test this possibility, we conducted a third experiment in which subjects learned either an intact map or a page of map features, with the same stimuli factorially varied at retrieval (Brooks, Werner-Bellman, Peterson, Stock, & Kulhavy, 1992). For text fact recall, the encoding-retrieval conditions ranked themselves: Intact Map-Intact Map > Intact Map-Feature Page = Feature Page-Intact Map > Feature Page-Feature Page (p < .05). These data are difficult for encoding specificity to explain, since the group seeing the page of features at b6th encoding and retrieval should have out performed the different stimulus conditions. We conclude that a strong version of encoding specificity does not account for the retrieval differences in the initial two studies.

Reference Maps as a Frameworkfor Remembering Text

159

Stimulus order effects We have just completed two additional unpublished experiments in which we varied the order in which the map and text were studied. We reasoned that order of presentation should influence how effectively subjects can form associations between map and text information. Our assumptions regarding working memory activities suggest an asymmetry in recall, depending on which stimulus is learned first. We have already discussed the events that occur when maps are presented before text, encoded as intact images, and used to cue associated text recall. However, when the text is presented before the map and encoded as a sequence of propositions the demands on working memory are different. When the map is presented second, subjects retrieve text propositions into memory in an attempt to associate them with the information on the map (especially when they have been specifically instructed to use the first stimulus to help learn the second). However, keeping propositions in working memory requires considerable resources, and subjects are faced with the choice of using their processing capacity to, (a) learn the map, or (b) make as many map-text associations as possible. Whatever course subjects take, the number of possible map-text associations is reduced, simply because the computational advantage of the intact map representation is lost. This argument leads us to predict that subjects seeing the map first will recall more text information that those seeing it after the text. In both experiments the maps and text were ecologically valid (Ceylon and the city of Rome) and instructions given to both map-text and textmap subjects were identical. In the first study the subjects recalled about 30% more associated facts from the text when the map was the stimulus first seen. The map-first condition also remembered more map features, and the joint probability of recalling a text fact, given that the related feature was correctly located on a map reconstruction, was significantly higher. In the second experiment we crossed stimulus order with intact versus individual feature presentation. The results were identical for stimulus order, with the map-first group remembering more text facts and map features, and producing higher values of P(text fact I correctly located feature). In addition, the intact map condition outperformed the group seeing individual features across each of the same independent measures.

160

R.M. Kulhavy, W.A. Stock & 1~C. Caterino

Conclusions Taken together, the results of eight experiments indicate that structural information from a map is important for the recall of associated text. These results support the dual coding model developed earlier, and suggest that the organizational properties of the map image are important for cuing retrieval of related information in the verbal store. Encoding the structural information present in complex visual displays, such as maps, increases the efficiency of working memory operations, and allows access to more map image information that can be used to cue retrieval of associated text.

RESUME

Cet article developpe une theorie fondee sur le double codage qui explique pourquoi l'etude d'une carte ameliore La retention des informations textuelies qui y sont associees. I/hypothese de base est que les representations visuelies telies que les cartes sont encodees sous forme d'images intactes qui conservent les proprietes spatiales du stimulus. Ces images ont un avantage computationnel en memoire de travail car les informations qu'elles contiennent sont simultanement disponibles pour indicer La recuperation du texte associe. L'article decrit huit experiences qui toutes sont en faveur a La fois des hypotheses de codage image et de memoire de travail.

Reference Maps as a Frameworkfor Remembering Text

161

References Abel, RR., & Kulhavy, R.W. (1986). Maps, mode of text presentation, and children's prose learning. American Educational Research Journal, 23, 263-274. Anderson, RC., & Pearson, P.O. (1984). A schema-theoretic view of basic processes in reading comprehension. In P. D. Pearson (Ed.), Handbook of Reading Research (pp. 255-291). New York: Longman. Bertin, J. (1983). Semiology of graphics: Diagrams, networks, maps. Madison, WI: University of Wisconsin Press (Translated by William J. Berg). Bradshaw, a.L., & Anderson, J.R (1982). Elaborative encoding as an explanation of levels of processing. Journal of Verbal Learning and Verbal Behavior, 21, 165-174. Bransford, lD., & Johnson, M.K. (1972). Contextual prerequisites for understanding: Some investigations of comprehension and recall. Journal of Verbal Learning and Verbal Behavior, 11, 717-726. Brooks, RB., Werner-Bellman, L., Peterson, S.E., Stock, W.A., & Kulhavy, RW. (April, 1992). Retrieving text events associated with maps: Examining the encoding specificity hypothesis. Paper at American Educational Research Association, San Francisco, California. Dean, RS., & Kulhavy, RW. (1981). Influence of spatial organization in prose learning. Journal of Educational Psychology, 73,57-64. Kulhavy, R.W., Lee, J.B., & Caterino, L.c. (1985). Conjoint retention of maps and related discourse. Contemporary Educational Psychology, 10, 28-37. Kulhavy, RW., & Stock, W.A. (April, 1992). Feature and structural information in the recall of text. Research Colloquium, Victoria University of Wellington, New Zealand. Kulhavy, R.W., Stock, W.A., Peterson, S.E., Pridemore, D.R, & Klein, J.D. (1992). Using maps to retrieve text: A test of conjoint retention. Contemporary Educational Psychology, 17,56-70. Kulhavy, R.W., Stock, W.A., Woodard, K.A., & Haygood, RC. (in press). Comparing elaboration and dual coding theories: The case of maps and text. American Journal of Psychology, 106, 483-498. Larkin, J.H., & Simon, H.A. (1987). Why a diagram is (sometimes) worth ten thousand words. Cognitive Science, 11, 65-99. Mastropieri, M.A., & Peters, E.E. (1987). Increasing prose recall of learning disabled and reading disabled students via spatial organizers. Journal of Educational Research, 80, 272-276.

162

R.M. Kulhavy, W.A. Stock &

i.c.

Caterino

Paivio, A. (1986). Mental representations: A dual coding approach. New York: Oxford University Press. Reynolds, J.H. (1968). Cognitive transfer in verbal learning: Transfer effects after prefamiliarization with integrated versus partially integrated verbal-perceptual structures. Journal of Educational Psychology, 59, 133-138. Royer, J.M. & Cable, G.W. (1976). Illustrations, analogies, and facilitative transfer in prose learning. Journal of Educational Psychology, 68, 205-209. Schwartz, N.H., & Kulhavy, R.W. (1981). Map features and the recall of discourse. Contemporary Educational Psychology, 6, 151-158. Tu1ving, E. & Thomson, n.M. (1973). Encoding specificity and retrieval processes in episodic memory. Psychological Review, 80, 352-373. Werner-Bellman, L., Klein, J.D., Brooks, R.B., Kulhavy, R.W., & Stock, W.A. (June, 1992). Remembering maps and text: Effects of color and organization. Paper at American Psychological Society, San Diego, California. Willows, D.M., & Houghton, H.A. (1987). The psychology of illustration (Vol. 1). New York: Springer-Verlag.