- Email: [email protected]
Images and Semantic Representations
LANGUAGE AND COMPREHENSION J.·F. Le NV. W. Kintseh (editors) © North·Holiand Publishing Company. 1982
17
IMAGES AND SEMANTIC REPRESENTATIONS Michel Denis Laboratoire de Psycho10gie Universite de Paris VIII Saint-Denis, France
In this chapter are summarized and discussed selected data from a research program on the relationships between imagery and semantic processing. Within the framework of a featura1 approach to conceptual representations, imagery is viewed as a constructive process that selectively applies to, and transiently actualizes, specific subsets of semantic features, the so-called figurative features. INTRODUCTI ON The last twelve years of research in psychology have been characterized by a number of theoretical and empirical efforts with the aim of giving mental imagery - a long-standing theme in our discipline - a renewed theoretical status. Research has simultaneously developed in two main directions. On the one hand, imagery was investigated as an independent variable, i.e., as a process intervening or likely to intervene in the course of many psychological activities, such as memorization, comprehension of sentences, problem solving, etc. As a general conclusion, it may be stated that the use of imagery has positive effects on most of these activities, whatever the indicators of performance. However this kind of approach is unlikely to throw much light on what an image is and how the intrinsic properties of this psychological event may account for its effimncy. Thus, what can be named the "second generation" of contemporaneous research in imagery is more concerned with questions such as : What are the characteristics of images, in contrast with other ways people evoke concepts? What is the structure of this psychological event? To what extent shall we consider it as an "analogical" mode of representation? Must we consider that images have genuine spatial properties? How shall we consider their temporal characteristics? Whatever the researchers' specific focus as regards imagery, it is obvious that research in this field is fundamentally concerned with the question of the relationships between images and knowledge. No doubt that images are a privileged modality for actualization of some aspects of knowledge, but a number of other aspects are unable to be figuratively represented. In any case, a persistent feature in the last ten or twelve years of psychology will have been the feeble effort, or the lack of a widespread effort, to systematically connect the study of imagery to the wider field of representation of knowledge. During a long time, research in imagery developed without any concern of converging with the models of semantic memory that attempted to describe the structure of knowledge and its organization in long term memory. We are still greatly unaware of the relationships between, on the one hand, activation of semantic structures leading to the construction of meaning in the mind of a reader who has to verify a statement (such as : A canary has wings ... ) and, on the other hand, the imagery elicited by this statement. These questions in fact were expressed in several empirical approaches with various theoretical backgrounds, but with the common concern of deliberately introducing imagery in the study of semantic activities - contrary to what was the case in several former, leading models (cf. Collins and Quillian, 1969 ; Smith, Shoben, and Rips, 1974). One widespread empirical approach was sentence
18
M. Denis
verification, in an attempt to measure verification latencies of sentences varying in their ability to elicit visual images that represent their content. Results initially presented by Jorgensen and Kintsch (1973), and later by Holmes and langford (1976), attested a facilitating effect of the imagery value of sentences on their verification. However such a relationship is difficult to interpret, mainly because of the difficulty in distinguishing the effects of imagery from the effects of the semantic complexity of sentences (cf. Holyoak, 1974). More recent experiments by Glass, Eddy and Schwanenflugel (1980) demonstrated that the reading of descriptive sentences may in fact be accompanied with an imagery activity, objectively detected by appropriate experimental procedures, but without allowing the researcher to ascribe to these images a decisive role in verification processes per se. Such a conclusion is itself in agreement with hypotheses concerning the abstract, amodal format of representation of sentence meaning (cf. Potter, Valian,and Faulconer, 1977). An other procedure, in some respects similar to the former one, was equally used for examining the possible role of imagery in sentence verification - the mental comparison technique. In these tasks, the subject has to evaluate as true or false sentences asserting a relation, such as : A lobster is larger than a crayfish. In opposition to Paivio's (1975, 1978) arguments favoring the explanatory role of visual imagery in the evaluation of such sentences, critical arguments, such as Friedman's (1978), were developed in favor of different forms of mental representation, other than "analogical". The most recent trends seem to favor mixed models, postulating parallel processing of analogical representations, such as visual images, and propositional representations (cf. Kosslyn, 1980 ; Kosslyn, Murphy, 8emesderfer, and Feinstein, 1977). lastly, a paradigm that looks relevant to the field of research discussed above is the so-called sentence-picture comparison paradigm, where a picture is to be compared by subjects with the semantic content of a descriptive sentence (such as: The plus is above the star ... ). After a long lasting controversy between proponents of "imaglstlc" lnterpretations (cf . Huttenlocher, 1968) and those of "propositional" accounts (cf. Carpenter and Just, 1975 ; Chase and Clark, 1972), this paradigm was more recently used in a new perspective, emphasizing strategies that differentiate individuals in such tasks (cf. Macleod, Hunt, and Mathews, 1978). Thus it is possible to identify subjects with clear preferences for "linguistic" strategies (they compare a linguistic description of the picture to the semantic representation of the sentence) and subjects inclined to "visual" strategies (they more readily compare the picture to the visual image of the pattern described in the sentence) (see also Beech, 1980 ; Dubois and Weil-Barais, 1980). The major idea stemming from these current research trends is that the role of radical imagistic positions in the field of language understanding has lessened and that presently more weight is given to propositional interpretations, calling upon abstract, amodal semantic representations, which are neither linguistic nor imaginal, but which have the open possibility of optional strategies making use of visual imagery. In this paper I briefly describe a series of steps which are part of a research program devoted to the question of the relationships between the domain of images - more specifically, images involved with linguistic activity, mainly comprehension of sentences - and the domain of semantics, by an approach initially concerned with lexical units, then with sentences. FIGURATIVE SEMANTIC FEATURES The major working hypothesis here is that there is no definite separation between the conceptual approaches to imaginal representations and to semantic representations. More precisely, the idea developed here is that imagery is a modality for the actualization of semantic representations of sentences, through a highly
Images andSemantic Representations
19
specific format of representation, mainly characterized by its structural analogy with perceptual events. Semantic representations can be analyzed into components, some of which may be actualized under a specific format and result in the elaboration of cognitive experiences"mimicking perceptual experiences to some extent (cf. Denis, 1979). The approach illustrated here relies upon the hypothesis that the format of the most elementary units of cognitive representations is lower than the lexical format. A full examination of the arguments supporting this claim, or rather the family of theories having this claim in common, is impossible here. Let us however concentrate on the idea that concepts to which lexical items are attached can be analyzed into more elementary units of meaning, or semantic features. This statement does not only mean that humans (among them, psychologists) are capable of analyzing a concept into smaller pieces of meaning, but that there exist arguments in support of the psychological validity of the notion of semantic features as elementary components of cognitive representations (e.g., Eysenck, 1979 ; Le Ny, 1979 ; Miller and Johnson-Laird, 1976). For instance, let us consider the meaning attached to the lexical unit knife. The concept to which the word knife refers may be analyzed into a set of features among which we would proba~ind : BLADE, HANDLE, LENGTHENED SHAPE, CUTS, USEFUL, DANGEROUS, etc. Two general characteristics of such feature sets should be underlined: (a) their hierarchical organization, which reflects the unequal contribution of these features to the concept; (b) their flexibility, i.e., the possibility of transient reorganizations of the hierarchy, depending on factors such as linguistic context, individual's cognitive orientation, etc. However, my paper will treat essentially the major characteristic of meaning according to this interpretation, that is : Meaning elicited by a word is a cognitive construct which results from the composition of a set of semantic features. Among the set of features that define the concept KNIFE, some features are related to the physical properties of knives, i.e., properties that have been perceptually experienced as such by the individual. This is the case of features such as BLADE, HANDLE, etc. These units are presumably involved in some privileged way not only in visual imagery elicited by the word knife, but also in any activity aiming at depicting the figural appearance of a k~ as in a pictorial representation. For this reason, I call such units of meaning "figurative semantic features", in contrast with other kinds of features that do not directly reflect the fiqural properties of objects. Such is probably the case for USEFUL and for DANGEROUS. We certainly are able to construct visual images that reflect through some kind of associative imagery the usefulness or the danger of a knife. However it is obvious that such an imagery is not as "direct" as when the features BLADE or HANDLE are used to evoke the concept KNIFE. That is why it seems more appropriate to qualify features as to the probability with which imagery processes may apply to them. In this manner it would be posslble to isolate from among the set of features defining a concept a subset of features to which imagery processes may apply with maximal probability. These are the features I shall call in abbreviated form "figurative (semantic) features". In my interpretation, imagery is to be viewed as a constructive process that gives a specific, limited subset of features a transient cognitive actuality, through a specific type of mental experience (mainly characterized by its structural analogy with perceptual experience). IMAGERY VALUE AND SEMANTIC COMPOSITION Several techniques have been developed in order to "measure" imagery elicited by linguistic inputs. One of them relies on the notion of "imagery value", to which research gave an empirical status at the time of the renewal of psychologists' interest for imagery (see Paivio, 1971). Imagery value of a word is defined as the ability of this word to elicit an imaginal representation in the individual, that is a cognitive event with a figurative content. It is operationally defined by the
20
M. Denis
mean value given to this word by a group of subjects on a rating scale, in most cases a seven-point scale, whose extreme points for instance are labeled, on the one hand, "no image" and, on the other hand, "highly vivid image", When subjects rate a sample of words on such a scale, not only is this task accepted as meaningful, and words in general scatter over the whole range of the scale, but this indicator also shows a high degree of inter- and intraindividua1 consistency. Defined in this way, the imagery value of words has proved to be a significant factor in many kinds of processing of linguistic material: comprehension, verification, memorization, etc. Linguistic messages made with high-imagery words are better memorized than messages made with low-imagery words. However, the questions that were surprisingly absent from recent discussions are : What does imagery value measure? What does it reflect, especially regarding the structure of knowledge about corresponding objects? In other words, what happens in someone's mind when he or she constructs an image in order to evaluate its vividness, its clarity, or its richness? tentative answer is that imaging results from a process that applies to the subset of figurative features of a concept. Constructing images in response to words is identified as the actualization, in a specialized representational system, of figurative components of concepts, through a specialized process whose major characteristic is the ability to construct mental events with a structure and an internal organization that are analogical with those of perceptual events. This actualization of figurative features presumably takes place within a "framework", a kind of a Gestalt spatial scheme, which is the representation of the global shape of the object (cf. the notion of "skeletal encoding" in Koss1yn, 1980). Whatever this hypothetical framework, the very first step in my research program consisted in testing the main consequence of the assumption that links imagery and semantic structures, i.e., one should predict the imagery value of words from the richness of the corresponding concepts in the so-called figurative features. My
The first experiment whose objective was to examine this hypothesis investigated the imagery value of general and specific words, and its relation with the semantic composition of these words. The generality-specificity dimension is closely related to a dimension that one may call richness of concepts in figurative semantic features. If one considers a branch in a taxonomy whose ultimate term is a physical object - for instance, LIVING BEING - ANIMAL - BIRD - PARROT, - it is obvious that, as one moves down the taxonomy, not only do concepts grow richer in semantic features, but they also grow richer in semantic figurative features. A corollary of this statement is that the more specific the concepts, the easier they are to evoke the objects as images, whereas images for more general concepts should either be very poor or consist in instantiations (that is, images of subconcepts richer in figurative features). It has been demonstrated that general terms, such as furniture, clothing, etc. are less likely to elicit images than more specific terms, such as chair, pants, etc. But this is mainly apparent in tasks emphasizing the speed witn-wnrch lmages are to be constructed. As a general result, imagery latencies are longer to general terms than to specific terms. This difference also holds for drawing latencies to these terms. However, when one considers imagery values (that are given without any temporal constraints), there is in most cases very little difference between general and specific terms. A likely interpretation is that when subjects have to rate for imagery value a general term that is poor in figurative features, they tend to look for a specific exemplar in the category, and they finally give the experimenter the imagery value for the associated specific term, not the imagery value for the general term itself. Figure 1 is a schematic representation of four hypothetical levels in a taxonomy, from the highest, more general level (I) to the lowest, more specific level (IV).
Images andSemantic Representations
21
At each level the features defining the corresponding concept are listed as lower-case letters. According to such an approach, a concept at any level is composed of the set of features defining the superordinate concept, plus a specific set of features. For instance, BIRD would comprise all the features for ANIMAL, plus some new ones: WINGS, FEATHERS, etc. At the lower level, PARROT would comprise all the features for BIRD, plus some new, specific ones: MANY-COLOURED, REPEATS, etc. (cf. Le Ny, 1979). In this framework, specification is therefore mainly viewed as a semantic enrichment. (a, b, c) II {a, b, c} & {i, e,
n
III {a, b, c} & {i, e,
n
& {!I' ~,
IV {a, b, c) & {i, e, f l & {!I,
~,
il il & {,t, .~,
1)
Figure 1 Schematic representation for four hypothetical levels in a taxonomy In Figure 1, underlined letters correspond to features that are supposed to be figurative ones, i.e., features reflecting object properties directly accessed by perceptual processes (visual, in the examples discussed here). Thus, in most taxonomies, the higher level concept would essentially be composed of non-figurative features, corresponding to very general, abstract properties (a, b, c). The immediatly lower concept would be composed of the same set of features (a, b, c), plus a further set (d, e, f), in which feature d would reflect a physical property and would be likely to be figuratively represented, either in a visual image or in a drawing. My contention is that such figurative features become more and more numerous as one goes down from one level to a lower level in the hierarchy. It is likely that such an analysis will make the positive relationship between imagery value and specificity more obvious, not only because more specific concepts are composed of a greater number of features, but because they are composed of a greater relative number of figurative features in the whole set of features defining these concepts. (I shall insist again that it is for brevity's sake that I speak of "figurative" versus "non-figurative" features. It is more likely indeed that any semantic feature is characterized by the degree or the probability with which it can be represented in an image.) My initial effort consisted in demonstrating that (a) general concepts are composed of a lesser number of features than more specific ones, and (b) specific concepts are composed of a relatively greater number of figurative features (or, according to an alternate formulation, the mean "figurativeness" of features composing specific concepts is higher than for more general concepts). In this experiment, pairs of substantives were used, one general and one specific. (One should however keep in mind that there are no absolute "general" nor "specific" words. Generality or specificity of a word is always relative to that of another word. That is why the strictest formulation would be that, in pairs of substantives, one was more general, or less specific, than the other one.) Two kinds of general concepts were considered: general concepts A (such as TREE, BIRD, FISH, etc.) and general concepts B (such as VEHICLE, FURNITURE, VEGETABLE, etc.). General concepts A subsume members that have a great number of figural characteristics in common. For instance, there is a high degree of similarity between most birds as to their global shape, the parts that compose them,and the relations between these parts. Such concepts are close to "base level concepts" according to Rosch's terminology (cf. Rosch, Mervis, Gray, Johnson, and BoyesBraem, 1976), that is concepts at the most inclusive level where the various members have a great number of properties in common, elicit similar behavioral
22
M. Denis
programs, and possess highly similar figural characteristics. These concepts are close also to what Hoffmann (1982) defined as "primary concepts" in a similar, independent approach. General concepts 8, on the other hand, subsume members with a high degree of figural discriminability. For instance, when one considers the category of vehicles, the figural appearance of a truck is strongly distinct from that of a bicycle, or a helicopter, or a space rocket. A specific term was paired with each general term. It was a high associate, non ambiguous word, whose frequency was equated with the frequency of the general term. A first group of subjects were presented with the selected terms (different subjects were presented with the general or the specific term in a pair), and they gave lists of properties for the designated classes of objects. From these lists were derived lists of features hypothetically defining each of the general and specific concepts. Features were lexically expressed either by substantives (for instance, for the concept SNAKE: SCALES, TONGUE, VENOM), or by adjectives (LONG), or by verbs (CRAWLS). Only responses given by at least 30 per cent of the subjects were considered. This, for instance, is the list of features defining the concept FLOWER, presented in decreasing order of their frequency: PETALS, STEM, LEAVES, STAMENS, POLLEN, PISTIL, SMELL. It was found, as expected, that the mean number of features was lower for general than for specific concepts. However, when considering only general concepts A and their specific counterparts, the difference is hardly noticeable, whereas the difference is very clear in the case of general concepts B. The next step consisted in obtaining an indicator of the "figurativeness" of features. This was performed by asking a new group of subjects to rate all the features on a scale similar to the imagery value scale. Then it was possible to sum the obtained mean values for all the features composing a concept, this resulting in a "composite imagery value" for each of the general and specific terms. When one looks at the whole material, it appears that the composite imagery value is lower for general than for specific terms. As previously seen, there is a very small difference between general terms A and their specific counterparts, whereas between general terms B and their specific counterparts the difference is large and significant. The composite imagery value of general terms B is the lowest. These terms do then refer, as previously argued, to concepts whose features have a very low degree of "figurativeness". Moreover the contrast beween these terms and all the others is much more pronounced than was the contrast previously observed when "global imagery values" were considered. It thus appears that "composite imagery value" gives a better reflection of the "figurativeness" of the semantic components of concepts, and that this indicator is much less likely to be biased by instantiation processes than is the "global imagery value". One however has to examine the following argument: The additive character of the hypothesis illustrated by Figure 1 is such that it is just statistically likely that the composite imagery values increase when the number of features taken into account increases. The answer is that when one computes the mean imagery value for a single feature, this value is still the lowest for general concepts, and still lower for features composing general concepts B than for features composing any of the other concepts. It must be noted however that the range of this indicator is limited, due to the fact that a11 the consi dered concepts are, on the whole, on the "positive" si de of imagery values. They all subsume physical objects or classes of physical objects. That is why I turned to other materials, in the hope that they would produce clearer differentiations. The general purpose was the same, with the additional attempt to demonstrate the relationship between the number of features composing a concept and their "figurativeness". In other words, concepts rich in semantic features - as was the case at the lowest levels of the taxonomies, according to semantic the previous approach - would by this very fact be rich in fi~urative features, whereas concepts poorer in semantic features - for lnstance, at the top of taxonomies - would be poorer in figurative semantic features.
Images and Semantic Representations
23
Having looked for a corpus whose elements would widely scatter on the rating scales to be used and furthermore would not be systematically hierarchized along the generality-specificity dimension, I turned to terms designating occupations. This semantic field is very large, with a wide variety of elements dispersed over the whole range of the imagery value scale, from occupations with salient figurative attributes, such as fireman or judge, to occupations with none or almost none of these attributes, such as editor-or-manager. From a list of 155 terms designating occupations that had been ~for~) imagery value by a group of subjects, I selected four groups of items (A, B, C, D) from the lowest to the highest imagery value. Without describing here the details of the analyses - which followed the same steps as in the previous experiment, - the main results were straightforward. The mean number of features per concept is positively correlated with the imagery value of corresponding terms. In other words, the imagery value of a word is closely dependent on the richness of the corresponding concept in semantic features. Furthermore, concepts richer in semantic features are richer in figurative semantic features. Actually, not only the composite imagery value attached to concepts increases from items A to items 0, but the mean imagery value for a single feature itself increases from items A to items D. A significant, positive correlation was computed between the global imagery value of words and the latter indicator. The mean imagery value for a single feature is thus likely to reflect the richness of the corresponding concept in figurative semantic features (cf. Denis, 1982 a).
ACTUALIZATION OF FIGURATIVE SEMANTIC FEATURES AND THE COMPREHENSION OF SENTENCES The present section is intended to give some support to the hypothesized validity of the featural approach. One could indeed argue that such an analysis is a purely theoretical, speculative matter, and that there is nothing in the human mind that resembles features, as components of lexical meaning. A further series of experiments were directed towards this point, i.e., to demonstrate that it is possible to actualize in someone's mind not only wholistic concepts, but also parts only of concepts, that is subsets of features. The following experiments were all concerned with selective actualization of figurative (or non-figurative) components of conceptual representations, mainly during sentence comprehension (see Denis, 1982 b).
The notion of selective actualization is central in a componential model of conceptual representations. When postulating that a concept can be analyzed into a series of components, one simultaneously postulates that activating processes may selectively apply to some of these components, or groups of correlated components. Cognitive actualization of a concept may be "global" when all the features that compose it are simultaneously activated. It may be "partial" as well when a smaller number of such components are activated. So I attempted to experimentally activate some specific components of a conceptual representation and then detect by some appropriate procedure the greater cognitive actuality of these components relatively to others. For instance, when figurative features of a concept have been activated, it may be expected that transiently persistent traces of this cognitive event will later facilitate the recognition of a pictorial stimulus showing the corresponding properties. In one experiment, subjects had to listen to lists of words designating objects in classes, for instance a list of musical instruments. They then had to search in their memory whether the instruments possessed such or such figurative property. For instance, when presented with words such as piano, violin, accordion, cymbal, etc., subjects had to classify them according to~r possession of certa~sical properties, such as has strings, or has keys. This task was purportedly activating in subjects' working memory some figurative features of the corresponding concepts and it was hypothesized that, as a result, these features would transien-
24
M. Denis
tly be maintained at some significant level of cognitive actuality. In an other condition, other subjects had to classify the same list of words, but according to other, non-figurative properties, such as is old (i .e., before the 20th century, as previously indicated), or is difficult to ~This task was purportedly activating some non-figurative (or very poorly figurative) features of the same concepts. After the classification, subjects were unexpectedly presented with pictures representing musical instruments, among which they had to recognize as fast as possible those previously classified, from distractors. Although all subjects, in both conditions, had previously processed the very same list of concepts, subjects in the first condition gave recognition responses systematically faster than subjects in the second condition. Such a result may be accounted for if one assumes that subjects in the first condition, at the end of the classification task, maintained in their working memory a transient trace of previously actualized figurative features. These subjects thus were in a favourable position for responding to pictures since the major characteristic of any pictorial material is to directly show the figurative properties of objects. Conversely, subjects in the second condition initially actualized non-figurative features of concepts and thus were less able than others to recognize stimuli (pictures) showing properties of another kind (figurative properties). Therefore selective actualization of figurative semantic features is reflected in the way subjects later process a pictorial material. One should also mention that when subjects had to give their recognition responses not to pictures but to written names of instruments, there was no longer any difference between the two conditions. On the whole, conclusions from this experiment are in agreement with results such as those presented by Johnson-Laird, Gibbs, and de Mowbray (1978), according to which the semantic processing of a word does not necessarily involve all the elements of its meaning. A further step in this research consisted in demonstrating that selective actualization may operate within the subset of figurative features as well. In line with a componential approach of mental representations (cf. Le Ny, 1979), the following hypothesis was examined, i.e., semantic analysis of a sentence actualizes in a reader's mind specified subsets of features, rather than whole sets of features composing all the concepts referred to in the sentence. For instance, the conceptual representation for the word eagle is likely to be composed with features such as HOOKED BEAK, LARGE WINGS, CLAW~LD, DANGEROUS, etc. However, when one reads a sentence as : The eagle suddenly swooeed down and snatched the weasel, it is supposed that a preferentlal actuallzatlon of the feature CLAWS occurs, while there is a lesser actualization of the other features, or even no actualization at all for some of them. There is some evidence for this in an experiment that was conducted with J.-F. Le Ny. After having read such a sentence, subjects were presented with a picture showing a part of an eagle, and they were instructed to say whether the picture illustrated a part of an object described in the sentence. When the subjects were presented with a picture showing an eagle's claws, their responses were faster than when they were presented with for instance eagle's wings. In the same way, after reading a sentence whose content focused on the feature WINGS, such as : The eagle soared slowly and majestically into the heavens, subjects now responded faster to the plcture of wlngs than to the picture or-craws. Such results may be accounted for if we admit that semantic processing of descriptive sentences unequally activates components of meaning. The figurative features focused on by the sentence are primarily activated and maintained in memory, whereas other figurative features are activated at a lower degree, or even not activated at all. It is worth noting that this difference in latencies was no longer observed when subjects had to respond, not to pictures, but to written words (in this example, the word claws or the word wings). This result was obtained in a further experiment, and-rr-suggests that the mental representation elaborated by the reader of the sentence, that is maintained while he or she is waiting for the target item, actually contains a specifically figurative information. Is this psychological event to be characterized as a visual image? Is it "only" a cognitive, non
Images and Semantic Representations
25
imaginal actualization of the figurative properties that have been evoked in the sentence? A definite answer is hard to give. I will however note that the existence of such transient mental representations containing some actually figurative information has been suggested by results derived from an other experiment, which was conducted with D. Dubois. Without discussing the details of this experiment, I will only mention that, in one condition, subjects had to read descriptive sentences, such as : A majestic swan was silently gliding on the lake in the park, and then to decide whether a plcture could be considered as an lllustratlon to the sentence. For pictures supposed to elicit positive responses, latencies were shorter to the picture of a typical swan than to the picture of a less typical one (such as a black swan). But the relevant finding here is that for pictures supposed to elicit negative responses, latencies were longer to pictures that contained more figurative characteristics of the typical item. For instance, negative correct responses were slower the picture of a goose (a bird that possesses a number of figurative properties in common with the typical swan, such as the color white, a long neck, etc.) than to the picture of an owl. So it is likely that the semantic representation of the things described in the sentence has an actually figurative content, but it is not possible to state that this content is always and for everybody expressed through a conscious cognitive experience, as a visual image. As a final comment, I will relate the research efforts described above to the current trend that gives a growing importance to partial actualization and focalization processes in cognitive activities. It is a matter of fact that relating the study of figurative representations to the study of language comprehension gave prominence to this significant characteristic of cognitive processes. More generally, having studied the data, there are good reasons for our believing in the fruitfulness of a mutual settlement in the approaches of researchers in language comprehension and of those in imagery. NOTE The research reported above and the preparation of this chapter were supported by the Laboratoire de Psychologie, Equipe de Recherche associee au C.N.R.S. n° 235. Author's address: Laboratoire de Psychologie, E.R.A. au C.N.R.S. n° 235, Universite de Paris VIII, 2, rue de la Liberte, 93526 Saint-Denis Cedex 2, France. REFERENCES (1) Beech, J.R., An alternative model to account for the Clark and Chase picture verification experiments, Journal of Mental Imagery, 1980, 4, 1-11. (2) Carpenter, P.A., and Just, M.A., Sentence comprehension: A psycholinguistic processing model of verification, Psychological Review, 1975, 82, 45-73. (3) Chase, W.G., and Clark, H.H., Mental operations in the comparison of sentences and pictures, in L.W. Gregg (ed.), Cognition in learning and memory, New York, Wiley, 1972. (4) Collins, A.M., and Quillian, M.P., Retrieval time from semantic memory, Journal of Verbal Learning and Verbal Behavior, 1969, 8, 240-247. (5) Denis, M., Les images mentales, Paris, Presses Universitaires de France, 1979. (6) Denis, M., Valeur d'imagerie et composition semantique : analyse de deux echantillons de substantifs, Universite de Paris VIII, Documents du Laboratoire de Psychologie, 1982 (a). (7) Denis, M., On figurative components of mental representations, in F. Klix, J. Hoffmann and E. van der Meer (eds.), Cognitive research in psychology:
26
M. Denis
Recent approaches, designs and results, Amsterdam, North-Holland, 1982 (b). (8) Dubois, D., and Weil-Barais, A., Activites anticipatrices dans une tache de comparaison de dessins et de phrases, in J. Requin (ed.), Anticipation et comportement, Paris, Editions du C.N.R.S., 1980. (9) Eysenck, M.W., The feeling of knowing a word's meaning, British Journal of Psychology, 1979, 70, 243-251. (10) Friedman, A., Memorial comparisons without the "mind's eye", Journal of Verbal Learning and Verbal Behavior, 1978, 17, 427-444. (11) Glass, A.L., Eddy, J.K., and Schwanenflugel, P.J., The verification of high and low imagery sentences, Journal of Experimental Psychology: Human Learning and Memory, 1980, 6, 692-704. (12) Hoffmann, J., Representation of concepts and the classification of objects, in F. Klix, J. Hoffmann and E. van der Meer (eds.), Cognitive research in psychology: Recent approaches, designs and results, Amsterdam, North-Holland, 1982. (13) Holmes, V.M., and Langford, J., Comprehension and recall of abstract and concrete sentences, Journal of Verbal Learning and Verbal Behavior, 1976, 15, 559-566. (14) Holyoak, K.J., The role of imagery in the evaluation of sentences; Imagery or semantic factors? Journal of Verbal Learning and Verbal Behavior, 1974, 13, 162-166. (15) Huttenlocher, J., Constructing spatial images A strategy in reasoning, Psychological Review, 1968, 75, 550-560. (16) Johnson-Laird, P.N., Gibbs, G., and de Mowbray, J., Meaning, amount of processing, and memory for words, Memory and Cognition, 1978, 6, 372-375. (17) Jorgensen, C.C., and Kintsch, W., The role of imagery in the evaluation of sentences, Cognitive Psychology, 1973, 4, 110-116. (18) Kosslyn, S.M., Image and mind, Cambridge, Mass., Harvard University Press, 1980. (19) Kosslyn, S.M., Murphy, G.L., Bemesderfer, M.E., and Feinstein, K.J., Category and continuum in mental comparisons, Journal of Experimental Psychology: General, 1977, 106, 341-375. (20) Le Ny, J.-F., La semantique psychologique, Paris, Presses Universitaires de France, 1979. (21) MacLeod, C.M., Hunt, E.B., and Mathews, N.N., Individual differences in the verification of sentence-picture relationships, Journal of Verbal Learning and Verbal Behavior, 1978, 17, 493-507. (22) Miller, G.A., and Johnson-Laird, P.N., Language and perception, Cambridge, Mass., Harvard University Press; Cambridge, England, Cambridge University Press, 1976. (23) Paivio, A., Imagery and verbal processes, New York, Holt, Rinehart &Winston, 1971 (Second printing, Hillsdale, N.J., Lawrence Erlbaum Associates, 1979). (24) Paivio, A., Perceptual comparisons through the mind's eye, Memory and Cognition, 1975, 3, 635-647. (25) Paivio, A., Mental comparisons involving abstract attributes. Memory and Cognition, 1978, 6, 199-208. (26) Potter, M.C., Valian, V.V., and Faulconer, B.A., Representation of a sentence and its pragmatic implications; Verbal, imagistic, or abstract? Journal of Verbal Learning and Verbal Behavior, 1977, 16, 1-12. (27) Rosch, E., Mervis, C.B., Gray, W., Johnson, D., and Boyes-Braem, P., Basic objects in natural categories, Cognitive Psychology, 1976, 8, 382-439.
Images andSemantic Representations
27
(28) Smith, E.E., Shoben, E.J., and Rips, L.J., Structure and process in semantic memory: A featural model fo~ semantic decisions, Psychological Review, 1974, 81, 214-241.
17
IMAGES AND SEMANTIC REPRESENTATIONS Michel Denis Laboratoire de Psycho10gie Universite de Paris VIII Saint-Denis, France
In this chapter are summarized and discussed selected data from a research program on the relationships between imagery and semantic processing. Within the framework of a featura1 approach to conceptual representations, imagery is viewed as a constructive process that selectively applies to, and transiently actualizes, specific subsets of semantic features, the so-called figurative features. INTRODUCTI ON The last twelve years of research in psychology have been characterized by a number of theoretical and empirical efforts with the aim of giving mental imagery - a long-standing theme in our discipline - a renewed theoretical status. Research has simultaneously developed in two main directions. On the one hand, imagery was investigated as an independent variable, i.e., as a process intervening or likely to intervene in the course of many psychological activities, such as memorization, comprehension of sentences, problem solving, etc. As a general conclusion, it may be stated that the use of imagery has positive effects on most of these activities, whatever the indicators of performance. However this kind of approach is unlikely to throw much light on what an image is and how the intrinsic properties of this psychological event may account for its effimncy. Thus, what can be named the "second generation" of contemporaneous research in imagery is more concerned with questions such as : What are the characteristics of images, in contrast with other ways people evoke concepts? What is the structure of this psychological event? To what extent shall we consider it as an "analogical" mode of representation? Must we consider that images have genuine spatial properties? How shall we consider their temporal characteristics? Whatever the researchers' specific focus as regards imagery, it is obvious that research in this field is fundamentally concerned with the question of the relationships between images and knowledge. No doubt that images are a privileged modality for actualization of some aspects of knowledge, but a number of other aspects are unable to be figuratively represented. In any case, a persistent feature in the last ten or twelve years of psychology will have been the feeble effort, or the lack of a widespread effort, to systematically connect the study of imagery to the wider field of representation of knowledge. During a long time, research in imagery developed without any concern of converging with the models of semantic memory that attempted to describe the structure of knowledge and its organization in long term memory. We are still greatly unaware of the relationships between, on the one hand, activation of semantic structures leading to the construction of meaning in the mind of a reader who has to verify a statement (such as : A canary has wings ... ) and, on the other hand, the imagery elicited by this statement. These questions in fact were expressed in several empirical approaches with various theoretical backgrounds, but with the common concern of deliberately introducing imagery in the study of semantic activities - contrary to what was the case in several former, leading models (cf. Collins and Quillian, 1969 ; Smith, Shoben, and Rips, 1974). One widespread empirical approach was sentence
18
M. Denis
verification, in an attempt to measure verification latencies of sentences varying in their ability to elicit visual images that represent their content. Results initially presented by Jorgensen and Kintsch (1973), and later by Holmes and langford (1976), attested a facilitating effect of the imagery value of sentences on their verification. However such a relationship is difficult to interpret, mainly because of the difficulty in distinguishing the effects of imagery from the effects of the semantic complexity of sentences (cf. Holyoak, 1974). More recent experiments by Glass, Eddy and Schwanenflugel (1980) demonstrated that the reading of descriptive sentences may in fact be accompanied with an imagery activity, objectively detected by appropriate experimental procedures, but without allowing the researcher to ascribe to these images a decisive role in verification processes per se. Such a conclusion is itself in agreement with hypotheses concerning the abstract, amodal format of representation of sentence meaning (cf. Potter, Valian,and Faulconer, 1977). An other procedure, in some respects similar to the former one, was equally used for examining the possible role of imagery in sentence verification - the mental comparison technique. In these tasks, the subject has to evaluate as true or false sentences asserting a relation, such as : A lobster is larger than a crayfish. In opposition to Paivio's (1975, 1978) arguments favoring the explanatory role of visual imagery in the evaluation of such sentences, critical arguments, such as Friedman's (1978), were developed in favor of different forms of mental representation, other than "analogical". The most recent trends seem to favor mixed models, postulating parallel processing of analogical representations, such as visual images, and propositional representations (cf. Kosslyn, 1980 ; Kosslyn, Murphy, 8emesderfer, and Feinstein, 1977). lastly, a paradigm that looks relevant to the field of research discussed above is the so-called sentence-picture comparison paradigm, where a picture is to be compared by subjects with the semantic content of a descriptive sentence (such as: The plus is above the star ... ). After a long lasting controversy between proponents of "imaglstlc" lnterpretations (cf . Huttenlocher, 1968) and those of "propositional" accounts (cf. Carpenter and Just, 1975 ; Chase and Clark, 1972), this paradigm was more recently used in a new perspective, emphasizing strategies that differentiate individuals in such tasks (cf. Macleod, Hunt, and Mathews, 1978). Thus it is possible to identify subjects with clear preferences for "linguistic" strategies (they compare a linguistic description of the picture to the semantic representation of the sentence) and subjects inclined to "visual" strategies (they more readily compare the picture to the visual image of the pattern described in the sentence) (see also Beech, 1980 ; Dubois and Weil-Barais, 1980). The major idea stemming from these current research trends is that the role of radical imagistic positions in the field of language understanding has lessened and that presently more weight is given to propositional interpretations, calling upon abstract, amodal semantic representations, which are neither linguistic nor imaginal, but which have the open possibility of optional strategies making use of visual imagery. In this paper I briefly describe a series of steps which are part of a research program devoted to the question of the relationships between the domain of images - more specifically, images involved with linguistic activity, mainly comprehension of sentences - and the domain of semantics, by an approach initially concerned with lexical units, then with sentences. FIGURATIVE SEMANTIC FEATURES The major working hypothesis here is that there is no definite separation between the conceptual approaches to imaginal representations and to semantic representations. More precisely, the idea developed here is that imagery is a modality for the actualization of semantic representations of sentences, through a highly
Images andSemantic Representations
19
specific format of representation, mainly characterized by its structural analogy with perceptual events. Semantic representations can be analyzed into components, some of which may be actualized under a specific format and result in the elaboration of cognitive experiences"mimicking perceptual experiences to some extent (cf. Denis, 1979). The approach illustrated here relies upon the hypothesis that the format of the most elementary units of cognitive representations is lower than the lexical format. A full examination of the arguments supporting this claim, or rather the family of theories having this claim in common, is impossible here. Let us however concentrate on the idea that concepts to which lexical items are attached can be analyzed into more elementary units of meaning, or semantic features. This statement does not only mean that humans (among them, psychologists) are capable of analyzing a concept into smaller pieces of meaning, but that there exist arguments in support of the psychological validity of the notion of semantic features as elementary components of cognitive representations (e.g., Eysenck, 1979 ; Le Ny, 1979 ; Miller and Johnson-Laird, 1976). For instance, let us consider the meaning attached to the lexical unit knife. The concept to which the word knife refers may be analyzed into a set of features among which we would proba~ind : BLADE, HANDLE, LENGTHENED SHAPE, CUTS, USEFUL, DANGEROUS, etc. Two general characteristics of such feature sets should be underlined: (a) their hierarchical organization, which reflects the unequal contribution of these features to the concept; (b) their flexibility, i.e., the possibility of transient reorganizations of the hierarchy, depending on factors such as linguistic context, individual's cognitive orientation, etc. However, my paper will treat essentially the major characteristic of meaning according to this interpretation, that is : Meaning elicited by a word is a cognitive construct which results from the composition of a set of semantic features. Among the set of features that define the concept KNIFE, some features are related to the physical properties of knives, i.e., properties that have been perceptually experienced as such by the individual. This is the case of features such as BLADE, HANDLE, etc. These units are presumably involved in some privileged way not only in visual imagery elicited by the word knife, but also in any activity aiming at depicting the figural appearance of a k~ as in a pictorial representation. For this reason, I call such units of meaning "figurative semantic features", in contrast with other kinds of features that do not directly reflect the fiqural properties of objects. Such is probably the case for USEFUL and for DANGEROUS. We certainly are able to construct visual images that reflect through some kind of associative imagery the usefulness or the danger of a knife. However it is obvious that such an imagery is not as "direct" as when the features BLADE or HANDLE are used to evoke the concept KNIFE. That is why it seems more appropriate to qualify features as to the probability with which imagery processes may apply to them. In this manner it would be posslble to isolate from among the set of features defining a concept a subset of features to which imagery processes may apply with maximal probability. These are the features I shall call in abbreviated form "figurative (semantic) features". In my interpretation, imagery is to be viewed as a constructive process that gives a specific, limited subset of features a transient cognitive actuality, through a specific type of mental experience (mainly characterized by its structural analogy with perceptual experience). IMAGERY VALUE AND SEMANTIC COMPOSITION Several techniques have been developed in order to "measure" imagery elicited by linguistic inputs. One of them relies on the notion of "imagery value", to which research gave an empirical status at the time of the renewal of psychologists' interest for imagery (see Paivio, 1971). Imagery value of a word is defined as the ability of this word to elicit an imaginal representation in the individual, that is a cognitive event with a figurative content. It is operationally defined by the
20
M. Denis
mean value given to this word by a group of subjects on a rating scale, in most cases a seven-point scale, whose extreme points for instance are labeled, on the one hand, "no image" and, on the other hand, "highly vivid image", When subjects rate a sample of words on such a scale, not only is this task accepted as meaningful, and words in general scatter over the whole range of the scale, but this indicator also shows a high degree of inter- and intraindividua1 consistency. Defined in this way, the imagery value of words has proved to be a significant factor in many kinds of processing of linguistic material: comprehension, verification, memorization, etc. Linguistic messages made with high-imagery words are better memorized than messages made with low-imagery words. However, the questions that were surprisingly absent from recent discussions are : What does imagery value measure? What does it reflect, especially regarding the structure of knowledge about corresponding objects? In other words, what happens in someone's mind when he or she constructs an image in order to evaluate its vividness, its clarity, or its richness? tentative answer is that imaging results from a process that applies to the subset of figurative features of a concept. Constructing images in response to words is identified as the actualization, in a specialized representational system, of figurative components of concepts, through a specialized process whose major characteristic is the ability to construct mental events with a structure and an internal organization that are analogical with those of perceptual events. This actualization of figurative features presumably takes place within a "framework", a kind of a Gestalt spatial scheme, which is the representation of the global shape of the object (cf. the notion of "skeletal encoding" in Koss1yn, 1980). Whatever this hypothetical framework, the very first step in my research program consisted in testing the main consequence of the assumption that links imagery and semantic structures, i.e., one should predict the imagery value of words from the richness of the corresponding concepts in the so-called figurative features. My
The first experiment whose objective was to examine this hypothesis investigated the imagery value of general and specific words, and its relation with the semantic composition of these words. The generality-specificity dimension is closely related to a dimension that one may call richness of concepts in figurative semantic features. If one considers a branch in a taxonomy whose ultimate term is a physical object - for instance, LIVING BEING - ANIMAL - BIRD - PARROT, - it is obvious that, as one moves down the taxonomy, not only do concepts grow richer in semantic features, but they also grow richer in semantic figurative features. A corollary of this statement is that the more specific the concepts, the easier they are to evoke the objects as images, whereas images for more general concepts should either be very poor or consist in instantiations (that is, images of subconcepts richer in figurative features). It has been demonstrated that general terms, such as furniture, clothing, etc. are less likely to elicit images than more specific terms, such as chair, pants, etc. But this is mainly apparent in tasks emphasizing the speed witn-wnrch lmages are to be constructed. As a general result, imagery latencies are longer to general terms than to specific terms. This difference also holds for drawing latencies to these terms. However, when one considers imagery values (that are given without any temporal constraints), there is in most cases very little difference between general and specific terms. A likely interpretation is that when subjects have to rate for imagery value a general term that is poor in figurative features, they tend to look for a specific exemplar in the category, and they finally give the experimenter the imagery value for the associated specific term, not the imagery value for the general term itself. Figure 1 is a schematic representation of four hypothetical levels in a taxonomy, from the highest, more general level (I) to the lowest, more specific level (IV).
Images andSemantic Representations
21
At each level the features defining the corresponding concept are listed as lower-case letters. According to such an approach, a concept at any level is composed of the set of features defining the superordinate concept, plus a specific set of features. For instance, BIRD would comprise all the features for ANIMAL, plus some new ones: WINGS, FEATHERS, etc. At the lower level, PARROT would comprise all the features for BIRD, plus some new, specific ones: MANY-COLOURED, REPEATS, etc. (cf. Le Ny, 1979). In this framework, specification is therefore mainly viewed as a semantic enrichment. (a, b, c) II {a, b, c} & {i, e,
n
III {a, b, c} & {i, e,
n
& {!I' ~,
IV {a, b, c) & {i, e, f l & {!I,
~,
il il & {,t, .~,
1)
Figure 1 Schematic representation for four hypothetical levels in a taxonomy In Figure 1, underlined letters correspond to features that are supposed to be figurative ones, i.e., features reflecting object properties directly accessed by perceptual processes (visual, in the examples discussed here). Thus, in most taxonomies, the higher level concept would essentially be composed of non-figurative features, corresponding to very general, abstract properties (a, b, c). The immediatly lower concept would be composed of the same set of features (a, b, c), plus a further set (d, e, f), in which feature d would reflect a physical property and would be likely to be figuratively represented, either in a visual image or in a drawing. My contention is that such figurative features become more and more numerous as one goes down from one level to a lower level in the hierarchy. It is likely that such an analysis will make the positive relationship between imagery value and specificity more obvious, not only because more specific concepts are composed of a greater number of features, but because they are composed of a greater relative number of figurative features in the whole set of features defining these concepts. (I shall insist again that it is for brevity's sake that I speak of "figurative" versus "non-figurative" features. It is more likely indeed that any semantic feature is characterized by the degree or the probability with which it can be represented in an image.) My initial effort consisted in demonstrating that (a) general concepts are composed of a lesser number of features than more specific ones, and (b) specific concepts are composed of a relatively greater number of figurative features (or, according to an alternate formulation, the mean "figurativeness" of features composing specific concepts is higher than for more general concepts). In this experiment, pairs of substantives were used, one general and one specific. (One should however keep in mind that there are no absolute "general" nor "specific" words. Generality or specificity of a word is always relative to that of another word. That is why the strictest formulation would be that, in pairs of substantives, one was more general, or less specific, than the other one.) Two kinds of general concepts were considered: general concepts A (such as TREE, BIRD, FISH, etc.) and general concepts B (such as VEHICLE, FURNITURE, VEGETABLE, etc.). General concepts A subsume members that have a great number of figural characteristics in common. For instance, there is a high degree of similarity between most birds as to their global shape, the parts that compose them,and the relations between these parts. Such concepts are close to "base level concepts" according to Rosch's terminology (cf. Rosch, Mervis, Gray, Johnson, and BoyesBraem, 1976), that is concepts at the most inclusive level where the various members have a great number of properties in common, elicit similar behavioral
22
M. Denis
programs, and possess highly similar figural characteristics. These concepts are close also to what Hoffmann (1982) defined as "primary concepts" in a similar, independent approach. General concepts 8, on the other hand, subsume members with a high degree of figural discriminability. For instance, when one considers the category of vehicles, the figural appearance of a truck is strongly distinct from that of a bicycle, or a helicopter, or a space rocket. A specific term was paired with each general term. It was a high associate, non ambiguous word, whose frequency was equated with the frequency of the general term. A first group of subjects were presented with the selected terms (different subjects were presented with the general or the specific term in a pair), and they gave lists of properties for the designated classes of objects. From these lists were derived lists of features hypothetically defining each of the general and specific concepts. Features were lexically expressed either by substantives (for instance, for the concept SNAKE: SCALES, TONGUE, VENOM), or by adjectives (LONG), or by verbs (CRAWLS). Only responses given by at least 30 per cent of the subjects were considered. This, for instance, is the list of features defining the concept FLOWER, presented in decreasing order of their frequency: PETALS, STEM, LEAVES, STAMENS, POLLEN, PISTIL, SMELL. It was found, as expected, that the mean number of features was lower for general than for specific concepts. However, when considering only general concepts A and their specific counterparts, the difference is hardly noticeable, whereas the difference is very clear in the case of general concepts B. The next step consisted in obtaining an indicator of the "figurativeness" of features. This was performed by asking a new group of subjects to rate all the features on a scale similar to the imagery value scale. Then it was possible to sum the obtained mean values for all the features composing a concept, this resulting in a "composite imagery value" for each of the general and specific terms. When one looks at the whole material, it appears that the composite imagery value is lower for general than for specific terms. As previously seen, there is a very small difference between general terms A and their specific counterparts, whereas between general terms B and their specific counterparts the difference is large and significant. The composite imagery value of general terms B is the lowest. These terms do then refer, as previously argued, to concepts whose features have a very low degree of "figurativeness". Moreover the contrast beween these terms and all the others is much more pronounced than was the contrast previously observed when "global imagery values" were considered. It thus appears that "composite imagery value" gives a better reflection of the "figurativeness" of the semantic components of concepts, and that this indicator is much less likely to be biased by instantiation processes than is the "global imagery value". One however has to examine the following argument: The additive character of the hypothesis illustrated by Figure 1 is such that it is just statistically likely that the composite imagery values increase when the number of features taken into account increases. The answer is that when one computes the mean imagery value for a single feature, this value is still the lowest for general concepts, and still lower for features composing general concepts B than for features composing any of the other concepts. It must be noted however that the range of this indicator is limited, due to the fact that a11 the consi dered concepts are, on the whole, on the "positive" si de of imagery values. They all subsume physical objects or classes of physical objects. That is why I turned to other materials, in the hope that they would produce clearer differentiations. The general purpose was the same, with the additional attempt to demonstrate the relationship between the number of features composing a concept and their "figurativeness". In other words, concepts rich in semantic features - as was the case at the lowest levels of the taxonomies, according to semantic the previous approach - would by this very fact be rich in fi~urative features, whereas concepts poorer in semantic features - for lnstance, at the top of taxonomies - would be poorer in figurative semantic features.
Images and Semantic Representations
23
Having looked for a corpus whose elements would widely scatter on the rating scales to be used and furthermore would not be systematically hierarchized along the generality-specificity dimension, I turned to terms designating occupations. This semantic field is very large, with a wide variety of elements dispersed over the whole range of the imagery value scale, from occupations with salient figurative attributes, such as fireman or judge, to occupations with none or almost none of these attributes, such as editor-or-manager. From a list of 155 terms designating occupations that had been ~for~) imagery value by a group of subjects, I selected four groups of items (A, B, C, D) from the lowest to the highest imagery value. Without describing here the details of the analyses - which followed the same steps as in the previous experiment, - the main results were straightforward. The mean number of features per concept is positively correlated with the imagery value of corresponding terms. In other words, the imagery value of a word is closely dependent on the richness of the corresponding concept in semantic features. Furthermore, concepts richer in semantic features are richer in figurative semantic features. Actually, not only the composite imagery value attached to concepts increases from items A to items 0, but the mean imagery value for a single feature itself increases from items A to items D. A significant, positive correlation was computed between the global imagery value of words and the latter indicator. The mean imagery value for a single feature is thus likely to reflect the richness of the corresponding concept in figurative semantic features (cf. Denis, 1982 a).
ACTUALIZATION OF FIGURATIVE SEMANTIC FEATURES AND THE COMPREHENSION OF SENTENCES The present section is intended to give some support to the hypothesized validity of the featural approach. One could indeed argue that such an analysis is a purely theoretical, speculative matter, and that there is nothing in the human mind that resembles features, as components of lexical meaning. A further series of experiments were directed towards this point, i.e., to demonstrate that it is possible to actualize in someone's mind not only wholistic concepts, but also parts only of concepts, that is subsets of features. The following experiments were all concerned with selective actualization of figurative (or non-figurative) components of conceptual representations, mainly during sentence comprehension (see Denis, 1982 b).
The notion of selective actualization is central in a componential model of conceptual representations. When postulating that a concept can be analyzed into a series of components, one simultaneously postulates that activating processes may selectively apply to some of these components, or groups of correlated components. Cognitive actualization of a concept may be "global" when all the features that compose it are simultaneously activated. It may be "partial" as well when a smaller number of such components are activated. So I attempted to experimentally activate some specific components of a conceptual representation and then detect by some appropriate procedure the greater cognitive actuality of these components relatively to others. For instance, when figurative features of a concept have been activated, it may be expected that transiently persistent traces of this cognitive event will later facilitate the recognition of a pictorial stimulus showing the corresponding properties. In one experiment, subjects had to listen to lists of words designating objects in classes, for instance a list of musical instruments. They then had to search in their memory whether the instruments possessed such or such figurative property. For instance, when presented with words such as piano, violin, accordion, cymbal, etc., subjects had to classify them according to~r possession of certa~sical properties, such as has strings, or has keys. This task was purportedly activating in subjects' working memory some figurative features of the corresponding concepts and it was hypothesized that, as a result, these features would transien-
24
M. Denis
tly be maintained at some significant level of cognitive actuality. In an other condition, other subjects had to classify the same list of words, but according to other, non-figurative properties, such as is old (i .e., before the 20th century, as previously indicated), or is difficult to ~This task was purportedly activating some non-figurative (or very poorly figurative) features of the same concepts. After the classification, subjects were unexpectedly presented with pictures representing musical instruments, among which they had to recognize as fast as possible those previously classified, from distractors. Although all subjects, in both conditions, had previously processed the very same list of concepts, subjects in the first condition gave recognition responses systematically faster than subjects in the second condition. Such a result may be accounted for if one assumes that subjects in the first condition, at the end of the classification task, maintained in their working memory a transient trace of previously actualized figurative features. These subjects thus were in a favourable position for responding to pictures since the major characteristic of any pictorial material is to directly show the figurative properties of objects. Conversely, subjects in the second condition initially actualized non-figurative features of concepts and thus were less able than others to recognize stimuli (pictures) showing properties of another kind (figurative properties). Therefore selective actualization of figurative semantic features is reflected in the way subjects later process a pictorial material. One should also mention that when subjects had to give their recognition responses not to pictures but to written names of instruments, there was no longer any difference between the two conditions. On the whole, conclusions from this experiment are in agreement with results such as those presented by Johnson-Laird, Gibbs, and de Mowbray (1978), according to which the semantic processing of a word does not necessarily involve all the elements of its meaning. A further step in this research consisted in demonstrating that selective actualization may operate within the subset of figurative features as well. In line with a componential approach of mental representations (cf. Le Ny, 1979), the following hypothesis was examined, i.e., semantic analysis of a sentence actualizes in a reader's mind specified subsets of features, rather than whole sets of features composing all the concepts referred to in the sentence. For instance, the conceptual representation for the word eagle is likely to be composed with features such as HOOKED BEAK, LARGE WINGS, CLAW~LD, DANGEROUS, etc. However, when one reads a sentence as : The eagle suddenly swooeed down and snatched the weasel, it is supposed that a preferentlal actuallzatlon of the feature CLAWS occurs, while there is a lesser actualization of the other features, or even no actualization at all for some of them. There is some evidence for this in an experiment that was conducted with J.-F. Le Ny. After having read such a sentence, subjects were presented with a picture showing a part of an eagle, and they were instructed to say whether the picture illustrated a part of an object described in the sentence. When the subjects were presented with a picture showing an eagle's claws, their responses were faster than when they were presented with for instance eagle's wings. In the same way, after reading a sentence whose content focused on the feature WINGS, such as : The eagle soared slowly and majestically into the heavens, subjects now responded faster to the plcture of wlngs than to the picture or-craws. Such results may be accounted for if we admit that semantic processing of descriptive sentences unequally activates components of meaning. The figurative features focused on by the sentence are primarily activated and maintained in memory, whereas other figurative features are activated at a lower degree, or even not activated at all. It is worth noting that this difference in latencies was no longer observed when subjects had to respond, not to pictures, but to written words (in this example, the word claws or the word wings). This result was obtained in a further experiment, and-rr-suggests that the mental representation elaborated by the reader of the sentence, that is maintained while he or she is waiting for the target item, actually contains a specifically figurative information. Is this psychological event to be characterized as a visual image? Is it "only" a cognitive, non
Images and Semantic Representations
25
imaginal actualization of the figurative properties that have been evoked in the sentence? A definite answer is hard to give. I will however note that the existence of such transient mental representations containing some actually figurative information has been suggested by results derived from an other experiment, which was conducted with D. Dubois. Without discussing the details of this experiment, I will only mention that, in one condition, subjects had to read descriptive sentences, such as : A majestic swan was silently gliding on the lake in the park, and then to decide whether a plcture could be considered as an lllustratlon to the sentence. For pictures supposed to elicit positive responses, latencies were shorter to the picture of a typical swan than to the picture of a less typical one (such as a black swan). But the relevant finding here is that for pictures supposed to elicit negative responses, latencies were longer to pictures that contained more figurative characteristics of the typical item. For instance, negative correct responses were slower the picture of a goose (a bird that possesses a number of figurative properties in common with the typical swan, such as the color white, a long neck, etc.) than to the picture of an owl. So it is likely that the semantic representation of the things described in the sentence has an actually figurative content, but it is not possible to state that this content is always and for everybody expressed through a conscious cognitive experience, as a visual image. As a final comment, I will relate the research efforts described above to the current trend that gives a growing importance to partial actualization and focalization processes in cognitive activities. It is a matter of fact that relating the study of figurative representations to the study of language comprehension gave prominence to this significant characteristic of cognitive processes. More generally, having studied the data, there are good reasons for our believing in the fruitfulness of a mutual settlement in the approaches of researchers in language comprehension and of those in imagery. NOTE The research reported above and the preparation of this chapter were supported by the Laboratoire de Psychologie, Equipe de Recherche associee au C.N.R.S. n° 235. Author's address: Laboratoire de Psychologie, E.R.A. au C.N.R.S. n° 235, Universite de Paris VIII, 2, rue de la Liberte, 93526 Saint-Denis Cedex 2, France. REFERENCES (1) Beech, J.R., An alternative model to account for the Clark and Chase picture verification experiments, Journal of Mental Imagery, 1980, 4, 1-11. (2) Carpenter, P.A., and Just, M.A., Sentence comprehension: A psycholinguistic processing model of verification, Psychological Review, 1975, 82, 45-73. (3) Chase, W.G., and Clark, H.H., Mental operations in the comparison of sentences and pictures, in L.W. Gregg (ed.), Cognition in learning and memory, New York, Wiley, 1972. (4) Collins, A.M., and Quillian, M.P., Retrieval time from semantic memory, Journal of Verbal Learning and Verbal Behavior, 1969, 8, 240-247. (5) Denis, M., Les images mentales, Paris, Presses Universitaires de France, 1979. (6) Denis, M., Valeur d'imagerie et composition semantique : analyse de deux echantillons de substantifs, Universite de Paris VIII, Documents du Laboratoire de Psychologie, 1982 (a). (7) Denis, M., On figurative components of mental representations, in F. Klix, J. Hoffmann and E. van der Meer (eds.), Cognitive research in psychology:
26
M. Denis
Recent approaches, designs and results, Amsterdam, North-Holland, 1982 (b). (8) Dubois, D., and Weil-Barais, A., Activites anticipatrices dans une tache de comparaison de dessins et de phrases, in J. Requin (ed.), Anticipation et comportement, Paris, Editions du C.N.R.S., 1980. (9) Eysenck, M.W., The feeling of knowing a word's meaning, British Journal of Psychology, 1979, 70, 243-251. (10) Friedman, A., Memorial comparisons without the "mind's eye", Journal of Verbal Learning and Verbal Behavior, 1978, 17, 427-444. (11) Glass, A.L., Eddy, J.K., and Schwanenflugel, P.J., The verification of high and low imagery sentences, Journal of Experimental Psychology: Human Learning and Memory, 1980, 6, 692-704. (12) Hoffmann, J., Representation of concepts and the classification of objects, in F. Klix, J. Hoffmann and E. van der Meer (eds.), Cognitive research in psychology: Recent approaches, designs and results, Amsterdam, North-Holland, 1982. (13) Holmes, V.M., and Langford, J., Comprehension and recall of abstract and concrete sentences, Journal of Verbal Learning and Verbal Behavior, 1976, 15, 559-566. (14) Holyoak, K.J., The role of imagery in the evaluation of sentences; Imagery or semantic factors? Journal of Verbal Learning and Verbal Behavior, 1974, 13, 162-166. (15) Huttenlocher, J., Constructing spatial images A strategy in reasoning, Psychological Review, 1968, 75, 550-560. (16) Johnson-Laird, P.N., Gibbs, G., and de Mowbray, J., Meaning, amount of processing, and memory for words, Memory and Cognition, 1978, 6, 372-375. (17) Jorgensen, C.C., and Kintsch, W., The role of imagery in the evaluation of sentences, Cognitive Psychology, 1973, 4, 110-116. (18) Kosslyn, S.M., Image and mind, Cambridge, Mass., Harvard University Press, 1980. (19) Kosslyn, S.M., Murphy, G.L., Bemesderfer, M.E., and Feinstein, K.J., Category and continuum in mental comparisons, Journal of Experimental Psychology: General, 1977, 106, 341-375. (20) Le Ny, J.-F., La semantique psychologique, Paris, Presses Universitaires de France, 1979. (21) MacLeod, C.M., Hunt, E.B., and Mathews, N.N., Individual differences in the verification of sentence-picture relationships, Journal of Verbal Learning and Verbal Behavior, 1978, 17, 493-507. (22) Miller, G.A., and Johnson-Laird, P.N., Language and perception, Cambridge, Mass., Harvard University Press; Cambridge, England, Cambridge University Press, 1976. (23) Paivio, A., Imagery and verbal processes, New York, Holt, Rinehart &Winston, 1971 (Second printing, Hillsdale, N.J., Lawrence Erlbaum Associates, 1979). (24) Paivio, A., Perceptual comparisons through the mind's eye, Memory and Cognition, 1975, 3, 635-647. (25) Paivio, A., Mental comparisons involving abstract attributes. Memory and Cognition, 1978, 6, 199-208. (26) Potter, M.C., Valian, V.V., and Faulconer, B.A., Representation of a sentence and its pragmatic implications; Verbal, imagistic, or abstract? Journal of Verbal Learning and Verbal Behavior, 1977, 16, 1-12. (27) Rosch, E., Mervis, C.B., Gray, W., Johnson, D., and Boyes-Braem, P., Basic objects in natural categories, Cognitive Psychology, 1976, 8, 382-439.
Images andSemantic Representations
27
(28) Smith, E.E., Shoben, E.J., and Rips, L.J., Structure and process in semantic memory: A featural model fo~ semantic decisions, Psychological Review, 1974, 81, 214-241.