Cognition, 6 (1978) 79-85 @Elsevier Sequoia S.A., Lausanne
Discussion - Printed
A criticism
in the Netherlands
Unfulfilled expectations : of Neisser’s theory of imagery
P. J. HAMPSON and P. E. MORRIS” Department University
of
Psychology
of Lancaster
Abstract Neisser’s recent theory of imagery (Neisser, 1976) is critically discussed and is found to contain several weaknesses. (I) It is not clear that the theory avoids the alleged difficulties of alternative accounts in explaining the lack of subjective confusion of images and percepts and the nature of the observer of mental operations. (2) In viewing imagery as a perceptual anticipation, Neisser resembles Ryle (1949). The conclusions and derivative arguments of both authors are criticized: Anticipation may well be a necessary but is not a sufficient component of imaging. (3) Neisser’s account has severe problems in explaining how images are manipulated and used in cognition. (4) The conscious experience of having an image is seen to differentiate imaging from just knowing - a possibility not admitted by Neisser. (5) Finally, Neisser’s view of introspection is criticized on the grounds that the use of real world descriptive terms does not imply a description of real world objects. Introspection, whilst not error free, still occurs. There has been much debate recently concerning the status of mental images. There are two main approaches. The “analogue” or “representational” approach (e.g., Shepard, 1975; Kosslyn and Pomerantz, 1977) maintains that images are a qualitatively distinct form of internal representation and that the experience of having a (visual) image resembles the experience of seeing the referent of the image. The “propositional” account (e.g., Pylyshyn, 1973, 1975) replaces the analogical representation by abstract structural descriptions accessed by computationally primitive semantic interpretation
*The contributions of the authors in the production of this paper were of equal weight. We wish to thank Andrew Young for his helpful comments on an earlier version of the manuscript. Requests for reprints should be sent to: Peter E. Morris, Department of Psychology, Fylde College, University of Lancaster, Lancaster LA1 4YF. England.
80
P. J. Hampson arzd P. E. Morris
functions and argues that the conscious image has no functional role in cognition. Recently, Neisser (1976) has rejected both of these approaches. He argues that both are versions of the linear cognitive model in which information is processed through successive levels from sensory reception input to consciousness. He wishes to replace the linear model with a cyclical model of cognitive activity which has three principle elements: the first are the objects in the real world and the information that specifies them, the latter modifies the second, the schemata that the subject possesses or constructs for perceiving the world, which in turn direct the third element, exploration of the environment, which controls sampling of the first element, objects in the real world. From this cyclic model Neisser derives a new theory of the nature of imagery which will be examined below. Central to Neisser’s account of cognition and imagery are schemata. He defines a schema as “that portion of the entire perceptual cycle which is internal to the perceiver, modifiable by experience and somehow specific to what is being perceived” (p. 54). He clarifies the functions of schemata by means of two analogies. In one sense, a schema is like a format in a computer programming language in that it accepts information but only in an appropriate form, otherwise it is ignored or leads to meaningless results. However, schemata do not merely accept input, they also lead to actions, and as such they are like the plans of Miller, Galanter and Pribram (1960). Neisser’s account of imaging is that images are “the anticipafory phases” of perceiving. They involve the activation of a schema independent of the usual cycle in which the schema plays a part. Thus “when you have an image of a unicorn at your elbow . . . you are making ready to pick up the visual information that the unicorn would provide” (p. 132). What are the advantages of Neisser’s account of imaging? Apart from the obvious one of making imaging compatible with his general theory of cognition he claims that it avoids the difficulty, inherent in both the analogue and propositional accounts, of explaining why images and percepts are not systematically confused. Neisser claims that his account escapes this problem because perceiving involves the continuous pickup of new information which is absent in imaging. It also escapes the problem of the observer of the image in the analogue account. Unfortunately, there are several weaknesses in Neisser’s theory of imaging. First it does not necessarily escape his own criticisms of the alternative accounts. The clear separations of images and percepts is one of the grounds for acceptance of the new theory. However, knowing that one is experiencing information pickup implies knowledge about one’s cognitive processing. If this knowledge is allowable in the case of his own account Neisser
A criticism of
Neisser’s theory of imagery
81
must allow it for the other accounts. Once knowledge of input or non input of information into the system from external sources is accepted, then any of the models can cope with the separation of image and percept. We, incidentally, believe that the separation of image and percept is not invariably possible, (e.g., hallucinations) and a complete model of imagery must allow for the occasional error. Neisser also hopes to avoid problems concerning the nature of the observer of the image. However, similar problems arise in accounting for who it is who would be aware of the presence or absence of information input and hence the distinction between images and percepts. Troublesome as the threat of an homonculus may be, it is better to face it and attempt to develop an adequate account for its apparent occurrences rather than attempt to define it out of existence. Another example of Neisser’s attempts to escape the homonculus problem via definition is his treatment of percepts. We maintain that percepts can be examined and their appearance described. For example, short sighted people can describe their percept as blurred. Neisser’s strategy in this case is to define “the “percept” . . . as the particular object that would most nearly provide the same information if it were actually present” (p. 32). This is a misleading definition. We will return to this point later when discussing introspection. Neisser places considerable emphasis on imaging being an anticipation of perception. In doing so he resembles Ryle (1949) who attempted to account for imaging as a disposition to see the thing imagined. The methods by which both authors derive this conclusion are also remarkably similar. Neisser argues from the employment of cognitive maps as unsupported anticipations during locomotion to imagery; Ryle claims that having an imaginary tune run in one’s head is similar to following a real tune. Both subsume imaging into the wider class of imaginings. Hannay’s (1971) criticism of Rule’s derivation of imagery from perception is thus appropriate to Neisser. He maintains that the two types of anticipation, imaginal and perceptual, are totally different in kind. Unfulfilled perceptual anticipations are well described in terms of disappointment or surprise. Yet such qualifications are wholly inappropriate as descriptions of images. While attempting to form an image, one may be said to be anticipating the formation of the image, but one is not anticipating a true perceptual experience of the thing imagined. Indeed, one would be surprised if such a true perceptual experience did follow imaging. Neisser’s use of “anticipation” is extremely misleading, since, at best, only a low level schema in the hierarchical system of schemata can be said to anticipate. Neisser restricts discussion to these levels because he has no room for any internal representations in his account. Further, while an anticipation may be a necessary part of the imaging process it is clearly not a sufficient one,
82
P. J. Hampson and P. E. Morris
since people can anticipate seeing without experiencing a mental image. Returning home one anticipates seeing one’s wife or one’s friends, but does not constantly experience an image of them. According to Neisser, anticipatory schemata are continually being used as the very essence of perception. They must be extremely rapidly and easily activated. Why then does it take effort and time to generate a mental image? Morris and Reid (1973) reported that the latency of forming an image to the referent of a concrete noun averaged over one second, while images incorporating two referents took over two seconds. Other difficulties concerning the time taken in imaging arise when we consider the inability of Neisser’s account to cope with the manipulation of images that have been studied by Cooper and Shepard (1973) and Kosslyn (1975). During normal perception, schemata are modified through the input of information from the real world and direct further exploration by acting as plans. By definition, no such input from external sources takes place during imaging, and the plan function is not discussed by Neisser, yet the ability to manipulate images is well established. It is difficult to see how a given schema could update or modify itself without exploration and input of information. Neisser could argue, of course, that higher order schemata control the ones used in imaging. This could mean one of two things: an image scheme might accept and explore stored information from a higher order schema; this would be tantamount to reintroducing an internal representation - a strategy that Neisser hopes to avoid. Alternatively the higher order schemata might simply “activate” or “switch on” the lower order schemata without providing any image referent specifying information; in which case, the problem of modification without input remains. In fact, Neisser’s solution to the problem is to suggest that the schemata used for, say, the rotation of images are of the type used when we perceive rotating objects. No explanation is given of how these “rotation schemata” might differ from other types. If they also modify themselves by information uptake and exploration then the problem of manipulation still remains. On the other hand, if image manipulation simply involves switching through a preexisting set of expectations, then how is new knowledge derived, or better still, “realized” by means of imaging? Neisser himself recognizes the weakness in his account. Image rotation takes place at a slow and regular rate, while we can perceive all kinds of speeds of rotation. Also, Hochberg and Gellman (1977) have shown that rotation rates are faster for those stimuli which contain “landmark” or orientation features. Without some appeal to properties of an image or internal representation it is hard to see why one rotation should be harder and take longer to perform than another.
A criticism of Neisser’s theory of imagery
83
The crucial aspect of imaging is the conscious experience. It is only the experience itself which makes imaging different from just knowing and from simply anticipating. For example, a person may know that the color of his front door is white, that it has a glass panel and so on. However, when he images the front door he does something more than just know these things. He has a conscious experience which at least partially resembles experiencing the door in real life, and he can give a description of the door using the terms normally used when describing the real world. Neisser does appreciate the close similarity of perceptual and imaginal experiences by claiming that imaging involves perceptual schemata. Yet, his account tends to shift our attention away from the real experience of the image to the anticipation of a percept that we do not really expect in any case ! The theory tells us nothing new about the nature of imagery. It asserts the relation to perception, but that is hardly novel. Nor is it clear that it answers the problems tackled by the propositional and analogue accounts. As soon as we go beyond accepting Neisser’s cyclical theory of cognition at a general level we will have to ask questions about how a specific schema is activated, what it is for a schema to be activated, what are schemata and so on. Further, how is one schema selected from the enormous number available for activation in imaging. Some answers will be necessary for such problems and it was to deal with such questions of image storage, construction and retrieval that the propositional account was formulated. Neisser’s elaboration of the nature of schemata would probably differ markedly from the propositional model, yet the need for elaboration remains. As, described earlier, Neisser does not cope adequately with the manipulation or use of images in cognition and, so, he is faced with the same problems that have led other researchers to prefer analogue models. The lack of specificity in Neisser’s theory leaves ample room for elaboration, but we feel that such elaboration is likely to destroy the very qualities of the theory which Neisser finds attractive. Finally, we must consider Neisser’s discussion of introspection. Like others, including Skinner and Ryle, before him, he blames dualistic metaphysics for the difficulties usually associated with introspection. “We seem to be describing something that is inside us rather than in our environment, something examinable by the mind’s eye and not by the body’s” (p. 172). “Children have not absorbed dualistic metaphysics so ‘Children do not have these hang ups’.” And “What seem to us like descriptions of images and cognitive maps are really . . . descriptions of potentially perceivable objects, of what one would see if such-and-such a thing were present. Introspection is a kind of preparation for exterospection” (p. 173). We are not sure where in the environment Neisser would locate pleasures, pains and other emotional experiences! No mention is made of these. At best,
84
P. J. Hampson
anu’ P. E. Morris
his account is only appropriate to some objects of introspection. The pink elephant that I claim to be imaging may be potentially perceivable as an object. However, this is irrelevant to whether or not I am consciously experiencing and describing an image of a pink elephant, or merely describing what a pink elephant might look like, without any such mental image. Only the individual giving the description knows whether he is experiencing an image or not. Nor does the fact that we use terms applicable to real world objects in any way imply that we are describing those same objects. What other words could we possibly use? It is true that there might be oblique references to the real world, in that, perhaps, any description of a visual mental experience could be translated into the description of a real thing, if such a thing existed, but, surely, this is little more than a communication device. The real intention of the introspector is to describe his conscious experiences. Also, the fact that certain terms applicable to mental experiences, such as “vivid”, “faithful” of “fuzzy”, are inapplicable ro real world objects suggests a difference between introspection and exterospection. We describe the quality of an image just as we might describe the texture of paint or the detail in a painting. Note, that we are not suggesting that introspection is error free. In common with Natsoulas (1970) we consider introspection to be the acquiring of beliefs, rather than knowledge, concerning one’s mental operations and experiences. Our conclusion is that whatever may be the merits of Neisser’s cyclical theory, its application to imagery adds nothing new to our understanding of the phenomenon, and may even prove detrimental if it distracts attention from the interesting and centrally important properties of the conscious experience itself.
References Cooper,
L. A. and Shepard, R. N. (1973) Chronometric studies of the rotation of mental images. In W. G. Chase (Ed.), Visual information processing. New York, Academic Press. Hannay, A. (1971). Mental images A defence. London, George Allen & Unwin. Hochberg, J. and Gellman, L. (1977). The effect of landmark features on “mental rotation” times. Mem. Cog., 5, 23-26. Kosslyn, S. M. (1975). Information representation in visual images. Cog. Psychol., 7, 341-370. Kosslyn, S. M. and Pomerantz, J. R. (1977). Imagery, propositions and the form of internal representations. Cog. Psychol., 9 52-76. Morris, P. E. and Reid, R. L. (1973). Recognition and recall: latency and recurrence of images. Er. J. Psychob, 64, 161-167. Natsoulas, T. (1970). Concerning introspective “knowledge”. Psychol. Bull, 73, 89-l 11. Neisser, U. (1976). Cognition and realify. San Francisco, Freeman.
A criticism of Neisser’s theory of imagery
85
Pylyshyn, Z. W. (1973). What the mind’s eye tells the mind’s brain: a critique of mental imagery. Psychol. Bull., 80, I-24. Pylyshyn, Z. W. (1975). Do we need images and analogues. Paper presented at the Conference on Theoretical Issues in Natural Language Processing. Massachusetts Institute of Technology. Boston, Mass. Ryle, G. (1949). The concepf ofmind. London, Hutchinson. Shepard, R. N. (1975). Form, formation and transformation of internal representations. In R. Solso (Ed.), Information processing the Loyola Symposium. Hillsdale, NJ., Lawrence Erlbaum.