- Email: [email protected]
Literature from cognitive psychology
ARTIFICIAL INTELLIGENCE
251
CORRESPONDENT'S REPORT
Literature from Cognitive Psychology Zenon W. Pylyshyn University of Western Ontario, London, Ontario, Canada
Cognitive psychology and artificial intelligence are in many respects indistinguishable (a claim I argued in Pylyshyn (1979)). Furthermore cognitive psychologists are extremely plentiful and are professionally a very productive breed. Consequently it requires more than a little gall and pretention to set out to c o m m e n t on some papers that A I people may find useful: There is no doubt someone who would find something of interest in almost every paper in the field. The best thing I could do then is to simply list all the best papers and r e c o m m e n d them to you. But not even I am so foolhardy as to pretend to that task. All I can do by way of a rather unsatisfactory compromise is to give you a very abbreviated skim based solely on what I happen to have been thinking about recently, which is naturally conditioned not only by my biased view of the field but also by what research I have been interested in this week. That accounts for why some of my own papers are on this list: its not that they are particularly good, it is just that I tend to think about them more frequently than some other papers! First a general remark about sources. Cognitive psychology research is reported in very many journals inasmuch as it covers a wide range of topics, from cognitive neuroscience to cognitive anthropology, including various specialized areas such as vision, audition, psychophysics, speech, psycholinguistics, h u m a n - m a c h i n e interactions, education, and so on, each of which has its own special journals. There are a number of journals, however, that are devoted to reporting research in the general area of cognition--especially as it is practiced in the approach known as 'cognitive science'. A m o n g those that the reader should look at in searching for psychological ideas are Cognitive Psychology (the second most cited journal in all of psychology!), Cognition, Cognitive Artificial Intelligence 19 (1982) 251-255 0004-3702/82/0000-0000/$02.75 O 1982 North-Holland
252
Z.W. PYLYSHYN
Science, Memory and Cognition, Journal of Experimental Psychology: Human Perception and Performance, and the extremely far-ranging journal of reviews and peer commentary, Behavioral and Brain Science. In addition if you want to get an overview of a wide range of research approaches and their motivations read the special tenth anniversary issue of Cognition (Vol. 10, 1981).
1. Modularity A hot topic in certain corners of both cognitive psychology and cognitive neuroscience is what has come to be called 'The Modularity of Mind' (after Jerry Fodor's widely circulated m a n u s c r i p t - - s o o n to be an M I T / B r a d f o r d Book by the same title). This is the proposal that the cognitive system is made up of a number of what might be thought of as compiled which work autonomously and communicate only in a limited way with a general reasoner. The best evidence for such modules comes from psychophysical research in low-level vision (e.g. Marr, 1982; Ullman, 1980) and psycholinguistic studies of sentence processing (e.g. Forster (1979) and other papers in Cooper and Walker (1979) including lexical lookup (e.g. Swinney, 1979). What these studies suggest is that there are certain processes whose operation appears to be impervious to context or to knowledge of the world: they are what I have called 'cognitively impenetrable' (Pylyshyn, 1980, 1981). O t h e r related studies (Shiffrin and Schneider, 1977) have focussed on the question of which processes are 'automatic' in the sense of being both data-driven and non-resource-consuming, and which processes require the application of resource limited attention. Although such modules are being explored both theoretically and empirically with considerable success in the case of input and output systems, there is little or no evidence for them in the case of central cognitive processes. The central processes most frequently investigated from this standpoint are those involved in imaginal reasoning, understanding (of stories, dialogues, and so on) and long term memory. Although regularities are found in all areas, and although it seems clear that such processes must have some properties that are special and knowledge-independent (i.e. reflect properties of the functional architecture), so far I think it is safe to say that in the case of central processes none of the proposals for what these properties are have withstood close scrutiny (see the discussion on this issue in Pylyshyn (1981)). It appears that (apart from some general and fairly easily alterable habits that people have developed for dealing with certain kinds of problems), most regularities in these areas are cognitively penetrable and best attributable to the operation of common-sense reasoning based on people's beliefs about the world, rather than in terms of modular processes built in (or compiled in) to the functional architecture as is the case with the input systems that have been studied.
CORRESPONDENT'S REPORT
253
2. Resource-limited Processing The idea of resource-limited processes is well established in psychology. Such processes are typically ones that are identified as being serial since when resources are limited, the usual way to deal with the limitation is to trade off the resource (e.g. memory) with time, creating a sequential process. While some people have treated resource limits merely as an aggregate parameter (e.g. Norman and Bobrow, 1975), there have also been a variety of proposals to explain the source of the limitation in computational terms. The most common assumption has been that only a small number of symbols can be attended to at one time, either because the size of short-term memory is fixed (as in the classical proposal of Miller (1956) or the development of this idea by Newell (1973)), or because only a fixed number of memory nodes can be activated at one time (e.g. Anderson, 1976). Other proposals include Newell's (1980) identification of the computational cost with the cost of a pattern match involving a variable in the pattern of a production and, in the case of low-level vision, Pylyshyn, Elcock, Marmor and Sander's (1978) proposal that there is a limited number of symbols available for internal referencing of intrinsic scene features (a precondition for evaluating any visual predicates over these features). Recently, evidence has been found for the involvement of resource-limited attentional processes in what might seem like rather low-level vision tasks. Triesman (1982) demonstrated the existence of certain perceptual illusions called 'conjunction illusions' (where a green A and a red B are misperceived as a red A and a green B), and Rock and Gutman (1981) have shown failures to recognize simple geometrical patterns, both of which occur when attention is being drawn away by another visual task. The whole question of which processes are automatic, nonattentional and cognitively impenetrable and which processes involve the use of limited attentional resources or knowledge is still very much a research frontier, but the pendulum (at least for input processes) appears to be swinging away from the popular top-down view that had everyone in psychology totally convinced in the 1950's (with the movement known as the 'new look in perception') and had the AI community convinced in the early 1970's (where the slogan 'heterarchy not hierarchy' played the same role).
3. Learning The same sort of pendulum swing can also be discerned in the study of learning and development. In developmental psychology more and more studies are showing that newborn infants' cognitive (though more especially, perceptual) capacities are much greater than most people had believed. Studies by Bower (1979), Spelke (in press) and others reported in Mehler (in press) show an
254
z.w. PYLYSHYN
extremely high level of structure in the 'initial state' of the cognitive system--a message that ought not to be lost on those members of the AI community trying to develop learning systems. The best theoretical analysis of the problems relating learning to initial state structure is now coming from psychologists interested in language learning (e.g. Gleitman, 1981) and particularly in the theory of learnability (Wexler and Culicover, 1980), though some work has also been done on the innate contraints on the learning of concepts (Osherson, 1978; Keil, 1979).
4. Applied Cognitive Science The most directly relevant psychological research, from the point of view of many people in AI developing expert systems, is relatively new in psychology. It concerns the study of how people understand anything from the operation of hand calculators (Young, 1981), elementary physics (see below)or mathematics (Brown and VanLehn, 1980; Young and O'Shea, 1981) to problems in general (Hayes and Simon, 1976; Greeno, 1977). Another area of considerable relevance to AI is the study of the so-called 'novice-expert shift' which compares the representation of specific task-relevant knowledge in experts and novices. There now exists a large and growing literature on this subject, much of which is published in Cognitive Science (e.g. Chi, Feltovich and Glazer, 1981; Johnson et al., 1981; Larkin, McDermott, Simon and Simon, 1980). Most of this work is straight-line cognitive science and probably very well known to people in AI, though it is worth mentioning here because it is so directly relevant. A large comprehensive review of research on expertise in problem solving will also be available shortly (Chi, Glaser and Rees, in press). REFERENCES 1. Anderson, J.R., Language, Memory, and Thought (Erlbaum, Hillsdale, N J, 1976). 2. Bower, T.G.R., Human Development (Freeman, San Fransisco, 1979). 3. Brown, J. and VanLehn, K., Repair theory: A generative theory of bugs in procedural skills, Cognitive Sci. 4 (1980) 379-426. 4. Chi, M.T.H., Feltovich, P.J. and Glaser, R., Categorization and representation of physics problems by expert and novices, Cognitive Sci. 5 (1981) 121-152. 5. Chi, M.T.H., Glaser, R. and Rees, E., Expertise in problem solving, in: R. Sternberg Ed., Advances in the Psychology of Intelligence (Erlbaum, Hillsdale, in press). 6. Cooper, w . and Walker, E. Eds., Sentence Processing: Psycholinguistic Studies Presented to Merrill Garrett (Erlbaum, Hillsdale, NJ, 1979). 7. Forster, K., Levels of processing and the structure of the language processor, in: W.E. Cooper and E.C.T. Walker Eds., Sentence Processing: Psycholinguistic Studies Presented to Merrill Garrett (Erlbaum, Hillsdale, NJ, 1979). 8. Gleitman, L., Maturational determinants of language growth, Cognition 10 (1981) 103-114. 9. Greeno, J., Process of understanding in problem solving, in: N.J. Castellan, D.B. Pisoni and G.R. Potts Eds., Cognitive Theory, Vol. 2 (Erlbaum, Hillsdale, NJ, 1977). 10. Hayes, J.R. and Simon, H.A., The understanding process: Problem isomorphs, Cognitive Psychol. 8 (1976) 165-190.
CORRESPONDENT'S REPORT
255
11. Johnson, P.E., Duran, A.S., Hassebrock, F., Moiler, J., Prietula, M., Feltovich, P.J. and Swanson, D.B., Expertise and error in diagnostic reasoning, Cognitive Sci. 5 (1981) 235-283. 12. Keil, F., Semantic and Conceptual Development: An Ontological Perspective (Harvard Univ. Press, Cambridge, 1979). 13. Marr, D., Vision: A Computational Investigation into the Human Representation and Processing of Visual Information (Freeman, San Francisco, 1982). 14. Mehler, J. Ed., Perspective in Cognitive Psychology (in press). 15. Miller, G., The magical number seven, plus or minus two: Some limits on our capacity for processing information, Psychol. Rev. 63 (1956) 81-97. 16. Newell, A., Production systems: Models of control structures, in: W.G. Chase Ed., Visual Information Processing (Academic Press, New York, 1973). 17. Newell, A., Harpy, production systems and human cognition, in: R. Cole Ed., Perception and Production of Fluent Speech (Erlbaum, Hillsdale, N J, 1980). 18. Norman, D. and Bobrow, D., On data-limited and resource-limited processes, Cognitive Psychol. 7 (1975) 44-64. 19. Osherson, D., Three conditions on conceptual naturalness, Cognition 6 (1978) 263-289. 20. Pylyshyn, Z.W., Computational models and empirical constraints, Behav. Brain Sci. 1 (1978) 93-127. 21. Pylyshyn, Z.W., Computation and cognition: Issues in the foundations of cognitive science, Behav. Brain Sei. 3 (1980) 111-169. 22. Pylyshyn, Z.W., The imagery debate: Analogue media versus tacit knowledge, Psychol. Rev. 88 (1981) 16-45. 23. Pylyshyn, Z.W., Elcock, E.W., Marmor, M. and Sander, P., Explorations in perceptual-motor spaces, Proc. Second Internat. Conf. of the Canadian Society for Computational Studies of Intelligence, Department of Computer Science, University of Toronto, 1978. 24. Rock, I. and Outman, D., The effect of inattention on form perception. J. Experimental Psychol. 7 (1981) 275-285. 25. Shitirin, R.M. and Schneider, W., Controlled and automatic information processing: II. Perceptual learning, automatic attending, and a general theory. Psychol. Rev. 84 (1977) 129-190. 26. Spelke, E., Perceptual knowledge of objects in infancy, in: J. Mehler Ed., Perspectives in Cognitive Psychology (in press). 27. Swinney, D., Lexical access during sentence comprehension: (Re)consideration of context effects, J. verb. Learn. verb. Behav. 18 (1979) 645-660. 28. Triesman, A. and Schmidt, H., Illusory conjunctions in the perception of objects, Cognitive Psychol. 14 (1982) 107-141. 29. Ullman, S., The Interpretation of Visual Motion (MIT Press, Cambridge, MA, 1980). 30. Young, R., The machine inside the machine: Users' models of pocket calculators. Internat. J. Man-Mach. Stud. 15 (1981) 51-85. 31. Young, R. and O'Shea, T., Errors in children's subtraction, Cognitive Sci. 5 (1981) 153-177.
251
CORRESPONDENT'S REPORT
Literature from Cognitive Psychology Zenon W. Pylyshyn University of Western Ontario, London, Ontario, Canada
Cognitive psychology and artificial intelligence are in many respects indistinguishable (a claim I argued in Pylyshyn (1979)). Furthermore cognitive psychologists are extremely plentiful and are professionally a very productive breed. Consequently it requires more than a little gall and pretention to set out to c o m m e n t on some papers that A I people may find useful: There is no doubt someone who would find something of interest in almost every paper in the field. The best thing I could do then is to simply list all the best papers and r e c o m m e n d them to you. But not even I am so foolhardy as to pretend to that task. All I can do by way of a rather unsatisfactory compromise is to give you a very abbreviated skim based solely on what I happen to have been thinking about recently, which is naturally conditioned not only by my biased view of the field but also by what research I have been interested in this week. That accounts for why some of my own papers are on this list: its not that they are particularly good, it is just that I tend to think about them more frequently than some other papers! First a general remark about sources. Cognitive psychology research is reported in very many journals inasmuch as it covers a wide range of topics, from cognitive neuroscience to cognitive anthropology, including various specialized areas such as vision, audition, psychophysics, speech, psycholinguistics, h u m a n - m a c h i n e interactions, education, and so on, each of which has its own special journals. There are a number of journals, however, that are devoted to reporting research in the general area of cognition--especially as it is practiced in the approach known as 'cognitive science'. A m o n g those that the reader should look at in searching for psychological ideas are Cognitive Psychology (the second most cited journal in all of psychology!), Cognition, Cognitive Artificial Intelligence 19 (1982) 251-255 0004-3702/82/0000-0000/$02.75 O 1982 North-Holland
252
Z.W. PYLYSHYN
Science, Memory and Cognition, Journal of Experimental Psychology: Human Perception and Performance, and the extremely far-ranging journal of reviews and peer commentary, Behavioral and Brain Science. In addition if you want to get an overview of a wide range of research approaches and their motivations read the special tenth anniversary issue of Cognition (Vol. 10, 1981).
1. Modularity A hot topic in certain corners of both cognitive psychology and cognitive neuroscience is what has come to be called 'The Modularity of Mind' (after Jerry Fodor's widely circulated m a n u s c r i p t - - s o o n to be an M I T / B r a d f o r d Book by the same title). This is the proposal that the cognitive system is made up of a number of what might be thought of as compiled which work autonomously and communicate only in a limited way with a general reasoner. The best evidence for such modules comes from psychophysical research in low-level vision (e.g. Marr, 1982; Ullman, 1980) and psycholinguistic studies of sentence processing (e.g. Forster (1979) and other papers in Cooper and Walker (1979) including lexical lookup (e.g. Swinney, 1979). What these studies suggest is that there are certain processes whose operation appears to be impervious to context or to knowledge of the world: they are what I have called 'cognitively impenetrable' (Pylyshyn, 1980, 1981). O t h e r related studies (Shiffrin and Schneider, 1977) have focussed on the question of which processes are 'automatic' in the sense of being both data-driven and non-resource-consuming, and which processes require the application of resource limited attention. Although such modules are being explored both theoretically and empirically with considerable success in the case of input and output systems, there is little or no evidence for them in the case of central cognitive processes. The central processes most frequently investigated from this standpoint are those involved in imaginal reasoning, understanding (of stories, dialogues, and so on) and long term memory. Although regularities are found in all areas, and although it seems clear that such processes must have some properties that are special and knowledge-independent (i.e. reflect properties of the functional architecture), so far I think it is safe to say that in the case of central processes none of the proposals for what these properties are have withstood close scrutiny (see the discussion on this issue in Pylyshyn (1981)). It appears that (apart from some general and fairly easily alterable habits that people have developed for dealing with certain kinds of problems), most regularities in these areas are cognitively penetrable and best attributable to the operation of common-sense reasoning based on people's beliefs about the world, rather than in terms of modular processes built in (or compiled in) to the functional architecture as is the case with the input systems that have been studied.
CORRESPONDENT'S REPORT
253
2. Resource-limited Processing The idea of resource-limited processes is well established in psychology. Such processes are typically ones that are identified as being serial since when resources are limited, the usual way to deal with the limitation is to trade off the resource (e.g. memory) with time, creating a sequential process. While some people have treated resource limits merely as an aggregate parameter (e.g. Norman and Bobrow, 1975), there have also been a variety of proposals to explain the source of the limitation in computational terms. The most common assumption has been that only a small number of symbols can be attended to at one time, either because the size of short-term memory is fixed (as in the classical proposal of Miller (1956) or the development of this idea by Newell (1973)), or because only a fixed number of memory nodes can be activated at one time (e.g. Anderson, 1976). Other proposals include Newell's (1980) identification of the computational cost with the cost of a pattern match involving a variable in the pattern of a production and, in the case of low-level vision, Pylyshyn, Elcock, Marmor and Sander's (1978) proposal that there is a limited number of symbols available for internal referencing of intrinsic scene features (a precondition for evaluating any visual predicates over these features). Recently, evidence has been found for the involvement of resource-limited attentional processes in what might seem like rather low-level vision tasks. Triesman (1982) demonstrated the existence of certain perceptual illusions called 'conjunction illusions' (where a green A and a red B are misperceived as a red A and a green B), and Rock and Gutman (1981) have shown failures to recognize simple geometrical patterns, both of which occur when attention is being drawn away by another visual task. The whole question of which processes are automatic, nonattentional and cognitively impenetrable and which processes involve the use of limited attentional resources or knowledge is still very much a research frontier, but the pendulum (at least for input processes) appears to be swinging away from the popular top-down view that had everyone in psychology totally convinced in the 1950's (with the movement known as the 'new look in perception') and had the AI community convinced in the early 1970's (where the slogan 'heterarchy not hierarchy' played the same role).
3. Learning The same sort of pendulum swing can also be discerned in the study of learning and development. In developmental psychology more and more studies are showing that newborn infants' cognitive (though more especially, perceptual) capacities are much greater than most people had believed. Studies by Bower (1979), Spelke (in press) and others reported in Mehler (in press) show an
254
z.w. PYLYSHYN
extremely high level of structure in the 'initial state' of the cognitive system--a message that ought not to be lost on those members of the AI community trying to develop learning systems. The best theoretical analysis of the problems relating learning to initial state structure is now coming from psychologists interested in language learning (e.g. Gleitman, 1981) and particularly in the theory of learnability (Wexler and Culicover, 1980), though some work has also been done on the innate contraints on the learning of concepts (Osherson, 1978; Keil, 1979).
4. Applied Cognitive Science The most directly relevant psychological research, from the point of view of many people in AI developing expert systems, is relatively new in psychology. It concerns the study of how people understand anything from the operation of hand calculators (Young, 1981), elementary physics (see below)or mathematics (Brown and VanLehn, 1980; Young and O'Shea, 1981) to problems in general (Hayes and Simon, 1976; Greeno, 1977). Another area of considerable relevance to AI is the study of the so-called 'novice-expert shift' which compares the representation of specific task-relevant knowledge in experts and novices. There now exists a large and growing literature on this subject, much of which is published in Cognitive Science (e.g. Chi, Feltovich and Glazer, 1981; Johnson et al., 1981; Larkin, McDermott, Simon and Simon, 1980). Most of this work is straight-line cognitive science and probably very well known to people in AI, though it is worth mentioning here because it is so directly relevant. A large comprehensive review of research on expertise in problem solving will also be available shortly (Chi, Glaser and Rees, in press). REFERENCES 1. Anderson, J.R., Language, Memory, and Thought (Erlbaum, Hillsdale, N J, 1976). 2. Bower, T.G.R., Human Development (Freeman, San Fransisco, 1979). 3. Brown, J. and VanLehn, K., Repair theory: A generative theory of bugs in procedural skills, Cognitive Sci. 4 (1980) 379-426. 4. Chi, M.T.H., Feltovich, P.J. and Glaser, R., Categorization and representation of physics problems by expert and novices, Cognitive Sci. 5 (1981) 121-152. 5. Chi, M.T.H., Glaser, R. and Rees, E., Expertise in problem solving, in: R. Sternberg Ed., Advances in the Psychology of Intelligence (Erlbaum, Hillsdale, in press). 6. Cooper, w . and Walker, E. Eds., Sentence Processing: Psycholinguistic Studies Presented to Merrill Garrett (Erlbaum, Hillsdale, NJ, 1979). 7. Forster, K., Levels of processing and the structure of the language processor, in: W.E. Cooper and E.C.T. Walker Eds., Sentence Processing: Psycholinguistic Studies Presented to Merrill Garrett (Erlbaum, Hillsdale, NJ, 1979). 8. Gleitman, L., Maturational determinants of language growth, Cognition 10 (1981) 103-114. 9. Greeno, J., Process of understanding in problem solving, in: N.J. Castellan, D.B. Pisoni and G.R. Potts Eds., Cognitive Theory, Vol. 2 (Erlbaum, Hillsdale, NJ, 1977). 10. Hayes, J.R. and Simon, H.A., The understanding process: Problem isomorphs, Cognitive Psychol. 8 (1976) 165-190.
CORRESPONDENT'S REPORT
255
11. Johnson, P.E., Duran, A.S., Hassebrock, F., Moiler, J., Prietula, M., Feltovich, P.J. and Swanson, D.B., Expertise and error in diagnostic reasoning, Cognitive Sci. 5 (1981) 235-283. 12. Keil, F., Semantic and Conceptual Development: An Ontological Perspective (Harvard Univ. Press, Cambridge, 1979). 13. Marr, D., Vision: A Computational Investigation into the Human Representation and Processing of Visual Information (Freeman, San Francisco, 1982). 14. Mehler, J. Ed., Perspective in Cognitive Psychology (in press). 15. Miller, G., The magical number seven, plus or minus two: Some limits on our capacity for processing information, Psychol. Rev. 63 (1956) 81-97. 16. Newell, A., Production systems: Models of control structures, in: W.G. Chase Ed., Visual Information Processing (Academic Press, New York, 1973). 17. Newell, A., Harpy, production systems and human cognition, in: R. Cole Ed., Perception and Production of Fluent Speech (Erlbaum, Hillsdale, N J, 1980). 18. Norman, D. and Bobrow, D., On data-limited and resource-limited processes, Cognitive Psychol. 7 (1975) 44-64. 19. Osherson, D., Three conditions on conceptual naturalness, Cognition 6 (1978) 263-289. 20. Pylyshyn, Z.W., Computational models and empirical constraints, Behav. Brain Sci. 1 (1978) 93-127. 21. Pylyshyn, Z.W., Computation and cognition: Issues in the foundations of cognitive science, Behav. Brain Sei. 3 (1980) 111-169. 22. Pylyshyn, Z.W., The imagery debate: Analogue media versus tacit knowledge, Psychol. Rev. 88 (1981) 16-45. 23. Pylyshyn, Z.W., Elcock, E.W., Marmor, M. and Sander, P., Explorations in perceptual-motor spaces, Proc. Second Internat. Conf. of the Canadian Society for Computational Studies of Intelligence, Department of Computer Science, University of Toronto, 1978. 24. Rock, I. and Outman, D., The effect of inattention on form perception. J. Experimental Psychol. 7 (1981) 275-285. 25. Shitirin, R.M. and Schneider, W., Controlled and automatic information processing: II. Perceptual learning, automatic attending, and a general theory. Psychol. Rev. 84 (1977) 129-190. 26. Spelke, E., Perceptual knowledge of objects in infancy, in: J. Mehler Ed., Perspectives in Cognitive Psychology (in press). 27. Swinney, D., Lexical access during sentence comprehension: (Re)consideration of context effects, J. verb. Learn. verb. Behav. 18 (1979) 645-660. 28. Triesman, A. and Schmidt, H., Illusory conjunctions in the perception of objects, Cognitive Psychol. 14 (1982) 107-141. 29. Ullman, S., The Interpretation of Visual Motion (MIT Press, Cambridge, MA, 1980). 30. Young, R., The machine inside the machine: Users' models of pocket calculators. Internat. J. Man-Mach. Stud. 15 (1981) 51-85. 31. Young, R. and O'Shea, T., Errors in children's subtraction, Cognitive Sci. 5 (1981) 153-177.