- Email: [email protected]
Connectionism and psychology: A psychological perspective on new connectionist research
178
Book reviews /Ada
Psychologica 85 (1994) 171-181
SSDI OOOl-6918(93)E0047-6
Philip Quinlan, Connectionism and Psychology : A Psychological Perspective on New Connectionist Research. Harvester Wheatsheaf, New York, 1991. ISBN 0-74500834-8. Many researchers in the fashionable field of neural network models or connectionism only pay lip service to psychology, but are often more concerned with considerations of practical (even commercial) applicability, mathematical tractability, philosophical significance, and programming elegance than by the psychological plausibility of their models. Although these may be interesting goals by themselves, many connectionists tend to forget that the primary goal of the approach is to simulate human functions, such as perception, attention, language, learning and memory (and perhaps even emotion and motivation, see Levine and Leven, 19921, while using elementary (microscopic) mechanisms and functions that have been inspired by the nervous system. Connectionism’s greatest strength probably lies in the opportunity it provides to couple (without fully reducing) macroscopic behavior to neural functions, much in the spirit of the ‘Cognitive Neuroscience’ approach (Gazzaniga, 1984) which is rapidly gaining field and which eventually may prove to form the deeper rationale of the connectionist movement. Without such coupling we may end up with a situation in psychology where several rival theories explain the behavioral data equally well but cannot be decided between without looking inside the ‘black box’ (Crick, 1979). The computational relevance of neural properties on the other hand may not become apparent unless we do simulations with them and see what collective network behavior they result in. If connectionism continues to drift sideways, it risks the same fate as the artificial intelligence approach and may, in fact, become indistinguishable from it. Connectionism and Psychology by Philip Quinlan has the great merit of stressing the importance of psychological, as opposed to mathematical, physical, philosophical, or engineering, issues in neural network models. Though there are many introductory neural network books available, only very few are suited as a textbook for teaching connectionism to second or third year psychology students. The only other books that come to mind are either too philosophically oriented, too biased towards a particular approach, or try to introduce experimental psychology from a connectionist point of view instead of the other way around. This is the only book I know of that introduces a lot of specific connectionist models (about 40 of them) and then evaluates them in a psychological context. It has the further advantage of providing a more or less coherent and balanced picture of the whole approach, instead of isolating very specific topics and treating them in depth as is done in the famous PDP volumes. The fact that the author sometimes expresses strong personal opinions, which I often find convincing, contributes to this coherence and makes the book very clear and readable and, at least in the course I gave, had the effect of inviting students to formulate their own opinions. The psychological orientation is strongly reflected by the organization of the book. The middle four chapters are each concerned with models for specific human functions, like learning and memory, vision, language, and higher-order
Book reviews /Acta Psychologica 85 (1994) 17X-181
179
aspects of cognition. The first chapter provides a historical perspective on the development of connectionism. The last chapter discusses a number of criticisms that have been raised against connectionism from psychological, neurobiological, and philosophical quarters. Though serious limitations of connectionism are set out and the author adds some severe criticisms, the tone of this book is more positive towards connectionism than from most of the critics discussed in the chapter. Quinlan ends on a cautiously hopeful note about the future and indeed the necessity of connectionist research. Though I think this is by far the best introductory book on connectionism I have seen, it also has some notable shortcomings, as could be expected from such an ambitious enterprise. The book appears to have grown from some short evaluation of connectionism or possibly even from a book review. Its evaluative character makes this book very lively and stimulates the formation of opinions by the reader, but particularly towards the end the explanation of the models becomes progressively shorter, and the comments on the models disproportionately long. Chapter 5 on higher-order aspects of cognition, in particular, is in my opinion one of the weakest chapters of the book, where it sometimes becomes impossible to understand essential features of the models without having read the original publications. If indeed the book started as a book review, most of the original text is probably to be found in the first chapter. Here a very good historical introduction is given and a discussion of the first connectionist wave together with the circumstances that led to its dismissal. Quinlan convincingly shows that this dismissal was based more on a curious sociological phenomenon in the scientific community than on a careful evaluation of the criticisms. Though the next chapter on learning and memory models closely follows the introductory chapter by introducing learning procedures which partly circumvent some of these criticisms, there is a remarkable discrepancy between this chapter and the following three chapters. Whereas these latter three chapters discuss models which actually simulate psychological phenomena (i.e., also experimental results), the second chapter presents models that, in principle, are able to learn and retain information, but simulations of actual results in the field of learning and memory are neglected here. In this chapter the psychological emphasis of the book seems forgotten for a moment. I have been using the book in an introductory course on connectionism to the satisfaction of both the students and myself. Due to its brevity of explanation and its omission of a few important topics, however, it was not sufficient to serve as the sole text for the course, but then I know of no books that cover all aspects I think should be covered. One aspect, the practical experience with actual models, cannot be expected because it falls outside the scope of this book, though I am still waiting for a text that provides detailed explanation together with some userfriendly, preferably graphical software, that can be used by students without much programming experience. To fill this gap we have been using the MacBrain 3.0 package from Neurix which has nice graphical features, but is rather slow on the Mats we have available. To cover the essential subjects in connectionism more extensively we added a reader with a number of original articles. A function, that is fundamental to almost
180
Book reviews /Acta Psychologica 85 (1994) 171-181
all connectionist models, namely simultaneous multiple constraint satisfaction, is mentioned only briefly as a distinctive notion in connectionism (p. 2391, but is not explained in the preceding five chapters. This is clearly an important topic, which may, moreover, be related to the very psychologically relevant function of attention and which needs further elaboration in supplementary reading. A conspicuous omission in the book concerns the complete neglect of competitive learning procedures. The majority and some of the best-known unsupervised learning procedures (a term which is discussed in the book) consist of such competitive learning schemes. Moreover, after the discussion of the importance of inhibition in neural networks in the first chapter, it would be expected that in the next chapter learning procedures, in which inhibition plays an important role, would be presented, but they were not. Also the neurobiological plausibility of such learning procedures seems somewhat higher than, for instance, that of back-propagation. Neurobiological plausibility, furthermore, is an aspect that might deserve more attention than it gets in the final chapter. The link with and the inspiration by the nervous system is what distinguishes connectionism from mathematical parameter-fitting methods or artificial intelligence programs. A comparison of connectionist models to the brain and an attempt to discuss the relevance of the parallels and differences would certainly have added to the balanced nature of the book. If this were done it would, for instance, become clear that the interconnection scheme (i.e., homogeneously connected or layered networks) of most networks is highly implausible due to the unlikely high number of connections required and that some form of modularity is probable. Though modularity is discussed shortly at several places, it seems to me that this is still too little in view of the growing evidence for modularity in the brain and the enhanced interest for modularity in connectionist models (e.g., McClelland, 1992). The inclusion of still other recent developments in connectionism would greatly extend the scope of the book and might go some way towards solving a few of the unsettled issues. A more than passing discussion of ‘nets with loops’ (p. 260) could show how these newer ‘simple recurrent networks’ can deal with sequential processing and rule-governed behavior. This would improve the somewhat meagre Chapter 5 and would allow for the treatment of modelling work in the field of implicit learning (more attention to implicit memory in Chapter 2 would probably also make this chapter more psychologically relevant). Finally, some discussion of the newer work on oscillations and synchronization in neural networks might also improve the currency of this introduction into connectionism. In spite of these minor deficiencies, I feel that this is a valuable (but not expensive) introduction to and evaluation of the connectionist approach. R. Hans Phaf Psychonomics Department University of Amsterdam Roetersstraat 15 1018 WB Amsterdam The Netherlands
Book reviews /Acta Psychologica 85 (1994) I71 -181
181
1. References Crick, F.H.C., 1979. Thinking about the brain. Scientific American 241, 181-188. Gazzaniga, MS., 1984. Handbook of cognitive neuroscience. New York: Plenum. Levine, D.S. and S.J. Leven, 1992. Motivation, emotion, and goal direction in neural networks. Hillsdale, NJ: Erlbaum. McClelland, J.L., 1992. ‘Toward a theory of information processing in graded, random, and interactive networks’. In: D.E. Meyer and S. Kornblum, Attention and performance XIV, Synergies in experimental psychology, artificial intelligence, and cognitive neuroscience (pp. 655-688). Cambridge, MA: MIT Press.
Book reviews /Ada
Psychologica 85 (1994) 171-181
SSDI OOOl-6918(93)E0047-6
Philip Quinlan, Connectionism and Psychology : A Psychological Perspective on New Connectionist Research. Harvester Wheatsheaf, New York, 1991. ISBN 0-74500834-8. Many researchers in the fashionable field of neural network models or connectionism only pay lip service to psychology, but are often more concerned with considerations of practical (even commercial) applicability, mathematical tractability, philosophical significance, and programming elegance than by the psychological plausibility of their models. Although these may be interesting goals by themselves, many connectionists tend to forget that the primary goal of the approach is to simulate human functions, such as perception, attention, language, learning and memory (and perhaps even emotion and motivation, see Levine and Leven, 19921, while using elementary (microscopic) mechanisms and functions that have been inspired by the nervous system. Connectionism’s greatest strength probably lies in the opportunity it provides to couple (without fully reducing) macroscopic behavior to neural functions, much in the spirit of the ‘Cognitive Neuroscience’ approach (Gazzaniga, 1984) which is rapidly gaining field and which eventually may prove to form the deeper rationale of the connectionist movement. Without such coupling we may end up with a situation in psychology where several rival theories explain the behavioral data equally well but cannot be decided between without looking inside the ‘black box’ (Crick, 1979). The computational relevance of neural properties on the other hand may not become apparent unless we do simulations with them and see what collective network behavior they result in. If connectionism continues to drift sideways, it risks the same fate as the artificial intelligence approach and may, in fact, become indistinguishable from it. Connectionism and Psychology by Philip Quinlan has the great merit of stressing the importance of psychological, as opposed to mathematical, physical, philosophical, or engineering, issues in neural network models. Though there are many introductory neural network books available, only very few are suited as a textbook for teaching connectionism to second or third year psychology students. The only other books that come to mind are either too philosophically oriented, too biased towards a particular approach, or try to introduce experimental psychology from a connectionist point of view instead of the other way around. This is the only book I know of that introduces a lot of specific connectionist models (about 40 of them) and then evaluates them in a psychological context. It has the further advantage of providing a more or less coherent and balanced picture of the whole approach, instead of isolating very specific topics and treating them in depth as is done in the famous PDP volumes. The fact that the author sometimes expresses strong personal opinions, which I often find convincing, contributes to this coherence and makes the book very clear and readable and, at least in the course I gave, had the effect of inviting students to formulate their own opinions. The psychological orientation is strongly reflected by the organization of the book. The middle four chapters are each concerned with models for specific human functions, like learning and memory, vision, language, and higher-order
Book reviews /Acta Psychologica 85 (1994) 17X-181
179
aspects of cognition. The first chapter provides a historical perspective on the development of connectionism. The last chapter discusses a number of criticisms that have been raised against connectionism from psychological, neurobiological, and philosophical quarters. Though serious limitations of connectionism are set out and the author adds some severe criticisms, the tone of this book is more positive towards connectionism than from most of the critics discussed in the chapter. Quinlan ends on a cautiously hopeful note about the future and indeed the necessity of connectionist research. Though I think this is by far the best introductory book on connectionism I have seen, it also has some notable shortcomings, as could be expected from such an ambitious enterprise. The book appears to have grown from some short evaluation of connectionism or possibly even from a book review. Its evaluative character makes this book very lively and stimulates the formation of opinions by the reader, but particularly towards the end the explanation of the models becomes progressively shorter, and the comments on the models disproportionately long. Chapter 5 on higher-order aspects of cognition, in particular, is in my opinion one of the weakest chapters of the book, where it sometimes becomes impossible to understand essential features of the models without having read the original publications. If indeed the book started as a book review, most of the original text is probably to be found in the first chapter. Here a very good historical introduction is given and a discussion of the first connectionist wave together with the circumstances that led to its dismissal. Quinlan convincingly shows that this dismissal was based more on a curious sociological phenomenon in the scientific community than on a careful evaluation of the criticisms. Though the next chapter on learning and memory models closely follows the introductory chapter by introducing learning procedures which partly circumvent some of these criticisms, there is a remarkable discrepancy between this chapter and the following three chapters. Whereas these latter three chapters discuss models which actually simulate psychological phenomena (i.e., also experimental results), the second chapter presents models that, in principle, are able to learn and retain information, but simulations of actual results in the field of learning and memory are neglected here. In this chapter the psychological emphasis of the book seems forgotten for a moment. I have been using the book in an introductory course on connectionism to the satisfaction of both the students and myself. Due to its brevity of explanation and its omission of a few important topics, however, it was not sufficient to serve as the sole text for the course, but then I know of no books that cover all aspects I think should be covered. One aspect, the practical experience with actual models, cannot be expected because it falls outside the scope of this book, though I am still waiting for a text that provides detailed explanation together with some userfriendly, preferably graphical software, that can be used by students without much programming experience. To fill this gap we have been using the MacBrain 3.0 package from Neurix which has nice graphical features, but is rather slow on the Mats we have available. To cover the essential subjects in connectionism more extensively we added a reader with a number of original articles. A function, that is fundamental to almost
180
Book reviews /Acta Psychologica 85 (1994) 171-181
all connectionist models, namely simultaneous multiple constraint satisfaction, is mentioned only briefly as a distinctive notion in connectionism (p. 2391, but is not explained in the preceding five chapters. This is clearly an important topic, which may, moreover, be related to the very psychologically relevant function of attention and which needs further elaboration in supplementary reading. A conspicuous omission in the book concerns the complete neglect of competitive learning procedures. The majority and some of the best-known unsupervised learning procedures (a term which is discussed in the book) consist of such competitive learning schemes. Moreover, after the discussion of the importance of inhibition in neural networks in the first chapter, it would be expected that in the next chapter learning procedures, in which inhibition plays an important role, would be presented, but they were not. Also the neurobiological plausibility of such learning procedures seems somewhat higher than, for instance, that of back-propagation. Neurobiological plausibility, furthermore, is an aspect that might deserve more attention than it gets in the final chapter. The link with and the inspiration by the nervous system is what distinguishes connectionism from mathematical parameter-fitting methods or artificial intelligence programs. A comparison of connectionist models to the brain and an attempt to discuss the relevance of the parallels and differences would certainly have added to the balanced nature of the book. If this were done it would, for instance, become clear that the interconnection scheme (i.e., homogeneously connected or layered networks) of most networks is highly implausible due to the unlikely high number of connections required and that some form of modularity is probable. Though modularity is discussed shortly at several places, it seems to me that this is still too little in view of the growing evidence for modularity in the brain and the enhanced interest for modularity in connectionist models (e.g., McClelland, 1992). The inclusion of still other recent developments in connectionism would greatly extend the scope of the book and might go some way towards solving a few of the unsettled issues. A more than passing discussion of ‘nets with loops’ (p. 260) could show how these newer ‘simple recurrent networks’ can deal with sequential processing and rule-governed behavior. This would improve the somewhat meagre Chapter 5 and would allow for the treatment of modelling work in the field of implicit learning (more attention to implicit memory in Chapter 2 would probably also make this chapter more psychologically relevant). Finally, some discussion of the newer work on oscillations and synchronization in neural networks might also improve the currency of this introduction into connectionism. In spite of these minor deficiencies, I feel that this is a valuable (but not expensive) introduction to and evaluation of the connectionist approach. R. Hans Phaf Psychonomics Department University of Amsterdam Roetersstraat 15 1018 WB Amsterdam The Netherlands
Book reviews /Acta Psychologica 85 (1994) I71 -181
181
1. References Crick, F.H.C., 1979. Thinking about the brain. Scientific American 241, 181-188. Gazzaniga, MS., 1984. Handbook of cognitive neuroscience. New York: Plenum. Levine, D.S. and S.J. Leven, 1992. Motivation, emotion, and goal direction in neural networks. Hillsdale, NJ: Erlbaum. McClelland, J.L., 1992. ‘Toward a theory of information processing in graded, random, and interactive networks’. In: D.E. Meyer and S. Kornblum, Attention and performance XIV, Synergies in experimental psychology, artificial intelligence, and cognitive neuroscience (pp. 655-688). Cambridge, MA: MIT Press.