Can computers think? Differences and similarities between computers and brains

Pergamon

0301-0082(93)E0002-K

Progress in Neurobiology Vol. 45, pp. 99 to 127, 1995 Copyright © 1995ElsevierScienceLtd Printed in Great Britain.All fights reserved 0301-0082/95[$29.00

CAN COMPUTERS THINK? DIFFERENCES A N D SIMILARITIES BETWEEN COMPUTERS AND BRAINS R. J. H A R V E Y Department of Anatomy and Structural Biology, and Neuroscience Research Centre, University of Otago Medical School, P.O. Box 913, Dunedin, New Zealand

CONTENTS 1. Introduction 2. Searle's demonstration 2.1. Introduction 2.2. The 'Chinese Room' 2.2.1. Comments on the Chinese Room 3. Computers and computer systems 3.1. Computers and programs 3.2. 'Items of information' 3.3. Different forms in which programs can exist 3.4. Levels of description 4. Simulations 4.1. Simulation of physical systems 4.2. Simulations of brains 4.2.1. Brains and symbol manipulation 4.2.2. Methods for assessing whether mental states exist 5. Mental phenomena 5.1. 'Intentionality' 5.2. Consciousness 5.2.1. Consciousness and memory 5.3. Ways in which understanding may arise 5.4. Thinking and algorithms 6. Brain processes 6.1. Development of the brain 6.2. Mechanisms of memory 6.3. Hierarchies in the central nervous system 6.4. Brain processes that may be associated with consciousness 6.5. Parallel proce~;singand brain simulation 7. Some practical differencesbetween organisms and computer systems 8. Final comments References

1. INTRODUCTION

99 100 100 101 102 104 104 106 107 108 109 !10 111 112 114 115 115 116 117 118 119 119 119 120 121 122 123 124 125 126

minds, and brains?) of philosophers and many other people for at least 2000 years. This particular question is often referred to as the mind-body problem. However, the problem that is addressed below, while closely related, is actually quite distinct. This problem is whether, given that some physical device, e.g. a human being, consisting of a brain in a body, is capable of generating or experiencing mental states, it is, in principle, possible for a quite different type of device to have comparable experiences. The debate in this area centres on whether a device such as a computer may be able to experience mental states. There is a wide spectrum of beliefs on whether computers can, in principle, think. At one end of the scale are those who appear to see no real problem in relation to computers thinking in essentially the same way as brains; among these are Turing (1950) and Moravec (1988). Moravec, by extrapolating from the developments in computer

There has been nmch debate over a number of years about the relative status of brains and computers, and whether computers are capable of performing the same sort of cognitive functions as brains. The particular part of this debate to which I wish to contribute is the question of whether or not it is possible in principle for a computer system to think, or understand, or have cognitive states in the same sort of ways that humans and their brains can. I am writing from the viewpoint of a neuroscientist with some knowledge of computers and with experience of using techniques of computer simulation to study the properties of single.,neurones and groups of neurones. The search for some explanation of the way in which mental states arise in a physical device such as the body (including its brain), has occupied the thoughts (and 99

R. J. Harvey

100

systems over the last 50 years, expects computers to reach the level of human intelligence in about 20 years time, while Turing, from a much more distant viewpoint, thought that this stage would be reached around the year 2000. At the other extreme are those who believe that it is impossible in principle for a computer to think, or experience mental states under any circumstances. Included in this group would be Lucas (1961), Searle (e.g. Searle, 1980, 1984, 1990a) and Penrose (1989). Searle has stated that he has produced a straightforward demonstration that computer programs are unable to think or understand under any circumstances. A large number of authors have responded to this claim (first made in Searle, 1980), and the majority have concluded that his claim is not fully justified. Despite a good deal of debate, Searle has produced answers to the points raised by his critics and has maintained his position essentially unchanged since this initial publication (from Searle, 1980 to Searle, 1990a). It would appear that much of the disagreement between the protagonists of the various points of view is, at least partially, a result of a lack of agreement on the precise definitions of terms such as 'thinking' and 'understanding'. However, it seems to me that Searle's demonstration, although plausible at first sight, does not show that computer thought is impossible, and it is actually irrelevant to the question, as I will attempt to show. I believe that there is at present no logical reason for believing that, in principle, a system which depends on the processing capabilities of a computer cannot think, and that the question is still open. We will have to wait and see what future developments bring. In what follows, I have tried to express the arguments in terms that are accessible to those who are not well-versed in the jargon either of neuroscience or of computers. I start with a description of Searle's demonstration, and continue with some details of current computers and computer systems, because these are very important to a consideration of what it may be possible in principle for a system based on a computer to do. This is followed by a discussion of some of the concepts that appear in Searle's arguments and have been raised by other authors who believe that computers are not, in principle, able to think or understand, and I conclude with some general observations on parallels between conceivable computer systems and living organisms (including their brains).

2. SEARLE'S DEMONSTRATION 2.1. Introduction

Searle says that he has no problem with accepting that a machine can think; a human being is a special sort of machine, which obeys the normal physical and chemical laws of nature, and a human being can think, so therefore a machine can think; I agree with this. The first chapter of Searle's book "Minds, Brains and Science" (Searle, 1984) describes the relationship between the mind and the brain as he sees it. This is an excellent account, which would be acceptable, at

least as a working hypothesis, to many (although by no means all) neuroscientists, although it does not deal with all problems in the relationship of the mind and brain (see, for example, Gray, 1987). In essence, he says that the mind/body problem is not really a problem at all, but that minds are just a particular manifestation of the sort of activity that goes on in brains: "Mental p h e n o m e n a . . , are caused by processes going on in the brain" and "the mind and the body interact, but they are not two different things, since mental phenomena just are features of the brain" (Searle, 1984). He states that there are four features that are fundamental to mental phenomena. These are: consciousness, intentionality, subjectivity and mental causation. Mental states are higher level features of the individual neurones and neuronal modules that make up the brain. However, Searle goes on to say that a computer cannot think, especially if it is running a computer program. In this, he is specifically attacking the claims of the supporters of what he refers to as "strong" Artificial Intelligence (AI). According to him, these claims are that a computer can think (and have mental states etc.)just by virtue of implementing a computer program. Searle divides AI research into two categories that he refers to as "strong" and "weak" AI. Weak AI, using his classification, takes the approach that computer models and simulations are very useful tools and may throw light on and give real insights into the mechanisms of the brain and mind. He is quite happy with this approach, and it would seem that the majority of philosophers and neuroscientists are too. On the other hand, strong AI, according to Searle's definition, works on the theory that a computer program could mimic the processes of the brain so well when it is running that its behaviour would be indistinguishable from that of a real person, and that it would then think and have mental states in the same way as a real person. A widely quoted test for determining whether something is able to think in the same sort of way as a human being is often referred to as the Turing test, after Alan Turing who first proposed this sort of approach (Turing, 1950). For this, an assessor carries on conversations with the device under test and with a human being. If on the basis of the responses, the assessor is unable to determine which is the human and which is the other device, then the device can be taken to be capable of thought. (Obviously these conversations must be carried out indirectly, e.g. via a keyboard and display, so that the assessor is unable to see who or what he or she is communicating with.) This type of test is therefore essentially a behavioural test for thinking, etc. Searle states that strong AI is unusual among theories of the mind in that it can be stated quite clearly, and that it can be decisively refuted. He gives (although without citing any references) the following as examples of statements that have been made about the theory underlying strong AI: "according to strong A I . . . the appropriately programmed computer really is a mind, in the sense that computers given the right programs can be literally said to understand and have other cognitive states" (Searle, 1980), and "~the mind is to the brain as the program is to the hardware" (Searle, 1980, 1990a). However, I believe that

Can Computers Think? the first of these ,statements is distinctly misleading, and, as discussed below, it is difficult to assign any precise meaning to the second of them. Moreover, as mentioned above, his refutation is essentially irrelevant to the question of whether a computer program running in a computer could in principle think in the same sort of way as a human being. It appears to me that Searle has missed the core of the question, partly at least, because he does not seem to have taken into account the full details of the relationship between a computer and a program that is running in it, and the relationship between them and the outside world. A number of those whom Searle identifies as proponents of strong AI have also not taken this into account, or at least have not made it clear in their writings, and there seems to have been a good deal of confusion and misunderstanding about this. I have tried to clarify these matters in Section 3. 2.2, The 'Chinese Room' Searle's argument hinges on a now famous thought experiment that he calls the Chinese room (see Fig. 1). Paraphrasing Searle slightly: Assume that you have no understanding at all of Chinese. Now, imagine that you are seated in a

101

room with supplies of symbols, which include the full set of Chinese characters, but may include other symbols as well. You also have a set of instructions which is written in a language you do understand (e.g. English). Someone passes in a piece ofpaper on which something is written in Chinese. You match the characters with symbols from your supplied set and, following the rules in the instructions, carry out various manipulations according to which characters they are. The process will finish with you copying some Chinese characters onto another piece of paper and passing it back out of the room. In this way, the person outside the room would be able to carry on a (type of) two way written conversation~ in which the pieces of paper that you passed out of the room carried responses to what was written on the pieces of paper that he or she had passed in to you. If the instructions were suitable and sufficiently comprehensive, these responses would be indistinguishable from those that would be produced by someone who fully understood Chinese. However, this conversation would have taken place even though you were not aware of the meaning of any of the characters (i.e. symbols) passed in or out of the room, or of any of those in the room, apart from those (English) symbols in the book, giving you the

f

I |

I

I

J |

I

Fig. 1. A bird's eye view of the 'Chinese Room'. The only means by which the "operator' can communicate with the outside is via the input and output slots. The operator, who does not understand Chinese, receives messages written in Chinese through the input slot, and produces messages in Chinese which are passed out through tlhe output slot. The instructions for dealing with the input message and producing the output message are included in the books on the table.

R. J. Harvey

102

instructions. You would have no understanding of what the various characters meant and would not understand Chinese. This system would appear to be able to satisfy the Turing test for understanding or thinking. However, Searle draws the conclusion that no understanding can be taking place; what is happening in the room is equivalent to what happens in a computer, with the instructions being the program that tells you, as the computer, how to manipulate the symbols. These symbols, as far as you are concerned, are meaningless. According to Searle, a program is purely 'formal' and consists of syntactic rules only, so that there is no way that it can indicate (at least to the computer) what any of the symbols mean. Therefore, of itself, a computer program cannot attribute meaning to anything and therefore cannot understand anything. 2.2.1. Comments on the Chinese Room The writing of the program needed to successfully operate the Chinese room would be a great intellectual feat. Its performance is well beyond what any language processing program has achieved (e.g. Schank, 1980, and I am not aware of any development since that time that materially alters the position), but Searle maintains that it or anything developed from it would not be able to understand. Searle (1984, 1990a) bases his discussion and arguments on a series of four axioms. These are: 1. Computer programs are formal (syntactic). 2. Human minds have mental contents (semantics). 3. Syntax by itself is neither constitutive of nor sufficient for semantics. 4. Brains cause minds. Of these, numbers 1, 2 and 4 seem fair enough, although, as we shall see, numbers 1 and 2 are somewhat over-restrictive in their scope if taken literally. Number 4 would not be accepted by everybody, and few people would accept that it is true for all brains. However, most neuroscientists and many philosophers would accept it, or perhaps something similar, such as, "minds are very closely related to activity occurring in brains", at least in relation to human brains. However, on some occasions Searle appears to take it as though it reads "only brains cause minds", without really justifying this reading, which, in any case, begs the whole question under discussion. Axiom number 3 creates some difficulties; Searle (1990a) states that at one level, it is true by definition, and that there is a distinction between formal elements that have no intrinsic meaning and those phenomena that have intrinsic content. However, he gives no indication what this distinction is and does not give any real argument in support of his claim, nor any idea as to how an element can come to be considered to have intrinsic content. Churchland and Churchland (1990) also stress the question-begging nature of this axiom. Under axiom 2, Searle appears to equate semantics with understanding, but does not give any definition of how he actually defines understanding. Presumably understanding in any sense must depend on the establishment of some sort of linkage between different concepts. In a brain (or computer system

come to that) a concept does not exist as such, but only in terms of an internal symbol or representation that stands for the particular concept. For brains, the different representations may well have been originally derived from information received by its containing organism via different sensory modalities (for example, understanding a written word by associating its internal representation with that for the corresponding spoken word, which is in turn associated with representations of what the word actually stands for), but this is not always the case. What is important is that understanding is not an all-or-nothing phenomenon, and various degrees of understanding are possible. What is often referred to as the depth of understanding presumably must depend in some way on the number and complexity of the associations linked to the particular symbol or constellation of symbols under consideration (and also that the correct sort of associations are made--misunderstandings would arise from incorrect associations!). The way in which these linkages occur is not known, but it is reasonable to presume that they depend on the neuronal activities involved in one representation, in combination with the patterns and strengths of the connections of the neurones involved, somehow setting off the pattern of neuronal activities that constitute the succeeding representation(s) to which it is linked. The precise way in which representations will link on to one another on any particular occasion will obviously depend on the current state of the remainder of the brain, in terms of neuronal activity, strengths and patterns of connections, etc. If, as seems highly likely, there is at least some element of synaptic modification occurring according to the rules first formulated by Hebb (1949) (i.e. that when, at a synapse, presynaptic activity occurs that is consistently related to activation of the postsynaptic neurone, this leads on to an increase in the effectiveness with which the synapse activates the postsynaptic neurone on future occasions), then linkages that have occurred previously are more likely to occur again, other things being equal. Precisely how such linkages, or series of linkages, give rise to understanding is not at all clear. Presumably at some stage there must be (or must previously have been) a linkage to some representation that has some meaning (see Section 5.3). Searle makes a complete logical separation between the computer and the program, which is not really justified. He treats a computer program as though it were a completely independent entity, because a given program can, in principle, operate and produce the same results when it is implemented on completely different forms of computers. While this may be true, it is only true up to a point, and Searle appears not to be aware that, while a computer program may possess potentiality on its own, it can only actually do anything when it is running on some computer or other. The same sort of thing is also true for the operations of a brain under normal circumstances, since it requires the transducers of its attached organism to enable it to actually do anything as far as the outside world is concerned--see Fig. 2. To achieve any effect, a computer program, or rather whoever designed and wrote the program, makes certain assumptions about what the computer can do in the way of communicating with the outside world, which

Can Computers Think?

Organism

Computer System

Nervous System

Transducers

103

Computer + program

I

Sense organs Musclesetc.

TOutside World Fig. 2. A diagram of an organism in relation to the interactions of its nervous system with the outside world. involves transducers, in a similar sort of way as for an organism, as indicated in Fig. 3. One could perhaps compare a comp~Lter program on its own to a brain, or rather to a whole person, who had been frozen and who was immersed in liquid nitrogen. Provided we could thaw people and their brains out successfully, the frozen person would have the potentiality for thought and understanding. However, not even the most optimistic supporter of the notion that brains cause minds would claim that a deep-frozen human (plus brain) was actually doing any thinking or understanding or being conscious. On the other hand, the logical separation that Searle makes between computers and programs makes a flaw in his reasoning more obvious, even within the over-limited view that he takes. He says that the person in the Chinese room does not understand Chinese as a result of performing the manipulations on the symbols, but, as he says, that person is fulfilling the role of the computer only, and so is 'outside' the program. Even so, Searle draws the conclusion that because the 'computer' does not understand anything, the program cannot be understanding anything either. However, the computer is only a part, although an essential part, of the overall system of computer and program (see Section 3). The fact that the computer does not understand therefore does not tell us anything about whether t]ae system as a whole understands anything. In any case, if the set of instructions has been written in the same sort of way as a computer program, as Searle suggests, it would not require any understanding at all on the part of the person in the Chinese room, only that he or she can follow instructions and carry out the necessary interactions with the outside world (a computer program does not require the actual computer hardware to understand anything, only that it can follow the instructions). One could perhaps make the same sort of statement about, for example, cochlear neurones (the nerve cells of the inner ear that are responsible for the detection of sound) in someone who is listening to spoken language. They are essential to understanding, but they do not themselves understand. I imagine that my cochlear neurones are doing the same sort of things as those of a natiw: Chinese speaker when listening to

Transducers

{ Keyboard Printer etc.

Outside World Fig. 3. A diagram of a computer system in relation to its interactions with the outside world. spoken Chinese, and while I would understand nothing, the Chinese speaker would understand what was being said. A number of writers have suggested that, even though the person in the room does not understand anything, the system as a whole does. Searle (1980) counters this argument along the following lines: suppose that you learnt the instructions and all of the characters and symbols by heart. You would then be able to make appropriate responses to Chinese writing that was given to you, although you would still not understand Chinese, exactly as before. This arrangement raises some interesting questions. Whether anyone would actually be able to memorise and so internalise such a complex and sophisticated system in this way is another matter; it would almost certainly be a great deal easier to learn Chinese! Searle claims that now the whole system is inside you, you encompass the whole system and, therefore, there is nothing in the system that is not in you, and since you do not understand, no understanding can be taking place. However, this actually makes no difference to the weakness of Searle's argument, as the situation is logically exactly the same as before. You would not understand Chinese, but that would still be irrelevant to the question of whether the whole system does. The argument that one would then have two independent understanding systems in the same person cannot be entirely compelling; apart from anything else, a patient who has had his or her corpus callosum divided can, under certain circumstances, show evidence for possessing two more or less independently conscious understanding systems (e.g. Sperry, 1986; see also Section 5.2). Even if it were possible to internalise the system in this way, I cannot help wondering how long it would be possible to continue to operate it without acquiring some understanding of Chinese--certainly one would learn to recognise and distinguish between individual Chinese characters-- but again, that is completely irrelevant to the actual point under discussion. We will be able to assess Searle's position better when translation programs, and other programs that manipulate language in relation to its meaning,

R. J. Harvey

104

improve from their present state of development. Computers systems are quite good at recognising very simple patterns, but, at present, bad at recognising complicated patterns or combinations of patterns. This is something that brains (incorporated into conscious humans or other animals) seem to be very good at, especially if other information (the 'context') has provided some idea of what to expect. The use of this sort of context information usually allows us to resolve the apparent ambiguities in language (i.e. the ambiguities that occur when language is taken in short segments) without any difficulty-- often without being aware of t h e m - - while existing programs are not at all good at this, unless the scope of what is being translated (or otherwise 'understood') is very restricted (see Dennett, 1984 for an interesting and entertaining discussion of this problem). The context is obviously very important for humans in their everyday life, and the amount of information that is needed to provide the basis for context is very difficult to estimate, but it is probably quite large. (See also Dreyfuss (1992), especially the "Introduction to the M.I.T. Press Edition" for further discussion of this.)

3. COMPUTERS AND COMPUTER SYSTEMS

3.1. Computers and Programs What is the relationship between a computer and a program, and how can this be compared to the various components of brains (or minds)? For the present, I will use the term 'computer' to stand for what should really be referred to as a 'general purpose digital computer'. However, as has often been shown, initially by Turing (1937), all 'finite state computers', a category that includes all current versions of general purpose digital computers, are, in principle, equivalent in what they can do, even though different ways of implementing computers may allow them to deal with various types of problem at very different speeds, I will use the term 'computer system' to refer to a computer, together with the program(s) that are running in (or on) it. These days, most people's image of a general purpose computer is that it is something with a video display, a keyboard, and a box containing the electronic components that constitute the 'works.' However, many other physical forms are available. If, for the moment, we think in terms of a standard personal computer system, the basic box + keyboard + display needs two additional invisible items before it is any use at all to most people. These are a source of power, and an operating system. The operating system is very important for keeping track of what is going on in the computer, and in controlling the interactions between the computer, or rather the programs that are running in it, and the outside world. Note that no computer or computer program can do anything that is apparent to any outside observer without some means of interacting with the outside world. A traditional computer contains three essential components: a central processing unit (CPU), a store or memory where information can be stored, and some

means of communicating with the outside world, as indicated diagrammatically in Fig. 4. In an electronic computer-- and all currently practical and useful computers are electronic-- the memory consists of a large number of identical circuits in which information, in a form which consists of what are essentially numbers (with a fixed number of digits-- see Section 3.2), can be stored. These 'numbers' will be referred to below as items of information, or sometimes just as items. Each individual item of information is stored in an individual set of circuits, and each set is assigned a unique numerical label, usually referred to as its 'address'-- see Fig. 4. An item of information can be stored by sending it to the appropriate address, which writes it into that set of circuits, or it can be retrieved by reading from that address, which leaves the contents unchanged. In addition, nearly all modern computers possess some form of device, usually a magnetic disc for our standard type of personal computer, where a large number of items of information can be stored in a form more permanent than when stored in the electronic circuits of a computer memory. (In current computers, most varieties of electronic circuit used for storing items of information have the disadvantage that the information stored in them disappears if the power fails or is turned off. Alternately, if the information does not disappear, it cannot be changed by sending a different item of information to that address. These unchangeable storage circuits are usually referred to as Read Only Memory, or ROM.) A very basic central processing unit is schematically illustrated in Fig. 5. Nearly all current CPUs contain many of each of the elements shown, together with other components, but the principles remain essentially the same. The basic CPU contains electronic circuits in which calculations take place (labelled "Accumulator" in Fig. 5), together with circuitry for containing items that represent addresses (in memory) from which items are to be read, or in which items are to be stored, usually to or from the accumulator. A particularly important item that must be stored in the CPU is the address from which is to be read the item which is the next instruction to be executed. The set of circuits that contain this is usually called the program counter. Items which are instructions are read from the address in memory specified by the program counter into a special set of circuits in the CPU which make up an 'instruction register'. This instruction register is a key part of the CPU, and the numerical value of the item that it contains specifies the particular one of its possible operations that the CPU carries out. As soon as one instruction has been set in motion, another item is read from memory into the instruction register, and specifies the next operation to be carried out. Usually, successive instructions are read from successive addresses in memory, and the program counter is automatically incremented after each instruction is read. However, certain instructions allow this sequence to be over-ridden by altering the program counter, so that the next instruction is taken from a different address. For some such instructions, a jump in the sequence will always take place when the instruction is carried out. For others, the program counter may or may not be altered, depending on the value of the result of some preceding operation, or on

Can Computers Think?

105

Input

Hemory

g

Output

1 - - 2

--3 J&

"Bus -

-

507 508

Z

50g, 510

511 512 m

z

CPU

Fig. 4. A diagram showing the essential components of a very basic electronic computer system. The computer interacts via the transducers labelled Input and Output with the outside world, from which it also requires a source of power. The 'Support' systems consist essentiallyof the power supply, which converts the input power into a form suitable for the electric circuits, etc. in the other components. CPU: Central Processing Units. The 'Bus' is the pathway by which information is sent from one component to another.

the value of a stored item of information. This allows the sequence of instructions of a program to be different under different circumstances. Some of the instructions that cause jumps may transfer the sequence of instructions to another part of its own program, or to a particular part of the operating system and also stc,re the address that is currently in the program counter, so that control can be returned to the position just after the jump by loading that address back into the program counter. Some of the instructions of the computer make the CPU send information to or obtain information from the transducers thrc,ugh which it makes its connections to the outside world. For our standard personal computer, the transducers are the keyboard for input, and the display for output (plus, often, a printer). These are usually connected to the CPU by a 'bus' (see Fig. 4), which is eL common pathway along which

information can be sent between the various components of the computer. The memory is usually attached to the bus as well. Conceptually, it is probably simpler to consider that all communication with the outside world is via the CPU, and under the control of the C P U - - o r rather under the control of programs that are running in the CPU (usually the operating system), Traditionally, computer systems have been divided up into 'hardware' and 'software,' along the lines suggested in Fig. 6. The hardware is made up of all the components that you can touch and see (in principle at least?), while the software consists of the programs that run on or are implemented by the hardware. In principle, the hardware is fixed, in the sense of being unchanging (apart from its internal states), while the software is readily changed. However, there is not a clear-cut distinction in a functional sense between

R. J. Harvey

106

what the hardware does and what the software does, and different computer systems vary much in which functions are carried out by specially designed circuits (i.e. hardware), and which are carried out by executing a special program in what might be called 'general purpose' hardware, like the CPU described above. In most existing personal computer systems, this variation comes mainly in the way that the functions of the operating system are carried out. In some, many of the operating system functions are carried out by programs running in the CPU, but whose instructions are stored in circuits that cannot be changed (i.e. ROM), or rather which can only be changed by changing the physical circuits (i.e. the hardware) in which they are stored. (In fact, all personal computer systems have some functions stored in this way, even if it is only functions that check out the system when the power is turned on, and that read the operating system into the normal (alterable) storage of the computer from where it is stored in a more permanent form on a magnetic disc or something similar.) Should we describe these functions as being implemented by hardware or by software? Certainly the distinction is not at all clear cut. Similarly, some of the devices associated with a computer system, such as disc drives and display controllers, may have their functions implemented by a program running in a general purpose CPU, or by special hardware. In some special purpose computer systems, although they may contain a 'general purpose' CPU, all the programs are stored in ROM, so that none of the functions of the system can be changed without changing the hardware. Such

lk Program Counter

]

Instruction Register

I

"Accumulator"

]

"'Address" Register

]

1

Fig. 5. An enlarged diagram of the CPU in Fig. 4, showing its essential internal registers. See text for further details.

a system is often referred to as being 'hard-wired,' even though the actual wiring for most of it may be identical to that of a general purpose computer based on the same type of CPU.

3.2. 'Items of Information' The entities that have been referred to as items of information have a format which is that of an integer number with a fixed number of digits. In all current electronic computers these are actually stored as binary digits (otherwise known as bits), i.e. as a series of states representing zeroes and ones. However, that is a convenience rather than a necessity, and, in the past, some computers which stored numbers as sets of decimal digits were produced. Obviously, a number with a fixed number of digits can only store a number within a given range. For example, a store that can contain a six digit decimal number plus a sign can be used for integers only in the range -999,999 to +999,999. To increase the range of magnitudes of numbers that can be accommodated, a series of individual items can be strung together end to end. Thus, two 6 digit stores end to end can contain a 12 digit number. An individual item, or, more usually, a series of (contiguous, in terms of their addresses) items can also be used to represent what is known as a 'floating point' number, which allows numbers other than integers to be stored and processed. The use of a floating point number increases the range of magnitude of numbers that can be represented and allows numbers between 0 and 1 to be represented, but they can only be represented with a limited accuracy. For example, if two 6 digit stores were used, l0 digits could be used to store the l0 most significant digits of the number, and the other two used to store a power of 10 to multiply them by. In this way (using a convention that assumes that there is a decimal point at the left hand end of the series of significant digits) numbers as large a s 1099 or as small a s 10 -99 can be represented at up to 10 figure accuracy. Although all the stored items can be treated as though they are numbers, they can be interpreted and are in fact used in a number of different ways. Items of information can represent: 1. numbers--in various formats, some of which are described above; 2. addresses of other items that are stored; 3. logical values, i.e. an indication of whether some proposition is true or false (usually used to control those instructions that may or may not produce a jump in the sequence in which instructions are processed); 4. characters, i.e. letters of the alphabet, numbers (in the sense of decimal digits), punctuation marks, and other symbols that can be shown on a display or printed out, or that appear on a keyboard; 5. patterns of colours of dots (or the presence or absence of dots) that can be combined to make up a graph or other picture on a display or printer; 6. and, very important, the instructions that tell the CPU of the computer what to do. All the types of item, apart from the instructions, are often referred to as data, while the instructions are referred to as code.

Can Computers Think?

107

Computer System Program

[

[Operating System I

Software t Software Hardware or

I

l

BIOS

I

Computer

I

l Transd --

Hardware

OutsideWorld Fig. 6. A diagram of a computer systemin terms of the stages through which a program runningin it interacts with the outside world. The ways in which the various stages are most commonly implemented is indicated at the right. 'Hardware' is used to indicate physical devices, while 'software' is used to indicate programs. See text for further details. Note that all but the very simplest of programs will normally need to store intermediate results and other information while they are actually operating. Because all computers are able to modify the sequence of a program on the basis of information stored in the computer or on the basis of the results of its calculations and other operations, the stored information can be q;onsidered to be an essential part of the program. Changing the stored information can have effects that are equivalent to changing the actual code of the program. In existing computer systems, it is unusual to have programs that modify their actions by altering their code, but almost all programs alter their actions according to the value of stored items of data. (It is very much easier for the designer of a program to predict or keep track of what might happen when using data to control the sequence of a program, rather than by altering the actual instructions. This is partly at least because all existing compilers and other programs that generate the code (e.g. assemblers)--see Section 3.3--have not been designed to produce code that could readily be modified. However, it is quite possible to produce programs that modiify their own code; indeed some computer 'viruses' have been written in that way to make them harder to detect.) This means that, as far as what the program actually does is concerned, there

is no clear logical distinction between the code and the data of the program. Alteration of either can affect the way the program operates and what it does.

3.3. Different Forms in Which Programs Can Exist What does a program actually consist of?. Unfortunately, the word program is used in a number of somewhat different senses, and this has led to a certain amount of confusion. A fairly wide, and widely acceptable, definition would be that a program incorporates the description and detailed instructions for performing the operations that enable some process, or series of processes to be carried out. A program can therefore represent a complete self-contained and independent system, even though it only has that potentiality (as mentioned above), since it needs to be implemented before it will actually do anything. Very commonly there will be different forms of the same program, which may be at different levels of description. One of these, at a 'high' level of description, could consist of a general outline of the problem and of the methods used to deal with it. These methods are often referred to as algorithms. An algorithm is, in essence, a recipe, or series of instructions for carrying out some particular task. Anything but a very small program is likely to contain

108

R. J. Harvey

a large number of sub-algorithms making up the overall algorithm that the whole program incorporates. One feature of the great majority of algorithms is that they are essentially deterministic, so that an algorithm will (or certainly should!) always produce the 'correct' answer to the problem that it is designed to solve. At the next lower level of description would be a representation in a particular higher-level computer language (e.g. F O R T R A N , Pascal, C). At the lowest level are the detailed instructions, usually referred to as machine code, that a particular computer requires. The machine code, as it exists in the computer when the program is running, consists of a series of items of information (and, as mentioned above, are therefore in exactly the same form as that in which numbers are represented). Some programs are written in a form that is directly related to the machine code, and this is usually referred to as assembly language. Assembly language is actually written in alphabetic and numeric characters that people find reasonably easy to deal with--at least very much easier to deal with than some direct representation of the numbers that the machine code is made up of. However, unlike the case for statements in a higher level language, there is a one-to-one correspondence between statements in assembly language and individual items of information stored in the computer's memory when the program is operating. These items of information make up the data storage that the program uses, as well as the individual instructions of the machine code. The advantage of using assembly language is that it gives the person writing the program much more direct control over the details of what the computer will actually do when the program is running. This makes it possible for the program to occupy less space in the computer's memory and/or to operate more quickly than if a higher level language had been used to write the program. The alphabetic and numeric characters making up a program written in assembly language are converted into machine code by a program which is called an assembler. The translation from a higher level computer language is carried out by another type of program called a compiler, which may produce a number of intermediate representations (one of which could be assembly language) between the higher level language and the machine code. In principle, a program in a higher-level language can be implemented on a number of different types of computer, by using the appropriate compiler to translate it into the correct machine code for the particular computer and operating system, and it should produce the same, or at least very similar results whatever computer is used. However, it is always written with the assumption that certain functions that are equivalent from one computer system to another will be available for communicating with the outside world (and also with devices such as discs where information can be stored). As mentioned above, these functions are normally provided by the operating system of the computer, in combination with the actual hardware. The compiler produces the appropriate instructions for interacting with the operating system (or it would be possible to incorporate instructions that interact directly with the hardware, although almost all current compilers only produce

instructions to interact with the operating system) so that the person writing a program does not need to be aware of any details of these functions. (Perhaps this is why the importance of the interactions of a computer system with the outside world have often been underemphasised.) Strictly speaking, the operating system is also a program (and even if it is implemented in hardware, it is functionally equivalent to a program). An operating system is often written in a higher level language, and so may exist in various forms at different levels of description, as for any other program. The actual input and output routines may be separate programs again (and, on a personal computer, are usually referred to as the BIOS---Basic Input Output System--see Fig. 6). These programs are often stored in Read Only Memory, and, as mentioned above, can then be considered to be part of the hardware of the particular computer. Thus most programs operate via two other layers of programs that provide particular functions. These functions may be implemented either in special purpose hardware, or in software, or in a combination of the two.

3.4. Levels of Description Levels of description in computer programs are somewhat analogous to levels of description in any other complicated system. For example, if one considers the properties of a piece of iron, these can be described at the level of macroscopic p h y s i c s ~ e n s i t y , hardness, magnetic properties, etc.--and this can give a complete and consistent description at the macroscopic level. However, at a lower level, we know that these properties depend on the properties of individual iron atoms, and more particularly on the way that the atoms interact. In a sense, the macroscopic properties depend entirely on the properties at the atomic level, but it is not at all easy to derive the higher level properties from the lower, even though they are entirely dependent on them. At the next level, the properties at the atomic level depend entirely on the properties and interactions of the sub-atomic particles (neutrons, protons, electrons) that make up the iron atoms, but the former are not easy to derive from the latter--and so on for one more level anyway. To fully understand the relationship between the properties at the different levels, it is necessary to work with the system at more than one level. Most existing computer programs are much simpler than a piece of iron, but it is still difficult to work out exactly what is going on at a higher level, that of the general description or of a higher level language, from what is going on at the machine code level. This is so even though the two are very directly related, and the functional properties of the program are entirely dependent on the program at the machine code level (whose properties are in turn dependent on those of the particular computer plus operating system). When it comes to the brain in an animal (e.g. human), we can again describe what is going on at a number of different levels. There is the level of overt behaviour, from which, in the case of humans at least, we can infer what is going on at the mental level--indeed we can ask someone what is going on at the mental level, so that we can even obtain evidence about it that is fairly

Can Computers Think? direct. We also know a good deal (for animals) about what is going on at the level of the individual neurones, and at a lower level still, ofinterconnections, synapses, neurotransmitters, receptor molecules, transmembrane ion channels, etc. In principle, we know how the properties at this level depend on the properties at the level of individual atoms etc. We are beginning to get some idea of how larger groups of neurones or modules of the brain operate, both in terms of how the activities of individual cells contribute to their operations, and in terms of the effects that these groups of neurones have on behaviour. However, there is still a very wide gap between the neurone level and the behavioural/mental level. This means that we are still a very long way from explaining behaviour in terms of physical and chemical events occurring in the brain, although the majority of neuroscientists believe and work on the assumption that this will eventually be possible in principle. Churchland and Churchland (1990, in an article published simultaneously with Searle, 1990a), presented a view contrary to that of Searle. They showed, using an analogy, that Searle's apparent demonstration of the impossibility of understanding occurring in a computer program may not be valid. This was, essentially, because Searle was looking at the Chinese room at the wrong level of description, so that he would not have observed any understanding, even if it had been taking place. I believe that there are even stronger arguments for doubting the validity of Searle's demonstration, some of which have been given above. Churchland and Churchland (1990) also argue that Searle was really considering the wrong type of computing system, and that understanding in a computer system is fa:r more likely to occur in a system based on hardware in which many operations are taking place in parallel, especially one based on the concept of neural networks (e.g. Rumelhart and McLelland, 1986)--see Section 6.5. Searle's response to Churchland and C]~urchland's view (Searle, 1990a) is that using a parallel system would make no difference to his argument, since the capabilities of all finite state machines are essentially identical (e.g. Turing, 1937). This is correct, but does not strengthen Searle's position in relation to any other counter argument. While the use of parallel processing may not make any difference to what a machine could perform in principle, it could make an enormous difference to the speed at which it could carry out its operations. This, in turn, could make it appear much more human-like and/or intelligent in its responses (e.g. Hofstadter, 1981)--see also Section 4.2.2.

4. SIMULATIONS The programs which are of particular interest here are those designed to simulate some function of the brain or mind. Such ]programs, and indeed the great majority of simulation programs of any type, incorporate a functional specification of a potential physical device, or series of physical devices. They must also include a specification of the ways in which the various components of the device(s) interact, and hence of the manipulations that it would perform on any input applied to it. For a simulation, the potential

109

device may be referred to as a 'virtual machine,' although this particular term is perhaps best reserved for those simulations in which one particular computer or computer system is simulated on another. The operation of the overall system would be the same whether some actual device with the properties specified in the program is constructed, or whether it is run as a program on a digital computer, provided that it has the right transduction processes at its input and output. For implementing a system such as the brain, which performs a very large number of operations in parallel, or even a very small subset of the brain, a serial machine like most currently available digital computers is really rather slow and inefficient. However, it is quite convenient; while a specially constructed hardware device might well operate very much faster, it might be difficult to make, and would certainly be much more difficult to modify than a computer program. Even so, considerable success in modelling a small part of the nervous system, a subset of the retina, has been achieved using a specially constructed piece of hardware, the 'silicon retina,' which consists of a large number of identical modules that operate in parallel. Each of the modules includes components that function to represent the photoreceptors, horizontal cells and bipolar cells of the retina (Mead and Mahowald, 1988; Mahowald and Mead, 1991). More generally, brain simulating hardware would presumably be somewhat similar in principle, and take the form of a large number of relatively simple elements connected together. For a simulation that consists of a program running in the hardware of a general purpose digital computer, it is important to remember that the actual computer hardware forms only a very small part of the overall system. For a brain simulation, it bears a relationship to the overall system that is perhaps roughly comparable to that which oxidative metabolism bears to the brain; it is a necessary basis for that particular implementation, but it is not logically an important part of the system. Indeed the precise form of the computer hardware (silicon chips, beer cans, pieces of paper etc.--to give some of the possibilities that Searle has mentioned) is totally irrelevant to what the system actually does--although it may make a very large difference to its speed of operation! The speed of operation is, of course, very important when the system has to interact with its environment in 'real time,' but does not affect what it can actually achieve in principle. However, when considering a system such as Searle's Chinese room, or any other system that is simulating one side of a conversation, something that responds to one's comments and questions with about the same delay as a human being might seem intelligent and to be thinking, while one that produced exactly the same answers after pauses of a couple of weeks or so would not appear very intelligent (see Hofstadter, 1981). It would certainly be very difficult to carry on a conversation with it. There are other problems with a system that operates very much slower than a human. A particular one would be its inability to keep up with ongoing changes in contextual information (see Section 4.2.2). The actual mode of operation of the brain is clearly quite different from that of a digital computer, or even from that of a program running in a digital computer,

110

R.J. Harvey

Chloroplast Light

Synthetic Chloroplast H20

Light

CO 2

j

A M e c h a n i s m of some sort

02

Glucose

Fig. 7. Diagram of the interactions of a chloroplast with the outside world, as far as photosynthesis is concerned. Some of the internal features of a chloroplast are indicated. 02 but can the brain, at least in principle, be described by a program? I do not see any compelling reason why it should not, even though we are still a very long way from being able to describe even the barest outlines of such a program in its entirety, let alone implement it. In a simulation system, consisting of a program running in a computer, the most important part is the program, which incorporates the description of the physical system being studied and its operations. For a brain, the vast majority of the system which is incorporated into the simulation program is embodied in the 'hardware' of the brain, that is in the details of the structure, the patterns ofinterconnections between the different neurones, in the effects of the different neurotransmitters, in the relative strengths of different synapses, etc. Nearly all of the system is therefore embedded in actual physical structures (if one counts individual molecules such as neurotransmitter receptor molecules as physical structures or parts of physical structures), and the principles by which the brain operates, or what may loosely be termed its program (i.e. the neurophysiology, most of which depends on structure at the molecular level or above, anyway) makes up only a very small, even if essential, part. There is really no program in the brain that is in any sense independent of its 'hardware'. The brain's program (and data) cannot be separated from the brain in the same way that a computer program can be separated from a computer. It cannot exist or do anything in relation to the functions of the nervous system apart from when it is actually part of a real brain. The individual properties of individual brains depend on differences in structure (at some level); all brains have essentially the same program (i.e. neurophysiological basis for their operations). In the brain, the structure is more nearly comparable in its scope to the program in a computer, and the program--i.e, the neurophysiology---comparable in scope to the actual computer hardware, although, of course, in both types of system, there is no clear logical dividing line between hardware and software,

Glucose

Fig. 8. Diagram of a device that would mimic the photosynthetic activities of a chloroplast. or between the structure and the principles of operation.

4.1. Simulation of Physical Systems Searle makes the point that computer simulations of processes, like those of the atmosphere used in meteorological models, or of biochemical processes, are clearly not the same as the real thing, because they only operate with symbols for the real physical objects that are under investigation. A simulation of a rainstorm does not make us wet, and a simulation of photosynthesis does not produce any glucose or remove any carbon dioxide from the air. Extending this, he says that simulations of brain processes cannot be like the real thing either. While it is true that the usual sort of simulations of meteorological and biochemical processes do not produce 'real' results, this is because they are not connected to the outside world by the right sort of transducers, and Searle has taken an overly restrictive view of their scope. Taking photosynthesis as an example, a chloroplast takes in carbon dioxide and water, and with the aid of light energy, converts them into oxygen and glucose, as indicated in Fig. 7. It might be possible to make a device that would take in water and carbon dioxide, and give out glucose and oxygen, with the aid of light as its sole source of energy, as in Fig. 8. This device would clearly pass a behavioural test for photosynthesis, even if its internal workings were nothing like those of a chloroplast. If one could make a device that not only duplicated the inputs and outputs, but also the internal biochemical steps of photosynthesis, this would be a functional duplicate of a chloroplast, even if its physical form were different. Computer simulations of biochemical processes, such as indicated in Fig. 9, are not functional duplicates of the real thing, however, because they do

Can Computers Think?

Simulated Chloroplast Symbols for:

1-I20

Light

111

Synthetic Simulated Chloroplast

CO2

H20

Light

CO2

$

$

$

Transducers

$

Computer System

Symbols for: H20

$

$

Light

CO2

~_--_-- Z--17 tl Ii Ii |l I I I I ~-' "7

(including program) Computer System (including program)

Symbols for:

02

Glucose V

Fig. 9. Diagram of a simulation of the photosynthetic activities of a chloroplast.

4.2. Sinmlations of Brains Searle extrapolates from the fact that a biochemical simulation and a real biochemical process are not equivalent, and states that a simulation of brain function and the operations of a real brain would not be equivalent. However, the same restrictions may not apply here as apply to biochemical (or physical) simulations. In considering brain functions, we really have to think about the whole system of which the brain is a part. This system is a complete organism, including its brain, as indicated in Fig. 11, and has to be considered in exac.tly the same way as a working

"7

ili Symbols for:

not have the same inputs, outputs or internal workings--all of these are in a symbolic or representational form. Even so, it is conceivable that it would be possible to construct a device such as that illustrated in Fig. 10, in which there are input transducers for detecting water, carbon dioxide and light, and converting 1:hem into some form of symbolic representation, and output transducers for converting other symbols into glucose and oxygen. The input and output transducers could be connected, as indicated by the outlined pathway on the right, so that the same carbon, oxygen and hydrogen atoms that had been detected (as water and carbon dioxide) at the input were incorporated into the products. Although such a device may be beyond current technology, it is not impossible in principle, and would be externally indistinguishable from a device such as that shown in Fig. 8. Suitable transducers could make other simulations actually produce the effects that they simulate, although in some cases, such as a large-scale meteorological simulation, it is unlikely that this will ever be feasible.

I I I I Ii I I I I I I I I I I I I Ii I I I I I I I I I I Ii

02

Glucose

Transducers

$

$

02

Glucose

I I I| II I I II |1 |1

Fig. 10. Diagram of a simulation of the photosynthetic activities of a chloroplast used as the internal mechanism for a device mimicking a chloroplast.

computer program. For this, we have to consider the whole computer system, including both computer and program. When the inputs and outputs of the organism are considered at a behavioural level, they need not be thought of in terms of biochemical events. They depend on biochemical events, but can be considered independently of those events (to a very considerable extent at least). The significant thing about the output is not its physical or chemical nature, but the fact that it is capable of exerting control, via suitable transducers--muscles, etc. in the case of an animal--over the system of which it is a part, and hence on its environment (e.g. Dennett, 1980). In the same sort of way as for a chloroplast, a functional duplicate of an organism could be produced by replacing its various parts, including its brain and input and output transducers, with functionally equivalent devices, as indicated in Fig. 12. For an animal and its brain, all sensory inputs are transduced into physico-cbemical events (i.e. nerve impulses), what goes on inside is in terms of physico-chemical events and related processes, and any output (usually a mechanical event) is produced by another transduction--of nerve impulses into the generation of

112

R.J. Harvey

Brain Outside World

ISensory transducers I

(Eyes, ears etc.) Symbols

Symbols

Motor transducers

I

( Muscles etc.)

Outside World Fig. 11. Diagram of a brain, plus the organism that contains it, in relation to the brain's functions that control the activities of the organism, in terms of its interactions with the outside world. force by muscles and hence into active movements. The system of an organism with its brain receives afferent input from its environment, and responds by producing actions on its environment. A stimulus does not itself contain information which specifies the (spatio-temporal) pattern of nerve impulses which it generates in nerve cells and axons of the nervous system; this depends on the transduction mechanisms at the sensory receptors. The pattern of nerve impulses is therefore a symbol for or representation of the stimulus. Since the input and output to the equivalent of the brain in a system such as that shown in Fig. 12 are already in the form of symbols, a computer-based simulation of the brain can be set up without the same sort of complications as there are for a chloroplast (see Fig. 13----compare Figs 9 and 10). It is not clear that transforming an external stimulus into a spatio-temporal pattern of action potentials is superior to or fundamentally different from any other possible form of transduction. Such a symbol should be equivalent to any other form of symbol. As far as the processing of sensory information is concerned, all the operations of the brain are in terms of

manipulation of symbols, and the output, before transduction, is also in terms of symbols. All these symbol manipulations must be in terms of rules that are set by the precise current state of the brain. None of the symbols that are being manipulated has any more intrinsic meaning than Chinese characters, or words written in any other language (Harvey, 1985). (Indeed, when processing sensory input derived from language, the symbols are symbols for symbols.) 4.2.1. Brains and Symbol Manipulation It is possible that the brain is a biological device whose function is to manipulate these symbols, and whose functional importance is related to this symbol manipulation. The precise way that it happens to do this symbol manipulation may well not matter at all, in the same way that the functional effect of a computer program does not depend on the precise form of the hardware on which it happens to be running. If this is so, and one could argue that this is the most parsimonious description of the structure and function of the brain, then it follows that an accurate

Can Computers Think?

113

Synthetic Brain

Simulated Brain

Outside',World

OutsideWorld

I

I Input (sensory) transducers ~ 1¢ ~ S Y mbOls

Computer System

A Mechanism of some sort

iI

$

(including p r o g r a m )

is

Output (motor) transducers I

Output (motor) transducers

OutsideWorld

OutsideWorld

Fig. 12. Diagram of a device that would mimic the activities of a brain plus its or[;anism,in terms of its interactions with the outside world.

simulation of brain function could consist only of something that simulates the symbol manipulation functions. This concept implicitly underlies a good deal of work in AL Clearly it is possible that such a system could be a great deal simpler than a system that simulates the brain neurone by neurone and synapse by synapse. There appears to be no logical way by which it is possible to predict whether a 'data processing' simulation would be possible for all brain functions, and the problem will have to resolved empirically. However, even if a synapse by synapse simulation turns oat to be necessary to simulate brain functions, the brain is still of finite extent, and, as Searle has said, operates according to the laws of chemistry and physics, so the actual process can be simulated on a finite state machine, or in other words on a general purpose digital computer. Thus there is no reason why a non-neural, or non-biochemical, device could not perform precisely the same functions as the brain in an animal, where the non-brain parts of the animal perform the transduction between external events and internal symbols (in both directions), and the brain performs the necessary JPN 45/2

Input (sensory) transducers

B

Fig. 13. Diagram of a deviceto mimic the activitiesof a brain plus its organism, in terms of its interactions with the outside world, in which the internal mechanismis a computer system simulating the brain.

symbol manipulation. This 'brain' could, in principle, be a program running on any sort of hardware, including non-electronic hardware, but this hardware p e r se is not part of the functional system in anything like the same sort of way that the brain is part of the functional system in a real biological organism. Searle 0985) counters this line of argument by stating that the patterns of action potentials in afferent nerves are not symbols, but "they are steps in a causal process the final terms of which are intentional mental states". For example, in relation to visual input, the patterns of action potentials in the optic nerve are not symbols "because they don't symbolise anything. They are the effects of the assault of the photons on the photoreceptor cells of the retina, and they in turn are the causes of visual experiences." From this it must presumably follow that nothing that the brain does is in terms of symbols. However, while for a visual stimulus, the pattern of action potentials in the optic nerve is a representation of the image on the retina, it is a highly abstracted representation that, as mentioned above, is only very indirectly related to the

114

R. J. Harvey

information in the pattern of photons arriving at the eye. When does a highly abstracted representation become a symbol? Certainly in this case there is a causal link (i.e. the eye and the visual system) between the stimulus and the exact form of the representation, so it is not an entirely arbitrary symbol, and it is reasonably consistently related to the stimulus, at least for a particular individual. On the other hand, nearly all of our ability to interpret and understand what we see, and recognise objects in the visual environment depends on previous visual experience, and has to be learnt, or at least encoded somehow in the structure of the brain as a result of visual experience, especially in early childhood. Those who are born blind and subsequently gain their sight, have a very limited ability to interpret what they then see; although this ability does improve to some extent with visual experience, it still has to be learnt (Gregory and Wallace, 1963; see also Zeki, 1993). Similarly, those young children who have a marked refractive error in one eye or a squint usually come to suppress the input to the primary visual cortices from one eye, giving rise to what is known as amblyopia. This (at least in experimental animals and almost certainly in man as well) is associated with a much reduced number of synaptic connections onto the cortical neurones from those cells of the lateral geniculate nucleus that transmit information from the suppressed eye. Amblyopia is thought to arise as a result of the competition between the synapses conveying information from the two eyes, and when no visual information is coming from one eye or when the two eyes are providing conflicting information, then the connections from the eye providing the better information become more numerous and more effective (e.g. Blakemore and Van Sluyters, 1974; Hubel and Wiesel, 1977). There seems to be a critical period for development of the connections to the primary visual cortex, and most of this occurs in humans before the age of 2 years. If good vision is restored to an abnormal eye before this age, then its vision develops more or less normally, but beyond the age of about 3 years full recovery becomes less and less likely with increasing age, although reasonable recovery can occur up to the age of nine and some degree of recovery can take place even in adults (see Rabinowicz, 1983 for a review). Visual perception is a product of the brain rather than a direct result of input from the eyes (e.g. "To obtain its knowledge of what is visible, the brain cannot therefore merely analyse the images presented to the retina; it must actively construct a visual world."--Zeki, 1992; see also Zeki, 1993, for more details). All this suggests that there is not an automatic direct connection between the representation of a visual object in terms of action potentials in the optic nerve and its conscious mental interpretation--see also Sections 6.1-6.4. Moreover, conscious visual perceptions can occur entirely independently of any action potentials in the optic nerve, as when recalling a visual memory (especially for those with eidetic memory) or when dreaming. Bear in mind that with the Chinese room, Searle is specifically addressing the problem of language processing, when the elements in the outside world are symbols that are essentially arbitrary. This means that any pattern of action potentials representing a word or

other language element must then be a representation of a symbol, and I feel that the relationship of that to the original object is no less arbitrary than for the relationship between the word and the object. However, Searle maintains that a brain manipulating such arbitrary symbols can understand, while a computer system cannot. As a further consideration, in order for language to be a useful means of communication, the relationship of symbols to what they represent must be consistent from one user of the language to another. This being so, one could then argue that comprehension depends on some linkage between the symbol and whatever is being symbolised and that this linkage is then embedded in the brains of the 'understanders' (Harvey, 1986). I believe that Searle would agree that this depends on something consistent going on in the brain of each 'understander,' and he would probably agree that this depends on some form of organisation within the brain. This therefore represents a causal relationship between the internal representation of the symbol, and the internal representation of whatever it is that has to be linked to this to lead to understanding. There are devices available that can receive visual input and send a representation of the visual scene to a computer system, e.g. a video camera. The relationship between the representation and the visual scene is then no less causal and consistent than the relationship between the visual scene and the pattern of action potentials in the optic nerve when the eye and visual system of an animal are involved. Presumably, Searle would allow that the brain manipulates these representations of visual input according to a definite set of rules. Thus a computer (with a suitable program) could also manipulate the representations according to a comparable set of rules, and finish up by producing a representation that could generate movements of the system--given appropriate output transducers. In principle, such a system could pass any behavioural test for thinking or consciousness or whatever. We are still a very long way from being able to produce such a system, because we still have very little understanding of the manipulations that the brain does on its internal representations or symbols. The question then would be whether the manipulations are merely simulations, as Searle would say, or whether they are sufficiently similar that they would be associated with mental phenomena in the same way as those in the brain are. In other words, are the manipulations of symbols of representations of symbols qualitatively totally different from the manipulations of representations of symbols? Searle argues that they are different, because, according to him, mental phenomena are restricted only to biological systems. 4.2.2. Methods for Assessing Whether Mental States Exist One of the problems in the literature on this area is that there appear to be no consistent criteria that different authors would accept for deciding whether some object or other is thinking or understanding. The great majority of authors would accept some behavioural criterion, such as the Turing test, as at least a basis for this; in other words, if something can

Can Computers Think? pass the Turing test, then it has at least a primafacie case for being considered to be capable of thought and/or understanding. A few authors appear to believe that understanding can only occur in very specific types of objects (e.$,,. humans) and that whatever types of behavioural test:~ other types of object may be able to pass, they cannot truly understand. Even among those authors who would accept some sort of behavioural test for understanding, there is still a wide range from those who would accept that being able to pass the Turing test, as Turing originally described it, is sufficient evidence for genuine understanding, to those who see it a:~ a necessary, but not a sufficient, condition (e.g. McDermott, 1982; Churchland and Churchland, 19901). Authors such as these demand that, in addition, what goes on in the artificial object should be demonstrably equivalent to what goes on in a human brain (or mind). Part of the disparity between different authors seems to stem from different interpretations of precisely what the terms 'thinking' and 'understanding' actually mean. Some authors appear to believe that there is of necessity something mysterious about understanding. For this reason, when something is demonstrated that appears to show some level of understanding, however primitive, such authors would say something like "Yes, that is very ingenious, but now that I can see how it happens, I can se:e that it has nothing to do with real understanding." Perhaps this is because some people feel that when something has been explained, it has been explained away, and therefore, in this sort of case, one still needs to look somewhat deeper for the true essence of understanding (e.g. Miller, 1981). Where does one stop? Apart from some criterion, such as the Turing test, how is it possible to know whether someone or something else understands anything? I have a fairly clear idea of what goes on in my own mind when I understand something, at least at some level. I believe that this is the result of something, about which I have very little idea, going on in my brain. In addition, it appears that a number of very 'intelligent' things that go on in the brain, such as many aspects of visual processing (e.g. Mart, 1982), do not have any parallel mental event, with only the end result of this processing reaching a mental level (see also Searle, 1990b and Section 6.4). From my observation of other human beings, I am fairly sure that the same sort of things go on in their minds (and presumably brains) as go on in my own, although I have to admit that it is more a matter of belief than of direct evidence. However, as well as making many aspects of behaviour easier to explain, there is a good deal of highly suggestive evidence, in the way of cues such as changes of expression, changes of posture, etc. ~md the responses to such cues in others, in addition to any verbal responses, that lead one to believe in the reality of the states of mind of other humans. Of course, if one were communicating with or via a computer system, as in the Turing test, one would only see written verbal symbols, so that none of these additional cues would be available. This makes it much mare difficult to assess whether there are in fact any internal states analogous to those that one is aware of in oneself. While such cues are very important in everyday life, and indeed form part of the

115

contextual information that helps to resolve ambiguities in spoken language, we are generally prepared to accept that something with which we have communicated only via written symbols can understand--provided it produces appropriate language! 5. MENTAL P H E N O M E N A A problem in this area is that the majority of the terms used to describe mental phenomena, such as thinking, consciousness, understanding, intentionality, etc. have been used in somewhat different senses by different writers, and there has been no entirely consistent definition of them. For example, it is possible to define thinking on the basis that it is a mental process that only humans can perform. If such a definition is used, then it is clear that nothing other than a human would be able think. However, most writers have used the terms in a somewhat wider sense, and have either implicitly or explicitly assumed that particular types of actions and responses reflect, or at least are highly suggestive of, thinking, consciousness, etc., even if displayed by a non-human. This type of definition will be assumed here.

5.1. 'Intentionality' Searle would fit into the category of those who believe that ability to pass the Turing test is a necessary but not sufficient indicator of real thinking or understanding. He has repeatedly stated that a necessary concomitant of understanding is what he refers to as intentionality, which he maintains is strictly a biological phenomenon. He places far more stringent requirements for acceptance of an ability to understand on a non-biological device than on a biological one. Moreover, his reliance on this assumption makes his arguments in relation to the Chinese room essentially circular. What Searle appears to be saying is: Assume that a program that could understand Chinese had been produced. (Of course, as mentioned above, to actually do anything, including understand, it needs to be implemented on a computer which has certain capabilities in terms of input and output, as well as being able to carry out the instructions of the program.) Now imagine that the program is being implemented by a computer consisting of a human operator, and that the program running in the computer consists of a set of instructions that the human operator comes out. While carrying out this program the operator would not come to understand Chinese. The only intentionality around is that of the human operator and he or she does not understand, therefore, no understanding can be taking place. His main justification for stating that the only intentionality around is that of the human operator appears to be that the program is non-biological and therefore cannot have any intentionality anyway. If intentionality as he defines it is necessary for understanding, then he is correct, but that definition short-circuits the whole argument, and makes impossible his stated initial assumption that an 'understanding' program could be written. His

116

R. J. Harvey

conclusion is based on a premise that precludes any other possibility. Alternatively, if the program (or, strictly speaking, the system of which it is the major part) did understand, in line with his initial assumption, then it must, by his definition, possess intentionality, even though it is not biological. This means that any intentionality that happens to be possessed by the particular form of computer on which the system is being implemented (i.e. the human operator) is irrelevant, as I have stressed elsewhere. By intentionality, Searle seems to mean, among other things, the relationship that exists in conscious thought between the subjects of internal thoughts and objects in the outside world, and, related to this, the fact that thoughts appear to reflect intentions to do something. These thoughts are often followed by some active interaction with the outside world, this interaction normally leading to something that is likely to be to the organism's advantage. As pointed out by Boden (1988), not all philosophers use the same definition of intentionality, and Searle's differs considerably from that of Dennett (e.g. Dennett, 1971, 1987). While intentionality may be a useful concept to assist thinking about humans' and animals' actions, it does not seem to me to be a necessary, or even useful, concept when considering what living organisms can do, except perhaps as a metaphor. It seems, at least in the way that Searle uses it, to be very much equivalent to a revised version of the klan vital. At one time, this was believed to be the essential, and essentially mysterious, property that distinguished living from non-living objects. There does not seem to be any good reason why intentionality is any more necessary than klan vital (see also Boden, 1988). On the other hand, the behaviour of animals, especially those with brains, clearly demonstrates something that looks like 'intentionality'. I would argue that this is not in the least surprising, and does not require anything beyond the ordinary physical and chemical properties of matter. Brains have evolved and persisted in organisms because they are inherited (or, at least, the genetic material (i.e. DNA) that contains the instructions for making them is inherited) and they increase the probability of survival of that organism and/or its offspring. This being so, the activities of any form of brain that has persisted will inevitably be directed towards the survival and reproduction of the organism containing it. The activities of brains therefore must demonstrate intentionality. Animals whose brains did not give rise to this sort of property have not survived. However, there does not seem to be any compelling reason why this sort of behaviour should be considered to be exclusively a biological phenomenon, even though biological devices are the only ones that we are currently aware of that demonstrate it. It is a phenomenon that would occur in any device that has to interact, or whose predecessors have had to interact, with a potentially hostile environment in a way that tends to preserve itself and/or its offspring. If some sort of device based on a computer system could demonstrate intentionality, in other words some purposeful interaction with its environment, would it then possess causal powers in line with Searle's (Searle, 1980) requirements that thinking devices must be "machines with internal causal powers equivalent to

those of brains"? For a brain, these causal powers depend on the input and output transducers in the body in which the brain resides. If it could be accepted that a computer system with suitable input and output transducers had such powers, presumably some of Searle's objections to considering devices running computer programs as machines capable of thinking would disappear, because he would then accept that the fact that the operator in the Chinese room does not understand Chinese would no longer be relevant. However, Searle could continue to argue that, although the device might appear to demonstrate intentionality and causal powers, he would know that it did not possess these attributes as soon as he realised that it was run by a computer program. Indeed, the demonstration of any such attribute, or mental phenomena of any sort, runs into the same difficulties. It is possible that Searle is not trying to make nearly such as strong a statement as is implied by the majority of what he has written. He has said in a number of places that understanding, etc. cannot be generated solely by instantiating or running a computer program (e.g. Searle, 1980, 1990a). If that statement is taken strictly literally, and only the computer program is considered, then one cannot but agree with him. This is because, as stated above, a computer program on its own may have potentiality, but until it is actually implemented, it cannot do anything. In order to operate, it requires a computer of some sort, and to do anything that is apparent to an outside observer, the computer needs to possess certain properties (could these possibly be causal powers?) that allow it to receive information from its surroundings and to send information back to its surroundings. However, this makes Searle's statement very restricted in its scope. It leaves it essentially irrelevant to the question of whether computer systems can, in principle, understand, and completely ignores the reality of how computer systems actually operate, as much of what has been written above attempts to show. 5.2. Consciousness

Consciousness appears to be an essential ingredient of understanding as we know it. Can consciousness exist only in strictly biological devices like brains in animals (specifically humans), or could it occur in other devices, such as computer systems running computer programs? In the same way as for understanding, there seems no reason why consciousness should be an all-or-none phenomenon (see also Miller, 1981). Most people would accept that humans are conscious (at least when they are not unconscious!) and that some other animals are too, but not plants, or rocks, or any other non-biological devices that we know of at present. However, where can we draw the line between conscious and non-conscious animals? On the whole, animals with more complex central nervous systems show more signs of the type that we accept as indicating consciousness in humans, but there is no agreement as to precisely where the watershed is. One of the features of consciousness in humans, at least as it appears from introspection, is that one is aware of only a very small number of things at any one time. Moreover, some external or internal stimulus can

Can Computers Think? drive everything else out of consciousness, so it depends on, or is at least associated with, the direction of attention to a very limited number of specific items at any one time. Indeed, there appear to be very specific structures within the brain that are involved in the processes related 'Lo attention (Posner and Petersen, 1990). Moreover, at least in humans, the unity of attention and consciousness depends on the connections between the two cerebral hemispheres provided by the cerebral coramisures, of which the largest by far is the corpus callosum. In normal humans, it is possible to obtain some evidence for two separate information processing systems, which show some features of independent consqziousness (e.g. Bogen, 1986). However, after surgical division of the corpus callosum, it is easy to demonstrate in the subject two consciousnesses that are in many respects independent of each other (e.g. Sperry, 1986). Interestingly, it is much harder to demonstrate two consciousnesses with a similar degree of independence in subjects in whom the corpus callosum has never developed (Milner and Jeeves, 1979). It is possible that consciousness is a strictly biological phenomenon, but no reason has been brought forward as to why this is necessarily so. Some biological devices do not show any evidence of consciousness under any circumstances, so it is certainly not a universal property of biological systems. It seems possible that consciousness is particularly relevant when it is important to distinguish between and act on the different possible interpretations of ambiguous sensory input. Attempts to write algorithms and computer programs to analyse scenes anything c,ther than very simple and restricted ones have made it clear that visual input is usually highly ambiguous, which has made the automation of visual processing a very difficult problem (e.g. Marr, 1982). Ambiguities of a similar sort apply in relation to other sensory modalities. Dawkins (1989) has made the suggestion that consciousness is a.n accompaniment of the ability of an organism with a ~brain to simulate the future (see also Llin~ts, 1987). As Dawkins puts it, simulation is a better strategy for living organisms than trial and error; "overt trial . . . takes time and e n e r g y . . , overt error is ... often fatal". He suggests that consciousness arises when the brain's simulation of the world is sufficiently complete that it includes a model of itself, or perhaps it would be better to say a model of its own actions. This is a somewhat similar explanation for the origin of the mind to that of Barlow (1987, 1990). Barlow suggests that minds develop in animals that have a great deal of social interaction (such as humans). It would be an advantage for them to be able to predict how other members of the species will react to what they do themselves. He proposes a three stage process. In the first, each individual observes others' behaviour; in the case of a human, initially this will nearly always be a baby observing his or her mother. In the second stage, the individual develops an internal representation of how others respond in different circumstances. In other words, the: individual uses concepts that are in effect other individuals' minds. Such a concept has been termed a "theory of mind" and allows an individual to make better predictions of another's

117

behaviour. Finally, as an extension of this, he or she develops a concept of his or her own mind, which is the entity with which other individuals are interacting. Barlow suggests that normal consciousness depends on interactions between individuals' minds and that, in humans, failure to develop a "theory of mind" leads to autism (Barlow, 1980); experimental evidence supports this idea (Baron-Cohen et al., 1985, see also Frith, 1993). Consciousness, in its attention directing aspects, seems to fulfil an important role in improving the chances of an animal's survival, so, once a heritable system that could support consciousness had arisen, it would tend to persist in the offspring of the animal that possessed it. Specifically, for survival, when the sensory input indicates some dangerous situation, such as approaching the edge of a cliff, or some advantageous situation, such as food, it is important to act on that information. If either a predator or some prey is nearby, paying attention to and acting on the presence of the predator or prey (in that order!) is essential. However, we know that this process is not quite as good as it might be; many animals, and humans too, can be lured into a dangerous situation by the presence of food or other potentially advantageous attention-arousing stimuli. In addition, some sort of consciousness may be particularly important when the future position of some moving object in the environment has to be predicted, as for example for a predator trying to catch moving prey, or for prey trying to escape a moving predator. Following up this line, it seems that more processing is required for predators than for prey, as predator species tend to have larger brains in proportion to their body size than closely related prey species. It is also possible that, in humans, a good deal of the survival value of consciousness may result from it leading to the very strongly interactive patterns ofbehaviour that humans demonstrate (Barlow, 1987).

5.2.1. Consciousness and Memory While the preceding three paragraphs give some suggestions as to what consciousness may provide an animal (or other object) that possesses it, and why, once consciousness had arisen (in animals at least), it would persist, they do not give any inkling as to exactly what this strange phenomenon is or how it could have arisen in the first place. A very important feature of consciousness, as it appears from our own experience and from that of interacting with other conscious entities, is that there appears to be a very large measure of continuity in it. Also, responses of conscious entities depend very much on what has happened in the past, and do not seem to be in any way entirely fixed. This suggests that some continuous updating of memory is a prerequisite, and it is difficult to attribute consciousness to something that does not show evidence of this. Thus, in the brain, and in any other device which can be considered conscious, the structure (or program, or data with which the program is operating) must be in a continual process of modification in the light of what is happening, both internally and externally. When the updating of memory is not taking place, but some functions of the brain are still active, as during sleep or general

118

R. J. Harvey

anaesthesia, humans are deemed not to be conscious. It is important to note that the memory of humans (and other animals), is very different from the usual type of memory in a computer system. Human memory is highly selective, and a great deal that is perceived is not remembered for more than a few seconds. In other words, only a small part of what passes through the short-term or working memory is retained in long-term memory (e.g. Baddelely, 1986, 1989; Goldman-Rakic, 1992). Even those things that are remembered for longer periods do not persist unchanged over time, but gradually fade and alter. Computer memories, in the sense described in Section 3.1, store information entire and unaltered until the information is overwritten (or the power is turned off), when all traces of the original information are lost. To carry on a normal conversation, and therefore to be able to pass the Turing test, would require a device that not only carried out updating of its memory all the time, but also retained its older memories. People who have suffered brain diseases or injuries, such as bilateral damage to the hippocampus (in the medial part of the temporal lobes of the cerebral hemispheres), that lead to a permanent loss of the ability to incorporate new information into long-term memory can still appear conscious, so it would appear that short-term memory or working memory (e.g. Baddelely, 1989) may be more important to consciousness than long-term memory. Provided that their long-term memories from the time preceding their injury or disease are resonably intact, they are able to carry on a normal 'social' type of conversation quite well, but their memory defects become apparent when ideas brought up earlier in the conversation are referred to again (e.g. Squire, 1986). Some sufferers from such disorders still appear to be able to make long-term use of new information, even though they cannot consciously recall it (Cohen and Squire, 1980; Cohen et al., 1985). This results from there being two (or possibly more) quite separate forms of long-term memory referred to as explicit (or declarative) and implicit (or procedural or non-declarative) memory to describe respectively the ability to learn material that can be consciously recalled and the ability to learn material that affects behaviour without being consciously recalled (e.g. Schacter et al., 1993; Zola-Morgan and Squire, 1993)--see Section 6.2.

5.3. Ways in Which Understanding May Arise How is it possible that understanding can occur in an organism? If some explanation of this were available, we would be in a much better position to assess whether computer systems could understand, and such an explanation might also provide the key as to how understanding could be incorporated into computer systems. Clearly, as mentioned above, understanding is not an all-or-none phenomenon. Most people would agree that the level of understanding of a very young human baby is relatively small (although it clearly operates in a way that is very different from any existing computer system). What a baby does have is a number of relatively stereotyped behavioural responses to certain sensory stimuli. Many of these responses are associated with emotional reactions, in other words,

reactions that involve the body as well as the brain, and the responses strongly suggest that the stimuli are meaningful for the baby and are therefore understood at some level. Many of the behavioural responses of an infant generate powerful emotional responses in nearby older humans, indicating that there are some fairly stereotyped responses involving the body as well as the brain that occur in older humans. (It has been suggested that we become aware of our emotional responses by the effect that they have on the body at least as much as by the effect that they have directly on our thoughts.) It is certainly possible to devise machines that show some sort of'emotional' response, such as the Machina speculatrix of Grey Walter (1961) that seeks its power supply when its battery voltage becomes low. Also, the devices described by Braitenberg (1984) show attributes, e.g. foresight, egotism, that have been considered to be characteristics of conscious animals, although the behaviour of such simple machines would not be considered by most people to be a reflection of consciousness or understanding. The apparent possession of attributes of this type by such a device depends on its active interaction with its environment. Most modern computer systems are much more complex in their organisation than such a device, but are not considered to show these attributes. An important factor in this is that the interactions of a standard type of computer system with its environment are, by comparison, very limited; the interactions involve symbols only and do not provide any measure of control over or active interaction with its environment. Given the possibility for consciousness and the ability to direct attention, it certainly seems quite plausible that something like a very young baby could learn to associate meaning with those things that happen to it that arouse emotional responses, because they have a built-in 'meaning' or relevance for it. Given the human brain's apparently innate potentiality for the processing of language, this would very easily be extended to the association of meaning with the internal representations of the symbols that are part of language, initially because the symbols would be associated with stimuli that aroused emotional responses. Thus, it would seem to be quite possible for any device with a minimum of built in responses and the ability or even necessity to make sense of its surroundings (as humans and other animals need to in order to survive), to be able to use its interactions with its environment to 'bootstrap' its understanding even of arbitrary external symbols. This raises another query relevant to the Chinese room. If some sort of interaction with the environment is very important for the development of understanding, is it possible for genuine understanding to take place in a program that interacts only via the passing in and out of symbols that represent words, either via the hardware of a computer system or via a human operator, as for the great majority of existing computer systems or for the Chinese room? Even if it could not develop its understanding, it would still possess whatever understanding had been incorporated in its original design. If the possession of intentionality is not an issue, this might be acceptable as understanding at some level, even if not the same as

Can Computers Think? human understanding. However, this brings us back to the difficulty of defining exactly what understanding is.

5.4. Thinking and Algorithms Another line of argument that has been used to suggest that brains cannot operate in the same way as computer systems, is that some aspects of thinking appear to involve steps that are not based on algorithms, while, essentially by definition, all computer programs must be. This has been developed at considerable length by Penrose (1989). The line of argument that Penrose uses is somewhat similar to that of Lucas (1961), artd one of its bases is the observation that human thinkers, when considering complicated logical systems, are able somehow to step outside the system and observe the inconsistencies that are inevitable in any closed logical system (provided it is sufficiently powerful). Such inconsistencies arise because it is possible to make clearly defined statements that are properly constructed in terms of the system but which cannot be shown either to be true or to be false within the system. However, it is often the case that the truth or falsehood of the statement is clear when it is looked at from outside the system. These inconsistencies were shown to exist in all finite logical systems by G6del (1931). The line of argument used by Lucas (1961) has been discussed at considerable length by Hofstadter (1979), and he concludes that i1: is not compelling. Hofstadter's argument can be paraphrased to something like the following: The brain is a finite system, and, at the level of neurones it must work in a consistent and logical way. Therefore: it is a closed logical system to which G6del's theorem must apply. However, while it is considering (or thinking about) systems that are considerably simpler than itself, it will have the ability to step outside that system to determine the truth or falsehood of otherwise indeterminable propositions. However, this will become progressively more difficult as the system under consideration becomes progressively more and more powerful and complex. When the complexity of the system under consideration approaches that of the brain [or thai: which the brain is capable of encompassing], the brain is no longer able to step outside, so that some propositions must remain indeterminable, except by some other system even more powerful or complex. As J. B. S. Haldane remarked, "the world is not only queerer than anyone has imagined, but queerer than anyone can imagine" (Clark, 1968). Chaitin (1982) showed that G/Sdel's theorem applies to a wide range of systems, and gave a variety of proofs for it. He states very clearly that a system has to be built on adequate foundations and that it cannot be used to derive conclusions beyond its foundations: "if one has ten pounds of axioms and a twenty-pound theorem, then that theorem cannot be derived from those axioms". In relation to the brain plus organism system that makes up a human being, a candidate for an indeterminable statement might be whether in fact a human being ha,; free will. If the brain is completely

119

deterministic in its operations, then free will should not be possible, and, as many authors have commented, free will would then be an illusion, although nevertheless an extremely powerful illusion. On the other hand, if free will exists, then the brain cannot be fully deterministic, and G6del's theorem may not apply. Penrose (1989) also addresses the problem of free will, but from another angle, and suggests a completely different type of explanation. He states that it is possible that some, as yet undiscovered, quantum mechanical process may be operating on the brain, or rather on assemblies of neurones within the brain, and that this may lead to it not acting strictly deterministically. This would mean that its behaviour could only be described in terms of probabilities of possible outcomes, in the same way as for the behaviour of sub-atomic particles etc. under the operations of quantum mechanics as currently understood. Thus, for a given starting state, more than one different outcome would be possible, each having its specific probability, and what would happen on a particular occasion would be a result of'chance'. If this is so, then clearly a computer system of the type currently in use could not simulate brain functions at this level of subtlety. This is because computers are designed to be very strictly deterministic in their operations, and so that quantum mechanical effects do not influence the results that they produce. If what Penrose suggests is true, then to simulate the operations of the brain, it would be necessary to incorporate some form of hardware on which these quantum mechanical processes could act in a way similar to that in which they act on assemblies of neurones in the brain. This would presumably mean that its elements would have to be of the same order of size as neurones, or whatever subunits of the brain were actually being affected by the quantum mechanical effects. Whatever may be the case in relation to quantum mechanics, it is clear that very small differences in the initial state of the brain may generate large differences in subsequent actions, and therefore the brain ( + body) is a powerful amplifier of very small effects, as, for example, in acting on the decision between two finely balanced alternatives. In this sense, some operations of the brain are chaotic, and it is quite possible that quantum mechanical effects would be sufficient to 'switch' the brain between two different alternative outcomes.

6. BRAIN PROCESSES 6.1. Development of the Brain Development in an animal is always from a single cell, the fertilised ovum. All cells of the animal are offspring of this single cell. At an early stage in development, different cells switch into different lines, and their offspring then develop into different types of cell in the adult. Some of this switching is believed to be due to some process that permanently disables some part of the genome, so that that cell and all of its offspring are unable to express it. The remainder of the switching process is due to some mechanism which leads to the expression of certain parts of the genome that are not expressed in other cells. During the process

120

R. J. Harvey

of development, the very large number of neurones that are present in the adult brain are formed by the repeated division of precursor cells known as neuroblasts. The distant offspring of some early neuroblasts may develop into any type of neurone, but after the early stage of development the majority of neuroblasts become specialised and their offspring develop only into a limited number of types of neurone. As the neurones mature, their axons and dendrites develop and eventually begin to make contact with the dendrites and axon terminals (respectively) of other neurones. The overall pattern of development, in terms of the location of neurones of a particular type, the pathways which particular axons follow and the different types of neurone with which the axon terminals and the dendrites make contact appears to be laid down in the genetic instructions in the D N A of the cells. The initial pattern of connections which the cells make seems to be specified only, or at least largely, on the basis of the cell types involved. In those pathways that show clear topographical localisation in the adult, the initial set of connections shows relatively little sign of a topographical arrangement, but this becomes more and more apparent during later development. Significant numbers of cells in many parts of the nervous system die during the later stages of development (e.g. Oppenheim, 1991). In the case of motoneurones, the number of cells that survive appears to be very closely related to the number of muscle precursor cells that are available for them to innervate (e.g. McLennan, 1988). However, in the case of other neurones, it is not clear whether the number of target neurones is always the major criterion for selecting the cells that will die (Oppenheim, 1991). The pattern of connections of those neurones that survive appears to be modified as a result of activity in the neuronal pathways, with some connections being strengthened, some new connections forming, some being weakened and others disappearing altogether. Localised groups of neurones come to receive strong connections from those terminals whose activity is synchronised with each others'. Since, at least during development, presynaptic neurones that receive inputs in common (and are therefore connected to a particular part of the body surface for somatosensory neurones, or a particular part of the retina for visual neurones) tend to fire synchronously, this process leads to a progressive refinement and sharpening of the topographical maps at the next step of the pathway (e.g. Shatz, 1992). In the early stages of development, changes in structure and patterns of connections appear to take place very readily in the cerebral cortex, but occur less readily after the initial phase of development. However, in young animals, the cortex still remains very easily modifiable, and in humans, this period seems to extend well beyond 2 years of age, and perhaps up to about 9 years, which is said to be the upper limit of easy acquisition of language (Penfield and Roberts, 1959) as well as the upper age limit for any large degree of functional recovery from amblyopia, as mentioned in Section 4.2.1. At greater ages, the capacity of the cortex to become modified declines, although it nevercompletely loses the ability--normal humans of any age can still encode and remember new information and the potentiality for this type of remodelling is never lost (e.g. Kaas, 1991).

The structure of the neocortex of the cerebral hemisperes seems to be basically very uniform, with, in primates, the possible exception of the areas that become the primary visual cortex; it has been suggested that the cytoarchitectonic differences between areas is a result of the different patterns of connections that they make with other areas of the brain, in particular the relative numbers of connections to and from the thalamus and to and from other areas of cerebral cortex (Rockel et al., 1980). This uniformity in structure appears to be associated with a uniformity of potentiality, since those subjects in which one cerebral hemisphere has been removed or parts of the cerebral cortex damaged at an early age may show relatively few functional deficits as an adult since other areas of cortex have taken over the usual functions of some at least of those areas that have been lost (e.g. Villablanca et al., 1984). For example many of the 'unilateral' functions, such as most aspects of speech and spatial perception that in most people mainly are carried out in the left and right hemisphere respectively, may be relatively normal in someone who has only one functional cerebral hemisphere, although he or she will still show signs of paralysis of voluntary movements of the side of the body opposite to that of the non-functional hemisphere (e.g. Ptito et al., 1987).

6.2. Mechanisms of Memory The mechanisms that are responsible for human and animal memory are still very far from clear. However, a plausible proposal is that its basis is processes occurring throughout the cerbral cortex (and other parts of the central nervous system, at least as far as procedural memory is concerned) that alter the strength of connections between different neurones, or may actually lead to new connections being formed and other connections disappearing, or at least becoming non-functional. The processes that lead to long-term changes in the potency of synapses may be similar, if not identical, to the long-term potentiation (LTP) that was first described by Bliss and Lomo (1973) in the hippocampus, where it occurs readily and has been very extensively studied, but it also occurs in other regions of the cerebral cortex (e.g. the motor cortex, Iriki et al., 1989; see Tsumoto, 1992 for a review). The converse of LTP, i.e. long-term depression (LTD), has also been described, originally in relation to the parallel fibre to Purkinje cell spine synapses in the cerebellar cortex (reviewed by Ito, 1984, 1989), but has been shown to take place in the cerebral cortex as well (see Siegelbaum and Kandel, 1991; Tsumoto, 1992). It has been suggested that nitric oxide is an important mediator that may be involved in both LTP and LTD (Snyder, 1992). In the absence of reinforcement, both LTP and LTD tend to gradually decay over time. LTP certainly appears to provide a mechanism by which the type of long-lasting modification of synaptic efficacy first proposed by Hebb (1949) could take place, in that activation of a presynaptic element, taking place close in time to activation of the post-synaptic neurone, is followed by an increase of its power to subsequently activate the post-synaptic neurone. That such mechanisms are implicated in explicit memory is supported by the fact that pharmacological agents and other procedures

Can Computers Think? that prevent LTP developing in the hippocampus after the correct stimulating pattern has been applied also interfere with the laying down of memories (e.g. Morris et al., 1986). On the above basis, memories are stored as long lasting changes in synapses within the same regions of the nervous system that normally participate in the processing, analysis and perception of the information that is being remembered (Squire, 1986). The initial stages of these changes are initiated rapidly, by a process that may involve a relatively short-term increase in the concentration of free calcium ions in the presynaptic terminals, analogous to the post-tetanic potentiation that occurs at the neuromuscular junction (Madison et al., 1991). The long-lasting changes that are associated with long-term memories take longer to develop, and a much longer time, which may be of the order of years in humans, to fully consolidate. Interference with normal brain activity (such as that caused by epileptiform activity or a blow on the head) a short time after the memory has been acquired may stop the long-term changes in the synapses from being retained so that the memory is not consolidated and is completely lost (Squire, 1986). Recalling a memory would depend on something taking place that initiated the same pattern of neuronal activity as occurred during the original event. Since the relevant synapses onto these neurones have been potentiated in the laying down of the memory, this pattern of activity ,=an now be triggered by an input which is only a subset of the input activity that originally set it into action. This in turn means that the responses of the n~urones that are activated by the neurones involved in the storage of the memory would now be very similar if not identical to those that occurred when the memory was first laid down, despite the input being only a subset of the original. They would therefore set in motion patterns of activity that very closely resembled those that occurred when the remembered even! first took place. This greater similarity of the patterns of activity in the later stages of the response to a stimulus would then correlate with the ability of humans and other animals to recognise patterns and respond appropriately to them, even when only a part of the pattern is used as a stimulus. It seems almost certain that a conscious memory depends on the pattern of activity in a considerable number of neurones, and that any individual neurone is part of the subset of neurones that participate in a number of different functional activity patterns. The pattern of strengths of the different synapses on the neurone will change over time, as the changes in synaptic efficacy generated as a result of a particular input decay and a~,~the synaptic strengths are altered due to the neurone receiving different patterns of input. This would mean that if a particular pattern of input occurs only very occasionally, it might not lead to quite the same pattern of activation of neurones as it had the previous time, or a greater proportion of the original input patt,~rn would have to be activated by the stimulus to set the memory pattern into action. This would correlate with the gradual fading of most memories and the increasing likelihood of particular stimuli not being recognised as familiar with increasing time since their previous occurrence, even to the stage of them rLo longer being elicitable at all (e.g.

121

Squire, 1986). In this way, information in the brain is stored as differences in synaptic strengths or patterns of connection in a distributed sort of way within the brain's structure, rather than in specific locations, as it is in a normal computer system (see Section 3.1). A corollary of this is that the brain automatically carries out operations that are determined by its current structure and state (in terms of the overall current pattern of neuronal activity within it), rather than executing a program in the same sort of way as a computer does. While the above scheme seems plausible for activating memories as a result of an external stimulus, it is much less clear how memories would be recalled in the absence of an external stimulus. What is the stimulus used when trying to recall a particular memory, and how is this stimulus generated? Why is it that an elusive memory sometimes appears in consciousness some time after an attempt to recall it, with no obvious immediate trigger? Most humans can actually bring into consciousness very little of what happens to them before the age of 2-3 years, even though it is very important to their subsequent functioning, which implies that whatever is remembered is stored as non-declarative or procedural memory. Included in this is the vast amount of learning that has to take place to develop the capacity to interpret visual images (see Section 4.2.1) and the capacity to process one's native language. A similar degree of learning is very likely to be necessary in relation to the interpretation of information reaching the cerebral cortex (and other regions of the brain) from other sensory modalities. This is particularly clear to anyone who has learned a foreign language as an adult. At first, the language appears to be a string of random sounds. One gradually learns to pick out words, and eventually (with luck and a lot of persistence!), the sounds make sense without having to be considered as sounds, and without having to be translated into one's native language. In the same sort of way, a great deal of learning is necessary in developing motor coordination. The vast majority of the skills learnt in the first 2 years or so of life operate below the level of consciousness, and are not accessible to introspection, even though they have major effects on behaviour. Even when learnt as an adult, motor and perceptual skills (such as those for language) that need conscious monitoring during the leaming process can become completely 'automatic' and come to operate below the level of consciousness (e.g. Sternberg, 1988)--as is also suggested by the learning of operational skills by those people who have lost their ability to lay down new declarative memories. Such subjects are able to learn new skills, but they are unable to recall the situation in which they learnt the skill and it seems to them to be completely unfamiliar (e.g. Squire, 1986). 6.3. Hierarchies in the Central Nervous System The mammalian nervous system is organised in a very hierarchical way, in the sense that, for the majority of functions--especially the so-called higher functions--there are many regions of the nervous system, operating at different levels, that all have a role in implementing the function. The highest level of the

122

R. J. Harvey

hierarchy is very often some area of the cerebral cortex. This sort of arrangement seems to be partly related to evolutionary changes in the nervous system, in that more recently evolved regions of the nervous system relating to a particular function do not take over all aspects of the function, and those parts of the nervous system that, by comparison with other modern animals, were almost certainly previously involved retain some aspects of the processing. For example, in relation to visual processing, in modern amphibia, and presumably in the common ancestors of modern amphibia and mammals, the optic tectum, corresponding to the superior colliculus in mammals, is the 'highest' part of the nervous system taking part in visual processing. Many aspects of visual processing in mammals have been taken over by the cerebral cortex, but some, for example focussing of the eyes, adjustment of the diameter of the pupils and moving the direction of the gaze to stimuli in the periphery of the visual field, still depend on the superior colliculus and adjacent regions of the midbrain. Some degree of visual processing of form, colour, etc. still takes place here as indicated by the ability of people who have lost the functions of the cerebral cortex of one side to identify stimuli in the 'blind' half of their visual fields, even if they cannot consciously perceive them (see Section 6.4). This hierarchical arrangement and the involvement of many separate regions of the nervous system may or may not be an essential feature of any sensory processing device. In relation to motor control, much of the detailed control of limb movement is organised in the spinal cord. For example, the succession of muscle contractions that are involved in locomotion and many other movements are brought about by a series of pattern generators in the spinal cord that, to a very large degree, operate autonomously. In addition, some aspects of motor control involve feedback from peripheral receptors direct to neurones in the spinal cord via spinal reflex mechanisms. These pattern generators and other mechanisms are regulated by axons descending from neurones of the brain, some of which originate from the cerebral cortex, but important regions of the brainstem involved include the red nucleus and tectum of the midbrain, the reticular formation of the pons and medulla and the vestibular nuclei. In generating a movement, one has to think consciously only of the movement to be produced. This presumably involves some part of the cerebral cortex. However, details of the movement, in terms of the particular muscles that have to be activated, the strength and timing of their activation, adjustment for load, position of the body and position of the target, are all taken care of by the lower levels of the hierarchy, which include the brainstem nuclei already mentioned as well as the basal ganglia, the cerebellum and, probably, the primary motor area of the cerebral cortex. If the cerebral cortex is damaged on one side, many active movements of the opposite side of the body can still occur, although they may not be under full voluntary control. As for sensory systems, whether this hierarchical arrangement is a necessary feature of a device that has the control properties of a nervous system or whether it is a by-product of the way in which the nervous system has evolved and developed is not at all clear--it may well

be possible to provide these functions without such an arrangement. However, it does have the advantage of not having a lot of low-level details cluttering up the highest level of the hierarchy, which is presumably somehow related to consciousness. Having too many details at the top of the hierarchy could well create difficulties in picking out and responding to the important features, in other words in separating out the 'frame' (see Dennett, 1984). As well as the systems and subsystems of the nervous system described above, between which and within which the pattern of connections is very highly organised, there are also systems in which a relatively small number of neurones project diffusely to very widely separated regions. No topographical organisation has yet been discovered in these projections which include the noradrenaline secreting neurones of the locus coeruleus and the 5-HT (serotonin) secreting neurones of the raphe nuclei. In relation to the latter, it has recently been suggested that it has considerable importance in relation to motor control: "5-HT serves an integrative overarching function, rather than being discretely and separately involved in the great diversity of behavioral and physiological processes in which it has been implicated." (Jacobs and Fornal, 1993). The functions of these systems is not yet resolved, but it is possible that they provide some functions that are essential for the correct operation of the nervous system, and which will have to be included in any overall simulation of the nervous system. 6.4. Brain Processes that may be Associated with Consciousness

It has been proposed that consciousness reflects the activity of the cerebral cortex as a whole. However, it is apparent from studies of visual (and other sorts of) sensory processing, that a great deal of complex processing in the cortex takes place at a level which is not accessible to consciousness, and that what appears in consciousness is a "construct of the brain" (Zeki, 1992). In visual processing, many operations on the input that reaches the cerebral cortex from the retina take place in parallel in spatially separated regions of the cerebral cortex, all of which contribute to the construct that reaches consciousness. Moreover, within each of these areas, processing of a particular 'style' of feature, e.g. colour, takes place in a large number of parallel subareas, each responding to a particular aspect of the feature. The construct that reaches consciousness appears to depend on a great deal of interconnection and feedback (sometimes referred to as re-entrance) between the various areas. For some of these areas, very specific losses in what is perceived occur if the area is damaged in some way, and the subject may not always be aware that there is any problem with his or her perception (reviewed in Zeki, 1993). If the primary visual cortex (area 17, also known as area V1) is damaged, the subject loses all conscious visual perception in the opposite half of his or her visual world. Despite this lack of conscious awareness, the subject is still able to point accurately at a stimulus in the blind region, detect movement in the blind region and show other evidence of quite detailed visual processing of stimuli in the blind region (see Weiskrantz, 1986, 1990; Celesia et aL, 1991); this

Can Computers Think? appears to depend at least partly on processing in other visual areas of the c,:rebral cortex, since patients with lesions only of the primary visual cortex show a wider range of unperceived visual processing than those who have lost a complete hemisphere (Ptito et al., 1991). Similarly, patients who have lost their ability to consciously perceiw,• spatial information, may nevertheless be able to correctly orient their hands in relation to spatial features that they have not consciously perceiw~d (Marshall and Halligan, 1988; Goodale et al., 199][). This suggests that only part of the neural activity in the cerebral cortex contributes to consciousness, even if it all contributes to behaviour. Moreover, there is evidence that regions outside the cerebral hemisphe:res can also affect conscious perception, such as the cerebellum in relation to the perception of movement (Ivry and Diener, 1991; Paulin, 1993) and other regions of the nervous system in relation to the perception of pain: "the appreciation of pain come about as a result of the coordinated interactions of all parts of the nervous system related to somatic sensitivil:y." (Albe-Fessard et al., 1985). In relation to the perception of vision, it has been suggested that the ~results of visual processing reach consciousness when the activities of the various neurones in the different visual areas of the cerebral cortex that respondL to the various properties of the stimulus all discharge in some sort of synchrony (Zeki, 1993). However, the neurones involved must include those of the primary visual cortex, which relays most of the information that reaches the other areas that work in parallel with one another to extract different higher level features of the visual input. This must be the case since no conscious visual perception occurs in the absence of the primary visual cortex, as mentioned above. It certainly seems to he the case that widely spaced neurones of the visual areas of the cerebral cortex that are responding to features of the same stimulus show syr~chronised oscillatory behaviour (e.g. Eckhorn et al., 1988; Gray et al., 1989). A similar degree ofsynchronisation has also been found between neurones in the viisual areas of the two separate cerebral hemispheres when responding to a common feature of the visual input (Engel et al., 1991). Apart from the suggestion by Zeki (1993) that this may be related to conscious perception, it has also been suggested that it is important for setting up 'cell assemblies' (Singer, 1990), in other words, groups of cells that contribute to the particular pattern of activity associated with the response to a particular stimulus and that have been proposed as the basis for memory (see Section 6.2). 6.5. Parallel Processing and Brain Simulation In this context, there are two types of parallel processing system that need to be considered. One is like the type of computer system described in Section 3.1, but having a large number of CPU's operating simultaneously in parallel. This would use a stored program (or series of stored programs) in just the same way as a computer with a single CPU, but it would, in principle, be able to operate faster by a factor close to the number of processors it had, which would be an advantage for the simulation of responses of the brain in 'real time' (see Section 4.2.2). Such systems have

123

been used to good effect in a number of types of supercomputer, especially those designed for processing arrays of data. In this type of system, each processor works on a subset of one array to produce results that are stored in a subset of another array; for some types of operation, they may all be able to use the same program, although each is likely to need its own memory for storing intermediate results etc. The other type of system is what has been called a neural network computer, or just a neural network. It consists of a large number of simple processing units in an array, and usually has a number of arrays of these units connected in series with each other (e.g. Rumelhart and McLelland, 1986; Caudill and Butler, 1990). Despite having a number of processing units, devices of each of these types are finite machines, and can therefore be simulated on a single processor digital computer, to any arbitrary degree of accuracy (although the speed of operation is likely to be much less) and, in practice, neural networks are at present nearly always tested by being run as simulations on a digital computer. In a neural network, the processing units act something like neurones in that they sum (or combine in some other way) the inputs that reach them, and produce an output according to the amount by which the processed input exceeds a threshold. Some authors have used networks with very simple processing units, whereas others have incorporated many details of the operations of real neurones (e.g. Ekeberg et al., 1991). The processors in the first array act upon a pattern of stimulation received along an array of input lines, each of which is connected to a large number of the processing units (most often to all of them). The output of the first layer of processing units is fed into the second layer of units, and their output fed to the third layer (if there is one), and so on. The output of the final layer is taken to be the output of the system. The strengths of the various connections can be changed, which will alter the pattern of output. Various procedures have been used for altering the weights of the connections to try to make the network improve its performance of some particular desired function. Some of these have allowed the whole network to 'learn' to perform a task well. An example of such a network is that developed by Grossberg and Merrill (1992), which demonstrates sophisticated learning abilities comparable to some of those of the nervous system, using methods that are firmly based on neurophysiological principles. An attraction of using a parallel processing system to simulate brain functions is that its organisation is very reminiscent of many aspects of the organisation of the central nervous system (see Caudill and Butler, 1990 for more details). This suggests that a parallel processing system may function in a way that is very similar to how the brain functions, although this has not been conclusively shown to be true. Another possible advantage of using a parallel, neural network type of machine, which could be of enormous importance, derives from the fact that neural networks appear to be able to learn to perform complex tasks, such as pattern recognition, without needing specific instructions on how to carry out the task. Indeed, given some very simple general rules of organisation, without any specific instructions at all, some neural networks, especially those with a number of layers of

124

R. J. Harvey

processing units, appear to be able to learn to carry out 'perceptual' tasks comparable to those that brains can perform, and both learn and perform the task in a way that may well be very similar to the way that brains carry them out (e.g. Linsker, 1990). If this proves to be the case, then it may be possible to construct a computing machine able to carry out functions comparable to the cognitive or other 'higher' functions of the brain in the same way as the brain does, without having to solve in advance the problem of the precise algorithms that are necessary. However, for the great majority of neural networks so far studied, learning seems to take a much larger number of trials than it does for an animal, even with the use of techniques for altering the strengths of the different connections that appear to be quite unlike those that the brain uses, and which, in effect, interpret the input for the network. On the other hand, many of the declarative memories of humans and other animals are set up by a single exposure to a unique event, with no outside interpretation being available.

7. S O M E PRACTICAL DIFFERENCES BETWEEN ORGANISMS AND COMPUTER SYSTEMS In some ways, brains in animals are very much more robust than computer systems, while, in other ways the reverse is true. Modern computer systems are much more robust than their predecessors, in that they require much less in the way of control of their physical environment and are much less easily damaged by mechanical shocks. However, they still require a much better controlled source of energy than animals. Most computers have very little energy storage, so that they stop functioning very soon after being disconnected from their power supply. Some battery powered computers are beginning to escape from this restriction, but still require closely specified sources of power to recharge their batteries (or they require new batteries of a particular specification). While brains need a fairly well regulated energy supply (in the form of glucose and oxygen), animals are much more flexible than computers in the timing and form of the energy supplies that they require. On the other hand, for most animals, a great deal of the function of the nervous system and body is devoted to securing the continuation of an energy supply and converting it to a form that the brain and body can use. In addition, animals are much more dependent on a maintained supply of energy; interruption of the energy supply for longer than its stores can last (or, for most animals, interruption of the oxygen supply for more than a few minutes) leads to an irreversible breakdown (i.e. death) of the whole animal since, especially in the brain, ongoing metabolism is required to maintain the physical structure at the cellular level. A computer, on the other hand, does not require any energy to maintain its physical structure, and it is easy to make a computer which saves the contents of all 'volatile' storage in a stable form (e.g. on a magnetic disc) if power is interrupted. If the computer is made so that it restores the contents of its volatile memory when power is reapplied, it can continue precisely from where it left off. For somewhat similar reasons, it is

possible to save the current state of a computer system exactly, and to make a duplicate from this information. For an animal, on the other hand, only the outline of the structure of the brain is laid down by genetic instructions, while the details are filled in as a result of preceding activity and sensory input (see Section 6.1). (Identical twins differ from each other at birth despite identical genetic instructions, and diverge further as they get older.) There is no way available at present for reading out the structure, which means that it is impossible to make an exact duplicate. An identical copy of an animal and its brain could only be made by making an atom-by-atom duplication, and this is only available in the realms of science fiction [see Hofstadter and Dennett (1981) for some potential consequences if this were possible]. However, note that even if an exact copy of a 'conscious' computer system were made, as soon as the two systems (the original and the copy) started to interact with their environments, they would rapidly diverge, both in their detailed structure and in their subsequent behaviour, as they would experience different events. Even when experiencing the same events, they would do so from different points of view. Present day computer systems and brains also differ in another important respect in their response to injuries or minor malfunctions. In a computer system, the failure of the smallest component may alter its behaviour a great deal, and may even stop it operating altogether. Brains on the other hand seem to tolerate minor degrees of injury or damage, or malfunctions (or even failure) of individual elements with much less dramatic effects. Their performance degrades, but the performance of an individual damaged brain does so in ways that may be very hard to detect. There is only very rarely a major qualitative change in observable behaviour unless a major injury has occurred. However, it is quite possible to make computer systems that are much less affected by damage to or malfunctions of the hardware, by having a number of each type of component running in parallel and taking the output of the majority to be the correct output. Any arbitrary degree of fault tolerance can be achieved by having a sufficient number of each component. Such a system in some respects then resembles the nervous system, where in nearly every region, there are a large number of apparently similar components operating in parallel. Neural network computers show even better fault tolerance (unless they are being simulated on a single CPU computer!) where, as in the brain, the operations depend on the strengths and patterns of a very large number of connections, and the loss of a few individual co,.nections or processing units has a very small effect on the overall operation (see Caudill and Butler, 1990). If it proves possible to make a computer based device that is capable of thought, its behaviour would differ considerably from that of any current computer system, even though a computer system would be its most important component. Moreover, it will probably need to pass new and more rigorous tests than those that have so far been proposed before its ability to think becomes widely accepted. Since the behaviour of such a computer based thinking machine would depend on its experience, its response in a particular situation would almost certainly be of a

Can Computers Think? degree of unpredictability comparable to that of humans or animals. However, it may well turn out to be difficult or even irapossible to make such a device think just like a human. This might mean that a system that was accepted to be able to think might be differentiable from a human on the basis of something comparable to the TuLring test or a new test developed to replace and extend it (see Section 4.2.2)--although it might well be able to learn to pass a test of this type for being human by consciously producing 'human' responses! Similarly, humans might have to consciously produce responses similar to those that would be generated by a computer based thinking machine in order to pass a test for being such a machine. This is because human thinking is, in most respects, so strongly influenced by the fact that we possess bodies of a particular form that are quite easily damaged, that require energy sources of a particular type, that reproduce in a certain way, and so on. Most physical aspects of a computer based thinking machine would clearly be very different. The properties of the body provide a basis for our emotions and emotional thinking, but in addition the state of the body can have a very powerful effecl: on the perceptions, not only of the body itself, but also of the outside world, as graphically described by Sacks (1991). The body and its properties underlies our 'higher', more rational and more logical thought to an extent that is very difficult to estimate. 'Thinking' computer systems, as mentioned above (Section 5.3), may need a basis of something comparable to the emotional responses of humans in order to develop their own understanding of the world, but th:is would inevitably be different from that of humans. ,One can imagine that a computer based thinking machine would have some difficulty in appreciating the finer points of a good meal, just as we cannot appreciate the finer points of different power outlets! It would be an interesting experience to converse with such a machine, and it would almost certainly provide us with some novel insights into many aspects of human thinking and behaviour. From the discussion above, it follows that a computer based thinking machine would not necessarily, and maybe could not under any circumstances, think jiust like a human being. For this reason, there may be many areas where it would not be possible or appropriate for computer based thinking machines to be used to take decisions that might affect humans. The sort of problems and difficulties that are likely to result, particularly as humans would very much like to have something that would relieve them of the responsibility of making decisions, have been discussed by Weizenbaum (1976). Even so, computer based thinking machines should be able to help humans reach decisions in a much more sensitive and flexible way than current computer systems can. On the other hand, there would be at least as much scope for them to be used inappropriately.

8. FINAL C O M M E N T S A number of difficulties are still painfully clear. First of all there are no universally acceptable or consistent definitions of such terms as thinking, consciousness or understanding. Equally there are no widely accepted

125

criteria for determining whether some entity is conscious or is able to think or understand. It seems to me that there are three unresolved questions, which are independent, although closely related. These are: 1. how consciousness, thought and other mental phenomena arise; 2. what is required in the way of physical and logical organisation in order that any device (brain, computer system, etc.) is able to think, etc.; and 3. whether non-biological devices are in principle able to think and to possess other 'mental' attributes. The third of these questions is the one to which the majority of the above has been directed, while the first is the classical mind/body problem, which I have very largely avoided up to now. The second lies somewhere between the other two, but is still completely separate from either. If it could be conclusively shown that the third question had a positive answer, then a concerted attack on the second question might allow us to answer the first. In relation to the first of the above questions, the views expressed above may be considered to be strong support for a materialist theory of the mind, thinking, etc. A materialist theory states that mental phenomena simply are features of the brain and its activities, in the sort of way that is described in Chapter l of Searle (1984). In our present state of knowledge, a theory of this type would be the most parsimonious explanation for the phenomena of mind (e.g. Churchland, 1988). However, that does not necessarily mean that it is correct, and because the three questions given above are independent, the viewpoint followed here does not exclude other possibilities. For example, it is not entirely incompatible with the idea that human and all other consciousness depends on some form of universal consciousness that pervades the universe. On this basis, the consciousness of individuals has been considered to be something like waves on the surface of this universal consciousness. As set out by Campbell (1973), entities with less well developed consciousness generate smaller waves than humans (and, in any case, different humans make waves of different sizes). If consciousness is universal in a way similar to that which Campbell describes, then non-biological devices should be able to make waves in it in the same sort of manner as biological ones. Equally, it does not necessarily exclude the dualist-interactionist view set out by Popper and Eccles (1977), and it is conceivable that the brain in their scheme could be replaced by a suitable computer system. Such a computer system would require some sort of liaison module, equivalent to the liaison brain of Popper and Eccles. The precise form that this would have to take is not clear. However, it would be required to interact with the inner sense, etc. of World 2 of Eccles and Popper, where, according to their formulation, consciousness and intentions reside. Eccles (1986, 1990) has proposed that the interaction between World 2 and the brain takes place in the cerebral cortex, in relation to units that he refers to as dendrons, which are structures consisting of the closely apposed apical dendrites of a number of pyramidal nerve cells. Eccles suggests that

R. J. Harvey

126

this i n t e r a c t i o n d e p e n d s o n q u a n t u m m e c h a n i c a l effects. If this were so, t h e n t h e liaison m o d u l e in a c o m p u t e r s y s t e m w o u l d h a v e to r e s p o n d to q u a n t u m m e c h a n i c a l effects in t h e s a m e s o r t o f w a y as t h e d e n d r o n s . Eccles' p r o p o s a l t h e r e f o r e h a s a n u m b e r o f parallels w i t h t h a t o f P e n r o s e (1989). In a n s w e r to t h e s e c o n d o f t h e q u e s t i o n s at t h e b e g i n n i n g o f this section, we c a n s a y ( m o r e o r less b y definition) t h a t a h u m a n b r a i n (plus b o d y ) h a s t h e r i g h t sort o f p h y s i c a l a n d logical o r g a n i s a t i o n to be able to t h i n k , etc. b u t we d o n o t yet h a v e a n y g o o d i n f o r m a t i o n o n w h e t h e r this is t h e o n l y f o r m o f o r g a n i s a t i o n t h a t c o u l d p o s s e s s t h e s e abilities, n o r w h a t t h e l o w e r limits o f c o m p l e x i t y are. S o m e a n i m a l s t h a t h a v e b r a i n s t h a t a r e less c o m p l e x t h a n t h o s e o f h u m a n s s h o w b e h a v i o u r t h a t m a n y p e o p l e a c c e p t as showing some degree of consciousness, thought or understanding. In r e l a t i o n to t h e t h i r d q u e s t i o n , Searle h a s c l a i m e d that he has demonstrated that computers running c o m p u t e r p r o g r a m s a r e u n a b l e to t h i n k o r u n d e r s t a n d , a n d t h a t n o n - b i o l o g i c a l d e v i c e s in g e n e r a l h a v e to m e e t much more stringent requirements than biological o n e s b e f o r e t h e y c a n be c o n s i d e r e d to be able to t h i n k . H o w e v e r , I believe t h a t I h a v e e s t a b l i s h e d t h a t t h e s e c l a i m s a r e n o t justified, a n d t h a t his m a i n a r g u m e n t in r e l a t i o n to c o m p u t e r p r o g r a m s is a c t u a l l y i r r e l e v a n t to t h e q u e s t i o n . T h e r e d o e s n o t c u r r e n t l y a p p e a r to be a n y c o m p e l l i n g r e a s o n to a s s u m e t h a t t h i n k i n g , etc. m u s t be r e s t r i c t e d to b i o l o g i c a l d e v i c e s r a t h e r t h a n b e i n g p o s s i b l e also in n o n - b i o l o g i c a l o n e s . T h i s is d e s p i t e t h e fact t h a t all c u r r e n t devices t h a t t h e g r e a t m a j o r i t y o f p e o p l e a r e p r e p a r e d to a c c e p t as b e i n g able to t h i n k , u n d e r s t a n d , etc. a r e biological.

REFERENCES Albe-Fessard, D., Berkley, K. J., Kruger, L., Ralston, H. J. and Willis, W. D. (1985) Diencephalic mechanisms of pain sensation. Brain Res. Rev. 9, 217-296. Baddelely, A. (1986) Working Memory. Oxford University Press: Oxford. Baddelely, A. (1989) The psychology of remembering and forgetting. In: Memory: History, Culture and the Mind, pp. 33-60. Ed. E. Butler. Basil Blackwell: Oxford. Barlow, H. (1980) Nature's joke. In: Consciousness and the Physical World, pp 81-94. Eds B. D. Josephson and V. S. Ramachandran. Pergamon Press: Oxford. (Cited from Barlow, 1987). Barlow, H. (1987) The biological role of consciousness. In: Mindwaves, pp. 361-374. Eds C. Blakemore and S. Greenfield. Basil Blackwell: Oxford. Barlow, H. (19903 The mechanical mind. Ann. Rev. Neurosci. 13, 15-24. Baron-Cohen, S., Leslie, A. M. and Frith, U. (19853 Does the autistic child have a "theory of mind?" Cognition 21, 37-46. Blakemore, C. B. and Van Sluyters, R. C. (19743 Reversal of the physiological effects of monocular deprivation in kittens: Further evidence for a critical period. J. Physiol. 237, 195-216. Bliss, T. V. P. and Lomo, T. (1973) Long-lasting potentiation of synaptic transmission in the dentate area of anaesthetized rabbit following stimulation of the perforant path. J. Physiol. 233, 334-356. Braitenberg, V. (19843 Vehicles: Experiments in Synthetic Psychology. MIT Press: Cambridge (MA). Boden, M. (19883 Computer Models of Mind, Chapter 8. Cambridge University Press: Cambridge. Bogen, J. E. (19863 One brain, two brains, or both? In: Two Hemispheres

One Brain: Functions o[' the Corpus Callosum,

pp. 21-34. Eds E. Leporr, M. Ptito and H. H. Jasper. Alan R. Liss: New York. Campbell, A. ( 1973) Seven States of Consciousness. Victor Gollancz: London. Caudill, M. and Butler, C. (19903 Naturally Intelligent Systems. MIT Press: Cambridge (MA). Celesia, G. C., Bushnell, D., Toleikis, S. C. and Brigell, M. G. (1991) Cortical blindness and residual vision: Is the "second" visual system in humans capable of more than rudimentary visual perception? Neurology 41, 862-869. Chaitin, G. W. (1982) G6del's theorem and information. Int. J. theor. Phys. 21,941 954. Churchland, P. M. (19883 Matter and Consciousness (revised edition). MIT Press: Cambridge (MA). Churchland, P. M. and Churchland, P. S. (19903 Could a machine think? Sci. Am. 262(!), 26-31. Clark, R. (1968) J. B. S. The Life and Work of J. B. S. Haldane. Hodder and Stoughton: London. Cohen, N. J., Eichenbaum, H., Deacedo, B. S. and Corkin, S. (1985) Different memory systems underlying acquisition of procedural and declarative knowledge. Ann. N.Y. Acad. Sci. 444, 54-71. Cohen, N. J. and Squire, L. R. (1980) Preserved learning and retention of pattern analyzing skill in amnesia: Dissociation of knowing how and knowing that. Science 210, 207-210. Dawkins, R. (1989) The Selfish Gene (edn 2). Oxford University Press: Oxford. Dennett, D. C. (1971) Intentional systems. J. Phil. 68, 87 106. Dennen, D. C. (1980) The milk of human intentionality. Behav. Brain Sci. 3, 428-430. Dennett, D. C. (1984) Cognitive wheels: The frame problem of AI. In: Minds, Machines and Evolution: Philosophical Studies, pp 129-15 I. Ed. C. Hookway. Cambridge University Press: Cambridge. Dennett, D. C. (19873 The Intentional Stance. MIT Press: Cambridge (MA). Dreyfuss, H. (1992) What Computers Still Can't Do. MIT Press: Cambridge (MA). Eccles, J. C. (1986) Do mental events cause neural events analogously to the probability fields of quantum mechanics? Proc. R. Soc. B 227, 411-428. Eccles, J. C. (1990) A unitary hypothesis of mind-brain interaction in the cerebral cortex. Proc. R. Soc. B 240, 433-451. Eckhorn, R., Bauer, R., Jordan, W., Brosch, M., Kruse, W., Munk, M. and Reitboeck, H. J. (1988) Coherent oscillations: A mechanism of feature linking in the visual cortex? Biol. Cybernet. 60, 121-130. Ekeberg, O., Wallrn, P., Lansner, A. Tr~vrn, H., Brodin, L. and Grillner, S. (1991 ) A computer based model for realistic simulations of neural networks. Biol. Cybernet. 65, 81-90. Engel, A. K., K6nig, P., Kreiter, A. K. and Singer, W. (19913 lnterhemispheric synchronization of oscillatory responses in cat visual cortex. Science 252, 1177-1179. Frith, U. (1993) Autism. Sci. Am. 268(6), 78-84. Grdel, K. (1931) Uber formal unentscheidebare S~itze der Principia Mathematica und verwandter Systeme, I. Mh. Math. Phy. 38, 173--198. Translated as: On Formally Undecidable Propositions (1962). Basic Books: New York. Goldman-Rakic, P. S. (1992) Working memory and the mind. Sci. Am. 267(3), 73-79. Goodale, M. A., Milner, A. D., Jakobson, L. S. and Carey, D. P. ( 19913 A neurological dissociation between perceiving objects and grasping them. Nature 349, 154-156. Gray, C. M., Krnig, P., Engel, A. K. and Singer, W. (1989) Oscillatory responses in cat visual cortex exhibit inter-columnar synchronization which reflects global stimulus properties. Nature 338, 334-337. Gray, J. (1987) The mind-brain identity theory as a scientific hypothesis: A second look. In: Mindwaves, pp. 461-483. Eds C. Blakemore and S. Greenfield. Basil Blackwell: Oxford. Gregory, R. L. and Wallace, J. (1963) Recoveo'./?om In[ant Blindness: A Case Study. Experimental Psychology Socieo' Monograph H.

Heifer: Cambridge. Grossberg, S. and Merrill, J. W. L. (1992) A neural network model of adaptively timed reinforcement learning and hippoeampal dynamics. Cogn. Brain Res. I, 3-38. Harvey, R. J. (19853 On the nature of programs, simulations, and organisms. Behav. Bra#l Sci. 8, 741-742. Harvey, R. J. (1986) Language processing and computer programs. Behav. Brain Sci. 9, 549-550. Hebb, D. O. (1949) The Organization t~['Behavior. John Wiley: New York.

Can Computers Think? Hofstadter, D. R. (1979) Gi~deL Eseher, Bach: An Eternal Golden Braid. Basic Books: New York. Hofstadter, D. R. (1981)'Reflections' on Searle (1980). In: TheMinds Eye, pp. 373-382. Eds D. R. Hofstadter and D. C. Dennett. Harvester Press: Brighton. Hofstadter, D. R. and Dermett, D. C. (Eds) (1981) The Mind's Eve. Harvester Press: Brighton. Hubel, D. H. and Wiesel, T. N. (1977) Functional architecture of macaque monkey visual cortex. Proc. R. Soc. B 198, 1-59. Iriki, A., Pavlides, C., Keller, A. and Asanuma, H. (1989) Long-term potentiation in the motor cortex. Science 2,45, 1385-1387. Ito, M. (1984) The Cerebellum and Neural Control. Raven: New York. Ito, M. (1989) Long-term depression. Ann. Rev. Neurosci. 12, 85-102. lvry, R. B. and Diener, H. C. (1991) Impaired velocity perception in patients with lesions of the cerebellum. J. Cogn. Neurosci. 3, 355-366. Jacobs, B. L. and Fornal, C. A. (1993) 5-HT and motor control: A hypothesis. Trends Neurosci. 16, 346-352. Kaas, J. H. (1991) Plasticity of sensory and motor maps in adult mammals. Ann. Rev. Neurosci. 14, 137-167. Linsker, R. (1990) Perceptual neural organization: Some approaches based on network models and information theory. Ann. Rev. Neurosci. 13, 257-281. Llinfis, R. (1987) 'Mindness' as a functional state of the brain. In: Mindwaves, pp. 339-358 Eds C. Blakemore and S. Greenfield. Basil Blackwell: Oxford. Lucas, J. R. (1961) Minds, machines and Grdel. Philosophy 36, 120-124. Lycan, W. G. (1980) The fu:actionalist reply (Ohio State). Behav. Brain Sci. 3, 434-435. McDermott (1982) Minds, brains, programs, and persons. Behav. Brain Sci. 5, 339-341. McLennan, I. S. (1988) Quantitative relationships between motoneurons and mtLscle development in Xenopus laevis: Implications for motoneuronal cell death and motor unit formation. J. comp. NeuroL 271, 19-29. Madison, D. V., Malenka, R. C. and Nicoll, R. A. (1991) Mechanisms underlying long-term potentiation of synaptic transmission. Ann. Ret,. Neurosci. 14, 379-397. Mahowald, M. A. and Mead, C. (1991) The silicon retina. Sci. Am. 264(5), 40-46. Marr, D. (1982) Vision. W. H. Freeman: San Francisco (CA). Marshall, J. C. and Halligan, P. W. (1988) Blindsight and insight in visuo-spatial neglect. Nature 336, 766-767. Mead, C. and Mahowald, M. A. (1988) A silicon model of early visual processing. Neural Networks 1, 91-97. Miller, R. (1981) Meaning and Purpose in the Intact Brain. Oxford University Press: Oxford. Milner, A. D. and Jeeves, M. A. (1979) A review of behavioural studies of agenesis of the corpus callosum. In: Structure and Function of Cerebral Commisures, pp. 428-448. Eds i. S. Russell, M. W. van Hof and G. Berlucchi. University Park Press: Baltimore (MD). Moravec, H. (1988) Mind Children: The Future of Robot and Human Intelligence. Harvard Ulaiversity Press: Cambridge (MA). Morris, R. G. M., Anderson, E., Lynch, G. S. and Baudry, M. (1986) Selective impairment of learning and blockage of long-term potentiation by an N-me Lhyl-D-aspartate receptor antagonist, AP5. Nature 319, 774-776. Oppenheim, R. W. (1991~ Cell death during development of the nervous system. Ann. Rev. Neurosci. 14, 453-501. Paulin, M. G. (1993) The role of the cerebellum in motor control and perception. Brain Behav. Et,ol. 41, 39-50. Penfield, W. and Roberts, L. (1959) Speech and Brain Mechanisms. Princeton University Pr,:ss: Princeton. Penrose, R. (1989) The Emperor's New Mind: Concerning Computers, Minds and the Laws of Physics. Oxford University Press: Oxford. Popper, K. R. and Eccles, J. C. (1977) The Selfandits Brain. Springer: Heidelberg.

127

Posner, M. I. and Petersen, S. E. (1990) The attention system of the human brain. Ann. Bet,. Neurosci. 13, 25-42. Ptito, A., Lassonde, M., Leporr, F. and Ptito, M. (1987) Visual discrimination in hemispherectomized patients. Neuropsychologia 25, 869-879. Ptito, A., Lepor& F., Ptito, M. and Lassonde, M. (1991) Target detection and movement discrimination in the blind field of hemispherectomized patients. Brain 114, 497-512. Rabinowicz, I. M. (1983) Amblyopia. In: Pediatric Ophthalmology, pp 293-348. Ed. R. D. Harley. W. B. Saunders: Philadelphia. Rockel, A. J., Hiorns, R. W. and Powell, T. P. S. (1980) The basic uniformity in structure of the neocortex. Brain 103, 221-244. Rumelhart, D. E. and McLelland, J. L. (Eds) (1986) Parallel Distributed Processing: Explorations in the Microstructure of Cognition, VoL 1: Foundations: Vol. 2: Psychological and Biological Models. MIT Press: Cambridge (MA). Sacks, O. (1991) .4 Leg to Stand On. Picador: London. Schacter, D. L., Chiu, C.-Y. K. and Ochsner, K. N. (1993) Implicit memory: A selective review. Ann. Rev. Neurosci. 16, 159-182. Schank, R. C. (1980) Understanding Searle. Behav. Brain Sci. 3, 446-447. Searle, J. R. (1980) Minds, brains, and programs. Behav. Brain Sci. 3, 417-457. Searle, J. R. (1984) Minds, Brains and Science. Penguin: London. Searle, J. R. (1985) Patterns, symbols, and understanding. Behav. Brain Sci. 8, 742-743. Searle, J. R. (1990a) Is the brain's mind a computer program? Sci. Am. 262(1), 20-25. Searle, J. R. (1990b) Consciousness, explanatory inversion, and cognitive science. Behav. Brain Sci. 13, 585-642. Shatz, C. J. (1992) The developing brain. Sci. Am. 267(3), 35-41. Siegelbaum, S. A. and Kandel, E. R. (1991) Learning-related synaptic plasticity: LTP and LTD. Curr. Opin. Neurobiol. 1, 113-120. Singer, W. (1990) Search for coherence: A basic principle of cortical self-organization. Concepts Neurosci. 1, 1-26. Snyder, S. H. (1992) Nitric oxide and neurons. Curr. Opin. Neurobiol. 2, 323-327. Sperry, R. W. (1986) Consciousness, personal identity, and the divided brain. In: Two Hemispheres--One Brain: Functions of the Corpus Callosum, pp 3-20. Eds E. Leporr, M. Ptito and H. H. Jasper. Alan R. Liss: New York. Squire, L. R. (1986) Mechanisms of memory. Science 232, 1612-1619. Sternberg, R. J. (1988) The Triarchic Mind: A New Theory of Human Intelligence. Viking Penguin: New York. Tsumoto, T. (1992) Long-term potentiation and long-term depression in the neocortex. Prog. Neurobiol. 39, 209-228. Turing, A. M. (1937) On computable numbers, with an application to the Entscheidungsproblem. Proc. Lond. math. Soc. (Series 2) 42, 230-265. Turing, A. M. (1950) Computing machinery and intelligence. Mind59, 433-460. Villablanca, J. R., Burgess, J. W. and Sonnier, B. J. (1984) Neonatal cerebral hemispherectomy: A model for postlesion reorganization of the brain. In: Early Brain Damage. Vol. 2. Neurobiology and Behavior, pp. 179-228. Eds S. Finger and C. R. Almli. Academic Press: Orlando. Walter, W. G. (1961) The Lh,ing Brain. Penguin Books: Harmondsworth. Weiskrantz, L. 0986) Blindsight. Oxford University Press: Oxford. Weiskrantz, L. (1990) Outlooks for blindsight: Explicit methodologies for implicit processes. Proc. R. Soc. B 239, 247-278. Weizenbaum, J. (1976) Computer Power and Human Reason. W. H. Freeman: San Francisco. Zeki, S. (1992) The visual image in mind and brain. Sci. Am. 267(3), 42-50. Zeki, S. (1993) A Vision of the Brain. Basil Blackwell: Oxford. Zola-Morgan, S. and Squire, L. R. (1993) Neuroanatomy of memory. Ann. Rev. Neurosci. 16, 547-563.