On the use of theories, models and cybernetical toys in brain research

201

BRAIN RESEARCH

Review Article

O N THE USE OF THEORIES, M O D E L S A N D C Y B E R N E T I C A L TOYS IN BRAIN RESEARCH*

V. BRAITENBERG Laboratorio di Cibernetica del C.N.R., Istituto di Fisica Teorica, Universitgt di Napoli, Naples (Italy)

This article is by no means intended to be a complete review of neuro-cybernetical literature but rather the rettexion of a neurologist's experience with conceptual models provided by others. The distinction of theories, models and toys is of course arbitrary but will serve to articulate this paper into two main parts and an appendix. In the neurological terminology, the phenomena which some recent theories and models depict go by the name of nervous integration. Not long ago, the mysterious 'Nissl Grau', a substance reputed to be beyond the possibilities of analysis with existing means, was often invoked as the locus of 'integration', a function which was consequently implicitly marred by the same mark of unanalysability. It is my impression that the terms 'molecular layer', 'molecular substance', etc. for various anatomical formations, combined with the recent discovery of an enormous amount of complexity of biological matter at the molecular level, have unfortunate conceptual consequences even today. The models that will be mentioned here are for the most part neuronal models, explanations in terms of the complexity of nerve nets built out of relatively simple neurons, models which strengthen our belief that not only is there no room for Nissl's gray, but that there is no need for it, since before we have to resort to structure at the molecular level, we have at our disposal a large amount of complexity at a semi-macroscopical level, namely that of the texture of nerve nets out of neuronal elements. THEORIES There are two theories which have gained outstanding importance in quite different ways, the first as a conceptual basis for an enormous amount of experimental work, the second as one of the nuclei of crystallization of the new science which goes by the name of cybernetics. Theory o f the brain as an ensemble o f co-operative and competing organs. Although this theory has never been explicitly formulated, we have all occasionally talked about the brain in terms of a collection of organs (nuclei, grisea in the language of O. Vogt) * This review article was prepared in 1965 for the Third International Summer School of Brain Research on 'Structure and Function of the Cerebellum'. Brain Research, 6 (1967) 201-216

202

v. BRAITENBERG

which may suppress, release, control each other. They may compete for access to the motor output, communicate with each other or relay information from one to another. When we use this language, we are not concerned with the internal structure of each of these organs, but rather define their individuality according to their position in the scheme of their relations which represent the brain. It is characteristic for this view that the relations themselves are expressed in words befitting the social relations between organisms, in particular between men, and the individual organ is attributed functions (decision, control, memory) which in other contexts are concepts to be explained by the theories of the brain themselves. Although progress in neurology has been largely identified with progress beyond this organological scheme, in the sense that explanations are sought at a more elementary level, there are some arguments in favour of this kind of theory. First of all, experimental evidence is plentiful at this level, since destruction or global stimulation of such grisea have been for some time standard tools of experimental neurology and yield effects which are often easily describable in ordinary language. Secondly, there is no doubt that brains (not only of vertebrates) rely on a principle of specialization of function in a number of suborgans which are readily identified with the anatomical subdivisions (ganglia, nuclei, grisea, cortices etc.), each characterized by a relatively uniform texture, markedly different from one to the other, and often characterized also as organ-units by the mapping, within one such unit, of a co-ordinate system corresponding to that of an entire sensory or motor field. Finally, the use of behavioural terms for the functions and relations of the unit organs is reasonable, not only as a practical mnemotechnical device for the quick construction of hypotheses, but also on the grounds that the complexity of such units (or at least the order of magnitude of the number of cells in them) is closer to that of the whole brain than to that of the unitary meshes of their texture. Thus, if the unitary mesh may be explained in terms of mechanistic concepts, for the function of the entire brain the language of psychology is still appropriate, and it may serve also at the level of the grisea. This level of analysis has its constructive counterpart in the use ofbtock diagrams in engineering in situations in which, just as in the brain, the level intermediate between the functional units and the whole has sufficient structure to present problems of its own; although, of course, blocks labelled 'control' or 'decision' etc. in a machine diagram are not meant in the anthropomorphic sense which is tolerable in neurology but must yield to precise functional descriptions in the machine. MacKay's analysis (1962a) of constancy phenomena in the perception of space illustrates the application of the engineer's language to a neurological puzzle, and is particularly interesting in comparison with the analysis of the same phenomena by the biologist E. yon Hoist (to be discussed later), both concerned with block schemes rather than with neuronal wiring diagrams. Theory o f the brain in terms o f all-or-none neurons on a discrete time scale. McCulloch and Pitts's theory (1943) of nerve nets occupies a singular position in that the wide acceptance it has gained outside neurology gives it the appearance, to many, of something like a standard theory of the brain, while on the other hand 1 am not able to quote an example of its exposition in a textbook of neurophysiology, or even an Brain Research, 6 (1967) 201-216

THEORIES, MODELS AND CYBERNETICAL TOYS

203

experimental physiological study based on it. The booklet by Braines et al. (1964), written in the spirit of physiology and using McCulloch and Pitts's language extensively, is the exception which confirms the rule, since it is disconnected from the bulk of electrophysiology and neuroanatomy and therefore rather proves the point. The conceptual basis is well known. The neuron doctrine, corroborated 19ya large collection of anatomical and experimental facts and strongly supported by Cajal even if the ultimate morphological proof of its correctness had to wait for recent electron microscopy, rejects the notion of continuous nerve nets and emphasizes (implicitly) that transmission through the nodes of the net (i.e. in most instances, in vertebrates, through neural cell bodies) must obey radically different rules from the transmission through fibres. Hence, the ensemble of neurons is essentially an ensemble of discrete but connected elements. The early discovery of spikes adds to this picture the discreteness of the successions of signals that are transmitted. The types of interaction of two (or more) signals converging at a node are suggested by macrosc3pical observations, mainly in spinal physiology: (a) The observation of facilitation at the macroscopical level, i.e. the fact that a reflex response produced by stimulation of afferent bundles of fibres may sometimes be more massive than the sum of the responses obtained by stimulation of each bundle individually, implies that there are neurons which respond only if two (or more) afferent fibres are activated together and do not respond at all to the individual afferent lines. (b) The observation, in some other experimental situations, of a combined reflex response less massive than the sum of the individual responses (occlusion) may be interpreted on the basis of neurons which can be activated by either afferent line and which in the response to both stimuli imparted together appear only once: hence the response is less massive than the sum of individual responses. (c) Finally, the response to two (roughly) contemporaneous stimuli may be less than that to one of the two stimuli by itself: this is inhibition and implies that some neurons will be active only if some of their afferent fibres are not activated. These concepts were firmly established and diagrammatically described in some well-known diagrams by Sherrington as early as 1929. Add to these a supposition cautiously stated in the introduction to McCulloch and Pitts's p a p e r - - t h a t the delays between the firing of a set of neurons and the firing of another set excited by the first are roughly the same for all neurons, because the difference in thelength of the axons leading from one neuron to the next may be compensated by different conduction velocities (long fibres tend to be thicker and therefore faster conducting) - - and we have assembled the neurophysiological evidence which made this theory plausible in 1943. We shall see later that its weakest point, and I presume, the reason for which it has not entered the textbooks of physiology, is precisely the latter supposition. Meantime 1 shall try to state the reasons for which, on the contrary, McCulloch and Pitts's theory has encountered such unusual favour among mathematically minded people. Symbolic logic and nerve nets. I f sensory and other neurons either clearly emit a spike or do not (fire or d o n ' t fire) at a certain time, their state can be described by a simple proposition 'neuron N fires at time t' which is either true or false, without any intermediate readings being possible. Symbolic logic deals with such propositions, Brain Research, 6 (1967) 201-216

204

V. BRAITENBERG

either clearly true or false, representable by variables which assume only two values, truth or falsehood. All that happens within a nerve net (brain) in a lifetime should be describable by assigning values to a finite set of such binary variables, each referring to a certain neuron at a certain time, and should therefore fall within the scope of symbolic logic. Here the assumption that all delays are equal becomes important, for if they are not, we are n0t justified in describing the states of the nerve net only for a succession of discrete instants of time; as a matter of fact we have to consider, in general, continuous time, and the mathematical model becomes more complicated. For argument's sake let us accept quantized time. The upshot is essentially the following. The situation in which two neurons A and B must be active together in order to activate a third neuron C (a situation which we have seen to occur in so-called facilitation) corresponds exactly to the mode of composition of two propositions a and b into a third proposition c such that c is true only i f a and b are both true: conjunction in symbolic logic (let a be 'A fires', b 'B fires' and c 'C fires'). I f a neuron C fires when either of its afferents A and B are activated, the situation, known to occur in some neurons in the case of occlusion, corresponds to disjunction in symbolic logic: c is true when a is true and/or b is true. Finally, the inhibition of one (spontaneously active) neuron A by another B corresponds to negation in symbolic logic: a is true when b is not true. Now, since, using conjunction, disjunction and negation, we can construct propositions in symbolic logic which are true when particular sets of some other (elementary) propositions are true, and false otherwise, we can do something analogous for neurons, constructing neural nets with junctions of the three kinds described, culminating in a neuron that will fire only if particular sets of other neurons of the net have fired. Hence, at this level of the elementary propositional calculus, we may use propositions or neurons, with the appropriate connexions, interchangeably. Representation of events. This implies that any constellation of active neurons in the sense organs, and even any set of such constellations, may in principle correspond to a particular neuron of the brain which fires only when the particular constellation (or any of the set of constellations) occurs. In other words, calling such a set of constellations an event (e.g. the set of all constellations of active neurons in the retina which lie on the sides of a triangle correspond to the event'triangle'), we may say that events are represented by neurons of the brain. Just how literally this is to be taken is debatable. It is very likely that events of the kind that would ordinarily deserve this name in common language, such as a car approaching me or somebody whistling a tune ! know, are nowhere in the brain compressed into a single line, but are always represented by a collection of neurons rather than by individual elements, down to the set of motoneurons which I activate in response to the event. But in the visual system of the frog Lettvin et al. (1959) and GriJsser-Cornehls et al. (1963) have shown conclusively that local constellations of sensory neurons, so to say the building blocks of images, are actually represented by single fibres, and for some other vertebrates and invertebrates the situation seems to be similar. Be it as it may, there are some theoretical questions connected with the idea of representation of events. What kind of an event is represented by a neuron equipped with a recurrent axon through which, once activated by other afferents, it will re-

Brain Research, 6 (1967) 201-216

THEORIES, MODELS AND CYBERNETICAL TOYS

205

excite and reactivate itself indefinitely? If we agree that initially it was inactive, the proposition corresponding to the firing of such a neuron is of the kind 'at some time in the past the constellation so and so (in the afferents to that neuron) has occurred' and the event represented by it is therefore no more, as in the case of a neuron without feedback, an event referred to the immediate past, but one whose time of occurrence is not specified at all and which, if the brain were infinitely old, could have happened infinitely long ago. This raises the question whether there are any bounds at all to the kinds of events which a neuron in a nerve net may represent. Indeed such bounds do exist, since the type of events representable in McCulloch and Pitts nerve nets (or better, in any net of elements which behave like their formal abstraction of neurons) corresponds exactly to the class of events defined as 'regular events' by Kleene (1956). Essentially, if I have two nerve nets which represent two different events, I can easily (a) construct another nerve net which will represent the disjunction of the two events, and I can construct (b) one that represents the events corresponding to the first event following the second and even (c) one event following the other one an indefinite number of times. But no other ways of increasing the power of the representation are possible. To make this more concrete, let us find examples in language. Let us take words as elementary events. Then, when I hear a certain word (the event occurs in my input) and I say I understand it, something has happened in my brain which is analogous to a 'representation of the event' (and analogous probably to 'a concept evoked by the word' in philosophical terminology). The same concept may be evoked by different words, e.g. 'King crab', 'horse-shoe crab', 'Limulus','Pfeilschwanzkrabbe', hence in understanding any of these words, my brain is capable of a function which is analogous to the composition of events by disjunction of type (a) above. Also, words may be compounded by concatenation (composition of type (b) above) into events which I understand as a whole, 'horse-shoe crab' being again a good example. Finally, linguistic events of indefinite length may be produced by concatenation (type (c) above), such as in relative sentences, strings of adjectives, etc. This ready availability of examples from language, coupled with the suspicion that, apart from learning processes, the conceptual digestion of language may be in principle fully described by the scheme of Kleene's regular events, raises an important point. No matter how crude a model, neurophysiologically speaking, these formal nerve nets with their discrete time and no-dead-time neurons may be, their mathematical analysis produces results which are again interpretable in terms of macroscopic observations from psychology. Essentially, we have learned from the theorems on regular events how a mechanism which has a finite grasp of facts both in space and in time, i.e. a mechanism which at any moment is informed only on a finite region of space-time, may indefinitely extend its own reach (in time) by the use of the operation of concatenation (which in other contexts we may call 'conditioning of one state by a preceding one') and iteration (which corresponds to reverberation of states in the mechanism). And this, I think, is valid for many non-linear mechanisms under the appropriate conditions, not just for McCulloch and Pitts neurons. Other discrete nerve net models. Network of 'neurons' essentially similar to the preceding ones, i.e. composed of elements which add their inputs, weigh them against Brain Research, 6 (1967) 201-216

206

v. BRAITENBERG

a threshold and eventually transmit pulses to other elements, have been considered by various authors from different points of view. Connexions changing with function are in the foreground in some approaches (e.g. Uttley, 1956) while global statistical predictions dominate others (e.g. Ashby et al., 1962). The field is continuous with that of automata theory where similar objects are considered without reference to biology. The theoretical physicist's approach is exemplified by Caianiello's paper (1961) which contributes a discussion of the problem how the treatment of the short range behaviour of such nets may be coupled with that of the long range behaviour characterized by memory changes, i.e. learning. It appears that in joining these two levels cycles of neurons again become important. Thus neuronal reverberations. first postulated by various authors on purely speculative or anatomical (Lorente de N6, 1943) grounds, acquire a crucial role in the logical theory of nerve nets (McCulloch and Pitts) as well as in any attempt to embellish the logic by the assumption of structures themselves changing as a function of their activity (Caianiello). Theories borrowed;from physics. The description of neuronal activity in terms of propositions of symbolic logic or any equivalent system based on binary variables does not permit the ready application of the mathematical apparatus of physics for the transition from the micro- to the macroscopical. Confronted with this situation, many a physicist feels uneasy and will be inclined to apply to nerve nets the type of mathematical model which is familiar to him from similar situations in physics. Two examples come to mind. Cragg and Temperley (Temperley, 1965) point out the similarity between the binary (all or none) behaviour of neurons and the orientation of elementary magnets in a magnetic body, which occupy a finite number of orientations in space, in the simplest case in two directions only, parallel and opposed to an applied magnetic field. The analogy with nerve nets is further supported by the interactions of these elements on the one hand, which are comparable to synaptic relations between neurons since they provide for a dependence of the probabilities of orientation of the elementary magnets from the orientation of neighbouring magnets, and on the other hand by the effect of thermal agitation, which is comparable to the effect of random noise (or random connexions) in nerve nets. Organization of elementary magnets in spontaneously magnetized domains occurs below a certain critical temperature when, with a low level of thermal agitation, the effect of the interactions becomes more pronounced. These domains have a certain degree of stability which provides an analogy with memory phenomena. An interesting comparison is possible between the critical effect of temperature in magnetic materials and the idea of threshold control in nerve nets (consensual changes of the thresholds in all neurons, imposed from outside), which corresponds to a global change in the strength of the interactions. Such a mechanism has often been postulated as indispensable for maintaining nerve nets in the critical region between 'all neurons active' and 'all neurons inactive'. The observation that discrete acts of the nervous system, such as certain standard patterns of instinctive behaviour or certain motor p a t t e r n s - - t h e various gaits of a horse etc. - - may be interpretable as the discrete normal modes of the brain viewed as a linear system, constitutes the central point of Greene's theory (1962, quoted from

Brain Research, 6 (1967) 201-216

THEORIES~ MODELS AND CYBERNETICAL TOYS

207

Arbib, 1964). It is appealing to think that excitation of a resonant frequency of a nerve net will excite a normal mode and therefore produce a particular distribution of excitation in space and time, since this is equivalent to mapping the relevant patterns of activity on to a single (frequency-) co-ordinate, but neurophysiology to my knowledge has no direct support for such an idea. MODELS OF PARTICULAR FUNCTIONS

While the preceding items, called theories for the purpose of this review, could be discussed with very little or only very general reference to the experimental facts of neurophysiology, since their main interest was in general statements about the function of idealized 'nerve nets', I shall mention some examples of models in which, on the contrary, a particular set of experimental observations in a particular brain was the guiding principle. Lateral inhibition in the eye q[Limulus. Below the level of sensory receptors in the compound eye of Limulus (the horse-shoe crab), the fibres conducting impulses from the (about 1000) ommatidia ( = elements of the compound eye) toward the brain communicate through collaterals which, if the terminology of vertebrate physiology is applicable, provide rich axo-axonal synapses at this level. The influence exerted through these fibres is inhibitory, and obeys simple quantitative rules (Hartline and Ratliff, 1957). The spike frequency in one fibre, descending from an illuminated ommatidium, will decrease when a neighbouring ommatidium is also excited. This inhibitory effect has a threshold, in the sense that below a certain level of excitation of the inhibiting fibre no effect is detectable in the inhibited fibre. Beyond this threshold the inhibition is, within a wide range, simply a linear function of excitation. The coefficient of proportionality is, however, dependent on the distance between the two ommatidia, and moreover the threshold of inhibition is higher for more distant ommatidia. The firing frequencies of a set of fibres corresponding to illuminated ommatidia can be precisely determined once the constants are known and matrix algebra becomes applicable (Reichardt, 1961; Varjt], 1965). The successful mathematical analysis of this system is particularly interesting since it is based on a continuous treatment of spike frequencies, in other words, the discontinuity inherent in the spike mechanism is completely forgotten and linearity ensues at a higher level. Biologically, the system is interesting both as a mechanism for enhancement of contours, one of the most useful devices in perception (analogous to drawing as opposed to painting) and, possibly (Reichardt, 1961), as a correction of an optical imperfection, since the blurring of the image in the compound eye, due to overlap of the visual fields of ommatidia, could be precisely counteracted by an inverse transformation in the layer of inhibitory fibres if the function describing the optical blurring on one hand and the neurological sharpening on the other hand are matched, as they appear to be. Perception oJ movement in the eyes o/ insects. Again in the field of invertebrate vision, the analyses of Hassenstein, Reichardt and others resulted in a set of quantiBrain Research, 6 (1967) 201-216

208

v. BRAITENBERG

tative observations which could be compounded into a mathematical model having a high degree of plausibility not only because it predicts further observations (which have been verified), but also because of its intrinsic simplicity. Moreover, the model is expressed in terms of components which may be readily interpreted as neuronal mechanisms. These virtues give it an outstanding position among models of neurological functions. I shall try to summarize the model in simple intuitive terms Insects, like many other animals, tend to follow a moving pattern, the reaction being perhaps part of a feedback which operates as a stabilizer of the animal's position in its optical environment. The question now arises: how is movement perceived, and further, how is velocity measured. If the environment consisted in a periodic pattern with known period, the measurement of relative velocity could of course be readily done. But these optomotor reactions will occur for movements of any sort of regular or irregular patterns, including patterns which may be taken as approximations of purely casual distributions, i. e. with no statistical correlation between different portions of the pattern, which forces us to look for some mechanism more sophisticated than simply counting the number of regularly spaced objects which pass in front of a beetle's ommatidium. It appears that in Chlorophanus, the beetle studied by Hassenstein, the elementary movement perceiving device consists in just two ommatidia (facets) separated by no more than one ommatidium, i.e. couples of neighbours or neighbours but one. The first astonishing fact is that, while movement perceived by just one such elementary couple of ommatidia (all others are blinded) is sufficient to elicit a reaction, when the reaction is due to movement perceived in many elementary couples, e.g. an entire row across the beetle's eye, it is quantitatively precisely the sum of the reactions due to all individual couples of ommatidia involved. This is astonishing especially to anyone used to thinking in terms of McCulloch and Pitts nerve nets, for within that theory linear superposition of effects is the least plausible thing to happen. The other very striking observation is that, within one elementary movement perceiving couple of ommatidia a multiplicative law is valid, in the sense that the reaction is proportionate to the product of the change of light in one ommatidium at some time and the change of light in the other ommatidium after a fixed delay. Thus dimming of one ommatidium followed by dimming of the other one produces a reaction of the same sign as brightening of one followed by brightening of the other, but dimming of one followed by brightening of the other produces a reaction of opposite sign (i.e. a turning movement in the opposite direction). The magnitude of the reaction is further dependent on the delay between the changes induced in the two ommatidia; it decreases for longer delays, which is to say, some phenomenon traversing the beetle's visual field will induce a reaction which is (of course, up to a limit) the stronger the higher its velocity. These facts are sufficient to describe a model in a crude form. Multiplication of a function x (t) by the same function shifted in time, f ( t + ~), followed by an averaging operation is the essence of the autocorrelation function. It gives, as a function of r, a measure of correlation of values within x (t), so to say, a measure of its structure. Two neighbouring ommatidia, confronted with a moving environment, will actually

Brain Research, 6 (1967) 201-216

THEORIES,

MODELS AND CYBERNETICAL

209

TOYS

receive as input the same function (describing the light flux from successive positions in the environment), but delayed in one ommatidium with respect to the other by an amount depending inversely on the velocity of the movement. Now, if the outputs of the two ommatidia are multiplied, as the experiments suggest, and are averaged, one should expect to get something like the autocorrelation of the moving pattern for a which depends inversely on the velocity of the movement. A wiring scheme connecting couples of ommatidia with a multiplying unit and an averager (low pass filter) is easily made, and is neurologically not too implausible (Fig. 1a). It would readily work as a device for measuring velocity, since (for random

~

J I..

k.)

\J

O~'IMATIDIA "fZ~"

....... MULTIPLIER ....... ~'

fl-VJ",R AG F tt AND SUB I'IiA( TOI{

a

I

b

Fig. 1. To explain the Hassenstein-Reichardt model of the perception of movement in insects. For explanation see text. functions) the autocorrelation is a monotonically decreasing function of T, and therefore, in this system, a monotonically increasing function of the velocity of movements. It has, however, two drawbacks. First of all it gives no indication of the direction of movement. This can be corrected by making the system asymmetrical, e.g. by introducing a low pass filter into one of the lines connecting an ommatidium to the multiplier (Fig. lb). This will increase the correlation for movement in one direction but not in the other, since now the original function, derived from one ommatidium, will be compared to the same function passed through the filter which functions, so to say, as a short term memory. The second drawback, however, is not yet eliminated. Namely, the system will show correlation whether it is due to movement or to actual correlation within the pattern such as always exists, but in varying degree, and will not be able to distinguish between the two sources of correlation and hence not be able to furnish a measure of velocity independently from the characteristic of the moving patterns. This again is overcome (Fig. lc) by using two of the asymmetrical systems of Fig. lb in push-pull,Wi.e, arranging them in mirror symmetry and subtracting their output. The correlation pre-existing in the pattern will have no effect, since it will cancel out in the subtraction, while the correlation due to the low pass filters in the asymmetrical system will be detected and will produce positive output for movement in one direction, and negative output for movement in the other direction. This model (which, I am afraid, I had to state somewhat superficially in the above Brain Research, 6 (1967) 201-216

210

v. BRAtTENBERG

purely verbal account) would not have been so convincing had it not been for an experimental test which it suggested and which resulted in its favour. If the autocorrelation function is the essential trick involved, the mechanism should give the same output for different patterns having the same spectral components (frequency components of the function describing the successions of shades in the moving patterns) but different phase, and it did. Details have to be sought in the original papers by Hassenstein and Reichardt (1956), Reichardt (1957) and Reichardt and Varj~t (1959). Again, as with the Limulus eye, what is astonishing to vertebrate physiologists is the description in terms of continuous variables, a description which at first sight would not seem appropriate for neurological networks in which we suspect spike trains to function as the messengers between the units. Also, the linearity which is apparent in this system (apart from the multiplicative j unction) poses some difficulties if one thinks in terms of spikes, as we have already mentioned. Direct neurophysiological evidence about the operation of the ganglia involved in the perception of movement by the beetle is not yet available, and it may well be that at least in invertebrates not all transmission is spike-like. It may also be that spike-like behaviour in the microscopical order of magnitude will give rise to continuous and perhaps linear macroscopical relations in the whole system. It is this transition from the microscopical to the macroscopical which the mathematical theories of nerve nets have up to now failed to clarify. Other 'systems'. The latter study is an example of the procedure wihch is sometimes called 'system analysis'. This term implies that one aims at a definition of the topological scheme of a system of relations which govern a collection of postulated elementary units. The input-output functions of the units are mathematically well defined and are often of a simple kind, but no particular emphasis is laid on the identification of these elementary functional units with known neurophysiological mechanisms. A very general analysis of this kind, applicable in principle to perception in various sensory modalities, resulted in a scheme called 'Reafferenzprinzip' by the authors von Hoist and Mittelstaedt (1950). I shall sketch it here because, among other reasons, again the model is stated in terms which at first sight seem to be in contrast with the spike-like behaviour of neurons. The observations are of the following kind. A fly (Eristalis) in a rotating cylinder carrying a pattern of stripes is subject to a 'reflex' which results in turning movements in the same direction as the cylinder goes, as if to stabilize the insect with respect to its optical environment. This 'optomotor reaction' we have already mentioned in the previous paragraph in connexion with the problem of perception of movement. If this is a reflex in the strict mechanistic sense, it could be expected to prevent a fly from executing any turning movements in a stationary striated cylinder, since a spontaneously initiated movement would produce an optical input analogous to the one produced by moving the cylinder in the opposite direction around the stationary fly. Do we have to suppose then that the reflex mechanism is turned off during voluntary locomotion? It turns out that it is not, since if the head of the fly is rotated 180 degrees

Brain Research, 6 (1967) 201-216

21 1

THEORIES, MODELS AND CYBERNETICAL TOYS

and fixed to the thorax in an upside-down position, voluntary movements will elicit the turning reflex, but of course now in the direction of the voluntary movement itself, with all the catastrophic consequences of such positive feedback. The answer is: any reflex, such as the turning movement of the insect in the rotating cylinder, is a consequence of what the animal perceives (if I am allowed to use this term in a sense more general than in human psychology), and perception at some level of the central nervous system appears to be the difference between what the animal expects to perceive and what actually comes in. Thus, with voluntary movement and the consequent expectation of a relative shift of the visual environment, which actually occurs, nothing is perceived and no 'reflex' is elicited. Another example from human perception (von Holst, 1957): the visual environment appears to remain stable with voluntary movements of the eye (if I direct my gaze to the right, the room does n o t appear to jump to the left), but when the eyeball is moved passively, e.g. by applying pressure with my finger, the same shift of the image over the retina is perceived as a jump of the world. Explanation: when I look to the right voluntarily, I subtract from perception an expected apparent movement of the environment to the left. Experimental confirmation is provided in the case of paralysed eye muscles, when an intended gaze to one side, say to the right, leaves the eyes immobile in their sockets but results in the perception of an apparent movement of the environment to the same side. This is very convincingly interpreted as the result of the subtraction of

EXPECTED PERCEPTION

1

Fig. 2. The reafference principle by yon Hoist and Mittelstaedt. For explanation see text. Brain Research, 6 (1967)201-216

212

v. BRAITENBERG

the expected perception (apparent movement to the left) from 0 (no real shift of the image over the retina due to the paralysis): --left ~- right. The general scheme underlying the interpretation of these observations is depicted in Fig. 2. What reaches a higher centre C as perception, p, is the sensory inflow from a sense organ S minus the expected perception e which is derived from the, so to say, voluntary outflow v from C. M is the motor organ that will induce changes in the relation between the animal and the world W. This terminology is somewhat different from von Holst and Mittelstaedt's and so is the diagram, but essentially, I believe, I have stated their point. It is interesting to compare this work with MacKay's independent analysis of the same kinds of phenomena, which arrives at similar conclusions. The emphasis of the authors in stating the reafference principle is on the idea that the action proceeding from S to M is not a reflex in the usual sense of the word, since the path contains a loop through the box labelled 'expected perception' whose state is determined by the rest of the nervous system. What interests us in trying to characterize the Holst-Mittelstaedt model in the context of the present review is again the linearity which is apparent in many experiments which support the reafference principle. The box labelled 2; appears to be truly, within wide limits, an adding device for the inflow from all sensory modalities (S, S', S" in our scheme)which contribute relevant information, as has been well demonstrated, e.g. by von Holst in some ingenious experiments in which he was able to vary the coefficients of S, S' etc. and consequently to weigh the corresponding sensory inflows against each other. When a statocyst is extirpated in a fish, the other statocyst has to contain a statolith twice as heavy as before in order to put the animal back in its upright position. Even between different sensory modalities, e.g. vision and labyrinthine sense which concur in regulating the posture of fishes, such linear summation appears to be the rule. This poses the question of how the relevant quantities are contained (coded) in the nervous system. For the vestibular system one may invoke, quite plausibly, the very regular resting discharges modulated by changes in position, which have been recorded in vestibular fibres (Viernstein and Grossmann, 1961), even if summation of spike frequencies is not, in general, a linear affair. But elsewhere, e.g. in trying to postulate the neurophysiological basis for the observations on the stability of the visual coordinates with voluntary eye movements, we are quite in the dark. Models derived.from histology. A procedure opposite in direction to the preceding ones, where the analysis of complex functions in terms of postulated elementary units had resulted in a 'wiring scheme', may be applicable in some cases where, on the contrary, the wiring scheme is provided by histology and the function'of the elementary boxes (various types of synaptic junctions) may be supposed to be known on the basis of physiological experiments. The function of the whole structure can then be directly read off the histological picture. It is debatable whether we wilt ever have at our disposal an efficient morpho-physiological dictionary which would enable us to translate all histological details in functional terms and so to carry through this type of analysis systematically, but an essay in this direction, on the cerebellar cortex, has been published (Braitenberg, 1967). Brain Research, 6 (1967) 201-216

THEORIES, MODELS AND CYBERNETICAL TOYS

213

Models oJ single neurons. If sufficiently realistic properties are taken into the formal definition of individual neurons, and if we were to build neuron models accordingly, the situation would become almost impenetrable if even only a few of such elements were coupled together to form a network. In order to overcome this difficulty, one may (a) search for simpler, mathematically more manageable elements hoping that they may still embody all the relevant properties, (b) investigate the properties of very small nets of various kinds of 'neurons', hoping to be able to generalize to larger numbers later on, or (c) build in hardware extremely cheap analogue neurons, so that the properties of connected ensembles of very many of them become amenable to experimental study. A number of studies along these and related lines has been published lately. I shall not attempt to compete with the excellent essay on neural modelling by Harmon (1964), but will mention some points in this field which are relevant to the themes of the preceding discussion. Interpretation oJ spikes sequences as messages. Electrophysiological records from individual neurons generally appear in the form of seemingly irregular sequences of spikes, both in conditions of spontaneous activity ( = stimulation not controlled) and as a consequence of some kind of external stimulation. Just how irregular these sequences are is still a matter of debate, and various theoretical models of the role of the individual neurons in the net have ensued. (a) Sometimes the spike sequences are such that, except for the absence of very short spike intervals, implying a dead time in the spike generating process, the statistics of the spike intervals are compatible with a Poisson model, which is to say, with the supposition of complete independence between the various occurrences of spikes in the sequence. Such a spike generating process would be completely defined by the average frequency and cannot, consequently, carry any other message but average intensity, with a precision, moreover, which is very low for short sequences. To my knowledge, this Poisson model is not seriously upheld by anybody any more, not only because of the poor information capacity of such neurons, but also because the resemblance of the statistics of spike sequences to that of a Poisson process has sometimes been found to be only superficial, because of evidence of correlation within the spike sequence (e.g. Braitenberg et al., 1965). (b) Frequency modulation or pulse interval modulation is essentially the role assigned to the neuron according to some other interpretations. In sensory systems, a dependence of the firing frequency in primary, secondary, tertiary and even higher order sensory neurons on the intensity of the stimulation is well known, and it is quite plausible that neurons in general do nothing but translate the level of afferent excitation, averaged over all incoming fibres and over some stretch of time, into the language of spike sequences. Even if neurophysiologically speaking this is a rather acceptable description of neurons, as regards information handling, these averaging neurons are curious devices: most of the large amount of information residing in the spatiotemporal pattern of afferent excitation at the level of the dendritic tree is lost, and a single quantity reaches the axon, where it is coded into spike sequences. The coding process itself has been represented in various ways in different models. Here are two examples. Brain Research, 6 (1967) 201-216

214

v. BRAt]'ENBERG

If the membrane potential, starting from a resting level, is pushed up and down by a random synaptic barrage of inhibitory and excitatory pulses, a certain fixed threshold which determines the firing of the neuron will be reached after times which depend on the preponderance of excitatory over inhibitory afferents and which should be distributed statistically according to a well-known distribution function (the stable distribution of order 1/2) derived from the theory of random walks. This model by Gerstein and Mandelbrot (1964) is supported by a good fit of the observed distribution of spike intervals in some neurons with said distribution function. This hypothesis requires a mechanism of immediate reset of the membrane potential to its resting level after each spike, which is not very plausible physiologically. On the contrary, the recovery process of the neuron itself may be invoked to explain frequency coding. If the threshold after a spike becomes very high and then decreases monotonically to its resting level, for a high level of excitation the neuron will fire again sooner than for lower levels. If this is the mechanism, and if the statistical distribution of levels of excitation in the input is supposed to be known (one may assume a Gaussian distribution), the recovery curve of the neuron can be computed from the distribution of spike intervals in the output (Braitenberg, 1965). (c) From the standpoint of economy of information transmission, the neuron would do best to code its input into spike intervals (MacKay and McCulloch, 1952) if other neurons are available precisely to detect coincidences and delayed coincidences of afferent spikes. This idea has been elaborated upon by MacKay (1962b), and also by Reiss (1964). If this principle were ever experimentally confirmed to be effective in brains, entirely new perspectives would be opened to neurophysiology. In fact, as MacKay points out, we may then search for memory traces, no longer solely by looking for variations in the strength of the connexions of neurons but also (or mainly) in the delay involved, since changing the delays slightly would be equivalent to radical changes in the patterns of connexions, if precise coincidence of pulses plays an important role in neurons. Also, if certain neuronal networks (involving delay lines) produce specific patterns of spike sequences, and others (involving delay lines and a coincidence detector) resonate specifically to these patterns, much of the specificity of the connexions within the brain could be ascribed to this mechanism rather than to a detailed 'wiring', since very diffuse, 'broadcasting' connexions would be sufficient for this scheme. Addition and multiplication of spike sequences. Many neuron models have been built or programmed on computers. Sometimes, no sooner were the well-known physiological properties incorporated into the electronic analogue than the model taught us something which we could easily have deduced from the physiology but which in the hardware sprang to the eye with much greater didactic force. This was so, for instance, with the neuron model of Kiipfmiiller and Jenik (1961) which, for two excitatory input lines, gives in the output, depending on the threshold, a frequency roughly proportional to the sum or to the product of the two input frequencies. This is obvious, since for a threshold low enough to let one input pulse produce one output pulse. all the input pulses on the two lines appear in the output (except for some loss due to dead time), while for a higher threshold, requiring coincidence of pulses on the two

Brain Research, 6 (1967) 201-216

THEORIES, MODELS AND CYBERNETICAL TOYS

215

lines to set off the output, the number of output pulses is the number o f coincidences which can be c o m p u t e d as the joint probability from the product of the two input frequencies. We have seen previously that both additive and multiplicative junctions are postulated in some of the models of complex brain functions. CYBERNETICAL TOYS

The Viennese Wolfgang von Kempelen, 18th century cyberneticist, ominously foreshadows a twofold attitude which is discernible a m o n g present day model builders. On the one hand he produced an ingenious and most respectable speaking machine, a solid piece of work which incorporated much of the a n a t o m y of the voice organs and seems to have functioned quite well, but on the other hand he was accused o f outright fakery when he exhibited a box which played quite good 'automatic chess' (and apparently contained a dwarf). We, too, not content with the use of models as analytical devices which are unavoidable where the theory fails, have occasionally constructed m o c k animals or brains or eyes etc., simply for our own or other people's amusement and without any reference to the real questions. The reference may, however, occasionally be implicit or involuntary, since such a toy, based on a simple trick which we repute to be far below the complexity of the true function which it is supposed to mock, may show so m a n y unsuspected facets in its behaviour when we play with it that we may be tempted to revise our original assumption of the superior complexity of the biological mechanism. This is a useful experience which we make, e.g. while playing with one of those mobile objects which follow us when we whistle etc. ACKNOWLEDGEMENT

The research reported in this d o c u m e n t has been sponsored by the 6570th Medical Research Laboratories under G r a n t A F E O A R 65-44 through the E u r o p e a n Office o f Aerospace Research ( O A R ) United States Air Force. REFERENCES ARBIB, M. A., (1964); Brains, Machines, and Mathematics. New York, McGraw-Hill Book Company. AsHav, W. R., YON FOERSTER,H., AND WALKER,C. C., (1962); Threshold and stability in large networks. Techn. Rep. No. 3, Elec. Eng. Res. Lab., University of Illinois. BRAINES,S. N., NAPLAKOW,A. W., ANDSWETSCHINSKI,W. B., (1964); Neurokybernetik. VEB Verlag Volk und Gesundheit, Berlin. BRAITENBERG,V., (1965); What can be learned from spike interval histograms about synaptic mechanisms. J. theor. Biol., 8, 419-425. BRAITENBERG, V., (1967); Is the cerebellar cortex a biological clock in the millisecond range? The Cerebellum. Progress in Brain Research, Volume 25. C. A. Fox and R. S. Snider, Editors. Elsevier, Amsterdam, pp. 334-346. BRAITENBERG,V., GAMBARDELLA,G., GH1GO, G., AND VOTA, U., (1965); Observations on spike sequences from spontaneously active Purkinje cells in the frog. Kybernetik, 2, 197-205. CA1ANIELLO, E. R., (1961); Outline of a theory of thought-processes and thinking machines. J. theor. Biol., 2, 204 235. GERSTEIN, G. L., AND MANDELBROT,B., (1964); Random walk models for the spike activity of a single neuron. Biophys. J., 4, 41. Brain Research, 6 (1967) 201-216

216

v. BRAITENBER~.

GREENE, P., (1962); On looking for neural networks and 'cell assemblies' that underlie behaviour. Bull. Math. Biophys., 24, 247-275, 395-411. GRi3SSER-CORNEHLS,U., GR0SSER,O. J., ANt) BULLOCK,TH. H., (1963); Unit responses in the frog's tectum to moving and nonmoving visual stimuli. Science, 141,820-822. HARMON, L. D., (1962); Problems in neural modeling. Neural Theory and Modeling, R. F. Reiss, Editor. Stanford University Press, pp. 9-42. HARTLINE, H. K., AND RATLIFF, F., (1957); Inhibitory interaction of receptor units in the eye of Limulus. J. gen. Physiol., 40, 357-376. HASSENSTEIN,B., UND REICttARt)T,W., (1956); Systemtheoretische Analyse der Zeit-Reihenfolgenund Vorzeichenauswertung bei der Bewegungsperzeption des Riisselkafers Chlorophanus. Z. Naturforsch., llb, 513. HOLST, E. VON, (1957) ; Aktive Leistungen der menschlichen Gesichtswahrnehmung. Studium Gener., 10, 231-243. HOLST, E. VON,UND MITTELSTAEDT,H., (1950); Das Reafferenzprinzip (Wechselwirkungen zwischen Zentralnervensystemund Peripherie). Naturwissenschaften, 37, 464--475. KEMPELEN, W. YON, (1791); Mechanismus der menschlichen Sprache, nebst der Beschreibung einer sprechenden Maschine. I. yon Degen, Wien. KLEENE, S. C., (1956); Representation of events in nerve nets and finite automata. Automata Studies. C. E. Shannon and J. McCarthy, Editors. Princeton University Press, pp. 3-41. Ki3PFMi3LLER, K., UND JENIK, F., (1961); Ober die Nachrichtenverarbeitung in der Nervenzelle. Kybernetik, 1, 1-6. LETTVtN, J. Y., MATURANA,H. R., McCuLLOCH, W. S., AND P1TTS,W. H., (1959); What the frog's eye tells the frog's brain. Proc. Inst. Radio Engrs., 47, 1940. MACKAY,D. M., (1962a); Theoretical models of space perception. Aspects of the Theory of Artificial Intelligence. C. A. Muses, Editor. Plenum Press, New York, pp. 83-103. MACKAY,D. M., (1962b); Self-organization in the time domain. Self-organizing Systems, M. C. Yovits G.T. Jacobi and G.D. Goldstein, Editors. Spartan Books, Washington 12, D.C., pp. 37-38, MACKAY,D. M., ANt) MCCULLOCH,W. S., (1952); The limiting information capacity of a neuronal link. Bull. Math. Biophys., 14, 127-135. McCULLOCH, W. S., AND PrrTs, W., (1943); A logical calculus of the ideas immanent in nervous activity. Bull. Math. Biophys., 5, 115-133. RErCHARt)T, W., (1957); Autokorrelationsauswertung als Funktionsprinzip des Zentralnervensystems. Z. Naturforsch., 12b, 447. REICHARDT,W., (1961); tOber das optische AufliSsungsvermogen von 'Limulus'. Kybernetik, 1, 59-69. REICHARDT, W., UNO VARJU, D., (1959); 1]bertragungseigenschaften im Auswertesystem ftir das Bewegungssehen. Z. Naturforsch., 14b, 447. REiss, R. F., (1964); A theory of resonant networks. Neural Theory and Modeling, R. F. Reiss, Editor. Stanford University Press, pp. 105-137. TEMPERLEY,H. N. V., (1965); Analogies between the engram and the magnetic domain: Cybernetics of Neural Processes. E. R. Caianiello, Editor. Consiglio Nazionale delle Ricerche, Rome, pp. 273-278. UTTLEY, A. M., 0956); Conditional probability machines and conditioned reflexes. Automata Studies. C. E. Shannon and J. McCarthy, Editors. Princeton University Press, pp. 253-275. VARJ6, D., (1965); On the theory of lateral inhibition. Cybernetics of Neural Processes, E. R. Caianiello, Editor. Consigiio Nazionale delle Ricerche, Rome. VERSTEIN,L. J., ANDGROSSMANN,R. G., (1961); Neural discharge patterns and simulation of synaptic operations in transmission of sensory information. Information Theory, C, Cherry, Editor. Butterworth, London, pp. 252-260.

Brain Research, 6 0967) 201-216