- Email: [email protected]
What can automaton theory tell us about the brain?
Physica D 45 (1990) 205-207 North-Holland
BIOLOGY
WHAT
CAN
AUTOMATON
THEORY
TELL US ABOUT
THE BRAIN?
J o n a t h a n D. V I C T O R Department of Neurology, Cornell University Medical College, 1300 York Avenue, New York, N Y 10021, USA and Laboratory of Biophysics, The Rockefeller University, 1230 York Avenue, New York, N Y 10021, USA Received 13 September 1989 Revised manuscript received 11 January 1990
The potential contribution of the theory of cellular a u t o m a t a to understanding of the brain is considered.
1. I n t r o d u c t i o n How the brain works is one of the fundamental problems in biology. Despite dramatic advances in understanding at the molecular and cellular level, a n u m b e r of basic issues remain. These include, for example, the mechanism for the determination of neural connectivity by genetic and epigenetic factors, and the basis for the difference between the intellectual capacity of the h u m a n brain and that of other species. Satisfactory answers require integration of understanding at multiple levels of structure into a coherent whole. Thus, a theoretical framework, as well as experiment, is required. But what can one reasonably expect from theory, and what is its relationship to experimental investigations? W i t h these questions in mind, I would like to outline areas of inquiry in which I believe that theoretical insights will make significant contributions towards understanding the brain. The best place to begin is to consider the landm a r k contribution of yon N e u m a n n [1]. Von Neum a n n constructed a cellular a u t o m a t o n capable of universal c o m p u t a t i o n and self-reproduction. This construction d e m o n s t r a t e d t h a t a small set of local rules acting on a large repetitive array can result in a structure with very complex behavior. The von N e u m a n n construction thus immediately suggests how an organ with behavior as complex as the brain's can be specified from limited genetic information.
T h a t each unit of the von N e u m a n n a u t o m a ton had twenty-nine states and four neighbors is inessential to its biologic import; what is important is that the construction could be done at all. The property of universal c o m p u t a t i o n is only indirectly important: we do not need to think of the brain as a universal ~hring machine in order to a p p l y the cellular a u t o m a t o n metaphor; universal c o m p u t a t i o n is simply a rigorous way to guarantee that the von N e u m a n n construction has behavior that, all would agree, is complex. The p r o p e r t y of self-reproduction is relevant in a similar fashion, in that it is a rigorous way to guarantee a rich behavioral repertoire. The observation that the cerebral cortex is composed of a large number of local neural assemblies that are iterated throughout its extent, by itself, is not an existence proof that complex behavior m a y result from a network of simple elements. Von N e u m a n n ' s construction is necessary to show that such a structure is indeed capable of complex behavior, without the need to invoke region-to-region variability, long-range interactions, stochastic components, or mysticism. W h e t h e r the yon N e u m a n n m e t a p h o r is qualitatively correct is open to question. However, even if we agree t h a t it does capture an essential feature of brain organization, there is more to be done than to turn the m e t a p h o r into an allegory: several challenging areas of theoretical inquiry remain.
0167-2789/90/$ 03.50 (~) 1990 - Elsevier Science Publishers B.V. (North-Holland)
206
2.
J.D. Victor
The problem
/ Automaton
of robustness
One major qualitative difference between the behavior of the von Neumann construction and that of the brain is that of robustness, or stability to perturbation. Alteration of the state of a single unit of the von Neumann machine typically leads to catastrophic failure; malfunction of a single neuron or neural assembly should have no measurable effect. To some extent, the instability of the cellular automaton model may be a consequence of the discretization of states - but how many states are needed to provide for robustness? Closely related is the problem of robustness of behavior on an imperfect lattice. The cellular automaton metaphor becomes very inattractive if it would require an ezactly periodic brain. Presumably, neural connections are formed on the basis of a regular overall plan, but with variability at the level of the individual neuron. If the four neighbors of a von Neumann unit were selected at chance from a somewhat larger local neighborhood, the construction would fail catastrophically. Again, this failure is probably a consequence of the small number of neighbors in the von Neumann machine - but how many neighbors are necessary? Yet another kind of robustness is that of relative insensitivity to global alterations of the transition rule itself. In this regard, the brain does not appear to be robust. Rather, special homeostatic mechanisms such as autoregulation of cerebral blood flow act to minimize changes in the metabolic milieu that might induce global changes in neural function. When the homeostatic mechanisms fail in even apparently subtle ways, the result is a gross disturbance of consciousness. It is plausible that the cost of ensuring some measure of insensitivity to mild changes in the transition rule itself is much higher than that of ensuring insensitivity to state change or connectivity - but can this notion be made precise? Introduction of more states and more neighbors leads to other questions. What is the tradeoff between number of states and number of neighbors in providing a given level of reliability? A manyneighbor (100 or more) many-state (20 or more) automaton is biologically implausible unless there is some regularity to the state space and the transition rule. An “irregular” rule is also very likely to
theory and brain
be unstable to perturbation. For a plausible rule, it should be possible to view the states as discrete points in a continuous state space, and the transition rule should be at least piecewise continuous on state space. But what can be said about the topology and dimension of that space, and how complex must the transition rule be? These abstract questions can be made quite concrete if, say, we consider a von Neumann unit to be a neuron. Does a single variable (such as a transmembrane voltage) suffice to specify a neuron’s state, or must additional variables (perhaps concentration of calcium or a trophic factor) be considered? Such factors are crucial for development and synaptic plasticity. But are they required for the moment-by-moment informationprocessing as well? Questions about the transition rule become questions about the informationprocessing capacity of a single neuron. Does it suffice to lump all the neighbors’ inputs into a few pools which combine additively (e.g. an excitatory pool and an inhibitory pool), or is it necessary to hypothesize more complex interactions among a neuron’s inputs? Such interactions, such as presynaptic inhibition, are well known - but are they a requirement, an efficiency, or an inessential byproduct of evolution from a more primitive nervous system?
3. The problem
of scaling
To understand the behavior of a model neural network, it often appears necessary to create an explicit computer model. Indeed, if the behavior of the model could be predicted in a simple fashion from its axioms, the model would likely be criticized as not possessing the “emergent” properties that are the essence of an acceptable model for cortex. But even the most ambitious explicit computer simulations are dwarfed by the number of elements in a real cortex. The scale of a computer model (lo4 to lo5 elements) is about halfway between that of just a single unit and that of a real brain (lo8 to 10’ elements). We need to know how characteristic lengths and times scale as the number of network elements increase from that of a computer model to that of a real brain. A network whose settling time increases even linearly
J.D. Victor
/ Automaton
with network size will have very different behavior when scaled up by four or five orders of magnitude. More generally, we need to be able to understand what happens to global dynamics (number, dimension, and stability of attractors, for example), and whether additional qualitative properties of the model will “emerge” at a more realistic scale. A thorough understanding of scaling behavior may permit a clearer answer to the question of whether the qualitatively distinct features of human brain function may simply be viewed as consequences of its size, or rather, whether other processes (such as the development of new, specialized brain regions) must be invoked.
4. The problem
of model
testing
Let us now assume that we have a model in hand, with acceptable robustness, a biologically reasonable transition rule, and scaling behavior within grasp. How, and to what extent, can the model be tested? Model testing requires two levels of investigation. (i) Does the overall behavior of the model correspond to that of the brain? (ii) Is there a detailed correspondence between parts of the model and parts of the brain? These questions seem inextricably related: it only makes sense to inquire about a detailed correspondence if overall behavior is acceptable, but how do we ask about overall behavior if we do not know what are the model counterparts of biologic observables? Theory can help in two ways. Firstly, theory may be able to identify certain kinds of observables that are relatively insensitive to hypotheses about detailed correspondence. Possibilities for such observables might include dimensionality and stability of limit trajectories, or, how mutual information at two points in the model scales with separation or time lag. Such ideas might suggest new ways to interpret anatomic, single-cell, and gross-potential data.
theory
and brain
207
Secondly, we would like to know to what extent two models which have different internal descriptions may have similar overall behavior. This is perhaps the most important contribution that theory can make. A relatively minor benefit is that competing models that can only be distinguished on the basis of detailed biologic correspondence will be recognized as such. The major benefit is that such an understanding will identify the critical features of internal structure which do affect overall behavior - and thus, define the critical experimental questions.
5.
Conclusion
I hope that the questions raised above will serve as a focus for theoretical efforts, and as a framework for the design and interpretation of experiments. The viewpoint implicit in these questions is likely to be considered by experimentalists to be an apology for automaton theory, and by theorists, to be one of dissatisfaction with the state of the art. It is neither. My view is that the questions raised above are difficult but not unanswerable. Progress will be made, but progress will require new theoretical insights and mathematical analysis, and not merely brute-force computer simulations. Von Neumann’s work has had a profound impact on neuroscience; if we can continue in his tradition, there will be further rewards.
Acknowledgements
This acknowledges the support, in part, of the McKnight Foundation and NIH grant EY7977.
References [1] J. van Neumann, in: Theory of Self-Reproducing Automata, ed. A.W. Burks (University of Illinois, Urbana, IL, 1966).
BIOLOGY
WHAT
CAN
AUTOMATON
THEORY
TELL US ABOUT
THE BRAIN?
J o n a t h a n D. V I C T O R Department of Neurology, Cornell University Medical College, 1300 York Avenue, New York, N Y 10021, USA and Laboratory of Biophysics, The Rockefeller University, 1230 York Avenue, New York, N Y 10021, USA Received 13 September 1989 Revised manuscript received 11 January 1990
The potential contribution of the theory of cellular a u t o m a t a to understanding of the brain is considered.
1. I n t r o d u c t i o n How the brain works is one of the fundamental problems in biology. Despite dramatic advances in understanding at the molecular and cellular level, a n u m b e r of basic issues remain. These include, for example, the mechanism for the determination of neural connectivity by genetic and epigenetic factors, and the basis for the difference between the intellectual capacity of the h u m a n brain and that of other species. Satisfactory answers require integration of understanding at multiple levels of structure into a coherent whole. Thus, a theoretical framework, as well as experiment, is required. But what can one reasonably expect from theory, and what is its relationship to experimental investigations? W i t h these questions in mind, I would like to outline areas of inquiry in which I believe that theoretical insights will make significant contributions towards understanding the brain. The best place to begin is to consider the landm a r k contribution of yon N e u m a n n [1]. Von Neum a n n constructed a cellular a u t o m a t o n capable of universal c o m p u t a t i o n and self-reproduction. This construction d e m o n s t r a t e d t h a t a small set of local rules acting on a large repetitive array can result in a structure with very complex behavior. The von N e u m a n n construction thus immediately suggests how an organ with behavior as complex as the brain's can be specified from limited genetic information.
T h a t each unit of the von N e u m a n n a u t o m a ton had twenty-nine states and four neighbors is inessential to its biologic import; what is important is that the construction could be done at all. The property of universal c o m p u t a t i o n is only indirectly important: we do not need to think of the brain as a universal ~hring machine in order to a p p l y the cellular a u t o m a t o n metaphor; universal c o m p u t a t i o n is simply a rigorous way to guarantee that the von N e u m a n n construction has behavior that, all would agree, is complex. The p r o p e r t y of self-reproduction is relevant in a similar fashion, in that it is a rigorous way to guarantee a rich behavioral repertoire. The observation that the cerebral cortex is composed of a large number of local neural assemblies that are iterated throughout its extent, by itself, is not an existence proof that complex behavior m a y result from a network of simple elements. Von N e u m a n n ' s construction is necessary to show that such a structure is indeed capable of complex behavior, without the need to invoke region-to-region variability, long-range interactions, stochastic components, or mysticism. W h e t h e r the yon N e u m a n n m e t a p h o r is qualitatively correct is open to question. However, even if we agree t h a t it does capture an essential feature of brain organization, there is more to be done than to turn the m e t a p h o r into an allegory: several challenging areas of theoretical inquiry remain.
0167-2789/90/$ 03.50 (~) 1990 - Elsevier Science Publishers B.V. (North-Holland)
206
2.
J.D. Victor
The problem
/ Automaton
of robustness
One major qualitative difference between the behavior of the von Neumann construction and that of the brain is that of robustness, or stability to perturbation. Alteration of the state of a single unit of the von Neumann machine typically leads to catastrophic failure; malfunction of a single neuron or neural assembly should have no measurable effect. To some extent, the instability of the cellular automaton model may be a consequence of the discretization of states - but how many states are needed to provide for robustness? Closely related is the problem of robustness of behavior on an imperfect lattice. The cellular automaton metaphor becomes very inattractive if it would require an ezactly periodic brain. Presumably, neural connections are formed on the basis of a regular overall plan, but with variability at the level of the individual neuron. If the four neighbors of a von Neumann unit were selected at chance from a somewhat larger local neighborhood, the construction would fail catastrophically. Again, this failure is probably a consequence of the small number of neighbors in the von Neumann machine - but how many neighbors are necessary? Yet another kind of robustness is that of relative insensitivity to global alterations of the transition rule itself. In this regard, the brain does not appear to be robust. Rather, special homeostatic mechanisms such as autoregulation of cerebral blood flow act to minimize changes in the metabolic milieu that might induce global changes in neural function. When the homeostatic mechanisms fail in even apparently subtle ways, the result is a gross disturbance of consciousness. It is plausible that the cost of ensuring some measure of insensitivity to mild changes in the transition rule itself is much higher than that of ensuring insensitivity to state change or connectivity - but can this notion be made precise? Introduction of more states and more neighbors leads to other questions. What is the tradeoff between number of states and number of neighbors in providing a given level of reliability? A manyneighbor (100 or more) many-state (20 or more) automaton is biologically implausible unless there is some regularity to the state space and the transition rule. An “irregular” rule is also very likely to
theory and brain
be unstable to perturbation. For a plausible rule, it should be possible to view the states as discrete points in a continuous state space, and the transition rule should be at least piecewise continuous on state space. But what can be said about the topology and dimension of that space, and how complex must the transition rule be? These abstract questions can be made quite concrete if, say, we consider a von Neumann unit to be a neuron. Does a single variable (such as a transmembrane voltage) suffice to specify a neuron’s state, or must additional variables (perhaps concentration of calcium or a trophic factor) be considered? Such factors are crucial for development and synaptic plasticity. But are they required for the moment-by-moment informationprocessing as well? Questions about the transition rule become questions about the informationprocessing capacity of a single neuron. Does it suffice to lump all the neighbors’ inputs into a few pools which combine additively (e.g. an excitatory pool and an inhibitory pool), or is it necessary to hypothesize more complex interactions among a neuron’s inputs? Such interactions, such as presynaptic inhibition, are well known - but are they a requirement, an efficiency, or an inessential byproduct of evolution from a more primitive nervous system?
3. The problem
of scaling
To understand the behavior of a model neural network, it often appears necessary to create an explicit computer model. Indeed, if the behavior of the model could be predicted in a simple fashion from its axioms, the model would likely be criticized as not possessing the “emergent” properties that are the essence of an acceptable model for cortex. But even the most ambitious explicit computer simulations are dwarfed by the number of elements in a real cortex. The scale of a computer model (lo4 to lo5 elements) is about halfway between that of just a single unit and that of a real brain (lo8 to 10’ elements). We need to know how characteristic lengths and times scale as the number of network elements increase from that of a computer model to that of a real brain. A network whose settling time increases even linearly
J.D. Victor
/ Automaton
with network size will have very different behavior when scaled up by four or five orders of magnitude. More generally, we need to be able to understand what happens to global dynamics (number, dimension, and stability of attractors, for example), and whether additional qualitative properties of the model will “emerge” at a more realistic scale. A thorough understanding of scaling behavior may permit a clearer answer to the question of whether the qualitatively distinct features of human brain function may simply be viewed as consequences of its size, or rather, whether other processes (such as the development of new, specialized brain regions) must be invoked.
4. The problem
of model
testing
Let us now assume that we have a model in hand, with acceptable robustness, a biologically reasonable transition rule, and scaling behavior within grasp. How, and to what extent, can the model be tested? Model testing requires two levels of investigation. (i) Does the overall behavior of the model correspond to that of the brain? (ii) Is there a detailed correspondence between parts of the model and parts of the brain? These questions seem inextricably related: it only makes sense to inquire about a detailed correspondence if overall behavior is acceptable, but how do we ask about overall behavior if we do not know what are the model counterparts of biologic observables? Theory can help in two ways. Firstly, theory may be able to identify certain kinds of observables that are relatively insensitive to hypotheses about detailed correspondence. Possibilities for such observables might include dimensionality and stability of limit trajectories, or, how mutual information at two points in the model scales with separation or time lag. Such ideas might suggest new ways to interpret anatomic, single-cell, and gross-potential data.
theory
and brain
207
Secondly, we would like to know to what extent two models which have different internal descriptions may have similar overall behavior. This is perhaps the most important contribution that theory can make. A relatively minor benefit is that competing models that can only be distinguished on the basis of detailed biologic correspondence will be recognized as such. The major benefit is that such an understanding will identify the critical features of internal structure which do affect overall behavior - and thus, define the critical experimental questions.
5.
Conclusion
I hope that the questions raised above will serve as a focus for theoretical efforts, and as a framework for the design and interpretation of experiments. The viewpoint implicit in these questions is likely to be considered by experimentalists to be an apology for automaton theory, and by theorists, to be one of dissatisfaction with the state of the art. It is neither. My view is that the questions raised above are difficult but not unanswerable. Progress will be made, but progress will require new theoretical insights and mathematical analysis, and not merely brute-force computer simulations. Von Neumann’s work has had a profound impact on neuroscience; if we can continue in his tradition, there will be further rewards.
Acknowledgements
This acknowledges the support, in part, of the McKnight Foundation and NIH grant EY7977.
References [1] J. van Neumann, in: Theory of Self-Reproducing Automata, ed. A.W. Burks (University of Illinois, Urbana, IL, 1966).