On the dynamical realization of automata

J. theor. Biol. (1975) 54, 109-120

On the Dynamical Realization of Automata ROBERT ROSEN

Center for Theoretical Biology, State University of New York at Buffalo, Amherst, New York 14226, U.S.A. (Received I 1 November 1974, and in revised form 17 March 1975) A notion of dynamical realizability, meaningful for an arbitrary mapping process f:A---* B between arbitrary sets, is defined. It is shown that the next-state map of a finite automaton is always dynamically realizable, and indeed the realization of such an automaton is closely related to the class of systems we called autonomous state classifiers (Rosen, 1972a). On the basis of these realizations, we have at our disposal a class of systems simultaneously describable in dynamical and automata-theoretic terms, and in particular we can ask whether the dynamical description subsumes the automata-theoretic one. It appears that such a subsumption cannot be extracted from the dynamical laws alone, but require additional constraints; hence the two descriptions in general'are complementary. A number of other features of these dynamical realizations are discussed. Many systems of biological interest, such as the central nervous system, the control of genetic expression in bacterial and other ceils, and the immune system, admit of alternate continuous and discrete descriptions. The continuous descriptions take the form of systems of first-order differential equations, describing the manner in which certain system observables (state variables) are changing in time, as a function of their instantaneous values and the external forces impressed on the system. The study of such systems is loosely called control theory, and mathematically is a part of analysis. Discrete descriptions, on the other hand, involve a finite set of states, which are not further analyzable within the description, and a transition rule specifying how the state changes in an arbitrary but discrete time frame, as a function of the present state and the present external input or forcing. The study of these discrete systems forms the basis of automata theory, which is a part of algebra. Naturally, the properties of these two descriptions, and the interpretation of results obtained via their study, are quite different in emphasis and structure. Many authors have attempted to develop relationships between these two modes of description. One kind of approach is based on the obvious 109

110

R. ROSEN

homologies between the forms of the discrete and continuous system descriptions; automata theory is regarded here as a paraphrase of control theory to a discrete setting (Rosen, 1962, 1969a, 1970) (Arbib, 1965, 1966). These involve attempts at reduction of one kind of description to the other. Another kind of viewpoint arises from the fact that, in science, it is the same real system which is being described in two different languages, so tllat a correspondence principle relating these different descriptions of the same system is what is desired. Underlying this view is the hypothesis that such alternate descriptions of the same system (e.g. the central nervous system) are conveying essentially different information, and therefore that neither is directly reducible to the other (Pattee, 1969, 1974, 1976)(Rosen, 1969b). Pattee, for example (1974) has stressed the fact that the dynamical element of automata theory is, in some ways, artificial and extraneous; automata theory is a way of talking about algorithms, which are essentially timeindependent processes, while control theory depends crucially on the rates at which system variables are changing. In order to facilitate the study of such matters, it would be most helpful to have available a variety of specific systems which are well-characterized, and which simultaneously admit a continuous and an automata-theoretic description. The present note is concerned with the development of such a class of systems. The first basic ingredient of our construction will involve the dynamical realization of an arbitrary mapping process. We will take the viewpoint that an abstract mapping processf:A ~ B can be regarded as the "processing" of input elements A by a system (represented here by the mapping f ) in such a way as to produce output elementsf(a) ~ B. This kind of representation of the behavior of a system is time-independent, or algorithmic, and as such it falls more nearly within the purview of the questions considered in automata theory. But since a "real" processing occurs in real time in a rate-dependent fashion, it is necessary to seek "realizations" of this abstract mapping process involving rates of change of continuous variables in a continuous timeframe. This can be accomplished in the following way.

Definition: An abstract mapping processf:A ~ B is said to be dynamically realizable if we can find (a) a manifold ~, (b) 1 - 1 embeddings ~o: A ~ fl, $: B ~ fl, (c) a dynamics on f~, i.e. a one-parameter family {Tt} of transformations o f f l onto itself, such that, for each a ~ A, we have lim Tt(9(a)) = ~b(f(a)). t " ~ oO

We have explored some of the properties of this concept elsewhere (Rosen, 1973, in press); the main result necessary for our purpose here is the following:

DYNAMICAL

REALIZATION

OF A U T O M A T A

111

Theorem 1 : If two mappingsf'A ~ B, g : B ~ C are dynamically realizable, so is their composite 9 f ' A ~ C. The hypothesis that f is realizable gives us a diagram

f A

--,B

t D

--,fl

{r,} while the realizability of 9 gives another diagram g B

oC

l

t

D'

-,D' {T;}

Since ~0 and ~0' are monomorphisms, we can find a map a : f l -~ fl' such that the diagram

q, B

\

~D

\

I

commutes, by writing c~(~b(b)) = ~0'(b), all b ~ B. Now the relation between the dynamics in D and in D' is indicated pictorially in Fig. 1, where the trajectories in D are under the dynamics {T~}, and the trajectories in D' are under the dynamics {T't}. Now it is clear that any dynamics on D' which gives the trajectories drawn in f~' in Fig. 1 will likewise realize the mapping g. What we desire to do is to replace the given dynamics T~ by a new dynamics ~ which will realize g, and which will also allow us to embed A in fl', and realize the mapping f Looking again at Fig. 1, it is clear that we could do this if we could move the fl-trajectories from D down to D', by an appropriate extension of the

112

R. ROSEN

""~......~(b111 FIG. 1. Relation between the dynamics in [2 and in ~'.

mapping ~. We can do this by making the points ~p'(b) in fl' conditionally stable. To realize g, we need trajectories leaving these points, which requires instability; if we want to move the f2-trajectories down to f~', we will need also trajectories entering these points, which means converting them to saddle points. The required behavior is shown in Fig. 2. We cannot realize the composite mapping gf by extending ~ in such a way that, in Fig. 2,

Io I I I I

I~ .-o I I I I

'(g(b~l)

FIG. 2. The required behavior.

113

D Y N A M I C A L R E A L I Z A T I O N OF A U T O M A T A

q~(al) and to(a2) map directly on the points Pl, P2 respectively. However, by mapping q~(at) and q~(a2) very near to these points, we will obtain a hyperbolic trajectory which will, in fact serve to realize the composite map g f in every case. Therefore, extending the mapping ~: f~ ~ I'~' as indicated, and replacing the original dynamics {T;} on f~' with a new dynamics {~} with the properties mentioned, we can construct the diagram

A

gJ ..... ~ C

$

$

f~'

,-~ f~,

{T,} which proves the dynamical realizability of gf. The above theorem is important, since it shows that a pair of dynamically realizable maps, each realized by a dynamics on a different manifold, can always be concurrently realized by dynamics on the same manifold. The concept of dynamical realizability makes sense for an arbitrary mapping process. Therefore, we can inquire about the dynamical realizability of automata. In the present note, we shall consider a simplified automaton, consisting of a finite set S of states, a finite input alphabet A, and a nextstate map 5 : S x A ~ S. The dynamical realizability of such an automaton then reduces to the dynamical realizability of its next-state map 5. The discussion can easily be generalized to the situation in which an output map 2 : S x A -* B must be realized as well. If we can produce a dynamical realization, in the above sense, of an automaton, we shall in fact have constructed a system which admits alternate discrete and continuous descriptions, and in which the study of the reducibility or complementarity of these descriptions may be studied. The remainder of this note is devoted to producing such a realization. The main tool we shall use is the class of dynamical systems which we termed autonomous state classifiers (Rosen, 1972a; Engel, 1972). These systems arose in an analysis of measurement or classification processes, in which the elements of some input set of states or patterns are to be classified into a finite number of classes on the basis of the value of some observable or descriptor defined on that set. It will be recalled that the classifiers were defined by the following properties: (1) The state set Z of the classifier can be decomposed into N open regions T.B.

8

114 U1 . . . . .

R. ROSEN ON, such that each U1 is invariant under the system dynamics,

and such that N

X=

u U~. i=1

(2) The intersection of the closures O , i = 1. . . . . N, consists of a single state tro, the "zero-state" or reference state of the classifier. (3) Each U~ contains a unique attractor, so that every trajectory in U i tends to the same asymptotic behavior. If i ¢: j, these asymptotic behaviors are different. Intuitively, a "classification" or "measurement" occurs through the coupling of the classifier, in its reference state tro, with an object to be classified. This results in a displacement of the state of the classifier into one or another of the regions U~, as a function of the object to be classified. Once in the region Ui, the autonomous dynamics of the classifier determines a unique (macroscopic) limiting behavior. As will be seen, these classifiers provide a natural vehicle for the realization of automata. To motivate the construction which follows, we shall adopt the following picture of the operation of a finite automaton. If s i e S is any state of the automaton at an instant, then the automaton can be regarded at this instant as a particular kind of classifier of the elements of the input set A. More specifically, if there are N states si . . . . . aN in S, then we can decompose the input alphabet .4 into N equivalence classes Aij, relative to S~ where a ~ Aij iff tS(si,a) = sj. (Of course, some of these classes may be empty.) In this sense, an automaton in a given initial state s~ can be regarded as an N-ary classifier, and since there are N possible initial states, the whole automaton can be looked on as that of a family of N different N-ary classifiers. What the above analysis amounts to is to decompose the given mapping 6 : S x A ~ S into a set o f N individual maps ~51. . . . . 3N, each of which is a map from A into S, defined by ,L(a) = ,~(s,a). Since each of these can be regarded as a classification of the elements o f A, we shall consider how each of these can be dynamically realized as an autonomous state classifier. We shall then combine the realizations of each of these into one big dynamical realization of the original automaton. The realization of each 6~, i = 1. . . . . N in terms of an N-ary autonomous state classifier is very easily demonstrated. Let C~ denote such a classifier; we shall use the same symbol to denote both the classifier and its set o f states. Let UI . . . . . UN be the N domains of attraction of this classifier, and

D Y N A M I C A L R E A L I Z A T I O N OF A U T O M A T A

115

let ~1 . . . . , au be the corresponding attracting states. If we then define tpi: A ~ C~

simply by stipulating that ~oi(a) e Ut when 6i(a ) = s t, and the map

0i: S ~ C~ by writing ~b~(sj) -- trj, then it is immediate that the N-ary classifier C~ is a dynamical realization of the mapping 6 i, for each i = 1, I . . , N. We now need to put these classifiers together. The way to do this is indicated by the sequential character of the operation of the automaton. Let us suppose that si was the initial state, and that 6~(a) = st for the input element a ~ A. Then at the "next" instant, we have moved from the classifier C~ to the classifier C~, and therefore there must be an identification of the reference state of C~ with the limiting behavior of trajectories in the classifier C~. The best way to see how this happens is to consider a simple specific example; the topological arguments seem complicated, but, as will be seen, they result in a simple and familiar picture. Consider as an illustration the following specific automaton: S = {st, s2}; A = {al, az, aa}; 6 : S x A -.+ S is given by the table Si

S2

al

S2

$1

a2

S1

$2

"a 3

52

s2

The map 61:A -~ S defined by 61(a) = 6(sl,a) classifies A into two classes: A11 = { a t A161(a) = sl} = {a2}

Aaz = {a ~ A161(a) = sz} = {ax, a3} Likewise, the mapping 62 :A ~ S defined by 62(a) = 6(s2,a) classifies the input set A into two classes:

A ~ = {a ~ Ala~(a) = s~} = {~, a~} The trajectories in the two corresponding classifiers C~, Czz, considered separately, can be rcpresentcd geometrically as in Fig. 3.

116

R. ROSEN

oz~

oa2

c~

Fl~. 3. Geometric representation of trajectories in C~2 and C~ considered separately. To take account of the sequential operation of the automaton, we want to make the following identifications: (1) The limiting states o'1~, ~r2~, which intuitively leave the automaton in the state &, with the origin of the classifier C~. (2) The limiting states a~2, a22, which intuitively leave the automaton in the state s2, with the origin of the classifier C~. When we make these identifications in Fig. 3, and draw the corresponding trajectories, the manifold of trajectories of the resulting system will be as shown in Fig. 4. In doing this we make essential use of Theorem 1 above.

o5

FIG. 4..The manifold of trajectories of the system. This will be recognized as nothing other than the "state transition graph" for the automaton, but in this situation it has a completely different interpretation. Each arrow in the state transition graph represents a bundle of trajectories under the classifier dynamics, in separate pieces of manifolds corresponding to the corresponding classifier regions Ui. The actual stitching

DYNAMICAL REALIZATION

OF A U T O M A T A

117

together of these classifier regions is complicated to describe in topological terms, .but the essence should be clear by straightforward generalization from the above example. We have one further aspect of the above construction to make. The system whose trajectories are sketched in Fig. 4 is already a dynamical realization of the given sequential machine. However, it is not itself an autonomous state classifier, although it was constructed from such classifiers. By one additional refinement, we can turn this realization itself into a classifier. To do this, we simply observe that putting the automaton into an initial state, or determining that the automaton actually lies in a particular one of its N possible states, is itself a classification operation, and can be performed by a particular N-ary classifier CoN. If we identify the attracting states of this classifier CoN, representing the initial states of the automaton, with the appropriate origins of the classifiers C~, i --- 1. . . . . N, and again utilizing theorem 1, we turn the entire system into a classifier, which realizes the original automaton in a straightforward fashion. For instance, for the automaton realized in Fig. 4, we should obtain the classifier realization shown in Fig. 5.

FIG. 5. Classifier realization for automaton realized in Fig. 4.

Thus we have in effect proved the Theorem 2. Theorem 2: Every finite automaton can be dynamically realized, and in fact can be realized by an appropriate autonomous state classifier. Before we proceed to some applications and implications of this construction, there is a technical point which needs to be mentioned. The realization of a mapping in dynamical terms which we have defined is an asymptotic realization, in which the actual value of the function being realized is strictly attained only at t = oo. However, we view this as being

118

R. ROSEN

essentially a mathematical artifact of the representation; by the s~ne kind of argument we can say that a chemical reaction never comes to equilibrium. It is well known that such artifacts can be avoided by exploiting continuity of state representations, so that the functional values of the functions being realized will be effectively reached in finite times. What these times will actually be will depend essentially on the realizing dynamics, of course, and each. realizing dynamics will impose a real-time significance on the arbitrary discrete time-scale of the automaton. It is quite easy to see, for instance by using oo-ary classifiers, that Theorem 3 follows. Theorem 3: Not every dynamical or control system is a realization of a finite automaton. Thus there are many dynamical systems which cannot be described in automata-theoretic terms. Indeed, in order for a dynamical system to be so describable, it is in some sense necessary and probably sufficient, that it be closely related to an autonomous state classifier. This points up another significant aspect of the properties of such classifiers. One significant mathematical property of the classifiers might be mentioned here. It was shown in Rosen (1972a) that the classifiers C M (for N > 2) are not linearizable, and hence not structurally stable. It might be of some interest to discover how the set of classifiers is embedded in the set of all dynamical systems on a particular set of states. We conclude with three general remarks, whose implications will be explored in subsequent papers. (1) The main advantage of a dynamical description, from the standpoint of biological interpretation, is the capacity in principle to identify the state variables occurring in a dynamical representation with those manifested in a real experimental system, and thereby make detailed, experimentally testable predictions about the time course of system behavior. In addition, we can also build specific (e.g. chemical) realizations, whose kinetics satisfy the same equations as the realization, and study their properties; these then become "model systems". For instance, we can ask for dynamical realizations of the simplest kind of automaton, the McCultoch-Pitts formal neuron. Any such realization will be a "model neuron"; any physical system which satisfies the dynamical laws governing the realization will be a "neuromime". One can then compare the properties of these realizations with real neurons, or with excitable element models derived from quite different principles, such as those satisfying the Hodgkin-Huxley equations, or the equations posited by Zeeman (1972) in his beautiful analysis, based on apparently quite different considerations than those employed above.

D Y N A M I C A L R E A L I Z A T I O N OF A U T O M A T A

119

(2) We began our analysis of dynamical realization of automata by asking whether, in a system which admits both kinds of description, the discrete description is reducible to the continuous one, or whether they convey essentially different kinds of information, and are therefore complementary. In the analysis we have given above, it appears that both possibilities coexist. In purely temporal aspects, it appears that the dynamical description is surely more general, and hence subsumes the discrete description. However, a glance at Fig. 4 will reveal the curious topology of the state space of the dynamical realization; a topology which, in conventional kinetic terms, means that there are severe constraints, of a non-holonomic character, imposed upon the dynamical realization. The importance of such constraints, and their independence from the dynamics itself, has been repeatedly emphasized by Pattee (1969). We see here in a particularly graphic way how these constraints manifest themselves in dynamical behavior which admits an automata-theoretic description. Indeed, it has been argued by Polanyi (1968) that, since these constraints themselves obey no dynamical law, we cannot understand biological organization in purely dynamical terms. (3) One of the hopes in expressing automata in dynamical terms is that we will be able to reformulate important automata-theoretic problems in such a way that the powerful tools of analysis may be brought to bear upon them. For instance, it would be interesting to know what, if any, analytical correlate there might be of the concept of computability. The analysis we have given above does not seem to throw any immediate light on such questions. For one thing, the input alphabet A is not explicitly visible in our discussion (it appears only in the cartesian product S x A); and even less therefore is the free semi-group A* which is the basis for posing decidability and computability questions. Indeed, if we cannot readily pose such automata-theoretic questions in analytic terms, this fact will itself provide further grounds for asserting that the automata-theoretic and dynamical descriptions of any particular machine are complementary, and not reducible one to the other. (4) Finally, the definition of dynamical realizability makes sense for any arbitrary mapping, between arbitrary mathematics structures. For example, we can ask whether the composition map A x A ~ A, which turns the set A into a semigroup, or group, is realizable; for finite sets the answer is always affirmative (simply by repeating the above arguments, choosing A = S and giving 6 the appropriate properties). We thus obtain a dynamical representation of an algebraic object. Homomorphisms between these algebraic objects then can be realized as "deformations" of their dynamical representatives. These facts may have interesting consequences of a categorytheoretic nature, since they offer the possibility of functorially embedding

120

R. ROSEN

categories o f arbitrary mathematical objects into categories o f dynamical systems. This paper was prepared with the support of N I H Grant No. 2R01 HD05136-04, Grant No. 1P01 HD07328-01, and N A S A Grant No. NGP,33015002, while the author was in residence at the Salk Institute for Biological Studies. We are most grateful to D r Salk for the facilities he made available during this time. REFERENCES ARnm, M. A. (1965). d. SIAM Control, Ser. A 3, 206. A ~ m , M. A. (1966). Automatica 3, 161. ENos, A. (1972). Thesis, State University of New York at Buffalo. PATrEE, H. H. (1969). In Towards a Theoretical Biology, vol. 2 ((2. H. Waddington, ed.). Edinburgh: Edinburgh University Press. PATrEE, H. H. (1974). In The Physics and Mathematics of the Nervous System (H. Gottinger & M. Conrad, eds). New York: springer-Verlag. PAITI~, H. H. (1976). J'. theor. Biol. (in press). POtANYI,M. (1968). Science, N.Y. 160, 1308. ROSEN, R. (1962). Bull. Math. Biophys. 24, 375. ROSEN, R. (1969a). Proc. Helgoland 3rd International Symposium on Quant. Biol. and Metabolism. New York: Springer-Verlag. ROSEN, R. (1969b). In Symposium on Information Processing in the CNS (K. N. Leibovic, ed.). New York: Springer-Verlag. ROSEr~, R. (1970). Dynamical Systems Theory in Biology. New York: John Wiley & Sons. ROSEN, R. (1972a). Mathl Biosci. 14, 151. ROSEN, R. (1972b). Mathl Biosci. 14, 305. ROSEt~, R. (1973). Bull. math. Biol. 35, 1. ZEEMAN, C. (1972). In Towards a Theoretical Biology, vol. 4 (C. H. Waddington, ed.). Chicago: Aldine.