- Email: [email protected]
Two neuron dynamics and adiabatic elimination
PHYSICAti
Physica D 67 (1993) 224-236 North-Holland
SDI: 0167-2789( 93 )E0063-H
Two neuron dynamics and adiabatic elimination Z e n g Z h a o j u e l, W.C. Schieve a n d P r a n a b K. Das llva Prigogine Center for Statistical Mechanics and Department qf Physics, University (ff Texas, Austin, TX 7S712, US'A Received 7 April 1992 Revised manuscript received 25 N o v e m b e r 1992 Accepted 27 N o v e m b e r 1992 C o m m u n i c a t e d by G. Ahlers
We consider an effective neuron network obtained from the Hopfield model by adiabatic elimination. The validity ol the slaving procedure is first numerically examined. The selfqntcraction of the effective neuron network may lead to non-zero attractors (far from the origin [ / , - 0). The qualitative structure of the attractors for this two effective neuron model with ./leJ:~ > 0 or <(I is analysed in detail. In the latter a limit cycle appears.
1. Introduction
The seminal work of Hopfield [1,2] has generated great interest in physics and engineering in mathematical neuron networks having analogous properties of their biological counterparts [3-8]. With symmetric synaptic connections the model is equivalent to a spin-glass dynamics and the ideas of mean field theory have been utilized in the description [3,9-12]. As argued by Hopfield, these simplifications of the biological have led to new insights into cognitive processes as well as suggest new non-linear methods of computation. The Hopfield-like model which is the basis of our discussion is of the form
C d Ui i-~-=
?~' ~ Jii tanh UI
1#i=1
(i- 1,2,3 .....
U, R,
(1.1)
N).
Here Ui is the potential of the ith neuron with ' P e r m a n e n t address: G u a n g D o n g , China.
Jia
/1167-2789/93/$06.00 © 1993
Ying
University,
MieZhou,
input capacitance C and intermembranc resistance &. The non-linear term represents (biologically) the input to the soma from the other neurons ( j ) with characteristic saturation of their fring rates, J# tanh Ui. We take Jii = Jii and important for this paper J,~ =0. There is no selfconnection in the "bare" network. A good discussion of these network equations is given by Shamma [41. Recently there has been a growth of interest in few (even single) neuron non-linear dynamics with or without stochastic noise. Babcock and Westervelt [13] examined models involving two non-linear threshold switching elements. They introduced inertial terms and numerically demonstrated a bifurcation sequence including dynamical chaos. Hopfield [14] introduced a few non-linear oscillator models of the olfactory bulb. In addition, Skarda and Freeman [15] have analysed the electroencephalograph of the olfactory bulb successfully with simple Hopfield-type models. One of us has given simple neuron models a broader validity [16,191 by arguing after the slaving principle of Haken [17,18], that for large
Elsevier Science Publishers B.V. All rights reserved
Z. Zhaojue et al. / Two neuron dynamics and adiabatic elimination
groups ( j ) of neurons in eq. (1.1) (by an adjustm e n t of parameters, the resistance R 1 >> Rj, for instance) may be in a steady state on a long time scale characteristic of a few neurons and thus effectively follow the non-linear dynamics of a few (k = 1, 2 , . . . ,n ~ N ) neurons. This was carried out in a deterministic and stochastic context. T h e resulting effective neuron equations are N
Ck(jk __
Uk + ~
_ _ _ _
Rk
+ ~
2
Jkjtanh Uj
j~k -1
n),
(1.2a)
and
Ci(Ji=O (Ri~Rk, i=n+ l,n+2 . . . .
,N). (1.2b)
A meaning was, thus, given to a self-connection having a simple form for a single effective neuron (k = 1). In the case N
Jl~ = ~
J :ljRj •
nection with an electronic trigger circuit. Two cases will be examined for the two effective neuron system with J12J2~> 0 or < 0. It is particularly the latter which will lead to interesting new qualitative features. In the last section, we will c o m m e n t on the new limit cycle and its properties in the case J~zJ21 < 0.
2. Numerical examination of "slaving" in a neural network
JkjRjtanhUk
]=n+l
(k : 1,2, 3 . . . . .
225
(1,3)
]-2
In a sense the neuron is "clothed" (renormalized in a field theory sense). If a self-term is included, ab initio, then (1.3) would be a correction [20]. T h e adiabatic elimination in this case both deterministic and stochastic has been carried out [19]. The ab initio self-term may be considered as an internal neuron feedback due to its own complex structure. This approach is not adopted in this paper. H e r e we will first, in section 2, examine numerically the scaling indicated by (1.2) and the time scale for the validity of the slaving assumed. In section 3, we examine the qualitative structure of the steady states of two effective neurons (n = 2) with the others slaved. The steady states for k = 1, N = 2 have in fact been examined by A n d r o n o v et al. [21] some time ago in con-
As discussed in [16], an adiabatic elimination procedure can be applied to the Hopfield model of neural networks. In this section we consider various computational measures of the accuracy of that approximation and the applicability of the basic concepts of H a k e n ' s procedure [17,18] to this system. We first consider the dynamics of the coupled non-linear equations for the network. Since the adiabatic elimination procedure hinges on the dominant time dependence of the dynamical variables, we present a measure of the exponential approach to steady state for the nodes in certain p a r a m e t e r regimes. We also u n d e r t a k e a numerical comparison of the solutions of a first order effective neuron to the solution of a single node in the coupled system. We provide a quantitative measure of the accuracy of the approximation for specific choices of the system parameters. Adiabatic elimination can be applied if time dependences of the dynamical variables are of different scales [17]. In the neural network, the dynamical variables are the individual neuron potentials U i. The m e m b r a n e resistance Ri can be seen to provide a measure of the time dependence of the equations of motion. In first order, our dynamical equations give an exponential time dependence U i = exp(-t/R~Ci). Hence, the resistances Ri are indicated as parameterizing the relative relaxation times of the neurons. E v e n at relatively low ratios of resistances (and hence the time constants), slaving is present. The accuracy of the first order effective
22~
Z. Zhaojue et al. / Two neuron dynamics and adiabatic elimination
neuron in simulating the dynamical behavior of a priviledged " m a s t e r " neuron is seen to depend on the p a r a m e t e r s of the network. In a limited example of eight neurons, we examined the time dependence of the potentials as a function of the ratio of m e m b r a n e resistances. Neuron 1 was taken to have unit resistance, with all other resistances varied together. All couplings were taken to be symmetrical and all cross-couplings to be equal. Fig. 1 shows a characteristic dynamical plot of the convergence of the neurons to their steady states. The upper trace is the slow variable Uj while the lower traces are the other potentials. This is a log plot of the absolute value of the potential. As is clear from fig. 1, the initial phase of the dynamics is nearly exponential. The final convergence is slower than exponential. Fig. 2 is a plot of the ratio of convergence times as a function of R~/ R / ~ . As the ratio R~/R,~j grows, the disparity between convergence times grows. The slope is seen to be 0.49. The time dependence of the network as shown in figs. 1 and 2 clearly make the system eligible for treatment with adiabatic elimination for
Log (101
Time
Fig. 1 The dynamics of the neuron potentials U, for eq. (1.1) for R~/R,+, = 4.0. log(IU~]) is plotted against time. The upper trace shows the convergence of U, :, the lower traces the behavior of the other neurons with varying initial values. Here, Jii = 2.6 and J / = 0.3 Vi, j.
i
U ~ - U :-:
5
f
RI/Ri
~]g. 2. Ratio o| convergence times for LT~ and L; ~ ~crsus R:,'I?,.~ as in fig. I. The slope is 0.49. Parameter values arc the same as those in fig. [.
reasonably small values of R~/R,¢~. Since the relative relaxation times are close to linear in the ratio of the resistances, a sufficiently large disparity in settling time can be obtained with a ratio of resistances of the same order of magnitude. In the presence of a fairly small variation of m e m b r a n e resistance between neurons, slaving takes place. If, in a many-neuron system, this kind of disparity of resistances is present, the dynamics of the whole network should hinge on the behavior of only the slower, higher resistance neurons. This is the essence of the effective neuron concept. The slaving of the fast neurons to the slow master neuron allows for an elimination of variables and the reduction of the dimensionality of the system of equations. These results suggest that a ratio of resistances less than ten is sufficient to achieve an effective slaving of the U,~: to U, = 1. We expect then to find reasonable correlation between the effective neuron's steady state and that of U , - 1 as computed from the coupled n-body system. Fig. 3 shows the ratio of the difference between U k_l (as calculated from eq. (1.2a) (n = 1, N = 8)) and U i = I (as calculated from eq. (1.1) with N - - 8 ) and U, = 1 versus the ratio R~/Ri~ ~.
Z. Zhaojue et al. / Two neuron dynamics and adiabatic elimination 7"
Here Ck = assumed Jk tanh U~ depending [16]. We
Ii
6"
5" ¢~1
x
4x
3"
2
x
x x
x x
x
4
6
8
10
12
227
14
R1/Rn
Fig. 3. Ratio of the difference between the steady state value of Uk 1 in eq. (1,2a) ( n = l , N = 8 ) and Uland U,~lin eq. (1.1) (N = 8) and U~_~v e r s u s RI/R,= I. The deviation of Uk 1 peaks at 38% of U,~ at R~/R,~ = 3.8 and falls rapidly. By R~/R,~ 2 = 10.0, the deviation is within 5%. Parameters are as in above figures. T h e difference between the steady state value of the effective neuron and U 1 is seen to peak at 38% of U 1 and to fall rapidly to less than 5% for R I / R ~ 1 = 10.0. For the specific p a r a m e t e r choices of fig. 3, the first order effective neuron is shown to be a good approximation of the behavior of the slow or " m a s t e r " neuron in the coupled 8 node system. As the ratio of the p a r a m e t e r s JH/J12 falls, the necessary ratio of resistances grows somewhat. For small or absent self-terms, a second or higher order effective neuron formulation is required as the first order a p p r o a c h fails. We have not yet developed such a higher order theory.
1 and R k = 1. Each effective neuron is to have a self-excitation term, (Jk > 0). The details of the form of Jk on Jkj (1 > n) does not concern us assume that the cross-connections Jkj (k ~ j) may be symmetric or asymmetric. Although symmetric (Jij = Jji) and antisymmetric (Jij = - J j i ) connections a m o n g neurons are easy to implement in an electronic circuit mimicking the biological neuron network, asymmetric connections a m o n g neurons may appear in the biological neuron network [22-24]. Now, we consider how the phase space flow of eq. (3.1) lead to additional attractors (far from the origin) by virtue of the self-excitation. First, we look at the flow near the origin. The relative rate of change of volume V in phase space, under the action of the flow (3.1), is given by the Lie derivative [25,26]
1 dV_ V dt
~ k=l
O(Jk_ ~ Ofk OUk k = l OUk"
(3.2)
We have from (3.1)
V dt
- 1 + Jk sech2Uk
t
,
(3.3)
and we conclude that if
•
Jk > n ,
(3.4)
k=l
3. Qualitative analysis of steady states of an effective neuron network
then the volume V of the flow (3.1) would dilate when [Ukl is small,
3.1. The flows in phase space
1 dV V d-'t- ) 0 "
Let us first consider an effective neuron network described by the deterministic equation /-)k = -- Uk + Jk tanh U k +
~
J~/tanh U/--= fk
j~k=l
(k=1,2,3,...
,n).
(3.1)
(3.5)
The origin is unstable, and the flow (3.1) in the vicinity of the origin will point out. But, it does not go to infinity. Secondly, let us look at the flow (3.1) near infinity. We may write (3.1) as
228
ukdUkdt -
Z . Z h a o j u e el al.
/ Two neuron dynamics and adiabatic elimination
U~ + JkUk tanh U k
+ 2
J,jU k tanh Uj
.
(3.6)
or negative. We will discuss the two cases of Ji2J21 > 0 and Jl2J21 < 0 . First we look for fixed points of cq. (3.8) which are solutions to the system
jck = 1
f,
Since Itanh U k ] < 1, then
,,( - U ~ + JkUk tanh Uk
d' L aT(, iU ~ ) : 2 k ~ = ,
+ ~ ]~k-
JkjU~ tanh Uj) 1
<-2<. k
(3.7)
I
T h e Liapounov function, ~k:~v" U-k, shows that the flow (3.1) from infinity will move into an n-dimensional hypershell. The two tendencies of the flow (3.1) illustrated by (3.5) and (3.7) show that the flow (3.1) will end at some forms of attractors located in the n-dimensional hypershell. Therefore, the appearance of a self-excitation term causes an effective neuron network to have additional attractors. They may be fixed points, limit cycles or tori, or possibly a strange attractor leading to chaos. A strange attractor, in fact, has been numerically found in a four effective neuron model. The details are given elsewhere [21]. In the following sections, we qualitatively analyse in detail these attractors for the case n = 2. H e r e , of course, a strange attractor is not possible.
3.2. A two effective neuron system ]'or Jt2J21 > 0
0,
]~:0.
(3.9a, b)
The solution to the transcendental system (3.9) cannot be easily represented as an analytical function of p a r a m e t e r s J~, J2, Jt2 and J21- As an alternative, we shall explore the qualitative features of curve (3.9a), illustrated in fig. 4. When J~-< 1, curve (3.9a) has no peak or valley. And when Jr > 1, curve (3.9a) has two antisymmetric peaks at (U I.... U~,,,) and ( - U I , , , U2,,). Here. U~,, and U2, , are determined by the solution to system
c°sh2 Ulm-
(3.10a)
JI "
[tanh U2,~] =
UI,, - J~ tanh U,, n
(3.10b)
J12
The larger J, and the smaller [J,zl, the higher the p e a k becomes. We emphasize that curve (3.9a) will divide into three discontinuous manifolds when eq. (3.10b) gives Itanh U2,,, I -> 1. For examU2
,
(3)
I
(3)
(1) --1
1(13)
~,
U~
As a specific detailed example, consider the two effective neuron system which is described by 01 - - U~ + J~ tanh U l + J12 tanh U~ = fl , 02 = - U, + J2 tanh U2 + J21 tanh UI = f2,
(3.8)
where the self-connection J1 > 0 and J2 > 0, and the cross-connections J12 and J2! may be positive
Fig. 4. The curve, U~ J, tanh U, + J~2tanh U:, in the three cases: curve (1) (J~ = 0 . 9 , J~,= t ) with no peak, curve (21 (J~ 1.4, J~2 = 1 ) with two peaks, and curve (3) (J, 1.5, J~, - ~ ) divides into three manifolds,
229
Z. Zhaojue et al. / Two neuron dynamics and adiabatic elimination
pie, when J ~ = l . 5 and 1J211->0.207 546, the curve (3.9) will divide into three discontinuous manifolds, shown by curve (3) in fig. 4. The asymptotes of the three manifolds of curve (3) in fig. 4 are perpendicular to the Ul-axis. The two asymptotes of the inner manifold of the curve (3.9a) approximately a r e U 1 = ~ IJ12l/(Jl 1), and the asymptotes of outer manifolds of curve (3.9a) approximately are U~ = ± (J~ + IJ12]) and U1 = ~ (J1 - ]J121)" When J~2 changes sign, c u r v e (3.9a) is reflected about the U~-axis. The features of curve (3.9b) are similar to those of curve (3.9a). By virtue of the two curves (3.9a) and (3.9b), it is easy to see there may exist one, three, five, or nine equilibrium points in the two effective neuron system with J12J21 > 0 or < 0. Fig. 5 shows the overlay of (3.9a) and (3.9b) for the case of one continuous and three discontinuous manifolds. Here the presence of nine fixed points is seen. The other possibilities are deduced similarly. For example, fig. 6 illustrates the case of nine fixed points in the two effective neuron system with J12J21 > 0. They are the origin, four nonzero stable fixed points (labeled in fig. 6 A 1, A 2 , A 3 , A 4 ) and four unstable saddle points (labeled B1, B 2, B3, B4). The stability of these
U~
151
r 2 n , ~
A,
I
T~a,
B2
-
U1
_
i
-1.
Fig. 6. Nine fixed points, A t , A2, A3, A4, BI, B2, B3, B 4 and O, in the model 6"~ = - U ~ + 1.4 tanh U~ + 0 . 2 t a n h U2; 02 = - U2 + 1.4 tanh/-]2 + 0.2 tanh U1. The A are stable and B unstable fixed points, TIA~, T2~ ~ indicate the tangents at point A~. Note the arrows, similarly for the other A, B points.
points are determined by the characteristic equation [20] - 1 + J~ s e c h 2 U 1 - A J21 sech2Ux
I
J12 s e c h 2 U 2 - 1 + J2 s e c h 2 U 2 - A = 0
(3.11)
Its roots are
U1
AI.2 = ½(a +- ~%-5-c2+ 4 b ) ,
(3.12)
where a =
2
.....
3
U2
J1 sech2Ul + J2 sech2U2
Fig. 5. Two solution curves intersect at nine points indicating the positions of the nine fixed points of the network. Here the parameters are J11 = 1.6, J12 = - 0 . 2 , J21 = -0.845, Jzz = 2.4.
2,
c = IJ1 s e c h Z U 1 - J2 sech2U2] , b = J12J21
,
-
sech2Ul sech2Uz.
Here, (UI, U2) are the coordinates of the fixed point, A or B type. Now, we develop a qualitative method for determining the stability of the fixed points without direct numerical calculation of (3.9) and (3.12). Let Tie be the tangent of curve (3.9a) at
Z. Zhaojue et al. ,; Two neuron dynamic.v and adiabatic elimination
23{I
a singular point, P, and Tee be the o t h e r tangent of curve (3.9b) at the s a m e point, shown, for instance, in fig. 6 at point A ~ or B t (labeled Tt~~, Te, , at point A.) T h e arrows indicate the two t a n g e n t s at A, and B~. F r o m eq. (3.9a), we have
Ji2Tlll sechZU.
1 - . I I sech2Ul .
For a m o r e detailed analysis of the two effective n e u r o n n e t w o r k s with J12J21 2> 0, we plot a bifurcation d i a g r a m fig. 7 which d e p e n d s only on p a r a m e t e r s Jz, JÈ and the product J12J21. It is d r a w n by analysing the stability of the origin (U t = U, = 0) given by the two roots a~. 2, where
(3.13) A,.2 = ~ [ ( J t + J+
A n d , f r o m eq. (3.9b). we have
J21T2t" sech2Ui
1
.I, secheU~ .
(Jt2J:t (3.14)
T h e n , the p a r a m e t e r s a and c in (3.12) can be r e p r e s e n t e d by the two tangents at the singular point. a = -(J12Tt/,
secheU~ + Jel T2t, secheUi ) ,
(3.15) ," = IJ,eTjtl s e c h : U ,
J21T2,, sccheU, I •
(3. lO)
First, at a saddle point such as point B~ in fig. 6, if the two tangents, T.~, and T2B I, have opposite signs, then ]a] < c. C o n s i d e r the case when the c o o r d i n a t e s of BI, U ~ , is much larger than Uez~l, then 4b < c z. T h e n , one root of (3.12) is positive, and the o t h e r negative. This m e a n s that point B~ is unstable, a saddle point. If on the o t h e r hand, at a fixed point such as A ~ in fig. 6, the two tangents have s a m e signs, then ]a[ > c. C o n s i d e r a point A~ which lies far from the U~and U,-axis, then, J~2 secheU: ~ 0. Thus, the two roots of (3.12) are negative, or A~ is a stable fixed point. Finally, we conclude that a fixed point is a saddle point if it lies at least on an inner manifold of curve (3.9a) or (3.9b), and a stable fixed point if it lies at the intersection of the two o u t e r m a n i f o l d s of (3.9a) and (3.9b). In the light of this, A ~ , A 2 , A ~ , A 4 in fig. 6 are stable fixed points, and B~, B e, B~,, B a are saddle points. T h e qualitative m e t h o d for determining the stability is easily e x t e n d e d to those cases of five and three fixed points when J12J21 > 0, and to the case of J12J2~ < 0 discussed in the s u b s e q u e n t section.
2) ++-~/( J, - J2)2 + 4J12J2,]
(3.17)
:> [1) .
In region 1 of fig. 7, the origin is the unique stable fixed point. In region 2, the origin is a saddle point and there also exist two non-zero stable fixed points. In region 3, the origin is an unstable node and there m a y now exist four or eight n o n - z e r o fixed points, half of t h e m arc stable, half are saddle points. This is p r o v e d by the P o i n c a r d - B e n d i x s o n t h e o r e m [25,20] (see a p p e n d i x A). In fig. 7, the b o u n d a r y curve s e p a r a t i n g the two regions of four and two fixed points could not be shown as a curve, because it is sensitive to all four p a r a m e t e r s J , / o f eq. (3.8). For m o r e details a b o u t this situation, we show a n o t h e r bifurcation d i a g r a m , fig. 8 (see appendix B). T h e two cases of nine and five fixed points
J2
4~
Z
3
I
I--
Fig. 7. The bifurcation graph with ./~. J, and J~:J:~ =/I.2. The boundary curves are (J~ I ) ( J : - 1) = Jt,J~l. The two roots of (3.17) for the origin arc, A ~ : < 0 in region 1, A , - 0 and A , < I I in region 2 and A ~ : - t l i n region 3.
Z. Zhaolue et al. / Two neuron dynamics and adiabatic elimination
231
&
J~2l
4
2..5
3.5"
2
4(I)
1.5
?
3 2.
1
3
0.5 0.5
z
1.s
2
2.5
3
3.5
~
J1
Fig. 8. The bifurcation graph with J1 and IJlZl for the case Jt - J2 and Ji2Jz~. In region 1, the origin is the unique stable node. In region 2, the origin is a saddle point and there are two non-zero stable nodes. In region 3 and 4, the origin is an unstable node. Two stable nodes and two saddle points are in region 3, and four stable nodes and four saddle points are in region 4.
do not appear in the usual Hopfield model of two neurons. Finally, there is no limit cycle when J12J?l > 0 in the two effective neuron system. This is the same as the Hopfield model of two neurons (see appendix A).
0.5
1.s
~
215
3'.5
~
J~
Fig. 9. The bifurcation graph with J1, Je and Ji2J, t = -0.25. The two roots of (3.18) for the origin are, Re A~,2<0 in regions 1 and 2, Re A1,2> 0 in regions 4 and 5, At > 0 and A2 < 0 in region 3. The dashed lines are J~ = 1.968 and J2 = 1.968. For JlzJ2~ < 0 parameter region 5(3) contains no stable fixed points and thus a limited cycle may exist, as discussed in section 4 of the text.
Jl + J2 - 2 = 0 , [ J 1 - J21 = 2V;IJlzJ2~] ,
3.3. A two effective neuron system f o r JleJel < 0
(J1
In light of the analysis of section 3.2, the curves (3.9a) and (3.9b) in a two effective neuron network with J12J21 < 0 may divide into three manifolds for sufficiently large J~ and J2. As discussed in section 3.2, there may exist one, three, five, or nine fixed points in the system, The cataloging of these fixed points depends on the stability of the origin (U~ = U 2 = 0) given by the two roots
These curves depend only on J1, J2, and the p r o d u c t J12J21, three of the four parameters in the flow (3.8). The origin (U 1 = U2 = 0) is a stable focus in region 1 of fig. 9, and a stable node in region 2. Then, in the two regions, the origin is the unique stable fixed point and there is no saddle point. In region 3, the origin becomes a saddle point, and there also exist two stable fixed points far from the origin. In region 4, the origin is an unstable node, and there may exist four or eight non-zero fixed points in subregions 4(2) or 4(1). The characters of these points in regions 2, 3, and 4 are the same as that when J12J21 2> 0. In region 5 of fig. 9, the origin may be an unstable focus (not attracting). Thus, there exists eight non-zero fixed points in subregion 5(1), four non-zero fixed points in subregion 5(2), and no stable fixed point but rather a stable limit cycle in
A ,2 = ½[(J1 + J2 - 2) -+
- J 2 ) 2 - 41J12J2111. (3.18)
By the analysis of the two roots of (3.18), a bifurcation diagram, fig. 9, is drawn to show the character of the other stable fixed points and in this case limit cycle when J12J21 < O. The curves separating the regions of fig. 9 are
q- J 2 - -
2) 2 = (J~ - J2) e - 41J12J2,].
(3.19)
232
Z. Zhaojue et al. I Two neuron dynamics' arid adiabauc elirnination
subregion 5(3). As mentioned in appendix A, half of these non-zero fixed points in region 5 (and 4) are stable fixed points and the other half are saddle points. The three cases in subregions 5(1), 5(2) and 5(3), do not appear when Ji2Jel > 0. In the general case, the boundary curves separating the two cases of five and nine fixed points in the regions 4 and 5 of fig. 9 cannot be shown simply as fixed curves. However, when J12J21 are constant, these curves can be seen in fig. 9. For a given Jl2 (or J21 )' we can obtain a critical J~(Jl2) (or JC(J2t)) by means of eqs. (3.10). For example, if J~2 = 0 . 5 , then J~(0.5) 1.968. When (3.20a)
J, >>-J ~ ( J l 2 ) ,
and (3.20b)
J~ > J~(J,_~ ) .
The curves (3.9a) and (3.9b) will divide into three manifolds. In fig. 9, we assumed J ~ 2 J 2 i 0.5, then the boundary curves are J~ = 1.968 and J, - 1.968 which are shown as dashed lines. They divide region 4 into two subregions (4(1) and 4(2)), and region 5 into three subregions (5(1), 5(2) and 5(3)). With fixed J~ (or Jx), we may obtain another critical J~2(J1) (or JCl(J2) ) by means of eqs. (3.10). For example, i f J 1 = 2, then J~2(2) --0.54. If IJ~2i<<-J~2(J1) (or IJ2,l<-J2i(J2)), then the curve (3.9a) (or (3.9b)) will divide into three manifolds. But in this case, the two boundary curves cannot be drawn in fig. 9. As an example, fig. 10 shows the case of nine fixed points appearing in the subregion 5(1) of fig. 9. In light of the analysis of section 3.2, it is easy to show that A I , A 2 , A 3 , A 4 are stable points, and BI, B 2, B3, B 4 are saddle points. In fig. 10, the four inner manifolds of curves (3.9a) and (3.9b) are the approximate boundaries of the basins of attraction of these fixed points because they pass through saddle points, and are c
c
U2
Fig. 10. Nine singular t~oints, A~, A:, A~, A 4, B,, B e. B~, B, and O, in the model UL U~ + 1.6 tanh U, + (I.25 tanh U,: /~), U, k 1.6 ianh ~,"~ (I.25 tanh {"~. nearly straight lines. Therefore, these basins of attractions are separable, and the stable fixed points, A ~, A 2, A ~ , A 4 correspond to four memory states. This has also been numerically observed. Finally, let us draw some conclusions concerning the two effective neuron system with Jt~J21 < 0. First, more fixed points and more saddle points a p p e a r in the system with the selfconnections Jl and J2 increases. In contrast to the Hopfield model of two neurons, when Jl2Jzl < 0 there is no other attractor except the origin. The a p p e a r a n c e of a limit cycle in the two effective neuron system with J12J2~ < 0 is the main qualitative difference between the two cases of Ji2J2t :%0.
4. The limit cycle (J~zJ2~ < O) In an effective neuron network, the appearance of limit cycles of periodic solutions are also possible. First, we obtain the sufficient condition for the appearance of a limit cycle when J~2J2~ < 0. In the light of section 3.1, if the origin (U~ = U z = 0) is unstable and there is no non-zero stable fixed point, then the flow of (3.8) must
233
Z. Zhaojue et al. / Two neuron dynamics and adiabatic elimination
end at a limit cycle. T h e bifurcation d i a g r a m fig. 9 has shown there is a subregion 5(3) in the p a r a m e t e r space, for which the origin is an unstable focus and no n o n - z e r o stable fixed point exists. A s a result, a stable limit cycle must appear. Secondly, we want to show that the limit cycle will n o t a p p e a r w h e n there exist n o n - z e r o stable fixed points in the system. Consider the Poincar6-Bendixson t h e o r e m [21,26]. T h e limit cycle, if it exists, must enclose the origin or all stable fixed points. This is impossible. If there were a limit cycle and stable fixed points, it w o u l d intersect with the phase curves linking the origin (as a saddle point) to infinity, or that linking a n o t h e r saddle point to infinity, or to stable nodes and the origin (as an unstable node or focus). F o r e x a m p l e , in fig. 10, if there were a limit cycle, then it intersects with these phase curves linking saddle points B 1, B2, B3, B 4 to infinity, or stable fixed points A , , A2, A 3, A 4 and the origin. T h e r e f o r e , the condition for the a p p e a r a n c e of a limit cycle in the two effective n e u r o n system with J 1 2 J 2 1 < 0 is that the origin is an unstable focus and there is no stable fixed point. L e t us n o w construct an a p p r o x i m a t e solution of the limit cycle. We focus on the s y m m e t r i c case for simplicity. We write eq. (3.8) as /]]" =
- - U 1 ~-
/f12 = ((1!('02 -- 1)U2 -- /3031
--U2
-[-
dp 2
dt
dO
= 2[(w, + 032)a - 2lp 2 ,
d-7
(4.4)
Here p
2
2
2
= O 1 nt- U 2 ,
U1
0 -- a r c t a n --.
U2
T h e n , the solution to (4.5) is P = P0 exp[2(03a - 1 ) t ] ,
(4.5a)
0 = 00 + / 3 f 03 d t .
(4.5b)
0
If c~03> 1, p will increase with time. With the increase of p, w, and w e will decrease until a critical value, % , w h e r e am c - 1 = 0. T h e n p = Pe and does not increase. Pc is the radius of the limit cycle. O n the o t h e r hand, f o r ' l a r g e IU,[ and IU21 (or p) then a03 < 1, eq. (4.5a) shows that p will d e c r e a s e with time until p = Pc. This m e a n s that the solution (4.5) shows a stable limit cycle. T h e a m p l i t u d e Pc and frequency /3wc of the limit cycle can be obtained. First, consider a point, U, = U 2 on the limit cycle, then
13:'tanh U, + / 3 tanh U 2 , O/ tanh U 2 - / 3 tanh U, .
(4.3)
or in polar coordinates,
tanh Pc 601 =
[ff2 =
U1,
(02 >
-
-
,
Pc
(4.1)
(4.6)
and First introduce 03, = t a n h U~/U, and 032 = tanh U 2 / U 2. F o r IU, I ~ 1 and lull ~ 1, 03, and 032 are nearly equal to one. H o w e v e r , we will ass u m e that 03, and 032 are constant as a first a p p r o x i m a t i o n because they vary m o r e slowly t h a n tanh U, and tanh U 2. A s s u m e 601 ~
032 ~
03 "
(4.2)
t a n h Pc _< _1
(4.7)
Pc
T h e lower limit of & is o b t a i n e d by solving (4.7). F o r e x a m p l e , if a = 1.05, then Pc > 0.39. Second, consider a point (U I = 0 and U 2 = Pc) on the limit cycle, or 03, = 1 and 032 = tanh pc~pc. Thus we have
T h u s eq. (4.1) is
U, = (a03,-
1 ) U 1 +/3032 U2,
tanh Pc Pc
-
>
2 a
-
-
1.
(4.8)
234
Z. Zhao]ue et al. / Two neuron dynamics and adiabatic elimination
U2
i~ U1
Fig. 11. A limil for /~7,- U l + I . O 5 t a n h U ! + O . 5 t a n h U , : U, [,L + 1.115tanh/_7~ 0.5 tanh U~.
For c~ - 1.05, then p< < 0 . 4 7 . Computer simulation of the case of ~ = 1.05 and /3 =11.5 illustrated in fig. ll, shows that there is a stable limit cycle with radius p~.- 0.45. It agrees well with above estimates of p~.. In the asymmetric cases when d~ ¢ J, a n d / o r J i 2 . l ~ l , the limit cycle is not a circle but deformed into an ellipse around the origin.
5. Conclusions In the preceding sections, we have considered a network of two effective neurons. Our analysis suggests that the effective neuron formulation is not only a valid approximation of the larger dimensional Hopfield network, but also that dynamics of considerable complexity ensues in even this low dimensional system. Further, we have constructed explicit bifurcation diagrams for the two neuron network and demonstrated analytically the stability structure inherent in the system. Finally, we consider the onset and behavior of the limit cycle solution and apply the Poincar6-Bendixson theorem to show that limit cycles and fixed points will not coexist in the two dimensional case. Our intent here has been to present a thorough inspection of the simplest multiple effective neuron network both as an interesting dynamical system and as a paradigm for more
complicated, higher dimensional nets. As a n a~sociative memory, this network is capable of storing from zero to four memories as steady states or a single, periodic memory. In a future report, we will discuss the capacities of higher dimension networks and the implications of the periodic solutions. As is clear from the bifurcation diagrams presented here, qualitative changes in the dynamics are produced by movement through the space of the connection strengths. The bifurcations are then seen as resulting in the creation and annihilation of fixed points, of emergence of a limit cycle (and, as previously reported, emergence of chaos in larger dimensional networks). The dynamical systems approach taken here enables us to glean precise information regarding the behavior of the network while investigating a greatly reduced number of effective neurons. In generality, our results pertain to all self-connected Hopfield networks (effective or otherwise). While the parameter spaces grow rapidly for larger networks and exhaustive construction of bifurcation diagrams becomes more difficult, the two neuron case can be seen as a step toward and a module in such a project. In forthcoming reports, we consider the dynamics of systems of three and four neurons.
Acknowledgements A first draft of this work was completed while one of us (W.C.S.) was recipient of a summer Senior Humboldt Foundation Award at the MaxPlanck Institute for Quantumoptics, Garching, Germany. Z.Z. and P.K.D. acknowledge support of the R A Welch Foundation of Texas.
Appendix A In the light of the curves (3.9a) and (3.9b), there may exist one, three, five, or nine fixed points in the two effective neuron system gov-
z. Zhaojue et al. / Two neuron dynamics and adiabatic elimination
erned by eq. (3.8). The origin (U 1 = U 2 = 0 ) is the unique fixed point which is always present. Most of the characteristics of the other fixed points or the limit cycle of the system are obtained by the character of the origin and the P o i n c a r 6 - B e n d i x s o n theorem [20,25]. Consider the flow in (3.8). Infinity is absolutely unstable. Its Poincar6 index is +1, then the sum of the Poincar6 indexes of all fixed points of eq. (3.8) must equal +1. First, if there are nine (or five') fixed points and the origin is an unstable node (the Poincar6 index is +1), then half of the eight (or four) non-zero fixed points are stable fixed points, and the other half are saddle points (the Poincard index saddle point is - 1 ) . They are shown in region 3 of fig. 7 and in regions 4 and 5 of fig. 9. Now, if the origin is a saddle point, then there must exist two non-zero stable fixed points. They are shown in region 2 of fig. 7 and in region 3 of fig. 9. There always exist stable fixed points when JzzJz~ > 0. As a result, there is no limit cycle in these systems. If it were to exist, it would intersect the phase curves linking infinity to unstable fixed points, and linking saddle points to stable fixed points.
Appendix B In fig. 8, the curve separating the region of five fixed points and the region of nine fixed points (with J12J21 > 0) is x = ~/yY ~ _i In(v'-y + ~¢/-y- i ) .
(A.I)
Here X :
J| --IJ12[ ,
y = Jl + IJl=l •
In the case of J1 = J2 and J12 = J21 ~> 0, if there exist nine fixed points, then there are four stable nodes, shown in fig. 6 as the symmetric couple of A 1 and A 3 and the asymmetric couple of A 2 ( - U a, Ua) and A 4 ( U a, - U , ) . From eq. (3.8),
235
U,( > 0) is obtained by Oa = (J1 -
[J121) tanh U ~ .
(A.2)
In the light of (A.1), if the two asymmetric points are present, they must be stable fixed points. As a result, the appearance of the unsymmetric stable fixed points marks the appearance of the case of nine fixed points. Consider the characteristic equation (3.12), two roots for the point A 2 or A 4 are h l , 2 ~" --1 + J~ sec h 2 Ua + [JI21 sech 2 U~.
(A.3)
When (J1 +
IJ~2])secheua =
I ,
(A.4)
it gives the condition for the appearance of the case of the nine points. The fixed points A 2 and A 4 are stable fixed points. Eq. (A.1) is obtained from eq. (A.2) and (A.4) by eliminating U~.
References [1] J. Hopfield, Proc. Natl. Sci. USA 79 (1982) 2554. [2] J. Hopfield, Proc. Natl. Sci. USA 81 (1984) 3088. [3] D. Amit, Modeling Brain Function (Cambridge University, New York, 1989). [4] S. Shamma, in: Methods in Neuronal Modeling, eds. C. Koch and I. Seger (MIT Press, Cambridge, MA, 1989). [5] B. Muller and J. Reinhardt, Neural Networks, an Introduction (Springer, Berlin, 1990). [6] H. Haken, Synergetic Computers and Cognition (Springer, Berlin, 1990). [7] F. Hoppenstaedt, An Introduction to the Mathematics of Neurons (Cambridge Univ. Press, London, 1989). [8] J. Buhmann and K. Schulten, Biol. Cybern. 54 (1986) 319. [9] D. Amit, H. Guttfreund and H. Sompolinsky, Phys. Rev. A 32 (1985) 1007. [10] H. Sompolinsky, Phys. Today 41 (1988) 70. [11] D. Amit, H. Guttfreund and H. Sompolinsky, Ann. Phys. (NY) 173 (1987) 30. [12] P. Peretto, Biol. Cybern. 50 (1984) 51. [13] K. Babcock and R. Westervelt, Physica D 28 (1987) 305. [14] Z. Li and L. Hopfield, Biol. Cybern. 61 (1989) 379. [15] C. Skarda and W. Freeman, Behav. Brain Sci. 10 (1987) 161.
236
Z. Zhaojue et al. / Two neuron dynamics and adiabatic' elimination
[16] W.C. Schieve, A.R. Bulsara and G.M, Davis, Phys. Rev. A 43 (1991) 2613. [17] H. Haken, Synergetics (Springer, Berlin, 1977). [18] H. Haken, Advanced Synergetics (Springer, Berlin, 1983). [19] A. Bulsara and W.C. Schieve, Phys. Rev. A 44 (I991) 7913. [2/I] A. Andronov, A. gitt and S. Khaikin, Theory of Oscillations (Dover, New York, 1966).
[21] P. Das, W.C. Schieve and Zeng Zhajuc, Phys. Lett. A 161 (1991) 60. [22[ J. Denker, Physica D 22 (1986) 216. [23] B. Baird, Physica D 22 (1986) 150. [24] B. Baird, Physica D 42 (1991)) 365. [25] F. Verhulst, Nonlinear Differential Equations and Dynamic Systems (Springer. Berlin, 1980). [26] P. Berge, Y. Pomeau and C. Vidal, Order within Chaos (Wiley, New York, 1987).
Physica D 67 (1993) 224-236 North-Holland
SDI: 0167-2789( 93 )E0063-H
Two neuron dynamics and adiabatic elimination Z e n g Z h a o j u e l, W.C. Schieve a n d P r a n a b K. Das llva Prigogine Center for Statistical Mechanics and Department qf Physics, University (ff Texas, Austin, TX 7S712, US'A Received 7 April 1992 Revised manuscript received 25 N o v e m b e r 1992 Accepted 27 N o v e m b e r 1992 C o m m u n i c a t e d by G. Ahlers
We consider an effective neuron network obtained from the Hopfield model by adiabatic elimination. The validity ol the slaving procedure is first numerically examined. The selfqntcraction of the effective neuron network may lead to non-zero attractors (far from the origin [ / , - 0). The qualitative structure of the attractors for this two effective neuron model with ./leJ:~ > 0 or <(I is analysed in detail. In the latter a limit cycle appears.
1. Introduction
The seminal work of Hopfield [1,2] has generated great interest in physics and engineering in mathematical neuron networks having analogous properties of their biological counterparts [3-8]. With symmetric synaptic connections the model is equivalent to a spin-glass dynamics and the ideas of mean field theory have been utilized in the description [3,9-12]. As argued by Hopfield, these simplifications of the biological have led to new insights into cognitive processes as well as suggest new non-linear methods of computation. The Hopfield-like model which is the basis of our discussion is of the form
C d Ui i-~-=
?~' ~ Jii tanh UI
1#i=1
(i- 1,2,3 .....
U, R,
(1.1)
N).
Here Ui is the potential of the ith neuron with ' P e r m a n e n t address: G u a n g D o n g , China.
Jia
/1167-2789/93/$06.00 © 1993
Ying
University,
MieZhou,
input capacitance C and intermembranc resistance &. The non-linear term represents (biologically) the input to the soma from the other neurons ( j ) with characteristic saturation of their fring rates, J# tanh Ui. We take Jii = Jii and important for this paper J,~ =0. There is no selfconnection in the "bare" network. A good discussion of these network equations is given by Shamma [41. Recently there has been a growth of interest in few (even single) neuron non-linear dynamics with or without stochastic noise. Babcock and Westervelt [13] examined models involving two non-linear threshold switching elements. They introduced inertial terms and numerically demonstrated a bifurcation sequence including dynamical chaos. Hopfield [14] introduced a few non-linear oscillator models of the olfactory bulb. In addition, Skarda and Freeman [15] have analysed the electroencephalograph of the olfactory bulb successfully with simple Hopfield-type models. One of us has given simple neuron models a broader validity [16,191 by arguing after the slaving principle of Haken [17,18], that for large
Elsevier Science Publishers B.V. All rights reserved
Z. Zhaojue et al. / Two neuron dynamics and adiabatic elimination
groups ( j ) of neurons in eq. (1.1) (by an adjustm e n t of parameters, the resistance R 1 >> Rj, for instance) may be in a steady state on a long time scale characteristic of a few neurons and thus effectively follow the non-linear dynamics of a few (k = 1, 2 , . . . ,n ~ N ) neurons. This was carried out in a deterministic and stochastic context. T h e resulting effective neuron equations are N
Ck(jk __
Uk + ~
_ _ _ _
Rk
+ ~
2
Jkjtanh Uj
j~k -1
n),
(1.2a)
and
Ci(Ji=O (Ri~Rk, i=n+ l,n+2 . . . .
,N). (1.2b)
A meaning was, thus, given to a self-connection having a simple form for a single effective neuron (k = 1). In the case N
Jl~ = ~
J :ljRj •
nection with an electronic trigger circuit. Two cases will be examined for the two effective neuron system with J12J2~> 0 or < 0. It is particularly the latter which will lead to interesting new qualitative features. In the last section, we will c o m m e n t on the new limit cycle and its properties in the case J~zJ21 < 0.
2. Numerical examination of "slaving" in a neural network
JkjRjtanhUk
]=n+l
(k : 1,2, 3 . . . . .
225
(1,3)
]-2
In a sense the neuron is "clothed" (renormalized in a field theory sense). If a self-term is included, ab initio, then (1.3) would be a correction [20]. T h e adiabatic elimination in this case both deterministic and stochastic has been carried out [19]. The ab initio self-term may be considered as an internal neuron feedback due to its own complex structure. This approach is not adopted in this paper. H e r e we will first, in section 2, examine numerically the scaling indicated by (1.2) and the time scale for the validity of the slaving assumed. In section 3, we examine the qualitative structure of the steady states of two effective neurons (n = 2) with the others slaved. The steady states for k = 1, N = 2 have in fact been examined by A n d r o n o v et al. [21] some time ago in con-
As discussed in [16], an adiabatic elimination procedure can be applied to the Hopfield model of neural networks. In this section we consider various computational measures of the accuracy of that approximation and the applicability of the basic concepts of H a k e n ' s procedure [17,18] to this system. We first consider the dynamics of the coupled non-linear equations for the network. Since the adiabatic elimination procedure hinges on the dominant time dependence of the dynamical variables, we present a measure of the exponential approach to steady state for the nodes in certain p a r a m e t e r regimes. We also u n d e r t a k e a numerical comparison of the solutions of a first order effective neuron to the solution of a single node in the coupled system. We provide a quantitative measure of the accuracy of the approximation for specific choices of the system parameters. Adiabatic elimination can be applied if time dependences of the dynamical variables are of different scales [17]. In the neural network, the dynamical variables are the individual neuron potentials U i. The m e m b r a n e resistance Ri can be seen to provide a measure of the time dependence of the equations of motion. In first order, our dynamical equations give an exponential time dependence U i = exp(-t/R~Ci). Hence, the resistances Ri are indicated as parameterizing the relative relaxation times of the neurons. E v e n at relatively low ratios of resistances (and hence the time constants), slaving is present. The accuracy of the first order effective
22~
Z. Zhaojue et al. / Two neuron dynamics and adiabatic elimination
neuron in simulating the dynamical behavior of a priviledged " m a s t e r " neuron is seen to depend on the p a r a m e t e r s of the network. In a limited example of eight neurons, we examined the time dependence of the potentials as a function of the ratio of m e m b r a n e resistances. Neuron 1 was taken to have unit resistance, with all other resistances varied together. All couplings were taken to be symmetrical and all cross-couplings to be equal. Fig. 1 shows a characteristic dynamical plot of the convergence of the neurons to their steady states. The upper trace is the slow variable Uj while the lower traces are the other potentials. This is a log plot of the absolute value of the potential. As is clear from fig. 1, the initial phase of the dynamics is nearly exponential. The final convergence is slower than exponential. Fig. 2 is a plot of the ratio of convergence times as a function of R~/ R / ~ . As the ratio R~/R,~j grows, the disparity between convergence times grows. The slope is seen to be 0.49. The time dependence of the network as shown in figs. 1 and 2 clearly make the system eligible for treatment with adiabatic elimination for
Log (101
Time
Fig. 1 The dynamics of the neuron potentials U, for eq. (1.1) for R~/R,+, = 4.0. log(IU~]) is plotted against time. The upper trace shows the convergence of U, :, the lower traces the behavior of the other neurons with varying initial values. Here, Jii = 2.6 and J / = 0.3 Vi, j.
i
U ~ - U :-:
5
f
RI/Ri
~]g. 2. Ratio o| convergence times for LT~ and L; ~ ~crsus R:,'I?,.~ as in fig. I. The slope is 0.49. Parameter values arc the same as those in fig. [.
reasonably small values of R~/R,¢~. Since the relative relaxation times are close to linear in the ratio of the resistances, a sufficiently large disparity in settling time can be obtained with a ratio of resistances of the same order of magnitude. In the presence of a fairly small variation of m e m b r a n e resistance between neurons, slaving takes place. If, in a many-neuron system, this kind of disparity of resistances is present, the dynamics of the whole network should hinge on the behavior of only the slower, higher resistance neurons. This is the essence of the effective neuron concept. The slaving of the fast neurons to the slow master neuron allows for an elimination of variables and the reduction of the dimensionality of the system of equations. These results suggest that a ratio of resistances less than ten is sufficient to achieve an effective slaving of the U,~: to U, = 1. We expect then to find reasonable correlation between the effective neuron's steady state and that of U , - 1 as computed from the coupled n-body system. Fig. 3 shows the ratio of the difference between U k_l (as calculated from eq. (1.2a) (n = 1, N = 8)) and U i = I (as calculated from eq. (1.1) with N - - 8 ) and U, = 1 versus the ratio R~/Ri~ ~.
Z. Zhaojue et al. / Two neuron dynamics and adiabatic elimination 7"
Here Ck = assumed Jk tanh U~ depending [16]. We
Ii
6"
5" ¢~1
x
4x
3"
2
x
x x
x x
x
4
6
8
10
12
227
14
R1/Rn
Fig. 3. Ratio of the difference between the steady state value of Uk 1 in eq. (1,2a) ( n = l , N = 8 ) and Uland U,~lin eq. (1.1) (N = 8) and U~_~v e r s u s RI/R,= I. The deviation of Uk 1 peaks at 38% of U,~ at R~/R,~ = 3.8 and falls rapidly. By R~/R,~ 2 = 10.0, the deviation is within 5%. Parameters are as in above figures. T h e difference between the steady state value of the effective neuron and U 1 is seen to peak at 38% of U 1 and to fall rapidly to less than 5% for R I / R ~ 1 = 10.0. For the specific p a r a m e t e r choices of fig. 3, the first order effective neuron is shown to be a good approximation of the behavior of the slow or " m a s t e r " neuron in the coupled 8 node system. As the ratio of the p a r a m e t e r s JH/J12 falls, the necessary ratio of resistances grows somewhat. For small or absent self-terms, a second or higher order effective neuron formulation is required as the first order a p p r o a c h fails. We have not yet developed such a higher order theory.
1 and R k = 1. Each effective neuron is to have a self-excitation term, (Jk > 0). The details of the form of Jk on Jkj (1 > n) does not concern us assume that the cross-connections Jkj (k ~ j) may be symmetric or asymmetric. Although symmetric (Jij = Jji) and antisymmetric (Jij = - J j i ) connections a m o n g neurons are easy to implement in an electronic circuit mimicking the biological neuron network, asymmetric connections a m o n g neurons may appear in the biological neuron network [22-24]. Now, we consider how the phase space flow of eq. (3.1) lead to additional attractors (far from the origin) by virtue of the self-excitation. First, we look at the flow near the origin. The relative rate of change of volume V in phase space, under the action of the flow (3.1), is given by the Lie derivative [25,26]
1 dV_ V dt
~ k=l
O(Jk_ ~ Ofk OUk k = l OUk"
(3.2)
We have from (3.1)
V dt
- 1 + Jk sech2Uk
t
,
(3.3)
and we conclude that if
•
Jk > n ,
(3.4)
k=l
3. Qualitative analysis of steady states of an effective neuron network
then the volume V of the flow (3.1) would dilate when [Ukl is small,
3.1. The flows in phase space
1 dV V d-'t- ) 0 "
Let us first consider an effective neuron network described by the deterministic equation /-)k = -- Uk + Jk tanh U k +
~
J~/tanh U/--= fk
j~k=l
(k=1,2,3,...
,n).
(3.1)
(3.5)
The origin is unstable, and the flow (3.1) in the vicinity of the origin will point out. But, it does not go to infinity. Secondly, let us look at the flow (3.1) near infinity. We may write (3.1) as
228
ukdUkdt -
Z . Z h a o j u e el al.
/ Two neuron dynamics and adiabatic elimination
U~ + JkUk tanh U k
+ 2
J,jU k tanh Uj
.
(3.6)
or negative. We will discuss the two cases of Ji2J21 > 0 and Jl2J21 < 0 . First we look for fixed points of cq. (3.8) which are solutions to the system
jck = 1
f,
Since Itanh U k ] < 1, then
,,( - U ~ + JkUk tanh Uk
d' L aT(, iU ~ ) : 2 k ~ = ,
+ ~ ]~k-
JkjU~ tanh Uj) 1
<-2<. k
(3.7)
I
T h e Liapounov function, ~k:~v" U-k, shows that the flow (3.1) from infinity will move into an n-dimensional hypershell. The two tendencies of the flow (3.1) illustrated by (3.5) and (3.7) show that the flow (3.1) will end at some forms of attractors located in the n-dimensional hypershell. Therefore, the appearance of a self-excitation term causes an effective neuron network to have additional attractors. They may be fixed points, limit cycles or tori, or possibly a strange attractor leading to chaos. A strange attractor, in fact, has been numerically found in a four effective neuron model. The details are given elsewhere [21]. In the following sections, we qualitatively analyse in detail these attractors for the case n = 2. H e r e , of course, a strange attractor is not possible.
3.2. A two effective neuron system ]'or Jt2J21 > 0
0,
]~:0.
(3.9a, b)
The solution to the transcendental system (3.9) cannot be easily represented as an analytical function of p a r a m e t e r s J~, J2, Jt2 and J21- As an alternative, we shall explore the qualitative features of curve (3.9a), illustrated in fig. 4. When J~-< 1, curve (3.9a) has no peak or valley. And when Jr > 1, curve (3.9a) has two antisymmetric peaks at (U I.... U~,,,) and ( - U I , , , U2,,). Here. U~,, and U2, , are determined by the solution to system
c°sh2 Ulm-
(3.10a)
JI "
[tanh U2,~] =
UI,, - J~ tanh U,, n
(3.10b)
J12
The larger J, and the smaller [J,zl, the higher the p e a k becomes. We emphasize that curve (3.9a) will divide into three discontinuous manifolds when eq. (3.10b) gives Itanh U2,,, I -> 1. For examU2
,
(3)
I
(3)
(1) --1
1(13)
~,
U~
As a specific detailed example, consider the two effective neuron system which is described by 01 - - U~ + J~ tanh U l + J12 tanh U~ = fl , 02 = - U, + J2 tanh U2 + J21 tanh UI = f2,
(3.8)
where the self-connection J1 > 0 and J2 > 0, and the cross-connections J12 and J2! may be positive
Fig. 4. The curve, U~ J, tanh U, + J~2tanh U:, in the three cases: curve (1) (J~ = 0 . 9 , J~,= t ) with no peak, curve (21 (J~ 1.4, J~2 = 1 ) with two peaks, and curve (3) (J, 1.5, J~, - ~ ) divides into three manifolds,
229
Z. Zhaojue et al. / Two neuron dynamics and adiabatic elimination
pie, when J ~ = l . 5 and 1J211->0.207 546, the curve (3.9) will divide into three discontinuous manifolds, shown by curve (3) in fig. 4. The asymptotes of the three manifolds of curve (3) in fig. 4 are perpendicular to the Ul-axis. The two asymptotes of the inner manifold of the curve (3.9a) approximately a r e U 1 = ~ IJ12l/(Jl 1), and the asymptotes of outer manifolds of curve (3.9a) approximately are U~ = ± (J~ + IJ12]) and U1 = ~ (J1 - ]J121)" When J~2 changes sign, c u r v e (3.9a) is reflected about the U~-axis. The features of curve (3.9b) are similar to those of curve (3.9a). By virtue of the two curves (3.9a) and (3.9b), it is easy to see there may exist one, three, five, or nine equilibrium points in the two effective neuron system with J12J21 > 0 or < 0. Fig. 5 shows the overlay of (3.9a) and (3.9b) for the case of one continuous and three discontinuous manifolds. Here the presence of nine fixed points is seen. The other possibilities are deduced similarly. For example, fig. 6 illustrates the case of nine fixed points in the two effective neuron system with J12J21 > 0. They are the origin, four nonzero stable fixed points (labeled in fig. 6 A 1, A 2 , A 3 , A 4 ) and four unstable saddle points (labeled B1, B 2, B3, B4). The stability of these
U~
151
r 2 n , ~
A,
I
T~a,
B2
-
U1
_
i
-1.
Fig. 6. Nine fixed points, A t , A2, A3, A4, BI, B2, B3, B 4 and O, in the model 6"~ = - U ~ + 1.4 tanh U~ + 0 . 2 t a n h U2; 02 = - U2 + 1.4 tanh/-]2 + 0.2 tanh U1. The A are stable and B unstable fixed points, TIA~, T2~ ~ indicate the tangents at point A~. Note the arrows, similarly for the other A, B points.
points are determined by the characteristic equation [20] - 1 + J~ s e c h 2 U 1 - A J21 sech2Ux
I
J12 s e c h 2 U 2 - 1 + J2 s e c h 2 U 2 - A = 0
(3.11)
Its roots are
U1
AI.2 = ½(a +- ~%-5-c2+ 4 b ) ,
(3.12)
where a =
2
.....
3
U2
J1 sech2Ul + J2 sech2U2
Fig. 5. Two solution curves intersect at nine points indicating the positions of the nine fixed points of the network. Here the parameters are J11 = 1.6, J12 = - 0 . 2 , J21 = -0.845, Jzz = 2.4.
2,
c = IJ1 s e c h Z U 1 - J2 sech2U2] , b = J12J21
,
-
sech2Ul sech2Uz.
Here, (UI, U2) are the coordinates of the fixed point, A or B type. Now, we develop a qualitative method for determining the stability of the fixed points without direct numerical calculation of (3.9) and (3.12). Let Tie be the tangent of curve (3.9a) at
Z. Zhaojue et al. ,; Two neuron dynamic.v and adiabatic elimination
23{I
a singular point, P, and Tee be the o t h e r tangent of curve (3.9b) at the s a m e point, shown, for instance, in fig. 6 at point A ~ or B t (labeled Tt~~, Te, , at point A.) T h e arrows indicate the two t a n g e n t s at A, and B~. F r o m eq. (3.9a), we have
Ji2Tlll sechZU.
1 - . I I sech2Ul .
For a m o r e detailed analysis of the two effective n e u r o n n e t w o r k s with J12J21 2> 0, we plot a bifurcation d i a g r a m fig. 7 which d e p e n d s only on p a r a m e t e r s Jz, JÈ and the product J12J21. It is d r a w n by analysing the stability of the origin (U t = U, = 0) given by the two roots a~. 2, where
(3.13) A,.2 = ~ [ ( J t + J+
A n d , f r o m eq. (3.9b). we have
J21T2t" sech2Ui
1
.I, secheU~ .
(Jt2J:t (3.14)
T h e n , the p a r a m e t e r s a and c in (3.12) can be r e p r e s e n t e d by the two tangents at the singular point. a = -(J12Tt/,
secheU~ + Jel T2t, secheUi ) ,
(3.15) ," = IJ,eTjtl s e c h : U ,
J21T2,, sccheU, I •
(3. lO)
First, at a saddle point such as point B~ in fig. 6, if the two tangents, T.~, and T2B I, have opposite signs, then ]a] < c. C o n s i d e r the case when the c o o r d i n a t e s of BI, U ~ , is much larger than Uez~l, then 4b < c z. T h e n , one root of (3.12) is positive, and the o t h e r negative. This m e a n s that point B~ is unstable, a saddle point. If on the o t h e r hand, at a fixed point such as A ~ in fig. 6, the two tangents have s a m e signs, then ]a[ > c. C o n s i d e r a point A~ which lies far from the U~and U,-axis, then, J~2 secheU: ~ 0. Thus, the two roots of (3.12) are negative, or A~ is a stable fixed point. Finally, we conclude that a fixed point is a saddle point if it lies at least on an inner manifold of curve (3.9a) or (3.9b), and a stable fixed point if it lies at the intersection of the two o u t e r m a n i f o l d s of (3.9a) and (3.9b). In the light of this, A ~ , A 2 , A ~ , A 4 in fig. 6 are stable fixed points, and B~, B e, B~,, B a are saddle points. T h e qualitative m e t h o d for determining the stability is easily e x t e n d e d to those cases of five and three fixed points when J12J21 > 0, and to the case of J12J2~ < 0 discussed in the s u b s e q u e n t section.
2) ++-~/( J, - J2)2 + 4J12J2,]
(3.17)
:> [1) .
In region 1 of fig. 7, the origin is the unique stable fixed point. In region 2, the origin is a saddle point and there also exist two non-zero stable fixed points. In region 3, the origin is an unstable node and there m a y now exist four or eight n o n - z e r o fixed points, half of t h e m arc stable, half are saddle points. This is p r o v e d by the P o i n c a r d - B e n d i x s o n t h e o r e m [25,20] (see a p p e n d i x A). In fig. 7, the b o u n d a r y curve s e p a r a t i n g the two regions of four and two fixed points could not be shown as a curve, because it is sensitive to all four p a r a m e t e r s J , / o f eq. (3.8). For m o r e details a b o u t this situation, we show a n o t h e r bifurcation d i a g r a m , fig. 8 (see appendix B). T h e two cases of nine and five fixed points
J2
4~
Z
3
I
I--
Fig. 7. The bifurcation graph with ./~. J, and J~:J:~ =/I.2. The boundary curves are (J~ I ) ( J : - 1) = Jt,J~l. The two roots of (3.17) for the origin arc, A ~ : < 0 in region 1, A , - 0 and A , < I I in region 2 and A ~ : - t l i n region 3.
Z. Zhaolue et al. / Two neuron dynamics and adiabatic elimination
231
&
J~2l
4
2..5
3.5"
2
4(I)
1.5
?
3 2.
1
3
0.5 0.5
z
1.s
2
2.5
3
3.5
~
J1
Fig. 8. The bifurcation graph with J1 and IJlZl for the case Jt - J2 and Ji2Jz~. In region 1, the origin is the unique stable node. In region 2, the origin is a saddle point and there are two non-zero stable nodes. In region 3 and 4, the origin is an unstable node. Two stable nodes and two saddle points are in region 3, and four stable nodes and four saddle points are in region 4.
do not appear in the usual Hopfield model of two neurons. Finally, there is no limit cycle when J12J?l > 0 in the two effective neuron system. This is the same as the Hopfield model of two neurons (see appendix A).
0.5
1.s
~
215
3'.5
~
J~
Fig. 9. The bifurcation graph with J1, Je and Ji2J, t = -0.25. The two roots of (3.18) for the origin are, Re A~,2<0 in regions 1 and 2, Re A1,2> 0 in regions 4 and 5, At > 0 and A2 < 0 in region 3. The dashed lines are J~ = 1.968 and J2 = 1.968. For JlzJ2~ < 0 parameter region 5(3) contains no stable fixed points and thus a limited cycle may exist, as discussed in section 4 of the text.
Jl + J2 - 2 = 0 , [ J 1 - J21 = 2V;IJlzJ2~] ,
3.3. A two effective neuron system f o r JleJel < 0
(J1
In light of the analysis of section 3.2, the curves (3.9a) and (3.9b) in a two effective neuron network with J12J21 < 0 may divide into three manifolds for sufficiently large J~ and J2. As discussed in section 3.2, there may exist one, three, five, or nine fixed points in the system, The cataloging of these fixed points depends on the stability of the origin (U~ = U 2 = 0) given by the two roots
These curves depend only on J1, J2, and the p r o d u c t J12J21, three of the four parameters in the flow (3.8). The origin (U 1 = U2 = 0) is a stable focus in region 1 of fig. 9, and a stable node in region 2. Then, in the two regions, the origin is the unique stable fixed point and there is no saddle point. In region 3, the origin becomes a saddle point, and there also exist two stable fixed points far from the origin. In region 4, the origin is an unstable node, and there may exist four or eight non-zero fixed points in subregions 4(2) or 4(1). The characters of these points in regions 2, 3, and 4 are the same as that when J12J21 2> 0. In region 5 of fig. 9, the origin may be an unstable focus (not attracting). Thus, there exists eight non-zero fixed points in subregion 5(1), four non-zero fixed points in subregion 5(2), and no stable fixed point but rather a stable limit cycle in
A ,2 = ½[(J1 + J2 - 2) -+
- J 2 ) 2 - 41J12J2111. (3.18)
By the analysis of the two roots of (3.18), a bifurcation diagram, fig. 9, is drawn to show the character of the other stable fixed points and in this case limit cycle when J12J21 < O. The curves separating the regions of fig. 9 are
q- J 2 - -
2) 2 = (J~ - J2) e - 41J12J2,].
(3.19)
232
Z. Zhaojue et al. I Two neuron dynamics' arid adiabauc elirnination
subregion 5(3). As mentioned in appendix A, half of these non-zero fixed points in region 5 (and 4) are stable fixed points and the other half are saddle points. The three cases in subregions 5(1), 5(2) and 5(3), do not appear when Ji2Jel > 0. In the general case, the boundary curves separating the two cases of five and nine fixed points in the regions 4 and 5 of fig. 9 cannot be shown simply as fixed curves. However, when J12J21 are constant, these curves can be seen in fig. 9. For a given Jl2 (or J21 )' we can obtain a critical J~(Jl2) (or JC(J2t)) by means of eqs. (3.10). For example, if J~2 = 0 . 5 , then J~(0.5) 1.968. When (3.20a)
J, >>-J ~ ( J l 2 ) ,
and (3.20b)
J~ > J~(J,_~ ) .
The curves (3.9a) and (3.9b) will divide into three manifolds. In fig. 9, we assumed J ~ 2 J 2 i 0.5, then the boundary curves are J~ = 1.968 and J, - 1.968 which are shown as dashed lines. They divide region 4 into two subregions (4(1) and 4(2)), and region 5 into three subregions (5(1), 5(2) and 5(3)). With fixed J~ (or Jx), we may obtain another critical J~2(J1) (or JCl(J2) ) by means of eqs. (3.10). For example, i f J 1 = 2, then J~2(2) --0.54. If IJ~2i<<-J~2(J1) (or IJ2,l<-J2i(J2)), then the curve (3.9a) (or (3.9b)) will divide into three manifolds. But in this case, the two boundary curves cannot be drawn in fig. 9. As an example, fig. 10 shows the case of nine fixed points appearing in the subregion 5(1) of fig. 9. In light of the analysis of section 3.2, it is easy to show that A I , A 2 , A 3 , A 4 are stable points, and BI, B 2, B3, B 4 are saddle points. In fig. 10, the four inner manifolds of curves (3.9a) and (3.9b) are the approximate boundaries of the basins of attraction of these fixed points because they pass through saddle points, and are c
c
U2
Fig. 10. Nine singular t~oints, A~, A:, A~, A 4, B,, B e. B~, B, and O, in the model UL U~ + 1.6 tanh U, + (I.25 tanh U,: /~), U, k 1.6 ianh ~,"~ (I.25 tanh {"~. nearly straight lines. Therefore, these basins of attractions are separable, and the stable fixed points, A ~, A 2, A ~ , A 4 correspond to four memory states. This has also been numerically observed. Finally, let us draw some conclusions concerning the two effective neuron system with Jt~J21 < 0. First, more fixed points and more saddle points a p p e a r in the system with the selfconnections Jl and J2 increases. In contrast to the Hopfield model of two neurons, when Jl2Jzl < 0 there is no other attractor except the origin. The a p p e a r a n c e of a limit cycle in the two effective neuron system with J12J2~ < 0 is the main qualitative difference between the two cases of Ji2J2t :%0.
4. The limit cycle (J~zJ2~ < O) In an effective neuron network, the appearance of limit cycles of periodic solutions are also possible. First, we obtain the sufficient condition for the appearance of a limit cycle when J~2J2~ < 0. In the light of section 3.1, if the origin (U~ = U z = 0) is unstable and there is no non-zero stable fixed point, then the flow of (3.8) must
233
Z. Zhaojue et al. / Two neuron dynamics and adiabatic elimination
end at a limit cycle. T h e bifurcation d i a g r a m fig. 9 has shown there is a subregion 5(3) in the p a r a m e t e r space, for which the origin is an unstable focus and no n o n - z e r o stable fixed point exists. A s a result, a stable limit cycle must appear. Secondly, we want to show that the limit cycle will n o t a p p e a r w h e n there exist n o n - z e r o stable fixed points in the system. Consider the Poincar6-Bendixson t h e o r e m [21,26]. T h e limit cycle, if it exists, must enclose the origin or all stable fixed points. This is impossible. If there were a limit cycle and stable fixed points, it w o u l d intersect with the phase curves linking the origin (as a saddle point) to infinity, or that linking a n o t h e r saddle point to infinity, or to stable nodes and the origin (as an unstable node or focus). F o r e x a m p l e , in fig. 10, if there were a limit cycle, then it intersects with these phase curves linking saddle points B 1, B2, B3, B 4 to infinity, or stable fixed points A , , A2, A 3, A 4 and the origin. T h e r e f o r e , the condition for the a p p e a r a n c e of a limit cycle in the two effective n e u r o n system with J 1 2 J 2 1 < 0 is that the origin is an unstable focus and there is no stable fixed point. L e t us n o w construct an a p p r o x i m a t e solution of the limit cycle. We focus on the s y m m e t r i c case for simplicity. We write eq. (3.8) as /]]" =
- - U 1 ~-
/f12 = ((1!('02 -- 1)U2 -- /3031
--U2
-[-
dp 2
dt
dO
= 2[(w, + 032)a - 2lp 2 ,
d-7
(4.4)
Here p
2
2
2
= O 1 nt- U 2 ,
U1
0 -- a r c t a n --.
U2
T h e n , the solution to (4.5) is P = P0 exp[2(03a - 1 ) t ] ,
(4.5a)
0 = 00 + / 3 f 03 d t .
(4.5b)
0
If c~03> 1, p will increase with time. With the increase of p, w, and w e will decrease until a critical value, % , w h e r e am c - 1 = 0. T h e n p = Pe and does not increase. Pc is the radius of the limit cycle. O n the o t h e r hand, f o r ' l a r g e IU,[ and IU21 (or p) then a03 < 1, eq. (4.5a) shows that p will d e c r e a s e with time until p = Pc. This m e a n s that the solution (4.5) shows a stable limit cycle. T h e a m p l i t u d e Pc and frequency /3wc of the limit cycle can be obtained. First, consider a point, U, = U 2 on the limit cycle, then
13:'tanh U, + / 3 tanh U 2 , O/ tanh U 2 - / 3 tanh U, .
(4.3)
or in polar coordinates,
tanh Pc 601 =
[ff2 =
U1,
(02 >
-
-
,
Pc
(4.1)
(4.6)
and First introduce 03, = t a n h U~/U, and 032 = tanh U 2 / U 2. F o r IU, I ~ 1 and lull ~ 1, 03, and 032 are nearly equal to one. H o w e v e r , we will ass u m e that 03, and 032 are constant as a first a p p r o x i m a t i o n because they vary m o r e slowly t h a n tanh U, and tanh U 2. A s s u m e 601 ~
032 ~
03 "
(4.2)
t a n h Pc _< _1
(4.7)
Pc
T h e lower limit of & is o b t a i n e d by solving (4.7). F o r e x a m p l e , if a = 1.05, then Pc > 0.39. Second, consider a point (U I = 0 and U 2 = Pc) on the limit cycle, or 03, = 1 and 032 = tanh pc~pc. Thus we have
T h u s eq. (4.1) is
U, = (a03,-
1 ) U 1 +/3032 U2,
tanh Pc Pc
-
>
2 a
-
-
1.
(4.8)
234
Z. Zhao]ue et al. / Two neuron dynamics and adiabatic elimination
U2
i~ U1
Fig. 11. A limil for /~7,- U l + I . O 5 t a n h U ! + O . 5 t a n h U , : U, [,L + 1.115tanh/_7~ 0.5 tanh U~.
For c~ - 1.05, then p< < 0 . 4 7 . Computer simulation of the case of ~ = 1.05 and /3 =11.5 illustrated in fig. ll, shows that there is a stable limit cycle with radius p~.- 0.45. It agrees well with above estimates of p~.. In the asymmetric cases when d~ ¢ J, a n d / o r J i 2 . l ~ l , the limit cycle is not a circle but deformed into an ellipse around the origin.
5. Conclusions In the preceding sections, we have considered a network of two effective neurons. Our analysis suggests that the effective neuron formulation is not only a valid approximation of the larger dimensional Hopfield network, but also that dynamics of considerable complexity ensues in even this low dimensional system. Further, we have constructed explicit bifurcation diagrams for the two neuron network and demonstrated analytically the stability structure inherent in the system. Finally, we consider the onset and behavior of the limit cycle solution and apply the Poincar6-Bendixson theorem to show that limit cycles and fixed points will not coexist in the two dimensional case. Our intent here has been to present a thorough inspection of the simplest multiple effective neuron network both as an interesting dynamical system and as a paradigm for more
complicated, higher dimensional nets. As a n a~sociative memory, this network is capable of storing from zero to four memories as steady states or a single, periodic memory. In a future report, we will discuss the capacities of higher dimension networks and the implications of the periodic solutions. As is clear from the bifurcation diagrams presented here, qualitative changes in the dynamics are produced by movement through the space of the connection strengths. The bifurcations are then seen as resulting in the creation and annihilation of fixed points, of emergence of a limit cycle (and, as previously reported, emergence of chaos in larger dimensional networks). The dynamical systems approach taken here enables us to glean precise information regarding the behavior of the network while investigating a greatly reduced number of effective neurons. In generality, our results pertain to all self-connected Hopfield networks (effective or otherwise). While the parameter spaces grow rapidly for larger networks and exhaustive construction of bifurcation diagrams becomes more difficult, the two neuron case can be seen as a step toward and a module in such a project. In forthcoming reports, we consider the dynamics of systems of three and four neurons.
Acknowledgements A first draft of this work was completed while one of us (W.C.S.) was recipient of a summer Senior Humboldt Foundation Award at the MaxPlanck Institute for Quantumoptics, Garching, Germany. Z.Z. and P.K.D. acknowledge support of the R A Welch Foundation of Texas.
Appendix A In the light of the curves (3.9a) and (3.9b), there may exist one, three, five, or nine fixed points in the two effective neuron system gov-
z. Zhaojue et al. / Two neuron dynamics and adiabatic elimination
erned by eq. (3.8). The origin (U 1 = U 2 = 0 ) is the unique fixed point which is always present. Most of the characteristics of the other fixed points or the limit cycle of the system are obtained by the character of the origin and the P o i n c a r 6 - B e n d i x s o n theorem [20,25]. Consider the flow in (3.8). Infinity is absolutely unstable. Its Poincar6 index is +1, then the sum of the Poincar6 indexes of all fixed points of eq. (3.8) must equal +1. First, if there are nine (or five') fixed points and the origin is an unstable node (the Poincar6 index is +1), then half of the eight (or four) non-zero fixed points are stable fixed points, and the other half are saddle points (the Poincard index saddle point is - 1 ) . They are shown in region 3 of fig. 7 and in regions 4 and 5 of fig. 9. Now, if the origin is a saddle point, then there must exist two non-zero stable fixed points. They are shown in region 2 of fig. 7 and in region 3 of fig. 9. There always exist stable fixed points when JzzJz~ > 0. As a result, there is no limit cycle in these systems. If it were to exist, it would intersect the phase curves linking infinity to unstable fixed points, and linking saddle points to stable fixed points.
Appendix B In fig. 8, the curve separating the region of five fixed points and the region of nine fixed points (with J12J21 > 0) is x = ~/yY ~ _i In(v'-y + ~¢/-y- i ) .
(A.I)
Here X :
J| --IJ12[ ,
y = Jl + IJl=l •
In the case of J1 = J2 and J12 = J21 ~> 0, if there exist nine fixed points, then there are four stable nodes, shown in fig. 6 as the symmetric couple of A 1 and A 3 and the asymmetric couple of A 2 ( - U a, Ua) and A 4 ( U a, - U , ) . From eq. (3.8),
235
U,( > 0) is obtained by Oa = (J1 -
[J121) tanh U ~ .
(A.2)
In the light of (A.1), if the two asymmetric points are present, they must be stable fixed points. As a result, the appearance of the unsymmetric stable fixed points marks the appearance of the case of nine fixed points. Consider the characteristic equation (3.12), two roots for the point A 2 or A 4 are h l , 2 ~" --1 + J~ sec h 2 Ua + [JI21 sech 2 U~.
(A.3)
When (J1 +
IJ~2])secheua =
I ,
(A.4)
it gives the condition for the appearance of the case of the nine points. The fixed points A 2 and A 4 are stable fixed points. Eq. (A.1) is obtained from eq. (A.2) and (A.4) by eliminating U~.
References [1] J. Hopfield, Proc. Natl. Sci. USA 79 (1982) 2554. [2] J. Hopfield, Proc. Natl. Sci. USA 81 (1984) 3088. [3] D. Amit, Modeling Brain Function (Cambridge University, New York, 1989). [4] S. Shamma, in: Methods in Neuronal Modeling, eds. C. Koch and I. Seger (MIT Press, Cambridge, MA, 1989). [5] B. Muller and J. Reinhardt, Neural Networks, an Introduction (Springer, Berlin, 1990). [6] H. Haken, Synergetic Computers and Cognition (Springer, Berlin, 1990). [7] F. Hoppenstaedt, An Introduction to the Mathematics of Neurons (Cambridge Univ. Press, London, 1989). [8] J. Buhmann and K. Schulten, Biol. Cybern. 54 (1986) 319. [9] D. Amit, H. Guttfreund and H. Sompolinsky, Phys. Rev. A 32 (1985) 1007. [10] H. Sompolinsky, Phys. Today 41 (1988) 70. [11] D. Amit, H. Guttfreund and H. Sompolinsky, Ann. Phys. (NY) 173 (1987) 30. [12] P. Peretto, Biol. Cybern. 50 (1984) 51. [13] K. Babcock and R. Westervelt, Physica D 28 (1987) 305. [14] Z. Li and L. Hopfield, Biol. Cybern. 61 (1989) 379. [15] C. Skarda and W. Freeman, Behav. Brain Sci. 10 (1987) 161.
236
Z. Zhaojue et al. / Two neuron dynamics and adiabatic' elimination
[16] W.C. Schieve, A.R. Bulsara and G.M, Davis, Phys. Rev. A 43 (1991) 2613. [17] H. Haken, Synergetics (Springer, Berlin, 1977). [18] H. Haken, Advanced Synergetics (Springer, Berlin, 1983). [19] A. Bulsara and W.C. Schieve, Phys. Rev. A 44 (I991) 7913. [2/I] A. Andronov, A. gitt and S. Khaikin, Theory of Oscillations (Dover, New York, 1966).
[21] P. Das, W.C. Schieve and Zeng Zhajuc, Phys. Lett. A 161 (1991) 60. [22[ J. Denker, Physica D 22 (1986) 216. [23] B. Baird, Physica D 22 (1986) 150. [24] B. Baird, Physica D 42 (1991)) 365. [25] F. Verhulst, Nonlinear Differential Equations and Dynamic Systems (Springer. Berlin, 1980). [26] P. Berge, Y. Pomeau and C. Vidal, Order within Chaos (Wiley, New York, 1987).