Learning rules for an oscillator network

Physics LettersA 174 (1993) 289-292 North-Holland

PHYSICS L E T T E R S A

Learning rules for an oscillator network H i d e t s u g u Sakaguchi Department of Physics, College of General Education, Kyushu University, Fukuoka 81 O, Japan Received 7 October 1992; revised manuscript received 11 December 1992; acceptedfor publication 18 December 1992 Communicatedby A.R. Bishop

Two learning rules are studied for a simple oscillator network. One is a Boltzmann-machine-typeof supervised learning rule and the other is a generalized Hebbian rule for the self-organizationof connections. Synchronization is one of the peculiar phenomena observed in coupled oscillator systems [ 1,2]. Recently Eckhorn et al. and Gray and Singer found that neurons in the primary visual cortex of a cat can exhibit oscillatory responses and that the oscillation is synchronized over relatively large distances [ 3,4]. Sompolinsky et al. used a smooth phase oscillator or an X Y spin as a dynamical unit and analyzed the phase coherence of the oscillation [ 5 ]. Chawanya et al. used an integrate-and-fire-type element and studied mutual synchronization among the neurons [ 6 ]. Abbott used a Fitzhugh-Nagumo type element and proposed phase locking memories [7]. Synaptic connections are fixed in their oscillator networks. However, modification of synaptic connections is characteristic of a neural system. We want to consider a learning process in an oscillator network and study the interplay of mutual synchronization and learning. In this Letter we use a simple phase oscillator model and propose simple learning rules, since such phase models are expected to retain the essence of the interplay of learning and synchronization and are mathematically tractable. A different kind of learning rule that may occur in a network of fireflies is discussed by Ermentrout [ 8 ]. We assume that each oscillator can be described with its phase and the phase oscillators obey the coupled phase equation [ 2,5,9 ] d~i d-T = to~ - ~ W~j sin ( g), - q)j) + ~ ( t ) ,

dc/~ dt - -

OH 0-~ + ~ ( t ) ,

i= l, ..., N ,

(2)

where H = - E~j W~j cos ( ¥~- C/j). Equation (2) is the Langevin equation for C/~. The equilibrium distribution P_ ((C/,-)) is written as e--H/r P - ({C/")) = f2o~ . . . f ~ d C / l ...dC/~e - z / r "

(3)

This equilibrium distribution is equivalent to the equilibrium distribution of the X Y spin system with temperature T. The connections W,-j can be modified slowly by a learning rule in a neural system. We propose a learning rule which is a simple extension of the Boltzmann machine learning [ 10,11 ]. In the Boltzmann machine learning the connections are changed as the Kullback divergence is decreased. The Kullbach divergence G is defined as 2n

2n

G= ~...~dC/,...dC/jvP+((C/i))In(P+/P_),(4)

J

i = 1,...,N,

where ~ is the phase of the ith oscillator, to~ is its natural frequency, W~j represents the synaptic strength between the ith and jth oscillators, and ~ represents noise. The noise is taken to be Gaussian white noise with ( ~ i ( t ) ~ ( t ' ) ) = 2 T ~ i f l ( t - t ' ) . The connection is assumed to be symmetric: Wij= Wj.~, and Wi,~= 0. I f each oscillator has a constant natural frequency too, ¥~= ~J-toot obeys the coupled equations

(1)

O

O

0375-9601/93/$ 06.00 © 1993 Elsevier Science Publishers B.V. All fights reserved.

289

Volume 174, number 4

PHYSICS LETTERS A

where P+ is a desirable probability distribution and P_ is the equilibrium distribution (3). The Kullback divergence G is generally positive and G = 0 only when P+ = P_. It represents a distance between P+ and P_. When the connections are changed as the Kullback divergence is decreased, the equilibrium distribution P_ is expected to approach the desirable distribution P+. The learning rule is written as

dW~,j OG r dt =--Tow~,j 2x

=T 0 2n

dy,...dyNp

({y./),

0

W~a = - 0.11

a wi,j

2~

= f ... f d y l . . . d y N C O S ( y i - Y j ) o o × [P+ ({Y~})-P_ ({y~}) ]

=+-_,

(5)

where z is the time constant for the learning and < > + and < > _ represent respectively the average with respect to P+ and P_. In the original Boltzmann machine learning cos ( y~- YJ) is replaced by S:g where s~ and sj represent the firing state of the ith and jth neuron and take the value + 1 or - 1. The quantity cos(y~-yg) describes the phase coherence between the ith andjth oscillators and it is a quantity similar to s~sjin the Ising system, since sisj also describes the spin correlation between the ith andjth spins. Equation (5) says that the connections are modified as the average phase coherence _ approaches the desirable phase coherence < c0s ( y ~ - yj) > +. In the original Boltzmann machine each spin variable can take only two values, + 1 or - 1. But in our oscillator network each oscillator takes a continous value between 0 and 2n. We can therefore approximate the probability distribution P_ to a larger class of the probability distribution P+ with our oscillator network. As an example we have carried out a numerical simulation of eqs. ( 1 ) and (5). We consider a system of six oscillators and the desirable phase coherences < c o s ( y ~ - y j ) > + are assumed to be + = < c o s ( y 3 - y , ) > + : + =0.95

290

and < c o s ( y ~ - y j ) > = - 0 . 4 for the other pairs. Namely, the six oscillators are classified into three groups: ( 1, 2), (3, 4) and (5, 6); the pair in each group is attractive and the three groups are repulsive. The temperature T is 0.1 and the thermal average < c o s ( y ~ - y j ) ) _ is replaced by a long time average of cos [ Yi(t) - yg(t) ]. Initially all connections W~j are assumed to be 0. The result of the simulation is W1, 2 = W3, 4 = W5, 6 =0.87,

2~x

...

15 March 1993

for the other pairs.

(6)

As is expected the connections in each group are positive and the connections between two groups are negative. Figure 1 shows an equilibrium configuration of the six oscillators in the plane (x, y ) = (cos(y), s i n ( y ) ) for the above values of the connections and T = 0. Two oscillators in each group are completely synchronized and each group is -~n away from the other two groups. The above learning rule is a kind of supervised learning rule, since a desirable phase configuration is given to the system. Next we propose an unsupervised learning rule. We consider a system of N oscillators (1) where the natural frequencies are generally different. As a learning rule we consider a generalized Hebbian rule:

dWi,j z dt = - D W , j+g(Oi-Oj).

(7)

The first term represents a simple damping term and the connection is strengthened by the second term.

34 6 o

12 I

i

1

O0

-T O. O0

J

I 1.00

X

Fig. 1. Equilibrium configuration for the six oscillators with the connections (6) and T = 0.

Volume 174, number 4

PHYSICSLETTERSA

15 March 1993

In the neural network composed of the Ising spin si, like the Hopfield model, the Hebbian rule is written as d W~,j/dt ocs~sj [12]. As a simple generalization of the Hebbian rule we assume g ( ~ t - ¢j) =cos(Oi-Oj), since c o s ( ~ ; - ¢ j ) represents a similar correlation to s~sj in the Ising system. Since the time constant z is very large, cos ( ~ - ~j-) can be replaced by its average

0

(cos(¢~-~j))_. We consider at first a system of two oscillators and study theoretically the generalized Hebbian learning rule. The phase difference ~ u = ~ - ~ 2 obeys d~/ dt - Aco--2Wsin(¢,) + ~ ( t ) ,

(8)

~;o. oo

'

0. 2 0

'

o.',o

'

o.',o

'

o.',o

'

~.'oo

W

Fig. 2. Phase coherence (cos(~=-~2))_ for Ato=0, 0.2 and T= 0.1 as a functionof W.

where Aco=col--092, W= I4II,2 and ~=~l-~2, which satisfies ( ~ ( t ) ~ ( t ' ) ) = 4 T J ( t - t'). The probability distribution P_ (~u) obeys

OP_ O Ot - -- ~ { [Aog-EWsin(~,) ]P_}

7.

+ 2 T o2P-

(9)

0~¢2 "

The stationary solution of eq. (9) satisfying the periodic boundary condition P (~) = P (~u+ 2n) is given by P - (~u) = e x p ( - - 2 W + Aco¢¢+ ~-~ 2 W cos ( ¥ ) ) r n (0)

X ( I + [exp( - 2nAog/2T) - 1 ] X f~ d~'exp{ [ -Ato~u- 2 Wcos(~u) ]/2T} fo2x d~ exp{ [ - A t o ¥ - 2 Wcos(~,) ] / 2 T } ] '

(10) where P _ ( 0 ) is determined by the normalization condition fo2x d y P _ (~) = 1. Substitution of (10) into eq. (7) yields a learning equation for W:

dW

z--~ =-DW+

2x

j- d ~ c o s ( ~ ) P _ ( ~ ) .

(11)

0

Figure 2 shows the second term of (11 ), i.e., c = ( c o s ( ~ , ) ) _ as a function of W for T=0.1, and A~o=0 and 0.2. The stationary solutions ofeq. ( 11 ) are obtained as the intersection of the curve of fig. 2 and the straight line c=DW. If the damping con-

0. 00

t. 00

2, 00

3. 00

4. O0

5. 00

D

Fig. 3. Stationary solutions o f the learning equation ( i 1 ) for Ato=0 and 0.2.

stant D is large, the only solution is c = W = 0 . Namely, the connection cannot be generated. But if D is smaller than a critical value, the zero solution becomes unstable and a nontrivial connection appears. The phase coherence appears owing to the connection and the connection is strengthened by the generalized Hebbian rule. This positive feedback is the origin of the self-organization of the mutual connection. If W is small enough, the phase coherence c= (cos(~u))_ can be expanded in powers of W / T and then the stationary solution W satisfies the equation

DW=

2T

AO)2+4T2

1 W - -4-T-~ gW

3

(12)

where g = 4 / ( A 0 9 2 + 16T 2) --8TAto/(Ao)2+4T2) 2. The critical damping constant Dc = 2 T~ (Ato 2 + 4 T 2 ) and when g is positive (negative), the transition is supercritical (subcritical). Figure 3 shows the stationary solutions of eq. ( I 1 ) as a function of D for 291

Volume 174, number 4

PHYSICS LETTERS A

T = 0 . 1 a n d Ao9=0 a n d 0.2. W h e n Aog=0, the transition occurs at D = 5 a n d it is supercritical. On the other h a n d when Ao9=0.2, the transition occurs at D = 2.5 and it is subcritical. Between D = 2.5 a n d 2.7 the system is bistable. The critical d a m p i n g constant Dc becomes small when the difference from the natural frequency is large. We have carried out a numerical simulation o f six oscillators. T h e natural frequencies are o91= 1.1, 092=0.5, o93=0.1, o 9 4 = - 0 . 1 , o 9 5 = - 0 . 5 a n d 096= - 1.1. Initially W~j=0.5, D = 1.5 a n d T = 0 . 1 a n d the d a m p i n g constant D is increased gradually. Figure 4 shows the connections WL,2, W2,3 a n d W3.4 as a function o l D . At D = 1.5 all connections are not zero. The critical d a m p i n g constant Dc for the self-organization is smaller for the larger frequency difference. So at first the connections Wt,2 a n d I415,6 b e c o m e zero

15 March 1993

discontinuously at D ~ 1.85 a n d the first and sixth oscillators are isolated from the oscillator network. Next, the second and fifth oscillator are isolated from the oscillator network at D ~ 2.15 and finally all connections become zero at D ~ 2.75. In s u m m a r y we have investigated two learning rules for the connections o f a coupled oscillator system. One is a simple extension o f the Boltzmann machine learning. A desirable phase coherence can be realized by the learning rule. The learning rule can be easily extended to include higher h a r m o n i c interaction like W,j sin [ n ( ~ - ~j) ]. The other is a generalized H e b b i a n rule, for which the connections can be self-organized in a certain p a r a m e t e r region. The self-organization o f connections is easier for the oscillator pair whose frequency difference is small. The models considered in this Letter seem to be too simple and we would like to study m o r e biologically realistic models in the future.

References

,=::;

'

t

40

i.

80

I

/ ~ 2. 20

I

' 2, 50

3.00

D

Fig. 4. Connection strength W~.2, 1412.3and W3.4as the damping constant D is gradually increased. The natural frequencies of the six oscillators are cot=l.l, co2=0.5, ca3=0.1, co4=-0.1, o~n= -0.5 and co6= - 1.1.

292

[ 1] A.T. Winfree, The geometry of biological time (Springer, Berlin, 1980). [ 2 ] Y. Kuramoto, Chemical oscillation, waves, and turbulence (Springer, Berlin, 1984). [3] R. Eckhorn, R. Bauer, W. Jordan, M. Brosch, W. Kruse, M. Munk and R.J. Reitboeck, Biol. Cybern. 60 ( 1988 ) 12 I. [4 ] C.M. Gray and W. Singer, Proc. Natl. Acad. Sci. 86 ( 1988 ) 1698. [ 5 ] H. Sompolinsky, D. Golomb and D. Kleinfeld, Phys. Rev. A43 (1991) 6990. [ 6 ] T. Chawanya, T. Aoyagi, I. Nishikawa, K. Okuda and Y. Kuramoto, to appear in Biol. Cybern. [7] L.F. Abbott, J. Phys. A 23 (1990) 3835. [8] B. Ermentrout, J. Math. Biol. 29 ( 1991 ) 571. [9] H. Sakaguchi, Prog. Theor. Phys. 79 (1988) 39. [ 10] D.H. Ackley, G.E. Hinton and T.J. Sejnowski, Cogn. Sci. 9 (1985) 147. [ 11 ] H. Sakaguchi, Prog. Theor. Phys. 83 (1990) 693. [ 12 ] J.J. Hopfield, Proc. Natl. Acad. Sci. 79 ( 1982 ) 2254.