- Email: [email protected]
Stabilization of metastable dynamical rotating waves in a ring of unidirectionally coupled nonmonotonic neurons
Neurocomputing 225 (2017) 148–156
Contents lists available at ScienceDirect
Neurocomputing journal homepage: www.elsevier.com/locate/neucom
Stabilization of metastable dynamical rotating waves in a ring of unidirectionally coupled nonmonotonic neurons
MARK
Yo Horikawa Faculty of Engineering, Kagawa University, Takamatsu 761-0396, Japan
A R T I C L E I N F O
A BS T RAC T
Communicated by Y. Liu
Effects of nonmonotonicity in a sigmoidal output function of neurons on rotating waves in a ring of unidirectionally coupled neurons are studied. It is known that transient rotating waves show dynamical metastability when the output of neurons is monotonic; their duration increases exponentially with the number of neurons. Stable rotating waves of three types are generated as the nonmonotonicity in the output function of neurons increases. The type of stable rotating wave depends on whether the number of neurons is even or odd. Kinematical description for the propagation of wave fronts shows that an overshoot of the state of a neuron after the passage of the wave front due to the nonmonotonic output of a neuron causes the stabilization of rotating waves.
Keywords: Ring neural network Metastable dynamics Rotating wave Nonmonotonic output
1. Introduction A ring of unidirectionally coupled sigmoidal neurons has been studied from several points of view although its structure is simple. It has been shown that a ring of unidirectionally coupled neurons causes a stable oscillation when the number of inhibitory couplings is odd [1]. Such a ring is equivalent to a ring oscillator, which is a closed loop of an odd number of inverters and is widely used as one of variablefrequency oscillators in electronic circuits. Mathematically, a ring of unidirectionally coupled sigmoidal neurons belongs to a kind of cyclic feedback system and its dynamics has been investigated [2]. It has also been studied as a simple model for recurrent neural networks [3,4] and examined on the basis of a group-theoretic approach due to its symmetry [5–8]. It is a special type of cellular neural networks, which have been employed for signal processing and pattern recognition, and its properties on the convergence to multiple equilibria and the existence and stability of periodic solutions has been investigated [9– 12]. It has been shown that a discrete-time version of a ring of coupled sigmoidal neurons has multiple stable orbits [13,14]. Further, effects of delays on the stability and bifurcations of equilibria and periodic solutions in a ring of unidirectionally coupled sigmoidal neurons have been widely studied [15–19]. It has also been shown that the ring causes long-lasting transient oscillations if delays exist [20–22]. The oscillation in a ring of unidirectionally coupled sigmoidal neurons is a rotating wave propagating in the ring, in which neurons form two bumps. For the sake of simplicity, we consider a ring in which all couplings are excitatory. The states of neurons in one bump are positive and those in the other bump are negative. Then, boundaries
between the bumps (wave fronts) propagate in the direction of coupling, and one rotation of them in the ring corresponds to one period of the oscillation of each neuron. This rotating wave is unstable and the system reaches one of stable spatially uniform steady states eventually. Recently, kinematical analysis showed that such unstable rotating waves in a ring of unidirectionally coupled neurons show dynamical metastability [23]. That is, the duration (life time) of transient rotating waves increases exponentially with the number of neurons. Then, the rotating waves and the oscillations of neurons last extremely long time when the number of neurons is large even though there are no delays. It has been proven that the dominant Floquet multiplier of a periodic rotating wave solution in a ring of piecewise linear sigmoidal neurons with asymmetric bidirectional coupling (including unidirectional coupling as a special case) converges to unity exponentially with the number of neurons [24,25]. A simple sigmoidal input-output relation of a neuron has physiological relevance although a biological neuron generates a sequence of spikes (action potentials). It is because the firing rates (firing frequency) of most neurons in nervous systems increase as stimulus intensity increases, and a sigmoidal function represents the firing rate of a neuron or the population of neurons. A sigmoidal neuron is referred to as a rate model and is widely used in artificial neural networks since it is tractable compared with biologically more plausible spiking models, e.g., the Hodgkin-Huxley model. However, it has been known that the firing rates of many neurons in the auditory system (the inferior colliculus and the auditory cortex) show nonmonotonic growth functions to sound levels [26–29]. The firing rates (rate-intensity functions) of nonmonotonic neurons once increase and then decrease
E-mail address: [email protected]. http://dx.doi.org/10.1016/j.neucom.2016.11.027 Received 14 March 2016; Received in revised form 3 November 2016; Accepted 14 November 2016 Available online 16 November 2016 0925-2312/ © 2016 Elsevier B.V. All rights reserved.
Neurocomputing 225 (2017) 148–156
Y. Horikawa
as the sound intensity increases. Then, nonmonotonic neurons fire at their maximum rate for a specific sound intensity and have intensitytuning properties. In artificial neural networks, a nonmonotonic output function of a neuron has been introduced into associative memory for the first time as far as the author knows [30–34] (see [35] for review). It has been shown that the recall dynamics and memory capacity of autocorrelation associative memory are improved by the nonmonotonicity in the output function of neurons. The stability of equilibria in attractor neural networks consisting of various kinds of nonmonotonic output functions has also been investigated recently. In this paper, we consider rotating waves in a ring of unidirectionally coupled neurons with a nonmonotonic output function of Gaussian type. Because of the nonmonotonicity in the output function of neurons, the recovery of the state of a neuron to its steady state causes an overshoot. This overshoot in the recovery process of a neuron can stabilize rotating waves. The bifurcation analysis of solutions to a model equation shows that stable rotating waves are generated as nonmonotonicity in the output function of neurons increases. When the number of neurons is even, two kinds of stable symmetric rotating waves, which consist of two bumps with the equal lengths (the same numbers of neurons), are generated through pitchfork bifurcations and the Neimark-Sacker bifurcation. When the number of neurons is odd, however, a stable asymmetric rotating wave, which consists of two bumps with the unequal lengths, is generated through a saddle-node bifurcation. It can be shown that larger nonmonotonicity in the output function makes rotating waves chaotic as well as a pair of steady states. Then, kinematical equations for the propagation of wave fronts in rotating waves are derived by using piecewise linear output functions of neurons. It is shown that the propagation time (the inverse of the propagation speed) of a wave front becomes a nonmonotonic function of the length of the forward bump of the wave front. This nonmonotonicity in the propagation time of a wave front due to the nonmonotonicity in the output function of neurons stabilizes rotating waves. Although the kinematical analysis carried out in this paper is qualitative and is not mathematically rigorous, it helps intuitive understanding of the cause of the stabilization of rotating waves. For more rigorous analysis of rotating waves in a ring of unidirectionally coupled neurons, please refer to the references cited above. The rest of the paper is organized as follows. In Section 2, a model equation for a ring of unidirectionally coupled nonmonotonic neurons is presented and the bifurcations and stabilization of its rotating wave solutions are shown. In Section 3, kinematical equations for the propagation of wave fronts in three kinds of stable rotating waves are derived. It is shown that the dependence of the propagation time of a wave front on the forward bump length is nonmonotonic, which results in the stabilization of rotating waves. The discussion and conclusion are given in Sections 4 and 5, respectively.
Fig. 1. Nonmonotonic output function f(x) of neurons with g=1.0 (a solid line) g=e (=2.718∙∙∙) (a dashed line) and g=3.5 (a dotted line) with the line f=x (a thin dash-dotted line).
pitchfork bifurcation at g =1.0 and a pair of stable spatially uniform nonzero steady states (xn=f(xn)≡ ± xs (xs=[2log(g)]1/2 > 0), 1≤n≤N) is generated as g increases. A further increase in g causes the Hopf bifurcation of the origin at g=1/cos(π/N) and an unstable periodic solution with the wavenumber one is generated. The generated periodic solution is a rotating wave with two bumps, in which the states of a half of neurons are positive (xn > 0) and the states of the other half are negative (xn < 0) when the number N of neurons is even (N=2M). Please see lower panels in Fig. 3 for g=3.3. When the output function of neurons is monotonic, this symmetric rotating wave is always unstable [23]. In transients, a rotating wave consisting of two bumps which are comprised of different numbers of neurons is generated. Then, the number of neurons in a smaller bump decreases and the small bump disappears eventually so that the states of all neurons converge to one of stable spatially uniform steady states. However, the duration of a transient rotating wave increases exponentially with the number of neurons in the smaller bump (dynamical metastability). When the output function of neurons is nonmonotonic, the unstable symmetric rotating wave can be stabilized as g increases over unity. We consider the bifurcations of rotating waves with an increase in g. The bifurcations of solutions to Eq. (1) were calculated by using the software package AUTO [36]. Fig. 2 shows the bifurcation diagrams of periodic rotating wave solutions to Eq. (1) with the number of neurons: N=14 (a) and 15 (b), in which the maximum values of x1 in rotating waves are plotted against the output gain g. The branches of stable and unstable rotating waves are plotted with thick and thin lines, respectively. The Hopf bifurcation points (open squares), pitchfork bifurcation points (open triangles), saddle-node bifurcation points (open diamonds) and the Neimark-Sacker bifurcation points (open circles) are also plotted. When N=14 (a), an unstable symmetric rotating wave is generated from the origin through the Hopf bifurcation at g≈1.11. The rotating wave undergoes pitchfork bifurcations at g≈3.14 and 3.19 successively. The rotating wave generated at g≈3.14 is stabilized through a pitchfork bifurcation at g≈3.17 while the rotating wave generated at g≈3.19 is stabilized through the NeimarkSacker bifurcation at g≈3.23. (The branch of an unstable quasiperiodic rotating wave generated at the same time is not plotted.) Stabilized rotating waves undergo further bifurcations and change into stable quasiperiodic rotating waves as g increases over four. Further, it can be shown that chaotic rotating waves emerge over g≈5.83. However, bifurcations and chaos in rotating waves after stabilization are beyond
2. A model and the bifurcations of rotating waves A model equation is given by
dxn /dt = −xn + f (xn −1) f (x ) = gx exp(−x 2 /2)
(1 ≤ n ≤ N ,
xn ± N = xn ,
g > 0)
(1)
where xn is the state of the nth neuron, f(x) is the nonmonotonic output function of neurons and g is an output gain. The output f(xn−1) of the n−1st neuron is transmitted to the nth neuron, and a total of N neurons make a closed loop. Fig. 1 shows the nonmonotonic output function f(x) of neurons with g=1.0 (a solid line) g=e(=2.718∙∙∙) (a dashed line) and g=3.5 (a dotted line) with the line f=x (a thin dash-dotted line). The output function is odd (f(−x)=−f(x)) and takes extreme values ± gexp(−1/2) at x= ± 1 (Here and in the following, the upper signs belong together, and do the lower signs.) and approaches zero as x→ ± ∞. The origin (xn=0, 1≤n≤N) is always a steady state of Eq. (1) and is globally stable when 0 < g < 1.0. The origin is destabilized through a 149
Neurocomputing 225 (2017) 148–156
Y. Horikawa
Fig. 2. Bifurcation diagrams of rotating waves in Eq. (1) with the number of neurons N=14 (a) and 15 (b). Maximum values of x1 in rotating waves vs the output gain g. Thick lines: stable rotating waves, thin lines: unstable rotating waves. Open squares: the Hopf bifurcation points, open triangles: pitchfork bifurcation points, open diamonds: saddlenode bifurcation points, open circles: the Neimark-Sacker bifurcation points. Inset in (a) is a magnification.
Fig. 3. Time courses of the states x1 and x2 of two adjacent neurons (upper panels) and snapshots of xn (1≤n≤N) (lower panels) in stable rotating waves in Eq. (1) with g=3.3 and N=14 (a) (b), 15 (c).
in (b). On the one hand, the amplitudes and waveforms of the oscillations in the states of neurons differ between a positive half cycle (x1, 2 > 0) and a negative half cycle (x1, 2 < 0) in (a). The amplitude is large and the state of a neuron changes almost monotonically in one half cycle while the amplitude is small and the state of a neuron changes nonmonotonically with overshoots in the other half cycle. The state of each neuron alternates these two kinds of half cycles. Further, the signs of these half cycles differ between x1 and x2, i.e., the state x1 (x2) takes positive (negative) monotonic half cycles and negative (positive) nonmonotonic half cycles. On the other hand, the state x1 always takes monotonic half cycles while x2 always takes nonmonotonic half cycles in (b). Because of the symmetry of Eq. (1), the waveforms of all odd-numbered neurons (x2m−1) are the same as x1 while those of all even-numbered neurons (x2m) are the same as x2 in both (a) and (b). Thus, odd-numbered and even-numbered neurons oscillate in a monotonic and nonmonotonic form, respectively, or vice versa in (b). We refer to a stable rotating wave in (a) as type-PF, which is generated through the pitchfork bifurcation, and refer to a stable rotating wave in (b) as type-TR, which is generated through the Neimark-Sacker bifurcation. (The term “TR” is used because the Neimark-Sacker bifurcation is referred to as a torus bifurcation in some cases and its abbreviation NS might be confused with an abbreviation SN of a saddle-node bifurcation, which will be used below.) In both rotating waves of type-PF and type-TR in a ring of an even number of neurons, their spatial patterns are symmetric. That is, the lengths l and N−l of
the scope of this paper and they will be dealt with in a ring with a smaller number of neurons, e.g., three neurons. When N=15 (b), an unstable symmetric rotating wave is generated from the origin through the Hopf bifurcation at g≈1.09. The rotating wave undergoes the Neimark-Sacker bifurcations at g≈3.10 and 4.16 twice, but no stable rotating waves are generated. (The branches of generated unstable quasiperiodic rotating waves are not plotted.) Instead, a pair of a stable and unstable rotating wave is generated through a saddle-node bifurcation at g≈3.12 and disappears at g≈3.48. That is, a stable rotating wave exists only in 3.12 < g < 3.48. When the number of neurons is even (N=2M), stable rotating waves seem to be bifurcated from a symmetric rotating wave generated from the origin through the Hopf bifurcation. When N≥14, two stable rotating waves seem to be generated as shown in Fig. 2(a). When 8≤N≤12, it can be shown that one stable rotating wave is generated through the Neimark-Sacker bifurcation. Fig. 3 shows the time courses of the states x1 and x2 of two adjacent neurons (upper panels) and snapshots of xn (1≤n≤N) (lower panels) in the stable rotating waves in Eq. (1) with g=3.3 and N=14 (a) (b), 15 (c). Eq. (1) was numerically integrated using the Runge-Kutta method with a time step 0.01. In the case of N=14, the rotating wave stabilized through the pitchfork bifurcation at g≈3.17 is shown in (a) while the rotating wave stabilized through the Neimark-Sacker bifurcation at g≈3.23 is shown
150
Neurocomputing 225 (2017) 148–156
Y. Horikawa
of the forward bumps of the fronts be l and N−l (l=l2–l1, N−l=l1–l2 (mod N)), respectively. The propagation of the wave fronts and changes in the lengths of the bumps are described by
two bumps (the numbers of neurons in two bumps) are the same (l=N−l=N/2); both bumps are comprised of seven neurons in Fig. 3(a) (b). When the number of neurons is odd (N=2M +1) and is not less than 15 (N≥15), a stable rotating wave seems to be generated through a saddle-node bifurcation as shown in Fig. 2(b). In the stable rotating wave generated through a saddle-node bifurcation at g≈3.12 for N=15 shown in Fig. 3(c), one monotonic cycle with the large amplitude alternates with the other nonmonotonic cycle with the small amplitude and overshoots in the state of each neuron. One period of the oscillation of each neuron is comprised of these two cycles, i.e., the period is about the double of the period of the oscillation in the case of even N. When a wave front passes a neuron, the sign of the state of the neuron changes. Two rotations of a wave front in a ring correspond to one period of the oscillations in the states of neurons. We refer to this stable rotating wave in a ring of an odd number of neurons as type-SN. The lengths of two bumps in a type-SN rotating wave differ from each other. These differences in the waveforms of the states of neurons in rotating waves come from the bifurcations of a pair of stable spatially uniform nonzero steady states (xn=f(xn) (= ± xs), 1≤n≤N). As the output gain g increases, the slope of f( ± xs) at ± xs becomes negative and decreases to –1 so that a pair of the spatially uniform steady states (xn= ± xs, 1≤n≤N) is destabilized. When the number of neurons is even, each nonzero steady state is destabilized through a pitchfork bifurcation at g=e (=2.718∙∙∙), and a stable steady state with the spatial period two is generated (x2m−1=f(x2m), x2m=f(x2m−1), x2m−1≠x2m, 1≤n≤N/2). Then, the states of adjacent neurons take different values from each other. We let two values of the states of neurons in the stabilized steady state of spatial period two be ± xh and ± xl, where xh > xs > xl > 0 (xh=f(xl), xl=f(xh)). They are solutions to the equation x2(1+g2exp(−x2))=4log(g), and xh=xs=xl=21/2 at g=e. When the number of neurons is odd, each nonzero steady state is destabilized through the Hopf bifurcation at g=exp{[1+1/cos(π/N)]/2}. Then, a stable oscillation around ( ± )xs with the small amplitude is generated, in which the signs of the states of all neurons are the same and unchanged. The oscillation is a traveling wave, in which inconsistency (the adjacent same states) propagates in the direction of coupling as (∙∙∙, xn−1, xn, xn +1, ∙∙∙) =(∙∙∙, xh, xh, xl, ∙∙∙) → (∙∙∙, xh, xl, xl, ∙∙∙) → (∙∙∙, xh, xl, xh, ∙∙∙), i.e., it is a mini ring oscillator. In a rotating wave, the states of adjacent neurons in the same bump alternately approach ± xh and ± xl in the steady state of spatial period two because the signs of the states of neurons in the bump are the same. That is, if xn−1 approaches xh, then xn approaches xl after the passage of a wave front. These alternate states of neurons in the same bumps can be seen in the lower panels in Fig. 3; x5, x7, x9 → xh while x6, x8, x10 → xl in (a), for instance. When xn−1 approaches xh, its output f(xn−1) changes nonmonotonically since |xn−1| increases over unity at which its output takes the maximum value (f( ± 1)=gexp(−1/2)). This nonmonotonic change in f(xn−1) causes a nonmonotonic change in the state xn with the overshoots. These changes in the states of neurons in rotating waves will be dealt with in Section 3.
dl1/dt = 1/ tp (l ),
dl2 /dt = 1/ tp (N − l )
dl /dt = d (l2 − l1)/dt = 1/ tp (N − l ) − 1/ tp (l ),
(2)
d (N − l )/dt = −dl /dt
(3)
Eq. (3) has a steady solution l=N/2 (dl/dt=0), which corresponds to a symmetric rotating wave with the bumps of equal lengths (l=N−l=N/ 2). The linear stability of this steady solution is evaluated by the derivative dtp/dl, and it is stable (unstable) if dtp(N/2)/dl < 0 ( > 0). That is, a symmetric rotating wave is stable (unstable) if the propagation time tp(N/2+Δl) with 0 < Δl«1 of a wave front with a longer forward bump is smaller (larger) than the propagation time tp(N/2−Δl) of a wave front with a shorter forward bump. In the case of a monotonic sigmoidal output function of neurons, it has been shown that the propagation time of a wave front increases with the length of the forward bump (dtp/dl > 0) and a symmetric rotating wave is always unstable [23]. Here, it will be shown that the propagation time of a wave front changes nonmonotonically with the forward bump length, which causes the stabilization of a rotating wave. We will construct piecewise linear functions for the output functions of neurons in Section 3.1, which will be based on the time courses of the states of neurons obtained with computer simulation. Then, we will deal with a type-TR, type-PF and type-SN rotating wave in Sections 3.2–3.4, respectively. 3.1. Piecewise linear output functions of neurons As shown in Fig. 3, the oscillations of neurons consist of two kinds of half cycles: monotonic and nonmonotonic. The waveforms obtained with computer simulation show that the state xn of a neuron approaches ± xh (xh > xs) monotonically in one half cycle after a wave front passes xn (xn=0) while the state of a neuron approaches ± xl (xl < xs) nonmonotonically in the other half cycle. In all types of rotating waves, adjacent neurons take these monotonic and nonmonotonic half cycles alternately as a wave front passes. That is, if xn−1 takes a monotonic (nonmonotonic) half cycle after the passage of a wave front, then xn takes a nonmonotonic (monotonic) half cycle after the passage of the wave front. We refer to a wave front as an upward front when the sign of the state of a neuron changes from negative to positive as the front passes the neuron, and refer to a wave front as a downward front when the sign of the state of a neuron changes from positive to negative at the passage of the front. In a monotonic half cycle of the oscillation in the state xn of the nth neuron, in which an upward front passes the nth neuron, we approximate a change in xn by (4)
dxn /dt = −xn + xh xn (t ) = (xn (0) − x h ) exp(−t ) + x h
(t > 0)
(5)
That is, we approximate the output f(xn−1) of the n−1st neuron by the following step function.
⎧x fSTEP (x ) = ⎨ h ⎩− xh
3. Kinematics of the propagation of wave fronts In this section, we consider the propagation of wave fronts in a rotating wave with two bumps. Let a wave front pass the n−1st neuron at t=0, i.e., let the state xn−1 cross zero at t=0 (xn−1(0)=0). We define the propagation time of the wave front at the nth neuron by tp ( > 0) at which the state xn crosses zero: xn(tp)=0. That is, the propagation time tp is the inverse of the propagation speed of the wave front per neuron. We will derive the propagation time of a wave front as a function of the length of the forward bump (the number of neurons in the forward bump) of the wave front. Although the number of neurons is discrete, we deal with the length of a bump as a continuous variable. Let the locations of wave fronts be l1 and l2 (0 < l1, l2 < N), and let the lengths
(x > 0) (x < 0)
(6)
It is because the state xn−1 changes toward xl after the passage of the upward wave front, at which the output f(xn−1) takes the value xh (f(xl)=xh). Since xl < 1 for large g ( > 3.1), the state xn−1 moves in a monotonic region of f (df/dx > 0 for |x| < 1). Although the approach of xn−1 to xl is nonmonotonic, its output f(xn−1) approaches xh quickly and remains at xh with a small change, which is shown in Fig. 4(a nonmonotonic cycle). To consider a nonmonotonic half cycle of the oscillation in the state xn of the nth neuron after the passage of a wave front, we use the following piecewise linear function fPL as the output function f(xn−1) 151
Neurocomputing 225 (2017) 148–156
Y. Horikawa
respectively). First, we consider the propagation time of a wave front at the nth neuron in a monotonic half cycle. Let an upward wave front with the forward negative bump of length N−l (=N/2) pass the n−1st neuron at t=0 (xn−1(0)=0) and the sign of the state xn−1 change from negative to positive. We then approximate the state xn(0) of the nth neuron in the negative bump at t=0 by −xh for N−l »1. The state xn then increases toward xh according to Eqs. (4) and (5) so that the nth neuron comes into a positive bump of length l. The state xn(τ/2) at which a downward wave front passes the n−1st neuron (xn−1(τ/2)=0) and the sign of xn−1 changes from positive to negative is given by
Fig. 4. Time courses of the state xn−1 of a neuron and its output f(xn−1). In a monotonic cycle of xn−1, its output f(xn−1) changes nonmonotonically as f(xn−1) → ± xh → ± xl. In a nonmonotonic cycle of xn−1, its output f(xn−1) approaches ± xh quickly and remains near ± xh. (Note that a nonmonotonic change in f(xn−1) just before its zerocrossing in a nonmonotonic cycle is not intrinsic.).
xn (τ/2) = −2xh exp (−τ/2) + x h
( > 0,
τ > > 1)
(8)
Then, the state xn decreases toward −xh, and the propagation time tp1 of the downward front at the nth neuron (xn(τ/2+tp1)=0) is obtained as (9)
dxn /dt = −xn − xh
tp1 = log {2[1 − exp(−τ/2)]}
(11)
dxn +1/dt = −xn +1 + fPL (xn ) = − xn +1 + fPL (−xh exp (−t + tp10 ) + x h ) = − xn +1 + x l + (xh − xl )exp(−t + tp10 )
of the n−1st neuron.
(x > 0) (x < 0)
(10)
Note that the propagation time tp1 depends on τ/2, during which the positive bump passes the n−1st neuron, and hence depends on the length l of the forward bump of the downward front. Next, we consider the propagation time of a wave front at the n+1st neuron in a nonmonotonic half cycle with the change in xn and its output f(xn)=fPL(xn) in Eq. (7). The state xn changes for t > 0 according to Eqs. (4) and (5) with xn(0)=−xh as above, and then an upward wave front passes the nth neuron at t=tp10=log2 (xn(tp10)=0). We then approximate the state xn+1(tp10) of the n +1st neuron in the negative bump at t=tp10 by −xl. The state xn+1 then increases toward xl with fPL(xn) in Eq. (7) for the output f(xn) of the nth neuron as
Fig. 5. Piecewise linear function fPL(x) in Eq. (5) with xl=0.85 and xh=2.07. Open circles: the steady points ( ± xh, ± fPL(xh)) (=( ± xh, ± xl)).
⎧ f (x ) = xh + (xl / xh − 1) x fPL (x ) = ⎨ + ⎩ f− (x ) = −xh + (xl / xh − 1) x
xn (τ/2 + tp1) = (xn (τ/2) + x h ) exp(− tp1) − x h=0
(7)
(t > tp10 )
(12)
The state xn+1(tp10+τ/2) at which a downward wave front passes the nth neuron (xn(tp10+τ/2) =0) is given by
Fig. 5 shows fPL(x) with xl=0.85 and xh=2.07, in which open circles denote the steady points ( ± xh, ± fPL(xh)) (=( ± xh, ± xl)) for the steady state of spatial period two. Since the state xn−1 increases xh monotonically after the passage of an upward wave front (xn−1=0) according to Eqs. (4) and (5), its output f(xn−1) increases to xh and then decreases to xl, which is shown in Fig. 4(a monotonic cycle). This decrease in f(xn−1) from xh to xl is taken into consideration in Eq. (7), i.e., the output fPL(xn−1) decreases from xh to xl as the state xn−1 increases from 0 to xh. This decrease is intrinsic for causing a nonmonotonic change in the state xn of the nth neuron.
xn +1 (tp10 + τ/2) = xl + [−2x l + (x h−x l )τ/2]exp(−τ/2)
( > 0)
(13)
The state xn changes for t > tp10+τ/2 as (14)
dxn /dt = −xn − xh xn (tp10 + τ/2 + t ) = xh exp (−t ) − x h
( < 0)
(15)
Then, the state xn+1 decreases toward −xl with fPL(xn) as
dxn +1/dt = −xn +1 + fPL (xn ) = − xn +1 − x l + (xl − xh )exp(−t + tp10 + τ/2)
3.2. Type-TR rotating wave
(t
> tp10 + τ/2)
(16)
xn +1 (tp10 + τ/2 + t ) = − xl + [xn +1 (tp10 + τ/2) + xl + (xl − xh ) t ]exp(−t )
We let the period of a rotating wave (the period of the oscillations of neurons) be τ. The period is related to the mean propagation time tpm of a wave front as τ=Ntpm in a type-TR and Type-SN rotating wave. The lengths of two bumps in a type-TR rotating wave are the same (l=N−l=N/2) as mentioned in Section 2 and thus the length of every half cycle is τ/2. Each wave front in the symmetric rotating wave passes a half (N/2) of the number of neurons in a half cycle (τ/2). In a type-TR rotating wave, the oscillations of adjacent neurons consist of different half cycles from each other: monotonic and nonmonotonic (Fig. 3(b)). We let xn take monotonic half cycles and let xn ± 1 take nonmonotonic half cycles. When the length of a bump (the number of neurons in a bump) and the length of a half cycle are large (l, τ/2 → ∞), the states xn and xn +1 of the adjacent neurons in the bump approach ( ± )xh and ( ± )xl in the steady state of spatial period two, respectively. (Positive (+) and negative (−) signs correspond to a positive and negative bump,
=−xl + {2xl + [−2xl + (xh − xl )τ/2]exp(−τ/2) + (xl − xh ) t} exp(−t ) (17) The propagation time tp2 of the downward front at the n+1st neuron (xn+1(tp10+τ/2+tp2)=0) is given by a solution to the following transcendental equation.
2 − [2 + (1 − xh / xl )τ/2]exp(−τ/2) + (1 − xh / xl ) tp2 = exp(tp2 )
(18)
The mean propagation time of a wave front is given by tpm1=(tp1+tp2)/2 since the states of adjacent neurons take monotonic and nonmonotonic half cycles alternately. The length l (=N/2) of a bump in a symmetric rotating wave is approximated by τ/(2tpm10), where tpm10=(tp10+tp20)/2 with tp10=log2 and tp20=log[2+(xh/ 152
Neurocomputing 225 (2017) 148–156
Y. Horikawa
downward front. Let an upward wave front with the forward negative bump pass the n−1st neuron at t=0 (xn−1(0)=0). We approximate the state xn(0) of the nth neuron in the negative bump at t=0 by –xl since xn takes a negative nonmonotonic half cycle and approaches −xl for t < 0. Then, the state xn increases toward xh according to Eqs. (4) and (5) in a monotonic half cycle so that the nth neuron comes into a positive bump of length l. Then, xn(τ/2) at which a downward wave front passes the n−1st neuron (xn−1(τ/2) =0) is given by
xn (τ/2) = −(xl + x h ) exp(−τ/2) + x h
( > 0,
τ > > 1)
(19)
The state xn−1 changes for t > τ/2 as
dxn −1/dt = −xn −1 − xh
(20)
(t > τ/2)
xn −1 (τ/2 + t ) = xh exp (−t ) − x h
( < 0)
(21)
Then, the state xn decreases toward –xl in a nonmonotonic half cycle with fPL(xn−1) in Eq. (7) for f(xn−1) as
dxn /dt = −xn + fPL (xn −1) = − xn − x l + (xl − xh )exp(−t )
(22)
xn (τ/2 + t ) = −xl + [xn (τ/2) + xl + (xl − xh ) t ]exp(−t ) =−xl + {(xl + xh )[1 − exp(−τ/2)] + (xl − xh ) t}exp(−t ) (23) The propagation time tp3 of the downward front at the nth neuron (xn(τ/2+tp3)=0) is given by a solution to the following transcendental equation.
(1 + xh / xl )[1 − exp(−τ/2)] + (1 − xh / xl ) tp3 = exp(tp3)
(24)
Next, we consider the propagation time tp4 of a downward front at the n+1st neuron. The state xn changes for t > 0 according to Eqs. (4) and (5) with xn(0)=−xl as above, and then an upward wave front passes the nth neuron at t=tp40=log(1+xl/xh) (xn(tp40) =0). We then approximate the state xn+1(tp40) of the n+1st neuron in the negative bump at t=tp40 by −xh. The state xn+1 then increases toward xl with fPL(xn) in Eq. (7) for f(xn) as
Fig. 6. Mean propagation time tpm1 of a type-TR wave front (a) and tpm2 of a type-PF wave front (b) vs the forward bump length l (=N/2).
xl−1)tp20] in the limit of τ → ∞. Then, the mean propagation time tpm1 of a wave front is obtained as a function of the forward bump length l. Although we considered a symmetric rotating wave with the equal bump lengths (l=N−l=N/2), this dependence of the mean propagation time on the forward bump length is applicable to an asymmetric rotating wave with the unequal bump lengths (l≠N−l) from its derivation. Fig. 6(a) shows the mean propagation time tpm1(=(tp1+tp2)/2) of a wave front against the forward bump length l (=N/2) calculated with Eqs. (11) and (18). The values of xh and xl are set to be 2.07 and 0.85, respectively, which are based on the states of neurons in the stable steady state of spatial period two in Eq. (1) with g=3.5. As the forward bump length increases, the propagation time increases to the maximum (≈0.5382) at l=lm1 ≈8.3 and decreases to tpm1(∞) (≈0.532). Thus, a symmetric type-TR rotating wave is stable when l (=N/2) > lm1, hence N > 16, according to Eq. (3). This condition for the stabilization of a type-TR rotating wave agrees qualitatively with that for Eq. (1), in which a stable type-TR rotating wave is generated for N≥8.
dxn +1/dt = −xn +1 + fPL (xn ) = − xn +1 + fPL (−xh exp (−t + tp 40 ) + x h ) = − xn +1 + x l + (xh − xl )exp(−t + tp 40 )
(t > tp 40 )
(25)
The state xn+1(tp40+τ/2) at which a downward wave front passes the nth neuron (xn(tp40+τ/2) =0) is obtained with xn+1(tp40)=−xh as
xn +1 (tp 40 + τ/2) = xl + [−(xh + x l ) + (x h−x l )τ/2]exp(−τ/2)
( > 0) (26)
Then, the state xn+1 decreases toward −xh as
dxn +1/dt = −xn +1 − xh
(t > tp 40 + τ/2)
xn +1 (tp 40 + τ/2 + t ) = −xh + [xn +1 (tp 40 + τ/2) + xh]exp(−t )
(27) (28)
Substituting Eq. (26) into xn+1(tp40+τ/2) in Eq. (28) and letting xn+1(tp40+τ/2+tp4)=0, we obtain the propagation time tp4 of the downward front at the n+1st neuron as
tp 4 = log {1 + xl / xh + [−(1 + xl / xh ) + (1 − xl / xh )τ/2]exp(−τ/2)}
(29)
The mean propagation time of a wave front is given by tpm2=(tp3+tp4)/2. The length l (=N/2) of a bump in a symmetric rotating wave is approximated by τ/(2tpm20), where tpm20=(tp30+tp40)/2 with tp30=log[1+xh/xl+(1−xh/xl)tp30] and tp40=log(1+xl/xh) in the limit of τ→∞. Fig. 6(b) shows the mean propagation time tpm2(=(tp3+tp4)/2) of a wave front against the forward bump length l (=N/2) calculated with Eqs. (24) and (29) (xh=2.07 and xl=0.85). As the forward bump length increases, the propagation time increases to the maximum (≈0.5811) at l=lm2≈9.7 and decreases to tpm2(∞) (≈0.5804). A symmetric type-PF rotating wave is stable when l (=N/2) > lm2, hence N > 18. The bump lengths
3.3. Type-PF rotating wave We consider a type-PF rotating wave, which also consists of two bumps with the equal lengths. The state of each neuron in a type-PF rotating wave takes monotonic and nonmonotonic half cycles alternately (Fig. 3(a)). We let xn take positive monotonic half cycles after the passage of an upward front and take negative nonmonotonic half cycles after the passage of a downward front. Then, the states xn ± 1 take positive nonmonotonic half cycles after the passage of an upward front and take negative monotonic half cycles after the passage of a 153
Neurocomputing 225 (2017) 148–156
Y. Horikawa
lm1 and lm2 at the maxima of the propagation times of wave fronts in a type-TR and type-PF rotating wave differ: lm1 (≈8.3) < lm2 (≈9.7). Thus, a type-TR rotating wave is stabilized in a ring with a smaller number of neurons than a type-PF rotating wave. This condition agrees qualitatively with that for Eq. (1), in which a stable type-TR rotating wave is generated for N≥8 while a stable type-PF rotating wave is generated for N≥14. 3.4. Type-SN rotating wave When the number of neurons is odd (N=2M+1), a stable rotating wave is generated through a saddle-node bifurcation (a type-SN rotating wave) as shown in Fig. 2(b). In a type-SN rotating wave, one period of the oscillation in the state of a neuron consists of a monotonic and nonmonotonic cycle, i.e., two rotations of a wave front in a ring. Each neuron takes a monotonic and nonmonotonic cycle alternately, and adjacent neurons take these two cycles alternately as a wave front passes. Consequently, two wave fronts take different time courses as they pass neurons. The states of all neuron change as ± xh→∓xl or ± xl→∓xh at the passage of one wave front while the states of all neuron change as ∓xh→ ± xh or ∓xl→ ± xl at the passage of the other wave front. In Fig. 3(c), for instance, the states of neurons change as −xh→xl or −xl→xh at the passage of an upward wave front while the states of neurons change as xh→−xh or xl→−xl at the passage of a downward wave front That is, one wave front takes the same time course as a wave front in a type-TR rotating wave, while the other front takes the same time course as a wave front in a type-PF rotating wave. We refer to this two kinds of wave fronts as a type-TR and type-PF wave front. We give a qualitative explanation as to why a stable type-SN rotating wave exists only in the restricted range of g as shown in Fig. 2(b). This restriction of its existence is attributed to the difference between the propagation times of a type-TR and type-PF wave front. That is, the propagation times of two wave fronts must be the same in a periodic rotating wave. If the propagation times of a type-TR and typePF wave front, which depend on the lengths of the forward bumps, differ from each other, a type-SN rotating wave cannot exist. Fig. 7(a) shows the propagation times of wave fronts in a type-TR and type-PF rotating wave in Eq. (1) with N=14 against the output gain g of neurons. The propagation times decreases as the output gain increases, and the propagation time of a type-TR wave front (a solid line) becomes smaller than that of a type-PF wave front (a dashed line). This difference between the propagation times of two wave fronts increases as the output gain increases, which results in the disappearance of a type-SN rotating wave. Since the propagation times of wave fronts in a type-TR and typePF rotating wave derived in Sections 3.2 and 3.3 are based on qualitative piecewise linear output functions of neurons, the results are not applicable directly. That is, the absolute values of the propagation times of wave fronts in two types of rotating waves do not overlap with each other, e.g., tpm1(lm1) (≈0.5382) < tpm2(∞) (≈0.5804). However, the disappearance of a type-SN rotating wave with an increase in g can be shown through an increase in the ratio xh/xl of the values of the states of neurons in the steady state of spatial period two. Fig. 7(b) shows the ratio xh/xl in the steady states of neurons in Eq. (1) against the output gain g of neurons. The ratio xh/xl increases from 1 to 3 as g increases from e to 4. The propagation times tpm1(∞) and tpm2(∞) of a type-TR and type-PF wave front, respectively, for τ→ ∞ derived in Sections 3.2 and 3.3 are functions of xh/xl. Then, Fig. 7(c) shows the propagation times tpm1(∞) and tpm2(∞) of the wave fronts against the ratio xh/xl. Both propagation times decrease and their difference tpm2(∞)−tpm1(∞) increases as xh/xl increases, hence as g increases. Fig. 8 shows changes in the propagation times tpm1(l) and tpm2(l) of the wave fronts due to an increase in g and xh/xl schematically. For the sake of simplicity, the propagation time tpm2 of a type-PF wave front is fixed (a solid line), and the propagation time tpm1 of a type-TR
Fig. 7. Propagation times tp of wave fronts in a type-TR (a solid line) and type-PF rotating wave (a dashed line) in Eq. (1) with N =14 vs the output gain g (a). Ratio xh/xl of the states of neurons in the steady states of spatial period two in Eq. (1) vs the output gain g (b). Propagation times tpm1(∞) of a type-TR wave front (a solid line) and tpm2(∞) of a type-PF wave front (a dashed line) vs the ratio xh/xl.
wave front is shifted downward (dashed, dotted and dash-dotted lines) as g increases. Also, the values lm1 and lm2 at the maximal points are fixed although they actually decrease as g increases. Open circles show the points of the minimum lengths of the forward bumps of a type-TR and type-PF wave front in a type-SN rotating wave for each tpm1 curve. As g increases, the maximum tpm1(lm1) of a type-TR wave front becomes smaller than the asymptotic value tpm2(∞) of a type-PF wave
154
Neurocomputing 225 (2017) 148–156
Y. Horikawa
occurs with other kinds of nonmonotonic output functions of neurons, e.g., f(x)=gxexp(−|x|) or f(x)=gx(1−x2) (g > 0). Rotating waves are stabilized through bifurcations of symmetric rotating waves in a similar manner when the number of neurons is even. However, the stabilization of rotating waves has not been observed with a discontinuous nonmonotonic function: f(x)=gsgn(x)exp(−|x|). The dependence of the occurrence of the stabilization of rotating waves on the form of the output function of neurons should be examined. Finally it should be noted that an increase in the absolute value of the state of a neuron just before the end of a nonmonotonic half cycle, which can be seen in Figs. 3 and 4, is not intrinsic for the stabilization of a rotating wave. In a positive half cycle, for instance, this increase in the state xn of the nth neuron is caused by an increase in the output f(xn−1) of the n−1st neuron due to a decrease in xn−1 from xh to xl. A kinematical equation in which this increase is taken into consideration can be made and it can be shown that the propagation time of a wave front becomes larger but its nonmonotonic dependence on the forward bump length is qualitatively the same.
Fig. 8. Schematic of changes in the propagation time tpm1(l) of a type-TR wave front (dashed, dotted and dash-dotted lines) with an increase in g and the propagation time tpm2(l) of a type-PF wave front (a solid line).
5. Conclusion front for l→∞. Then, a stable type-SN rotating wave cannot exist. Further, as mentioned in Section 2, it can be shown with computer simulation that the lengths of two bumps in a type-SN rotating wave differ from each other and that a type-PF wave front has a longer forward bump than a type-TR wave front. In computer simulation, the length of the forward bump of a type-PF wave front becomes longer as g increases, which agrees with the changes in Fig. 8.
Rotating waves in a ring of unidirectionally coupled neurons with the nonmonotonic output function were examined. It was shown that rotating waves are stabilized as the nonmonotonicity in the output function of neurons increases. The bifurcations of rotating waves and the waveforms of stabilized rotating waves depended on whether the number of neurons is even or odd. When the number of neurons was even, two types of rotating waves were stabilized through pitchfork bifurcations and the Neimark-Sacker bifurcation. When the number of neurons was odd, a stable rotating was generated through a saddlenode bifurcation. The propagation times of wave fronts in the stabilized rotating waves were derived by using the piecewise linear output functions of neurons, which were based on changes in the states of neurons obtained with computer simulation. The propagation time of a wave front depended on the length of the forward bump of the wave front and changed nonmonotonically as the forward bump length increased. The kinematical equations for the propagation of wave fronts showed that rotating waves are stabilized owing to this nonmonotonicity of the propagation time, which resulted from the nonmonotonicity in the output function of neurons. In this study, we restricted ourselves to a ring of neurons with simple unidirectional coupling. The effects of nonmonotonicity in the output function of neurons on metastable dynamical rotating waves in the presence of asymmetric bidirectional coupling and self-coupling are one of future areas of interest. Further, future work is needed to examine the effects of asymmetry in nonmonotonic output functions of neurons on the dynamical metastability and stabilization of rotating waves.
4. Discussion The stabilization of rotating waves is caused by an overshoot of the state of a neuron after the passage of a wave front in a nonmonotonic half cycle. After an upward front passes a neuron, for instance, the state of the neuron increases and overshoots the steady state. While converging to the steady state after the overshoot, the state of the neuron decreases until a downward front comes to the neuron. The time interval during which the state of a neuron decreases increases as the length of the forward bump of the downward front increases. Then, the propagation time of the downward front at the neuron decreases as its forward bump length increases since the state of the neuron decreases. Thus, the propagation time of a wave front with the longer forward bump is smaller than that of the other wave front with the shorter forward bump, so that the longer (shorter) bump decreases (increases). This results in the stabilization of a symmetric rotating wave with the bumps of equal lengths. Such stabilization of rotating waves is similar to that shown in a ring of unidirectionally coupled sigmoidal neurons with inertias, in which a damped oscillation in the state of a neuron due to the inertia makes an overshoot [37]. Then, there are the time intervals in which the state of a neuron decreases when converging to the steady state. When the time during which the forward bump of a wave front passes a neuron lies in these intervals, the propagation time of the wave front at the neuron decreases as the forward bump length increases. This change in the propagation time works to stabilize rotating waves. Similar stabilization of rotating waves has also been shown in a discrete-time system with coupled circle maps [38]. There are a pair of stable spatially uniform steady states, and each steady state undergoes a period doubling bifurcation as the strength of coupling increases. Then, a pair of stable states with the temporal and spatial period two is generated. In a rotating wave, the states of neurons in bumps also oscillate with period two and the oscillations work as overshoots. Consequently, the dependence of the propagation speed of a wave front on the bump length becomes nonmonotonic. The stabilization of rotating waves due to nonmonotonicity in the output function of neurons is not specific for the Gaussian-type function employed in this study. It can be shown that the stabilization
References [1] S. Amari, Mathematics of Neural Networks (in Japanese), Sangyo-Tosho, Tokyo, 1978. [2] T. Gedeon, Cyclic Feedback Systems. Memoirs of the American Mathematical Society, American Mathematical Society, Providence, 1998. [3] A.F. Atiya, P. Baldi, Oscillations and synchronizations in neural networks: an exploration of the labeling hypothesis, Int. J. Neural Syst. 1 (1989) 103–124. [4] M.H. Hirsh, Convergent activation dynamics in continuous time networks, Neural Netw. 2 (1989) 331–340. [5] M. Golubitsky, I. Stewart, D.G. Schaeffer, Singularities and Groups in Bifurcation Theory II, Springer, New York, 1988. [6] J.J. Collins, I. Stewart, A group-theoretic approach to rings of coupled biological oscillators, Biol. Cybern. 71 (1994) 95–103. [7] P.C. Bressloff, S. Coombes, B. de Souza, Dynamics of a ring of pulse-coupled oscillators: Group theoretic approach, Phys. Rev. Lett. 79 (1997) 2791–2794. [8] P.-L. Buono, M. Golubitsky, A. Palacios, Heteroclinic cycles in rings of coupled cells, Physica D 143 (2000) 74–108. [9] P. Thiran, Dynamics and Self-organization of Locally Coupled Neural Networks, Presses Polytechniques et Universitaires Romandes, Lausanne, 1997.
155
Neurocomputing 225 (2017) 148–156
Y. Horikawa
pp. 37–64. [29] C.E. Schreiner, B.J. Malone, Representation of loudness in the auditory cortex, in: G.G. Celesia, G. Hickok (Eds.), The Human Auditory System: Fundamental Organization and Clinical Disorders, Elsevier, Amsterdam, 2015, pp. 73–84. [30] M. Morita, S. Yoshizawa, K. Nakano, Analysis and improvement of the dynamics of autocorrelation associative memory (in Japanese) (Japanese ed.), IEICE Trans. Inf. Syst. J37-D-II (1990) 232–242. [31] M. Morita, Associative memory with nonmonotone dynamics, Neural Netw. 6 (1993) 115–126. [32] S. Yoshizawa, M. Morita, S. Amari, Capacity of associative memory using a nonmonotonic neuron model, Neural Netw. 6 (1993) 167–176. [33] G. Boffetta, R. Monasson, R. Zecchina, Symmetry breaking in non-monotonic neural networks, J. Phys. A 26 (1993) L507–L513. [34] N. Brunel, R. Zecchina, Response functions improving performance in attractor neural networks, Phys. Rev. E 49 (1994) R1823–R1826. [35] H. Zhang, Z. Wang, D. Liu, A Comprehensive review of stability analysis of continuous-time recurrent neural networks, IEEE Trans. Neural Netw. Learn. Syst. 25 (2014) 1229–1262. [36] E.J. Doedel, AUTO: Software for Continuation and Bifurcation Problems in Ordinary Differential Equations, Concordia University, Montreal, Canada, 1996 〈http://indy.cs.concordia.ca/auto/〉. [37] Y. Horikawa, H. Kitajima, Bifurcation and stabilization of oscillations in ring neural networks with inertia, Physica D 238 (2009) 2409–2418. [38] Y. Horikawa, H. Kitajima, Exponential transient rotating waves in a bistable ring of unidirectionally coupled maps, Physica D 241 (2012) 106–114.
[10] P. Thiran, G. Setti, M. Hasler, An approach to information propagation in 1-D cellular neural networks-part I: local diffusion, IEEE Trans. Circuits Syst. I 45 (1998) 777–789. [11] G. Setti, P. Thiran, C. Serpico, An approach to information propagation in 1-D cellular neural networks-part II: global propagation, IEEE Trans. Circuits Syst. I 45 (1998) 790–811. [12] M. Di Marco, M. Forti, M. Grazzini, L. Pancioni, Convergence of a class of cooperative standard cellular neural network arrays, IEEE Trans. Circuits Syst. I: Regul. Pap. 59 (2012) 772–783. [13] E.K. Blum, X. Wang, Stability of fixed points and periodic orbits and bifurcations in analog neural networks, Neural Netw. 5 (1992) 577–587. [14] F. Pasemann, Characterization of periodic attractors in neural ring networks, Neural Netw. 8 (1995) 421–429. [15] J. Wei, M. Velarde, Bifurcation analysis and existence of periodic solutions in a simple neural network with delays, Chaos 14 (2004) 940–953. [16] J. Wei, M.Y. Li, Global existence of periodic solutions in a tri-neuron network model with delays, Physica D 198 (2004) 109–119. [17] X. Li, J. Wei, Stability and bifurcation analysis on a delayed neural network model, Int. J. Bifurc. Chaos 15 (2005) 2883–2893. [18] J. Wei, C. Zhang, Bifurcation analysis of a class of neural networks with delays, Nonlinear Anal.: Real World Appl. 9 (2008), 2008, pp. 2234–2252. [19] S. Guo, L. Huang, Pattern formation and continuation in a trineuron ring with delays, Acta Math. Sin. 23 (2007) 799–818. [20] K.L. Babcock, R.M. Westervelt, Dynamics of simple electronic neural networks, Physica D 28 (1987) 305–316. [21] P. Baldi, A.F. Atiya, How delays affect neural dynamics and learning, IEEE Trans. Neural Netw. 5 (1994) 612–621. [22] K. Pakdaman, C.P. Malta, C. Grotta-Ragazzo, O. Arino, J.-F. Vibert, Transient oscillations in continuous-time excitatory ring neural networks with delay, Phys. Rev. E 55 (1997) 3234–3248. [23] Y. Horikawa, H. Kitajima, Duration of transient oscillations in ring networks of unidirectionally coupled neurons, Physica D 238 (2009) 216–225. [24] M. Forti, B. Garay, M. Koller, L. Pancioni, Long transient oscillations in a class of cooperative cellular neural networks, Int. J. Circ. Theory Appl. 43 (2015) 635–655. [25] M. Di Marco, M. Forti, B.M. Garay, M. Koller, L. Pancioni, Floquet multipliers of a metastable rotating wave in a Chua–Yang ring network, J. Math. Anal. Appl. 434 (2016) 798–836. [26] B.E. Pfingst, T.A. O'Connor, Characteristics of neurons in auditory cortex of monkeys performing a simple auditory task, J. Neurophysiol. 45 (1981) 16–34. [27] P.X. Joris, C.E. Schreiner, A. Rees, Neural processing of amplitude-modulated sounds, Physiol. Rev. 84 (2004) 541–577. [28] F.E. Musiek, J.A. Weihing, V.B. Oxholm, Anatomy and physiology of the central auditory nervous system: A clinical perspective, in: R.J. Roeser, M. Valente, H. Hosford-Dunn (Eds.), Audiology: Diagnosis2nd ed., Thieme, New York, 2007,
Yo Horikawa is a professor in the Faculty of Engineering at Kagawa University, Japan. He has received B. Eng., M. Eng. and Ph.D. (Eng.) degrees in mathematical engineering and information physics from the University of Tokyo in 1983, 1985 and 1994, respectively. His research interests include nonlinear dynamical systems and statistical pattern recognition.
156
Contents lists available at ScienceDirect
Neurocomputing journal homepage: www.elsevier.com/locate/neucom
Stabilization of metastable dynamical rotating waves in a ring of unidirectionally coupled nonmonotonic neurons
MARK
Yo Horikawa Faculty of Engineering, Kagawa University, Takamatsu 761-0396, Japan
A R T I C L E I N F O
A BS T RAC T
Communicated by Y. Liu
Effects of nonmonotonicity in a sigmoidal output function of neurons on rotating waves in a ring of unidirectionally coupled neurons are studied. It is known that transient rotating waves show dynamical metastability when the output of neurons is monotonic; their duration increases exponentially with the number of neurons. Stable rotating waves of three types are generated as the nonmonotonicity in the output function of neurons increases. The type of stable rotating wave depends on whether the number of neurons is even or odd. Kinematical description for the propagation of wave fronts shows that an overshoot of the state of a neuron after the passage of the wave front due to the nonmonotonic output of a neuron causes the stabilization of rotating waves.
Keywords: Ring neural network Metastable dynamics Rotating wave Nonmonotonic output
1. Introduction A ring of unidirectionally coupled sigmoidal neurons has been studied from several points of view although its structure is simple. It has been shown that a ring of unidirectionally coupled neurons causes a stable oscillation when the number of inhibitory couplings is odd [1]. Such a ring is equivalent to a ring oscillator, which is a closed loop of an odd number of inverters and is widely used as one of variablefrequency oscillators in electronic circuits. Mathematically, a ring of unidirectionally coupled sigmoidal neurons belongs to a kind of cyclic feedback system and its dynamics has been investigated [2]. It has also been studied as a simple model for recurrent neural networks [3,4] and examined on the basis of a group-theoretic approach due to its symmetry [5–8]. It is a special type of cellular neural networks, which have been employed for signal processing and pattern recognition, and its properties on the convergence to multiple equilibria and the existence and stability of periodic solutions has been investigated [9– 12]. It has been shown that a discrete-time version of a ring of coupled sigmoidal neurons has multiple stable orbits [13,14]. Further, effects of delays on the stability and bifurcations of equilibria and periodic solutions in a ring of unidirectionally coupled sigmoidal neurons have been widely studied [15–19]. It has also been shown that the ring causes long-lasting transient oscillations if delays exist [20–22]. The oscillation in a ring of unidirectionally coupled sigmoidal neurons is a rotating wave propagating in the ring, in which neurons form two bumps. For the sake of simplicity, we consider a ring in which all couplings are excitatory. The states of neurons in one bump are positive and those in the other bump are negative. Then, boundaries
between the bumps (wave fronts) propagate in the direction of coupling, and one rotation of them in the ring corresponds to one period of the oscillation of each neuron. This rotating wave is unstable and the system reaches one of stable spatially uniform steady states eventually. Recently, kinematical analysis showed that such unstable rotating waves in a ring of unidirectionally coupled neurons show dynamical metastability [23]. That is, the duration (life time) of transient rotating waves increases exponentially with the number of neurons. Then, the rotating waves and the oscillations of neurons last extremely long time when the number of neurons is large even though there are no delays. It has been proven that the dominant Floquet multiplier of a periodic rotating wave solution in a ring of piecewise linear sigmoidal neurons with asymmetric bidirectional coupling (including unidirectional coupling as a special case) converges to unity exponentially with the number of neurons [24,25]. A simple sigmoidal input-output relation of a neuron has physiological relevance although a biological neuron generates a sequence of spikes (action potentials). It is because the firing rates (firing frequency) of most neurons in nervous systems increase as stimulus intensity increases, and a sigmoidal function represents the firing rate of a neuron or the population of neurons. A sigmoidal neuron is referred to as a rate model and is widely used in artificial neural networks since it is tractable compared with biologically more plausible spiking models, e.g., the Hodgkin-Huxley model. However, it has been known that the firing rates of many neurons in the auditory system (the inferior colliculus and the auditory cortex) show nonmonotonic growth functions to sound levels [26–29]. The firing rates (rate-intensity functions) of nonmonotonic neurons once increase and then decrease
E-mail address: [email protected]. http://dx.doi.org/10.1016/j.neucom.2016.11.027 Received 14 March 2016; Received in revised form 3 November 2016; Accepted 14 November 2016 Available online 16 November 2016 0925-2312/ © 2016 Elsevier B.V. All rights reserved.
Neurocomputing 225 (2017) 148–156
Y. Horikawa
as the sound intensity increases. Then, nonmonotonic neurons fire at their maximum rate for a specific sound intensity and have intensitytuning properties. In artificial neural networks, a nonmonotonic output function of a neuron has been introduced into associative memory for the first time as far as the author knows [30–34] (see [35] for review). It has been shown that the recall dynamics and memory capacity of autocorrelation associative memory are improved by the nonmonotonicity in the output function of neurons. The stability of equilibria in attractor neural networks consisting of various kinds of nonmonotonic output functions has also been investigated recently. In this paper, we consider rotating waves in a ring of unidirectionally coupled neurons with a nonmonotonic output function of Gaussian type. Because of the nonmonotonicity in the output function of neurons, the recovery of the state of a neuron to its steady state causes an overshoot. This overshoot in the recovery process of a neuron can stabilize rotating waves. The bifurcation analysis of solutions to a model equation shows that stable rotating waves are generated as nonmonotonicity in the output function of neurons increases. When the number of neurons is even, two kinds of stable symmetric rotating waves, which consist of two bumps with the equal lengths (the same numbers of neurons), are generated through pitchfork bifurcations and the Neimark-Sacker bifurcation. When the number of neurons is odd, however, a stable asymmetric rotating wave, which consists of two bumps with the unequal lengths, is generated through a saddle-node bifurcation. It can be shown that larger nonmonotonicity in the output function makes rotating waves chaotic as well as a pair of steady states. Then, kinematical equations for the propagation of wave fronts in rotating waves are derived by using piecewise linear output functions of neurons. It is shown that the propagation time (the inverse of the propagation speed) of a wave front becomes a nonmonotonic function of the length of the forward bump of the wave front. This nonmonotonicity in the propagation time of a wave front due to the nonmonotonicity in the output function of neurons stabilizes rotating waves. Although the kinematical analysis carried out in this paper is qualitative and is not mathematically rigorous, it helps intuitive understanding of the cause of the stabilization of rotating waves. For more rigorous analysis of rotating waves in a ring of unidirectionally coupled neurons, please refer to the references cited above. The rest of the paper is organized as follows. In Section 2, a model equation for a ring of unidirectionally coupled nonmonotonic neurons is presented and the bifurcations and stabilization of its rotating wave solutions are shown. In Section 3, kinematical equations for the propagation of wave fronts in three kinds of stable rotating waves are derived. It is shown that the dependence of the propagation time of a wave front on the forward bump length is nonmonotonic, which results in the stabilization of rotating waves. The discussion and conclusion are given in Sections 4 and 5, respectively.
Fig. 1. Nonmonotonic output function f(x) of neurons with g=1.0 (a solid line) g=e (=2.718∙∙∙) (a dashed line) and g=3.5 (a dotted line) with the line f=x (a thin dash-dotted line).
pitchfork bifurcation at g =1.0 and a pair of stable spatially uniform nonzero steady states (xn=f(xn)≡ ± xs (xs=[2log(g)]1/2 > 0), 1≤n≤N) is generated as g increases. A further increase in g causes the Hopf bifurcation of the origin at g=1/cos(π/N) and an unstable periodic solution with the wavenumber one is generated. The generated periodic solution is a rotating wave with two bumps, in which the states of a half of neurons are positive (xn > 0) and the states of the other half are negative (xn < 0) when the number N of neurons is even (N=2M). Please see lower panels in Fig. 3 for g=3.3. When the output function of neurons is monotonic, this symmetric rotating wave is always unstable [23]. In transients, a rotating wave consisting of two bumps which are comprised of different numbers of neurons is generated. Then, the number of neurons in a smaller bump decreases and the small bump disappears eventually so that the states of all neurons converge to one of stable spatially uniform steady states. However, the duration of a transient rotating wave increases exponentially with the number of neurons in the smaller bump (dynamical metastability). When the output function of neurons is nonmonotonic, the unstable symmetric rotating wave can be stabilized as g increases over unity. We consider the bifurcations of rotating waves with an increase in g. The bifurcations of solutions to Eq. (1) were calculated by using the software package AUTO [36]. Fig. 2 shows the bifurcation diagrams of periodic rotating wave solutions to Eq. (1) with the number of neurons: N=14 (a) and 15 (b), in which the maximum values of x1 in rotating waves are plotted against the output gain g. The branches of stable and unstable rotating waves are plotted with thick and thin lines, respectively. The Hopf bifurcation points (open squares), pitchfork bifurcation points (open triangles), saddle-node bifurcation points (open diamonds) and the Neimark-Sacker bifurcation points (open circles) are also plotted. When N=14 (a), an unstable symmetric rotating wave is generated from the origin through the Hopf bifurcation at g≈1.11. The rotating wave undergoes pitchfork bifurcations at g≈3.14 and 3.19 successively. The rotating wave generated at g≈3.14 is stabilized through a pitchfork bifurcation at g≈3.17 while the rotating wave generated at g≈3.19 is stabilized through the NeimarkSacker bifurcation at g≈3.23. (The branch of an unstable quasiperiodic rotating wave generated at the same time is not plotted.) Stabilized rotating waves undergo further bifurcations and change into stable quasiperiodic rotating waves as g increases over four. Further, it can be shown that chaotic rotating waves emerge over g≈5.83. However, bifurcations and chaos in rotating waves after stabilization are beyond
2. A model and the bifurcations of rotating waves A model equation is given by
dxn /dt = −xn + f (xn −1) f (x ) = gx exp(−x 2 /2)
(1 ≤ n ≤ N ,
xn ± N = xn ,
g > 0)
(1)
where xn is the state of the nth neuron, f(x) is the nonmonotonic output function of neurons and g is an output gain. The output f(xn−1) of the n−1st neuron is transmitted to the nth neuron, and a total of N neurons make a closed loop. Fig. 1 shows the nonmonotonic output function f(x) of neurons with g=1.0 (a solid line) g=e(=2.718∙∙∙) (a dashed line) and g=3.5 (a dotted line) with the line f=x (a thin dash-dotted line). The output function is odd (f(−x)=−f(x)) and takes extreme values ± gexp(−1/2) at x= ± 1 (Here and in the following, the upper signs belong together, and do the lower signs.) and approaches zero as x→ ± ∞. The origin (xn=0, 1≤n≤N) is always a steady state of Eq. (1) and is globally stable when 0 < g < 1.0. The origin is destabilized through a 149
Neurocomputing 225 (2017) 148–156
Y. Horikawa
Fig. 2. Bifurcation diagrams of rotating waves in Eq. (1) with the number of neurons N=14 (a) and 15 (b). Maximum values of x1 in rotating waves vs the output gain g. Thick lines: stable rotating waves, thin lines: unstable rotating waves. Open squares: the Hopf bifurcation points, open triangles: pitchfork bifurcation points, open diamonds: saddlenode bifurcation points, open circles: the Neimark-Sacker bifurcation points. Inset in (a) is a magnification.
Fig. 3. Time courses of the states x1 and x2 of two adjacent neurons (upper panels) and snapshots of xn (1≤n≤N) (lower panels) in stable rotating waves in Eq. (1) with g=3.3 and N=14 (a) (b), 15 (c).
in (b). On the one hand, the amplitudes and waveforms of the oscillations in the states of neurons differ between a positive half cycle (x1, 2 > 0) and a negative half cycle (x1, 2 < 0) in (a). The amplitude is large and the state of a neuron changes almost monotonically in one half cycle while the amplitude is small and the state of a neuron changes nonmonotonically with overshoots in the other half cycle. The state of each neuron alternates these two kinds of half cycles. Further, the signs of these half cycles differ between x1 and x2, i.e., the state x1 (x2) takes positive (negative) monotonic half cycles and negative (positive) nonmonotonic half cycles. On the other hand, the state x1 always takes monotonic half cycles while x2 always takes nonmonotonic half cycles in (b). Because of the symmetry of Eq. (1), the waveforms of all odd-numbered neurons (x2m−1) are the same as x1 while those of all even-numbered neurons (x2m) are the same as x2 in both (a) and (b). Thus, odd-numbered and even-numbered neurons oscillate in a monotonic and nonmonotonic form, respectively, or vice versa in (b). We refer to a stable rotating wave in (a) as type-PF, which is generated through the pitchfork bifurcation, and refer to a stable rotating wave in (b) as type-TR, which is generated through the Neimark-Sacker bifurcation. (The term “TR” is used because the Neimark-Sacker bifurcation is referred to as a torus bifurcation in some cases and its abbreviation NS might be confused with an abbreviation SN of a saddle-node bifurcation, which will be used below.) In both rotating waves of type-PF and type-TR in a ring of an even number of neurons, their spatial patterns are symmetric. That is, the lengths l and N−l of
the scope of this paper and they will be dealt with in a ring with a smaller number of neurons, e.g., three neurons. When N=15 (b), an unstable symmetric rotating wave is generated from the origin through the Hopf bifurcation at g≈1.09. The rotating wave undergoes the Neimark-Sacker bifurcations at g≈3.10 and 4.16 twice, but no stable rotating waves are generated. (The branches of generated unstable quasiperiodic rotating waves are not plotted.) Instead, a pair of a stable and unstable rotating wave is generated through a saddle-node bifurcation at g≈3.12 and disappears at g≈3.48. That is, a stable rotating wave exists only in 3.12 < g < 3.48. When the number of neurons is even (N=2M), stable rotating waves seem to be bifurcated from a symmetric rotating wave generated from the origin through the Hopf bifurcation. When N≥14, two stable rotating waves seem to be generated as shown in Fig. 2(a). When 8≤N≤12, it can be shown that one stable rotating wave is generated through the Neimark-Sacker bifurcation. Fig. 3 shows the time courses of the states x1 and x2 of two adjacent neurons (upper panels) and snapshots of xn (1≤n≤N) (lower panels) in the stable rotating waves in Eq. (1) with g=3.3 and N=14 (a) (b), 15 (c). Eq. (1) was numerically integrated using the Runge-Kutta method with a time step 0.01. In the case of N=14, the rotating wave stabilized through the pitchfork bifurcation at g≈3.17 is shown in (a) while the rotating wave stabilized through the Neimark-Sacker bifurcation at g≈3.23 is shown
150
Neurocomputing 225 (2017) 148–156
Y. Horikawa
of the forward bumps of the fronts be l and N−l (l=l2–l1, N−l=l1–l2 (mod N)), respectively. The propagation of the wave fronts and changes in the lengths of the bumps are described by
two bumps (the numbers of neurons in two bumps) are the same (l=N−l=N/2); both bumps are comprised of seven neurons in Fig. 3(a) (b). When the number of neurons is odd (N=2M +1) and is not less than 15 (N≥15), a stable rotating wave seems to be generated through a saddle-node bifurcation as shown in Fig. 2(b). In the stable rotating wave generated through a saddle-node bifurcation at g≈3.12 for N=15 shown in Fig. 3(c), one monotonic cycle with the large amplitude alternates with the other nonmonotonic cycle with the small amplitude and overshoots in the state of each neuron. One period of the oscillation of each neuron is comprised of these two cycles, i.e., the period is about the double of the period of the oscillation in the case of even N. When a wave front passes a neuron, the sign of the state of the neuron changes. Two rotations of a wave front in a ring correspond to one period of the oscillations in the states of neurons. We refer to this stable rotating wave in a ring of an odd number of neurons as type-SN. The lengths of two bumps in a type-SN rotating wave differ from each other. These differences in the waveforms of the states of neurons in rotating waves come from the bifurcations of a pair of stable spatially uniform nonzero steady states (xn=f(xn) (= ± xs), 1≤n≤N). As the output gain g increases, the slope of f( ± xs) at ± xs becomes negative and decreases to –1 so that a pair of the spatially uniform steady states (xn= ± xs, 1≤n≤N) is destabilized. When the number of neurons is even, each nonzero steady state is destabilized through a pitchfork bifurcation at g=e (=2.718∙∙∙), and a stable steady state with the spatial period two is generated (x2m−1=f(x2m), x2m=f(x2m−1), x2m−1≠x2m, 1≤n≤N/2). Then, the states of adjacent neurons take different values from each other. We let two values of the states of neurons in the stabilized steady state of spatial period two be ± xh and ± xl, where xh > xs > xl > 0 (xh=f(xl), xl=f(xh)). They are solutions to the equation x2(1+g2exp(−x2))=4log(g), and xh=xs=xl=21/2 at g=e. When the number of neurons is odd, each nonzero steady state is destabilized through the Hopf bifurcation at g=exp{[1+1/cos(π/N)]/2}. Then, a stable oscillation around ( ± )xs with the small amplitude is generated, in which the signs of the states of all neurons are the same and unchanged. The oscillation is a traveling wave, in which inconsistency (the adjacent same states) propagates in the direction of coupling as (∙∙∙, xn−1, xn, xn +1, ∙∙∙) =(∙∙∙, xh, xh, xl, ∙∙∙) → (∙∙∙, xh, xl, xl, ∙∙∙) → (∙∙∙, xh, xl, xh, ∙∙∙), i.e., it is a mini ring oscillator. In a rotating wave, the states of adjacent neurons in the same bump alternately approach ± xh and ± xl in the steady state of spatial period two because the signs of the states of neurons in the bump are the same. That is, if xn−1 approaches xh, then xn approaches xl after the passage of a wave front. These alternate states of neurons in the same bumps can be seen in the lower panels in Fig. 3; x5, x7, x9 → xh while x6, x8, x10 → xl in (a), for instance. When xn−1 approaches xh, its output f(xn−1) changes nonmonotonically since |xn−1| increases over unity at which its output takes the maximum value (f( ± 1)=gexp(−1/2)). This nonmonotonic change in f(xn−1) causes a nonmonotonic change in the state xn with the overshoots. These changes in the states of neurons in rotating waves will be dealt with in Section 3.
dl1/dt = 1/ tp (l ),
dl2 /dt = 1/ tp (N − l )
dl /dt = d (l2 − l1)/dt = 1/ tp (N − l ) − 1/ tp (l ),
(2)
d (N − l )/dt = −dl /dt
(3)
Eq. (3) has a steady solution l=N/2 (dl/dt=0), which corresponds to a symmetric rotating wave with the bumps of equal lengths (l=N−l=N/ 2). The linear stability of this steady solution is evaluated by the derivative dtp/dl, and it is stable (unstable) if dtp(N/2)/dl < 0 ( > 0). That is, a symmetric rotating wave is stable (unstable) if the propagation time tp(N/2+Δl) with 0 < Δl«1 of a wave front with a longer forward bump is smaller (larger) than the propagation time tp(N/2−Δl) of a wave front with a shorter forward bump. In the case of a monotonic sigmoidal output function of neurons, it has been shown that the propagation time of a wave front increases with the length of the forward bump (dtp/dl > 0) and a symmetric rotating wave is always unstable [23]. Here, it will be shown that the propagation time of a wave front changes nonmonotonically with the forward bump length, which causes the stabilization of a rotating wave. We will construct piecewise linear functions for the output functions of neurons in Section 3.1, which will be based on the time courses of the states of neurons obtained with computer simulation. Then, we will deal with a type-TR, type-PF and type-SN rotating wave in Sections 3.2–3.4, respectively. 3.1. Piecewise linear output functions of neurons As shown in Fig. 3, the oscillations of neurons consist of two kinds of half cycles: monotonic and nonmonotonic. The waveforms obtained with computer simulation show that the state xn of a neuron approaches ± xh (xh > xs) monotonically in one half cycle after a wave front passes xn (xn=0) while the state of a neuron approaches ± xl (xl < xs) nonmonotonically in the other half cycle. In all types of rotating waves, adjacent neurons take these monotonic and nonmonotonic half cycles alternately as a wave front passes. That is, if xn−1 takes a monotonic (nonmonotonic) half cycle after the passage of a wave front, then xn takes a nonmonotonic (monotonic) half cycle after the passage of the wave front. We refer to a wave front as an upward front when the sign of the state of a neuron changes from negative to positive as the front passes the neuron, and refer to a wave front as a downward front when the sign of the state of a neuron changes from positive to negative at the passage of the front. In a monotonic half cycle of the oscillation in the state xn of the nth neuron, in which an upward front passes the nth neuron, we approximate a change in xn by (4)
dxn /dt = −xn + xh xn (t ) = (xn (0) − x h ) exp(−t ) + x h
(t > 0)
(5)
That is, we approximate the output f(xn−1) of the n−1st neuron by the following step function.
⎧x fSTEP (x ) = ⎨ h ⎩− xh
3. Kinematics of the propagation of wave fronts In this section, we consider the propagation of wave fronts in a rotating wave with two bumps. Let a wave front pass the n−1st neuron at t=0, i.e., let the state xn−1 cross zero at t=0 (xn−1(0)=0). We define the propagation time of the wave front at the nth neuron by tp ( > 0) at which the state xn crosses zero: xn(tp)=0. That is, the propagation time tp is the inverse of the propagation speed of the wave front per neuron. We will derive the propagation time of a wave front as a function of the length of the forward bump (the number of neurons in the forward bump) of the wave front. Although the number of neurons is discrete, we deal with the length of a bump as a continuous variable. Let the locations of wave fronts be l1 and l2 (0 < l1, l2 < N), and let the lengths
(x > 0) (x < 0)
(6)
It is because the state xn−1 changes toward xl after the passage of the upward wave front, at which the output f(xn−1) takes the value xh (f(xl)=xh). Since xl < 1 for large g ( > 3.1), the state xn−1 moves in a monotonic region of f (df/dx > 0 for |x| < 1). Although the approach of xn−1 to xl is nonmonotonic, its output f(xn−1) approaches xh quickly and remains at xh with a small change, which is shown in Fig. 4(a nonmonotonic cycle). To consider a nonmonotonic half cycle of the oscillation in the state xn of the nth neuron after the passage of a wave front, we use the following piecewise linear function fPL as the output function f(xn−1) 151
Neurocomputing 225 (2017) 148–156
Y. Horikawa
respectively). First, we consider the propagation time of a wave front at the nth neuron in a monotonic half cycle. Let an upward wave front with the forward negative bump of length N−l (=N/2) pass the n−1st neuron at t=0 (xn−1(0)=0) and the sign of the state xn−1 change from negative to positive. We then approximate the state xn(0) of the nth neuron in the negative bump at t=0 by −xh for N−l »1. The state xn then increases toward xh according to Eqs. (4) and (5) so that the nth neuron comes into a positive bump of length l. The state xn(τ/2) at which a downward wave front passes the n−1st neuron (xn−1(τ/2)=0) and the sign of xn−1 changes from positive to negative is given by
Fig. 4. Time courses of the state xn−1 of a neuron and its output f(xn−1). In a monotonic cycle of xn−1, its output f(xn−1) changes nonmonotonically as f(xn−1) → ± xh → ± xl. In a nonmonotonic cycle of xn−1, its output f(xn−1) approaches ± xh quickly and remains near ± xh. (Note that a nonmonotonic change in f(xn−1) just before its zerocrossing in a nonmonotonic cycle is not intrinsic.).
xn (τ/2) = −2xh exp (−τ/2) + x h
( > 0,
τ > > 1)
(8)
Then, the state xn decreases toward −xh, and the propagation time tp1 of the downward front at the nth neuron (xn(τ/2+tp1)=0) is obtained as (9)
dxn /dt = −xn − xh
tp1 = log {2[1 − exp(−τ/2)]}
(11)
dxn +1/dt = −xn +1 + fPL (xn ) = − xn +1 + fPL (−xh exp (−t + tp10 ) + x h ) = − xn +1 + x l + (xh − xl )exp(−t + tp10 )
of the n−1st neuron.
(x > 0) (x < 0)
(10)
Note that the propagation time tp1 depends on τ/2, during which the positive bump passes the n−1st neuron, and hence depends on the length l of the forward bump of the downward front. Next, we consider the propagation time of a wave front at the n+1st neuron in a nonmonotonic half cycle with the change in xn and its output f(xn)=fPL(xn) in Eq. (7). The state xn changes for t > 0 according to Eqs. (4) and (5) with xn(0)=−xh as above, and then an upward wave front passes the nth neuron at t=tp10=log2 (xn(tp10)=0). We then approximate the state xn+1(tp10) of the n +1st neuron in the negative bump at t=tp10 by −xl. The state xn+1 then increases toward xl with fPL(xn) in Eq. (7) for the output f(xn) of the nth neuron as
Fig. 5. Piecewise linear function fPL(x) in Eq. (5) with xl=0.85 and xh=2.07. Open circles: the steady points ( ± xh, ± fPL(xh)) (=( ± xh, ± xl)).
⎧ f (x ) = xh + (xl / xh − 1) x fPL (x ) = ⎨ + ⎩ f− (x ) = −xh + (xl / xh − 1) x
xn (τ/2 + tp1) = (xn (τ/2) + x h ) exp(− tp1) − x h=0
(7)
(t > tp10 )
(12)
The state xn+1(tp10+τ/2) at which a downward wave front passes the nth neuron (xn(tp10+τ/2) =0) is given by
Fig. 5 shows fPL(x) with xl=0.85 and xh=2.07, in which open circles denote the steady points ( ± xh, ± fPL(xh)) (=( ± xh, ± xl)) for the steady state of spatial period two. Since the state xn−1 increases xh monotonically after the passage of an upward wave front (xn−1=0) according to Eqs. (4) and (5), its output f(xn−1) increases to xh and then decreases to xl, which is shown in Fig. 4(a monotonic cycle). This decrease in f(xn−1) from xh to xl is taken into consideration in Eq. (7), i.e., the output fPL(xn−1) decreases from xh to xl as the state xn−1 increases from 0 to xh. This decrease is intrinsic for causing a nonmonotonic change in the state xn of the nth neuron.
xn +1 (tp10 + τ/2) = xl + [−2x l + (x h−x l )τ/2]exp(−τ/2)
( > 0)
(13)
The state xn changes for t > tp10+τ/2 as (14)
dxn /dt = −xn − xh xn (tp10 + τ/2 + t ) = xh exp (−t ) − x h
( < 0)
(15)
Then, the state xn+1 decreases toward −xl with fPL(xn) as
dxn +1/dt = −xn +1 + fPL (xn ) = − xn +1 − x l + (xl − xh )exp(−t + tp10 + τ/2)
3.2. Type-TR rotating wave
(t
> tp10 + τ/2)
(16)
xn +1 (tp10 + τ/2 + t ) = − xl + [xn +1 (tp10 + τ/2) + xl + (xl − xh ) t ]exp(−t )
We let the period of a rotating wave (the period of the oscillations of neurons) be τ. The period is related to the mean propagation time tpm of a wave front as τ=Ntpm in a type-TR and Type-SN rotating wave. The lengths of two bumps in a type-TR rotating wave are the same (l=N−l=N/2) as mentioned in Section 2 and thus the length of every half cycle is τ/2. Each wave front in the symmetric rotating wave passes a half (N/2) of the number of neurons in a half cycle (τ/2). In a type-TR rotating wave, the oscillations of adjacent neurons consist of different half cycles from each other: monotonic and nonmonotonic (Fig. 3(b)). We let xn take monotonic half cycles and let xn ± 1 take nonmonotonic half cycles. When the length of a bump (the number of neurons in a bump) and the length of a half cycle are large (l, τ/2 → ∞), the states xn and xn +1 of the adjacent neurons in the bump approach ( ± )xh and ( ± )xl in the steady state of spatial period two, respectively. (Positive (+) and negative (−) signs correspond to a positive and negative bump,
=−xl + {2xl + [−2xl + (xh − xl )τ/2]exp(−τ/2) + (xl − xh ) t} exp(−t ) (17) The propagation time tp2 of the downward front at the n+1st neuron (xn+1(tp10+τ/2+tp2)=0) is given by a solution to the following transcendental equation.
2 − [2 + (1 − xh / xl )τ/2]exp(−τ/2) + (1 − xh / xl ) tp2 = exp(tp2 )
(18)
The mean propagation time of a wave front is given by tpm1=(tp1+tp2)/2 since the states of adjacent neurons take monotonic and nonmonotonic half cycles alternately. The length l (=N/2) of a bump in a symmetric rotating wave is approximated by τ/(2tpm10), where tpm10=(tp10+tp20)/2 with tp10=log2 and tp20=log[2+(xh/ 152
Neurocomputing 225 (2017) 148–156
Y. Horikawa
downward front. Let an upward wave front with the forward negative bump pass the n−1st neuron at t=0 (xn−1(0)=0). We approximate the state xn(0) of the nth neuron in the negative bump at t=0 by –xl since xn takes a negative nonmonotonic half cycle and approaches −xl for t < 0. Then, the state xn increases toward xh according to Eqs. (4) and (5) in a monotonic half cycle so that the nth neuron comes into a positive bump of length l. Then, xn(τ/2) at which a downward wave front passes the n−1st neuron (xn−1(τ/2) =0) is given by
xn (τ/2) = −(xl + x h ) exp(−τ/2) + x h
( > 0,
τ > > 1)
(19)
The state xn−1 changes for t > τ/2 as
dxn −1/dt = −xn −1 − xh
(20)
(t > τ/2)
xn −1 (τ/2 + t ) = xh exp (−t ) − x h
( < 0)
(21)
Then, the state xn decreases toward –xl in a nonmonotonic half cycle with fPL(xn−1) in Eq. (7) for f(xn−1) as
dxn /dt = −xn + fPL (xn −1) = − xn − x l + (xl − xh )exp(−t )
(22)
xn (τ/2 + t ) = −xl + [xn (τ/2) + xl + (xl − xh ) t ]exp(−t ) =−xl + {(xl + xh )[1 − exp(−τ/2)] + (xl − xh ) t}exp(−t ) (23) The propagation time tp3 of the downward front at the nth neuron (xn(τ/2+tp3)=0) is given by a solution to the following transcendental equation.
(1 + xh / xl )[1 − exp(−τ/2)] + (1 − xh / xl ) tp3 = exp(tp3)
(24)
Next, we consider the propagation time tp4 of a downward front at the n+1st neuron. The state xn changes for t > 0 according to Eqs. (4) and (5) with xn(0)=−xl as above, and then an upward wave front passes the nth neuron at t=tp40=log(1+xl/xh) (xn(tp40) =0). We then approximate the state xn+1(tp40) of the n+1st neuron in the negative bump at t=tp40 by −xh. The state xn+1 then increases toward xl with fPL(xn) in Eq. (7) for f(xn) as
Fig. 6. Mean propagation time tpm1 of a type-TR wave front (a) and tpm2 of a type-PF wave front (b) vs the forward bump length l (=N/2).
xl−1)tp20] in the limit of τ → ∞. Then, the mean propagation time tpm1 of a wave front is obtained as a function of the forward bump length l. Although we considered a symmetric rotating wave with the equal bump lengths (l=N−l=N/2), this dependence of the mean propagation time on the forward bump length is applicable to an asymmetric rotating wave with the unequal bump lengths (l≠N−l) from its derivation. Fig. 6(a) shows the mean propagation time tpm1(=(tp1+tp2)/2) of a wave front against the forward bump length l (=N/2) calculated with Eqs. (11) and (18). The values of xh and xl are set to be 2.07 and 0.85, respectively, which are based on the states of neurons in the stable steady state of spatial period two in Eq. (1) with g=3.5. As the forward bump length increases, the propagation time increases to the maximum (≈0.5382) at l=lm1 ≈8.3 and decreases to tpm1(∞) (≈0.532). Thus, a symmetric type-TR rotating wave is stable when l (=N/2) > lm1, hence N > 16, according to Eq. (3). This condition for the stabilization of a type-TR rotating wave agrees qualitatively with that for Eq. (1), in which a stable type-TR rotating wave is generated for N≥8.
dxn +1/dt = −xn +1 + fPL (xn ) = − xn +1 + fPL (−xh exp (−t + tp 40 ) + x h ) = − xn +1 + x l + (xh − xl )exp(−t + tp 40 )
(t > tp 40 )
(25)
The state xn+1(tp40+τ/2) at which a downward wave front passes the nth neuron (xn(tp40+τ/2) =0) is obtained with xn+1(tp40)=−xh as
xn +1 (tp 40 + τ/2) = xl + [−(xh + x l ) + (x h−x l )τ/2]exp(−τ/2)
( > 0) (26)
Then, the state xn+1 decreases toward −xh as
dxn +1/dt = −xn +1 − xh
(t > tp 40 + τ/2)
xn +1 (tp 40 + τ/2 + t ) = −xh + [xn +1 (tp 40 + τ/2) + xh]exp(−t )
(27) (28)
Substituting Eq. (26) into xn+1(tp40+τ/2) in Eq. (28) and letting xn+1(tp40+τ/2+tp4)=0, we obtain the propagation time tp4 of the downward front at the n+1st neuron as
tp 4 = log {1 + xl / xh + [−(1 + xl / xh ) + (1 − xl / xh )τ/2]exp(−τ/2)}
(29)
The mean propagation time of a wave front is given by tpm2=(tp3+tp4)/2. The length l (=N/2) of a bump in a symmetric rotating wave is approximated by τ/(2tpm20), where tpm20=(tp30+tp40)/2 with tp30=log[1+xh/xl+(1−xh/xl)tp30] and tp40=log(1+xl/xh) in the limit of τ→∞. Fig. 6(b) shows the mean propagation time tpm2(=(tp3+tp4)/2) of a wave front against the forward bump length l (=N/2) calculated with Eqs. (24) and (29) (xh=2.07 and xl=0.85). As the forward bump length increases, the propagation time increases to the maximum (≈0.5811) at l=lm2≈9.7 and decreases to tpm2(∞) (≈0.5804). A symmetric type-PF rotating wave is stable when l (=N/2) > lm2, hence N > 18. The bump lengths
3.3. Type-PF rotating wave We consider a type-PF rotating wave, which also consists of two bumps with the equal lengths. The state of each neuron in a type-PF rotating wave takes monotonic and nonmonotonic half cycles alternately (Fig. 3(a)). We let xn take positive monotonic half cycles after the passage of an upward front and take negative nonmonotonic half cycles after the passage of a downward front. Then, the states xn ± 1 take positive nonmonotonic half cycles after the passage of an upward front and take negative monotonic half cycles after the passage of a 153
Neurocomputing 225 (2017) 148–156
Y. Horikawa
lm1 and lm2 at the maxima of the propagation times of wave fronts in a type-TR and type-PF rotating wave differ: lm1 (≈8.3) < lm2 (≈9.7). Thus, a type-TR rotating wave is stabilized in a ring with a smaller number of neurons than a type-PF rotating wave. This condition agrees qualitatively with that for Eq. (1), in which a stable type-TR rotating wave is generated for N≥8 while a stable type-PF rotating wave is generated for N≥14. 3.4. Type-SN rotating wave When the number of neurons is odd (N=2M+1), a stable rotating wave is generated through a saddle-node bifurcation (a type-SN rotating wave) as shown in Fig. 2(b). In a type-SN rotating wave, one period of the oscillation in the state of a neuron consists of a monotonic and nonmonotonic cycle, i.e., two rotations of a wave front in a ring. Each neuron takes a monotonic and nonmonotonic cycle alternately, and adjacent neurons take these two cycles alternately as a wave front passes. Consequently, two wave fronts take different time courses as they pass neurons. The states of all neuron change as ± xh→∓xl or ± xl→∓xh at the passage of one wave front while the states of all neuron change as ∓xh→ ± xh or ∓xl→ ± xl at the passage of the other wave front. In Fig. 3(c), for instance, the states of neurons change as −xh→xl or −xl→xh at the passage of an upward wave front while the states of neurons change as xh→−xh or xl→−xl at the passage of a downward wave front That is, one wave front takes the same time course as a wave front in a type-TR rotating wave, while the other front takes the same time course as a wave front in a type-PF rotating wave. We refer to this two kinds of wave fronts as a type-TR and type-PF wave front. We give a qualitative explanation as to why a stable type-SN rotating wave exists only in the restricted range of g as shown in Fig. 2(b). This restriction of its existence is attributed to the difference between the propagation times of a type-TR and type-PF wave front. That is, the propagation times of two wave fronts must be the same in a periodic rotating wave. If the propagation times of a type-TR and typePF wave front, which depend on the lengths of the forward bumps, differ from each other, a type-SN rotating wave cannot exist. Fig. 7(a) shows the propagation times of wave fronts in a type-TR and type-PF rotating wave in Eq. (1) with N=14 against the output gain g of neurons. The propagation times decreases as the output gain increases, and the propagation time of a type-TR wave front (a solid line) becomes smaller than that of a type-PF wave front (a dashed line). This difference between the propagation times of two wave fronts increases as the output gain increases, which results in the disappearance of a type-SN rotating wave. Since the propagation times of wave fronts in a type-TR and typePF rotating wave derived in Sections 3.2 and 3.3 are based on qualitative piecewise linear output functions of neurons, the results are not applicable directly. That is, the absolute values of the propagation times of wave fronts in two types of rotating waves do not overlap with each other, e.g., tpm1(lm1) (≈0.5382) < tpm2(∞) (≈0.5804). However, the disappearance of a type-SN rotating wave with an increase in g can be shown through an increase in the ratio xh/xl of the values of the states of neurons in the steady state of spatial period two. Fig. 7(b) shows the ratio xh/xl in the steady states of neurons in Eq. (1) against the output gain g of neurons. The ratio xh/xl increases from 1 to 3 as g increases from e to 4. The propagation times tpm1(∞) and tpm2(∞) of a type-TR and type-PF wave front, respectively, for τ→ ∞ derived in Sections 3.2 and 3.3 are functions of xh/xl. Then, Fig. 7(c) shows the propagation times tpm1(∞) and tpm2(∞) of the wave fronts against the ratio xh/xl. Both propagation times decrease and their difference tpm2(∞)−tpm1(∞) increases as xh/xl increases, hence as g increases. Fig. 8 shows changes in the propagation times tpm1(l) and tpm2(l) of the wave fronts due to an increase in g and xh/xl schematically. For the sake of simplicity, the propagation time tpm2 of a type-PF wave front is fixed (a solid line), and the propagation time tpm1 of a type-TR
Fig. 7. Propagation times tp of wave fronts in a type-TR (a solid line) and type-PF rotating wave (a dashed line) in Eq. (1) with N =14 vs the output gain g (a). Ratio xh/xl of the states of neurons in the steady states of spatial period two in Eq. (1) vs the output gain g (b). Propagation times tpm1(∞) of a type-TR wave front (a solid line) and tpm2(∞) of a type-PF wave front (a dashed line) vs the ratio xh/xl.
wave front is shifted downward (dashed, dotted and dash-dotted lines) as g increases. Also, the values lm1 and lm2 at the maximal points are fixed although they actually decrease as g increases. Open circles show the points of the minimum lengths of the forward bumps of a type-TR and type-PF wave front in a type-SN rotating wave for each tpm1 curve. As g increases, the maximum tpm1(lm1) of a type-TR wave front becomes smaller than the asymptotic value tpm2(∞) of a type-PF wave
154
Neurocomputing 225 (2017) 148–156
Y. Horikawa
occurs with other kinds of nonmonotonic output functions of neurons, e.g., f(x)=gxexp(−|x|) or f(x)=gx(1−x2) (g > 0). Rotating waves are stabilized through bifurcations of symmetric rotating waves in a similar manner when the number of neurons is even. However, the stabilization of rotating waves has not been observed with a discontinuous nonmonotonic function: f(x)=gsgn(x)exp(−|x|). The dependence of the occurrence of the stabilization of rotating waves on the form of the output function of neurons should be examined. Finally it should be noted that an increase in the absolute value of the state of a neuron just before the end of a nonmonotonic half cycle, which can be seen in Figs. 3 and 4, is not intrinsic for the stabilization of a rotating wave. In a positive half cycle, for instance, this increase in the state xn of the nth neuron is caused by an increase in the output f(xn−1) of the n−1st neuron due to a decrease in xn−1 from xh to xl. A kinematical equation in which this increase is taken into consideration can be made and it can be shown that the propagation time of a wave front becomes larger but its nonmonotonic dependence on the forward bump length is qualitatively the same.
Fig. 8. Schematic of changes in the propagation time tpm1(l) of a type-TR wave front (dashed, dotted and dash-dotted lines) with an increase in g and the propagation time tpm2(l) of a type-PF wave front (a solid line).
5. Conclusion front for l→∞. Then, a stable type-SN rotating wave cannot exist. Further, as mentioned in Section 2, it can be shown with computer simulation that the lengths of two bumps in a type-SN rotating wave differ from each other and that a type-PF wave front has a longer forward bump than a type-TR wave front. In computer simulation, the length of the forward bump of a type-PF wave front becomes longer as g increases, which agrees with the changes in Fig. 8.
Rotating waves in a ring of unidirectionally coupled neurons with the nonmonotonic output function were examined. It was shown that rotating waves are stabilized as the nonmonotonicity in the output function of neurons increases. The bifurcations of rotating waves and the waveforms of stabilized rotating waves depended on whether the number of neurons is even or odd. When the number of neurons was even, two types of rotating waves were stabilized through pitchfork bifurcations and the Neimark-Sacker bifurcation. When the number of neurons was odd, a stable rotating was generated through a saddlenode bifurcation. The propagation times of wave fronts in the stabilized rotating waves were derived by using the piecewise linear output functions of neurons, which were based on changes in the states of neurons obtained with computer simulation. The propagation time of a wave front depended on the length of the forward bump of the wave front and changed nonmonotonically as the forward bump length increased. The kinematical equations for the propagation of wave fronts showed that rotating waves are stabilized owing to this nonmonotonicity of the propagation time, which resulted from the nonmonotonicity in the output function of neurons. In this study, we restricted ourselves to a ring of neurons with simple unidirectional coupling. The effects of nonmonotonicity in the output function of neurons on metastable dynamical rotating waves in the presence of asymmetric bidirectional coupling and self-coupling are one of future areas of interest. Further, future work is needed to examine the effects of asymmetry in nonmonotonic output functions of neurons on the dynamical metastability and stabilization of rotating waves.
4. Discussion The stabilization of rotating waves is caused by an overshoot of the state of a neuron after the passage of a wave front in a nonmonotonic half cycle. After an upward front passes a neuron, for instance, the state of the neuron increases and overshoots the steady state. While converging to the steady state after the overshoot, the state of the neuron decreases until a downward front comes to the neuron. The time interval during which the state of a neuron decreases increases as the length of the forward bump of the downward front increases. Then, the propagation time of the downward front at the neuron decreases as its forward bump length increases since the state of the neuron decreases. Thus, the propagation time of a wave front with the longer forward bump is smaller than that of the other wave front with the shorter forward bump, so that the longer (shorter) bump decreases (increases). This results in the stabilization of a symmetric rotating wave with the bumps of equal lengths. Such stabilization of rotating waves is similar to that shown in a ring of unidirectionally coupled sigmoidal neurons with inertias, in which a damped oscillation in the state of a neuron due to the inertia makes an overshoot [37]. Then, there are the time intervals in which the state of a neuron decreases when converging to the steady state. When the time during which the forward bump of a wave front passes a neuron lies in these intervals, the propagation time of the wave front at the neuron decreases as the forward bump length increases. This change in the propagation time works to stabilize rotating waves. Similar stabilization of rotating waves has also been shown in a discrete-time system with coupled circle maps [38]. There are a pair of stable spatially uniform steady states, and each steady state undergoes a period doubling bifurcation as the strength of coupling increases. Then, a pair of stable states with the temporal and spatial period two is generated. In a rotating wave, the states of neurons in bumps also oscillate with period two and the oscillations work as overshoots. Consequently, the dependence of the propagation speed of a wave front on the bump length becomes nonmonotonic. The stabilization of rotating waves due to nonmonotonicity in the output function of neurons is not specific for the Gaussian-type function employed in this study. It can be shown that the stabilization
References [1] S. Amari, Mathematics of Neural Networks (in Japanese), Sangyo-Tosho, Tokyo, 1978. [2] T. Gedeon, Cyclic Feedback Systems. Memoirs of the American Mathematical Society, American Mathematical Society, Providence, 1998. [3] A.F. Atiya, P. Baldi, Oscillations and synchronizations in neural networks: an exploration of the labeling hypothesis, Int. J. Neural Syst. 1 (1989) 103–124. [4] M.H. Hirsh, Convergent activation dynamics in continuous time networks, Neural Netw. 2 (1989) 331–340. [5] M. Golubitsky, I. Stewart, D.G. Schaeffer, Singularities and Groups in Bifurcation Theory II, Springer, New York, 1988. [6] J.J. Collins, I. Stewart, A group-theoretic approach to rings of coupled biological oscillators, Biol. Cybern. 71 (1994) 95–103. [7] P.C. Bressloff, S. Coombes, B. de Souza, Dynamics of a ring of pulse-coupled oscillators: Group theoretic approach, Phys. Rev. Lett. 79 (1997) 2791–2794. [8] P.-L. Buono, M. Golubitsky, A. Palacios, Heteroclinic cycles in rings of coupled cells, Physica D 143 (2000) 74–108. [9] P. Thiran, Dynamics and Self-organization of Locally Coupled Neural Networks, Presses Polytechniques et Universitaires Romandes, Lausanne, 1997.
155
Neurocomputing 225 (2017) 148–156
Y. Horikawa
pp. 37–64. [29] C.E. Schreiner, B.J. Malone, Representation of loudness in the auditory cortex, in: G.G. Celesia, G. Hickok (Eds.), The Human Auditory System: Fundamental Organization and Clinical Disorders, Elsevier, Amsterdam, 2015, pp. 73–84. [30] M. Morita, S. Yoshizawa, K. Nakano, Analysis and improvement of the dynamics of autocorrelation associative memory (in Japanese) (Japanese ed.), IEICE Trans. Inf. Syst. J37-D-II (1990) 232–242. [31] M. Morita, Associative memory with nonmonotone dynamics, Neural Netw. 6 (1993) 115–126. [32] S. Yoshizawa, M. Morita, S. Amari, Capacity of associative memory using a nonmonotonic neuron model, Neural Netw. 6 (1993) 167–176. [33] G. Boffetta, R. Monasson, R. Zecchina, Symmetry breaking in non-monotonic neural networks, J. Phys. A 26 (1993) L507–L513. [34] N. Brunel, R. Zecchina, Response functions improving performance in attractor neural networks, Phys. Rev. E 49 (1994) R1823–R1826. [35] H. Zhang, Z. Wang, D. Liu, A Comprehensive review of stability analysis of continuous-time recurrent neural networks, IEEE Trans. Neural Netw. Learn. Syst. 25 (2014) 1229–1262. [36] E.J. Doedel, AUTO: Software for Continuation and Bifurcation Problems in Ordinary Differential Equations, Concordia University, Montreal, Canada, 1996 〈http://indy.cs.concordia.ca/auto/〉. [37] Y. Horikawa, H. Kitajima, Bifurcation and stabilization of oscillations in ring neural networks with inertia, Physica D 238 (2009) 2409–2418. [38] Y. Horikawa, H. Kitajima, Exponential transient rotating waves in a bistable ring of unidirectionally coupled maps, Physica D 241 (2012) 106–114.
[10] P. Thiran, G. Setti, M. Hasler, An approach to information propagation in 1-D cellular neural networks-part I: local diffusion, IEEE Trans. Circuits Syst. I 45 (1998) 777–789. [11] G. Setti, P. Thiran, C. Serpico, An approach to information propagation in 1-D cellular neural networks-part II: global propagation, IEEE Trans. Circuits Syst. I 45 (1998) 790–811. [12] M. Di Marco, M. Forti, M. Grazzini, L. Pancioni, Convergence of a class of cooperative standard cellular neural network arrays, IEEE Trans. Circuits Syst. I: Regul. Pap. 59 (2012) 772–783. [13] E.K. Blum, X. Wang, Stability of fixed points and periodic orbits and bifurcations in analog neural networks, Neural Netw. 5 (1992) 577–587. [14] F. Pasemann, Characterization of periodic attractors in neural ring networks, Neural Netw. 8 (1995) 421–429. [15] J. Wei, M. Velarde, Bifurcation analysis and existence of periodic solutions in a simple neural network with delays, Chaos 14 (2004) 940–953. [16] J. Wei, M.Y. Li, Global existence of periodic solutions in a tri-neuron network model with delays, Physica D 198 (2004) 109–119. [17] X. Li, J. Wei, Stability and bifurcation analysis on a delayed neural network model, Int. J. Bifurc. Chaos 15 (2005) 2883–2893. [18] J. Wei, C. Zhang, Bifurcation analysis of a class of neural networks with delays, Nonlinear Anal.: Real World Appl. 9 (2008), 2008, pp. 2234–2252. [19] S. Guo, L. Huang, Pattern formation and continuation in a trineuron ring with delays, Acta Math. Sin. 23 (2007) 799–818. [20] K.L. Babcock, R.M. Westervelt, Dynamics of simple electronic neural networks, Physica D 28 (1987) 305–316. [21] P. Baldi, A.F. Atiya, How delays affect neural dynamics and learning, IEEE Trans. Neural Netw. 5 (1994) 612–621. [22] K. Pakdaman, C.P. Malta, C. Grotta-Ragazzo, O. Arino, J.-F. Vibert, Transient oscillations in continuous-time excitatory ring neural networks with delay, Phys. Rev. E 55 (1997) 3234–3248. [23] Y. Horikawa, H. Kitajima, Duration of transient oscillations in ring networks of unidirectionally coupled neurons, Physica D 238 (2009) 216–225. [24] M. Forti, B. Garay, M. Koller, L. Pancioni, Long transient oscillations in a class of cooperative cellular neural networks, Int. J. Circ. Theory Appl. 43 (2015) 635–655. [25] M. Di Marco, M. Forti, B.M. Garay, M. Koller, L. Pancioni, Floquet multipliers of a metastable rotating wave in a Chua–Yang ring network, J. Math. Anal. Appl. 434 (2016) 798–836. [26] B.E. Pfingst, T.A. O'Connor, Characteristics of neurons in auditory cortex of monkeys performing a simple auditory task, J. Neurophysiol. 45 (1981) 16–34. [27] P.X. Joris, C.E. Schreiner, A. Rees, Neural processing of amplitude-modulated sounds, Physiol. Rev. 84 (2004) 541–577. [28] F.E. Musiek, J.A. Weihing, V.B. Oxholm, Anatomy and physiology of the central auditory nervous system: A clinical perspective, in: R.J. Roeser, M. Valente, H. Hosford-Dunn (Eds.), Audiology: Diagnosis2nd ed., Thieme, New York, 2007,
Yo Horikawa is a professor in the Faculty of Engineering at Kagawa University, Japan. He has received B. Eng., M. Eng. and Ph.D. (Eng.) degrees in mathematical engineering and information physics from the University of Tokyo in 1983, 1985 and 1994, respectively. His research interests include nonlinear dynamical systems and statistical pattern recognition.
156