- Email: [email protected]
Multistability of competitive neural networks with time-varying and distributed delays
Nonlinear Analysis: Real World Applications 10 (2009) 928–942 www.elsevier.com/locate/nonrwa
Multistability of competitive neural networks with time-varying and distributed delays Xiaobing Nie ∗ , Jinde Cao Department of Mathematics, Southeast University, Nanjing 210096, China Received 24 September 2007; accepted 16 November 2007
Abstract In this paper, with two classes of general activation functions, we investigate the multistability of competitive neural networks with time-varying and distributed delays. By formulating parameter conditions and using inequality technique, several novel delayindependent sufficient conditions ensuring the existence of 3 N equilibria and exponential stability of 2 N equilibria are derived. In addition, estimations of positively invariant sets and basins of attraction for these stable equilibria are obtained. Two examples are given to show the effectiveness of our theory. c 2007 Elsevier Ltd. All rights reserved.
Keywords: Competitive neural networks; Time-varying delays; Distributed delays; Multistability
1. Introduction It is well known that neural networks play important roles in many applications, such as classification, associative memory, image processing, pattern recognition, parallel computation, optimization problem, and decision making etc [1–4]. The theory on the dynamics of the networks has been developed according to the purposes of the applications. On the one hand, in the applications to parallel computation and optimization problem, the existence of a computable solution for all possible initial states is the best situation. Mathematically, this means that an equilibrium of the networks exists and any state in the neighborhood converges to the equilibrium, which is called “monostability” of networks. On the other hand, existence of many equilibria is a necessary feature in the applications of neural networks to associative memory storage, pattern recognition, and decision making. The notion of “multistability” of networks describes coexistence of multiple stable patterns such as equilibria or periodic orbits. Competitive neural networks (CNNs) with different time scales are proposed in [5], which model the dynamics of cortical cognitive maps with unsupervised synaptic modifications. In this model, there are two types of state variables, that of the short-term memory (STM) describing the fast neural activity and that of the long-term memory (LTM) describing the slow unsupervised synaptic modifications. A typical form for multitime scale CNNs with time-varying ∗ Corresponding author.
E-mail address: [email protected] (X. Nie). c 2007 Elsevier Ltd. All rights reserved. 1468-1218/$ - see front matter doi:10.1016/j.nonrwa.2007.11.014
X. Nie, J. Cao / Nonlinear Analysis: Real World Applications 10 (2009) 928–942
929
and distributed delays is described by the following functional differential equations N N X X STM : dxi (t) = −ai xi (t) + D f (x (t)) + Diτj f j (x j (t − τi j (t))) ij j j dt j=1 j=1 Z t N P X X + D K (t − s) f (x (s)) ds + B m i j (t)y j + Ii , i j i j j j i −∞ j=1 j=1 dm i j (t) = −m i j (t) + y j f i (xi (t)), i = 1, 2, . . . , N , j = 1, 2, . . . , P, LTM : dt
(1)
where xi (t) is the neuron current activity level, ai > 0 is the time constant of the neuron, f j (x j (t)) is the output of neurons, m i j (t) is the synaptic efficiency, y j is the constant external stimulus, Di j represents the connection weight between the ith neuron and the jth neuron, Bi is the strength of the external stimulus, is the time scale of STM state, Diτj and D i j represent the synaptic weight of delayed feedback, Ii is the constant input, τi j (t) corresponds to the transmission delay and satisfies 0 < τi j (t) < τi j (τi j is a positive constant). PP T T After setting Si (t) = j=1 m i j (t)y j = y mi (t), where y = (y1 , y2 , . . . , y P ) , mi (t) = (m i1 (t), m i2 (t), . . . , m i P (t))T , and summing up the LTM over j, the networks (1) can be rewritten in the following form N N X X dxi (t) STM : = −a x (t) + D f (x (t)) + Diτj f j (x j (t − τi j (t))) i i i j j j dt j=1 j=1 Z t N X Di j + K i j (t − s) f j (x j (s)) ds + Bi Si (t) + Ii , −∞ j=1 dSi (t) = −Si (t) + |y|2 f i (xi (t)), i = 1, 2, . . . , N , LTM : dt
(2)
where |y|2 = y12 + · · · + y 2P is a constant, without loss of generality, the input stimulus y is assumed to be normalized with unit magnitude |y|2 = 1, and the fast time-scale parameter is also assumed to be unit, then the above networks are simplified as N N X X dxi (t) STM : = −a x (t) + D f (x (t)) + Diτj f j (x j (t − τi j (t))) i i i j j j dt j=1 j=1 Z t N X + Di j K i j (t − s) f j (x j (s)) ds + Bi Si (t) + Ii , −∞ j=1 dSi (t) = −Si (t) + f i (xi (t)), i = 1, 2, . . . , N , LTM : dt
(3)
where the delay kernels K i j (s) : [0, +∞) → [0, +∞) are piecewise continuous integral functions, and they satisfy Z +∞ Z +∞ K i j (s)ds = 1, K i j (s) eµs ds < +∞ (4) 0
0
for some positive constant µ and i, j = 1, 2, . . . , N . In the past few years, the monostability analysis of neural networks with time-varying and/or distributed delays has been developed [6–13]. In particular, the theory of unique equilibrium point and global convergence to the equilibrium point for CNNs with their various generalizations has been extensively studied, see [5,14–17]. Recently, the multistability analysis of neural networks has attracted the attention of many researchers [18–21]. In [18], with unsaturated piecewise linear activation function f (x) = max{0, x}, the multistability of system (3) without delay, that is Diτj = D i j = 0 (i, j = 1, 2, . . . , N ), was investigated by using local inhibition and constructing an energy-like function. The result therein strongly relies on the piecewise linearity and unsaturation of activation functions. In [19, 20], the multistability of neural networks without and with constant delays was studied by formulating parameter conditions. Inspired by [19,20], in this paper, we shall study the multistability of system (3) with two classes of general activation functions. Firstly, we derive conditions ensuring the existence of 3 N equilibria for system (3) with
930
X. Nie, J. Cao / Nonlinear Analysis: Real World Applications 10 (2009) 928–942
two classes of general activation functions, through constructing parameter conditions by a geometrical observation. Secondly, we establish a series of new criteria on the exponential stability of 2 N equilibria for the networks above by means of inequality technique. In addition, estimations of positively invariant sets and basins of attraction for these stable stationary solutions are derived. This paper is organized as follows. In Section 2, we consider two classes of activation functions which are commonly employed in neural networks. We then obtain conditions for the existence of 3 N equilibria. In Section 3, we show that, with additional conditions, there are 2 N regions in R2N , which are positively invariant under the flow generated by system (3). Subsequently, it is argued that these 2 N equilibria are exponentially stable. Two numerical simulations are given to illustrate our theory and distinct dynamical behaviors for different activation functions in Section 4. Finally, concluding remarks are summarized in Section 5. 2. Activation functions and multiple equilibria The initial conditions associated with system (3) are of the form xi (t) = φi (t), t ∈ (−∞, 0], Si (t) = ψi (t) ≡ ψi (0), t ∈ (−∞, 0]. For convenience, we introduce two notations. For any u(t) = (u 1 (t), u 2 (t), . . . , u N (t))T ∈ R N , define #q/ p " N X p/q , p ≥ q > 0. ku(t)k = |u i (t)| i=1
For any φ(s) = (φ1 (s), φ2 (s), . . . , φ N (s))T ∈ R N , s ∈ (−∞, 0], define " #q/ p N X p/q kφk = sup |φi (s)| , p ≥ q > 0, s∈(−∞,0] i=1
where φi (s) (i = 1, 2, . . . , N ) are continuous and bounded functions on (−∞, 0]. To prove our results, the following two lemmas are necessary. Lemma 1 ([22]). Let a, b ≥ 0, s ≥ 1, then a s−1 b ≤
s−1 s 1 s a + b . s s
Lemma 2 (C p Inequality [22]). Let a, b ≥ 0, p ≥ q > 0, then (a + b)q/ p ≤ a q/ p + bq/ p . In this paper, we shall consider the following two classes of activation functions f i for system (3): u i ≤ f i (ξ ) ≤ vi , f i0 (ξ ) > 0, 2 class A : f i ∈ C , 00 (ξ − σi ) f i (ξ ) < 0, for all ξ ∈ R \ {σi }, if − ∞ < ξ < pi , u i , increasing, if pi ≤ ξ ≤ qi , class B : f i ∈ C, vi , if qi < ξ < ∞, where u i , vi , pi , qi and σi are constants with u i < vi , pi < qi . Remark 1. Class A contains general bounded smooth sigmoidal functions, for example, functions 1+e1 −ξ , tanh ξ, arctan ξ are special cases of class A. Class B consists of nondecreasing functions with saturations, including the piecewise linear functions with two corner points at pi , qi : f i (ξ ) = u i +
vi − u i (ξ − pi ), qi − pi
(5)
931
X. Nie, J. Cao / Nonlinear Analysis: Real World Applications 10 (2009) 928–942
Fig. 1. The configurations of (a) smooth sigmoidal activation functions in class A and (b) saturated activation functions in class B.
and, in particular, the standard piecewise linear function: f i (ξ ) =
1 (|ξ + 1| − |ξ − 1|). 2
(6)
Typical configurations of these functions are depicted in Fig. 1. Note that the stationary equations for system (3) are −ai xi +
N X (Di j + Diτj + D i j ) f j (x j ) + Bi Si + Ii = 0,
(7)
j=1
−Si + f i (xi ) = 0,
i = 1, 2, . . . , N .
Equivalently, the above equations can be rewritten as G i (X ) := −ai xi +
N X (Di j + Diτj + D i j ) f j (x j ) + Bi f i (xi ) + Ii = 0,
(8)
j=1
−Si + f i (xi ) = 0,
i = 1, 2, . . . , N ,
where X = (x1 , x2 , . . . , x N )T . Now, define a map from R to R as gi (ξ ) := −ai ξ + (Dii + Diiτ + D ii + Bi ) f i (ξ ),
i = 1, 2, . . . , N .
Let us propose the first parameter condition for system (3) with activation functions in class A: (HA 1)
0 = inf f i0 (ξ ) < ξ ∈R
ai Dii +
Diiτ
+ D ii + Bi
< max f i0 (ξ ) = f i0 (σi ), ξ ∈R
i = 1, 2, . . . , N .
Lemma 3. There exist two points p˜ i , q˜i with p˜ i < σi < q˜i such that gi0 ( p˜ i ) = gi0 (q˜i ) = 0, i = 1, 2, . . . , N , under assumption (HA 1 ), for activation functions of class A. Proof. For each i, since gi0 (ξ ) = −ai + (Dii + Diiτ + D ii + Bi ) f i0 (ξ ), we have gi0 (ξ ) = 0 if and only if ai f i0 (ξ ) = . Consider the following functions τ Dii +Dii +D ii +Bi
Fi (ξ ) := f i0 (ξ ) −
ai Dii +
Diiτ
+ D ii + Bi
,
i = 1, 2, . . . , N .
0 It follows from (HA 1 ) and (9) that Fi (σi ) = f i (σi ) −
Moreover, noting that
limξ →±∞ f i0 (ξ )
ai Dii +Diiτ +D ii +Bi
(9) > 0 and Fi (ξ ) are continuous for all ξ ∈ R.
= 0, we have limξ →±∞ Fi (ξ ) = −
points p˜ i , q˜i with p˜ i < σi < q˜i such that
gi0 ( p˜ i )
=
gi0 (q˜i )
ai Dii +Diiτ +D ii +Bi
= 0. This completes the proof.
< 0. Thus, there exist two
932
X. Nie, J. Cao / Nonlinear Analysis: Real World Applications 10 (2009) 928–942
Fig. 2. The graphs of gˆi and gˇi induced from the activation function of class A.
Fig. 3. (a) The graphs of gˆi and gˇi induced from the activation function of class B. (b) The graphs of gˆi and gˇi induced from the standard activation function.
In the following, we define, for i = 1, 2, . . . , N , gˆi (ξ ) = −ai ξ + (Dii + Diiτ + D ii + Bi ) f i (ξ ) + ki+ , gˇi (ξ ) = −ai ξ + (Dii + Diiτ + D ii + Bi ) f i (ξ ) + ki− , P P where ki+ = Nj=1, j6=i ρ j |Di j +Diτj +D i j |+Ii , ki− = − Nj=1, j6=i ρ j |Di j +Diτj +D i j |+Ii with ρ j = max{|u j |, |v j |}. It follows that gˇi (xi ) ≤ G i (X ) ≤ gˆi (xi ) holds for all i = 1, 2, . . . , N , since | f i (xi )| ≤ max{|u i |, |vi |}. We consider the second (resp. first) parameter condition which is concerned with the existence of multiple equilibria for system (3) with activation functions in class A (resp. B): (HA 2 ) gˆ i ( p˜ i ) < 0, gˇ i (q˜i ) > 0, i = 1, 2, . . . , N . (HB 1 ) gˆ i ( pi ) < 0, gˇ i (qi ) > 0, i.e., −ai pi + (Dii + Diiτ + D ii + Bi )u i + ki+ < 0, −ai qi + (Dii +
Diiτ
+ D ii +
Bi )vi + ki−
> 0,
i = 1, 2, . . . , N , i = 1, 2, . . . , N .
B The configurations that motivates (HA ˆ i < bˆi < cˆi such 2 ) and (H1 ) are depicted in Figs. 2 and 3. There exist points a ˆ ˇ ˇ that gˆi (aˆ i ) = gˆi (bi ) = gˆi (cˆi ) = 0 as well as points aˇ i < bi < cˇi such that gˇi (aˇ i ) = gˇi (bi ) = gˇi (cˇi ) = 0, under A assumptions (HA 1 ) and (H2 ), for activation functions of class A. The same conclusion holds for activation functions of class B, under condition (HB 1 ).
Theorem 1. There exist 3 N equilibria for system (3) with activation functions of class A, under conditions (HA 1 ) and B ). (HA ). The same conclusion holds for system (3) with activation functions of class B, under condition (H 2 1 Proof. We prove only the case of class A, and similar arguments can be employed to prove the case of class B. A The equilibria of system (3) are zeros of (8). Under conditions (HA 1 ) and (H2 ), the graphs of gˆ i and gˇ i defined N above are depicted in Fig. 2. According to the configuration, there are 3 disjoint closed regions in R N . Set Ω α = {(x1 , x2 , . . . , x N )T ∈ R N |xi ∈ Ωiαi } with α = (α1 , α2 , . . . , α N )T , and αi = “l”, “m”, or “r”, where Ωil = [aˇ i , aˆ i ],
Ωim = [bˆi , bˇi ],
Ωir = [cˇi , cˆi ].
(10)
X. Nie, J. Cao / Nonlinear Analysis: Real World Applications 10 (2009) 928–942
933
Herein, “l”, “m”, “r” means, respectively, “left”, “middle” and “right”. Consider any fixed one of these regions Ω α . For a given x˜ = (x˜1 , x˜2 , . . . , x˜ N )T ∈ Ω α , we solve h i (xi ) := −ai xi + (Dii + Diiτ + D ii + Bi ) f i (xi ) +
N X
(Di j + Diτj + D i j ) f j (x˜ j ) + Ii = 0
j=1, j6=i
for i = 1, 2, . . . , N . Note that the graph of h i is a vertical shift of the graph of gˆi or gˇi and lies between the graphs of gˆi and gˇi , thus, one can always find three solutions xi , and each of them lies in one of the regions in (10) for each i. Let us pick the one lying in Ωiαi and set it as x i for each i. We now define a map Hα : Ω α → Ω α by Hα (x) ˜ = (x) = (x 1 , x 2 , . . . , x N )T . Restated, we set x i = (h i |Ω l )−1 (0),
if αi = “l”,
x i = (h i |Ωim )−1 (0),
if αi = “m”,
i
x i = (h i |Ωir )
−1
(0),
if αi = “r”.
Since f i is continuous and h i is a vertical shift of function ξ 7→ −ai ξ + (Dii + Diiτ + D ii + Bi ) f i (ξ ) by the PN τ quantity j=1, j6=i (Di j + Di j + D i j ) f j ( x˜ j ) + Ii , the map Hα is continuous. It follows from Brouwer’s fixed point theorem that there exists one fixed point x¯ = (x¯1 , x¯2 , . . . , x¯ N )T of Hα in Ω α . Thus, we also obtain a zero (x¯1 , x¯2 , . . . , x¯ N , f 1 (x¯1 ), f 2 (x¯2 ), . . . , f N (x¯ N ))T of (8). Consequently, there exist 3 N zeros of (8), hence 3 N equilibria for system (3), and each of them lies in one of the 3 N regions Ω α . 3. Stability of multiple equilibria In this section, we shall give some positively invariant sets for system (3) and investigate stability of the equilibrium point in each invariant set. As a result, we also obtain a basin of attraction for each of the exponentially stable equilibria. Firstly, we give the third condition for system (3) with activation functions in class A and the second condition for system (3) with activation functions in class B: (HA 3 ) There exist constants p ≥ q > 0, µi > 0 (i = 1, 2, . . . , 2N ) such that " # N N X X µi pai − ( p − q) (|Di j | + |Diτj | + |D i j |) − ( p − q) − q (|Di j | + |Diτj | j=1 p/q
+ |D i j |)µ j η j p/q
−µi ηi
j=1
− qµ N +i |Bi | p/q > 0,
+ µ N +i > 0,
(11)
i = 1, 2, . . . , N
hold for all i = 1, 2, . . . , N , where η j , j = 1, 2, . . . , N are real numbers satisfying max{ f j0 (ξ )|ξ = cˇ j , aˆ j } < η j < min{ f j0 (ξ )|ξ = p˜ j , q˜ j }. (HB 2 ) There exist constants p ≥ q > 0, µi > 0 (i = 1, 2, . . . , 2N ) such that µi [ pai − ( p − q)] − qµ N +i |Bi | p/q > 0,
i = 1, 2, . . . , N .
(12)
For activation functions f j (·) in class A, we define d j = min{ξ | f j0 (ξ ) = η j }, d¯ j = max{ξ | f j0 (ξ ) = η j }. Then we have aˆ j < d j < p˜ j , q˜ j < d¯ j < cˇ j . For activation functions f j (·) in class B, we define, respectively, d j = p j , d¯ j = q j . For system (3), consider the following 2 N subsets of C((−∞, 0], R2N ). Let α = (α1 , α2 , . . . , α N )T with αi = “l” or “r”, and set Λα = {(φ1 , φ2 , . . . , φ N , ψ1 , ψ2 , . . . , ψ N )T |(φi , ψi )T ∈ Λli if αi = “l”, (φi , ψi )T ∈ Λri if αi = “r”}, where Λli = {(φi , ψi )T ∈ C((−∞, 0], R2 )|φi (θ ) ≤ d i , ψi (θ ) ≡ ψi (0) ≤ f i (d i ), ∀θ ∈ (−∞, 0]}, Λri = {(φi , ψi )T ∈ C((−∞, 0], R2 )|φi (θ ) ≥ d¯i , ψi (θ ) ≡ ψi (0) ≥ f i (d¯i ), ∀θ ∈ (−∞, 0]}.
(13)
934
X. Nie, J. Cao / Nonlinear Analysis: Real World Applications 10 (2009) 928–942
τ A α Theorem 2. Assume that (HA 1 ), (H2 ) and Dii ≥ 0, D ii ≥ 0, Bi ≥ 0 hold. Then each Λ is positively invariant with respect to the solution flow generalized by system (3) with activation functions of class A. The same conclusion holds τ for activation functions of class B, under conditions (HB 1 ), Dii ≥ 0, D ii ≥ 0 and Bi ≥ 0.
Proof. We only prove the case A, the proof of case B is similar to that in case A and here is omitted. Consider any one of the 2 N subsets Λα . For any initial condition (φ T , ψ T )T = (φ1 , φ2 , . . . , φ N , ψ1 , ψ2 , . . . , φ N )T ∈ Λα , we claim that the solution (x T (t; φ), S T (t; ψ))T remains in Λα for all t ≥ 0. If this is not true, without loss of generality, there exist a component xi (t) of x(t; φ) which is the first (or one of the first) escaping from Λli or Λri . Restated, there exist ∗ ∗ i ¯ ¯ some i and t ∗ such that either xi (t ∗ ) = d¯i , dx dt (t ) < 0, x i (t) ≥ di for −∞ < t ≤ t and S j (t; ψ) ≥ f j (d j ) for all dxi ∗ ∗ ∗ ∗ j = 1, 2, . . . , N and −∞ < t ≤ t or xi (t ) = d i , dt (t ) > 0, xi (t) ≤ d i for −∞ < t ≤ t and S j (t; ψ) ≤ f j (d j ) for all j = 1, 2, . . . , N and −∞ < t ≤ t ∗ . For the first case, we derive from (3) that 0>
N N X X dxi ∗ (t ) = −ai xi (t ∗ ) + Di j f j (x j (t ∗ )) + Diτj f j (x j (t ∗ − τi j (t ∗ ))) dt j=1 j=1
+
N X
Z
t∗
Di j
K i j (t ∗ − s) f j (x j (s)) ds + Bi Si (t ∗ ) + Ii
−∞
j=1
= −ai xi (t ) + Dii f i (xi (t ∗ )) + Diiτ f i (xi (t ∗ − τii (t ∗ ))) + Ii Z ∞ N X + D ii K ii (s) f i (xi (t ∗ − s)) ds + Bi Si (t ∗ ) + Di j f j (x j (t ∗ )) ∗
0
+
N X
j=1, j6=i
Diτj f j (x j (t ∗ − τi j (t ∗ ))) +
j=1, j6=i
N X
K i j (s) f j (x j (t ∗ − s)) ds
Di j 0
j=1, j6=i
≥ − ai d¯i + (Dii + Diiτ + D ii + Bi ) f i (d¯i ) −
∞
Z
N X
ρ j (|Di j | + |Diτj | + |D i j |) + Ii
j=1, j6=i
= gˇi (d¯i ) > gˇi (cˇi ) = 0, τ A ∗ ∗ ∗ ¯ ¯ due to (HA 1 ), (H2 ), Dii ≥ 0, D ii ≥ 0, Bi ≥ 0, | f j | ≤ ρ j , and f i (x i (t − τii (t ))) ≥ f i (di ), f i (x i (t − s)) ≥ f i (di ), ∗ from the monotonicity of f i and the definition of t . This is a contradiction. Hence, xi (t) ≥ d¯i , Si (t) ≥ f i (d¯i ) for all t ≥ 0. Similar arguments can be employed to show that xi (t) ≤ d i , Si (t) ≤ f i (d i ) for the second case. Therefore, each Λα is positively invariant with respect to the solution flow generated by system (3). This completes the proof. τ A A N Theorem 3. Under conditions (HA 1 ), (H2 ), (H3 ), and Dii ≥ 0, D ii ≥ 0, Bi ≥ 0, i = 1, 2, . . . , N , there exist 2 exponentially stable equilibria for system (3) with activation functions of class A.
Proof. Consider an equilibrium point (x ∗ T , S ∗ T )T = (x1∗ , x2∗ , . . . , x N∗ , S1∗ , S2∗ , . . . , S N∗ )T , where x ∗ ∈ Ω α for some (α1 , α2 , . . . , α N )T with αi = “l” or “r”, obtained in Theorem 1. We consider the following single-variable functions ωi (·), defined by " # N X τ ωi (λ) = µi pai − ( p − q) (|Di j | + |Di j | + |D i j |) − ( p − q) − qλ j=1
−q
N N X X p/q p/q (|Di j | + |D i j |)µ j η j − qeτ λ |Diτj |µ j η j − qµ N +i |Bi | p/q , j=1
(14)
j=1
where τ = max1≤i, j≤N τi j and i = 1, 2, . . . , N . By virtue of (HA 3 ), we have ωi (0) > 0 and ωi (λ) are continuous for λ ∈ [0, +∞). Moreover, ωi (λ) → −∞ as λ → +∞. Since (dωi (λ))/dλ < 0, ωi (λ) are strictly monotone decreasing functions on [0, +∞). Thus, there exist
X. Nie, J. Cao / Nonlinear Analysis: Real World Applications 10 (2009) 928–942
935
z i∗ > 0 such that " ωi (z i∗ )
= µi
N X pai − ( p − q) (|Di j | + |Diτj | + |D i j |) − ( p − q) − qz i∗
#
j=1
−q
N X
p/q
(|Di j | + |D i j |)µ j η j
− qeτ zi
∗
N X
|Diτj |µ j η j
p/q
− qµ N +i |Bi | p/q = 0
(15)
j=1
j=1
holds for all i = 1, 2, . . . , N . From (4) and (15), we can choose a sufficiently small positive number 0 < ε < min{z 1∗ , . . . , z ∗N , µ} such that " # N N X X p/q τ µi pai − ( p − q) (|Di j | + |Di j | + |D i j |) − ( p − q) − qε − q |Di j |µ j η j j=1
−q
N X
p/q
|D i j |µ j η j
j=1 +∞
Z
eεs K i j (s)ds − qeτ ε
N X
0
j=1
|Diτj |µ j η j
p/q
− qµ N +i |Bi | p/q > 0
(16)
j=1
holds for all i = 1, 2, . . . , N . Let (x T (t), S T (t))T = (x T (t; φ), S T (t; ψ))T be the solution to (3) with initial condition (φ T , ψ T )T ∈ Λα defined in (13), then N X d(xi (t) − xi∗ ) = −ai (xi (t) − xi∗ ) + Di j ( f j (x j (t)) − f j (x ∗j )) dt j=1
+
N X
Diτj ( f j (x j (t − τi j (t))) − f j (x ∗j )) + Bi (Si (t) − Si∗ )
j=1
+
N X
Z
t
Di j −∞
j=1
K i j (t − s)( f j (x j (s)) − f j (x ∗j )) ds,
(17)
d(Si (t) − Si∗ ) = −(Si (t) − Si∗ ) + f i (xi (t)) − f i (xi∗ ) dt for i = 1, 2, . . . , N . Let ϕi (t) = eεt |xi (t) − xi∗ | p/q , ζi (t) = eεt |Si (t) − Si∗ | p/q ,
i = 1, 2, . . . , N , i = 1, 2, . . . , N ,
(18)
then by using (4), (17) and (18), we can derive that εt
D ϕi (t) = εe |xi (t) − +
+
N X
xi∗ | p/q
p + eεt |xi (t) − xi∗ |( p−q)/q sgn(xi (t) − xi∗ ) q
Di j ( f j (x j (t)) − f j (x ∗j )) +
j=1
+
N X j=1
N X
" − ai (xi (t) − xi∗ )
Diτj ( f j (x j (t − τi j (t))) − f j (x ∗j ))
j=1
Z Di j 0
#
∞
K i j (s)( f j (x j (t − s)) − f j (x ∗j )) ds + Bi (Si (t) − Si∗ )
≤ εeεt |xi (t) − xi∗ | p/q −
N p εt p X ai e |xi (t) − xi∗ | p/q + eεt |Di j | f j0 (ς j )|x j (t) − x ∗j | q q j=1
× |xi (t) − xi∗ |( p−q)/q +
p εt e |Bi ||Si (t) − Si∗ | |xi (t) − xi∗ |( p−q)/q q
936
X. Nie, J. Cao / Nonlinear Analysis: Real World Applications 10 (2009) 928–942
+
N p εt X e |Diτj | f j0 (ξ j ) |x j (t − τi j (t)) − x ∗j | |xi (t) − xi∗ |( p−q)/q q j=1
N p X |D i j | + eεt q j=1
∞
Z 0
K i j (s) f j0 (ν j ) |x j (t − s) − x ∗j | |xi (t) − xi∗ |( p−q)/q ds,
where ς j , ξ j , ν j are three real numbers, respectively, lying between x j (t) and x ∗j , between x j (t − τi j (t)) and x ∗j as well as between x j (t − s) and x ∗j . Hence, D + ϕi (t) ≤ εeεt |xi (t) − xi∗ | p/q −
N p εt p X ai e |xi (t) − xi∗ | p/q + eεt |Di j | η j |x j (t) − x ∗j | q q j=1 N p εt X e |Diτj | η j |x j (t − τi j (t)) − x ∗j | |xi (t) − xi∗ |( p−q)/q q j=1
× |xi (t) − xi∗ |( p−q)/q + N p X + eεt |D i j | q j=1
+
∞
Z 0
K i j (s) η j |x j (t − s) − x ∗j | |xi (t) − xi∗ |( p−q)/q ds
p εt e |Bi | |Si (t) − Si∗ | |xi (t) − xi∗ |( p−q)/q . q
(19)
An application of Lemma 1 yields D + ϕi (t) ≤ εeεt |xi (t) − xi∗ | p/q − + eεt
N X
p/q
N p εt p − q εt X ai e |xi (t) − xi∗ | p/q + e |Di j | |xi (t) − xi∗ | p/q q q j=1
|x j (t) − x ∗j | p/q +
|Diτj | η j
|x j (t − τi j (t)) − x ∗j | p/q +
p/q |D i j | η j
Z
j=1
+ eεt
N X
N p − q εt X e |Diτj | |xi (t) − xi∗ | p/q q j=1
|Di j | η j
p/q
j=1
+e
εt
N X
∞
K i j (s) |x j (t − s) − x ∗j | p/q ds +
0
j=1 εt
N p − q εt X e |D i j | |xi (t) − xi∗ | p/q q j=1
p − q εt e |xi (t) − xi∗ | p/q q
+ e |Bi | |Si (t) − Si∗ | p/q (" # N X 1 qε − pai + ( p − q) (|Di j | + |Diτj | + |D i j |) + ( p − q) ϕi (t) ≤ q j=1 +q
p/q
N X
p/q
|Di j |η j
ϕ j (t) + qeετ
j=1
+q
N X
|Diτj |η j
p/q
ϕ j (t − τi j (t))
j=1
N X
p/q |D i j |η j
Z
j=1
∞
εs
e K i j (s)ϕ j (t − s)ds + q|Bi |
) p/q
ζi (t) .
(20)
0
Herein, the invariance property of Λα in Theorem 2 has been applied. Similarly, we have p/q
D + ζi (t) ≤ (ε − 1)ζi (t) + ηi
ϕi (t).
Let δ > 0 be an arbitrary real number and l0 =
(1 + δ)(kφ − x ∗ k p/q + kS(0) − S ∗ k p/q ) min {µi } 1≤i≤2N
(21)
X. Nie, J. Cao / Nonlinear Analysis: Real World Applications 10 (2009) 928–942
937
then ϕi (s) = eεs |xi (s) − xi∗ | p/q ≤ |xi (s) − xi∗ | p/q = |φi (s) − xi∗ | p/q < µi l0 , ζi (0) = |Si (0) −
Si∗ | p/q
∗ p/q
≤ kS(0) − S k
< µ N +i l0 ,
−∞ < s ≤ 0,
i = 1, 2, . . . , N .
In the following, we will prove that ϕi (t) < µi l0 ,
ζi (t) < µ N +i l0 ,
t > 0, i = 1, 2, . . . , N .
If (22) is not true, then there exist some i and ϕi (t ∗ ) = µi l0 , ϕ j (t) ≤ µ j l0
t∗
(22)
> 0 such that either
D + ϕi (t ∗ ) ≥ 0, (−∞ < t ≤ t ∗ , j = 1, 2, . . . , N ),
ζ j (t) ≤ µ N + j l0
(23)
(0 ≤ t ≤ t ∗ , j = 1, 2, . . . , N )
or ζi (t ∗ ) = µ N +i l0 ,
D + ζi (t ∗ ) ≥ 0,
ζ j (t) ≤ µ N + j l0 ϕ j (t) ≤ µ j l0
(0 ≤ t ≤ t ∗ , j = 1, 2, . . . , N ),
(24)
(−∞ < t ≤ t , j = 1, 2, . . . , N ). ∗
For the first case, it follows from (16) and (20) that # (" N X 1 D + ϕi (t ∗ ) ≤ qε − pai + ( p − q) (|Di j | + |Diτj | + |D i j |) + ( p − q) µi q j=1 +q
N X
p/q |Di j |µ j η j
+q
j=1 τε
+ qe
N X
p/q |D i j |µ j η j
Z
|Diτj |µ j
eεs K i j (s)ds
0
j=1 N X
+∞
) p/q ηj
+ qµ N +i |Bi |
p/q
l0 < 0,
(25)
j=1
that is a contraction. For the second case, by using (11) and (21), we have p/q
D + ζi (t ∗ ) ≤ (ε − 1)ζi (t ∗ ) + ηi
ϕi (t ∗ )
p/q
≤ [(ε − 1)µ N +i + ηi
µi ]l0 < 0,
(26)
that is also a contraction. So (22) holds. That is eεt |xi (t) − xi∗ | p/q ≤ K (kφ − x ∗ k p/q + kS(0) − S ∗ k p/q ), εt
e |Si (t) −
Si∗ | p/q
≤ K (kφ − x k
∗ p/q
∗ p/q
+ kS(0) − S k
),
i = 1, 2, . . . , N , i = 1, 2, . . . , N
(27)
for any t > 0, where K = (1+δ)(max1≤i≤2N {µi })/(min1≤i≤2N {µi }). Let M = 2(K N )q/ p , from (27) and Lemma 2, we get that kx(t) − x ∗ k + kS(t) − S ∗ k ≤ M(kφ − x ∗ k + kS(0) − S ∗ k) e−(qε/ p)t , for all t > 0. Therefore, (x T (t), S T (t))T is exponentially convergent to (x ∗ T , S ∗ T )T and the exponential convergence rate is qε/ p. This completes the proof. τ B N Theorem 4. Under conditions (HB 1 ), (H2 ), and Dii ≥ 0, D ii ≥ 0, Bi ≥ 0, i = 1, 2, . . . , N , there exist 2 exponentially stable equilibria for system (3) with activation functions of class B.
Proof. Consider an equilibrium (x ∗ T , S ∗ T )T = (x1∗ , x2∗ , . . . , x N∗ , S1∗ , S2∗ , . . . , S N∗ )T where x ∗ ∈ Ω α for some (α1 , α2 , . . . , α N )T with αi = “l” or “r”, obtained in Theorem 1. From (HB 2 ), we can choose a sufficiently small positive number ε > 0 such that µi [ pai − ( p − q) − qε] − qµ N +i |Bi | p/q > 0 for i = 1, 2, . . . , N . Let
938
X. Nie, J. Cao / Nonlinear Analysis: Real World Applications 10 (2009) 928–942
(x T (t), S T (t))T = (x T (t; φ), S T (t; ψ))T be the solution to (3) with the initial condition (φ T , ψ T )T ∈ Λα defined in (13). Note that the invariance property of Λα and f i (xi (t; φ)) ≡ u i or vi when (φ T , ψ T )T ∈ Λα , then d(xi (t) − xi∗ ) = −ai (xi (t) − xi∗ ) + Bi (Si (t) − Si∗ ), dt d(Si (t) − Si∗ ) = −(Si (t) − Si∗ ), i = 1, 2, . . . , N . dt
i = 1, 2, . . . , N , (28)
Let ϕi (t) = eεt |xi (t) − xi∗ | p/q , εt
ζi (t) = e |Si (t) −
Si∗ | p/q ,
i = 1, 2, . . . , N ,
(29)
i = 1, 2, . . . , N ,
then it follows from (28) and (29) that p εt e |xi (t) − xi∗ |( p−q)/q sgn(xi (t) − xi∗ ) q × [−ai (xi (t) − xi∗ ) + Bi (Si (t) − Si∗ )]
D + ϕi (t) = εeεt |xi (t) − xi∗ | p/q +
≤
1 [(qε − pai + p − q)ϕi (t) + q|Bi | p/q ζi (t)]. q
Similarly, we have p + D ζi (t) = ε − ζi (t). q
(30)
(31)
The remaining details of the proof is similar to those in Theorem 3 and here are omitted. This completes the proof.
Corollary 1. Each of these 2 N subsets Λα of C((−∞, 0], R2N ), defined in (13), lies in the basin of attraction for the unique equilibrium point in Λα , under the assumptions of Theorem 3 or Theorem 4. B If we take p = q = 1 in (11) and (12), then (HA 3 ), (H2 ) reduces to the following conditions, respectively, 0
(HA 3 ) There exist constants µi > 0 (i = 1, 2, . . . , 2N ) such that µi ai −
N X (|Di j | + |Diτj | + |D i j |)µ j η j − µ N +i |Bi | > 0, j=1
(32)
−µi ηi + µ N +i > 0 hold for all i = 1, 2, . . . , N , where η j , j = 1, 2, . . . , N are real numbers satisfying max{ f j0 (ξ )|ξ = cˇ j , aˆ j } < η j < min{ f j0 (ξ )|ξ = p˜ j , q˜ j }. 0
(HB 2 ) There exist constants µi > 0 (i = 1, 2, . . . , 2N ) such that µi ai − µ N +i |Bi | > 0,
i = 1, 2, . . . , N .
(33)
Applying Theorems 3 and 4 above, respectively, we can easily obtain the following corollaries: τ A A N Corollary 2. Under conditions (HA 1 ), (H2 ), (H3 ), and Dii ≥ 0, D ii ≥ 0, Bi ≥ 0, i = 1, 2, . . . , N , there exist 2 exponentially stable equilibria for system (3) with activation functions of class A. 0
τ Corollary 3. Under conditions gˆi (−1) < 0, gˇi (1) > 0, (HB 2 ), and Dii ≥ 0, D ii ≥ 0, Bi ≥ 0, i = 1, 2, . . . , N , N there exist 2 exponentially stable equilibria for system (3) with the standard piecewise linear function. 0
Remark 2. Since there exist time delays, the method used in [18] cannot be applied to system (3). Moreover, our method used here is valid for a class of functions and the stability analysis of neural networks with time-varying and distributed delays.
X. Nie, J. Cao / Nonlinear Analysis: Real World Applications 10 (2009) 928–942
939
4. Two illustrative examples We consider the following two-dimensional competitive neural networks with time-varying and distributed delays 2 2 X X dxi (t) = −a x (t) + D f (x (t)) + Diτj f j (x j (t − τi j (t))) i i i j j j dt j=1 Z t j=1 Z t + D i1 K i1 (t − s) f 1 (x1 (s)) ds + D i2 K i2 (t − s) f 2 (x2 (s)) ds (34) −5 −6 +Bi Si (t) + Ii , i = 1, 2, dSi (t) = −S (t) + f (x (t)), i = 1, 2 i i i dt for t > 0, where K i1 (s) = satisfy the assumptions (4).
2 e−2s , 1−e−10
K i2 (s) =
e−s 1−e−6
(i = 1, 2). We can easily check that K i j (s) (i, j = 1, 2) above
1 τ = D τ Example 1. For system (34), take a1 = 1, a2 = 2, D11 = D11 11 = B1 = 2 , D22 = D22 = D 22 = τ τ B2 = 1, D12 = D12 = 0.15, D21 = D21 = 0.25, D 12 = D 21 = −0.1, f i (ξ ) ≡ f (ξ ) = arctan ξ, τi j (t) ≡ τ (t) = e−t , Ii = 0, (i, j = 1, 2). Direct computation gives gˆ 1 (ξ ) = −ξ + 2 arctan ξ + 0.1 π, gˇ 1 (ξ ) = −ξ + 2 arctan ξ − 0.1 π , gˆ 2 (ξ ) = −2 ξ + 4 arctan ξ + 0.2 π , gˇ 2 (ξ ) = −2 ξ + 4 arctan ξ − 0.2 π . Herein, the parameters satisfy conditions in Corollary 2: Condition (HA 1 ):
0<
a1 D11 +
τ D11
=
+ D 11 + B1
1 < 1, 2
0<
a2 D22 +
τ D22
=
+ D 22 + B2
1 < 1. 2
Condition (HA 2 ): gˆ 1 ( p˜ 1 ) = −0.2566 < 0,
gˇ 1 (q˜1 ) = 0.2566 > 0,
gˆ 2 ( p˜ 2 ) = −0.5133 < 0,
gˇ 2 (q˜2 ) = 0.5133 > 0.
Condition
0 (HA 3 ):
τ τ | + |D 11 |) − η2 (|D12 | + |D12 | + |D 12 |) − |B1 | = 0.025 > 0, a1 − η1 (|D11 | + |D11 τ τ | + |D 21 |) − η2 (|D22 | + |D22 | + |D 22 |) − |B2 | = 0.1 > 0, a2 − η1 (|D21 | + |D21 0
where µ1 = µ2 = µ3 = µ4 , η1 = η2 = 0.25 are chosen in (HA ˆ 1 = aˆ 2 3 ); p˜ 1 = p˜ 2 = −1, q˜1 = q˜2 = 1, a ˆ ˆ ˇ ˇ = −1.8248, b1 = b2 = −0.3384, cˆ1 = cˆ2 = 2.7607, aˇ 1 = aˇ 1 = −2.7607, b1 = b2 = 0.3384, cˇ1 = cˇ2 = 1.8248, d 1 = d 2 = −1.7321, d¯1 = d¯2 = 1.7321. From Corollary 2, it follows that system (34) has four equilibria, which is exponentially stable. Figs. 4 and 5 depict the time responses of state variables x1 (t), x2 (t), S1 (t), S2 (t) with step h = 0.01. From Figs. 4 and 5, it is easy to see that there are four exponentially stable equilibria (x1∗ , x2∗ , S1∗ , S2∗ )T = (2.6664, 2.6664, 1.2120, 1.2120)T , (−2.6664, −2.6664, −1.2120, −1.2120)T , (1.9888, −1.9888, 1.1049, −1.1049)T , (−1.9888, 1.9888, −1.1049, 1.1049)T in this system, as confirmed by our theory. τ = D τ τ Example 2. For system (34), take D11 = D11 11 = 2, D12 = D12 = D 12 = D21 = D21 = D 21 = 1, D22 = 1 τ D22 = D 22 = 3, I1 = 1.5, I2 = 3, ai = 2, Bi = 1, f i (ξ ) ≡ f (ξ ) = 2 (|ξ +1|−|ξ −1|), τi j (t) ≡ τ (t) = e−t (i, j = 1, 2). Direct computation gives gˆ 1 (ξ ) = −2 ξ +7 f (ξ )+4.5, gˇ 1 (ξ ) = −2 ξ +7 f (ξ )−1.5, gˆ 2 (ξ ) = −2 ξ +10 f (ξ )+6, gˇ 2 (ξ ) = −2 ξ + 10 f (ξ ). The parameters satisfy the conditions in Corollary 3:
gˆ 1 (−1) = −0.5 < 0, 0
gˇ 1 (1) = 3.5 > 0,
gˆ 2 (−1) = −2 < 0,
gˇ 2 (1) = 8 > 0.
Condition (HB 2 ): ai > |Bi |, i = 1, 2, 0 where µ1 = · · · = µ4 are chosen in (HB ˆ 1 = −1.25, bˆ1 = −0.9, cˆ1 = 5.75, aˇ 1 = −4.25, bˇ1 = 0.3, cˇ1 = 2 ); a 2.75, aˆ 2 = −2, bˆ2 = −0.75, cˆ2 = 8, aˇ 2 = −5, bˇ2 = 0, cˇ2 = 5, d 1 = d 2 = −1, d¯1 = d¯2 = 1. From Corollary 3,
940
X. Nie, J. Cao / Nonlinear Analysis: Real World Applications 10 (2009) 928–942
Fig. 4. Transient response of state variables x1 (t) and x2 (t).
Fig. 5. Transient response of state variables S1 (t) and S2 (t).
we can conclude that this system has four exponentially stable equilibria. Figs. 6 and 7 depict the time responses of state variables x1 (t), x2 (t), S1 (t), S2 (t) with step h = 0.01. From Figs. 6 and 7, it is also easy to see that there are four exponentially stable equilibria (x1∗ , x2∗ , S1∗ , S2∗ )T = (5.75, 8, 1, 1)T , (−4.25, −5, −1, −1)T , (2.75, −2, 1, −1)T , (−1.25, 5, −1, 1)T in this system, as confirmed by our theory. 5. Conclusions In this paper, two classes of activation functions, which are commonly employed in neural networks, have been considered. By means of geometrical method and inequality technique, several novel sufficient conditions have been derived ensuring the existence of 3 N equilibria and exponential stability of 2 N equilibria for competitive neural networks with time-varying and distributed delays. Compared with [18], the method used here is valid for a class of functions and the stability analysis of neural networks with time-varying and distributed delays. Acknowledgments The authors appreciate the editor’s work and the reviewer’s insightful comments and constructive suggestions. This work was jointly supported by the National Natural Science Foundation of China under Grant 60574043 and the Natural Science Foundation of Jiangsu Province of China under Grant BK2006093.
X. Nie, J. Cao / Nonlinear Analysis: Real World Applications 10 (2009) 928–942
941
Fig. 6. Transient response of state variables x1 (t) and x2 (t).
Fig. 7. Transient response of state variables S1 (t) and S2 (t).
References [1] L.O. Chua, L. Yang, Cellular neural networks: Theory, IEEE Trans. Circuits Syst. 35 (1988) 1257–1272. [2] J. Hopfield, Neurons with graded response have collective computational properties like those of two state neurons, Proc. Natl. Acad. Sci. USA 81 (1984) 3088–3092. [3] M. Morita, Associative memory with non-monotone dynamics, Neural Netw. 6 (1993) 115–126. [4] J. Foss, A. Longtin, B. Mensour, J. Milton, Multistability and delayed recurrent loops, Phys. Rev. Lett. 76 (1996) 708–711. [5] A. Meyer-Baese, F. Ohl, H. Scheich, Singular perturbation analysis of competitive neural networks with different time-scales, Neural Comput. 8 (1996) 1731–1742. [6] J. Cao, J. Wang, Asymptotic stability of a general class of recurrent neural networks with time-varying delays, IEEE Trans. Circuits Syst. I 50 (1) (2003) 34–44. [7] J. Cao, Q. Song, Stability in Cohen–Grossberg-type bidirectional associative memory neural networks with time-varying delays, Nonlinearity 19 (2006) 1601–1607. [8] K. Yuan, J. Cao, Exponential stability and periodic solutions of fuzzy cellular neural networks with time-varying delays, Neurocomputing 69 (2006) 1619–1627. [9] Q. Zhang, X. Wei, J. Xu, Stability analysis for cellular neural networks with variable delays, Chaos Solitons Fractals 28 (2006) 331–336. [10] B. Liu, L. Huang, Existence and stability of almost periodic solutions for shunting inhibitory cellular neural networks with continuously distributed delays, Phys. Lett. A 349 (2006) 177–186. [11] Q. Song, J. Cao, Z. Zhao, Periodic solutions and its exponential stability of reaction-diffusion recurrent neural networks with continuously distributed delays, Nonlinear Anal. Real World Appl. 7 (2006) 65–80.
942
X. Nie, J. Cao / Nonlinear Analysis: Real World Applications 10 (2009) 928–942
[12] H. Jiang, Z. Teng, Global exponential stability of cellular neural networks with time-varying coefficients and delays, Neural Netw. 17 (10) (2004) 1415–1425. [13] H. Jiang, Z. Teng, A new criterion on the global exponential stability for cellular neural networks with multiple time-varying delays, Phys. Lett. A 338 (2005) 461–471. [14] A. Meyer-Baese, S. Pilyugin, Y. Chen, Global exponential stability of competitive neural networks with different time scales, IEEE Trans. Neural Netw. 14 (3) (2003) 716–719. [15] A. Meyer-Baese, S. Pilyugin, A. Wismuller, S. Foo, Local exponential stability of competitive neural networks with different time scales, Eng. Appl. Artifical Intell. 17 (2004) 227–232. [16] H. Lu, A. Shun-ichi, Global exponential stability of multitime scale competitive neural networks with nonsmooth functions, IEEE Trans. Neural Netw. 17 (5) (2006) 1152–1164. [17] H. Lu, Z. He, Global exponential stability of delayed competitive neural networks with different time scales, Neural Netw. 18 (2005) 243–250. [18] Y. Mao, Multistability of competitive neural networks with different time scales, in: Communications, Circuits and Systems, 2005. Proceedings. 2005 International Conference on, vol. 2, 2005, pp. 939–943. [19] C.Y. Cheng, K.H. Lin, C.W. Shih, Multistability in recurrent neural networks, SIAM J. Appl. Math. 66 (4) (2006) 1301–1320. [20] C.Y. Cheng, K.H. Lin, C.W. Shih, Multistability and convergence in delayed neural networks, Physica D 225 (2007) 61–74. [21] Z.G. Zeng, J. Wang, Multiperiodicity and exponential attractivity evoked by periodic external inputs in delayed cellular neural networks, Neural Comput. 18 (2006) 848–870. [22] J. Kuang, Applied Inqualities, 3rd edn, Shandong Science and Technology Press, 2004 (in Chinese).
Multistability of competitive neural networks with time-varying and distributed delays Xiaobing Nie ∗ , Jinde Cao Department of Mathematics, Southeast University, Nanjing 210096, China Received 24 September 2007; accepted 16 November 2007
Abstract In this paper, with two classes of general activation functions, we investigate the multistability of competitive neural networks with time-varying and distributed delays. By formulating parameter conditions and using inequality technique, several novel delayindependent sufficient conditions ensuring the existence of 3 N equilibria and exponential stability of 2 N equilibria are derived. In addition, estimations of positively invariant sets and basins of attraction for these stable equilibria are obtained. Two examples are given to show the effectiveness of our theory. c 2007 Elsevier Ltd. All rights reserved.
Keywords: Competitive neural networks; Time-varying delays; Distributed delays; Multistability
1. Introduction It is well known that neural networks play important roles in many applications, such as classification, associative memory, image processing, pattern recognition, parallel computation, optimization problem, and decision making etc [1–4]. The theory on the dynamics of the networks has been developed according to the purposes of the applications. On the one hand, in the applications to parallel computation and optimization problem, the existence of a computable solution for all possible initial states is the best situation. Mathematically, this means that an equilibrium of the networks exists and any state in the neighborhood converges to the equilibrium, which is called “monostability” of networks. On the other hand, existence of many equilibria is a necessary feature in the applications of neural networks to associative memory storage, pattern recognition, and decision making. The notion of “multistability” of networks describes coexistence of multiple stable patterns such as equilibria or periodic orbits. Competitive neural networks (CNNs) with different time scales are proposed in [5], which model the dynamics of cortical cognitive maps with unsupervised synaptic modifications. In this model, there are two types of state variables, that of the short-term memory (STM) describing the fast neural activity and that of the long-term memory (LTM) describing the slow unsupervised synaptic modifications. A typical form for multitime scale CNNs with time-varying ∗ Corresponding author.
E-mail address: [email protected] (X. Nie). c 2007 Elsevier Ltd. All rights reserved. 1468-1218/$ - see front matter doi:10.1016/j.nonrwa.2007.11.014
X. Nie, J. Cao / Nonlinear Analysis: Real World Applications 10 (2009) 928–942
929
and distributed delays is described by the following functional differential equations N N X X STM : dxi (t) = −ai xi (t) + D f (x (t)) + Diτj f j (x j (t − τi j (t))) ij j j dt j=1 j=1 Z t N P X X + D K (t − s) f (x (s)) ds + B m i j (t)y j + Ii , i j i j j j i −∞ j=1 j=1 dm i j (t) = −m i j (t) + y j f i (xi (t)), i = 1, 2, . . . , N , j = 1, 2, . . . , P, LTM : dt
(1)
where xi (t) is the neuron current activity level, ai > 0 is the time constant of the neuron, f j (x j (t)) is the output of neurons, m i j (t) is the synaptic efficiency, y j is the constant external stimulus, Di j represents the connection weight between the ith neuron and the jth neuron, Bi is the strength of the external stimulus, is the time scale of STM state, Diτj and D i j represent the synaptic weight of delayed feedback, Ii is the constant input, τi j (t) corresponds to the transmission delay and satisfies 0 < τi j (t) < τi j (τi j is a positive constant). PP T T After setting Si (t) = j=1 m i j (t)y j = y mi (t), where y = (y1 , y2 , . . . , y P ) , mi (t) = (m i1 (t), m i2 (t), . . . , m i P (t))T , and summing up the LTM over j, the networks (1) can be rewritten in the following form N N X X dxi (t) STM : = −a x (t) + D f (x (t)) + Diτj f j (x j (t − τi j (t))) i i i j j j dt j=1 j=1 Z t N X Di j + K i j (t − s) f j (x j (s)) ds + Bi Si (t) + Ii , −∞ j=1 dSi (t) = −Si (t) + |y|2 f i (xi (t)), i = 1, 2, . . . , N , LTM : dt
(2)
where |y|2 = y12 + · · · + y 2P is a constant, without loss of generality, the input stimulus y is assumed to be normalized with unit magnitude |y|2 = 1, and the fast time-scale parameter is also assumed to be unit, then the above networks are simplified as N N X X dxi (t) STM : = −a x (t) + D f (x (t)) + Diτj f j (x j (t − τi j (t))) i i i j j j dt j=1 j=1 Z t N X + Di j K i j (t − s) f j (x j (s)) ds + Bi Si (t) + Ii , −∞ j=1 dSi (t) = −Si (t) + f i (xi (t)), i = 1, 2, . . . , N , LTM : dt
(3)
where the delay kernels K i j (s) : [0, +∞) → [0, +∞) are piecewise continuous integral functions, and they satisfy Z +∞ Z +∞ K i j (s)ds = 1, K i j (s) eµs ds < +∞ (4) 0
0
for some positive constant µ and i, j = 1, 2, . . . , N . In the past few years, the monostability analysis of neural networks with time-varying and/or distributed delays has been developed [6–13]. In particular, the theory of unique equilibrium point and global convergence to the equilibrium point for CNNs with their various generalizations has been extensively studied, see [5,14–17]. Recently, the multistability analysis of neural networks has attracted the attention of many researchers [18–21]. In [18], with unsaturated piecewise linear activation function f (x) = max{0, x}, the multistability of system (3) without delay, that is Diτj = D i j = 0 (i, j = 1, 2, . . . , N ), was investigated by using local inhibition and constructing an energy-like function. The result therein strongly relies on the piecewise linearity and unsaturation of activation functions. In [19, 20], the multistability of neural networks without and with constant delays was studied by formulating parameter conditions. Inspired by [19,20], in this paper, we shall study the multistability of system (3) with two classes of general activation functions. Firstly, we derive conditions ensuring the existence of 3 N equilibria for system (3) with
930
X. Nie, J. Cao / Nonlinear Analysis: Real World Applications 10 (2009) 928–942
two classes of general activation functions, through constructing parameter conditions by a geometrical observation. Secondly, we establish a series of new criteria on the exponential stability of 2 N equilibria for the networks above by means of inequality technique. In addition, estimations of positively invariant sets and basins of attraction for these stable stationary solutions are derived. This paper is organized as follows. In Section 2, we consider two classes of activation functions which are commonly employed in neural networks. We then obtain conditions for the existence of 3 N equilibria. In Section 3, we show that, with additional conditions, there are 2 N regions in R2N , which are positively invariant under the flow generated by system (3). Subsequently, it is argued that these 2 N equilibria are exponentially stable. Two numerical simulations are given to illustrate our theory and distinct dynamical behaviors for different activation functions in Section 4. Finally, concluding remarks are summarized in Section 5. 2. Activation functions and multiple equilibria The initial conditions associated with system (3) are of the form xi (t) = φi (t), t ∈ (−∞, 0], Si (t) = ψi (t) ≡ ψi (0), t ∈ (−∞, 0]. For convenience, we introduce two notations. For any u(t) = (u 1 (t), u 2 (t), . . . , u N (t))T ∈ R N , define #q/ p " N X p/q , p ≥ q > 0. ku(t)k = |u i (t)| i=1
For any φ(s) = (φ1 (s), φ2 (s), . . . , φ N (s))T ∈ R N , s ∈ (−∞, 0], define " #q/ p N X p/q kφk = sup |φi (s)| , p ≥ q > 0, s∈(−∞,0] i=1
where φi (s) (i = 1, 2, . . . , N ) are continuous and bounded functions on (−∞, 0]. To prove our results, the following two lemmas are necessary. Lemma 1 ([22]). Let a, b ≥ 0, s ≥ 1, then a s−1 b ≤
s−1 s 1 s a + b . s s
Lemma 2 (C p Inequality [22]). Let a, b ≥ 0, p ≥ q > 0, then (a + b)q/ p ≤ a q/ p + bq/ p . In this paper, we shall consider the following two classes of activation functions f i for system (3): u i ≤ f i (ξ ) ≤ vi , f i0 (ξ ) > 0, 2 class A : f i ∈ C , 00 (ξ − σi ) f i (ξ ) < 0, for all ξ ∈ R \ {σi }, if − ∞ < ξ < pi , u i , increasing, if pi ≤ ξ ≤ qi , class B : f i ∈ C, vi , if qi < ξ < ∞, where u i , vi , pi , qi and σi are constants with u i < vi , pi < qi . Remark 1. Class A contains general bounded smooth sigmoidal functions, for example, functions 1+e1 −ξ , tanh ξ, arctan ξ are special cases of class A. Class B consists of nondecreasing functions with saturations, including the piecewise linear functions with two corner points at pi , qi : f i (ξ ) = u i +
vi − u i (ξ − pi ), qi − pi
(5)
931
X. Nie, J. Cao / Nonlinear Analysis: Real World Applications 10 (2009) 928–942
Fig. 1. The configurations of (a) smooth sigmoidal activation functions in class A and (b) saturated activation functions in class B.
and, in particular, the standard piecewise linear function: f i (ξ ) =
1 (|ξ + 1| − |ξ − 1|). 2
(6)
Typical configurations of these functions are depicted in Fig. 1. Note that the stationary equations for system (3) are −ai xi +
N X (Di j + Diτj + D i j ) f j (x j ) + Bi Si + Ii = 0,
(7)
j=1
−Si + f i (xi ) = 0,
i = 1, 2, . . . , N .
Equivalently, the above equations can be rewritten as G i (X ) := −ai xi +
N X (Di j + Diτj + D i j ) f j (x j ) + Bi f i (xi ) + Ii = 0,
(8)
j=1
−Si + f i (xi ) = 0,
i = 1, 2, . . . , N ,
where X = (x1 , x2 , . . . , x N )T . Now, define a map from R to R as gi (ξ ) := −ai ξ + (Dii + Diiτ + D ii + Bi ) f i (ξ ),
i = 1, 2, . . . , N .
Let us propose the first parameter condition for system (3) with activation functions in class A: (HA 1)
0 = inf f i0 (ξ ) < ξ ∈R
ai Dii +
Diiτ
+ D ii + Bi
< max f i0 (ξ ) = f i0 (σi ), ξ ∈R
i = 1, 2, . . . , N .
Lemma 3. There exist two points p˜ i , q˜i with p˜ i < σi < q˜i such that gi0 ( p˜ i ) = gi0 (q˜i ) = 0, i = 1, 2, . . . , N , under assumption (HA 1 ), for activation functions of class A. Proof. For each i, since gi0 (ξ ) = −ai + (Dii + Diiτ + D ii + Bi ) f i0 (ξ ), we have gi0 (ξ ) = 0 if and only if ai f i0 (ξ ) = . Consider the following functions τ Dii +Dii +D ii +Bi
Fi (ξ ) := f i0 (ξ ) −
ai Dii +
Diiτ
+ D ii + Bi
,
i = 1, 2, . . . , N .
0 It follows from (HA 1 ) and (9) that Fi (σi ) = f i (σi ) −
Moreover, noting that
limξ →±∞ f i0 (ξ )
ai Dii +Diiτ +D ii +Bi
(9) > 0 and Fi (ξ ) are continuous for all ξ ∈ R.
= 0, we have limξ →±∞ Fi (ξ ) = −
points p˜ i , q˜i with p˜ i < σi < q˜i such that
gi0 ( p˜ i )
=
gi0 (q˜i )
ai Dii +Diiτ +D ii +Bi
= 0. This completes the proof.
< 0. Thus, there exist two
932
X. Nie, J. Cao / Nonlinear Analysis: Real World Applications 10 (2009) 928–942
Fig. 2. The graphs of gˆi and gˇi induced from the activation function of class A.
Fig. 3. (a) The graphs of gˆi and gˇi induced from the activation function of class B. (b) The graphs of gˆi and gˇi induced from the standard activation function.
In the following, we define, for i = 1, 2, . . . , N , gˆi (ξ ) = −ai ξ + (Dii + Diiτ + D ii + Bi ) f i (ξ ) + ki+ , gˇi (ξ ) = −ai ξ + (Dii + Diiτ + D ii + Bi ) f i (ξ ) + ki− , P P where ki+ = Nj=1, j6=i ρ j |Di j +Diτj +D i j |+Ii , ki− = − Nj=1, j6=i ρ j |Di j +Diτj +D i j |+Ii with ρ j = max{|u j |, |v j |}. It follows that gˇi (xi ) ≤ G i (X ) ≤ gˆi (xi ) holds for all i = 1, 2, . . . , N , since | f i (xi )| ≤ max{|u i |, |vi |}. We consider the second (resp. first) parameter condition which is concerned with the existence of multiple equilibria for system (3) with activation functions in class A (resp. B): (HA 2 ) gˆ i ( p˜ i ) < 0, gˇ i (q˜i ) > 0, i = 1, 2, . . . , N . (HB 1 ) gˆ i ( pi ) < 0, gˇ i (qi ) > 0, i.e., −ai pi + (Dii + Diiτ + D ii + Bi )u i + ki+ < 0, −ai qi + (Dii +
Diiτ
+ D ii +
Bi )vi + ki−
> 0,
i = 1, 2, . . . , N , i = 1, 2, . . . , N .
B The configurations that motivates (HA ˆ i < bˆi < cˆi such 2 ) and (H1 ) are depicted in Figs. 2 and 3. There exist points a ˆ ˇ ˇ that gˆi (aˆ i ) = gˆi (bi ) = gˆi (cˆi ) = 0 as well as points aˇ i < bi < cˇi such that gˇi (aˇ i ) = gˇi (bi ) = gˇi (cˇi ) = 0, under A assumptions (HA 1 ) and (H2 ), for activation functions of class A. The same conclusion holds for activation functions of class B, under condition (HB 1 ).
Theorem 1. There exist 3 N equilibria for system (3) with activation functions of class A, under conditions (HA 1 ) and B ). (HA ). The same conclusion holds for system (3) with activation functions of class B, under condition (H 2 1 Proof. We prove only the case of class A, and similar arguments can be employed to prove the case of class B. A The equilibria of system (3) are zeros of (8). Under conditions (HA 1 ) and (H2 ), the graphs of gˆ i and gˇ i defined N above are depicted in Fig. 2. According to the configuration, there are 3 disjoint closed regions in R N . Set Ω α = {(x1 , x2 , . . . , x N )T ∈ R N |xi ∈ Ωiαi } with α = (α1 , α2 , . . . , α N )T , and αi = “l”, “m”, or “r”, where Ωil = [aˇ i , aˆ i ],
Ωim = [bˆi , bˇi ],
Ωir = [cˇi , cˆi ].
(10)
X. Nie, J. Cao / Nonlinear Analysis: Real World Applications 10 (2009) 928–942
933
Herein, “l”, “m”, “r” means, respectively, “left”, “middle” and “right”. Consider any fixed one of these regions Ω α . For a given x˜ = (x˜1 , x˜2 , . . . , x˜ N )T ∈ Ω α , we solve h i (xi ) := −ai xi + (Dii + Diiτ + D ii + Bi ) f i (xi ) +
N X
(Di j + Diτj + D i j ) f j (x˜ j ) + Ii = 0
j=1, j6=i
for i = 1, 2, . . . , N . Note that the graph of h i is a vertical shift of the graph of gˆi or gˇi and lies between the graphs of gˆi and gˇi , thus, one can always find three solutions xi , and each of them lies in one of the regions in (10) for each i. Let us pick the one lying in Ωiαi and set it as x i for each i. We now define a map Hα : Ω α → Ω α by Hα (x) ˜ = (x) = (x 1 , x 2 , . . . , x N )T . Restated, we set x i = (h i |Ω l )−1 (0),
if αi = “l”,
x i = (h i |Ωim )−1 (0),
if αi = “m”,
i
x i = (h i |Ωir )
−1
(0),
if αi = “r”.
Since f i is continuous and h i is a vertical shift of function ξ 7→ −ai ξ + (Dii + Diiτ + D ii + Bi ) f i (ξ ) by the PN τ quantity j=1, j6=i (Di j + Di j + D i j ) f j ( x˜ j ) + Ii , the map Hα is continuous. It follows from Brouwer’s fixed point theorem that there exists one fixed point x¯ = (x¯1 , x¯2 , . . . , x¯ N )T of Hα in Ω α . Thus, we also obtain a zero (x¯1 , x¯2 , . . . , x¯ N , f 1 (x¯1 ), f 2 (x¯2 ), . . . , f N (x¯ N ))T of (8). Consequently, there exist 3 N zeros of (8), hence 3 N equilibria for system (3), and each of them lies in one of the 3 N regions Ω α . 3. Stability of multiple equilibria In this section, we shall give some positively invariant sets for system (3) and investigate stability of the equilibrium point in each invariant set. As a result, we also obtain a basin of attraction for each of the exponentially stable equilibria. Firstly, we give the third condition for system (3) with activation functions in class A and the second condition for system (3) with activation functions in class B: (HA 3 ) There exist constants p ≥ q > 0, µi > 0 (i = 1, 2, . . . , 2N ) such that " # N N X X µi pai − ( p − q) (|Di j | + |Diτj | + |D i j |) − ( p − q) − q (|Di j | + |Diτj | j=1 p/q
+ |D i j |)µ j η j p/q
−µi ηi
j=1
− qµ N +i |Bi | p/q > 0,
+ µ N +i > 0,
(11)
i = 1, 2, . . . , N
hold for all i = 1, 2, . . . , N , where η j , j = 1, 2, . . . , N are real numbers satisfying max{ f j0 (ξ )|ξ = cˇ j , aˆ j } < η j < min{ f j0 (ξ )|ξ = p˜ j , q˜ j }. (HB 2 ) There exist constants p ≥ q > 0, µi > 0 (i = 1, 2, . . . , 2N ) such that µi [ pai − ( p − q)] − qµ N +i |Bi | p/q > 0,
i = 1, 2, . . . , N .
(12)
For activation functions f j (·) in class A, we define d j = min{ξ | f j0 (ξ ) = η j }, d¯ j = max{ξ | f j0 (ξ ) = η j }. Then we have aˆ j < d j < p˜ j , q˜ j < d¯ j < cˇ j . For activation functions f j (·) in class B, we define, respectively, d j = p j , d¯ j = q j . For system (3), consider the following 2 N subsets of C((−∞, 0], R2N ). Let α = (α1 , α2 , . . . , α N )T with αi = “l” or “r”, and set Λα = {(φ1 , φ2 , . . . , φ N , ψ1 , ψ2 , . . . , ψ N )T |(φi , ψi )T ∈ Λli if αi = “l”, (φi , ψi )T ∈ Λri if αi = “r”}, where Λli = {(φi , ψi )T ∈ C((−∞, 0], R2 )|φi (θ ) ≤ d i , ψi (θ ) ≡ ψi (0) ≤ f i (d i ), ∀θ ∈ (−∞, 0]}, Λri = {(φi , ψi )T ∈ C((−∞, 0], R2 )|φi (θ ) ≥ d¯i , ψi (θ ) ≡ ψi (0) ≥ f i (d¯i ), ∀θ ∈ (−∞, 0]}.
(13)
934
X. Nie, J. Cao / Nonlinear Analysis: Real World Applications 10 (2009) 928–942
τ A α Theorem 2. Assume that (HA 1 ), (H2 ) and Dii ≥ 0, D ii ≥ 0, Bi ≥ 0 hold. Then each Λ is positively invariant with respect to the solution flow generalized by system (3) with activation functions of class A. The same conclusion holds τ for activation functions of class B, under conditions (HB 1 ), Dii ≥ 0, D ii ≥ 0 and Bi ≥ 0.
Proof. We only prove the case A, the proof of case B is similar to that in case A and here is omitted. Consider any one of the 2 N subsets Λα . For any initial condition (φ T , ψ T )T = (φ1 , φ2 , . . . , φ N , ψ1 , ψ2 , . . . , φ N )T ∈ Λα , we claim that the solution (x T (t; φ), S T (t; ψ))T remains in Λα for all t ≥ 0. If this is not true, without loss of generality, there exist a component xi (t) of x(t; φ) which is the first (or one of the first) escaping from Λli or Λri . Restated, there exist ∗ ∗ i ¯ ¯ some i and t ∗ such that either xi (t ∗ ) = d¯i , dx dt (t ) < 0, x i (t) ≥ di for −∞ < t ≤ t and S j (t; ψ) ≥ f j (d j ) for all dxi ∗ ∗ ∗ ∗ j = 1, 2, . . . , N and −∞ < t ≤ t or xi (t ) = d i , dt (t ) > 0, xi (t) ≤ d i for −∞ < t ≤ t and S j (t; ψ) ≤ f j (d j ) for all j = 1, 2, . . . , N and −∞ < t ≤ t ∗ . For the first case, we derive from (3) that 0>
N N X X dxi ∗ (t ) = −ai xi (t ∗ ) + Di j f j (x j (t ∗ )) + Diτj f j (x j (t ∗ − τi j (t ∗ ))) dt j=1 j=1
+
N X
Z
t∗
Di j
K i j (t ∗ − s) f j (x j (s)) ds + Bi Si (t ∗ ) + Ii
−∞
j=1
= −ai xi (t ) + Dii f i (xi (t ∗ )) + Diiτ f i (xi (t ∗ − τii (t ∗ ))) + Ii Z ∞ N X + D ii K ii (s) f i (xi (t ∗ − s)) ds + Bi Si (t ∗ ) + Di j f j (x j (t ∗ )) ∗
0
+
N X
j=1, j6=i
Diτj f j (x j (t ∗ − τi j (t ∗ ))) +
j=1, j6=i
N X
K i j (s) f j (x j (t ∗ − s)) ds
Di j 0
j=1, j6=i
≥ − ai d¯i + (Dii + Diiτ + D ii + Bi ) f i (d¯i ) −
∞
Z
N X
ρ j (|Di j | + |Diτj | + |D i j |) + Ii
j=1, j6=i
= gˇi (d¯i ) > gˇi (cˇi ) = 0, τ A ∗ ∗ ∗ ¯ ¯ due to (HA 1 ), (H2 ), Dii ≥ 0, D ii ≥ 0, Bi ≥ 0, | f j | ≤ ρ j , and f i (x i (t − τii (t ))) ≥ f i (di ), f i (x i (t − s)) ≥ f i (di ), ∗ from the monotonicity of f i and the definition of t . This is a contradiction. Hence, xi (t) ≥ d¯i , Si (t) ≥ f i (d¯i ) for all t ≥ 0. Similar arguments can be employed to show that xi (t) ≤ d i , Si (t) ≤ f i (d i ) for the second case. Therefore, each Λα is positively invariant with respect to the solution flow generated by system (3). This completes the proof. τ A A N Theorem 3. Under conditions (HA 1 ), (H2 ), (H3 ), and Dii ≥ 0, D ii ≥ 0, Bi ≥ 0, i = 1, 2, . . . , N , there exist 2 exponentially stable equilibria for system (3) with activation functions of class A.
Proof. Consider an equilibrium point (x ∗ T , S ∗ T )T = (x1∗ , x2∗ , . . . , x N∗ , S1∗ , S2∗ , . . . , S N∗ )T , where x ∗ ∈ Ω α for some (α1 , α2 , . . . , α N )T with αi = “l” or “r”, obtained in Theorem 1. We consider the following single-variable functions ωi (·), defined by " # N X τ ωi (λ) = µi pai − ( p − q) (|Di j | + |Di j | + |D i j |) − ( p − q) − qλ j=1
−q
N N X X p/q p/q (|Di j | + |D i j |)µ j η j − qeτ λ |Diτj |µ j η j − qµ N +i |Bi | p/q , j=1
(14)
j=1
where τ = max1≤i, j≤N τi j and i = 1, 2, . . . , N . By virtue of (HA 3 ), we have ωi (0) > 0 and ωi (λ) are continuous for λ ∈ [0, +∞). Moreover, ωi (λ) → −∞ as λ → +∞. Since (dωi (λ))/dλ < 0, ωi (λ) are strictly monotone decreasing functions on [0, +∞). Thus, there exist
X. Nie, J. Cao / Nonlinear Analysis: Real World Applications 10 (2009) 928–942
935
z i∗ > 0 such that " ωi (z i∗ )
= µi
N X pai − ( p − q) (|Di j | + |Diτj | + |D i j |) − ( p − q) − qz i∗
#
j=1
−q
N X
p/q
(|Di j | + |D i j |)µ j η j
− qeτ zi
∗
N X
|Diτj |µ j η j
p/q
− qµ N +i |Bi | p/q = 0
(15)
j=1
j=1
holds for all i = 1, 2, . . . , N . From (4) and (15), we can choose a sufficiently small positive number 0 < ε < min{z 1∗ , . . . , z ∗N , µ} such that " # N N X X p/q τ µi pai − ( p − q) (|Di j | + |Di j | + |D i j |) − ( p − q) − qε − q |Di j |µ j η j j=1
−q
N X
p/q
|D i j |µ j η j
j=1 +∞
Z
eεs K i j (s)ds − qeτ ε
N X
0
j=1
|Diτj |µ j η j
p/q
− qµ N +i |Bi | p/q > 0
(16)
j=1
holds for all i = 1, 2, . . . , N . Let (x T (t), S T (t))T = (x T (t; φ), S T (t; ψ))T be the solution to (3) with initial condition (φ T , ψ T )T ∈ Λα defined in (13), then N X d(xi (t) − xi∗ ) = −ai (xi (t) − xi∗ ) + Di j ( f j (x j (t)) − f j (x ∗j )) dt j=1
+
N X
Diτj ( f j (x j (t − τi j (t))) − f j (x ∗j )) + Bi (Si (t) − Si∗ )
j=1
+
N X
Z
t
Di j −∞
j=1
K i j (t − s)( f j (x j (s)) − f j (x ∗j )) ds,
(17)
d(Si (t) − Si∗ ) = −(Si (t) − Si∗ ) + f i (xi (t)) − f i (xi∗ ) dt for i = 1, 2, . . . , N . Let ϕi (t) = eεt |xi (t) − xi∗ | p/q , ζi (t) = eεt |Si (t) − Si∗ | p/q ,
i = 1, 2, . . . , N , i = 1, 2, . . . , N ,
(18)
then by using (4), (17) and (18), we can derive that εt
D ϕi (t) = εe |xi (t) − +
+
N X
xi∗ | p/q
p + eεt |xi (t) − xi∗ |( p−q)/q sgn(xi (t) − xi∗ ) q
Di j ( f j (x j (t)) − f j (x ∗j )) +
j=1
+
N X j=1
N X
" − ai (xi (t) − xi∗ )
Diτj ( f j (x j (t − τi j (t))) − f j (x ∗j ))
j=1
Z Di j 0
#
∞
K i j (s)( f j (x j (t − s)) − f j (x ∗j )) ds + Bi (Si (t) − Si∗ )
≤ εeεt |xi (t) − xi∗ | p/q −
N p εt p X ai e |xi (t) − xi∗ | p/q + eεt |Di j | f j0 (ς j )|x j (t) − x ∗j | q q j=1
× |xi (t) − xi∗ |( p−q)/q +
p εt e |Bi ||Si (t) − Si∗ | |xi (t) − xi∗ |( p−q)/q q
936
X. Nie, J. Cao / Nonlinear Analysis: Real World Applications 10 (2009) 928–942
+
N p εt X e |Diτj | f j0 (ξ j ) |x j (t − τi j (t)) − x ∗j | |xi (t) − xi∗ |( p−q)/q q j=1
N p X |D i j | + eεt q j=1
∞
Z 0
K i j (s) f j0 (ν j ) |x j (t − s) − x ∗j | |xi (t) − xi∗ |( p−q)/q ds,
where ς j , ξ j , ν j are three real numbers, respectively, lying between x j (t) and x ∗j , between x j (t − τi j (t)) and x ∗j as well as between x j (t − s) and x ∗j . Hence, D + ϕi (t) ≤ εeεt |xi (t) − xi∗ | p/q −
N p εt p X ai e |xi (t) − xi∗ | p/q + eεt |Di j | η j |x j (t) − x ∗j | q q j=1 N p εt X e |Diτj | η j |x j (t − τi j (t)) − x ∗j | |xi (t) − xi∗ |( p−q)/q q j=1
× |xi (t) − xi∗ |( p−q)/q + N p X + eεt |D i j | q j=1
+
∞
Z 0
K i j (s) η j |x j (t − s) − x ∗j | |xi (t) − xi∗ |( p−q)/q ds
p εt e |Bi | |Si (t) − Si∗ | |xi (t) − xi∗ |( p−q)/q . q
(19)
An application of Lemma 1 yields D + ϕi (t) ≤ εeεt |xi (t) − xi∗ | p/q − + eεt
N X
p/q
N p εt p − q εt X ai e |xi (t) − xi∗ | p/q + e |Di j | |xi (t) − xi∗ | p/q q q j=1
|x j (t) − x ∗j | p/q +
|Diτj | η j
|x j (t − τi j (t)) − x ∗j | p/q +
p/q |D i j | η j
Z
j=1
+ eεt
N X
N p − q εt X e |Diτj | |xi (t) − xi∗ | p/q q j=1
|Di j | η j
p/q
j=1
+e
εt
N X
∞
K i j (s) |x j (t − s) − x ∗j | p/q ds +
0
j=1 εt
N p − q εt X e |D i j | |xi (t) − xi∗ | p/q q j=1
p − q εt e |xi (t) − xi∗ | p/q q
+ e |Bi | |Si (t) − Si∗ | p/q (" # N X 1 qε − pai + ( p − q) (|Di j | + |Diτj | + |D i j |) + ( p − q) ϕi (t) ≤ q j=1 +q
p/q
N X
p/q
|Di j |η j
ϕ j (t) + qeετ
j=1
+q
N X
|Diτj |η j
p/q
ϕ j (t − τi j (t))
j=1
N X
p/q |D i j |η j
Z
j=1
∞
εs
e K i j (s)ϕ j (t − s)ds + q|Bi |
) p/q
ζi (t) .
(20)
0
Herein, the invariance property of Λα in Theorem 2 has been applied. Similarly, we have p/q
D + ζi (t) ≤ (ε − 1)ζi (t) + ηi
ϕi (t).
Let δ > 0 be an arbitrary real number and l0 =
(1 + δ)(kφ − x ∗ k p/q + kS(0) − S ∗ k p/q ) min {µi } 1≤i≤2N
(21)
X. Nie, J. Cao / Nonlinear Analysis: Real World Applications 10 (2009) 928–942
937
then ϕi (s) = eεs |xi (s) − xi∗ | p/q ≤ |xi (s) − xi∗ | p/q = |φi (s) − xi∗ | p/q < µi l0 , ζi (0) = |Si (0) −
Si∗ | p/q
∗ p/q
≤ kS(0) − S k
< µ N +i l0 ,
−∞ < s ≤ 0,
i = 1, 2, . . . , N .
In the following, we will prove that ϕi (t) < µi l0 ,
ζi (t) < µ N +i l0 ,
t > 0, i = 1, 2, . . . , N .
If (22) is not true, then there exist some i and ϕi (t ∗ ) = µi l0 , ϕ j (t) ≤ µ j l0
t∗
(22)
> 0 such that either
D + ϕi (t ∗ ) ≥ 0, (−∞ < t ≤ t ∗ , j = 1, 2, . . . , N ),
ζ j (t) ≤ µ N + j l0
(23)
(0 ≤ t ≤ t ∗ , j = 1, 2, . . . , N )
or ζi (t ∗ ) = µ N +i l0 ,
D + ζi (t ∗ ) ≥ 0,
ζ j (t) ≤ µ N + j l0 ϕ j (t) ≤ µ j l0
(0 ≤ t ≤ t ∗ , j = 1, 2, . . . , N ),
(24)
(−∞ < t ≤ t , j = 1, 2, . . . , N ). ∗
For the first case, it follows from (16) and (20) that # (" N X 1 D + ϕi (t ∗ ) ≤ qε − pai + ( p − q) (|Di j | + |Diτj | + |D i j |) + ( p − q) µi q j=1 +q
N X
p/q |Di j |µ j η j
+q
j=1 τε
+ qe
N X
p/q |D i j |µ j η j
Z
|Diτj |µ j
eεs K i j (s)ds
0
j=1 N X
+∞
) p/q ηj
+ qµ N +i |Bi |
p/q
l0 < 0,
(25)
j=1
that is a contraction. For the second case, by using (11) and (21), we have p/q
D + ζi (t ∗ ) ≤ (ε − 1)ζi (t ∗ ) + ηi
ϕi (t ∗ )
p/q
≤ [(ε − 1)µ N +i + ηi
µi ]l0 < 0,
(26)
that is also a contraction. So (22) holds. That is eεt |xi (t) − xi∗ | p/q ≤ K (kφ − x ∗ k p/q + kS(0) − S ∗ k p/q ), εt
e |Si (t) −
Si∗ | p/q
≤ K (kφ − x k
∗ p/q
∗ p/q
+ kS(0) − S k
),
i = 1, 2, . . . , N , i = 1, 2, . . . , N
(27)
for any t > 0, where K = (1+δ)(max1≤i≤2N {µi })/(min1≤i≤2N {µi }). Let M = 2(K N )q/ p , from (27) and Lemma 2, we get that kx(t) − x ∗ k + kS(t) − S ∗ k ≤ M(kφ − x ∗ k + kS(0) − S ∗ k) e−(qε/ p)t , for all t > 0. Therefore, (x T (t), S T (t))T is exponentially convergent to (x ∗ T , S ∗ T )T and the exponential convergence rate is qε/ p. This completes the proof. τ B N Theorem 4. Under conditions (HB 1 ), (H2 ), and Dii ≥ 0, D ii ≥ 0, Bi ≥ 0, i = 1, 2, . . . , N , there exist 2 exponentially stable equilibria for system (3) with activation functions of class B.
Proof. Consider an equilibrium (x ∗ T , S ∗ T )T = (x1∗ , x2∗ , . . . , x N∗ , S1∗ , S2∗ , . . . , S N∗ )T where x ∗ ∈ Ω α for some (α1 , α2 , . . . , α N )T with αi = “l” or “r”, obtained in Theorem 1. From (HB 2 ), we can choose a sufficiently small positive number ε > 0 such that µi [ pai − ( p − q) − qε] − qµ N +i |Bi | p/q > 0 for i = 1, 2, . . . , N . Let
938
X. Nie, J. Cao / Nonlinear Analysis: Real World Applications 10 (2009) 928–942
(x T (t), S T (t))T = (x T (t; φ), S T (t; ψ))T be the solution to (3) with the initial condition (φ T , ψ T )T ∈ Λα defined in (13). Note that the invariance property of Λα and f i (xi (t; φ)) ≡ u i or vi when (φ T , ψ T )T ∈ Λα , then d(xi (t) − xi∗ ) = −ai (xi (t) − xi∗ ) + Bi (Si (t) − Si∗ ), dt d(Si (t) − Si∗ ) = −(Si (t) − Si∗ ), i = 1, 2, . . . , N . dt
i = 1, 2, . . . , N , (28)
Let ϕi (t) = eεt |xi (t) − xi∗ | p/q , εt
ζi (t) = e |Si (t) −
Si∗ | p/q ,
i = 1, 2, . . . , N ,
(29)
i = 1, 2, . . . , N ,
then it follows from (28) and (29) that p εt e |xi (t) − xi∗ |( p−q)/q sgn(xi (t) − xi∗ ) q × [−ai (xi (t) − xi∗ ) + Bi (Si (t) − Si∗ )]
D + ϕi (t) = εeεt |xi (t) − xi∗ | p/q +
≤
1 [(qε − pai + p − q)ϕi (t) + q|Bi | p/q ζi (t)]. q
Similarly, we have p + D ζi (t) = ε − ζi (t). q
(30)
(31)
The remaining details of the proof is similar to those in Theorem 3 and here are omitted. This completes the proof.
Corollary 1. Each of these 2 N subsets Λα of C((−∞, 0], R2N ), defined in (13), lies in the basin of attraction for the unique equilibrium point in Λα , under the assumptions of Theorem 3 or Theorem 4. B If we take p = q = 1 in (11) and (12), then (HA 3 ), (H2 ) reduces to the following conditions, respectively, 0
(HA 3 ) There exist constants µi > 0 (i = 1, 2, . . . , 2N ) such that µi ai −
N X (|Di j | + |Diτj | + |D i j |)µ j η j − µ N +i |Bi | > 0, j=1
(32)
−µi ηi + µ N +i > 0 hold for all i = 1, 2, . . . , N , where η j , j = 1, 2, . . . , N are real numbers satisfying max{ f j0 (ξ )|ξ = cˇ j , aˆ j } < η j < min{ f j0 (ξ )|ξ = p˜ j , q˜ j }. 0
(HB 2 ) There exist constants µi > 0 (i = 1, 2, . . . , 2N ) such that µi ai − µ N +i |Bi | > 0,
i = 1, 2, . . . , N .
(33)
Applying Theorems 3 and 4 above, respectively, we can easily obtain the following corollaries: τ A A N Corollary 2. Under conditions (HA 1 ), (H2 ), (H3 ), and Dii ≥ 0, D ii ≥ 0, Bi ≥ 0, i = 1, 2, . . . , N , there exist 2 exponentially stable equilibria for system (3) with activation functions of class A. 0
τ Corollary 3. Under conditions gˆi (−1) < 0, gˇi (1) > 0, (HB 2 ), and Dii ≥ 0, D ii ≥ 0, Bi ≥ 0, i = 1, 2, . . . , N , N there exist 2 exponentially stable equilibria for system (3) with the standard piecewise linear function. 0
Remark 2. Since there exist time delays, the method used in [18] cannot be applied to system (3). Moreover, our method used here is valid for a class of functions and the stability analysis of neural networks with time-varying and distributed delays.
X. Nie, J. Cao / Nonlinear Analysis: Real World Applications 10 (2009) 928–942
939
4. Two illustrative examples We consider the following two-dimensional competitive neural networks with time-varying and distributed delays 2 2 X X dxi (t) = −a x (t) + D f (x (t)) + Diτj f j (x j (t − τi j (t))) i i i j j j dt j=1 Z t j=1 Z t + D i1 K i1 (t − s) f 1 (x1 (s)) ds + D i2 K i2 (t − s) f 2 (x2 (s)) ds (34) −5 −6 +Bi Si (t) + Ii , i = 1, 2, dSi (t) = −S (t) + f (x (t)), i = 1, 2 i i i dt for t > 0, where K i1 (s) = satisfy the assumptions (4).
2 e−2s , 1−e−10
K i2 (s) =
e−s 1−e−6
(i = 1, 2). We can easily check that K i j (s) (i, j = 1, 2) above
1 τ = D τ Example 1. For system (34), take a1 = 1, a2 = 2, D11 = D11 11 = B1 = 2 , D22 = D22 = D 22 = τ τ B2 = 1, D12 = D12 = 0.15, D21 = D21 = 0.25, D 12 = D 21 = −0.1, f i (ξ ) ≡ f (ξ ) = arctan ξ, τi j (t) ≡ τ (t) = e−t , Ii = 0, (i, j = 1, 2). Direct computation gives gˆ 1 (ξ ) = −ξ + 2 arctan ξ + 0.1 π, gˇ 1 (ξ ) = −ξ + 2 arctan ξ − 0.1 π , gˆ 2 (ξ ) = −2 ξ + 4 arctan ξ + 0.2 π , gˇ 2 (ξ ) = −2 ξ + 4 arctan ξ − 0.2 π . Herein, the parameters satisfy conditions in Corollary 2: Condition (HA 1 ):
0<
a1 D11 +
τ D11
=
+ D 11 + B1
1 < 1, 2
0<
a2 D22 +
τ D22
=
+ D 22 + B2
1 < 1. 2
Condition (HA 2 ): gˆ 1 ( p˜ 1 ) = −0.2566 < 0,
gˇ 1 (q˜1 ) = 0.2566 > 0,
gˆ 2 ( p˜ 2 ) = −0.5133 < 0,
gˇ 2 (q˜2 ) = 0.5133 > 0.
Condition
0 (HA 3 ):
τ τ | + |D 11 |) − η2 (|D12 | + |D12 | + |D 12 |) − |B1 | = 0.025 > 0, a1 − η1 (|D11 | + |D11 τ τ | + |D 21 |) − η2 (|D22 | + |D22 | + |D 22 |) − |B2 | = 0.1 > 0, a2 − η1 (|D21 | + |D21 0
where µ1 = µ2 = µ3 = µ4 , η1 = η2 = 0.25 are chosen in (HA ˆ 1 = aˆ 2 3 ); p˜ 1 = p˜ 2 = −1, q˜1 = q˜2 = 1, a ˆ ˆ ˇ ˇ = −1.8248, b1 = b2 = −0.3384, cˆ1 = cˆ2 = 2.7607, aˇ 1 = aˇ 1 = −2.7607, b1 = b2 = 0.3384, cˇ1 = cˇ2 = 1.8248, d 1 = d 2 = −1.7321, d¯1 = d¯2 = 1.7321. From Corollary 2, it follows that system (34) has four equilibria, which is exponentially stable. Figs. 4 and 5 depict the time responses of state variables x1 (t), x2 (t), S1 (t), S2 (t) with step h = 0.01. From Figs. 4 and 5, it is easy to see that there are four exponentially stable equilibria (x1∗ , x2∗ , S1∗ , S2∗ )T = (2.6664, 2.6664, 1.2120, 1.2120)T , (−2.6664, −2.6664, −1.2120, −1.2120)T , (1.9888, −1.9888, 1.1049, −1.1049)T , (−1.9888, 1.9888, −1.1049, 1.1049)T in this system, as confirmed by our theory. τ = D τ τ Example 2. For system (34), take D11 = D11 11 = 2, D12 = D12 = D 12 = D21 = D21 = D 21 = 1, D22 = 1 τ D22 = D 22 = 3, I1 = 1.5, I2 = 3, ai = 2, Bi = 1, f i (ξ ) ≡ f (ξ ) = 2 (|ξ +1|−|ξ −1|), τi j (t) ≡ τ (t) = e−t (i, j = 1, 2). Direct computation gives gˆ 1 (ξ ) = −2 ξ +7 f (ξ )+4.5, gˇ 1 (ξ ) = −2 ξ +7 f (ξ )−1.5, gˆ 2 (ξ ) = −2 ξ +10 f (ξ )+6, gˇ 2 (ξ ) = −2 ξ + 10 f (ξ ). The parameters satisfy the conditions in Corollary 3:
gˆ 1 (−1) = −0.5 < 0, 0
gˇ 1 (1) = 3.5 > 0,
gˆ 2 (−1) = −2 < 0,
gˇ 2 (1) = 8 > 0.
Condition (HB 2 ): ai > |Bi |, i = 1, 2, 0 where µ1 = · · · = µ4 are chosen in (HB ˆ 1 = −1.25, bˆ1 = −0.9, cˆ1 = 5.75, aˇ 1 = −4.25, bˇ1 = 0.3, cˇ1 = 2 ); a 2.75, aˆ 2 = −2, bˆ2 = −0.75, cˆ2 = 8, aˇ 2 = −5, bˇ2 = 0, cˇ2 = 5, d 1 = d 2 = −1, d¯1 = d¯2 = 1. From Corollary 3,
940
X. Nie, J. Cao / Nonlinear Analysis: Real World Applications 10 (2009) 928–942
Fig. 4. Transient response of state variables x1 (t) and x2 (t).
Fig. 5. Transient response of state variables S1 (t) and S2 (t).
we can conclude that this system has four exponentially stable equilibria. Figs. 6 and 7 depict the time responses of state variables x1 (t), x2 (t), S1 (t), S2 (t) with step h = 0.01. From Figs. 6 and 7, it is also easy to see that there are four exponentially stable equilibria (x1∗ , x2∗ , S1∗ , S2∗ )T = (5.75, 8, 1, 1)T , (−4.25, −5, −1, −1)T , (2.75, −2, 1, −1)T , (−1.25, 5, −1, 1)T in this system, as confirmed by our theory. 5. Conclusions In this paper, two classes of activation functions, which are commonly employed in neural networks, have been considered. By means of geometrical method and inequality technique, several novel sufficient conditions have been derived ensuring the existence of 3 N equilibria and exponential stability of 2 N equilibria for competitive neural networks with time-varying and distributed delays. Compared with [18], the method used here is valid for a class of functions and the stability analysis of neural networks with time-varying and distributed delays. Acknowledgments The authors appreciate the editor’s work and the reviewer’s insightful comments and constructive suggestions. This work was jointly supported by the National Natural Science Foundation of China under Grant 60574043 and the Natural Science Foundation of Jiangsu Province of China under Grant BK2006093.
X. Nie, J. Cao / Nonlinear Analysis: Real World Applications 10 (2009) 928–942
941
Fig. 6. Transient response of state variables x1 (t) and x2 (t).
Fig. 7. Transient response of state variables S1 (t) and S2 (t).
References [1] L.O. Chua, L. Yang, Cellular neural networks: Theory, IEEE Trans. Circuits Syst. 35 (1988) 1257–1272. [2] J. Hopfield, Neurons with graded response have collective computational properties like those of two state neurons, Proc. Natl. Acad. Sci. USA 81 (1984) 3088–3092. [3] M. Morita, Associative memory with non-monotone dynamics, Neural Netw. 6 (1993) 115–126. [4] J. Foss, A. Longtin, B. Mensour, J. Milton, Multistability and delayed recurrent loops, Phys. Rev. Lett. 76 (1996) 708–711. [5] A. Meyer-Baese, F. Ohl, H. Scheich, Singular perturbation analysis of competitive neural networks with different time-scales, Neural Comput. 8 (1996) 1731–1742. [6] J. Cao, J. Wang, Asymptotic stability of a general class of recurrent neural networks with time-varying delays, IEEE Trans. Circuits Syst. I 50 (1) (2003) 34–44. [7] J. Cao, Q. Song, Stability in Cohen–Grossberg-type bidirectional associative memory neural networks with time-varying delays, Nonlinearity 19 (2006) 1601–1607. [8] K. Yuan, J. Cao, Exponential stability and periodic solutions of fuzzy cellular neural networks with time-varying delays, Neurocomputing 69 (2006) 1619–1627. [9] Q. Zhang, X. Wei, J. Xu, Stability analysis for cellular neural networks with variable delays, Chaos Solitons Fractals 28 (2006) 331–336. [10] B. Liu, L. Huang, Existence and stability of almost periodic solutions for shunting inhibitory cellular neural networks with continuously distributed delays, Phys. Lett. A 349 (2006) 177–186. [11] Q. Song, J. Cao, Z. Zhao, Periodic solutions and its exponential stability of reaction-diffusion recurrent neural networks with continuously distributed delays, Nonlinear Anal. Real World Appl. 7 (2006) 65–80.
942
X. Nie, J. Cao / Nonlinear Analysis: Real World Applications 10 (2009) 928–942
[12] H. Jiang, Z. Teng, Global exponential stability of cellular neural networks with time-varying coefficients and delays, Neural Netw. 17 (10) (2004) 1415–1425. [13] H. Jiang, Z. Teng, A new criterion on the global exponential stability for cellular neural networks with multiple time-varying delays, Phys. Lett. A 338 (2005) 461–471. [14] A. Meyer-Baese, S. Pilyugin, Y. Chen, Global exponential stability of competitive neural networks with different time scales, IEEE Trans. Neural Netw. 14 (3) (2003) 716–719. [15] A. Meyer-Baese, S. Pilyugin, A. Wismuller, S. Foo, Local exponential stability of competitive neural networks with different time scales, Eng. Appl. Artifical Intell. 17 (2004) 227–232. [16] H. Lu, A. Shun-ichi, Global exponential stability of multitime scale competitive neural networks with nonsmooth functions, IEEE Trans. Neural Netw. 17 (5) (2006) 1152–1164. [17] H. Lu, Z. He, Global exponential stability of delayed competitive neural networks with different time scales, Neural Netw. 18 (2005) 243–250. [18] Y. Mao, Multistability of competitive neural networks with different time scales, in: Communications, Circuits and Systems, 2005. Proceedings. 2005 International Conference on, vol. 2, 2005, pp. 939–943. [19] C.Y. Cheng, K.H. Lin, C.W. Shih, Multistability in recurrent neural networks, SIAM J. Appl. Math. 66 (4) (2006) 1301–1320. [20] C.Y. Cheng, K.H. Lin, C.W. Shih, Multistability and convergence in delayed neural networks, Physica D 225 (2007) 61–74. [21] Z.G. Zeng, J. Wang, Multiperiodicity and exponential attractivity evoked by periodic external inputs in delayed cellular neural networks, Neural Comput. 18 (2006) 848–870. [22] J. Kuang, Applied Inqualities, 3rd edn, Shandong Science and Technology Press, 2004 (in Chinese).