Analysis of multiple quasi-periodic orbits in recurrent neural networks

Neurocomputing 162 (2015) 85–95

Contents lists available at ScienceDirect

Neurocomputing journal homepage: www.elsevier.com/locate/neucom

Analysis of multiple quasi-periodic orbits in recurrent neural networks R.L. Marichal n, J.D. Piñeiro Department of Computer and Systems Sciences, University of La Laguna, Avda. Francisco Sánchez S/N, Edf. Informática, 38206 Tenerife, Canary Islands, Spain

art ic l e i nf o

a b s t r a c t

Article history: Received 30 November 2014 Received in revised form 27 February 2015 Accepted 1 April 2015 Communicated by Rongrong Ji Available online 17 April 2015

In this paper we consider a recurrent neural network model consisting of two neurons and analyze its stability using the associated characteristic model. In order to analyze the multiple quasi-periodic orbits, the strong resonance of this system, in particular that known as the R2 bifurcation, is also studied. In the case of two neurons, one necessary condition that yields the bifurcation is found. In addition, the direction of the R2 bifurcation is determined by applying normal form theory and the center manifold theorem. The simple conditions for ensuring the existence of multiple quasi-periodic orbits are given. The strong resonance phenomenon is analyzed using numerical simulations and is related with the codimension-two bifurcation of the high-iteration map. & 2015 Elsevier B.V. All rights reserved.

Keywords: Recurrent neural networks Bifurcation Strong resonance Quasi-periodic orbit

1. Introduction The purpose of this paper is to present certain results on the analysis of the dynamics of a recurrent neural network. The particular network in which we are interested is the Williams– Zipser network, also known as a discrete recurrent neural network in [1]. Its two-neuron state equation is x1 ðk þ 1Þ ¼ f ðw11 x1 ðkÞ þ w12 x2 ðkÞÞ

ð1aÞ

x2 ðk þ 1Þ ¼ f ðw21 x1 ðkÞ þ w22 x2 ðkÞÞ;

ð1bÞ

where xi ðkÞ is the ith neuron output, wij are the weight factors of the neuron outputs, and f ðÞ is a continuous, bounded, monotonically increasing function, such as the hyperbolic tangent. From the point of view of dynamic system theory, it is interesting to study the equilibrium or fixed points. The dynamics at these points do not change in time. Their character or stability determines the local behavior of nearby trajectories. A fixed point systems can attract (sink), repel (source) or have directions of attraction and repulsion (saddle) of close trajectories [2, Chapter 3]. Besides fixed points, there exist periodic trajectories, quasi-periodic trajectories or even chaotic sets, each with its own stability characterization. All of these features are similar in a class of topologically equivalent systems [3, Chapter 2]. With respect to recurrent neural networks as systems, several dynamics-related results are available in the literature. The most general result is derived in Marcus and Westervelt [4] using the Lyapunov stability n

Corresponding author. Tel.: þ 34 922845234; fax: þ 34 922318288. E-mail address: [email protected] (R.L. Marichal).

http://dx.doi.org/10.1016/j.neucom.2015.04.001 0925-2312/& 2015 Elsevier B.V. All rights reserved.

theorem. They establish that the only stable equilibrium states that can exist for a symmetric weight matrix are either fixed points or period-two cycles. More recently, Cao [5] proposed less restrictive but more complex conditions. Wang [6] describes an interesting type of trajectory, the quasi-periodic orbits. Passeman [7] obtains some experimental results, such as the coexistence of periodic cycles, chaotic attractors and quasi-periodic trajectories. In [8], Tino gives the position, number and stability types of fixed points for a two-neuron discrete recurrent network with non-zero weights. The rest of this paper is divided into four additional sections. Section 2 consists of an introduction to bifurcation theory. In Section 3, the local stability of the recurrent neural network and the necessary conditions for the onset of the R2 strong resonance bifurcation are analyzed. In Section 4, conditions for the direction of the bifurcation and for the existence of quasi-periodic orbits are established. In Section 5, we show the bifurcation diagram and dynamic behavior simulations of the network with the hyperbolic tangent as the activation function.

2. Bifurcation theory overview In general, when system parameters are slowly changed, the system dynamics varies smoothly. Those dynamics belong to the same class of topologically equivalent systems. Sometimes, the variation of parameters can reach a critical point at which it is no longer topologically equivalent. This is called a bifurcation point [9, Chapter 3], and the system will exhibit new behaviors. In order to determine the new dynamics associated to the bifurcation point, normal form theory is a useful tool [9, Chapter 2]. This theory is a

86

R.L. Marichal, J.D. Piñeiro / Neurocomputing 162 (2015) 85–95

technique for transforming the nonlinear dynamical system in critical situation into certain simple standard forms known as normal forms of the bifurcation. In fact, the dynamical behavior of these forms is known and the dynamical system is locally topologically equivalent to them on the bifurcation point. The simplest bifurcations are those associated with the fixed points of the system under analysis. These bifurcations occur at certain critical eigenvalues of the linearized equivalent system at the fixed point. If the bifurcation is characterized by a one-dimensional manifold in parameter space is called a codimension-one bifurcation. More generally, the codimension of a bifurcation is the number of parameters which must be varied for the bifurcation to occur. The three simplest codimension-one bifurcations in discrete-time systems are known as Neimark–Sacker, fold and period-doubling which appear when one eigenvalue is 1, þ 1 and a pair of complex conjugate eigenvalues have a unit modulus, respectively [9, Chapter 4]. The Neimark–Sacker bifurcation is related with an interesting dynamical behavior characterized by the presence of one quasi-periodic orbit. The periodicity of this particular trajectory is irrational, that makes that the set of points it traverses looks like a continuous closed orbit. In the bifurcation curve exist some points where the periodicities of trajectories are rational and appears as a new codimension-two bifurcation. The nomenclature of this rational periodicity is represented by q : p, this number indicates that in p mapping iterations, the state completes q revolutions. This terminology is related to the phase locking phenomenon [9, Chapter 7] (also called entrainment or synchronization). Phase locking produces a periodic solution that persists generically as parameters are varied. In contrast, quasiperiodicity is a codimension-one phenomenon, which is thus generically destroyed by perturbation. The result is a wellknown bifurcation diagram in the two-parameter plane called the “Arnold tongue” scenario [9, Chapter 7]. The Arnold tongues are situated next to the unit circle, where the rotational number [2] is constant and rational and has a value of p/q, and is surrounded by zones in which it is irrational. For example, in an integrate-and-fire continuous-time neural network model with sinusoidal input, the rational q : p mode-locked solution is identified by a spike train in which p firing events occur in period qf, where f is the forcing input period. In general, codimension-two bifurcations are related with two critical eigenvalues. For example, the rotational number mentioned above is rational then appears a codimension-two bifurcation with the following condition with respect to the eigenvalues: λ1;2 ¼ e 7 iθ0 ;

θ0 ¼

2πp : q

These bifurcations are known as resonances or strong resonances and they are related with the destruction of quasi-periodic orbits associated to Neimark–Sacker bifurcations. In particular, the R2 bifurcation is a resonance represented by the rotational number 1:2. In fact, for neural network (1), the numerical simulations on Arnold tongues [10] conclude that the most important Arnold tongue is associated with a 1:2 rotational number, that is, the most frequent resonance is 1:2. Additionally, in R2 resonance the most interesting dynamical behavior is the presence of multiple quasiperiodic orbits under some conditions of normal form coefficients (see Section 4). With respect to the analysis of bifurcations in neural networks there exist some previous studies. Refs. [11,12] study the discrete-time Hopfield neural network and specifically, their Neimark–Sacker bifurcation and the stability of the associated quasi-periodic orbit. In [10], a simple stability condition for the Neimark–Sacker bifurcation in a two-neuron discrete recurrent neural network is given and the Arnold tongues (related to the phase-locking phenomenon) are studied. In [13], numerical estimates of the Neimark–Sacker bifurcation direction in a

Hopfield neural network with two neurons and one time delay are given. Refs. [14,15] complete the bifurcation results for twoneuron discrete-time Hopfield neural networks with time delay and only self-connections between the neurons (no interactions between them). In [16,13,17], results are generalized to n-neuron Hopfield neural networks. In [18], the discrete-time dynamics of a two-neuron network with recurrent connectivity, known as ring neural networks, are studied, showing for specific parameter configurations the relationship between dynamics and the evolution of the external outputs. In [19], the authors consider a system of delayed differential equations representing a simple model for a ring of neurons with some restrictions on the parameters, giving the geometric locus in parameter space that results in a Hopf bifurcation. In [20], a discrete neural network with two neurons is considered and the period-doubling bifurcation is analyzed. In this paper the stability of the bifurcation focuses on the zero fixed point. In contrast, [21] studies the saddle-node, pitchfork and Hopf bifurcations in a recurrent neural network. Additionally, Guo [16] presents some results for a codimension-two bifurcation ring neural network. Finally, Folias and Ermentrout [22] analyze the strong resonance (1 : 2) of a biological neural network model. With regard to these references, this paper introduces the study of period-doubling at fixed points different from zero, and also includes novel results for strong resonance in high-iteration maps of Hopfield discrete neural networks. Generally, the typical procedure is to analyze the quasi-periodic orbit associated with the Neimark–Sacker bifurcation [16,13,17,10], or to propose conditions for the non-existence of said quasi-periodic orbits by ensuring that the system has fixed stable points [5]. The main novelty of this paper with respect to previous studies is the analysis of multiple quasi-periodic orbits and the destruction process of a quasiperiodic from a numerical simulation approach. Additionally, we show the relationships with the codimension-two bifurcation of the high-iteration map in R2 strong resonance.

3. Local stability and resonance bifurcation conditions In the exposition below, a two-neuron neural network is considered. It is usual for the activation function to be a sigmoid function or a tangent hyperbolic function. Here we only need the following assumption: f A C 1 ðRÞ;

f ð0Þ ¼ 0;

0

f ð0Þ a 0;

ðH:1Þ

1

where C ðRÞ is the functions set with continuous first derivative. In order to simplify the notation we denote (x1, x2) as (x; y). First, the analytical condition of a fixed point can be shown as x ¼ f ðw11 x þ w12 yÞ

ð2aÞ

y ¼ f ðw21 x þ w22 yÞ:

ð2bÞ

Taking into consideration assumption (H.1), it is clear that (0,0) is a fixed point. Introducing the new variables σ1 and σ2, which depend on the diagonal weights and the weight matrix determinant, we have 0

σ1 ¼

w11 f ðf 0

1

σ 2 ¼ j W j f ðf

0

ð0ÞÞ þ w22 f ðf 2

1

ð0ÞÞ2 :

1

ð0ÞÞ

ð3Þ ð4Þ

The Jacobian matrix of the linearized system evaluated at the fixed point is " # 0 1 0 1 w11 f ðf ð0ÞÞ w12 f ðf ð0ÞÞ A¼ ð5Þ 0 1 0 1 w21 f ðf ð0ÞÞ w22 f ðf ð0ÞÞ

R.L. Marichal, J.D. Piñeiro / Neurocomputing 162 (2015) 85–95

87

σ2

The characteristic equation of the above matrix is 2

λ 2σ 1 λ þ σ 2 ¼ 0:

ð6Þ

Neimark−Sacker Bifurcation

With respect to fixed point eigenvalues, the bifurcations occur at certain critical eigenvalues. These critical eigenvalues appear at certain parameter values of the system. The normal form associated with codimension-two bifurcations is characterized by two parameters. System (1) shows codimension-two bifurcations known as strong resonance, in particular the 1:2 resonance (R2). This bifurcation appears when both eigenvalues are minus one, that is, when λ1 ¼ λ2 ¼  1:

σ =1

R2

2

σ1=−1

ð7Þ

σ1

Firstly, the parameter condition associated with the R2 bifurcation is given by the following lemma.

Period−Doubling Bifurcation

Lemma 1. If σ 1 ¼ 1

ð8Þ

σ2=−2σ1−1

and σ 2 ¼ 1;

ð9Þ

then the R2 bifurcation is present. Proof. Considering eigenvalue equation (6), qffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi □ λ 7 ¼ σ 1 7 ðσ 1 Þ2  σ 2 ¼  1:

Fig. 1. Regions of stability and the R2 bifurcation point.

ð10Þ otherwise

In order to simplify the notation, we use σ n to indicate the parameter values where the bifurcation appears, that is, considering expressions (8) and (9), σ n1 ¼ 1 and σ n2 ¼ 1, respectively. The R2 bifurcation is related with the Neimark–Sacker and period-doubling bifurcations. In fact, the R2 eigenvalues condition (λ1 ¼ λ2 ¼  1) satisfy simultaneously the Neimarks–Sacker (j λ1 j ¼ j λ1 j ¼ 1 and λ1 ¼ Conjugateðλ2 Þ) and period-doubling (λ1 ¼  1 or λ2 ¼  1) eigenvalues conditions. The two following lemmas establish relation between σ n1 and σ n2 parameters corresponding to these codimensionone bifurcations.

λ þ ¼ 1:

□

The R2 occurs while moving along the period-doubling bifurcation [9, Chapter 4]. Fig. 1 shows the period-doubling bifurcation and Neimark–Sacker bifurcation [9, Chapter 4] curves and the R2 bifurcation represented by R2 point. Furthermore, the R2 point corresponds to the intersection between both codimension-one bifurcation curves.

Lemma 2. If σ n2 ¼ 1

ð11Þ

and j σ n1 j o 1;

ð12Þ

then the Neimark–Sacker bifurcation is present. Proof. Considering eigenvalue equation (6) and lemma condition (12), qffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi ð13Þ λ 7 ¼ σ n1 7 i σ n2  ðσ n1 Þ2 with the other lemma condition (11) the modulus of complex eigenvalues is qffiffiffiffiffi □ j λ 7 j ¼ σ n2 ¼ 1: Lemma 3. If σ n2 ¼ 2σ n1  1;

ð14Þ

then the period-doubling bifurcation is present. Proof. Taking into account the lemma condition and the eigenvalue equation (6) yields qffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi λ 7 ¼ σ n1 7 ðσ n1 þ 1Þ2 n

and it supposes that σ 1 4  1 then λ  ¼  1;

4. Direction 1:2 resonance bifurcation In order to determine the direction of the R2 bifurcation, it is necessary to use normal form theory [9, Chapter 5]. In this section, a formula for determining the direction of the R2 bifurcation of system (1) at σ 1 ¼ σ n1 and σ 2 ¼ σ n2 is presented by employing the normal form method and the center manifold theorem. For most of the models in the literature [1], the transfer function f is the hyperbolic tangent function. Here, we assume that the transfer functions in (1) satisfy f A C 3 ðRÞ;

f ð0Þ ¼ f ″ð0Þ ¼ 0;

0

f ð0Þf ‴ð0Þ a 0;

ðH:2Þ

where C 3 ðRÞ is the functions set with continuous third derivative. In order to apply normal form theory, it is necessary to redefine the neural network map (1) as ! ! ! ! F 1 ðx; σ n Þ w11 w12 x1 ðkÞ x~1 0 ¼ f ð0Þ ; ð15Þ þ F 2 ðx; σ n Þ w21 w22 x2 ðkÞ x~2 where x ¼ ðx1 ; x2 ÞT A R2 , σ n ¼ ðσ n1 ; σ n2 ÞT A R2 and x~1 , x~2 are the k þ1iteration of x1 and x2, respectively. In (H.1), the Fi terms are F 1 ðx; σ n Þ ¼ 12 B1 ðx; xÞ þ 16 C 1 ðx; x; xÞ þ Oð‖x‖4 Þ F 2 ðx; σ n Þ ¼ 12 B2 ðx; xÞ þ 16 C 2 ðx; x; xÞ þ Oð‖x‖4 Þ;

88

R.L. Marichal, J.D. Piñeiro / Neurocomputing 162 (2015) 85–95

In order to determine the c and d parameters in the neural network, the relationships involving the neural network weight derived in the following lemmas are necessary.

where  2 X ∂2 F i ðξ; σ 1 ; σ 2 Þ Bi ðx; yÞ ¼  ∂ξj ∂ξk σ j;k ¼ 1 2 X

¼ f ″ð0Þ

xj yk 1

¼ σ n1 ;σ 2 ¼ σ nn2

wij wik xj yk ¼ 0

ð16Þ

j;k ¼ 1

0

α2 ¼ β2 ¼ w12 w21 f ð0Þ2

 2 X ∂3 F i ðξ; σ 1 ; σ 2 Þ C i ðx; y; zÞ ¼ ∂ξ ∂ξ ∂ξ  j

j;k;l ¼ 1

2 X

¼ f ‴ð0Þ

k

l

xj yk zl

0

α ¼ 1 þf ð0Þw11 :

σ 1 ¼ σ n1 ;σ 2 ¼ σ n2

ð17Þ

wij wik wil xj yk zl

It is also necessary to calculate the Jacobian matrix and the Jacobian transpose matrix generalized eigenvectors. We thus have Aðσ n1 ; σ n2 Þq0 ¼  q0 ;

Aðσ n1 ; σ n2 Þq1 ¼  q1 þq0

ð18Þ

and AT ðσ n1 ; σ n2 Þp0 ¼  p0 ;

AT ðσ n1 ; σ n2 Þp1 ¼  p1 þ p0 ;

ð19Þ

where Aðσ n1 ; σ n2 Þ represents the Jacobian matrix given by Eq. (5) evaluated at the fixed point ð0; 0Þ with bifurcation parameters σ n1 and σ n2 . In addition, q0, q1 are the generalized eigenvectors of the Jacobian matrix and p0, p1 are the generalized eigenvectors of the transpose Jacobian matrix. These eigenvectors satisfy the normalization conditions 〈p0 ; q1 〉 ¼ 〈p1 ; q0 〉 ¼ 1;

ð20Þ

where 〈:; :〉 is the standard scalar product. In the neural network mapping (15), q0, q1, p0 and p1 are 2 3 " w w #  2w βf 0 ð0Þ 11 22 21 2w21 ð21Þ ; q1 ¼ 4 β þ 2 5 q0 ¼ β 1 β

p0 ¼ β

 w22 2w12

11

2

#

1

;

p1 ¼ 4

 2w

β 0 12 f ð0Þ

βþ2 β

ð26Þ

Proof. Taking into account the variable definitions of σ1 and σ2, given by Eq. (4), and the bifurcation parameter σ n2 ¼ 1 yields 0

0

ðw11 w22  w12 w21 Þf ð0Þ2 ¼ ðw11 þ w22 Þf ð0Þ  1 or 0

0

0

ðw11 f ð0Þ þ 1Þðw22 f ð0Þ þ 1Þ ¼ w12 w21 f ð0Þ2 Considering the definitions of α and β, given by (26) and (23), respectively, 0

αβ ¼ w12 w21 f ð0Þ2 :

ð27Þ n

On the other hand, with the bifurcation parameter σ 1 ¼  1 0

ðw11 þ w22 Þf ð0Þ ¼  2 the relationship between α and β is α ¼  β:

ð28Þ

Substituting the above equation in (27) 0

α2 ¼ β2 ¼ w12 w21 f ð0Þ2 :

□

Lemma 5. Given (H.1), (H.2) and critical bifurcation parameter σ n1 ¼  1, then 0

f ð0Þðw11  w22 Þ ¼  2β:

3 5;

ð25Þ

where

j;k;l ¼ 1

"w

Lemma 4. Given (H.1), (H.2) and critical bifurcation parameters, σ n1 ¼  1 and σ n2 ¼ 1, then

ð22Þ

ð29Þ

Proof. Considering the bifurcation parameter σ n1 ¼  1 0

ðw11 þ w22 Þf ð0Þ ¼  2

where 0

β ¼ 1 þ w22 f ð0Þ:

ð23Þ

and subtracting 2w22 from both sides of this expression 0

0

ðw11  w22 Þf ð0Þ ¼  2ð1 þ w22 f ð0ÞÞ;

The dynamic behavior of system (1) can be represented for σ1, σ2 near σ n1 and σ n2 , respectively, by

with β variable (23)

η1 ðk þ 1Þ ¼ η1 ðkÞ þ η2 ðkÞ

ð24aÞ

ðw11  w22 Þf ð0Þ ¼  2β:

ð24bÞ

Lemma 6. Given (H.1), (H.2) and critical bifurcation parameters σ n1 ¼  1 and σ n2 ¼ 1, then

η2 ðk þ 1Þ ¼ γ 1 η1 ðkÞ þðγ 2  1Þη2 ðkÞ þ 16 cðγ 1 ; γ 2 Þη31 ðkÞ þ 12 dðγ 1 ; γ 2 Þη21 ðkÞη2 ðkÞ;

where η1, η2 and γ1, γ2 are state coordinates and parameters of the normal form, respectively. The R2 bifurcation appears when the parameter values are zero. The above discrete map is known as the normal form associated to the bifurcation [9, Chapter 9]. In order to determine the c and d expression we can apply the normal form technique and the center manifold reduction [9, Chapter 9] with the following theorem. Theorem 1. If conditions (H.1) and (H.2) hold, then system (1) is topologically equivalent to the normal form (24) with parameters c ¼ 〈p0 ; Cðq0 ; q0 ; q0 Þ〉 and d ¼ 〈p0 ; Cðq0 ; q0 ; q1 Þ〉þ 〈p1 ; Cðq0 ; q0 ; q0 Þ〉; where C is a vector whose elements correspond to the Ci coefficients given by (17). Proof. The proof is postponed to Appendix A.

0

□

w211 þ 2w12 w21  w11 w22 ¼ 2

1 0

f ð0Þ2

α:

ð30Þ

Proof. The left hand side of the above expression can be simplified through successive uses of Eqs. (26)–(29), that is w211 þ 2w12 w21  w11 w22 ¼ w11 ðw11  w22 Þ þ2w12 w21 w11 β ¼ 2 0 þ 2w12 w21 f ð0Þ   w11 β α  w11 þ 0 ¼2 0 f ð0Þ f ð0Þ β ¼2 0 2 f ð0Þ α ¼  2 0 2: □ f ð0Þ

ð31Þ

Taking into account the relationships in the above lemmas, (25), (29), (30), and bifurcation condition (8), parameter c in the

R.L. Marichal, J.D. Piñeiro / Neurocomputing 162 (2015) 85–95

Proof. Given Lemma 4, we can deduce that w12 w21 o0. Then, taking into account the expression for parameter c (31)

neural network system is c ¼ 〈p0 ; Cðq0 ; q0 ; q0 Þ〉 ¼ ¼ ¼

0

βf ‴ð0Þ

0

2f ð0Þ6 w12 w321 f ‴ð0Þ

0

0

βf ‴ð0Þ

c¼ 

ð  2w12 w321 f ð0Þ3 þ α3 ðw22  w11 ÞÞ 0

2f ð0Þ6 w221

C 1 ðq0 ; q0 ; q1 Þ ¼ 

0

α2 f ‴ð0Þ

0

0

2βf ð0Þ5 w321 0

2f ð0Þ

6

w321

α3 f ‴ð0Þ 0

2f ð0Þ6 w321

C 2 ðq0 ; q0 ; q1 Þ ¼ 

0

;

d1 ¼ 〈p0 ; Cðq0 ; q0 ; q1 Þ〉 0

f ‴ð0Þ

2f ð0Þ3 w21

ðα3 βðw11 w22 Þ þ 2w12 w321 f ð0Þ3 ðβ  2ÞÞ

ðw12 α w21 ðβ  2ÞÞ:

The second right-hand term in the expression for the d coefficient can be simplified using (14) and condition (8), that is d2 ¼ 〈p1 ; Cðq0 ; q0 ; q0 Þ〉 0

f ‴ð0Þ

0

2f ð0Þ7 w12 w321

¼

f ‴ð0Þ

0

2f ð0Þ3 w21

ðα3 β2  f ð0Þ4 ðβ þ 2ÞÞ

ðw12 α w21 ðα 2ÞÞ:

Taking into account the above expressions and (8), the d parameter is given by d ¼ d1 þ d2 ¼ 

f ‴ð0Þ 0

f ð0Þ3 w21

ðw12 α þ 2w21 Þ:

ð33Þ

The dynamic behavior of the normal form (24) depends on the sign of c and D1 ¼  c  d and defines the bifurcation scenario under the generic perturbation of system parameters around the critical bifurcation values [9, Chapter 9]. Considering (8), the expression for D1 in the neural network system is D1 ¼  c  d ¼

f ‴ð0Þ 0

f ð0Þ3

0

SgnðD1 Þ ¼ Sgnðf ‴ð0Þf ð0Þðβ þ 2ÞÞ:

SgnðD1 Þ ¼  Sgnðβ þ 2Þ:

with (29) and (14) then

0

Theorem 3. Suppose that conditions (H.1) and (H.2) hold, then ð37Þ

Corollary 2. Suppose (H.1) and (H.2) hold, f ð0Þf ‴ð0Þ o 0, then

;

2βf ð0Þ3

f ‴ð0Þ

Taking the equation for D1 leads to the following theorem.

ð38Þ

ð2α þβÞ

ðβ  2Þf ‴ð0Þ

4f ð0Þ6 w12 w321

ð36Þ

0

ðw11 β2  f ð0Þw12 w21 ðβ þ 2ÞÞÞ

and taking into account bifurcation condition (8)

¼

□

SgnðcÞ ¼ Sgnðβðj w21 j  j w12 j ÞÞ: ð32Þ

α2 f ‴ð0Þ

¼

¼

ðj w21 j  j w12 j Þ;

Corollary 1. Suppose (H.1) and (H.2) hold, f ð0Þf ‴ð0Þ o 0; then 0

ð  w221 f ð0Þ3 þ f ð0Þ2 α2 Þ

To determine the d we calculate the first and second right-hand terms of (A.15) separately. As regards the first term, given (30), (14) and (8)

0

βf ‴ð0Þ

0

f ð0Þ6 w221   βf ‴ð0Þ w12 : ¼  0 3 1þ w21 f ð0Þ

¼

0

f ð0Þ3 j w21 j

the proof of the theorem follows directly.

0

ð  2βw221 f ð0Þ3 þ f ð0Þ2 α2 ðw22  w11 ÞÞ

¼

89

ð2 þ βÞ:

ð34Þ

Based on the expression for the c parameter, we can establish conditions for the sign using the following theorem. Theorem 2. Suppose that conditions (H.1) and (H.2) hold, then 0

SgnðcÞ ¼ Sgnðf ‴ð0Þf ð0Þβðj w21 j  j w12 j ÞÞ; where Sgn is the signum function.

ð35Þ

In the next bifurcation diagram explanation we assume, without loss of generality, that D1 o 0; otherwise, reverse time [9, Chapter 9]. This R2 bifurcation is associated with the normal form (24) and its dynamic behavior depends on the value of the c coefficient (31). The richest bifurcation diagram corresponds to a negative value. In this section we focus on this situation. The normal form associated with this bifurcation can be described by six regions and the boundaries between these regions are codimension-one bifurcations. In region 1 there is a trivial stable fixed point. It undergoes a non-degenerate Neimark–Sacker bifurcation on the half-line NSð2Þ given above, giving rise to a stable quasi-periodic cycle (region 2). Additionally, the unstable trivial fixed point becomes a saddle, while one unstable 2cycle arises when we cross the upper half-axis of the period-doubling bifurcation F ð1Þ from region 2 to region 3. In region 3, all 2-cycles are located inside the surrounding large quasi-periodic orbit that is still present. At the half-line, the nontrivial 2-cycles simultaneously yield two NSð2Þ bifurcations. These bifurcations lead to the appearance of two small unstable quasi-periodic orbits around the nontrivial 2cycle. The 2-cycles themselves become stable. Therefore, in region 4 we have three quasi-periodic orbits: one large and two small. Along the curve the small cycles disappear via a symmetric figure-eight homoclinic bifurcation [9, Chapter 9] and the trivial fixed point on the saddle has two simultaneous homoclinic orbits. Crossing this bifurcation from region 4 to a new region 5 results in the destruction of a small, quasi-periodic stable orbit and in the appearance of a large, quasi-periodic unstable orbit. In region 5, two large quasiperiodic orbits appear: the outer one is stable, while the inner one is unstable. These two large cycles collide and disappear along the curve due to the quasi-periodic orbit fold bifurcation. After the fold bifurcation, no limit cycles are left in the system. In region 6 we have the trivial saddle and a stable 2-cycle. The 2-cycle disappears due to a period-doubling bifurcation of the trivial fixed point, taking us back to region 1. Fig. 2 shows the bifurcation diagram with the different region mentioned above. In the original map (1), with respect to the R2 normal form, if we consider higher-order terms, then the homoclinic bifurcation between regions 4 and 5 and the quasi-periodic orbit fold bifurcation between regions 5 and 6 do not exist. The instant collisions of the saddle invariant manifolds and closed invariant curves are substituted by an infinite series of bifurcations in which homoclinic structures are involved. Such structures imply the existence of long-period cycles appearing and disappearing via fold bifurcations as one crosses the corresponding bifurcation set. In general, the complete bifurcation picture is unknown. This bifurcation structure is shown in neural network map (1). With respect to the analyses of neural networks the most interesting phenomenon is

90

R.L. Marichal, J.D. Piñeiro / Neurocomputing 162 (2015) 85–95

the presence of various quasi-periodic orbit (region 4) and the phenomenon of disappearance in the quasi-periodic orbit. Additionally, the bifurcation diagram corresponding to the case that the

0

δ

2

coefficient c is positive the dynamical behavior (region 4) with multiple quasi-periodic orbits disappear [9, Chapter 9]. In order to determine the necessary condition respect to the weight parameter of (1) necessary to appear different quasiperiodic orbit we plan the following theorem. Theorem 4. Given conditions (H.1), (H.2), f ð0Þf ‴ð0Þ o 0, and the following weighted conditions:

(2)

NS

(1)

F+

0

f ð0Þðw22  w11 Þ 4  4

Reg. 3 Reg. 4

P

ð39Þ

and 0

f ð0Þðw22  w11 Þðj w21 tj  j w12 tj tÞ o0

Reg. 5 K

Reg. 2

ð40Þ

then multiple quasi-periodic orbits appear. Proof. First, it is necessary to prove that c and D1 coefficients are negatives to ensure the existence of various quasi-periodic orbits. Taking into account the result of Corollary 1 and Lemma 5 leads to

R

2

(1)

δ1

NS

0

SgnðcÞ ¼ Sgnðf ð0Þðw22 w11 Þðj w21 j  j w12 j ÞÞ:

Reg. 1

Based on the theorem condition (40) and the above expression, the value of coefficient c is negative. Considering Corollary 2 and Lemma 5, the signum of D1 is F(1)

0

SgnðD1 Þ ¼  Sgnðf ð0Þðw22  w11 Þ þ 4Þ;

−

with the theorem condition (39) the value of coefficient D1 is negative. □

Fig. 2. Regions of stability and bifurcation diagram of the R2 bifurcation point with c o 0 and D1 o 0.

2

NS(2) 2−cycle Neirmark−Sacker Bif. R ,R

GPD

1.9

2

(1) FR

1.8

1

F(1) Fixed Point Period−Doubling Bif. R

2

2

(24) CP

T

1.7

ð41Þ

24−cycle Fold Bif.

(24) TCP

1.6 |W|

3

1.5

4

2

1

1.4 1.3 1.2

CP NS(2) R ,R

1.1

R2/R1

2 1

1 −3

−2.9

−2.8

−2.7

−2.6

−2.5

−2.4

−2.3

−2.2

−2.1

−2

w11 +w22

1.5069

1.5068 (24)

T2,CP

(24)

T1,CP

1.5068 |W|

4

3’’

3

1.5067

1.5067

1.5066 −2.6036

−2.6036

−2.6035

−2.6035 w11 +w22

−2.6034

−2.6034

−2.6033

Fig. 3. Bifurcation diagram with some dynamic sample points. The weights are w11 ¼  2:732, w12 ¼  3:000, w21 ¼ 1:000, w22 ¼ 0:732 at the R2 bifurcation point, the bifurcation parameters are c ¼  3:464 o 0 and D1 ¼  3:732 o 0. (a) Bifurcation diagram and (b) 24-iteration fold bifurcation lines.

R.L. Marichal, J.D. Piñeiro / Neurocomputing 162 (2015) 85–95

91

1 0.8 0.6 0.4 0.2 Y

0 −0.2 −0.4 −0.6 −0.8 −1 −1

−0.8

−0.6

−0.4

−0.2

−0.8

−0.6

−0.4

−0.2

0 X

0.2

0.4

0.6

0.8

1

0

0.2

0.4

0.6

0.8

1

1 0.8 0.6 0.4 0.2 Y

0 −0.2 −0.4 −0.6 −0.8 −1 −1

X Fig. 4. Fig. 3(a) point dynamic behaviors; numbers 1 (a) and 2 (b). The black cross, black square and white squares are a saddle fixed point, unstable fixed point and unstable 2-cycles, respectively. Gray and black curves are unstable and stable manifolds associated with saddle fixed points, respectively.

Remark 1. The practical interpretation of Theorem 4 can be more 0 easily visualized assuming f ð0Þ o 0, since in that case when the weights of one neuron are more dominant than those of another, that is, when its associated weights are greater, several quasi0 periodic orbits appear. If f ð0Þ 4 0, there would have to be a cross effect involving the weights, meaning that if the feedback weight of one is smaller than the other's, the latter's interaction weight with the former is smaller. Remark 2. This theorem also allows certain control over the existence of multiple quasi-periodic orbits; that is, assuming a condition regarding the antisymmetric weights, the feedback weights could be varied to have said quasi-periodic orbits 0 appear/disappear. For example, assuming f ð0Þ 4 0, without loss of generality, we can select w22 while satisfying the critical situation D1 ¼ 0, considering (41) and the relationship between w11 and w22 in the bifurcation R2 σ n1 ¼  1, that is, wðcÞ 22 ¼ 

3 0 f ð0Þ

To ensure the sign of parameter c (c o 0), the antisymmetric weights must satisfy j w21 j 4 j w12 j . This way if w22 is slightly

greater/smaller than wðcÞ 22 , the quasi-periodic orbits will be dis0 appear/appear. If f ð0Þ o 0, the transition will be reversed, appear/ disappear.

5. Bifurcation diagram and dynamic behavior simulations In order to determine the bifurcation diagram for system (1), we consider as varying parameters the matrix weight determinant and the sum of the diagonal elements of the weight matrix. Fig. 3(a) shows the bifurcation diagram using the [23] software toolkit, which implements the numerical continuation technique [9, Chapter 10] to determine the diagram bifurcation. In this figure there are three additional codimension-two bifurcations, known as generalized period doubling (GPD), 1:1 resonance (R1) and cusp (CP) bifurcations [9, Chapter 9]. These bifurcations are produced, respectively, when: the period-doubling bifurcation normal form coefficient is zero; the eigenvalues associated with the fixed point are one; and the fold bifurcation normal form is zero. The GPD bifurcation is composed of three codimension-one bifurcations, two fixed point period-doubling bifurcations and a 2cycle fold bifurcation. In the CP bifurcation, the associated codimension-one are two fixed point fold bifurcations. Finally, the R1 bifurcation consists of two fold bifurcations, one Neimark–Sacker

92

R.L. Marichal, J.D. Piñeiro / Neurocomputing 162 (2015) 85–95

1 0.8 0.6 0.4 0.2 Y

0 −0.2 −0.4 −0.6 −0.8 −1 −1

−0.8

−0.6

−0.4

−0.2

0 X

0.2

0.4

0.6

0.8

1

−0.8

−0.6

−0.4

−0.2

0 X

0.2

0.4

0.6

0.8

1

1 0.8 0.6 0.4 0.2 Y

0 −0.2 −0.4 −0.6 −0.8 −1 −1

Fig. 5. Fig. 3(a) point dynamic behaviors; numbers 3 (a) and 4 (b). The black crosses and white triangles are saddle fixed points and stable 2-cycles, respectively. The gray and black curves are unstable and stable manifolds associated with saddle fixed points, respectively. 0.6

0.4

0.2

0 Y −0.2

−0.4

−0.6

−0.8 −1

−0.8

−0.6

−0.4

−0.2

0 X

0.2

0.4

0.6

0.8

1

Fig. 6. Fig. 3(b) point dynamic behaviors; number 300 . The black cross, white triangles, gray crosses and gray triangles are a saddle fixed point, stable 2-cycle, saddle 24-cycles and stable 24-cycles, respectively. Gray and black curves are unstable and stable manifolds associated with saddle fixed points, respectively.

bifurcation and one global bifurcation known as a saddle homoclinic bifurcation [9, Chapter 4], the main characteristic of which is the destruction of quasi-periodic orbits.

The bifurcation diagram, Fig. 3, shows these codimension-two bifurcations. One R2 bifurcation corresponds to the one-iteration map, one GPD and two R1 bifurcations are associated with the

R.L. Marichal, J.D. Piñeiro / Neurocomputing 162 (2015) 85–95

1

0.8

0.8

0.6

0.6

0.4

0.4

0.2

0.2 X

0

X

0

−0.2

−0.2

−0.4

−0.4

−0.6

−0.6

−0.8 −1

93

0

10

20

30

40 k

50

60

70

−0.8 0

80

10

20

30

40 k

50

60

70

80

10

20

30

40 k

50

60

70

80

−0.6

−0.4

−0.2

0.5

0.6

0.4 0.3

0.4

0.2

0.2 Y

0.1 Y

0

−0.2

−0.2

−0.3

−0.4

−0.4 −0.5 0

−0.6 −0.8

0 −0.1

0

10

20

30

40 k

50

60

70

80 0.5 0.4

1

0.3

0.8

0.2

0.6

0.1

0.4

Y

Y

0

0.2

−0.1

0

−0.2

−0.2

−0.3

−0.4

−0.4

−0.6

−0.5 −0.8

−0.8 −1 −1

−0.8

−0.6

−0.4

−0.2

0 X

0.2

0.4

0.6

0.8

1

Fig. 7. Trajectory of big quasi-periodic orbit corresponding to Fig. 3(a) dynamic behavior; X time evolution (a) Y time evolution (b) and X  Y trajectory (c). The initial state point is Xð0Þ ¼ 0:1000 and Yð0Þ ¼ 0:1000.

two-iteration map and one CP bifurcation point is associated with the 12-iteration map. The codimension-one bifurcation corresponds to different kinds of codimension-two bifurcations. For example, the NSð2Þ R2 ;R1 corresponds to the one-iteration map 2-cycle Neimark–Sacker and 2-iteration map fixed point Neimark-Sacker of R2 and R1 formal norms, respectively. Note in the bifurcation diagram that the fold bifurcation associated with the CP point is actually a double bifurcation, as shown in Fig. 3(b). The dynamic transition point parameters are represented in Fig. 3. The dynamics of the transition are shown in Figs. 4–6. The first point of the dynamic transition, represented by point 1 in Fig. 3(a), is shown in Fig. 4(a). In this case, there is a quasiperiodic orbit and one unstable fixed point, as in the above dynamic transition. This dynamic corresponds to region 2 associated with the R2 bifurcation mentioned above. Crossing F ð1Þ R2 , point 2 in Fig. 3(a), leads to region 3 of R2 where the stability of the trivial fixed point changes and one unstable 2-cycle appears (see Fig. 4(b)). Additionally, at point 3 in Fig. 3(a), the 2-cycle stability changes from stable to unstable via the Neimark–Sacker bifurcation NSRð2Þ2 ;R1 , while two small quasi-periodic orbits appear (see

0 X

0.2

0.4

0.6

0.8

Fig. 8. Trajectory of two small quasi-periodic orbits corresponding to Fig. 3(a) dynamic behavior; X time evolution (a) Y time evolution (b) and X  Y trajectory (c). The initial state point is Xð0Þ ¼  0:3928 and Yð0Þ ¼ 0:2447.

Fig. 5(a)). This behavior matches that of region 4 for bifurcation R2. Following the dynamic transition, at point 300 in Fig. 3(b), two 24cycles appear, saddle and stable, via 24-iteration fold bifurcation ð24Þ T 2;CP (see Fig. 5(b)). There is also a figure-eight saddle fixed point unstable manifold. Finally, at point 4 of Fig. 3(a), the 24-cycle disappears via the other 24-iteration fold bifurcation T ð24Þ 1;CP (see Fig. 6). This situation corresponds to region 6 of bifurcation R2. Note the absence of the homoclinic curve and the quasi-periodic fold bifurcation associated with the R2 bifurcation normal form. Also, region 5 is replaced by a 1:24 Arnold tongue. In region 8, a homoclinic point associated with the unstable manifold saddle fixed point appears because the stable manifold intersects at two points, known as homoclinic points [9, Chapter 6]. Finally, we study the influence of multiple quasi-periodic orbits on the neural network. In order to make this analysis, the temporal trajectories depict that the quasi-periodic orbits are calculated (see Figs. 7 and 8). Fig. 7 has been obtained by considering an initial configuration of the output neurons, this trajectory describes a big quasi-periodic orbit shown in Fig. 5(a). Furthermore, in Fig. 8 is considerate another initial configuration appears two small quasiperiodic orbits correspond to Fig. 5(a). In both cases, the small and big quasi-periodic orbits, the neurons are synchronized. In the

94

R.L. Marichal, J.D. Piñeiro / Neurocomputing 162 (2015) 85–95

first, the periodicity of synchronization is irrational, otherwise, in the second, is rational with value 2, this is motivated by these small quasi-periodic orbits that are related with the 2-cycle Neimark–Sacker bifurcation (see Section 4). In general, with respect to the applications related to neural network synchronization, the neurons have got an interesting behavior, that is, the neurons are synchronized with different periodicities (irrational or rational) that depend on the initial output values.

6. Conclusion In this paper we have analyzed a discrete recurrent neural network model consisting of one oscillator with two neurons. We discussed fixed-point stability with a characteristic equation. In addition, the condition necessary for producing the 1:2 resonance R2 bifurcation was stated. We showed the relationships with another codimension-two bifurcation in two-iteration and highiteration maps. We established the disappearance mechanism for quasi-periodic orbits associated with the Neimark–Sacker bifurcation. We also studied the differences between the R2 normal form and the neural-network map, and we gave simple conditions for controlling the existence of the multiple quasi-periodic orbits that appear in the aforementioned analysis.

Acknowledgments We are grateful to the anonymous referee for his/her constructive comments.

Appendix A. Proof of Theorem 1

In addition, taking into account condition (H.1) leads to B  0 by (16), and considering the η2-terms of (A.2) yields the following linear system: ðA  IÞh20 ¼ 0

ðA:6Þ

ðA  IÞh11 ¼  h20

ðA:7Þ

ðA  IÞh02 ¼  2h11 þ h20 ;

ðA:8Þ

where h20, h11 and h02 are the Taylor expansions (A.3) associated with η21 , η1 η2 and η22 , respectively. The matrix ðA  IÞ is nonsingular because A only has eigenvalues of minus one on the unit circle. We can thus can deduce that h20, h11 and h02 are zero. The cubic terms of (A.2) yield the equations ðA þ IÞh30 ¼ cq1  Cðq0 ; q0 ; q0 Þ

ðA:9Þ

ðA þ IÞh21 ¼ dq1 þh30  Cðq0 ; q0 ; q1 Þ

ðA:10Þ

ðA þ IÞh12 ¼ 2h21  h30 Cðq0 ; q1 ; q1 Þ

ðA:11Þ

ðA þ IÞh03 ¼ h30  Cðq1 ; q1 ; q1 Þ;

ðA:12Þ

where h30, h21, h12 and h03 are the Taylor expansions (A.3) associated with η31 , η21 η2 , η1 η22 and η32 , respectively. The above equations involve a singular matrix ðAþ IÞ because A only has eigenvalues of minus one on the unit circle. In this case the conventional methods for obtaining the parameters are not applicable. In fact, to determine c coefficient we must assume that the left side of the above equation is non-singular and the right satisfies Fredholm's solvability condition [9, Chapter 9], that is, 〈p; Req 〉 ¼ 0;

ðA:13Þ

where Req and p are the right-hand (A.9)–(A.12) and the generalized eigenvectors of the transpose Jacobian matrix, respectively. Taking into account equations (20), (A.4), (A.5), (A.9) and (A.13), we can determine the c coefficient: c ¼ 〈p0 ; Cðq0 ; q0 ; q0 Þ〉:

To determine the topological equivalence, let us consider the relationship between state variables x and η in the original system (1) and the normal form (24), respectively, by means of the center manifold. That is, locally the center manifold in the critical situation, when the Jacobian eigenvalues in the original system are minus one, can be parameterized as

The equation for determining d coefficient (A.10) includes the h30 vector. Considering the scalar product of (A.9) with p1 and the eigenvector equation (19) yields

x ¼ HðηÞ

Taking the scalar product of eigenvector p0 with (A.10) and considering (A.13), (20) and (A.14) we have

ðA:1Þ

where η is a vector whose elements correspond to the ηi normal form variables states given by (24). Additionally, the normal form can be redefined as

d ¼ 〈p0 ; Cðq0 ; q0 ; q1 Þ〉 þ 〈p1 ; Cðq0 ; q0 ; q0 Þ〉:

ðA:14Þ

ðA:15Þ

References

ηðk þ 1Þ ¼ GðηðkÞÞ; where G is the general function that defines the normal form map. The topology equivalence equation is HðGðηÞÞ ¼ FðHðηÞÞ

ðA:2Þ

where F is the original map (1). Given the center manifold Taylor expansion HðηÞ ¼ h1 η1 þ h2 η2 þ

〈p0 ; h30 〉 ¼  〈p1 ; Cðq0 ; q0 ; q0 Þ〉:

X 1 h ηi ηj ; i!j! ij 1 2 iþjZ2

ðA:3Þ

collecting the η-terms in (A.2) and taking into account the generalized eigenvector equations (18) and (19), the H terms associated with η1 and η2 are h1 ¼ q0

ðA:4Þ

h2 ¼ q1 ;

ðA:5Þ

where h1 and h2 are the Taylor expansions associated with η1 and η2, respectively.

[1] D.R. Hush, B.G. Horne, Progress in supervised neural networks, IEEE Signal Process. Mag. 10 (1) (1993) 8–39. [2] J. Hale, H. Koçak, Dynamics and Bifurcations. Texts in Applied Mathematics, Springer-Verlag, 1991 (ISBN 9780387971414). [3] R. Robinson, C. Robinson, Dynamical Systems: Stability, Symbolic Dynamics, and Chaos. Studies in Advanced Mathematics, CRC Press, 1999 (ISBN 9780849384950). [4] C.M. Marcus, R.M. Westervelt, Dynamics of iterated-map neural networks, Phys. Rev. A 40 (1) (1989) 501–504. [5] J.D. Cao, On stability of delayed cellular neural networks, Phys. Lett. A 261 (5–6) (1999) 303–308. [6] X. Wang, Discrete-time dynamics of coupled quasi-periodic and chaotic neural network oscillators, in: Proceedings of the International Joint Conference on Neural Networks, IJCNN'92, vol. 3, IEEE, 1992, pp. 517–522. [7] F. Pasemann, Complex dynamics and the structure of small neural networks, Netw.-Comput. Neural Syst. 13 (2) (2002) 195–216. [8] P. Tino, B.G. Horne, C.L. Giles, Attractive periodic sets in discrete-time recurrent networks (with emphasis on fixed-point stability and bifurcations in two-neuron networks), Neural Comput. 13 (6) (2001) 1379–1414. [9] Y. Kuznetsov, Elements of Applied Bifurcation Theory. Applied Mathematical Sciences, Springer, 2010 (ISBN 9781441919519). [10] R. Marichal, J. Piñeiro, E. González, J. Torres, Stability of quasi-periodic orbits in recurrent neural networks, Neural Process. Lett. 31 (3) (2010) 1–13.

R.L. Marichal, J.D. Piñeiro / Neurocomputing 162 (2015) 85–95

[11] W. He, J. Cao, Stability and bifurcation of a class of discrete-time neural networks, Appl. Math. Model. 31 (10) (2007) 2111–2122. [12] Z.H. Yuan, D.W. Hu, L.H. Huang, Stability and bifurcation analysis on a discretetime neural network, J. Comput. Appl. Math. 177 (1) (2005) 89–100. [13] E. Kaslik, S. Balint, Bifurcation analysis for a two-dimensional delayed discretetime Hopfield neural network, Chaos Solitons Fractals 34 (4) (2007) 1245–1253. [14] E. Kaslik, S. Balint, Bifurcation analysis for a discrete-time Hopfield neural network of two neurons with two delays and self-connections, Chaos Solitons Fractals 39 (1) (2009) 83–91. [15] C. Zhang, B. Zheng, Stability and bifurcation of a two-dimension discrete neural network model with multi-delays, Chaos Solitons Fractals 31 (5) (2007) 1232–1242. [16] S. Guo, X. Tang, L. Huang, Bifurcation analysis in a discrete-time singledirectional network with delays, Neurocomputing 71 (7–9) (2008) 1422–1435. [17] Z.H. Yuan, D.W. Hu, L.H. Huang, Stability and bifurcation analysis on a discretetime system of two neurons, Appl. Math. Lett. 17 (11) (2004) 1239–1245. [18] F. Pasemann, M. Hild, K. Zahedi, So(2)-networks as neural oscillators, in: Proceedings of the 7th International Conference on Computational Methods in Neural Modeling, IWANN'03, vol. 1, Springer-Verlag, Berlin, Heidelberg, 2003, pp. 144–151. ISBN 3-540-40210-1 〈http://dl.acm.org/citation.cfm?id=1762716. 1762737〉. [19] J.J. Wei, W.H. Jiang, Stability and bifurcation analysis in a neural network model with delays, Dyn. Contin. Discret. Impuls. Syst.—Ser. A—Math. Anal. 13 (2) (2006) 177–192. [20] R. Marichal, E. González, Period-doubling bifurcation in discrete Hopfield neural network, in: Proceedings of the 2011 International Conference on Multimedia Technology, ICMT'2011, vol. 1, IEEE, 2011, pp. 5966–5969. [21] B. Gao, W. Zhang, Equilibria and their bifurcations in a recurrent neural network involving iterates of a transcendental function, IEEE Trans. Neural Netw. 19 (5) (2008) 782–794. http://dx.doi.org/10.1109/TNN.2007.912321. [22] S.E. Folias, G.B. Ermentrout, Spatially localized synchronous oscillations in synaptically coupled neuronal networks: conductance-based models and discrete maps, SIAM J. Appl. Dyn. Syst. 9 (3) (2010) 1019–1060. http://dx.doi. org/10.1137/090780092. [23] A. Dhooge, W. Govaerts, Y. Kuznetsov, W. Mestrom, A. Riet, B. Sautois, Cl matcont: Continuation Software in Matlab 〈http://www.sourceforge.net/pro jects/matcont〉; 2011.

95

Roberto Marichal received his M.S. degree in Applied Physics in 1996 and his Ph.D. degree in Computer Science in 2003 from the University of La Laguna, Canary Islands, Spain. From 1996 to 2000, he was a Research Student in the Department of Applied Physics, Electronics and Systems at the same university. He was a Visitor at the University of Technical University of Denmark, Denmark, and University of Sevilla, Spain, on 2000 and 2004, respectively. Currently, he is an Assistant Professor in the Department of Computer and Systems Sciences of La Laguna University. His current research interests include process neural network, nonlinear control and bifurcation theory. Dr. Marichal received a National Award for Academic Excellence. He currently serves as a Reviewer in different journals.

Jose D. Piñeiro received his M.S. degree in Applied Physics in 1990 and his Ph.D. degree in Computer Science in 1996 from the University of La Laguna, Canary Islands, Spain. From 1990 to 1999 he was an Assistant Professor in the Department of Applied Physics, Electronics and Systems at the same university. Currently, he is an Associate Professor with the Department of Computer and Systems Sciences of La Laguna University. His current research interests include machine learning, neural networks and its applications to robotics and computer vision problems.