- Email: [email protected]
A new class of neural networks and its applications
Communicated by Prof. Duan Shukai
Accepted Manuscript
A New Class of Neural Networks and its Applications Kais Bouallegue PII: DOI: Reference:
S0925-2312(17)30487-3 10.1016/j.neucom.2017.03.006 NEUCOM 18226
To appear in:
Neurocomputing
Received date: Revised date: Accepted date:
27 April 2016 19 July 2016 2 March 2017
Please cite this article as: Kais Bouallegue, A New Class of Neural Networks and its Applications, Neurocomputing (2017), doi: 10.1016/j.neucom.2017.03.006
This is a PDF file of an unedited manuscript that has been accepted for publication. As a service to our customers we are providing this early version of the manuscript. The manuscript will undergo copyediting, typesetting, and review of the resulting proof before it is published in its final form. Please note that during the production process errors may be discovered which could affect the content, and all legal disclaimers that apply to the journal pertain.
ACCEPTED MANUSCRIPT
A New Class of Neural Networks and its Applications Kais Bouallegue1
CR IP T
Department of Electrical Engineering. Higher Institute of Applied Sciences and Technology of Sousse, Tunisia. kais [email protected].
Abstract
M
AN US
We present a new class of neural network using a variable structure model of neuron (VSMN). From this structure, we generate four models of neurons. For each model, we study different behaviors such as stable or equilibrium , degraded, hole, alternated, oscillator, harmonic, fractal, and chaos behaviors. Then we design different topologies and architectures of neural networks. These architectures are different from the classical ones; each layer of network contains different models of neurons, neurons can take four models by configuration of VSMN. We also present a numerical study describing the behavior of some models of neuron. We illustrate some results to show the efficiency of this new class of neural networks. We show that these neurons put tracks on their stimulators such as: signal track and half bounded region track with two high and low directions. Two applications in chaos and robotic are also given.
ED
Keywords: Variable structure model of neuron, neural networks, chaos, harmonic, oscillator, fractal, signature of neuron.
PT
1. Introduction
CE
After decades, many artificial neural networks have been proposed to develop tools that may lead to a better understanding of the behavior of brain. Their various versions have also been intensively and extensively studied in many aspects and successfully applied to many fields such as identifying pattern, recognizing voice, controlling system, processing signal systems, treating static image, and solving nonlinear algebraic system, etc. ,[21],[17], [12], [15],[7],[20],[3],[8], [1]. Hopfield’s neural network is a model of artificial neural network which appears to be a powerful tool for modeling biological neural networks [9]. It is known that the human brain has 1010 neurons, and each neuron has 103 to 104 connections with other neurons. Each neuron controls and drives the dynamical behavior of organs. Some neurons work all the time, others work only when
AC
5
10
✩ Kais
[email protected]
Preprint submitted to Elsevier
March 21, 2017
ACCEPTED MANUSCRIPT
25
30
CR IP T
20
AN US
15
their sensors are stimulated. In this work, we notice that when the neuron is in a “hole behavior” some information of its stimulator are lost. We think that this model of neuron, in this case, has great similarity with Parkinson disease. We also notice that when the sign of threshold position of neuron is switching between positive and negative, the neuron adopts an oscillating behavior. Probably, this is the model of Alzheimer disease. The results reached in this work shed some lights on neuroscience. Section 2, provides a mathematical approach of our models. Section 3 is devoted to a model of neuron with a variable structure governed by a specific function. Four models generated by this structure are studied. For each model, we find the maximum point of the associated function, and then we illustrate different behaviors of the models. In section 4, we present a multi behaviors of VSMN such as oscillator, fractal, harmonic and chaos behaviors some examples of each behavior are illustrated. Section 5, provides representations of VSMN which will be used in the topology of neural networks. In section 6, we design two examples of architectures of neural network using VSMN. In section 7, we apply our models of VSMN in chaotic attractors. Two examples of path of a robot are given (see Section 8). We close the paper by a conclusion. 2. A Mathematical Model of a VSMN
M
PT
ED
35
In this section, we elaborate a model of neurons with variable structure. The structure depends on four variables (n, p, q, k), where n, q are related to the behavior of the model, p is related to the model’s threshold position and k represents the neuron’s polarity. A Variable Structure Model of Neurons (VSMN, for short) is described by the following system of differential equations: + (u + p)vf (βv)f (λ(u + p)) u˙ = −(u+p) τ (1) v˙ = −αv + k(u + q)n αf 2 (λ(u + p)) t2
CE
40
with f (t) = e− 2 and k = ±1, τ is a time constant p, q are real numbers, λ, α, β are positive real numbers. Take g(u) := v˙ + αv = k(u + q)n αf 2 (λ(u + p)). Then to study the previous model, it suffices to focus on the following function: g(x) = k(x + q)n e−
(x+p)2 2
.
AC
Clearly, it is enough to consider the case k = +1. If n 6= 0 and n 6= 1, then the critical points of the function are given by the following equation: g 0 (x) = ((x + q)n−1 (n − (x + q)(x + p))e−
(x+p)2 2
= 0.
√ p+q+ ∆ −2
√
∆ The roots of this equation are x1 = −q, x2 = and x3 = p+q− −2 with ∆ = (p − q)2 + 4n. The values of the function at these critical points are
45
g(x1 ) = 0, g(x2 ) = (x2 + q)n e−
(x2 +p)2 2
and g(x3 ) = (x3 + q)n e− 2
(x3 +p)2 2
.
ACCEPTED MANUSCRIPT
In order to study the behavior of a VSMN influenced by a stimulator, we investigate four cases.
50
0
0.8
-0.2
0.6
-0.4
0.4
-0.6
0.2
-0.8
-4
-2
0
2
M
0
AN US
1
CR IP T
1. Case n = 0. −(x+p)2 The function becomes g(x) = e 2 ; the unique critical point is x = −p and g(−p) = 1 is the maximum value of the function.
4
-1 -4
(a) n = 0, k = +1, p = 0
-2
0
ED
Figure 1: Stable behavior of a VSMN for n = 0
AC
CE
PT
2. Case n = 1 −(x+p)2 If p = q, then the function becomes g(x) = (x + p)e 2 , the critical points are x1 = −p + 1 and x2 = −p − 1. The maximum value of the −1 −1 function is g(x1 ) = e 2 and the minimum value is g(x2 ) = −e 2 .
3
2
(b) n = 0, k = −1, p = 0
4
ACCEPTED MANUSCRIPT
0.6
1
0.4 0.5
0 0 -0.2 -0.4
-0.5
-1 -4
-2
0
2
(a) k = +1, p = q = 0
AN US
-0.6
CR IP T
0.2
4
-0.8 -4
-2
0
(b) k = −1, p = q = 0
Figure 2: Stable behavior of a VSMN for n = 1
3. Case n ≥ 2 and p = q In this case, the critical points are
M
55
x1 = −p, x2 = −p −
√
n and x3 = −p +
√
n.
ED
The values of g at these critical points are as follows: √ √ n n g(x1 ) = 0, g(x2 ) = (− n)n e− 2 and g(x3 ) = ( n)n e− 2 .
CE
PT
If the function g has two critical points with the same absolute maximal value, we say that the VSMN model has a stable behavior. - When n is even, the function has two maximum values √ n g(x2 ) = g(x3 ) = ( n)n e− 2
AC
Figure 3 displays the form of the model. This form contains two lobes with the same sizes, one on the left other on the right.
4
2
4
ACCEPTED MANUSCRIPT
0.8
CR IP T
0.6
0.4
0
-4
-2
AN US
0.2
0
2
4
Figure 3: Stable behavior of the model (for n = 2)
Figure 4 displays the form of the model. This form contains two lobes with alternate directions (up and down).
ED
60
M
- When n is odd, the function has a minimal value and a maximal maximum value with the same absolute value √ n g(x2 ) = ( n)n e− 2 = −g(x3 ).
1.5
PT
1
AC
CE
0.5
0
-0.5 -1 -1.5 -4
-2
0
2
4
Figure 4: Stable behavior of the model(for n = 3)
5
ACCEPTED MANUSCRIPT
4. Case n ≥ 2 and p = −q p p Here, the critical points are x1 = p, x2 = − p2 + n and x3 = p2 + n. The values of g at these points are
and g(x3 ) = (
p
p
p2 + n − p)n e−
p2 + n − p)n e−
√
(
√
(−
p2 +n+p)2 2
CR IP T
g(x1 ) = 0, g(x2 ) = (−
p2 +n+p)2 2
.
Notice that, if |p| is a large positive real number (approaching +∞), then g(x2 ) ≈ (−|p| − p)n e− and
(|p|+p)2 2
.
AN US
g(x3 ) ≈ (|p| − p)n e−
(−|p|+p)2 2
Then we consider two subcases,
70
• If p is a large positive real number, then g(x3 ) is a small positive number and |g(x2 )| is large. In this subcase, we say that the VSMN has a degrade behavior at the right. For a large value of p, Figure 5 displays only one point and the second point tends to zero.
ED
200
M
65
150
AC
CE
PT
100
50
0 -40
-20
0
20
40
Figure 5: Only one point appear (for n = 2)
we give implementation with small values of p and q to illustrate the two critical points.
6
ACCEPTED MANUSCRIPT
2
1
-1 1 -2 0.5
-4 -5
-5 0 5 (a) n = 2, p = −0.4, q = 0.4, k = +1
0 5 (b) n = 3, p = −0.4, q = 0.4, , k = +1
AN US
0
-3
CR IP T
0
1.5
Figure 6: Stable behavior of the model(for n = 3)
75
M
• If p is negative and |p| is large, then |g(x2 )| is small and g(x3 ) is a large positive number. In this subcase, we say that the VSMN has a degrade behavior at the left.
CE
0.5
AC
0
3 2
PT
1.5
1
4
ED
2
-5 0 5 (a) n = 3, p = 0.4, q = −0.4, , k = +1
1 0 -1 -5
0 5 (b) n = 3, p = 0.4, q = −0.4, k = +1
Figure 7: Stable behavior of the model(for n = 3)
In this case we say that the corresponding VSMN has a left peak (resp., right peak) when p is large positive (resp., negative). This will affect the information (or the signature) caused by the neuron on the structure of
7
ACCEPTED MANUSCRIPT
the stimulator; when p is large, the neuron makes a hole in the stimulator’s data (see Section 7 ).
80
3. Multi Models of Neurons Generated by VSMN
85
3.1. First Model of Neuron (n = 0)
CR IP T
In this section, we generate four models from the variable structure model of neuron. The results are similar to the previous section.
+ (u + p)vf (βv)f (λ(u + p)) u˙ = −(u+p) τ v˙ = −αv + kαf 2 (λ(u + p))
(a) k = −1, n = 0
AN US
This model is illustrated graphically as follows.
(2)
(b) k = +1, n = 0
M
Figure 8: First model of VSMN for n = 0
3.2. Second model of neuron (n = 1)
+ (u + p)vf (βv)f (λ(u + p)) u˙ = −(u+p) τ v˙ = −αv + k(u + p)αf 2 (λ(u + p))
AC
CE
PT
ED
8
(3)
(a) k = −1, n = 1
AN US
CR IP T
ACCEPTED MANUSCRIPT
(b) k = +1, n = 1
M
Figure 9: Second model n = 1 with alternated behavior
ED
PT
90
3.3. Third model of neurons (n = 2) 3.3.1. Stable behavior This model is given by the following system: u˙ = −(u+p) + (u + p)vf (βv)f (λ(u + p)) τ v˙ = −αv + k(u + p)2 αf 2 (λ(u + p))
AC
CE
Figure 10 shows the implementation of this model .
(a) Positive of polarity k = −1
(b) Negative of polarity k = +1
Figure 10: Third model n = 2 with stable behavior
3.3.2. Degraded behavior This behavior occurs when q = −p. 9
(4)
ACCEPTED MANUSCRIPT
u˙ = −(u+p) + (u + p)vf (βv)f (λ(u + p)) τ v˙ = −αv + k(u − p)2 αf 2 (λ(u + p))
(5)
(b)
AN US
(a)
CR IP T
Figure 11 shows different sizes of small lobs.
(c)
M
Figure 11: Different forms of degraded behavior
ED
(a) The small one is on the left
(b) The small one is on the right
3.3.3. Hole behavior of neuron in low and high directions If p is a large positive real number, then the VSMN has a hole behavior at the right. The information of the stimulator of the neuron will be lost. Figure 13 shows two results of implementation (see section .
AC
CE
95
PT
Figure 12: Degraded behavior of neuron on the left/right direction
10
CR IP T
ACCEPTED MANUSCRIPT
(b) Hole behavior of neuron(down direction) (a) Hole behavior of neuron(up direction)
100
AN US
Figure 13: Hole behavior of neuron on the up/down direction
3.4. Fourth model of neuron (n = 3) In this case the model of neuron is described as follows: u˙ = −(u+p) + (u + p)vf (βv)f (λ(u + p)) τ v˙ = −αv − (u + p)3 αf 2 (λ(u + p))
(6)
M
Figure 14 displays results of implementation.
ED
(b)
(a)
PT
Figure 14: Model of neuron n = 3
4. Multi behaviors of a VSMN
CE
4.1. Oscillator behavior of a VSMN Oscillations and chaos are ubiquitous phenomena that are encountered in many different areas of physics [14]. And oscillatory phenomena have attracted much interest in biology, chemistry, and neuroscience. It seems that when there is a chaos there are a hidden oscillators and when there is an oscillator there are hidden chaos. It turns out that the relationship between chaos and oscillations behavior is the common factor of parameters depending of the mathematical model. In this section, we will show with VSMN can genarate different types of oscillations. The input is a small sinusoidal given by
AC
105
110
xi+2 = 2cos(2π 11
κ )xi+1 − xi N
ACCEPTED MANUSCRIPT
AN US
CR IP T
. q = −m × 0.01 and p = 0 The equation of the model with input signal takes this form: u˙ = −(u+p) + v(u + p)f (βv)f (λ(u + p)) + xi+2 τ (7) v˙ = −αv − m(u − q)2 αf 2 (λ(u + p))
(a)
Figure 15: Different forms of Oscillator behavior of neuron
The inputs are given by
M
xi+2 = 2cos(2π and
yi+2 = 2cos(2π
κ )yi+1 − yi N
ED
.
The equation of the model with inputs takes this form: κ xi+2 = 2cos(2π N )xi+1 − xi κ y = 2cos(2π )y i+2 i+1 − yi N u = αx 2 v = αy2 u˙ = −(u+p) + v(u + p)f (βv)f (λ(u + p)) + x2 τ v˙ = −αv − m(u − q)2 αf 2 (λ(u + p))
CE
PT
115
κ )xi+1 − xi N
(8)
AC
We take these values q = −m × 0.034 and p = 0, result of simulation in figure 16 It shows symmetric oscillator without contact between points.
12
Figure 16: Oscillator without contact
CR IP T
ACCEPTED MANUSCRIPT
AN US
The equation of the model with input signal takes this form: u˙ = −(u+p) + v(u + p)f (βv)f (λ(u + p)) + xi+2 τ v˙ = −αv − m(u − q)2 αf 2 (λ(u + p))
(9)
The input is a small sinusoidal given by xi+2 = 2cos(2π
. q = −m0.01 and p = 0 with value of m is switching between ±1 Figure 17 shows symmetric oscillator with one contact.
CE
PT
ED
M
120
κ )xi+1 − xi N
Figure 17: Oscillator with contact
AC
4.1.1. Oscillator with degraded behavior The input is a small sinusoidal given by xi+2 = 2cos(2 × π
κ )xi+1 − xi N
. q = 0.01 and p = 0 The equation of the model with input signal takes this form: u˙ = −(u+p) + v(u + p)f (βv)f (λ(u + p)) + xi+2 τ (10) v˙ = −αv − (u − q)2 αf 2 (λ(u + p)) 13
ACCEPTED MANUSCRIPT
The value of the initial condition is as follows: x1 = 0.1x0 = 0.2 u0 = 0.001, v0 = −3, p = 0 κ = 1 and N = 113 After implementation, we obtain the result in figure 18.
CR IP T
125
AN US
Figure 18: Oscillator generated by a VSMN
(b) The small one is on the right
ED
M
We modify a value of parameter q = 0.02, and then we take q = −0.02, we obtain graphs illustrate on figures 19(a)and 19(b) with degraded form.
(a) The small one is on the left
First we modify the value of q by q = −0.008, we obtain result on figure 20(a). then for q = 0.008 we obtain result on figure20(b).
AC
CE
130
PT
Figure 19: Different forms of the degraded oscillators
14
CR IP T
ACCEPTED MANUSCRIPT
(a) Degraded on the left direction
AN US
(b) Degraded on the right direction Figure 20: An other example of degraded oscillator
We notice that the value of q has degraded the symmetry of two circles. 4.2. Fractal behavior of a VSMN
M
Fractal behavior can be generated by different classes such as: complex iterated map, iterated function system geometry and Lindmayer system. In this section we show that fractal behavior coexist on neural network. We generate a fractal behavior by one case of our model of neuron. We give the model of neuron as follows: u˙ = −(u+p) + (u + p)vf (βv)f (λ(u + p)) + 0.5xi+2 τ (11) v˙ = −αv − (u − q)2 αf 2 (λ(u + p))
ED
135
PT
The input is a small sinusoidal given by xi+2 = 2cos(2π
κ )xi+1 − xi N
CE
. with q = 0
140
AC
Figure 21 displays fractal behavior of neuron. A same motif repeated in different scales.
Figure 21: Fractal behavior of neuron
15
ACCEPTED MANUSCRIPT
4.3. Harmonic behavior of a VSMN
150
Harmonic studies have become an important aspect of signal theory and power system analysis and design in recent years. Up to now, there isn’t work appeared to show the coexisting harmonic behavior on Hopfield neural network. In this section, we present a harmonic behavior of neuron, we give four examples of neuron models configured by a variable structure model of neuron. The first example, we take three values of parameters n, p and q,the model is described as follows: p = q = 0
CR IP T
145
+ (u + p)vf (βv)f (λ(u + p)) + 4xi+2 u˙ = −(u+p) τ v˙ = −αv − (u − q)2 αf 2 (λ(u + p))
The input is a small sinusoidal given by
AN US
κ )xi+1 − xi N
(12)
xi+2 = 2cos(2π
M
.
Figure 22: Harmonic of stable behavior on neuron
ED
CE
PT
155
In the second example, we modify the model of neuron, we use a model with degraded behavior, keeping the same values of inputs p = 0 and q = −0.008
Figure 23: Harmonic of degraded behavior of neuron
AC
In the third example, we take a model of hole behavior with these values p = 0 and q = 1
16
AN US
CR IP T
ACCEPTED MANUSCRIPT
Figure 24: Harmonic of hole behavior of neuron
ED
M
In the fourth example, we use first model of neuron. + (u + p)vf (βv)f (λ(u + p)) + 4xi+2 u˙ = −(u+p) τ v˙ = −αv − (u + q)αf 2 (λ(u + p))
AC
CE
PT
Figure 25, shows harmonic signal with alternated behavior.
17
(13)
AN US
CR IP T
ACCEPTED MANUSCRIPT
Figure 25: Harmonic alternated behavior of neuron
160
4.4. Chaotic Behavior of a VSMN
4.4.1. Chaos on stable behavior In this section, we study the dynamical behaviors of the model of variable structure by computer simulation. We begin from the case of n = 2, with which the model generates coexisting chaotic attractor. Figure 26 shows chaotic attractor. Take the model of VSMN as follows: + v(u + p)f (βv)f (λ(u + p)) u˙ = −(u+p) τ (14) v˙ = −αv − (u − q)2 αf 2 (λ(u + p))
CE
PT
165
ED
M
In this section, we observe that the chaotic behavior exists in all some models of VSMN. We observe chaotic attractor with two scrolls with the same size, two scrolls with degraded scrolls and with hole scrolls.
AC
Consider a small sinusoidal external input given by
170
xi+2 = 2 × cos(2 × π ×
κ ) × xi+1 − xi N
.
The equation of the model with external input signal takes this form: u˙ = −(u+p) + v(u + p)f (βv)f (λ(u + p)) + xi+2 τ (15) v˙ = −αv − (u − q)2 αf 2 (λ(u + p)) 18
ACCEPTED MANUSCRIPT
PT
ED
M
AN US
CR IP T
175
We set the parameters be τ = 1, α = 1.6, = 0.002, p = q = 0, κ = 5 and N = 1. Figure 26 displays the phase plot of the attractor generated by the initial values u0 = 0.01 and v0 = −0.5
(a)
CE
Figure 26: Chaotic attractor with the same size of the two scrolls
AC
4.4.2. Chaos on degraded behavior In this subsection, we present two chaotic behaviors: the first one contains degraded scroll on the right, and the second has degraded scroll on the left. When we modify the values of q by q = 0.002, it becomes two scrolls with different sizes, so degraded scroll on the right, see figure 27(a). Figure 27(b) displays two scrolls with one degraded behavior when the value q = −0.02.
180
19
AN US
CR IP T
ACCEPTED MANUSCRIPT
(a) Degraded scroll on chaotic attrac- (b) Degraded scroll on chaotic attractor on the right direction tor on the left direction
5. Symbol of a VSMN
M
Figure 27: Different forms of degraded behavior
NBDn
(p,q)
Figure 28: Graph of neuron
CE
PT
ED
In this section, we represent a VSMN by the following symbol:
The structure of graph contains four parts, D=k, B,n, p,q. the parameter of n indicates the model of neuron. Neuron has two potentials positive and negative activity directions which means all states of neuron have positive or negative values. The symbol of D is direction of potential of neuron. We present some graphs of neurons generated by the VSMN. B it means the behavior of neuron such as: stable, hole ,degraded , oscillator, harmonic, fractal and chaos. we use some symbols to indicate in which steady neuron is. for example :
AC
185
190
• A is in stable behavior. • H is on hole behavior. 20
ACCEPTED MANUSCRIPT
• D is on degraded behavior.
195
• F is on fractal behavior. • M is on harmonic behavior.
CR IP T
• O is on oscillator behavior. • C is on chaos behavior. 200
5.1. Symbol of first model of neuron
AN US
In this graph, we take n = 0 and p = q. This model is used as membership layer see [2].
NA+0
=⇒
(p=q,q)
(b) Behavior of neuron on negative polarity
(a) Symbol of first model of neuron
NA−0
=⇒
M
(p=q,q)
(c) Symbol of first model of neuron
(d) Behavior of neuron on positive polarity
AC
CE
PT
ED
Figure 29: Graph of first model of neuron and its behavior
21
ACCEPTED MANUSCRIPT
NA+1
CR IP T
5.2. Symbol of second model of neuron
=⇒
(p=q,q)
AN US
(a) Symbol of second model of neuron
M
(b) Behavior of neuron begin with positive polarity
(p=q,q)
=⇒
ED
NA−1
CE
PT
(c) Symbol of second model of neuron
(d) Behavior of neuron begin with negative polarity
Figure 30: Graph of second model of neuron and its behavior
AC
205
5.3. Symbol of third model of neuron In this case, the value of n is equal to 2. This model as we demonstrated in the previous section has multi forms of behavior, stable behavior, degraded behavior left or right with two position of polarity and hole behavior.
22
ACCEPTED MANUSCRIPT
5.3.1. Symbol of stable behavior
NA+2
=⇒
(p=q,q)
(a) Symbol of third model of neuron
NA−2
CR IP T
210
(b) Behavior of neuron on positive polarity
(c) Symbol of third model of neuron
AN US
=⇒
(p=q,q)
(d) Behavior of neuron on negative polarity
M
Figure 31: Graph of third model of neuron and its behavior
PT
ED
5.3.2. Symbol of hole behavior
+ NH 2
=⇒
CE
(p=−q,q)
AC
(a) Symbol of second model of neuron
(b) Hole behavior on positive polarity Figure 32: Graph of third model of neuron with hole behavior
23
ACCEPTED MANUSCRIPT
− NH 2
(a) Symbol of second model of neuron
CR IP T
=⇒
(p=−q,q)
(b) Hole behavior on negative polarity Figure 33: Graph of third model of neuron with hole behavior
AN US
5.3.3. Symbol of degraded behavior up direction
+ ND 2
=⇒
(p,q=−p+)
+ ND 2
(p,q=−p−)
ED
M
(a) Symbol of second model of neuron
(b) Small lob is on right and high direction
=⇒
(c) Symbol of second model of neuron
PT
(d) Small lob is on left and high direction
AC
CE
Figure 34: Graph of third model of neuron with degraded behavior
24
ACCEPTED MANUSCRIPT
− ND 2
=⇒
(p,q=−p−)
(a) Symbol of second model of neuron
CR IP T
5.3.4. Symbol of degraded behavior down direction
AN US
(b) Small lob is on left and low direction
− ND 2
=⇒
M
(p,q=−p+)
ED
(c) Symbol of second model of neuron
PT
(d) Small lob is on right and low direction
Figure 35: Degraded with negative polarity and with small lob on the right
CE
215
5.4. symbol of fourth of model
AC
In this case, the value of n is equal to 3.
25
ACCEPTED MANUSCRIPT
NA+3
CR IP T
=⇒
(p=q,q)
(a) Symbol of fourth model of neuron
(b) Alternate behavior
− ND 3
=⇒
(p=q,q)
AN US
(c) Symbol of fourth model of neuron
(d) Alternate with degraded behavior
Figure 36: Graph of fourth model of neuron and its behavior
6. Multi neural networks
M
6.1. System of neural networks Here, we present a structure of a system of neural network by associating multiple neurons in a cascading manner. This structure starts with a set of initial conditions, a number of neurons. Let E be the complete metric unit, Φ a system of neural network in E such as:
PT
225
In this section, we show some examples of neural network with the previous models.
ED
220
E →E Φ:(ui , vi ) → (um , vm )
The equation system of neural network Φ is represented by: (u0 , v0 ) (ui+1 , vi+1 ) = N1 (ui , vi ) (u i+2 , vi+2 ) = N2 (ui+1 , vi+1 ) .. . Φ (uj+2 , vj+2 ) = Nj (uj+1 , vj+1 ) . .. (um , vm ) = Nm (um−1 , vm−1 )
CE AC
230
(16)
The dynamics of the neural network system is controlled by the assignment of ui+1 to ui and of vi+1 to vi . 26
ACCEPTED MANUSCRIPT
6.2. Topology of neural network 235
In this paragraph, we present two designs of neural networks the first topology with full connected. In the second topology, we give a design with a simple connection in a cascade.
NBDn
NBDn
D n NB
j1 ,(p,q)
ED
.. .
D n NB
M
i1 ,(p,q)
j0 ,(p,q)
AN US
i0 ,(p,q)
CR IP T
Example 6.2.1. Figure below shows a neural network with two layers, in each layer, we have k neurons.
D n NB
.. .
D n NB
jk ,(p,q)
AC
CE
PT
ik ,(p,q)
240
P
Example 6.2.2. Figure below shows a neural network with two layers, in each layer, we have k neurons, all neurons on each layer are connected in a cascade.
27
ACCEPTED MANUSCRIPT
NBDn
NBDn
i0 ,(p,q)
D n NB
D n NB
.. .
D n NB
i1 ,(p,q)
AN US
j1 (p,q)
CR IP T
j0 (p,q)
.. .
D n NB
ik ,(p,q)
PT
ED
M
jk (p,q)
P
6.3. Neural network with the third model of neuron (n=2) on positive polarity We built a neural network with five neurons generates by VSMN. In this network, for each neuron we use the model n = 2 and positive polarization (k = +1).
CE
245
AC
NA+2
(p=q,q=α1 )
NA+2
(p=q,q=α2 )
NA+2
(p=q,q=α3 )
NA+2
(p=q,q=α4 )
Figure 37: Neural Network with five neurons each neuron has its position of threshold
28
NA+2
(p=q,q=α5 )
ACCEPTED MANUSCRIPT
as:
Let E be the complete metric unit, Φ a system of neural network in E such E →E Φ:(ui , vi ) → (um , vm )
CR IP T
250
(17)
AN US
The equation system of neural network Φ is represented by: 1) + f (βv)fs (λ(u + p1 )) u˙1 = −(u+p NP1 τ 2 v ˙ = −αv + αf 1 s (λ(u + p1 )) −( u ˙ +p ) 1 2 u˙2 = + v˙1 (u˙1 + p2 )fb (βv1 )f (λ(u˙1 + p2 )) τ NP2 v ˙ = −α v ˙ + ( u ˙1 + p2 )2 αf 2 (λ(u˙1 + p2 )) 2 1 −(u˙2 +p3 ) + v˙2 (u˙2 + p3 )fb (βv2 )f (λ(u˙2 + p3 )) u˙3 = τ NP3 v ˙ = −α v ˙ + ( u ˙2 + p3 )2 αf 2 (λ(u˙2 + p3 )) 2 3 −( u ˙ +p ) 3 4 u˙4 = + v˙3 (u˙3 + p4 )fb (βv3 )f (λ(u˙3 + p4 )) τ NP4 v ˙ = −α v ˙ + ( u ˙3 + p4 )2 αf 2 (λ(u˙3 + p4 )) 3 4 −(u˙4 +p5 ) + v˙4 (u˙4 + p5 )fb (βv4 )f (λ(u˙4 + p5 )) u˙5 = τ NP5 v˙5 = −αv˙4 + (u˙4 + p5 )2 αf 2 (λ(u˙4 + p5 ))
M
The result is generated by five neurons, depicted in figure 38.
ED
Figure 38: Network with five neurons
6.4. Neural network with the third model of neuron on negative polarity
PT
We give some architectures of neural networks, all neurons are on negative polarization (k = −1).
CE
255
NA−2
AC
(p=q,q=α1 )
NA−2
(p=q,q=α2 )
NA−2
(p=q,q=α3 )
NA−2
(p=q,q=α4 )
Figure 39: Neural Network with five neurons each neuron has its position of threshold
Figure 40 displays curve which has symmetry the previous neural network.
29
NA−2
(p=q,q=α5 )
Figure 40: Network with five neurons
6.5. Neural network with bi potentials
NA−2
NA−2
(p=q,q=α1 )
NA+2
(p=q,q=α1 )
AN US
(p=q,q=α1 )
CR IP T
ACCEPTED MANUSCRIPT
Figure 41: Neuron with positive potential and position of threshold = α
Let E be the complete metric unit, Φ the equation system of neuronal network with chaotic attractor takes as follows:
1) u˙1 = −(u+p + f (βv)fs (λ(u + p1 )) τ v˙1 = −αv + αfs2 (λ(u + p1 )) u˙2 = −(u˙1τ+p2 ) + v˙1 (u˙1 + p2 )fb (βv1 )f (λ(u˙1 + p2 )) v˙2 = −αv˙1 + (u˙1 + p2 )2 αfb2 (λ(u˙1 + p2 )) u˙3 = −(u˙2τ+p3 ) + v˙2 (u˙2 + p3 )fb (βv2 )f (λ(u˙2 + p3 )) v˙3 = −αv˙2 − (u˙2 + p3 )2 αfb2 (λ(u˙2 + p3 ))
AC
CE
PT
ED
N P 1 NP2 N P3
M
E →E Φ:(u, v) → (u3 , v3 )
260
Figure 42: Network with three neurons
30
(18)
ACCEPTED MANUSCRIPT
6.6. Neural network with Hole behavior The equation system of neural network is represented by:
NA+2
(p=q,q=α1 )
+ NH 2
(p=q,q=α2 )
(p=q,q=α3 )
NA−2
CR IP T
− NH 2
(p=q,q=α4 )
Figure 43: Neural Network with alternated behaviors
265
Let E be the complete metric unit, Φ the equation system of neuronal network with chaotic attractor takes as follows:
M
1) + f (βv)fs (λ(u + p1 )) u˙1 = −(u+p τ v˙1 = −αv + αfs2 (λ(u + p1 )) u˙2 = −(u˙1τ+p2 ) + v˙1 (u˙1 + p2 )fb (βv1 )f (λ(u˙1 + p2 )) v˙2 = −αv˙1 + (u˙1 + p2 )2 αfb2 (λ(u˙1 + p2 )) u˙3 = −(u˙2τ+p3 ) + v˙2 (u˙2 + p3 )fb (βv2 )f (λ(u˙2 + p3 )) v˙3 = −αv˙2 + (u˙2 + p3 )2 αfb2 (λ(u˙2 + p3 )) u˙4 = −(u˙3τ+p4 ) + v˙3 (u˙3 + p4 )fb (βv3 )f (λ(u˙3 + p4 )) v˙4 = −αv˙3 + (u˙3 + p4 )2 αfb2 (λ(u˙3 + p4 ))
AC
CE
PT
ED
N P 1 N P2 N P3 NP4
AN US
E →E Φ:(u, v) → (u4 , v4 )
Figure 44: Network with four neurons
We give another result of neural network. 31
(19)
Figure 45: Network with five neurons
7. Application in chaos
275
In this section, we introduce various examples of neural networks designed by different models of neurons where each one of them prints its signatures on orbits chaotic attractor witch pass through position of threshold. The traces of neurons change the behavior of scrolls or orbits of chaotic attractor. Let E be the complete metric unit, Φ the equation system of neuronal network with chaotic attractor takes as follows:
AN US
270
CR IP T
ACCEPTED MANUSCRIPT
E →E Φ:(u, v) → (um , vm )
M
of neuronal network with chaotic attractor takes this x˙ 1 = f1 (x1 , x2 , x3 ) x˙ 2 = f2 (x1 , x2 , x3 ) x˙ 3 = f3 (x1 , x2 , x3 ) (u1 , v1 ) = N1 (x1 , x2 ) .. .
CE
PT
ED
The equation system form:
280
(ui+2 , vi+2 ) = N2 (ui+1 , vi+1 ) .. .
(20)
(uj+2 , vj+2 ) = Nj (uj+1 , vj+1 ) .. . (um , vm ) = Nm (um−1 , vm−1 )
7.1. Trace of neuron in Chua attractor in two lines of orbits
AC
In this subsection, we show the implementation results of a combination between Chua’s chaotic attractor and neuron. Let E be the complete metric unit, Φ the equation system of neuronal network with chaotic attractor takes as follows:
285
E →E Φ:(x1 , x2 ) → (u1 , v1 )
32
ACCEPTED MANUSCRIPT
(21)
CR IP T
x˙ 1 = f1 (x1 , x2 , x3 ) x˙ 2 = f2 (x1 , x2 , x3 ) x˙ 3 = f3 (x1 , x2 , x3 ) u˙1 = −(x˙ 1τ+p1 ) + f (β x˙ 2 )fs (λ(x˙ 1 + p1 )) v˙1 = −αx˙ 2 + αfs2 (λ(x˙ 1 + p1 ))
AN US
Figure 46: Track of neuron in Chua attractor
7.2. Trace of neuron in Chua attractor in all lines of orbits
Figure: Fig: 47(b) Fig: 47(c) Fig: 47(d) Fig: 47(e) Fig: 47(f) Fig: 47(g)
AC
CE
PT
ED
Model of neuron Degraded model Stable model Alternated model Fourth model First model Hole model
M
290
In this subsection, we stimulate Chua attractor with some models of neurons configured by variable structure model of neuron 13. In figures 47(b),47(c),47(d),47(e) and 47(g) we use the same position of threshold, in figure 47(f) we modify the position of the threshold.
33
(c) Track of neuron third model of neuron
AN US
(a) Chua attractor used for stimulation
(b) Track contains degraded behavior of neuron
CR IP T
ACCEPTED MANUSCRIPT
(e) Track of fourth model
(f) Track of first model
PT
ED
M
(d) Track of second model
(h) Zooming of hole behavior
Figure 47: Signature of neural network on its stimulate
AC
CE
(g) Track of neuron contains hole behavior
295
In the next example, we show that signatures of neurons are inserted in all lines of orbits. We give four neurons closed with four different positions of thresholds. In figure 48, it can be obviously seen the trace of neural network in Chua attractor. For all neurons we use the same model as follows:
34
ACCEPTED MANUSCRIPT
(22)
and we take the following value of neurons: p1 = −0.5, p2 = 0.5, p3 = −0.75 and on fourth neuron the value of threshold position is p4 = 0.75. Figure 48 shows scrolls with multi signatures of neurons.
M
AN US
CR IP T
300
u˙ = −(u+p) + v(u + p)f (βv)f (λ(u + p)) τ v˙ = −αv − (u + q)2 αf 2 (λ(u + p))
ED
Figure 48: Signature of neural network on its stimulate
The neuron makes signature in all lines of orbits when it passes by the position of threshold. 7.3. Trace of neurons in Chua attractor with different behaviors
PT
305
CE
We give example of neural network with Chua attractor. In this example, we stimulate Chua attractor by five neurons with different models of neurons and positions of threshold. The equation system of neural network is described by: Let E be the complete metric unit, Φ the equation system of neuronal network with chaotic attractor takes as follows:
AC
310
E →E Φ:(x1 , x2 ) → (u5 , v5 )
35
ACCEPTED MANUSCRIPT
= f1 (x1 , x2 , x3 ) = f2 (x1 , x2 , x3 ) =f3 (x1 , x2 , x3 ) u˙1 = −(x˙ 1τ+p1 ) + f (β x˙ 2 )f (λ(x˙ 1 + p1 )) NP1 x˙ 2 + α(x˙ 1 + p1 )f 2 (λ(x˙ 1 + p1 )) v˙1 = −α −(u˙1 +p2 ) + v˙1 (u˙1 + p2 )f (βv1 )f (λ(u˙1 + p2 )) u˙2 = τ NP2 2 v ˙ = −α v ˙ + αf 1 b (λ(u˙1 + p2 )) 2 −(u˙2 +p3 ) u˙3 = + v˙2 (u˙2 + p3 )f (βv2 )f (λ(u˙2 + p3 )) τ NP3 v˙3 = −αv˙2 + (u˙2 − p3 )2 αf 2 (λ(u˙2 + p3 )) u˙4 = −(u˙3τ+p4 ) + v˙3 (u˙3 + p4 )f (βv3 )f (λ(u˙3 + p4 )) NP4 v˙3 + (u˙3 + p4 )3 αf ( λ(u˙3 + p4 )) v˙4 = −α −(u˙4 +p5 ) + v˙4 (u˙4 + p5 )f (βv4 )f (λ(u˙4 + p5 )) u˙5 = τ NP5 v˙5 = −αv˙4 + (u˙4 + p5 )2 αf 2 (λ(u˙4 + p5 ))
CR IP T
x˙ 1 x˙ 2 x˙ 3
(23)
AN US
M
Figure 49 shows results of implementation.
7.4. Signature of neurons with degraded behaviors Let E be the complete metric unit, Φ a system of neural network in E such as: E →E Φ:(x1 , x2 ) → (u3 , v3 )
CE
PT
315
ED
Figure 49: Traces of five neurons in Chua attractor
AC
320
The equation system of neural network Φ is represented by: x˙ 1 = f1 (x1 , x2 , x3 ) x˙ 2 = f2 (x1 , x2 , x3 ) x˙ 3 = f3 (x1 , x2 , x3 ) (u1 , v1 ) = Pj (x˙ 1 , x˙ 2 ) (u2 , v2 ) = Pj (u1 , v1 )
(24)
if
(u2 < 0)(p = p1 )
u˙3 = −(u2τ+p1 ) + f (βv2 )f (λ(u2 + p1 )) v˙3 = −αv2 + α(u2 + p1 )f 2 (λ(u2 + p1 )) 36
(25)
ACCEPTED MANUSCRIPT
if (u2 > 0)(p = p2 ) (26)
Figure 50(a) illustrates a chaotic attractor with four scrolls, this result is output of response by stimulation of neuron. In the middle of chaotic behavior, we notice that neuron makes its trace by the same form of signature. Figure 50(b) shows response of stimulation with two different directions of traces. Figure 50(c), Figure 50(d) and Figure 50(e) display different cases of signatures.
AC
CE
PT
ED
M
AN US
325
u˙3 = −(u2τ+p2 ) + f (βv2 )f (λ(u2 + p2 )) v˙3 = −αv2 + α(u2 + p2 )f 2 (λ(u2 + p2 ))
CR IP T
37
CE
PT
ED
M
AN US
CR IP T
ACCEPTED MANUSCRIPT
AC
Figure 50: Different signatures of neurons in the same position of axis
330
7.5. Oscillator from one neuron in chaotic attractor In this example, both figures 51(a) and 51(b) contain oscillator. Each of them exhibits two forms of signatures from two model of neurons. And each one has its threshold position. We take the first neuron is as follows: u˙ = −(u+p) + v(u + p)f (βv)f (λ(u + p)) τ (27) v˙ = −αv − (u + q)2 αf 2 (λ(u + p)) 38
ACCEPTED MANUSCRIPT
AN US
With threshold position p = 2 and q = ±α.
(28)
CR IP T
with threshold position p = q = +2. And the second is given by: u˙ = −(u+p) + v(u + p)f (βv)f (λ(u + p)) τ v˙ = −αv − (u + q)2 αf 2 (λ(u + p))
(a) Nested oscillator and signature of neuron
(b) Oscillator with an opposition of phase
335
M
Figure 51: One oscillator in a chaotic attractor
7.6. Oscillators of two neurons in a chaotic attractor
AC
CE
PT
ED
Result in figure 52 shows traces of neurons which have two oscillators, one with degraded form and the other with stable one.
Figure 52: Two oscillators with two behaviors
Although figure53 shows an exact two signatures alternated with two oscillators, all are between scrolls.
39
CR IP T
ACCEPTED MANUSCRIPT
Figure 53: Two oscillators with the same behavior contain two scales of sizes
7.7. Multi scales of signatures in a chaotic attractor from neural networks
AN US
340
In this example, we use four neurons with different threshold positions of the same model as follows: i) u˙ = −(u+p + v(u + pi )f (βv)f (λ(u + pi )) τ (29) v˙ = −αv − (u + qi )2 αf 2 (λ(u + pi ))
PT
ED
M
The values of position threshold are p1 = q1 = +2, p2 = q2 = +1, p3 = q3 = +0, p4 = q4 = −1. Figure 54 shows signatures of four neurons on a chaotic attractor.
CE
Figure 54: Four signatures on two orbits of chaotic attractor
The order of threshold position of neuron has a great effect on the behavior of its stimulate. In this case the values of position threshold are p = q = −1,p = q = 0,p = q = +1, p = q = 2. Figure 55 shows signatures of four neurons on a chaotic attractor in opposite direction refereed to the previous result.
AC
345
40
CR IP T
ACCEPTED MANUSCRIPT
Figure 55: Direction of signatures change from left to right
In this example, we use four neurons with the same model as follows: 1. First neuron: u˙ = −(u+p) + v(u + p)f (βv)f (λ(u + p)) τ (30) v˙ = −αv − (u + q)2 αf 2 (λ(u + p))
AN US
350
With the threshold position p = q = +2 2. Second neuron: u˙ = −(u+p) + v(u + p)f (βv)f (λ(u + p)) τ v˙ = −αv + (u + q)2 αf 2 (λ(u + p))
M
With the threshold position p = q = +1 3. Third neuron: u˙ = −(u+p) + v(u + p)f (βv)f (λ(u + p)) τ v˙ = −αv − (u + q)2 αf 2 (λ(u + p))
ED
355
PT
With the threshold position p = q = +0 4. Fourth neuron u˙ = −(u+p) + v(u + p)f (βv)f (λ(u + p)) τ v˙ = −αv + (u + q)2 αf 2 (λ(u + p)) With threshold position is p = q = −1
AC
CE
Figure 56 shows result of traces on a chaotic attractor.
Figure 56: Four traces of neurons on a chaotic attractor
41
(31)
(32)
(33)
ACCEPTED MANUSCRIPT
360
8. Application in robotics 8.1. First example In this subsection, we give two sinusoids
CR IP T
x2 = 2 × cos(2.3.14.223/27) × x1 − x0 y2 = 2 × cos(2.3.14.223/27) × y1 − y0
u = αy2 and v = αx2 ; u˙ = −(u+p) + v(u + p)f (βv)f (λ(u + p)) τ v˙ = −αv + (u + q)2 αf 2 (λ(u + p))
With two threshold positions of neurons, the first neuron has the value p = −1 and the second p = −3 Figure 57(a) and Figure 57(b) show results of first example.
AN US
365
(34)
M
(a) Signature of one neuron in a (b) Traces of two neurons in a trajectory of robot trajectory of robot
ED
Figure 57: Robot path with signals of neurons
CE
PT
In the next example, we show a result of robot path with two signatures of two neurons, on the first neuron we use the position of threshold p1 = −1 and the second neuron p2 = +0 Model of first neuron is as follows: u˙ = −(u+p) + v(u + p)f (βv)f (λ(u + p)) τ (35) v˙ = −αv + (u + q)2 αf 2 (λ(u + p)) the second model is:
AC
370
u˙ = −(u+p) + v(u + p)f (βv)f (λ(u + p)) τ v˙ = −αv + (u + q)3 αf 2 (λ(u + p))
Figure 58 two different behavior of signatures in a trajectory of robot
42
(36)
(a) Traces of two models of neurons in trajectory of robot
CR IP T
ACCEPTED MANUSCRIPT
Figure 58: Robot path with signals of neurons
M
AN US
375
8.2. Second example We give a second example of robot path generated by neural networks. Figure 59(a)shows robot path without signature of neuron. Figure 59(b) displays path with trace of neuron. This trace can be used for avoid obstacle on robot path.
ED
(a) Robot path without signature of neuron (b) Robot path with signature of neuron Figure 59: Robot path with signal of neurons
This paper is an attempt to solve topological problems involved in relationship between chaos, fractal and neuroscience. By combining chaos and the Hopfield’s type of neural network, we have finally found out some main findings. First, we have developed a variable structure model of neuron. From this structure we generate four models, For each one of these models, its behavior can take as follows: Stable or equilibrium, degraded, hole, alternated, oscillator, fractal, harmonic, chaos behaviors. In addition, we have also elaborate a graph of these neurons and we design an architecture of neural networks, then we gave examples of topology with simulation. finally we applied our approach in significant application such as: chaos and robotic. It seems that, when neuron in oscillator behavior, that looks like Parkinson’s disease. When the neuron takes the hole behavior it gives an idea about Alzheimer disease behavior. I think that, there are some potential research directions that could be considered for the future works.
AC
CE
380
PT
9. Conclusion
385
390
43
ACCEPTED MANUSCRIPT
References
395
[1] Amitava, Kundu., Pritha, Das., A.B,Roy. ”Complex dynamics of a four neuron network model havinga pair of short-cut connections with multiple delays,” Int. J. Nonlinear Dyn 72, pp.643 − 662(2013).
405
[3] Cquan,Yuan. & Xiao-Song,Yang.”computer assisted verification of chaos in three-neuron cellular neural networks,” Int. J. Bifurcation and Chaos 17, 12, pp.4381 − 4386(2007).
[4] Chunguang, Li.& Guanrong, Chen. ”Coexisting chaotic attractors in a single neuron model with adapting feedback synapse,” Chaos, Solitons and Fractals 23, pp.1599 − 1604(2005).
AN US
400
CR IP T
[2] A. Mohammadzadeh, S. Ghaemi. A modified sliding mode approach for synchronization of fractional-order chaotic/hyper chaotic systems by using new self-structuring hierarchical type-2fuzzy neural network. Int. J. Neurocomputing (2016), http://dx.doi.org/10.1016/j.neucom.2015.12.098
[5] G. Lee and N.H. Farhat. ”The Bifurcating Neuron Network,” Neural Networks14, pp.115 − 131(2001).
M
410
[6] J.J. Hopfield. ” Neurons with graded response have collective computational properties like those of two-state neurons,” Proc. Natl. Acad. Sci. USA 81, 17, pp.3088 − 3092(1984).
415
ED
[7] Jianquan, Lu. Daniel W. C. HO& Jinde, CAO. ”Synchronization in an array of nonlinearly coupled chaotic neural networks with delay coupling ,” Int. J. Bifurcation and Chaos 18, 10pp.3101 − 3111(2008). [8] Juan Meng & Xing-yuan Wang. ”Robust anti-synchronization of a class of delayed chaotic neural networks,” Chaos, 17, 023113 − 7(2007).
PT
[9] Kreangkri, Ratchagit. ”Asymptotic stability of delay-difference system of Hopfield neural networks via matrix inequalities and application,” Int. J. Neural Systems 17, 5, pp.425 − 430(2007). [10] K. Bouallegue. ”Gallery of Chaotic Attractors Generated by Fractal Network,”Int. J. Bifurcation and Chaos 25, pp.1530002 − 18(2015).
CE 420
AC
[11] K. Bouallegue.”A new fractal model of chromosome and DNA processes ,” Chapter, Book. Proc. The 4th International Interdisciplinary Chaos Symposium Chaos and Complex Systems.pp.505 − 514(2012).
425
[12] Lenze.,J¨ org, Raddatz. ”Effects of dilation and translation on a perceptiontype learning rule for higer order Hopfield neural networks,” Int. J. Neural Systems12, 2, pp.83 − 93(2002).
[13] Marc Timme, Theo Geisel, and Fred Wolf. ”Speed of synchronization in complex networks of neural oscillators: Analytic results based on Random Matrix Theory”Chaos 16, 015108(2006) 44
ACCEPTED MANUSCRIPT
430
[14] M. Stork,”Energy feedback used for oscillators control,” Nonlinear Dyn 85 : 871 − 879(2016)
435
440
CR IP T
[15] Nimet,Korkmaz. & Recai, Kilic. ”Implementations of Modified Chaotic Neural Models with Analog Reconfigurable Hardware,” Int. J. Bifurcation and Chaos 2, 4, pp.11450046 − 1450061[2014]. [16] Qingdu, Lia. Xiao-Song,& Yangb. Fangyan, Yanga.” Hyperchaos in Hopfield-type neural networks,” Int. J. Neurocomputing 67, pp.275 − 280(2005). [17] Teijiro, Isokawa., Haruhiko, Nishimura., Naotake,Kamiura and Nobuyuki, Matsui. ”Associative memory in quaternionic Hopfield neural network,” Int. J. Neural Systems 18, 2, pp.135 − 145(2008).
445
AN US
[18] Shyan-Shiou Chen and Chih-Wen Shih.”Transversal homoclinic orbits in a transiently chaotic neural network”Chaos, 12, 3, (2002).
[19] RezaMazrooei-Sebdani, SaeedFarjami. ”On adiscrete-time-delayed Hopfield neural network with ring structures and different internal decays:Bifurcations analysis and chaotic behavior,” Int. J. Neurocomputing151, pp.188 − 195(2015).
[21] Wei-Yen, Hsu. ”Application of competitive Hopfield neural network to brain-computer interface systems,” Int. J. Neural Systems 22, 1, pp.51 − 62(2012).
AC
CE
PT
ED
450
M
[20] Yuanlong, Chen.,Tingwen, Huang. & Yu, Huang. ”Complex dynamics of a delayed discrete neural network of two nonidentical neurons,” AIP Chaos 24, pp.013108(1999).
45
AN US
CR IP T
ACCEPTED MANUSCRIPT
M
AC
CE
PT
460
ED
455
Kais Bouallegue is Assistant Professor in the Higher Institute of applied Sciences and Technology of Sousse, Tunisia. He got his PhD from National Engineering School of Sfax. He is active member in different industrial companies. He has served a reviewer for technical papers. Also he has mentored researchers at undergraduate and graduate levels. Moreover, He has been collaborating with many different international universities. His currently research interests include fractal, chaos, genomic and complex system
46
Accepted Manuscript
A New Class of Neural Networks and its Applications Kais Bouallegue PII: DOI: Reference:
S0925-2312(17)30487-3 10.1016/j.neucom.2017.03.006 NEUCOM 18226
To appear in:
Neurocomputing
Received date: Revised date: Accepted date:
27 April 2016 19 July 2016 2 March 2017
Please cite this article as: Kais Bouallegue, A New Class of Neural Networks and its Applications, Neurocomputing (2017), doi: 10.1016/j.neucom.2017.03.006
This is a PDF file of an unedited manuscript that has been accepted for publication. As a service to our customers we are providing this early version of the manuscript. The manuscript will undergo copyediting, typesetting, and review of the resulting proof before it is published in its final form. Please note that during the production process errors may be discovered which could affect the content, and all legal disclaimers that apply to the journal pertain.
ACCEPTED MANUSCRIPT
A New Class of Neural Networks and its Applications Kais Bouallegue1
CR IP T
Department of Electrical Engineering. Higher Institute of Applied Sciences and Technology of Sousse, Tunisia. kais [email protected].
Abstract
M
AN US
We present a new class of neural network using a variable structure model of neuron (VSMN). From this structure, we generate four models of neurons. For each model, we study different behaviors such as stable or equilibrium , degraded, hole, alternated, oscillator, harmonic, fractal, and chaos behaviors. Then we design different topologies and architectures of neural networks. These architectures are different from the classical ones; each layer of network contains different models of neurons, neurons can take four models by configuration of VSMN. We also present a numerical study describing the behavior of some models of neuron. We illustrate some results to show the efficiency of this new class of neural networks. We show that these neurons put tracks on their stimulators such as: signal track and half bounded region track with two high and low directions. Two applications in chaos and robotic are also given.
ED
Keywords: Variable structure model of neuron, neural networks, chaos, harmonic, oscillator, fractal, signature of neuron.
PT
1. Introduction
CE
After decades, many artificial neural networks have been proposed to develop tools that may lead to a better understanding of the behavior of brain. Their various versions have also been intensively and extensively studied in many aspects and successfully applied to many fields such as identifying pattern, recognizing voice, controlling system, processing signal systems, treating static image, and solving nonlinear algebraic system, etc. ,[21],[17], [12], [15],[7],[20],[3],[8], [1]. Hopfield’s neural network is a model of artificial neural network which appears to be a powerful tool for modeling biological neural networks [9]. It is known that the human brain has 1010 neurons, and each neuron has 103 to 104 connections with other neurons. Each neuron controls and drives the dynamical behavior of organs. Some neurons work all the time, others work only when
AC
5
10
✩ Kais
[email protected]
Preprint submitted to Elsevier
March 21, 2017
ACCEPTED MANUSCRIPT
25
30
CR IP T
20
AN US
15
their sensors are stimulated. In this work, we notice that when the neuron is in a “hole behavior” some information of its stimulator are lost. We think that this model of neuron, in this case, has great similarity with Parkinson disease. We also notice that when the sign of threshold position of neuron is switching between positive and negative, the neuron adopts an oscillating behavior. Probably, this is the model of Alzheimer disease. The results reached in this work shed some lights on neuroscience. Section 2, provides a mathematical approach of our models. Section 3 is devoted to a model of neuron with a variable structure governed by a specific function. Four models generated by this structure are studied. For each model, we find the maximum point of the associated function, and then we illustrate different behaviors of the models. In section 4, we present a multi behaviors of VSMN such as oscillator, fractal, harmonic and chaos behaviors some examples of each behavior are illustrated. Section 5, provides representations of VSMN which will be used in the topology of neural networks. In section 6, we design two examples of architectures of neural network using VSMN. In section 7, we apply our models of VSMN in chaotic attractors. Two examples of path of a robot are given (see Section 8). We close the paper by a conclusion. 2. A Mathematical Model of a VSMN
M
PT
ED
35
In this section, we elaborate a model of neurons with variable structure. The structure depends on four variables (n, p, q, k), where n, q are related to the behavior of the model, p is related to the model’s threshold position and k represents the neuron’s polarity. A Variable Structure Model of Neurons (VSMN, for short) is described by the following system of differential equations: + (u + p)vf (βv)f (λ(u + p)) u˙ = −(u+p) τ (1) v˙ = −αv + k(u + q)n αf 2 (λ(u + p)) t2
CE
40
with f (t) = e− 2 and k = ±1, τ is a time constant p, q are real numbers, λ, α, β are positive real numbers. Take g(u) := v˙ + αv = k(u + q)n αf 2 (λ(u + p)). Then to study the previous model, it suffices to focus on the following function: g(x) = k(x + q)n e−
(x+p)2 2
.
AC
Clearly, it is enough to consider the case k = +1. If n 6= 0 and n 6= 1, then the critical points of the function are given by the following equation: g 0 (x) = ((x + q)n−1 (n − (x + q)(x + p))e−
(x+p)2 2
= 0.
√ p+q+ ∆ −2
√
∆ The roots of this equation are x1 = −q, x2 = and x3 = p+q− −2 with ∆ = (p − q)2 + 4n. The values of the function at these critical points are
45
g(x1 ) = 0, g(x2 ) = (x2 + q)n e−
(x2 +p)2 2
and g(x3 ) = (x3 + q)n e− 2
(x3 +p)2 2
.
ACCEPTED MANUSCRIPT
In order to study the behavior of a VSMN influenced by a stimulator, we investigate four cases.
50
0
0.8
-0.2
0.6
-0.4
0.4
-0.6
0.2
-0.8
-4
-2
0
2
M
0
AN US
1
CR IP T
1. Case n = 0. −(x+p)2 The function becomes g(x) = e 2 ; the unique critical point is x = −p and g(−p) = 1 is the maximum value of the function.
4
-1 -4
(a) n = 0, k = +1, p = 0
-2
0
ED
Figure 1: Stable behavior of a VSMN for n = 0
AC
CE
PT
2. Case n = 1 −(x+p)2 If p = q, then the function becomes g(x) = (x + p)e 2 , the critical points are x1 = −p + 1 and x2 = −p − 1. The maximum value of the −1 −1 function is g(x1 ) = e 2 and the minimum value is g(x2 ) = −e 2 .
3
2
(b) n = 0, k = −1, p = 0
4
ACCEPTED MANUSCRIPT
0.6
1
0.4 0.5
0 0 -0.2 -0.4
-0.5
-1 -4
-2
0
2
(a) k = +1, p = q = 0
AN US
-0.6
CR IP T
0.2
4
-0.8 -4
-2
0
(b) k = −1, p = q = 0
Figure 2: Stable behavior of a VSMN for n = 1
3. Case n ≥ 2 and p = q In this case, the critical points are
M
55
x1 = −p, x2 = −p −
√
n and x3 = −p +
√
n.
ED
The values of g at these critical points are as follows: √ √ n n g(x1 ) = 0, g(x2 ) = (− n)n e− 2 and g(x3 ) = ( n)n e− 2 .
CE
PT
If the function g has two critical points with the same absolute maximal value, we say that the VSMN model has a stable behavior. - When n is even, the function has two maximum values √ n g(x2 ) = g(x3 ) = ( n)n e− 2
AC
Figure 3 displays the form of the model. This form contains two lobes with the same sizes, one on the left other on the right.
4
2
4
ACCEPTED MANUSCRIPT
0.8
CR IP T
0.6
0.4
0
-4
-2
AN US
0.2
0
2
4
Figure 3: Stable behavior of the model (for n = 2)
Figure 4 displays the form of the model. This form contains two lobes with alternate directions (up and down).
ED
60
M
- When n is odd, the function has a minimal value and a maximal maximum value with the same absolute value √ n g(x2 ) = ( n)n e− 2 = −g(x3 ).
1.5
PT
1
AC
CE
0.5
0
-0.5 -1 -1.5 -4
-2
0
2
4
Figure 4: Stable behavior of the model(for n = 3)
5
ACCEPTED MANUSCRIPT
4. Case n ≥ 2 and p = −q p p Here, the critical points are x1 = p, x2 = − p2 + n and x3 = p2 + n. The values of g at these points are
and g(x3 ) = (
p
p
p2 + n − p)n e−
p2 + n − p)n e−
√
(
√
(−
p2 +n+p)2 2
CR IP T
g(x1 ) = 0, g(x2 ) = (−
p2 +n+p)2 2
.
Notice that, if |p| is a large positive real number (approaching +∞), then g(x2 ) ≈ (−|p| − p)n e− and
(|p|+p)2 2
.
AN US
g(x3 ) ≈ (|p| − p)n e−
(−|p|+p)2 2
Then we consider two subcases,
70
• If p is a large positive real number, then g(x3 ) is a small positive number and |g(x2 )| is large. In this subcase, we say that the VSMN has a degrade behavior at the right. For a large value of p, Figure 5 displays only one point and the second point tends to zero.
ED
200
M
65
150
AC
CE
PT
100
50
0 -40
-20
0
20
40
Figure 5: Only one point appear (for n = 2)
we give implementation with small values of p and q to illustrate the two critical points.
6
ACCEPTED MANUSCRIPT
2
1
-1 1 -2 0.5
-4 -5
-5 0 5 (a) n = 2, p = −0.4, q = 0.4, k = +1
0 5 (b) n = 3, p = −0.4, q = 0.4, , k = +1
AN US
0
-3
CR IP T
0
1.5
Figure 6: Stable behavior of the model(for n = 3)
75
M
• If p is negative and |p| is large, then |g(x2 )| is small and g(x3 ) is a large positive number. In this subcase, we say that the VSMN has a degrade behavior at the left.
CE
0.5
AC
0
3 2
PT
1.5
1
4
ED
2
-5 0 5 (a) n = 3, p = 0.4, q = −0.4, , k = +1
1 0 -1 -5
0 5 (b) n = 3, p = 0.4, q = −0.4, k = +1
Figure 7: Stable behavior of the model(for n = 3)
In this case we say that the corresponding VSMN has a left peak (resp., right peak) when p is large positive (resp., negative). This will affect the information (or the signature) caused by the neuron on the structure of
7
ACCEPTED MANUSCRIPT
the stimulator; when p is large, the neuron makes a hole in the stimulator’s data (see Section 7 ).
80
3. Multi Models of Neurons Generated by VSMN
85
3.1. First Model of Neuron (n = 0)
CR IP T
In this section, we generate four models from the variable structure model of neuron. The results are similar to the previous section.
+ (u + p)vf (βv)f (λ(u + p)) u˙ = −(u+p) τ v˙ = −αv + kαf 2 (λ(u + p))
(a) k = −1, n = 0
AN US
This model is illustrated graphically as follows.
(2)
(b) k = +1, n = 0
M
Figure 8: First model of VSMN for n = 0
3.2. Second model of neuron (n = 1)
+ (u + p)vf (βv)f (λ(u + p)) u˙ = −(u+p) τ v˙ = −αv + k(u + p)αf 2 (λ(u + p))
AC
CE
PT
ED
8
(3)
(a) k = −1, n = 1
AN US
CR IP T
ACCEPTED MANUSCRIPT
(b) k = +1, n = 1
M
Figure 9: Second model n = 1 with alternated behavior
ED
PT
90
3.3. Third model of neurons (n = 2) 3.3.1. Stable behavior This model is given by the following system: u˙ = −(u+p) + (u + p)vf (βv)f (λ(u + p)) τ v˙ = −αv + k(u + p)2 αf 2 (λ(u + p))
AC
CE
Figure 10 shows the implementation of this model .
(a) Positive of polarity k = −1
(b) Negative of polarity k = +1
Figure 10: Third model n = 2 with stable behavior
3.3.2. Degraded behavior This behavior occurs when q = −p. 9
(4)
ACCEPTED MANUSCRIPT
u˙ = −(u+p) + (u + p)vf (βv)f (λ(u + p)) τ v˙ = −αv + k(u − p)2 αf 2 (λ(u + p))
(5)
(b)
AN US
(a)
CR IP T
Figure 11 shows different sizes of small lobs.
(c)
M
Figure 11: Different forms of degraded behavior
ED
(a) The small one is on the left
(b) The small one is on the right
3.3.3. Hole behavior of neuron in low and high directions If p is a large positive real number, then the VSMN has a hole behavior at the right. The information of the stimulator of the neuron will be lost. Figure 13 shows two results of implementation (see section .
AC
CE
95
PT
Figure 12: Degraded behavior of neuron on the left/right direction
10
CR IP T
ACCEPTED MANUSCRIPT
(b) Hole behavior of neuron(down direction) (a) Hole behavior of neuron(up direction)
100
AN US
Figure 13: Hole behavior of neuron on the up/down direction
3.4. Fourth model of neuron (n = 3) In this case the model of neuron is described as follows: u˙ = −(u+p) + (u + p)vf (βv)f (λ(u + p)) τ v˙ = −αv − (u + p)3 αf 2 (λ(u + p))
(6)
M
Figure 14 displays results of implementation.
ED
(b)
(a)
PT
Figure 14: Model of neuron n = 3
4. Multi behaviors of a VSMN
CE
4.1. Oscillator behavior of a VSMN Oscillations and chaos are ubiquitous phenomena that are encountered in many different areas of physics [14]. And oscillatory phenomena have attracted much interest in biology, chemistry, and neuroscience. It seems that when there is a chaos there are a hidden oscillators and when there is an oscillator there are hidden chaos. It turns out that the relationship between chaos and oscillations behavior is the common factor of parameters depending of the mathematical model. In this section, we will show with VSMN can genarate different types of oscillations. The input is a small sinusoidal given by
AC
105
110
xi+2 = 2cos(2π 11
κ )xi+1 − xi N
ACCEPTED MANUSCRIPT
AN US
CR IP T
. q = −m × 0.01 and p = 0 The equation of the model with input signal takes this form: u˙ = −(u+p) + v(u + p)f (βv)f (λ(u + p)) + xi+2 τ (7) v˙ = −αv − m(u − q)2 αf 2 (λ(u + p))
(a)
Figure 15: Different forms of Oscillator behavior of neuron
The inputs are given by
M
xi+2 = 2cos(2π and
yi+2 = 2cos(2π
κ )yi+1 − yi N
ED
.
The equation of the model with inputs takes this form: κ xi+2 = 2cos(2π N )xi+1 − xi κ y = 2cos(2π )y i+2 i+1 − yi N u = αx 2 v = αy2 u˙ = −(u+p) + v(u + p)f (βv)f (λ(u + p)) + x2 τ v˙ = −αv − m(u − q)2 αf 2 (λ(u + p))
CE
PT
115
κ )xi+1 − xi N
(8)
AC
We take these values q = −m × 0.034 and p = 0, result of simulation in figure 16 It shows symmetric oscillator without contact between points.
12
Figure 16: Oscillator without contact
CR IP T
ACCEPTED MANUSCRIPT
AN US
The equation of the model with input signal takes this form: u˙ = −(u+p) + v(u + p)f (βv)f (λ(u + p)) + xi+2 τ v˙ = −αv − m(u − q)2 αf 2 (λ(u + p))
(9)
The input is a small sinusoidal given by xi+2 = 2cos(2π
. q = −m0.01 and p = 0 with value of m is switching between ±1 Figure 17 shows symmetric oscillator with one contact.
CE
PT
ED
M
120
κ )xi+1 − xi N
Figure 17: Oscillator with contact
AC
4.1.1. Oscillator with degraded behavior The input is a small sinusoidal given by xi+2 = 2cos(2 × π
κ )xi+1 − xi N
. q = 0.01 and p = 0 The equation of the model with input signal takes this form: u˙ = −(u+p) + v(u + p)f (βv)f (λ(u + p)) + xi+2 τ (10) v˙ = −αv − (u − q)2 αf 2 (λ(u + p)) 13
ACCEPTED MANUSCRIPT
The value of the initial condition is as follows: x1 = 0.1x0 = 0.2 u0 = 0.001, v0 = −3, p = 0 κ = 1 and N = 113 After implementation, we obtain the result in figure 18.
CR IP T
125
AN US
Figure 18: Oscillator generated by a VSMN
(b) The small one is on the right
ED
M
We modify a value of parameter q = 0.02, and then we take q = −0.02, we obtain graphs illustrate on figures 19(a)and 19(b) with degraded form.
(a) The small one is on the left
First we modify the value of q by q = −0.008, we obtain result on figure 20(a). then for q = 0.008 we obtain result on figure20(b).
AC
CE
130
PT
Figure 19: Different forms of the degraded oscillators
14
CR IP T
ACCEPTED MANUSCRIPT
(a) Degraded on the left direction
AN US
(b) Degraded on the right direction Figure 20: An other example of degraded oscillator
We notice that the value of q has degraded the symmetry of two circles. 4.2. Fractal behavior of a VSMN
M
Fractal behavior can be generated by different classes such as: complex iterated map, iterated function system geometry and Lindmayer system. In this section we show that fractal behavior coexist on neural network. We generate a fractal behavior by one case of our model of neuron. We give the model of neuron as follows: u˙ = −(u+p) + (u + p)vf (βv)f (λ(u + p)) + 0.5xi+2 τ (11) v˙ = −αv − (u − q)2 αf 2 (λ(u + p))
ED
135
PT
The input is a small sinusoidal given by xi+2 = 2cos(2π
κ )xi+1 − xi N
CE
. with q = 0
140
AC
Figure 21 displays fractal behavior of neuron. A same motif repeated in different scales.
Figure 21: Fractal behavior of neuron
15
ACCEPTED MANUSCRIPT
4.3. Harmonic behavior of a VSMN
150
Harmonic studies have become an important aspect of signal theory and power system analysis and design in recent years. Up to now, there isn’t work appeared to show the coexisting harmonic behavior on Hopfield neural network. In this section, we present a harmonic behavior of neuron, we give four examples of neuron models configured by a variable structure model of neuron. The first example, we take three values of parameters n, p and q,the model is described as follows: p = q = 0
CR IP T
145
+ (u + p)vf (βv)f (λ(u + p)) + 4xi+2 u˙ = −(u+p) τ v˙ = −αv − (u − q)2 αf 2 (λ(u + p))
The input is a small sinusoidal given by
AN US
κ )xi+1 − xi N
(12)
xi+2 = 2cos(2π
M
.
Figure 22: Harmonic of stable behavior on neuron
ED
CE
PT
155
In the second example, we modify the model of neuron, we use a model with degraded behavior, keeping the same values of inputs p = 0 and q = −0.008
Figure 23: Harmonic of degraded behavior of neuron
AC
In the third example, we take a model of hole behavior with these values p = 0 and q = 1
16
AN US
CR IP T
ACCEPTED MANUSCRIPT
Figure 24: Harmonic of hole behavior of neuron
ED
M
In the fourth example, we use first model of neuron. + (u + p)vf (βv)f (λ(u + p)) + 4xi+2 u˙ = −(u+p) τ v˙ = −αv − (u + q)αf 2 (λ(u + p))
AC
CE
PT
Figure 25, shows harmonic signal with alternated behavior.
17
(13)
AN US
CR IP T
ACCEPTED MANUSCRIPT
Figure 25: Harmonic alternated behavior of neuron
160
4.4. Chaotic Behavior of a VSMN
4.4.1. Chaos on stable behavior In this section, we study the dynamical behaviors of the model of variable structure by computer simulation. We begin from the case of n = 2, with which the model generates coexisting chaotic attractor. Figure 26 shows chaotic attractor. Take the model of VSMN as follows: + v(u + p)f (βv)f (λ(u + p)) u˙ = −(u+p) τ (14) v˙ = −αv − (u − q)2 αf 2 (λ(u + p))
CE
PT
165
ED
M
In this section, we observe that the chaotic behavior exists in all some models of VSMN. We observe chaotic attractor with two scrolls with the same size, two scrolls with degraded scrolls and with hole scrolls.
AC
Consider a small sinusoidal external input given by
170
xi+2 = 2 × cos(2 × π ×
κ ) × xi+1 − xi N
.
The equation of the model with external input signal takes this form: u˙ = −(u+p) + v(u + p)f (βv)f (λ(u + p)) + xi+2 τ (15) v˙ = −αv − (u − q)2 αf 2 (λ(u + p)) 18
ACCEPTED MANUSCRIPT
PT
ED
M
AN US
CR IP T
175
We set the parameters be τ = 1, α = 1.6, = 0.002, p = q = 0, κ = 5 and N = 1. Figure 26 displays the phase plot of the attractor generated by the initial values u0 = 0.01 and v0 = −0.5
(a)
CE
Figure 26: Chaotic attractor with the same size of the two scrolls
AC
4.4.2. Chaos on degraded behavior In this subsection, we present two chaotic behaviors: the first one contains degraded scroll on the right, and the second has degraded scroll on the left. When we modify the values of q by q = 0.002, it becomes two scrolls with different sizes, so degraded scroll on the right, see figure 27(a). Figure 27(b) displays two scrolls with one degraded behavior when the value q = −0.02.
180
19
AN US
CR IP T
ACCEPTED MANUSCRIPT
(a) Degraded scroll on chaotic attrac- (b) Degraded scroll on chaotic attractor on the right direction tor on the left direction
5. Symbol of a VSMN
M
Figure 27: Different forms of degraded behavior
NBDn
(p,q)
Figure 28: Graph of neuron
CE
PT
ED
In this section, we represent a VSMN by the following symbol:
The structure of graph contains four parts, D=k, B,n, p,q. the parameter of n indicates the model of neuron. Neuron has two potentials positive and negative activity directions which means all states of neuron have positive or negative values. The symbol of D is direction of potential of neuron. We present some graphs of neurons generated by the VSMN. B it means the behavior of neuron such as: stable, hole ,degraded , oscillator, harmonic, fractal and chaos. we use some symbols to indicate in which steady neuron is. for example :
AC
185
190
• A is in stable behavior. • H is on hole behavior. 20
ACCEPTED MANUSCRIPT
• D is on degraded behavior.
195
• F is on fractal behavior. • M is on harmonic behavior.
CR IP T
• O is on oscillator behavior. • C is on chaos behavior. 200
5.1. Symbol of first model of neuron
AN US
In this graph, we take n = 0 and p = q. This model is used as membership layer see [2].
NA+0
=⇒
(p=q,q)
(b) Behavior of neuron on negative polarity
(a) Symbol of first model of neuron
NA−0
=⇒
M
(p=q,q)
(c) Symbol of first model of neuron
(d) Behavior of neuron on positive polarity
AC
CE
PT
ED
Figure 29: Graph of first model of neuron and its behavior
21
ACCEPTED MANUSCRIPT
NA+1
CR IP T
5.2. Symbol of second model of neuron
=⇒
(p=q,q)
AN US
(a) Symbol of second model of neuron
M
(b) Behavior of neuron begin with positive polarity
(p=q,q)
=⇒
ED
NA−1
CE
PT
(c) Symbol of second model of neuron
(d) Behavior of neuron begin with negative polarity
Figure 30: Graph of second model of neuron and its behavior
AC
205
5.3. Symbol of third model of neuron In this case, the value of n is equal to 2. This model as we demonstrated in the previous section has multi forms of behavior, stable behavior, degraded behavior left or right with two position of polarity and hole behavior.
22
ACCEPTED MANUSCRIPT
5.3.1. Symbol of stable behavior
NA+2
=⇒
(p=q,q)
(a) Symbol of third model of neuron
NA−2
CR IP T
210
(b) Behavior of neuron on positive polarity
(c) Symbol of third model of neuron
AN US
=⇒
(p=q,q)
(d) Behavior of neuron on negative polarity
M
Figure 31: Graph of third model of neuron and its behavior
PT
ED
5.3.2. Symbol of hole behavior
+ NH 2
=⇒
CE
(p=−q,q)
AC
(a) Symbol of second model of neuron
(b) Hole behavior on positive polarity Figure 32: Graph of third model of neuron with hole behavior
23
ACCEPTED MANUSCRIPT
− NH 2
(a) Symbol of second model of neuron
CR IP T
=⇒
(p=−q,q)
(b) Hole behavior on negative polarity Figure 33: Graph of third model of neuron with hole behavior
AN US
5.3.3. Symbol of degraded behavior up direction
+ ND 2
=⇒
(p,q=−p+)
+ ND 2
(p,q=−p−)
ED
M
(a) Symbol of second model of neuron
(b) Small lob is on right and high direction
=⇒
(c) Symbol of second model of neuron
PT
(d) Small lob is on left and high direction
AC
CE
Figure 34: Graph of third model of neuron with degraded behavior
24
ACCEPTED MANUSCRIPT
− ND 2
=⇒
(p,q=−p−)
(a) Symbol of second model of neuron
CR IP T
5.3.4. Symbol of degraded behavior down direction
AN US
(b) Small lob is on left and low direction
− ND 2
=⇒
M
(p,q=−p+)
ED
(c) Symbol of second model of neuron
PT
(d) Small lob is on right and low direction
Figure 35: Degraded with negative polarity and with small lob on the right
CE
215
5.4. symbol of fourth of model
AC
In this case, the value of n is equal to 3.
25
ACCEPTED MANUSCRIPT
NA+3
CR IP T
=⇒
(p=q,q)
(a) Symbol of fourth model of neuron
(b) Alternate behavior
− ND 3
=⇒
(p=q,q)
AN US
(c) Symbol of fourth model of neuron
(d) Alternate with degraded behavior
Figure 36: Graph of fourth model of neuron and its behavior
6. Multi neural networks
M
6.1. System of neural networks Here, we present a structure of a system of neural network by associating multiple neurons in a cascading manner. This structure starts with a set of initial conditions, a number of neurons. Let E be the complete metric unit, Φ a system of neural network in E such as:
PT
225
In this section, we show some examples of neural network with the previous models.
ED
220
E →E Φ:(ui , vi ) → (um , vm )
The equation system of neural network Φ is represented by: (u0 , v0 ) (ui+1 , vi+1 ) = N1 (ui , vi ) (u i+2 , vi+2 ) = N2 (ui+1 , vi+1 ) .. . Φ (uj+2 , vj+2 ) = Nj (uj+1 , vj+1 ) . .. (um , vm ) = Nm (um−1 , vm−1 )
CE AC
230
(16)
The dynamics of the neural network system is controlled by the assignment of ui+1 to ui and of vi+1 to vi . 26
ACCEPTED MANUSCRIPT
6.2. Topology of neural network 235
In this paragraph, we present two designs of neural networks the first topology with full connected. In the second topology, we give a design with a simple connection in a cascade.
NBDn
NBDn
D n NB
j1 ,(p,q)
ED
.. .
D n NB
M
i1 ,(p,q)
j0 ,(p,q)
AN US
i0 ,(p,q)
CR IP T
Example 6.2.1. Figure below shows a neural network with two layers, in each layer, we have k neurons.
D n NB
.. .
D n NB
jk ,(p,q)
AC
CE
PT
ik ,(p,q)
240
P
Example 6.2.2. Figure below shows a neural network with two layers, in each layer, we have k neurons, all neurons on each layer are connected in a cascade.
27
ACCEPTED MANUSCRIPT
NBDn
NBDn
i0 ,(p,q)
D n NB
D n NB
.. .
D n NB
i1 ,(p,q)
AN US
j1 (p,q)
CR IP T
j0 (p,q)
.. .
D n NB
ik ,(p,q)
PT
ED
M
jk (p,q)
P
6.3. Neural network with the third model of neuron (n=2) on positive polarity We built a neural network with five neurons generates by VSMN. In this network, for each neuron we use the model n = 2 and positive polarization (k = +1).
CE
245
AC
NA+2
(p=q,q=α1 )
NA+2
(p=q,q=α2 )
NA+2
(p=q,q=α3 )
NA+2
(p=q,q=α4 )
Figure 37: Neural Network with five neurons each neuron has its position of threshold
28
NA+2
(p=q,q=α5 )
ACCEPTED MANUSCRIPT
as:
Let E be the complete metric unit, Φ a system of neural network in E such E →E Φ:(ui , vi ) → (um , vm )
CR IP T
250
(17)
AN US
The equation system of neural network Φ is represented by: 1) + f (βv)fs (λ(u + p1 )) u˙1 = −(u+p NP1 τ 2 v ˙ = −αv + αf 1 s (λ(u + p1 )) −( u ˙ +p ) 1 2 u˙2 = + v˙1 (u˙1 + p2 )fb (βv1 )f (λ(u˙1 + p2 )) τ NP2 v ˙ = −α v ˙ + ( u ˙1 + p2 )2 αf 2 (λ(u˙1 + p2 )) 2 1 −(u˙2 +p3 ) + v˙2 (u˙2 + p3 )fb (βv2 )f (λ(u˙2 + p3 )) u˙3 = τ NP3 v ˙ = −α v ˙ + ( u ˙2 + p3 )2 αf 2 (λ(u˙2 + p3 )) 2 3 −( u ˙ +p ) 3 4 u˙4 = + v˙3 (u˙3 + p4 )fb (βv3 )f (λ(u˙3 + p4 )) τ NP4 v ˙ = −α v ˙ + ( u ˙3 + p4 )2 αf 2 (λ(u˙3 + p4 )) 3 4 −(u˙4 +p5 ) + v˙4 (u˙4 + p5 )fb (βv4 )f (λ(u˙4 + p5 )) u˙5 = τ NP5 v˙5 = −αv˙4 + (u˙4 + p5 )2 αf 2 (λ(u˙4 + p5 ))
M
The result is generated by five neurons, depicted in figure 38.
ED
Figure 38: Network with five neurons
6.4. Neural network with the third model of neuron on negative polarity
PT
We give some architectures of neural networks, all neurons are on negative polarization (k = −1).
CE
255
NA−2
AC
(p=q,q=α1 )
NA−2
(p=q,q=α2 )
NA−2
(p=q,q=α3 )
NA−2
(p=q,q=α4 )
Figure 39: Neural Network with five neurons each neuron has its position of threshold
Figure 40 displays curve which has symmetry the previous neural network.
29
NA−2
(p=q,q=α5 )
Figure 40: Network with five neurons
6.5. Neural network with bi potentials
NA−2
NA−2
(p=q,q=α1 )
NA+2
(p=q,q=α1 )
AN US
(p=q,q=α1 )
CR IP T
ACCEPTED MANUSCRIPT
Figure 41: Neuron with positive potential and position of threshold = α
Let E be the complete metric unit, Φ the equation system of neuronal network with chaotic attractor takes as follows:
1) u˙1 = −(u+p + f (βv)fs (λ(u + p1 )) τ v˙1 = −αv + αfs2 (λ(u + p1 )) u˙2 = −(u˙1τ+p2 ) + v˙1 (u˙1 + p2 )fb (βv1 )f (λ(u˙1 + p2 )) v˙2 = −αv˙1 + (u˙1 + p2 )2 αfb2 (λ(u˙1 + p2 )) u˙3 = −(u˙2τ+p3 ) + v˙2 (u˙2 + p3 )fb (βv2 )f (λ(u˙2 + p3 )) v˙3 = −αv˙2 − (u˙2 + p3 )2 αfb2 (λ(u˙2 + p3 ))
AC
CE
PT
ED
N P 1 NP2 N P3
M
E →E Φ:(u, v) → (u3 , v3 )
260
Figure 42: Network with three neurons
30
(18)
ACCEPTED MANUSCRIPT
6.6. Neural network with Hole behavior The equation system of neural network is represented by:
NA+2
(p=q,q=α1 )
+ NH 2
(p=q,q=α2 )
(p=q,q=α3 )
NA−2
CR IP T
− NH 2
(p=q,q=α4 )
Figure 43: Neural Network with alternated behaviors
265
Let E be the complete metric unit, Φ the equation system of neuronal network with chaotic attractor takes as follows:
M
1) + f (βv)fs (λ(u + p1 )) u˙1 = −(u+p τ v˙1 = −αv + αfs2 (λ(u + p1 )) u˙2 = −(u˙1τ+p2 ) + v˙1 (u˙1 + p2 )fb (βv1 )f (λ(u˙1 + p2 )) v˙2 = −αv˙1 + (u˙1 + p2 )2 αfb2 (λ(u˙1 + p2 )) u˙3 = −(u˙2τ+p3 ) + v˙2 (u˙2 + p3 )fb (βv2 )f (λ(u˙2 + p3 )) v˙3 = −αv˙2 + (u˙2 + p3 )2 αfb2 (λ(u˙2 + p3 )) u˙4 = −(u˙3τ+p4 ) + v˙3 (u˙3 + p4 )fb (βv3 )f (λ(u˙3 + p4 )) v˙4 = −αv˙3 + (u˙3 + p4 )2 αfb2 (λ(u˙3 + p4 ))
AC
CE
PT
ED
N P 1 N P2 N P3 NP4
AN US
E →E Φ:(u, v) → (u4 , v4 )
Figure 44: Network with four neurons
We give another result of neural network. 31
(19)
Figure 45: Network with five neurons
7. Application in chaos
275
In this section, we introduce various examples of neural networks designed by different models of neurons where each one of them prints its signatures on orbits chaotic attractor witch pass through position of threshold. The traces of neurons change the behavior of scrolls or orbits of chaotic attractor. Let E be the complete metric unit, Φ the equation system of neuronal network with chaotic attractor takes as follows:
AN US
270
CR IP T
ACCEPTED MANUSCRIPT
E →E Φ:(u, v) → (um , vm )
M
of neuronal network with chaotic attractor takes this x˙ 1 = f1 (x1 , x2 , x3 ) x˙ 2 = f2 (x1 , x2 , x3 ) x˙ 3 = f3 (x1 , x2 , x3 ) (u1 , v1 ) = N1 (x1 , x2 ) .. .
CE
PT
ED
The equation system form:
280
(ui+2 , vi+2 ) = N2 (ui+1 , vi+1 ) .. .
(20)
(uj+2 , vj+2 ) = Nj (uj+1 , vj+1 ) .. . (um , vm ) = Nm (um−1 , vm−1 )
7.1. Trace of neuron in Chua attractor in two lines of orbits
AC
In this subsection, we show the implementation results of a combination between Chua’s chaotic attractor and neuron. Let E be the complete metric unit, Φ the equation system of neuronal network with chaotic attractor takes as follows:
285
E →E Φ:(x1 , x2 ) → (u1 , v1 )
32
ACCEPTED MANUSCRIPT
(21)
CR IP T
x˙ 1 = f1 (x1 , x2 , x3 ) x˙ 2 = f2 (x1 , x2 , x3 ) x˙ 3 = f3 (x1 , x2 , x3 ) u˙1 = −(x˙ 1τ+p1 ) + f (β x˙ 2 )fs (λ(x˙ 1 + p1 )) v˙1 = −αx˙ 2 + αfs2 (λ(x˙ 1 + p1 ))
AN US
Figure 46: Track of neuron in Chua attractor
7.2. Trace of neuron in Chua attractor in all lines of orbits
Figure: Fig: 47(b) Fig: 47(c) Fig: 47(d) Fig: 47(e) Fig: 47(f) Fig: 47(g)
AC
CE
PT
ED
Model of neuron Degraded model Stable model Alternated model Fourth model First model Hole model
M
290
In this subsection, we stimulate Chua attractor with some models of neurons configured by variable structure model of neuron 13. In figures 47(b),47(c),47(d),47(e) and 47(g) we use the same position of threshold, in figure 47(f) we modify the position of the threshold.
33
(c) Track of neuron third model of neuron
AN US
(a) Chua attractor used for stimulation
(b) Track contains degraded behavior of neuron
CR IP T
ACCEPTED MANUSCRIPT
(e) Track of fourth model
(f) Track of first model
PT
ED
M
(d) Track of second model
(h) Zooming of hole behavior
Figure 47: Signature of neural network on its stimulate
AC
CE
(g) Track of neuron contains hole behavior
295
In the next example, we show that signatures of neurons are inserted in all lines of orbits. We give four neurons closed with four different positions of thresholds. In figure 48, it can be obviously seen the trace of neural network in Chua attractor. For all neurons we use the same model as follows:
34
ACCEPTED MANUSCRIPT
(22)
and we take the following value of neurons: p1 = −0.5, p2 = 0.5, p3 = −0.75 and on fourth neuron the value of threshold position is p4 = 0.75. Figure 48 shows scrolls with multi signatures of neurons.
M
AN US
CR IP T
300
u˙ = −(u+p) + v(u + p)f (βv)f (λ(u + p)) τ v˙ = −αv − (u + q)2 αf 2 (λ(u + p))
ED
Figure 48: Signature of neural network on its stimulate
The neuron makes signature in all lines of orbits when it passes by the position of threshold. 7.3. Trace of neurons in Chua attractor with different behaviors
PT
305
CE
We give example of neural network with Chua attractor. In this example, we stimulate Chua attractor by five neurons with different models of neurons and positions of threshold. The equation system of neural network is described by: Let E be the complete metric unit, Φ the equation system of neuronal network with chaotic attractor takes as follows:
AC
310
E →E Φ:(x1 , x2 ) → (u5 , v5 )
35
ACCEPTED MANUSCRIPT
= f1 (x1 , x2 , x3 ) = f2 (x1 , x2 , x3 ) =f3 (x1 , x2 , x3 ) u˙1 = −(x˙ 1τ+p1 ) + f (β x˙ 2 )f (λ(x˙ 1 + p1 )) NP1 x˙ 2 + α(x˙ 1 + p1 )f 2 (λ(x˙ 1 + p1 )) v˙1 = −α −(u˙1 +p2 ) + v˙1 (u˙1 + p2 )f (βv1 )f (λ(u˙1 + p2 )) u˙2 = τ NP2 2 v ˙ = −α v ˙ + αf 1 b (λ(u˙1 + p2 )) 2 −(u˙2 +p3 ) u˙3 = + v˙2 (u˙2 + p3 )f (βv2 )f (λ(u˙2 + p3 )) τ NP3 v˙3 = −αv˙2 + (u˙2 − p3 )2 αf 2 (λ(u˙2 + p3 )) u˙4 = −(u˙3τ+p4 ) + v˙3 (u˙3 + p4 )f (βv3 )f (λ(u˙3 + p4 )) NP4 v˙3 + (u˙3 + p4 )3 αf ( λ(u˙3 + p4 )) v˙4 = −α −(u˙4 +p5 ) + v˙4 (u˙4 + p5 )f (βv4 )f (λ(u˙4 + p5 )) u˙5 = τ NP5 v˙5 = −αv˙4 + (u˙4 + p5 )2 αf 2 (λ(u˙4 + p5 ))
CR IP T
x˙ 1 x˙ 2 x˙ 3
(23)
AN US
M
Figure 49 shows results of implementation.
7.4. Signature of neurons with degraded behaviors Let E be the complete metric unit, Φ a system of neural network in E such as: E →E Φ:(x1 , x2 ) → (u3 , v3 )
CE
PT
315
ED
Figure 49: Traces of five neurons in Chua attractor
AC
320
The equation system of neural network Φ is represented by: x˙ 1 = f1 (x1 , x2 , x3 ) x˙ 2 = f2 (x1 , x2 , x3 ) x˙ 3 = f3 (x1 , x2 , x3 ) (u1 , v1 ) = Pj (x˙ 1 , x˙ 2 ) (u2 , v2 ) = Pj (u1 , v1 )
(24)
if
(u2 < 0)(p = p1 )
u˙3 = −(u2τ+p1 ) + f (βv2 )f (λ(u2 + p1 )) v˙3 = −αv2 + α(u2 + p1 )f 2 (λ(u2 + p1 )) 36
(25)
ACCEPTED MANUSCRIPT
if (u2 > 0)(p = p2 ) (26)
Figure 50(a) illustrates a chaotic attractor with four scrolls, this result is output of response by stimulation of neuron. In the middle of chaotic behavior, we notice that neuron makes its trace by the same form of signature. Figure 50(b) shows response of stimulation with two different directions of traces. Figure 50(c), Figure 50(d) and Figure 50(e) display different cases of signatures.
AC
CE
PT
ED
M
AN US
325
u˙3 = −(u2τ+p2 ) + f (βv2 )f (λ(u2 + p2 )) v˙3 = −αv2 + α(u2 + p2 )f 2 (λ(u2 + p2 ))
CR IP T
37
CE
PT
ED
M
AN US
CR IP T
ACCEPTED MANUSCRIPT
AC
Figure 50: Different signatures of neurons in the same position of axis
330
7.5. Oscillator from one neuron in chaotic attractor In this example, both figures 51(a) and 51(b) contain oscillator. Each of them exhibits two forms of signatures from two model of neurons. And each one has its threshold position. We take the first neuron is as follows: u˙ = −(u+p) + v(u + p)f (βv)f (λ(u + p)) τ (27) v˙ = −αv − (u + q)2 αf 2 (λ(u + p)) 38
ACCEPTED MANUSCRIPT
AN US
With threshold position p = 2 and q = ±α.
(28)
CR IP T
with threshold position p = q = +2. And the second is given by: u˙ = −(u+p) + v(u + p)f (βv)f (λ(u + p)) τ v˙ = −αv − (u + q)2 αf 2 (λ(u + p))
(a) Nested oscillator and signature of neuron
(b) Oscillator with an opposition of phase
335
M
Figure 51: One oscillator in a chaotic attractor
7.6. Oscillators of two neurons in a chaotic attractor
AC
CE
PT
ED
Result in figure 52 shows traces of neurons which have two oscillators, one with degraded form and the other with stable one.
Figure 52: Two oscillators with two behaviors
Although figure53 shows an exact two signatures alternated with two oscillators, all are between scrolls.
39
CR IP T
ACCEPTED MANUSCRIPT
Figure 53: Two oscillators with the same behavior contain two scales of sizes
7.7. Multi scales of signatures in a chaotic attractor from neural networks
AN US
340
In this example, we use four neurons with different threshold positions of the same model as follows: i) u˙ = −(u+p + v(u + pi )f (βv)f (λ(u + pi )) τ (29) v˙ = −αv − (u + qi )2 αf 2 (λ(u + pi ))
PT
ED
M
The values of position threshold are p1 = q1 = +2, p2 = q2 = +1, p3 = q3 = +0, p4 = q4 = −1. Figure 54 shows signatures of four neurons on a chaotic attractor.
CE
Figure 54: Four signatures on two orbits of chaotic attractor
The order of threshold position of neuron has a great effect on the behavior of its stimulate. In this case the values of position threshold are p = q = −1,p = q = 0,p = q = +1, p = q = 2. Figure 55 shows signatures of four neurons on a chaotic attractor in opposite direction refereed to the previous result.
AC
345
40
CR IP T
ACCEPTED MANUSCRIPT
Figure 55: Direction of signatures change from left to right
In this example, we use four neurons with the same model as follows: 1. First neuron: u˙ = −(u+p) + v(u + p)f (βv)f (λ(u + p)) τ (30) v˙ = −αv − (u + q)2 αf 2 (λ(u + p))
AN US
350
With the threshold position p = q = +2 2. Second neuron: u˙ = −(u+p) + v(u + p)f (βv)f (λ(u + p)) τ v˙ = −αv + (u + q)2 αf 2 (λ(u + p))
M
With the threshold position p = q = +1 3. Third neuron: u˙ = −(u+p) + v(u + p)f (βv)f (λ(u + p)) τ v˙ = −αv − (u + q)2 αf 2 (λ(u + p))
ED
355
PT
With the threshold position p = q = +0 4. Fourth neuron u˙ = −(u+p) + v(u + p)f (βv)f (λ(u + p)) τ v˙ = −αv + (u + q)2 αf 2 (λ(u + p)) With threshold position is p = q = −1
AC
CE
Figure 56 shows result of traces on a chaotic attractor.
Figure 56: Four traces of neurons on a chaotic attractor
41
(31)
(32)
(33)
ACCEPTED MANUSCRIPT
360
8. Application in robotics 8.1. First example In this subsection, we give two sinusoids
CR IP T
x2 = 2 × cos(2.3.14.223/27) × x1 − x0 y2 = 2 × cos(2.3.14.223/27) × y1 − y0
u = αy2 and v = αx2 ; u˙ = −(u+p) + v(u + p)f (βv)f (λ(u + p)) τ v˙ = −αv + (u + q)2 αf 2 (λ(u + p))
With two threshold positions of neurons, the first neuron has the value p = −1 and the second p = −3 Figure 57(a) and Figure 57(b) show results of first example.
AN US
365
(34)
M
(a) Signature of one neuron in a (b) Traces of two neurons in a trajectory of robot trajectory of robot
ED
Figure 57: Robot path with signals of neurons
CE
PT
In the next example, we show a result of robot path with two signatures of two neurons, on the first neuron we use the position of threshold p1 = −1 and the second neuron p2 = +0 Model of first neuron is as follows: u˙ = −(u+p) + v(u + p)f (βv)f (λ(u + p)) τ (35) v˙ = −αv + (u + q)2 αf 2 (λ(u + p)) the second model is:
AC
370
u˙ = −(u+p) + v(u + p)f (βv)f (λ(u + p)) τ v˙ = −αv + (u + q)3 αf 2 (λ(u + p))
Figure 58 two different behavior of signatures in a trajectory of robot
42
(36)
(a) Traces of two models of neurons in trajectory of robot
CR IP T
ACCEPTED MANUSCRIPT
Figure 58: Robot path with signals of neurons
M
AN US
375
8.2. Second example We give a second example of robot path generated by neural networks. Figure 59(a)shows robot path without signature of neuron. Figure 59(b) displays path with trace of neuron. This trace can be used for avoid obstacle on robot path.
ED
(a) Robot path without signature of neuron (b) Robot path with signature of neuron Figure 59: Robot path with signal of neurons
This paper is an attempt to solve topological problems involved in relationship between chaos, fractal and neuroscience. By combining chaos and the Hopfield’s type of neural network, we have finally found out some main findings. First, we have developed a variable structure model of neuron. From this structure we generate four models, For each one of these models, its behavior can take as follows: Stable or equilibrium, degraded, hole, alternated, oscillator, fractal, harmonic, chaos behaviors. In addition, we have also elaborate a graph of these neurons and we design an architecture of neural networks, then we gave examples of topology with simulation. finally we applied our approach in significant application such as: chaos and robotic. It seems that, when neuron in oscillator behavior, that looks like Parkinson’s disease. When the neuron takes the hole behavior it gives an idea about Alzheimer disease behavior. I think that, there are some potential research directions that could be considered for the future works.
AC
CE
380
PT
9. Conclusion
385
390
43
ACCEPTED MANUSCRIPT
References
395
[1] Amitava, Kundu., Pritha, Das., A.B,Roy. ”Complex dynamics of a four neuron network model havinga pair of short-cut connections with multiple delays,” Int. J. Nonlinear Dyn 72, pp.643 − 662(2013).
405
[3] Cquan,Yuan. & Xiao-Song,Yang.”computer assisted verification of chaos in three-neuron cellular neural networks,” Int. J. Bifurcation and Chaos 17, 12, pp.4381 − 4386(2007).
[4] Chunguang, Li.& Guanrong, Chen. ”Coexisting chaotic attractors in a single neuron model with adapting feedback synapse,” Chaos, Solitons and Fractals 23, pp.1599 − 1604(2005).
AN US
400
CR IP T
[2] A. Mohammadzadeh, S. Ghaemi. A modified sliding mode approach for synchronization of fractional-order chaotic/hyper chaotic systems by using new self-structuring hierarchical type-2fuzzy neural network. Int. J. Neurocomputing (2016), http://dx.doi.org/10.1016/j.neucom.2015.12.098
[5] G. Lee and N.H. Farhat. ”The Bifurcating Neuron Network,” Neural Networks14, pp.115 − 131(2001).
M
410
[6] J.J. Hopfield. ” Neurons with graded response have collective computational properties like those of two-state neurons,” Proc. Natl. Acad. Sci. USA 81, 17, pp.3088 − 3092(1984).
415
ED
[7] Jianquan, Lu. Daniel W. C. HO& Jinde, CAO. ”Synchronization in an array of nonlinearly coupled chaotic neural networks with delay coupling ,” Int. J. Bifurcation and Chaos 18, 10pp.3101 − 3111(2008). [8] Juan Meng & Xing-yuan Wang. ”Robust anti-synchronization of a class of delayed chaotic neural networks,” Chaos, 17, 023113 − 7(2007).
PT
[9] Kreangkri, Ratchagit. ”Asymptotic stability of delay-difference system of Hopfield neural networks via matrix inequalities and application,” Int. J. Neural Systems 17, 5, pp.425 − 430(2007). [10] K. Bouallegue. ”Gallery of Chaotic Attractors Generated by Fractal Network,”Int. J. Bifurcation and Chaos 25, pp.1530002 − 18(2015).
CE 420
AC
[11] K. Bouallegue.”A new fractal model of chromosome and DNA processes ,” Chapter, Book. Proc. The 4th International Interdisciplinary Chaos Symposium Chaos and Complex Systems.pp.505 − 514(2012).
425
[12] Lenze.,J¨ org, Raddatz. ”Effects of dilation and translation on a perceptiontype learning rule for higer order Hopfield neural networks,” Int. J. Neural Systems12, 2, pp.83 − 93(2002).
[13] Marc Timme, Theo Geisel, and Fred Wolf. ”Speed of synchronization in complex networks of neural oscillators: Analytic results based on Random Matrix Theory”Chaos 16, 015108(2006) 44
ACCEPTED MANUSCRIPT
430
[14] M. Stork,”Energy feedback used for oscillators control,” Nonlinear Dyn 85 : 871 − 879(2016)
435
440
CR IP T
[15] Nimet,Korkmaz. & Recai, Kilic. ”Implementations of Modified Chaotic Neural Models with Analog Reconfigurable Hardware,” Int. J. Bifurcation and Chaos 2, 4, pp.11450046 − 1450061[2014]. [16] Qingdu, Lia. Xiao-Song,& Yangb. Fangyan, Yanga.” Hyperchaos in Hopfield-type neural networks,” Int. J. Neurocomputing 67, pp.275 − 280(2005). [17] Teijiro, Isokawa., Haruhiko, Nishimura., Naotake,Kamiura and Nobuyuki, Matsui. ”Associative memory in quaternionic Hopfield neural network,” Int. J. Neural Systems 18, 2, pp.135 − 145(2008).
445
AN US
[18] Shyan-Shiou Chen and Chih-Wen Shih.”Transversal homoclinic orbits in a transiently chaotic neural network”Chaos, 12, 3, (2002).
[19] RezaMazrooei-Sebdani, SaeedFarjami. ”On adiscrete-time-delayed Hopfield neural network with ring structures and different internal decays:Bifurcations analysis and chaotic behavior,” Int. J. Neurocomputing151, pp.188 − 195(2015).
[21] Wei-Yen, Hsu. ”Application of competitive Hopfield neural network to brain-computer interface systems,” Int. J. Neural Systems 22, 1, pp.51 − 62(2012).
AC
CE
PT
ED
450
M
[20] Yuanlong, Chen.,Tingwen, Huang. & Yu, Huang. ”Complex dynamics of a delayed discrete neural network of two nonidentical neurons,” AIP Chaos 24, pp.013108(1999).
45
AN US
CR IP T
ACCEPTED MANUSCRIPT
M
AC
CE
PT
460
ED
455
Kais Bouallegue is Assistant Professor in the Higher Institute of applied Sciences and Technology of Sousse, Tunisia. He got his PhD from National Engineering School of Sfax. He is active member in different industrial companies. He has served a reviewer for technical papers. Also he has mentored researchers at undergraduate and graduate levels. Moreover, He has been collaborating with many different international universities. His currently research interests include fractal, chaos, genomic and complex system
46