- Email: [email protected]
Non-linear dynamics in adaptive control
0005-1098/88 S3.00+ 0.00 Pergamon Presspie © 1988InternationalFederationof AutomaticControl
Automatica, Vol. 24, No. 4, pp. 485-497, 1988 Printed in Great Britain.
Non-linear Dynamics in Adaptive Control: Periodic and Chaotic Stabilization II. Analysis* IVEN M. Y. M A R E E L S t and R O B E R T R. BITMEAD$
A mathematical analysis of a simple adaptive control scheme in the presence of unmodeUed dynamics displays complex dynamical phenomena and indicates possibilities and limitations of robust adaptive control. Key Words--Adaptive control; non-linear systems; chaotic dynamics; robust control.
tions (Riedle and Kokotovic, 1986; Anderson et al., 1986). The importance of the results of Marcels and Bitmead (1986) was that they demonstrated two main features: firstly, that the non-linear dynamics of adaptive control systems could be manifested in their behaviour with the occurrence of complex periodic and chaotic motions; and, secondly, that these dynamics had the capacity for delivering very robustly globally stabilizing feedback control for a large class of plants---the motive aim of adaptive control. Our aim in this paper is to develop more fully the theoretical justification and proofs for the dynamic behaviour observed in Marcels and Bitmead (1986), and, in so doing, to highlight the extraordinary difficulty and perversity of carrying out the complete global analysis of a very simple robust adaptive control problem. In particular, we believe that this latter aim serves the useful purpose of delineating realistic expectations for analyses of adaptive controllers, all the more so because the large initial conditions transient behaviour of some more standard adaptive control procedures was shown in Marcels and Bitmead (1986) to duplicate these dynamics. The message conveyed here by this simplest of non-trivial robust adaptive control problems is that more general adaptive controllers will exhibit dynamics of at least this level of complexity. While this message appears to augur poorly for the prospects of adaptive controller analysis, it does help to focus investigations onto quantitatively meaningful problems and to raise questions, for example, about the efficacy of seeking both to produce global results and simultaneously to constrain the transient dynamics of adaptation. The inherent value of results on unqualified boundedness or asymptotic
Abstract--A particular non-linear equation arising in an adaptive control problem is presented in the first part of this paper. This equation displays vividly a wealth of non-linear phenomena. Depending on the value of one crucial parameter---characterizing the extent and presence of "non-design" circumstances--we demonstrate the existence of asymptotically periodic behaviour or chaotic dynamics in the feedback controller. This paper contains the mathematical analysis associated with the earlier results and, as such, serves two main purposes: to provide the theoretical proofs and support for these results, and to demonstrate more fully the extraordinary complexity and difficulty of conducting a thorough global analysis of even this simplest of robust adaptive problems. By performing this latter task we aim to delineate better what are reasonable objectives to be expected of robust adaptive control theory.
1. INTRODUCTION
IN THE FIRST part of this work (Marcels and Bitmead, 1986) an adaptive control problem was studied which exhibited very non-linear behaviour. This example was chosen to accentuate this non-linearity by incorporating fast adaptation, plant undermodelling and a severe controller objective. Classically one tends to think about the behaviour of an adaptive system as being asymptotically linear by relying upon parameter convergence (Goodwin and Sin, 1984) or local linearized manifold dynamics descrip*Received 26 May 1987; revised 6 November 1987; received in final form 13 December 1988. The original version of this paper was not presented at any IFAC meeting. This paper was recommended for publication in revised form by Associate Editor G. Kreisselmeier under the direction of Editor P. C. Parks. 5"Research Assistant with the National Fund for Scientific Research, Belgium, whose support is acknowledged. Department of Systems Engineering, University of Gent, Grote Steenweg Noord 2, B-9700, Gent (Zwijnaarde), Belgium. Current address: Department of Electrical and Computer Engineering, University of Newcastle, NSW 2308, Australia. ~:Department of Systems Engineering, Research School of Physical Sciences, Australian National University, GPO Box 4, Canberra, ACT 2601, Australia. Atrr 24-4-o
485
486
I.M.Y.
MAREELS and R. R. BITMEAD
optimality is brought into question. In our analysis, it is the control questions which are of prime concern. However, the techniques applied may be of interest in the broader study of non-linear dynamical systems as the multiplicative ergodic theorems are novel in their application to stability questions. The outline of the paper is as follows. In Section 2 we briefly derive the principal equation of interest from its adaptive control context. This equation's dynamics depend on a single parameter, b, representing the degree of plant undermodelling. Section 3 briefly touches upon the case of exact modelling, b = 0, when the adaptive controller performs "ideally". Section 4 treats the negative b case which produces globally attractive asymptotically two-periodic dynamics in the controller gain. Lyapunov techniques may be used here. Section 5 is devoted to the treatment of the positive b case where chaotic dynamics are generated. The analytical tools applied in this latter section incorporate both theoretical developments and numerical evidence to support theorems and conjectures concerning the global dynamics. Section 6 concludes. In presenting this analysis we are not only demonstrating some of the interesting and varied dynamical properties of adaptive controllers and the features of their underlying mathematical description but also providing an object lesson in the practical difficulties of a complete and thorough analysis of both transient and steadystate behaviour in adaptive control. Some conventions
We adopt the following notation. The first to fourth open quadrants of R 2 are denoted as al-a4. The closure of a set S in N2 will be denoted by S, its complement in R 2 by S c, its boundary by 6 ( S ) and its interior by int (S). 2. PROBLEM ORIGIN
The specific difference equation which is the main object of study of this paper is: rk=b(r 1-
1 )
1 rk--Z
keN;
ben
(1)
whose derivation will be briefly described shortly. This non-linear autonomous difference equation has a state in R 2 [given fo, ft the whole trajectory {rk, k e rN} with ro = ro and h = rl is uniquely determined by (1)]. In order to illustrate our results we often present phase plane pictures of the trajectories. Actually because the (numerical) dynamic range of the trajectories is too large we squeeze R 2 into the square ( - 1 , 1) x ( - 1 , 1) by the homeo-
morphism: h :l~2--~ (-1, I)x (-I, 1 ) : ( x , y ) ~ h ( x , y )
(x:,). l+lxl'l
h(x,y)=
Yl '
(2)
this preserves all topological features. Obviously in order to have the trajectories well defined for all k, rk should be non-zero (and bounded) for all k. Only these trajectories are relevant for the underlying adaptive scheme. Therefore, instead of extending the definition of the difference operator in order to make "division by zero" meaningful in some sense, we exclude from the phase plane all initial conditions for which there is some k such that rk = 0. The collection of these "pre-images of zero" is the one-dimensional set:
G = { (x), 3k :rk(x, y) = O}. Y
We study the difference equation (1) on the complement of this set in R 2. On this set the difference operator corresponding to (1) is a diffeomorphism. Equation (1) is derived in Mareels and Bitmead (1986) via the interconnection of a second-order, linear, time-invariant plant system Yk = ayk-1 +
byk-2 + Uk-t,
a, b e ~
(3)
with an adaptive controller based upon a first order model Yk = t~yk-1 + Uk-1,
~ e ~.
(4)
The adaptive controller consists of a deadbeat identifier and a deadbeat feedback control law gZk+l = ak + 1/y~(Yk+l -- akyk -- Uk),
Yk :/: 0
~k.l = ak, Uk= --akyk.
Yk= 0
(5) (6)
Connecting (3), (5) and (6) and introducing rk = Yk/Yk-1
(7)
yields (1). Note that rk relates to the feedback gain and the plant output via Yk = rkyk-1 = rkrk-~ " " " r o y - i ak = a -- b / r k - 1 .
(8) (9)
The original closed loop Yk system has a state vector in N3; but because of the system's specific structure it is sufficient to study (1) (with a state in Nz) in order to capture the generic dynamics of the adaptive system. It is not too difficult to analyse the behaviour of the trajectories for which a "division by zero" could occur-however, this does not add anything fundamental to our knowledge of the system, and
Non-linear dynamics in adaptive control moreover this only gives us information about a two-dimensional subset of initial conditions in the state space of the original system. Therefore we do not pursue this here. In the following sections we study (1) in detail. By way of preliminary analysis we note that as a function of the parameter b we can at most distinguish three topologically different types of dynamical behaviour for (1). Indeed, rescaling as vk = r k l V ~ l ,
b ~0
(10)
487
into two fixed points: k l"Ik
Wk=(--1) ~7~,
keN;
(13)
the equation governing wk becomes: Wk =
1) ,
+ 141k-1
(14)
k e~.
Wk- 2
Introduce the following state space representation; define the state as:
leads to
(1
Vk=sign(b) v_x
1)
vZ-2 "
,11,
Xk=(
Wk ],
(15)
k•N
\Wk-1/
)
define the state transition map F as:
F(Y'/:/1(--'+'
So it is possible to consider three situations bO.
In the sequel we discuss all three types of dynamical behaviour. 3. T H E C L O S E D L O O P D Y N A M I C S I; b = 0
In this ideal situation, the dynamics are quite obvious, as one could expect from the combination of a deadbeat identifier and deadbeat control law. The closed loop reduces to: Plant: Yk ayk-I + Uk-l, Identifier: ak = a, Control law: u~-l = --ayk-1, Hence: Yk -~ O, =
Vk (Y-1 ~ 0), Vk Vk Vk
• N >10 >- 1 >! 2.
In the situation that Y-1 = 0, the identifier does not identify anything useful, but there is also no need to identify anything, as in this case Yk - 0, Vk • N and ak = a - l , Vk e N; hence the control objective is met. In this situation the equation (1) governing the ratio of successive outputs is redundant, except to highlight a definitional problem with the deadbeat identifier when the deadbeat controller actually works. (Notice however, that it still captures the generic dynamics.)
,,.
t?'y'y,
the difference equation represented as
;
(14)
Xk+t = F(Xk); x l ,
can
.6, then
k • ~.
be (17)
xl is the initial condition. This recursion is properly defined on the domain: DF=R2\,t..~JoF-"({(~),(O);
y•l~}).
(18)
In precise terms we are interested in the dynamics of the continuously differentiable map F restricted to the domain De. Note that the inverse map is given by:
\Y2/
-- Y2
]- _7
4.1. The characteristics o f
"
(19)
DF
DF is most easily characterized by investigating some basic properties of the family of curves
{C.,ne~}: (20)
4. T H E C L O S E D L O O P D Y N A M I C S II; b < 0
In this section we study the equation v~=
1 Uk-1
1 t---,
k•N
;
(12)
neN,
n>-I
(21)
Uk-2
which describes the evolution of the normalized ratio of successive plant outputs in the case b < 0 , (10) and (11). We demonstrate the existence of a "globally" attractive periodic orbit of period two, and investigate the consequence of this periodic behaviour for the closed adaptive loop via (8)-(10). We introduce a simple time-dependent scaling transformation, which maps the periodic orbits
which has to be deleted from the phase plane to obtain De. The following result holds. L e m m a 4.1. (i) The curves Cn are a subset of
0,. (ii) The curves (7, shrink towards the origin as n increases:
488
I.M.Y.
MAR~BLS and R. R. BITMEAD
(iii) The curves C~, n ~ 3 are the boundaries of a sequence of compact sets, denoted by S~, containing in their interior Cn+~. [] The proof of this lemma relies on the following result which investigates the dynamics of F -1 on 02U04 •
Lemma 4.2. The union of the closed second and fourth quadrants (~)2U 04) is invariant under F-l, and is contained in the domain of attraction of the origin, i.e. F-~(x)-"~O as n~'oo V x e 0 2 U 0 4
[]
Proof of Lemma 4.2. From the definition of F -L, (4.8), it follows that F -1 is well defined on
C3 encloses a compact set in 02 U 04 (denoted by Ss). Because F -1 is a homeomorphism on Q2 U 0,\{('~), y e R0}, it follows that C, N C,+1 = ((0 0)v}, Vn e N and because F -1 contracts 0 2 U 0 4 to the origin, C.+1 must be contained in the compact set enclosed by C, denoted by Sn (6(S~) = C,, S, = C~.1). (If Cn-a is contained in S, the (7,-1 and (7, intersect in a point different from (0 0), contradicting the fact that F -~ is a homeomorphism on 0 2 U 0 4 \ ((y 0)r,y e ~0} with only the origin as fixed point.) [] The previous lemmata allow us to partition the domain De as follows: Do = O3 U 01
Q2 u 04 and that F-1(02 U 0 , ) = Q2 U 04.
(22)
Defining on Q2 u 04, the following Lyapunov function:
V(x) =
tyd + ly21;
x = (yl
DI= { (Yl):ylY2
lY2I - lYll > 0}
l(yY2) :YlY2 < O,
lY21- tYll < 0,
192=
lYll > 0} lY21- 2y2 + 1
y2)r
and evaluating V along an orbit {F-~(x) = x~;
n e N} for an initial condition x in Q2 u 04,
we obtain that
V(F-("+l)(x))
-
= lyll,+, + ly21.+1- ly, I, -ly21, (t see below)
- 2 lyly21._l
2 lyly21~ ~<0. 1 + 2 lYlY21n-1 1 + 2 lYlY2!n
n~>3.
By construction, DF = U D~, the boundaries of these sets being ,~.0
,5(Do) = Co 6(Dn) = (7, U C~-1,
n I> 1.
(23)
One can easily verify that:
Equality can hold iff and
[yxy2[,=0
but as this implies that Xk ~ 0, Vk Wn + 1, we conclude that the origin is the unique fixed point of F -1 with a domain of attraction containing u
,y,, 01\s
lY21 2y12+ 1
D.+I = int (S~\S.+O,
V(F-"(x))
ly~y2[,_a=0
Y2 :yly2 < 0,
04.
[]
F(On n Qz) = Dn-1 N 04 F(D~ O a4) = D~_I n Q2 F(D.) = O
-i
(24)
F(Do) ¢ Do.
Summarizing the expressions (24) we have:
Proof of Lemma 4.1. Parts (i) and (ii) are direct consequences of the Lemma 4.2. Part (iii) follows readily from the observation that y T R} l + 2 y Z ) ' Y~ _y T ff~} C 3 = { ( 1 +y2y 2 1 +4y~) ' Y~ :
C2={( -y
Lemma 4.3. For almost all initial conditions--excluding initial conditions on D ~ = U Cn--the orbits are well defined. Initial conditions in Dn will travel through the sets Dk, O<,k<-n, alternating between Q2 and Q4, finally reaching Do = Q1 u Q3 and remain there. [] This lemma is illustrated in Fig. 1, which displays some of the curves Cn (C0-C4) and regions Dn (Do-D,).
5-ly,.I,, i = 1, 2, stands for the absolute value of the ith componentof x., analogousl)~ly2y21,stands for the absolute value of the product YtY~ where Yt and Y2 are the componentsof xn.
4.2. The dynamics of F restricted to Q1 or 03 Because of the symmetry, we limit ourselves to F restricted to Q1. Clearly, F(Q 0 c Qt and
Non-linear dynamics in adaptive control
489
Proof of Lemma 4.6. Because of symmetry it is
1.0-
sufficient to look at the situations a¢ I> fl >~ 1 and 3 -1 ~> ~/> 1 and 0¢-t >1 fl 1> 1. In the first case
Co
0.5-
( 1,1) a' = max a', j6, ~
¢N4 0 o
2~xfl) =max ( ,x, ~x+fl2~xfl , a1, x+/31~>1
x
-0.5-
1 = min(a~' fl' 1-' a~ fl) -I.0-
'
-1.0
'
'
'
.
I
.
.
.
-0.5
.
.
.
.
I
'
'
'
=min(ol, 2aq3 1, ~x+fl~ a~+fl'a' 2~xfl/<~l; in the second case
'
0.5
t.0
X1
FIG. 1. Domain of definition.
also F((1 1)T)=(1 1) T. The local stability properties of the fixed point (1 1)r are summarized in: Lemma 4.4. The fixed point (1 1)T is locally
exponentially stable in the sense of Lyapunov. []
1 =max(a~, r, l , 1) 1 c~+fl]>~l 2acfl] ~>max ( a~, ol+fl2ocfl, ,x, fl=min( a'fl'l-'a~ ~) ( 1 at+r) min o¢, ~2 +a ~ , o¢' 2o¢fl / ~<1;
Proof. Directly from linearization of F in the neighbourhood of (1 1)r: _1
(25)
o
which is values:
in the last case :
- = max a, fl, -
strictly stable matrix, with eigen=max(or, ot+fl, 1 ;t1'2 =
-1 +iV~ 4
2a~fl
(26)
IZLzl = @2 < 1.
:
(27)
[] The global stability properties of (1 1)a" are stated in: Lemma 4.5. For any initial condition x • Q1, the orbit {F'(x), n • N} tends to (1 1)r. []
In the proof we make use of the following lemma: Lemma 4.6. For all strictly positive numbers
=min e, oi + fl' oc' 2eft ] From this the lemma follows. Remark also that equality (in 28) can hold iff max ( 10¢,)
= max ( o¢, r, __1 o: ' ~)
min(~' 1 ) = min(~' fl' 1o~' ~) "
ol + fl/
~1
(28a) 1, ~)1 ~min( o:, 2aft ( min ~, fi, o~ o¢ +
1 e+ff~l. 2otfl /
define the following "Lyapunov" function, V(x) = max Yl, Y2, Y:
x = (Yl Y2)T e Q:. This function is positive definite: V(x)>~O; x • a a
(28b) []
(29b)
Proof of Lemma 4.5. In the domain Q:, we
a,, fl • R, we have that: 1 1 ~max a~, , max e, fl, a~' 2a~fl '
(29a)
V((1 1)T ) = 0
and is radially unbounded: lim V ( x ) - - +~.
x~,~(Q~)
490
I.M.Y.
MAREELS and R. R. BITMEAD
Evaluated along an orbit { F " ( x ) = x , , n e ~ } , x e Q1 we obtain: (denote Xk = (Y~kY2,)T) - V(x
A_, = (Do N Q3) U (D2.-~ n Q2)
)
n~l
U (DE. n Q4).
= max Ylk+~, Y2k+t, Ylk+l
(lye)
=max(Ylk+Y2 k
2y'kY2______.~k
- max(ylk, 1 ,
Yak,
(30)
(iii) Locally at the fixed points the convergence is exponential. []
- max Y~k,Yzk, Y~,
\
point ( - 1 - 1 ) T is:
This result has immediate consequences for the adaptive closed loop system. Recall that the parameter estimate/feedback gain is given by:
~)
•
ak = a +
Ylk
(31) Wk - l
Hence, from the previous lemma we obtain: V ( x k + , ) - V(x
) <. o
Yk = (--1)k 2V2V2V2V~WkYk-1. I
with equality only holding if 1
1
From these expressions theorem we obtain:
1
Ylk Consequently V(xk+l) - V(Xk) -- 0 Y~k-= 1, this proves the result.
~
"
iff
Y2k-[]
Remark 4.1. It is possible to obtain good estimates for local (but no e-small) regions containing (1 1)T wherein F is a contraction in some norm. Because this does not add substantially to our knowledge about the dynamics of F, we do not pursue this result in the sequel, local exponential stability will prove to be sufficient. 4.4. Global dynamics---consequences for the adaptive system Linking the previous lemmata, we obtain the following picture for the global dynamics of F on Dr. Theorem 4.1 (i) For any initial condition Xo in DF (= for almost all initial conditions in ~2) the orbit {Fk(xo), k e N} [see (16), (17)1 is well defined: there exist positive numbers m, M depending on Xo, such that O
and that the output of the plant Yk is given by:
Vk.
(ii) Any orbit converges to either (1 1)T or ( - 1 - 1 ) T. The domain of attraction A~ of the fixed point (1 1)T is:
and
the
U (/92, n
Q2).
The domain of attraction A_~ of the fixed
previous
Theorem 4.2. For almost all initial conditions, except for a set of Lebesgue measure zero, and for all parameter values a, b < 0 the adaptive closed-loop system described by (1)-(9) produces a bounded parameter estimate feedback gain, which exponentially becomes periodic with period two: ~k--~a +(--1) 4
as
k~.
[]
Pro@ Follows directly from Theorem 4.1 and (31). [] Theorem 4.3. For almost all initial conditions, and for all parameter values a, the adaptive closed loop system is stable for all b:0 ~>b ~> - I , in the sense that the state vector is bounded. Moreover for 0 ~>b > -½ the output is regulated exponentially to zero: yk--'~0 as
k~'~ 0 ~ b > - ½ .
For b < - I the adaptive system is unstable, and Yk diverges exponentially. []
Proof. Because, Yk = (--1)kV'2 Ibt wky~-~ and because [Wk[ converges exponentially to 1, we have that, for almost all initial conditions Y0, Y-l, Y-2:
lYkl < C(yo, y-~, y _ 2 ) ( V ~ ) * A, = (Do n Q,) U (Dz._, n Q,)
(32)
(33)
where C is a positive constant, possibly depending on the initial conditions, but bounded. Equation (33) establishes exponential stability for 0~>b > - ½ . As for almost all initial
Non-linear dynamics in adaptive control conditions Yo, Y-I, Y-2, IWkl and Yk are bounded away from zero, and because Iwkl converges exponentially to 1, we have that if b < -½ re:0 < e < ~ ] -
1
3ko(e, Yo, Y-I, Y-2) lYkl/> (1 + e) lYk-tl
(34)
Vk I---ko.
This completes the proof.
[]
Remark 4.2. It is possible to give an explicit estimate for the rate of exponential convergence, and to prove that the convergence can be faster than exponential. 5. T H E C L O S E D L O O P D Y N A M I C S III; b > 0
In this section we present both analytical and numerical evidence for the presence of chaos in the equation: 1 1 Uk k• N (35) =
-
-
Uk-I
Uk- 2
which describes the dynamics of the normalized ratios of the successive plant outputs in the situation b > 0 [see (10) and (11)], and investigate what this implies for the closed loop adaptive system. We use the following state space representation. Define the state vector as:
Xk=( vk ) ~k-1
(36)
(1, /
and the state transition map as:
Y2
\
Yl
the difference equation represented as Xk+l = G(Xk)
We will describe the dynamics of (38) on the set D = D c - ~ N D c . In precise terms we are interested in the dynamics on the twodimensional open set D of the map G: D ~ D. On D, G is a diffeomorphism. Remark 5.1. The complement of D in R 2, is a set of measure zero, because it is the union of a countable number of curves in R 2. D c is partly illustrated in Fig. 2. The figure shows the collection of the first four curves G-"({(0 y)T, (y 0)r; y • R } ) n = 0 , 1,2,3 which have to be deleted from R 2. They are labelled as Co, C1, C2, C3, respectively. 5.1. Fixed points--periodic orbits--local stability properties Searching for fixed points and periodic orbits requires solving GP(x)=x, p e N , or alternatively solving a set of algebraic equations [derived from (35)]: UI~
1 vp
k • IN.
Vk-1
The recursion (38) is well defined on:
(41) 1
k = 3 . . . . . p.
Vk- 2
It is easily demonstrated that G P ( x ) = x has no real solutions for p < 4. There is a unique orbit of period four:
x, = t_vf-z--
/-V2+
x'=t
be (38)
1 vp
1 Vk=--
/ then
1 vp-1
1 v2 . . . . vl
(37)
(35) can
491
j;
{-~/2 - V~'~
Z2
t (42)
J;
x, i=1,...,4 are the fixed points of G 4. Numerically, we verified the existence of periodic orbits for periods up to 55, by solving the set of algebraic equations (41). Despite the 1.0
the recursion (38) can be inverted: Xk = G-I(Xk+I)
(39)
0.5
where G -1 is defined by: 0
=
\z2/
z2 1 -[ZlZ2
;
(40)
the backward recursion is well defined on the set
ce -0.5 .
- 1 . 0
-
-1.0
,
-0.5
xl
n~O
\L \I/Z/
,
,
',
0.5
Fro. 2. Domain of definition.
1.0
492
I.M.Y.
MAREELS and R. R. BITMEAD
apparent symmetry, we were not able to prove analytically that there indeed exist periodic orbits of all periods ~>4. Through linearization, we analysed the local stability properties of these periodic orbits. Numerically we verified that all periodic orbits we could establish (up to period 55) are of saddle type, i.e. they possess a one-dimensional stable and a one-dimensional unstable manifold. In particular, for the periodic orbit of period 4; we have that the Jacobian of 6 4 at x~ (which determines the local stability properties) is given by
3+ DG'(xl) =
2 -2~/2-
2 ~_2~2 )
(43)
)~,(DG4(x,)) = (5 4-V~) 2> l ).,(DG4(xl)) = (5 4 - ~ ) 2 <1.
(44)
Locally, at xl, the stable manifold is tangent to: = (-0.043115~ ~s \-0.9990701 the eigenvector of DG4(x~) corresponding to ).~; and the unstable manifold is tangent to: = ( 0.758652] ~u \-0.6514971
three neighbourhoods U-l, Uo, U1, intervals on three straight lines oriented along the unstable eigenvector ~u, U0 centred on Xo and U_~, U1 centred respectively on xl - Au, xl + Au. (See Fig. 3a for the precise configuration; Fig. 3a displays a neighbourhood of the fixed point x~ enclosed by S-1, S÷1, U-1 and U÷~.) We find that the curves
GSi G-'"'(SI) i = -1, O, 1 GUj= G""(U/) j= -1, O, 1 ----
intersect each other, for ns and nu sufficiently large, transversally in "new" points, called homoclinic points. [See Fig. 3b for precise configuration, the transversal intersections are denoted by HP. Notice in particular that the separate curves GSi are indistinguishable, as are the curves GU/. This illustrates the fact that the described method for constructing the manifolds is (numerically) stable.] In view of this fact, and in view of the continuity properties of G and because "transversal intersection" is a property which persists under slight perturbations, i.e. is structurally stable, we conclude that the stable and unstable manifold of the periodic orbit of period 4 intersect transversally in a homoclinic -4.1 ----~'T . . . . . . . . . . . . .
-4.2-
~-4.3 " 7 Q
the eigenvector of DG4(xa) corresponding to ~.~.
5.2. A horseshoe in an iteration of G The global stable and global unstable manifold are defined respectively as the union of all backward iterations of the local stable manifold and as the union of all forward iterations of the local unstable manifold (Guckenheimer and Holmes, 1983). Because the stable manifold is attractive under the inverse map, and the unstable manifold is attractive under the forward map, these manifolds can be computed in a numerically stable way, using the definition to construct them. In this way we constructed partially the stable and unstable manifolds of the fixed point xl of 6 4. (This is also part of the stable and unstable manifold of the periodic orbit of period 4 of G.) We iterated backwards, under G -4, three neighbourhoods S_1, So, S~; intervals on three parallel straight lines oriented along the stable eigenvector ~s, So centred on xl, S-a and $1 centred respectively on x~-As, x~+As. Analogously, we iterated forwards, under G 4,
\\\ .......
-4.5!
.......
!,
$-1 -4.5 J 6.3
6.4
6.5
X1
(!o-~)
6.fi
6.7
FIO. 3a. Local manifolds (S, stable; U, unstable; FP, fixed point). i.O
0.5-~
G(S)
o
-0,5
-i.O
-~.0
-0,5
0
1
0.5
FIG. 3b. Global manifolds (HP, homoclinic point).
~..0
Non-linear dynamics in adaptive control point. The existence of a homoclinic point established, the Smale-Birkhoff homoclinic theorem (Guckenheimer and Holmes, 1984, section 5.3) asserts that:
Conjecture 5.1.
There exists a zero-dimensional hyperbolic invariant set on which an iteration of G 4 is topologically equivalent to a subshift of finite type. []
Remark 5.2.
That an iteration of the map G is topologically isomorphic to a subshift of finite type on a zero-dimensional hyperbolic invariant set is a compact way of stating that in the dynamics of the map G a horseshoe is present, which is a prototype for chaotic dynamics. For a description of Smale's horseshoe and for a crictical look at chaos we refer to Mees (1982). A formal definition of "subshift of finite type" or "Bernoulli shift" can be found in Guckenheimer and Holmes (1983). The important geometric picture connected with the horseshoe is the stretching and folding back over itself of an area through the action of the map G. The stretching is along the unstable manifolds, whilst the bending in the present situation is done at infinity.
Remark 5.3.
It is clear from the presence of the asymptotic lines in the manifolds (Fig. 3b) that arbitrarily close to the fixed point xl there are points belonging to D c, i.e. for which not all iterations of G or its inverse are defined. More precisely, for any given e-neighbourhood of xl, there exist points x ~ D ¢, belonging to this neighbourhood for which there exist positive integers nl, n2 depending on e: x~G-n'({(~),
(~),
y~})
\t~l/z/ Remark 5.4.
The reason for Conjecture 5.1 not being promoted to the rank of theorem is that numerical evidence is utilized to support the existence of a homoclinic point, albeit persuasive numerical evidence. Further, G operates on a non-compact domain. Resolution of these technical difficulties is probably possible but would add little to our understanding. By using the mapping of (2) we may examine the behaviour of G on a compactification of R 2 where infinity is mapped to the boundary of the square. Using this device we see that, in the construction of the horseshoe using the homoclinic point theorem, the local area around the
493
fixed point is stretched to infinity before it is bent over itself. Hence the divergence (convergence) along the unstable manifold under the action of G (of G -1) or along the stable manifold under the action of G -1 (of G) must be faster than exponential. Only "locally" at xl do points separate (converge) exponentially. The phenomenon is of course due to the highly non-linear features of the map G, and its peculiar behaviour at infinity.
Remark 5.5. In a similar manner, we can proceed for the other periodic orbits. The results are the same, yielding more horseshoes for different iterations of G. 5.3. A perioddoublingrouteto chaosin G Having established the existence of chaotic behaviour in some iterations of G, and noting the symmetry of the problem and the importance of the behaviour at infinity, it is not hard to believe that G itself must be chaotic. In this section we strengthen this by looking at the bifurcation diagram for the one parameter family of maps:
G[yl~ ( Y_~. C\y2/= c + y 2
Y2 )
c+y2
; c>0.
(45)
Yl On D, Gc converges pointwise to G as c approaches zero from above: G = lim Gc I D. cJ, o
(46)
Some easily established results are: (i) The origin is a fixed point Vc > 0, which is globally attractive for c > 1 and a locally unstable node for all c < 1. (ii) For c < 1 there is an orbit of period six ( ( ~ ) ' (0a~)' (Otr) ' ( - - ~ ) ' ( L ) '
(0))(47)
~ = lX/i--~-c which locally is exponentially stable for all c : 0.40775 ~
which is initially, c close to 0.25, locally exponentially stable; then bifurcates into a saddle periodic orbit, forming two asymmetric stable periodic orbits of period 4. This orbit converges to the periodic orbit of period 4 of G.
494
I.M.Y.
MAREELS and R. R. BITMEAD
Figure 4 gives the numerically established bifurcation diagram. Horizontally, the parameter c is represented and vertically the set of stable oJ-limit is represented--as a sequence of numbers, the first coordinate of the state (e.g. the 6-periodic orbit is represented as (or, 0, -or, -a~, 0, a~}). Although this is not the complete bifurcation diagram, it certainly indicates the period doubling route to chaos. Notice that a symmetric periodic orbit first bifurcates into two stable asymmetric periodic orbits of the same period (symmetry breaking bifurcation), then for a smaller value of the parameter these asymmetric periodic orbits undergo a period doubling bifurcation. (Observable for the periodic orbit of period 6 and 4.) For small c values, after a series of period doubling bifurcations, finally chaos emerges. [This information is not obtainable from the picture in Fig. 4, due to the projection of the state onto its first coordinate; but is apparent from the way these bifurcations have to operate (Guckenheimer and Holmes, 1983) and from the simulations.] The diagram is obtained by running the difference equation for consecutive values of c, detecting periodic behaviour and then plotting out the o)-limit set for that value of the parameter. This is repeated for different initial conditions in order to obtain the two copies of every asymmetrical periodic orbit. (Because of the symmetry of the equation it suffices to take an initial condition out of the first and third quadrant in order to obtain the whole picture. Notice that the vertical axis, representing ~ , is transformed into the interval ( - 1 , 1 ) , in accordance with all other pictures.) The diagram in Fig. 4 contains the information of 1000 c-values. The only easily recognizable bifurcations are the bifurcations involving the
orbits of period 6 and 4. If we would use a finer resolution in c, and magnify the scale of both axes, the now dark patches would show similar bifurcations (symmetry breaking followed by period doubling) of periodic orbits of periods 5, 7, 11 and so on.
Remark. Although the family Gc does not depend continuously on c at c = 0 (which can best be seen from the fact that the symmetric 6-periodic orbit can not exist for c = 0, at least not with finite amplitude) we believe that this period doubling sequence indeed provides evidence for the case that G is chaotic. Remark 5.7. This sequence does provide substantial evidence for the earlier observation that all periodic orbits are unstable! 5.4. "Cycle slipping" Analysing the time sequence {v~, k e t~} [as defined by (35)] one observes that the time sequence consists of certain segments of apparent almost periodic behaviour, separated by a short transistion characterized by large deviations. (See Fig. 5, which displays a sample of the time sequence of (Vk, k • ~}.) Assuming that all periodic orbits are either of saddle type or completely unstable, of which we are strongly convinced in the light of the previous observations, this phenomenon becomes easy to understand. Orbits "close" to a stable manifold belonging to a certain periodic orbit approach this periodic orbit, hence approach the unstable manifold of this periodic orbit and are consequently repelled away from it, to be captured by a stable manifold belonging to another periodic orbit. This whole cycle keeps repeating itself. This intuitive picture, "cycle slipping", gives
a m
P 1 i t
u
d
¥
e
SEOUENC E OF PERIOD ~OU6L~NG BIFURCATIONS
o.o
o.t
o,2
o.3
0.4
o.s
o.s
o.?
o,e
o.9
t.o
parameter FIG. 4. Bifurcation diagram (SB, symmetry breaking; HB, Hopf bifurcation; PD, period doubling).
0
100
200
300
k
FIG. 5. Time sequencevx
400
500
Non-linear dynamics in adaptive control us the impression that the union of all the unstable manifolds of the periodic orbits of saddle type might be a one-dimensional strange attractor.
Remark 5.8. In principle it would be possible that there are strictly stable periodic orbits, with such a small basin of attraction that they are unobservable from a computer simulation. However, we have never established such orbits, and also the period doubling sequence seems to exclude this possibility--although this is not analytically established. 5.5. The "gumleaf attractor" Figure 6 represents the orbit of a "typical" initial condition in the phase plane. It consists of 40 000 iterations. Comparing the "curves" traced by this orbit with the unstable manifold of the periodic orbit of period 4, we find that they are virtually identical. Therefore combining, all previous observations, we conjecture:
495
previous analytical evidence points this way. (An important fact to note is that it appears as if the unstable manifold of the periodic orbit of period 4 is dense in this attractor. This is a property of an hyperbolic, connected attractor.) It even appears as if the whole of D, the open manifold on which G is defined, forms the basin of attraction of this attractor. A theorem of Sinai-Bowen-Ruelle (Guckenheimer and Holmes, 1983) states that for diffeomorphism F defined on a compact manifold possessing a hyperbolic strange attractor S there exists a measure/~, invariant with respect to the diffeomorphism, supported on the hyperbolic attractor, such that for all initial conditions x in the basin of attraction of the attractor, and for all real-valued continuous functions g the cesaro mean evaluated along the orbit generated by x exists and converges to the ensemble average of g over the attractor: liml •
N1"==/~k=0
g(Fk(x) ) =
fs
g d~.
(49)
Conjecture 5.2. The union of all unstable manifolds of the periodic orbits of saddle type form a one-dimensional hyperbolic strange attractor, denoted by S. []
In other terms F is ergodic. We conjecture that this result also holds for the present situation.
It is clear that the union of all the unstable manifolds of the periodic orbits of saddle type is a one-dimensional set, which is attracting. Moreover, if it is an attractor it is a strange attractor because it contains several transversal homoclinic orbits---and the corresponding horseshoes. The conjecture would be proven if all periodic orbits were unstable, and if all the periodic orbits of saddle type had transversally intersecting stable and unstable manifolds. The closure of the stable manifolds of the periodic orbits would then form the basin of attraction for this attractor, as well as the foliation of stable manifolds of this attractor. All numerical and
The technical difficulties--apart from those encountered for the previous conjecture--stem from the fact that it is not clear whether G can be extended to a diffeomorphism G ~ defined on a compact manifold D 1 containing D as a submanifold. If this were possible, then this conjecture follows from the previous one by the quoted theorem of Sinai, Bowen and Ruelle. Note that the "cycle slipping" idea strongly suggests that the cesaro mean of continuous functions evaluated along typical orbits should exist and be independent of the particular orbit. Indeed orbits spent most of their time in the neighbourhood of periodic orbits. Clearly, on the periodic orbits the cesaro mean is well defined. The invariant measure would then simply attribute different weights to different periodic orbits according to the relative amount of time spent by a typical orbit in their neighbourhood.
1 . 0 . ~ i
"X"
.
0.5X2
-
Conjecture 5.3. G is ergodic.
[]
0.
5.6. Implications for the closed loop adaptive system -0.5-
-! .0 -t.0
-0.5
' 0
'
'
'
I 0.5
1(1
FIG. 6. "Oumleafattractor".
1.0
Firstly, because of (9) and (10) it is clear that the parameter estimate feedback gain behaves chaotically. What does a chaotic feedback gain imply for the stability of the closed loop? Recall that
Yk = Vb VkYk-i
(50)
496
I.M.Y.
MAREELS and R. R. BITMEAD
[see (8) and (9)]. Hence, we are interested in the products of "chaotic" signals: Yk =
Vt
o.
(51)
Using the ergodicity property, we can immediately investigate the cesaro mean of the logarithm of the absolute values of the v~s; evaluating this numerically we obtain: 1
N
lim ~ k=~llog Irk I ~ ½log 2.
(52)
Remark 5.9. We have exhaustively evaluated (52) for different initial conditions and for different sample sizes. For sufficiently large N, N > 10 000 appears adequate, the same value (1og2)/2 was obtained in a "statistically" consistent way. This observation strongly supports the ergodicity property. Consequently,
lim 1 log lyN[--~½log2b.
Nr~N
(53)
We conclude therefore that Yk converges exponentially to zero for all n: 0 < b < ½, in the sense that, for almost all (Lebesgue) initial conditions, there exists a constant or> 1, independent of the initial conditions such that: o:kfykl--*O
as
kT~
(54)
and diverges exponentially for all b > ½, in the sense that there exists a constant fl < 1, such that for almost all initial conditions: flk lYkl-" + ~
as
k~'~.
o~, fl only depend on b; 1 1 < o~<~--~ ; 0 < b < ½ 1
(55)
;Tgg
< 1; ½
Remark 5.10. The present definition of exponential convergence is different from the classical definition, but is, however, frequently used in the context of stochastic processes (Bitmead, 1984). Notice in particular that (54) does not imply the existence of an exponentially decaying overbound for tY~I. Furthermore the adaptive feedback gain ak given by ~tk = a - - - Uk
[see (9) and (10)] behaves in a chaotic fashion
because v, does, hence does not yield directly any information about the system's parameters. 6. CONCLUSIONS
We have presented a careful analysis of an equation arising in adaptive control. The equation of interest describes the adaptive feedback gain used in an adaptive deadbeat controller, based on the current parameter estimates. These estimates fit a first-order model for a second-order plant. The parameter b, indicator of the model-plant mismatch, determines the topology of the dynamics of the adaptive loop. This topology varies discontinuously with b. For b = 0, no model error, the topology is trivially simple, but for b :# 0, the dynamics vary drastically: b < 0 yields limit cycle behaviour, b > 0 gives chaos. Based on the conjecture that the dynamics are ergodic for b > 0 (a conjecture we supported both with analytical and numerical evidence) we were able to characterize in the parameter space those systems for which the controller could regulate the output to zero, i.e. achieve its design objective. More information on this equation, and specifically about the implications for adaptive control are presented in Mareels and Bitmead (1986). We believe that this explicit demonstration of the inherent non-linear features of adaptive control is an initial attempt to understand the global dynamics of adaptive control algorithms; much more work remains to be done. Other researchers have independently reported other instances of chaotic behaviour in continuous-time adaptive control (Salam and Bai, 1986; Rubio et al., 1985). Our results differ considerably from these. Salam and Bai (1986) are concerned with a "Melnikov" route to chaos, due to external excitation of the algorithm, whilst in Rubio et al. (1985) the chaos is due not to the adaptive algorithm itself but rather to the non-linear features of the plant to be controlled. We have dealt with these complicated dynamics in a constructive way proving that chaos is not necessarily a destructive property. REFERENCES Anderson, B. D. O., R. R. Bitmead, C. R. Johnson, Jr, P. V. Kokotovic, R. L. Kosut, I. M. Y. Mareels, L. Praly and B. D. Riedle (1986). Stability of Adaptive Systems: Passivity and Averaging Analysis. MIT Press, Cambridge, MA. Bitmead, R. R. (1984). Persistence of excitation conditions and the convergence of adaptive schemes. IEEE Trans. Inform. Theory, |T-30, 183-191. Goodwin, G. C. and K. S. Sin (1984). Adaptive Filtering Prediction and Control, Information and Systems Science Series. Prentice-Hall, Englewood Cliffs, NJ.
Non-linear dynamics in adaptive control and P. Holmes (1983). Nonlinear Oscillations, Dynamical Systems and Bifurcations of Vector Fields, Applied Mathematics Series, Vol. 42. Springer,
Guckenheimer, J.
Berlin. Mareels, I. M. Y. and R. R. Bitmead (1986). Nonlinear dynamics in adaptive control: chaotic and periodic stabilization. Automatica, 22, 641-655. Mees, A. I. (1982). Dynamics of Feedback Systems. Wiley Interscience, New York.
497
Riedle, B. D. and P. V. Kokotovic (1986). Integral manifold approach to slow adaptation. IEEE Trans. Aut. Control, AC-31, 316-324. Rubio, F. R., J. Aracil and E. F. Camacho (1985). Chaotic motion in an adaptive control system. Int. J. Control, 42, 353-360. Salam, F. M. A. and S. Bai (1986). Disturbance-generated bifurcation in a simple adaptive control system: simulation evidence. Syst. Control Lett., 7, 269-280.
Automatica, Vol. 24, No. 4, pp. 485-497, 1988 Printed in Great Britain.
Non-linear Dynamics in Adaptive Control: Periodic and Chaotic Stabilization II. Analysis* IVEN M. Y. M A R E E L S t and R O B E R T R. BITMEAD$
A mathematical analysis of a simple adaptive control scheme in the presence of unmodeUed dynamics displays complex dynamical phenomena and indicates possibilities and limitations of robust adaptive control. Key Words--Adaptive control; non-linear systems; chaotic dynamics; robust control.
tions (Riedle and Kokotovic, 1986; Anderson et al., 1986). The importance of the results of Marcels and Bitmead (1986) was that they demonstrated two main features: firstly, that the non-linear dynamics of adaptive control systems could be manifested in their behaviour with the occurrence of complex periodic and chaotic motions; and, secondly, that these dynamics had the capacity for delivering very robustly globally stabilizing feedback control for a large class of plants---the motive aim of adaptive control. Our aim in this paper is to develop more fully the theoretical justification and proofs for the dynamic behaviour observed in Marcels and Bitmead (1986), and, in so doing, to highlight the extraordinary difficulty and perversity of carrying out the complete global analysis of a very simple robust adaptive control problem. In particular, we believe that this latter aim serves the useful purpose of delineating realistic expectations for analyses of adaptive controllers, all the more so because the large initial conditions transient behaviour of some more standard adaptive control procedures was shown in Marcels and Bitmead (1986) to duplicate these dynamics. The message conveyed here by this simplest of non-trivial robust adaptive control problems is that more general adaptive controllers will exhibit dynamics of at least this level of complexity. While this message appears to augur poorly for the prospects of adaptive controller analysis, it does help to focus investigations onto quantitatively meaningful problems and to raise questions, for example, about the efficacy of seeking both to produce global results and simultaneously to constrain the transient dynamics of adaptation. The inherent value of results on unqualified boundedness or asymptotic
Abstract--A particular non-linear equation arising in an adaptive control problem is presented in the first part of this paper. This equation displays vividly a wealth of non-linear phenomena. Depending on the value of one crucial parameter---characterizing the extent and presence of "non-design" circumstances--we demonstrate the existence of asymptotically periodic behaviour or chaotic dynamics in the feedback controller. This paper contains the mathematical analysis associated with the earlier results and, as such, serves two main purposes: to provide the theoretical proofs and support for these results, and to demonstrate more fully the extraordinary complexity and difficulty of conducting a thorough global analysis of even this simplest of robust adaptive problems. By performing this latter task we aim to delineate better what are reasonable objectives to be expected of robust adaptive control theory.
1. INTRODUCTION
IN THE FIRST part of this work (Marcels and Bitmead, 1986) an adaptive control problem was studied which exhibited very non-linear behaviour. This example was chosen to accentuate this non-linearity by incorporating fast adaptation, plant undermodelling and a severe controller objective. Classically one tends to think about the behaviour of an adaptive system as being asymptotically linear by relying upon parameter convergence (Goodwin and Sin, 1984) or local linearized manifold dynamics descrip*Received 26 May 1987; revised 6 November 1987; received in final form 13 December 1988. The original version of this paper was not presented at any IFAC meeting. This paper was recommended for publication in revised form by Associate Editor G. Kreisselmeier under the direction of Editor P. C. Parks. 5"Research Assistant with the National Fund for Scientific Research, Belgium, whose support is acknowledged. Department of Systems Engineering, University of Gent, Grote Steenweg Noord 2, B-9700, Gent (Zwijnaarde), Belgium. Current address: Department of Electrical and Computer Engineering, University of Newcastle, NSW 2308, Australia. ~:Department of Systems Engineering, Research School of Physical Sciences, Australian National University, GPO Box 4, Canberra, ACT 2601, Australia. Atrr 24-4-o
485
486
I.M.Y.
MAREELS and R. R. BITMEAD
optimality is brought into question. In our analysis, it is the control questions which are of prime concern. However, the techniques applied may be of interest in the broader study of non-linear dynamical systems as the multiplicative ergodic theorems are novel in their application to stability questions. The outline of the paper is as follows. In Section 2 we briefly derive the principal equation of interest from its adaptive control context. This equation's dynamics depend on a single parameter, b, representing the degree of plant undermodelling. Section 3 briefly touches upon the case of exact modelling, b = 0, when the adaptive controller performs "ideally". Section 4 treats the negative b case which produces globally attractive asymptotically two-periodic dynamics in the controller gain. Lyapunov techniques may be used here. Section 5 is devoted to the treatment of the positive b case where chaotic dynamics are generated. The analytical tools applied in this latter section incorporate both theoretical developments and numerical evidence to support theorems and conjectures concerning the global dynamics. Section 6 concludes. In presenting this analysis we are not only demonstrating some of the interesting and varied dynamical properties of adaptive controllers and the features of their underlying mathematical description but also providing an object lesson in the practical difficulties of a complete and thorough analysis of both transient and steadystate behaviour in adaptive control. Some conventions
We adopt the following notation. The first to fourth open quadrants of R 2 are denoted as al-a4. The closure of a set S in N2 will be denoted by S, its complement in R 2 by S c, its boundary by 6 ( S ) and its interior by int (S). 2. PROBLEM ORIGIN
The specific difference equation which is the main object of study of this paper is: rk=b(r 1-
1 )
1 rk--Z
keN;
ben
(1)
whose derivation will be briefly described shortly. This non-linear autonomous difference equation has a state in R 2 [given fo, ft the whole trajectory {rk, k e rN} with ro = ro and h = rl is uniquely determined by (1)]. In order to illustrate our results we often present phase plane pictures of the trajectories. Actually because the (numerical) dynamic range of the trajectories is too large we squeeze R 2 into the square ( - 1 , 1) x ( - 1 , 1) by the homeo-
morphism: h :l~2--~ (-1, I)x (-I, 1 ) : ( x , y ) ~ h ( x , y )
(x:,). l+lxl'l
h(x,y)=
Yl '
(2)
this preserves all topological features. Obviously in order to have the trajectories well defined for all k, rk should be non-zero (and bounded) for all k. Only these trajectories are relevant for the underlying adaptive scheme. Therefore, instead of extending the definition of the difference operator in order to make "division by zero" meaningful in some sense, we exclude from the phase plane all initial conditions for which there is some k such that rk = 0. The collection of these "pre-images of zero" is the one-dimensional set:
G = { (x), 3k :rk(x, y) = O}. Y
We study the difference equation (1) on the complement of this set in R 2. On this set the difference operator corresponding to (1) is a diffeomorphism. Equation (1) is derived in Mareels and Bitmead (1986) via the interconnection of a second-order, linear, time-invariant plant system Yk = ayk-1 +
byk-2 + Uk-t,
a, b e ~
(3)
with an adaptive controller based upon a first order model Yk = t~yk-1 + Uk-1,
~ e ~.
(4)
The adaptive controller consists of a deadbeat identifier and a deadbeat feedback control law gZk+l = ak + 1/y~(Yk+l -- akyk -- Uk),
Yk :/: 0
~k.l = ak, Uk= --akyk.
Yk= 0
(5) (6)
Connecting (3), (5) and (6) and introducing rk = Yk/Yk-1
(7)
yields (1). Note that rk relates to the feedback gain and the plant output via Yk = rkyk-1 = rkrk-~ " " " r o y - i ak = a -- b / r k - 1 .
(8) (9)
The original closed loop Yk system has a state vector in N3; but because of the system's specific structure it is sufficient to study (1) (with a state in Nz) in order to capture the generic dynamics of the adaptive system. It is not too difficult to analyse the behaviour of the trajectories for which a "division by zero" could occur-however, this does not add anything fundamental to our knowledge of the system, and
Non-linear dynamics in adaptive control moreover this only gives us information about a two-dimensional subset of initial conditions in the state space of the original system. Therefore we do not pursue this here. In the following sections we study (1) in detail. By way of preliminary analysis we note that as a function of the parameter b we can at most distinguish three topologically different types of dynamical behaviour for (1). Indeed, rescaling as vk = r k l V ~ l ,
b ~0
(10)
487
into two fixed points: k l"Ik
Wk=(--1) ~7~,
keN;
(13)
the equation governing wk becomes: Wk =
1) ,
+ 141k-1
(14)
k e~.
Wk- 2
Introduce the following state space representation; define the state as:
leads to
(1
Vk=sign(b) v_x
1)
vZ-2 "
,11,
Xk=(
Wk ],
(15)
k•N
\Wk-1/
)
define the state transition map F as:
F(Y'/:/1(--'+'
So it is possible to consider three situations b
In the sequel we discuss all three types of dynamical behaviour. 3. T H E C L O S E D L O O P D Y N A M I C S I; b = 0
In this ideal situation, the dynamics are quite obvious, as one could expect from the combination of a deadbeat identifier and deadbeat control law. The closed loop reduces to: Plant: Yk ayk-I + Uk-l, Identifier: ak = a, Control law: u~-l = --ayk-1, Hence: Yk -~ O, =
Vk (Y-1 ~ 0), Vk Vk Vk
• N >10 >- 1 >! 2.
In the situation that Y-1 = 0, the identifier does not identify anything useful, but there is also no need to identify anything, as in this case Yk - 0, Vk • N and ak = a - l , Vk e N; hence the control objective is met. In this situation the equation (1) governing the ratio of successive outputs is redundant, except to highlight a definitional problem with the deadbeat identifier when the deadbeat controller actually works. (Notice however, that it still captures the generic dynamics.)
,,.
t?'y'y,
the difference equation represented as
;
(14)
Xk+t = F(Xk); x l ,
can
.6, then
k • ~.
be (17)
xl is the initial condition. This recursion is properly defined on the domain: DF=R2\,t..~JoF-"({(~),(O);
y•l~}).
(18)
In precise terms we are interested in the dynamics of the continuously differentiable map F restricted to the domain De. Note that the inverse map is given by:
\Y2/
-- Y2
]- _7
4.1. The characteristics o f
"
(19)
DF
DF is most easily characterized by investigating some basic properties of the family of curves
{C.,ne~}: (20)
4. T H E C L O S E D L O O P D Y N A M I C S II; b < 0
In this section we study the equation v~=
1 Uk-1
1 t---,
k•N
;
(12)
neN,
n>-I
(21)
Uk-2
which describes the evolution of the normalized ratio of successive plant outputs in the case b < 0 , (10) and (11). We demonstrate the existence of a "globally" attractive periodic orbit of period two, and investigate the consequence of this periodic behaviour for the closed adaptive loop via (8)-(10). We introduce a simple time-dependent scaling transformation, which maps the periodic orbits
which has to be deleted from the phase plane to obtain De. The following result holds. L e m m a 4.1. (i) The curves Cn are a subset of
0,. (ii) The curves (7, shrink towards the origin as n increases:
488
I.M.Y.
MAR~BLS and R. R. BITMEAD
(iii) The curves C~, n ~ 3 are the boundaries of a sequence of compact sets, denoted by S~, containing in their interior Cn+~. [] The proof of this lemma relies on the following result which investigates the dynamics of F -1 on 02U04 •
Lemma 4.2. The union of the closed second and fourth quadrants (~)2U 04) is invariant under F-l, and is contained in the domain of attraction of the origin, i.e. F-~(x)-"~O as n~'oo V x e 0 2 U 0 4
[]
Proof of Lemma 4.2. From the definition of F -L, (4.8), it follows that F -1 is well defined on
C3 encloses a compact set in 02 U 04 (denoted by Ss). Because F -1 is a homeomorphism on Q2 U 0,\{('~), y e R0}, it follows that C, N C,+1 = ((0 0)v}, Vn e N and because F -1 contracts 0 2 U 0 4 to the origin, C.+1 must be contained in the compact set enclosed by C, denoted by Sn (6(S~) = C,, S, = C~.1). (If Cn-a is contained in S, the (7,-1 and (7, intersect in a point different from (0 0), contradicting the fact that F -~ is a homeomorphism on 0 2 U 0 4 \ ((y 0)r,y e ~0} with only the origin as fixed point.) [] The previous lemmata allow us to partition the domain De as follows: Do = O3 U 01
Q2 u 04 and that F-1(02 U 0 , ) = Q2 U 04.
(22)
Defining on Q2 u 04, the following Lyapunov function:
V(x) =
tyd + ly21;
x = (yl
DI= { (Yl):ylY2
lY2I - lYll > 0}
l(yY2) :YlY2 < O,
lY21- tYll < 0,
192=
lYll > 0} lY21- 2y2 + 1
y2)r
and evaluating V along an orbit {F-~(x) = x~;
n e N} for an initial condition x in Q2 u 04,
we obtain that
V(F-("+l)(x))
-
= lyll,+, + ly21.+1- ly, I, -ly21, (t see below)
- 2 lyly21._l
2 lyly21~ ~<0. 1 + 2 lYlY21n-1 1 + 2 lYlY2!n
n~>3.
By construction, DF = U D~, the boundaries of these sets being ,~.0
,5(Do) = Co 6(Dn) = (7, U C~-1,
n I> 1.
(23)
One can easily verify that:
Equality can hold iff and
[yxy2[,=0
but as this implies that Xk ~ 0, Vk Wn + 1, we conclude that the origin is the unique fixed point of F -1 with a domain of attraction containing u
,y,, 01\s
lY21 2y12+ 1
D.+I = int (S~\S.+O,
V(F-"(x))
ly~y2[,_a=0
Y2 :yly2 < 0,
04.
[]
F(On n Qz) = Dn-1 N 04 F(D~ O a4) = D~_I n Q2 F(D.) = O
-i
(24)
F(Do) ¢ Do.
Summarizing the expressions (24) we have:
Proof of Lemma 4.1. Parts (i) and (ii) are direct consequences of the Lemma 4.2. Part (iii) follows readily from the observation that y T R} l + 2 y Z ) ' Y~ _y T ff~} C 3 = { ( 1 +y2y 2 1 +4y~) ' Y~ :
C2={( -y
Lemma 4.3. For almost all initial conditions--excluding initial conditions on D ~ = U Cn--the orbits are well defined. Initial conditions in Dn will travel through the sets Dk, O<,k<-n, alternating between Q2 and Q4, finally reaching Do = Q1 u Q3 and remain there. [] This lemma is illustrated in Fig. 1, which displays some of the curves Cn (C0-C4) and regions Dn (Do-D,).
5-ly,.I,, i = 1, 2, stands for the absolute value of the ith componentof x., analogousl)~ly2y21,stands for the absolute value of the product YtY~ where Yt and Y2 are the componentsof xn.
4.2. The dynamics of F restricted to Q1 or 03 Because of the symmetry, we limit ourselves to F restricted to Q1. Clearly, F(Q 0 c Qt and
Non-linear dynamics in adaptive control
489
Proof of Lemma 4.6. Because of symmetry it is
1.0-
sufficient to look at the situations a¢ I> fl >~ 1 and 3 -1 ~> ~/> 1 and 0¢-t >1 fl 1> 1. In the first case
Co
0.5-
( 1,1) a' = max a', j6, ~
¢N4 0 o
2~xfl) =max ( ,x, ~x+fl2~xfl , a1, x+/31~>1
x
-0.5-
1 = min(a~' fl' 1-' a~ fl) -I.0-
'
-1.0
'
'
'
.
I
.
.
.
-0.5
.
.
.
.
I
'
'
'
=min(ol, 2aq3 1, ~x+fl~ a~+fl'a' 2~xfl/<~l; in the second case
'
0.5
t.0
X1
FIG. 1. Domain of definition.
also F((1 1)T)=(1 1) T. The local stability properties of the fixed point (1 1)r are summarized in: Lemma 4.4. The fixed point (1 1)T is locally
exponentially stable in the sense of Lyapunov. []
1 =max(a~, r, l , 1) 1 c~+fl]>~l 2acfl] ~>max ( a~, ol+fl2ocfl, ,x, fl=min( a'fl'l-'a~ ~) ( 1 at+r) min o¢, ~2 +a ~ , o¢' 2o¢fl / ~<1;
Proof. Directly from linearization of F in the neighbourhood of (1 1)r: _1
(25)
o
which is values:
in the last case :
- = max a, fl, -
strictly stable matrix, with eigen=max(or, ot+fl, 1 ;t1'2 =
-1 +iV~ 4
2a~fl
(26)
IZLzl = @2 < 1.
:
(27)
[] The global stability properties of (1 1)a" are stated in: Lemma 4.5. For any initial condition x • Q1, the orbit {F'(x), n • N} tends to (1 1)r. []
In the proof we make use of the following lemma: Lemma 4.6. For all strictly positive numbers
=min e, oi + fl' oc' 2eft ] From this the lemma follows. Remark also that equality (in 28) can hold iff max ( 10¢,)
= max ( o¢, r, __1 o: ' ~)
min(~' 1 ) = min(~' fl' 1o~' ~) "
ol + fl/
~1
(28a) 1, ~)1 ~min( o:, 2aft ( min ~, fi, o~ o¢ +
1 e+ff~l. 2otfl /
define the following "Lyapunov" function, V(x) = max Yl, Y2, Y:
x = (Yl Y2)T e Q:. This function is positive definite: V(x)>~O; x • a a
(28b) []
(29b)
Proof of Lemma 4.5. In the domain Q:, we
a,, fl • R, we have that: 1 1 ~max a~, , max e, fl, a~' 2a~fl '
(29a)
V((1 1)T ) = 0
and is radially unbounded: lim V ( x ) - - +~.
x~,~(Q~)
490
I.M.Y.
MAREELS and R. R. BITMEAD
Evaluated along an orbit { F " ( x ) = x , , n e ~ } , x e Q1 we obtain: (denote Xk = (Y~kY2,)T) - V(x
A_, = (Do N Q3) U (D2.-~ n Q2)
)
n~l
U (DE. n Q4).
= max Ylk+~, Y2k+t, Ylk+l
(lye)
=max(Ylk+Y2 k
2y'kY2______.~k
- max(ylk, 1 ,
Yak,
(30)
(iii) Locally at the fixed points the convergence is exponential. []
- max Y~k,Yzk, Y~,
\
point ( - 1 - 1 ) T is:
This result has immediate consequences for the adaptive closed loop system. Recall that the parameter estimate/feedback gain is given by:
~)
•
ak = a +
Ylk
(31) Wk - l
Hence, from the previous lemma we obtain: V ( x k + , ) - V(x
) <. o
Yk = (--1)k 2V2V2V2V~WkYk-1. I
with equality only holding if 1
1
From these expressions theorem we obtain:
1
Ylk Consequently V(xk+l) - V(Xk) -- 0 Y~k-= 1, this proves the result.
~
"
iff
Y2k-[]
Remark 4.1. It is possible to obtain good estimates for local (but no e-small) regions containing (1 1)T wherein F is a contraction in some norm. Because this does not add substantially to our knowledge about the dynamics of F, we do not pursue this result in the sequel, local exponential stability will prove to be sufficient. 4.4. Global dynamics---consequences for the adaptive system Linking the previous lemmata, we obtain the following picture for the global dynamics of F on Dr. Theorem 4.1 (i) For any initial condition Xo in DF (= for almost all initial conditions in ~2) the orbit {Fk(xo), k e N} [see (16), (17)1 is well defined: there exist positive numbers m, M depending on Xo, such that O
and that the output of the plant Yk is given by:
Vk.
(ii) Any orbit converges to either (1 1)T or ( - 1 - 1 ) T. The domain of attraction A~ of the fixed point (1 1)T is:
and
the
U (/92, n
Q2).
The domain of attraction A_~ of the fixed
previous
Theorem 4.2. For almost all initial conditions, except for a set of Lebesgue measure zero, and for all parameter values a, b < 0 the adaptive closed-loop system described by (1)-(9) produces a bounded parameter estimate feedback gain, which exponentially becomes periodic with period two: ~k--~a +(--1) 4
as
k~.
[]
Pro@ Follows directly from Theorem 4.1 and (31). [] Theorem 4.3. For almost all initial conditions, and for all parameter values a, the adaptive closed loop system is stable for all b:0 ~>b ~> - I , in the sense that the state vector is bounded. Moreover for 0 ~>b > -½ the output is regulated exponentially to zero: yk--'~0 as
k~'~ 0 ~ b > - ½ .
For b < - I the adaptive system is unstable, and Yk diverges exponentially. []
Proof. Because, Yk = (--1)kV'2 Ibt wky~-~ and because [Wk[ converges exponentially to 1, we have that, for almost all initial conditions Y0, Y-l, Y-2:
lYkl < C(yo, y-~, y _ 2 ) ( V ~ ) * A, = (Do n Q,) U (Dz._, n Q,)
(32)
(33)
where C is a positive constant, possibly depending on the initial conditions, but bounded. Equation (33) establishes exponential stability for 0~>b > - ½ . As for almost all initial
Non-linear dynamics in adaptive control conditions Yo, Y-I, Y-2, IWkl and Yk are bounded away from zero, and because Iwkl converges exponentially to 1, we have that if b < -½ re:0 < e < ~ ] -
1
3ko(e, Yo, Y-I, Y-2) lYkl/> (1 + e) lYk-tl
(34)
Vk I---ko.
This completes the proof.
[]
Remark 4.2. It is possible to give an explicit estimate for the rate of exponential convergence, and to prove that the convergence can be faster than exponential. 5. T H E C L O S E D L O O P D Y N A M I C S III; b > 0
In this section we present both analytical and numerical evidence for the presence of chaos in the equation: 1 1 Uk k• N (35) =
-
-
Uk-I
Uk- 2
which describes the dynamics of the normalized ratios of the successive plant outputs in the situation b > 0 [see (10) and (11)], and investigate what this implies for the closed loop adaptive system. We use the following state space representation. Define the state vector as:
Xk=( vk ) ~k-1
(36)
(1, /
and the state transition map as:
Y2
\
Yl
the difference equation represented as Xk+l = G(Xk)
We will describe the dynamics of (38) on the set D = D c - ~ N D c . In precise terms we are interested in the dynamics on the twodimensional open set D of the map G: D ~ D. On D, G is a diffeomorphism. Remark 5.1. The complement of D in R 2, is a set of measure zero, because it is the union of a countable number of curves in R 2. D c is partly illustrated in Fig. 2. The figure shows the collection of the first four curves G-"({(0 y)T, (y 0)r; y • R } ) n = 0 , 1,2,3 which have to be deleted from R 2. They are labelled as Co, C1, C2, C3, respectively. 5.1. Fixed points--periodic orbits--local stability properties Searching for fixed points and periodic orbits requires solving GP(x)=x, p e N , or alternatively solving a set of algebraic equations [derived from (35)]: UI~
1 vp
k • IN.
Vk-1
The recursion (38) is well defined on:
(41) 1
k = 3 . . . . . p.
Vk- 2
It is easily demonstrated that G P ( x ) = x has no real solutions for p < 4. There is a unique orbit of period four:
x, = t_vf-z--
/-V2+
x'=t
be (38)
1 vp
1 Vk=--
/ then
1 vp-1
1 v2 . . . . vl
(37)
(35) can
491
j;
{-~/2 - V~'~
Z2
t (42)
J;
x, i=1,...,4 are the fixed points of G 4. Numerically, we verified the existence of periodic orbits for periods up to 55, by solving the set of algebraic equations (41). Despite the 1.0
the recursion (38) can be inverted: Xk = G-I(Xk+I)
(39)
0.5
where G -1 is defined by: 0
=
\z2/
z2 1 -[ZlZ2
;
(40)
the backward recursion is well defined on the set
ce -0.5 .
- 1 . 0
-
-1.0
,
-0.5
xl
n~O
\L \I/Z/
,
,
',
0.5
Fro. 2. Domain of definition.
1.0
492
I.M.Y.
MAREELS and R. R. BITMEAD
apparent symmetry, we were not able to prove analytically that there indeed exist periodic orbits of all periods ~>4. Through linearization, we analysed the local stability properties of these periodic orbits. Numerically we verified that all periodic orbits we could establish (up to period 55) are of saddle type, i.e. they possess a one-dimensional stable and a one-dimensional unstable manifold. In particular, for the periodic orbit of period 4; we have that the Jacobian of 6 4 at x~ (which determines the local stability properties) is given by
3+ DG'(xl) =
2 -2~/2-
2 ~_2~2 )
(43)
)~,(DG4(x,)) = (5 4-V~) 2> l ).,(DG4(xl)) = (5 4 - ~ ) 2 <1.
(44)
Locally, at xl, the stable manifold is tangent to: = (-0.043115~ ~s \-0.9990701 the eigenvector of DG4(x~) corresponding to ).~; and the unstable manifold is tangent to: = ( 0.758652] ~u \-0.6514971
three neighbourhoods U-l, Uo, U1, intervals on three straight lines oriented along the unstable eigenvector ~u, U0 centred on Xo and U_~, U1 centred respectively on xl - Au, xl + Au. (See Fig. 3a for the precise configuration; Fig. 3a displays a neighbourhood of the fixed point x~ enclosed by S-1, S÷1, U-1 and U÷~.) We find that the curves
GSi G-'"'(SI) i = -1, O, 1 GUj= G""(U/) j= -1, O, 1 ----
intersect each other, for ns and nu sufficiently large, transversally in "new" points, called homoclinic points. [See Fig. 3b for precise configuration, the transversal intersections are denoted by HP. Notice in particular that the separate curves GSi are indistinguishable, as are the curves GU/. This illustrates the fact that the described method for constructing the manifolds is (numerically) stable.] In view of this fact, and in view of the continuity properties of G and because "transversal intersection" is a property which persists under slight perturbations, i.e. is structurally stable, we conclude that the stable and unstable manifold of the periodic orbit of period 4 intersect transversally in a homoclinic -4.1 ----~'T . . . . . . . . . . . . .
-4.2-
~-4.3 " 7 Q
the eigenvector of DG4(xa) corresponding to ~.~.
5.2. A horseshoe in an iteration of G The global stable and global unstable manifold are defined respectively as the union of all backward iterations of the local stable manifold and as the union of all forward iterations of the local unstable manifold (Guckenheimer and Holmes, 1983). Because the stable manifold is attractive under the inverse map, and the unstable manifold is attractive under the forward map, these manifolds can be computed in a numerically stable way, using the definition to construct them. In this way we constructed partially the stable and unstable manifolds of the fixed point xl of 6 4. (This is also part of the stable and unstable manifold of the periodic orbit of period 4 of G.) We iterated backwards, under G -4, three neighbourhoods S_1, So, S~; intervals on three parallel straight lines oriented along the stable eigenvector ~s, So centred on xl, S-a and $1 centred respectively on x~-As, x~+As. Analogously, we iterated forwards, under G 4,
\\\ .......
-4.5!
.......
!,
$-1 -4.5 J 6.3
6.4
6.5
X1
(!o-~)
6.fi
6.7
FIO. 3a. Local manifolds (S, stable; U, unstable; FP, fixed point). i.O
0.5-~
G(S)
o
-0,5
-i.O
-~.0
-0,5
0
1
0.5
FIG. 3b. Global manifolds (HP, homoclinic point).
~..0
Non-linear dynamics in adaptive control point. The existence of a homoclinic point established, the Smale-Birkhoff homoclinic theorem (Guckenheimer and Holmes, 1984, section 5.3) asserts that:
Conjecture 5.1.
There exists a zero-dimensional hyperbolic invariant set on which an iteration of G 4 is topologically equivalent to a subshift of finite type. []
Remark 5.2.
That an iteration of the map G is topologically isomorphic to a subshift of finite type on a zero-dimensional hyperbolic invariant set is a compact way of stating that in the dynamics of the map G a horseshoe is present, which is a prototype for chaotic dynamics. For a description of Smale's horseshoe and for a crictical look at chaos we refer to Mees (1982). A formal definition of "subshift of finite type" or "Bernoulli shift" can be found in Guckenheimer and Holmes (1983). The important geometric picture connected with the horseshoe is the stretching and folding back over itself of an area through the action of the map G. The stretching is along the unstable manifolds, whilst the bending in the present situation is done at infinity.
Remark 5.3.
It is clear from the presence of the asymptotic lines in the manifolds (Fig. 3b) that arbitrarily close to the fixed point xl there are points belonging to D c, i.e. for which not all iterations of G or its inverse are defined. More precisely, for any given e-neighbourhood of xl, there exist points x ~ D ¢, belonging to this neighbourhood for which there exist positive integers nl, n2 depending on e: x~G-n'({(~),
(~),
y~})
\t~l/z/ Remark 5.4.
The reason for Conjecture 5.1 not being promoted to the rank of theorem is that numerical evidence is utilized to support the existence of a homoclinic point, albeit persuasive numerical evidence. Further, G operates on a non-compact domain. Resolution of these technical difficulties is probably possible but would add little to our understanding. By using the mapping of (2) we may examine the behaviour of G on a compactification of R 2 where infinity is mapped to the boundary of the square. Using this device we see that, in the construction of the horseshoe using the homoclinic point theorem, the local area around the
493
fixed point is stretched to infinity before it is bent over itself. Hence the divergence (convergence) along the unstable manifold under the action of G (of G -1) or along the stable manifold under the action of G -1 (of G) must be faster than exponential. Only "locally" at xl do points separate (converge) exponentially. The phenomenon is of course due to the highly non-linear features of the map G, and its peculiar behaviour at infinity.
Remark 5.5. In a similar manner, we can proceed for the other periodic orbits. The results are the same, yielding more horseshoes for different iterations of G. 5.3. A perioddoublingrouteto chaosin G Having established the existence of chaotic behaviour in some iterations of G, and noting the symmetry of the problem and the importance of the behaviour at infinity, it is not hard to believe that G itself must be chaotic. In this section we strengthen this by looking at the bifurcation diagram for the one parameter family of maps:
G[yl~ ( Y_~. C\y2/= c + y 2
Y2 )
c+y2
; c>0.
(45)
Yl On D, Gc converges pointwise to G as c approaches zero from above: G = lim Gc I D. cJ, o
(46)
Some easily established results are: (i) The origin is a fixed point Vc > 0, which is globally attractive for c > 1 and a locally unstable node for all c < 1. (ii) For c < 1 there is an orbit of period six ( ( ~ ) ' (0a~)' (Otr) ' ( - - ~ ) ' ( L ) '
(0))(47)
~ = lX/i--~-c which locally is exponentially stable for all c : 0.40775 ~
which is initially, c close to 0.25, locally exponentially stable; then bifurcates into a saddle periodic orbit, forming two asymmetric stable periodic orbits of period 4. This orbit converges to the periodic orbit of period 4 of G.
494
I.M.Y.
MAREELS and R. R. BITMEAD
Figure 4 gives the numerically established bifurcation diagram. Horizontally, the parameter c is represented and vertically the set of stable oJ-limit is represented--as a sequence of numbers, the first coordinate of the state (e.g. the 6-periodic orbit is represented as (or, 0, -or, -a~, 0, a~}). Although this is not the complete bifurcation diagram, it certainly indicates the period doubling route to chaos. Notice that a symmetric periodic orbit first bifurcates into two stable asymmetric periodic orbits of the same period (symmetry breaking bifurcation), then for a smaller value of the parameter these asymmetric periodic orbits undergo a period doubling bifurcation. (Observable for the periodic orbit of period 6 and 4.) For small c values, after a series of period doubling bifurcations, finally chaos emerges. [This information is not obtainable from the picture in Fig. 4, due to the projection of the state onto its first coordinate; but is apparent from the way these bifurcations have to operate (Guckenheimer and Holmes, 1983) and from the simulations.] The diagram is obtained by running the difference equation for consecutive values of c, detecting periodic behaviour and then plotting out the o)-limit set for that value of the parameter. This is repeated for different initial conditions in order to obtain the two copies of every asymmetrical periodic orbit. (Because of the symmetry of the equation it suffices to take an initial condition out of the first and third quadrant in order to obtain the whole picture. Notice that the vertical axis, representing ~ , is transformed into the interval ( - 1 , 1 ) , in accordance with all other pictures.) The diagram in Fig. 4 contains the information of 1000 c-values. The only easily recognizable bifurcations are the bifurcations involving the
orbits of period 6 and 4. If we would use a finer resolution in c, and magnify the scale of both axes, the now dark patches would show similar bifurcations (symmetry breaking followed by period doubling) of periodic orbits of periods 5, 7, 11 and so on.
Remark. Although the family Gc does not depend continuously on c at c = 0 (which can best be seen from the fact that the symmetric 6-periodic orbit can not exist for c = 0, at least not with finite amplitude) we believe that this period doubling sequence indeed provides evidence for the case that G is chaotic. Remark 5.7. This sequence does provide substantial evidence for the earlier observation that all periodic orbits are unstable! 5.4. "Cycle slipping" Analysing the time sequence {v~, k e t~} [as defined by (35)] one observes that the time sequence consists of certain segments of apparent almost periodic behaviour, separated by a short transistion characterized by large deviations. (See Fig. 5, which displays a sample of the time sequence of (Vk, k • ~}.) Assuming that all periodic orbits are either of saddle type or completely unstable, of which we are strongly convinced in the light of the previous observations, this phenomenon becomes easy to understand. Orbits "close" to a stable manifold belonging to a certain periodic orbit approach this periodic orbit, hence approach the unstable manifold of this periodic orbit and are consequently repelled away from it, to be captured by a stable manifold belonging to another periodic orbit. This whole cycle keeps repeating itself. This intuitive picture, "cycle slipping", gives
a m
P 1 i t
u
d
¥
e
SEOUENC E OF PERIOD ~OU6L~NG BIFURCATIONS
o.o
o.t
o,2
o.3
0.4
o.s
o.s
o.?
o,e
o.9
t.o
parameter FIG. 4. Bifurcation diagram (SB, symmetry breaking; HB, Hopf bifurcation; PD, period doubling).
0
100
200
300
k
FIG. 5. Time sequencevx
400
500
Non-linear dynamics in adaptive control us the impression that the union of all the unstable manifolds of the periodic orbits of saddle type might be a one-dimensional strange attractor.
Remark 5.8. In principle it would be possible that there are strictly stable periodic orbits, with such a small basin of attraction that they are unobservable from a computer simulation. However, we have never established such orbits, and also the period doubling sequence seems to exclude this possibility--although this is not analytically established. 5.5. The "gumleaf attractor" Figure 6 represents the orbit of a "typical" initial condition in the phase plane. It consists of 40 000 iterations. Comparing the "curves" traced by this orbit with the unstable manifold of the periodic orbit of period 4, we find that they are virtually identical. Therefore combining, all previous observations, we conjecture:
495
previous analytical evidence points this way. (An important fact to note is that it appears as if the unstable manifold of the periodic orbit of period 4 is dense in this attractor. This is a property of an hyperbolic, connected attractor.) It even appears as if the whole of D, the open manifold on which G is defined, forms the basin of attraction of this attractor. A theorem of Sinai-Bowen-Ruelle (Guckenheimer and Holmes, 1983) states that for diffeomorphism F defined on a compact manifold possessing a hyperbolic strange attractor S there exists a measure/~, invariant with respect to the diffeomorphism, supported on the hyperbolic attractor, such that for all initial conditions x in the basin of attraction of the attractor, and for all real-valued continuous functions g the cesaro mean evaluated along the orbit generated by x exists and converges to the ensemble average of g over the attractor: liml •
N1"==/~k=0
g(Fk(x) ) =
fs
g d~.
(49)
Conjecture 5.2. The union of all unstable manifolds of the periodic orbits of saddle type form a one-dimensional hyperbolic strange attractor, denoted by S. []
In other terms F is ergodic. We conjecture that this result also holds for the present situation.
It is clear that the union of all the unstable manifolds of the periodic orbits of saddle type is a one-dimensional set, which is attracting. Moreover, if it is an attractor it is a strange attractor because it contains several transversal homoclinic orbits---and the corresponding horseshoes. The conjecture would be proven if all periodic orbits were unstable, and if all the periodic orbits of saddle type had transversally intersecting stable and unstable manifolds. The closure of the stable manifolds of the periodic orbits would then form the basin of attraction for this attractor, as well as the foliation of stable manifolds of this attractor. All numerical and
The technical difficulties--apart from those encountered for the previous conjecture--stem from the fact that it is not clear whether G can be extended to a diffeomorphism G ~ defined on a compact manifold D 1 containing D as a submanifold. If this were possible, then this conjecture follows from the previous one by the quoted theorem of Sinai, Bowen and Ruelle. Note that the "cycle slipping" idea strongly suggests that the cesaro mean of continuous functions evaluated along typical orbits should exist and be independent of the particular orbit. Indeed orbits spent most of their time in the neighbourhood of periodic orbits. Clearly, on the periodic orbits the cesaro mean is well defined. The invariant measure would then simply attribute different weights to different periodic orbits according to the relative amount of time spent by a typical orbit in their neighbourhood.
1 . 0 . ~ i
"X"
.
0.5X2
-
Conjecture 5.3. G is ergodic.
[]
0.
5.6. Implications for the closed loop adaptive system -0.5-
-! .0 -t.0
-0.5
' 0
'
'
'
I 0.5
1(1
FIG. 6. "Oumleafattractor".
1.0
Firstly, because of (9) and (10) it is clear that the parameter estimate feedback gain behaves chaotically. What does a chaotic feedback gain imply for the stability of the closed loop? Recall that
Yk = Vb VkYk-i
(50)
496
I.M.Y.
MAREELS and R. R. BITMEAD
[see (8) and (9)]. Hence, we are interested in the products of "chaotic" signals: Yk =
Vt
o.
(51)
Using the ergodicity property, we can immediately investigate the cesaro mean of the logarithm of the absolute values of the v~s; evaluating this numerically we obtain: 1
N
lim ~ k=~llog Irk I ~ ½log 2.
(52)
Remark 5.9. We have exhaustively evaluated (52) for different initial conditions and for different sample sizes. For sufficiently large N, N > 10 000 appears adequate, the same value (1og2)/2 was obtained in a "statistically" consistent way. This observation strongly supports the ergodicity property. Consequently,
lim 1 log lyN[--~½log2b.
Nr~N
(53)
We conclude therefore that Yk converges exponentially to zero for all n: 0 < b < ½, in the sense that, for almost all (Lebesgue) initial conditions, there exists a constant or> 1, independent of the initial conditions such that: o:kfykl--*O
as
kT~
(54)
and diverges exponentially for all b > ½, in the sense that there exists a constant fl < 1, such that for almost all initial conditions: flk lYkl-" + ~
as
k~'~.
o~, fl only depend on b; 1 1 < o~<~--~ ; 0 < b < ½ 1
(55)
;Tgg
< 1; ½
Remark 5.10. The present definition of exponential convergence is different from the classical definition, but is, however, frequently used in the context of stochastic processes (Bitmead, 1984). Notice in particular that (54) does not imply the existence of an exponentially decaying overbound for tY~I. Furthermore the adaptive feedback gain ak given by ~tk = a - - - Uk
[see (9) and (10)] behaves in a chaotic fashion
because v, does, hence does not yield directly any information about the system's parameters. 6. CONCLUSIONS
We have presented a careful analysis of an equation arising in adaptive control. The equation of interest describes the adaptive feedback gain used in an adaptive deadbeat controller, based on the current parameter estimates. These estimates fit a first-order model for a second-order plant. The parameter b, indicator of the model-plant mismatch, determines the topology of the dynamics of the adaptive loop. This topology varies discontinuously with b. For b = 0, no model error, the topology is trivially simple, but for b :# 0, the dynamics vary drastically: b < 0 yields limit cycle behaviour, b > 0 gives chaos. Based on the conjecture that the dynamics are ergodic for b > 0 (a conjecture we supported both with analytical and numerical evidence) we were able to characterize in the parameter space those systems for which the controller could regulate the output to zero, i.e. achieve its design objective. More information on this equation, and specifically about the implications for adaptive control are presented in Mareels and Bitmead (1986). We believe that this explicit demonstration of the inherent non-linear features of adaptive control is an initial attempt to understand the global dynamics of adaptive control algorithms; much more work remains to be done. Other researchers have independently reported other instances of chaotic behaviour in continuous-time adaptive control (Salam and Bai, 1986; Rubio et al., 1985). Our results differ considerably from these. Salam and Bai (1986) are concerned with a "Melnikov" route to chaos, due to external excitation of the algorithm, whilst in Rubio et al. (1985) the chaos is due not to the adaptive algorithm itself but rather to the non-linear features of the plant to be controlled. We have dealt with these complicated dynamics in a constructive way proving that chaos is not necessarily a destructive property. REFERENCES Anderson, B. D. O., R. R. Bitmead, C. R. Johnson, Jr, P. V. Kokotovic, R. L. Kosut, I. M. Y. Mareels, L. Praly and B. D. Riedle (1986). Stability of Adaptive Systems: Passivity and Averaging Analysis. MIT Press, Cambridge, MA. Bitmead, R. R. (1984). Persistence of excitation conditions and the convergence of adaptive schemes. IEEE Trans. Inform. Theory, |T-30, 183-191. Goodwin, G. C. and K. S. Sin (1984). Adaptive Filtering Prediction and Control, Information and Systems Science Series. Prentice-Hall, Englewood Cliffs, NJ.
Non-linear dynamics in adaptive control and P. Holmes (1983). Nonlinear Oscillations, Dynamical Systems and Bifurcations of Vector Fields, Applied Mathematics Series, Vol. 42. Springer,
Guckenheimer, J.
Berlin. Mareels, I. M. Y. and R. R. Bitmead (1986). Nonlinear dynamics in adaptive control: chaotic and periodic stabilization. Automatica, 22, 641-655. Mees, A. I. (1982). Dynamics of Feedback Systems. Wiley Interscience, New York.
497
Riedle, B. D. and P. V. Kokotovic (1986). Integral manifold approach to slow adaptation. IEEE Trans. Aut. Control, AC-31, 316-324. Rubio, F. R., J. Aracil and E. F. Camacho (1985). Chaotic motion in an adaptive control system. Int. J. Control, 42, 353-360. Salam, F. M. A. and S. Bai (1986). Disturbance-generated bifurcation in a simple adaptive control system: simulation evidence. Syst. Control Lett., 7, 269-280.