A perspective on convergence of adaptive control algorithms

Automatica, Vol. 20, No. 5, pp. 519 531, 1984

0005 1098/84$3.00+ 0.00 Pergamon Press Ltd. (~ 1984International Federationof AutomaticControl

Printed in Great Britain.

A Perspective on Convergence of Adaptive Control Algorithms* G. C. G O O D W l N , t D. J. HILLt and M. PALANISWAMIt An overview of the current status of convergence theory for adaptive control algorithms reveals that considerable unification is possible. Key Words--Adaptivecontrol; convergence;least-squaresestimation; model referenceadaptive control; parameter estimation; pole placement; recursivealgorithms; robust control; stochasticcontrol. • A convergence theory helps distinguish between good and bad algorithms. • A convergence theory suggests ways in which algorithms might be improved. For these reasons, there has been considerable research effort on the question of convergence of adaptive control algorithms. The results have been summarized in numerous surveys, papers and books. See for example Bellman (1961), Felbaum (1965), Landau (1979), Egardt (1979), Narendra and Monopoli (1980), Harris and Billings (1981) and Goodwin and Sin (1984). An adaptive controller is actually nothing more than a special nonlinear control algorithm which has been motivated by combining on-line parameter estimation with on-line control. However, the essential non-linearity of the algorithms has proven to be a major stumbling block in establishing the necessary convergence properties. In fact, given that some of the algorithms are nonlinear and timevarying and operate in a stochastic environment, it is surprising that anything substantial can be proved at all. Faced by the complexity of the convergence question, researchers initially concentrated their efforts on a restricted class of algorithms for which the control system synthesis task was essentially trivial. This class of algorithms used pole-zero cancellation methods. The algorithms have a variety of names including one-step-ahead adaptive control, model reference adaptive control and, for the stochastic case, self-tuning regulator (Landau, 1979; ,~str6m et al., 1977). Even for this simple class of algorithms, the convergence analysis proved to be very difficult. It took the combined efforts of many researchers over two decades to resolve the convergence problem. An early global convergence result for the simple class of all-pole discrete-time systemswas published by Jeanneau and de Larminat (1975). Initial proofs for general systems concentrated on continuoustime systems having relative degree (excess of poles

Abstract--This paper presents an overview of the current status of convergence theory for adaptive control algorithms. Rather than giving a comprehensive survey, the paper aims to emphasize the conceptual c o m m o n ground between different approaches. Possible areas for future research are also discussed.

1. I N T R O D U C T I O N

THE IDEA of adaptive control has its origins in the early days of control. However, it was not until the 1950s that serious attempts were made to design practical adaptive control systems (Gregory, 1959). The initial attempts were hampered by two principal difficulties: lack of suitable computer technology for implementation and absence of adequate supporting theory. With improvements in computer technology, it became feasible to experiment with various strategies for adaptive control. The methods ranged from simple gain adjustment procedures to sophisticated algorithms which attempted to achieve optimal regulation in a stochastic environment. This culminated in several successful experiments involving adaptive control. See for example/~str6m et al. (1977) and Narendra and Monopoli (1980). Our purpose in this paper is to examine the question of convergence of adaptive control algorithms. By 'convergence' we mean that the control objective is asymptotically achieved and all system variables remain bounded for the given class of initial conditions. Before going into the convergence question, we should perhaps ask, "Why is a convergence theory helpful?" This is partially answered as follows: • A convergence proof, albeit one derived under idealized assumptions, lends credibility to the practical application of the algorithm.

*Received 25 October 1983; revised 13 April 1984. The original version of this paper was not presented at any IFAC meeting. This paper was recommended for publication in revised form by guest editor L. Ljung. t Department of Electrical and Computer Engineering, University of Newcastle, New South Wales, 2308 Australia. 519

520

G . C . GOODWIN. D. J. HILL and M. PALANISWAMI

over zeros) of one (see for example Narendra and Valavani, 1978). The general case of arbitrary (but known) relative degree was resolved in the late 1970s (Feuer and Morse, 1978). This algorithm was quite complicated. However, the key point had been made that adaptive control algorithms could, indeed, be shown to converge. After the appearance of these initial results, other simpler adaptive control algorithms were shown to be globally convergent for both discrete-time systems (Goodwin, Ramadge and Caines, 1980; Egardt, 1980) and continuous-time systems (Morse, 1980; Narendra, Lin and Valavani, 1980). Also, the class of control algorithms and systems has since been expanded considerably. This paper describes some of the principal approaches that have been useful in establishing convergence of adaptive control algorithms. For simplicity, we have chosen to give emphasis to the discrete-time results though occasional reference will be made to the corresponding continuous case. Since the algorithms are invariably implemented on a computer, the discrete-time setting is more appropriate for applications. In Section 2 of the paper, we describe a basic single-input single-output model reference adaptive control convergence result. In Section 3, we show how the basic algorithm can be modified to cope with certain types of modelled uncertainty. In particular, we address the question of deterministic disturbances, bounded disturbances and stochastic disturbances. In Section 4, we discuss the performance of the algorithms in the presence of various types of unmodelled uncertainty including the common situation when the model order is less than the plant order. Section 5 is concerned with timevarying plants which actually was the original motivation for adaptive control. The above discussion has been restricted to the model reference type of algorithm. In Section 6, we turn to the question of more complicated design procedures including pole assignment and linear quadratic optimal control. We show that a difficulty with these algorithms is that it may not be feasible to evaluate the control law in certain cases. A typical situation is when the estimated model contains an unstable pole zero cancellation, i.e. the estimated model is non-stabilizable. One technique for overcoming this difficulty is to project the estimates into a region in which the model is known to be stabilizable (Goodwin and Sin, 1981). However, this means that only local convergence can be estalished. It has been known for a long time that the above difficulty would be resolved if it could be shown that the estimated parameters converged to their true values. Thus there has been considerable work on the question of parameter convergence. It turns out that once boundedness of the system variables has

been established, then it is a relatively straightforward matter to show that the parameters converge provided an external set point perturbation is applied. Using this approach parameter convergence has been established for model reference (Anderson and Johnson, 1982) and stochastic self-tuning algorithms (Chen, 1983; Chen and Caines, 1983a; Moore, 1983). However, the same approach fails for the pole-assignment type algorithms where boundedness of the system variables cannot be established independently of parameter convergence. In Section 7, we discuss some recent results which establish parameter convergence without first requiring boundedness of the system variables. These results, in turn, allow one to establish global convergence of the pole-assignment algorithms. In Section 8, we discuss various open questions and suggest some possible avenues for future research. 2. A BASIC MODEL REFERENCE CONVERGENCE RESULT The first question that must be resolved is how to parameterize the system model. Various alternatives exist, but amongst these there is a special significance attached to the observer-form state space model (Kailath, 1980). The key feature of this model is its linearity in the parameters. We shall briefly mention the more general situation later. It can be readily shown (Goodwin and Sin, 1984) that the observer form is more compactly expressed as a deterministic autoregressive moving average (DARMA) model: A ( q - ~)y(t) = B(q - ~)u(t)

(1)

where q-1 is the unit delay operator, {u(t)l, {y(t)} denote the input and output respectively and A(q - 1), B(q - 1) are polynomials in q- 1: A(q -~) = 1 + a l q - ' + . . . a , q - "

(2)

B(q - 1 ) = q - d ( b o + blq -1 + ...b,,q m);ho=~O = q-dB,(q

1).

(3)

Model reference control aims to cause the output {y(t)} to track the output {y*(t)} of a reference model: E(q-~)y*(t) = q aH(q-X)r(t)

(4)

where r(t) is a reference input. The control law which achieves the above objective is simply found by solving E(q -1) = F ( q - 1 ) A ( q -~) + q-riG(q-i)

(5)

Convergence of adaptive control algorithms where

521

(i) ]10(t) - 0o1[ ~ ]]0(t - 1 ) - 0o[1 ~ 1]0(0) F(q -~)

=.)Co + f ~ q - '

+ ...fa_tq

- 0o11; t ~ 1

(6)

-a+'

(15)

where 00 is the true parameter vector

and G(q - 1 ) = go + g l q - 1 + . . . g , - l q

(7)

-"+1.

The feedback control law is then given by

(ii)

N

e(t)2 N ~ o ~ t = l C + (a(t - 1)T~b(t -- 1) < ~ lim

where

F ( q - l)B'(q - 1)u(t) + G ( q - l)y(t) = H(q - 1)r(t).

e(t) = y ( t ) -

c~(t -

(8) To make the above control law adaptive, one simply needs to replace the true parameters by estimated parameters. A great many algorithms exist for parameter estimation. However, most algorithms are variants of the following:

[0o -

1)r0(t - 1) = ~b(t - 1) r O(t -

1)~

and this implies: e(t)

(a)

t~lim [c + qS(t - 1)r ~b(t -

1 ) ] 1/2 =

~ ,;b(t- 1) T O ( t - l ) e ( t ) 2 ~c+~t 7 i-)~(i_~i)]2

(b) lim N~,t=

0

(17)

< ~ (18)

1

Gradient scheme

N

(c) O(t) = O(t -

(16)

1) +

c + ~(t

[y(t)-

lim

I]0(t) - O(t - 1)112 <

(19)

I[0(t) - O(t - k)l] 2 < o0

(20)

N~o~t= 1

~(t-

1)

-

1)T 4,(t -

c~(t- 1)r0(t-

N

1)

(d)

1)]

N~vt=k

(9)

where

lim

(e)

l i m [10(t) - O(t - k)[[ = 0

(21)

t~ac,

O(t - 1) r = [ - y ( t

-

1)..... - y ( t

for any finite k.

- n), u ( t - d ) ,

.... u(t - d -

m)].

(10)

(i) Subtracting 0o from both sides of (9) and using (1) and (10):

Least squares scheme O(t) = O(t -

Proof

1) + P ( t - 1)qS(t - 1) [ y ( t ) - ()(t - 1 ) r o ( t - 1)]

(11)

0(t) = 0(t -

l) -

4,(t - 1) c + 4,(t - 1)r O(t -

where

4,(t - 1) r 0(t - 1) 1)=P(t-2) P ( t - 2)q~(t - 1)~b(t - 1) T P ( t - 2) I + ~b(t- 1) TP(t - 2)~b(t - 1)

P(t-

where 0(t) = O(t) - 0o. Hence (12)

The convergence properties of the above parameter estimators can be studied using a Lyapunov type analysis based on the following functions. Gradient

V l ( t ) = (O(t) - Oo)r(O(t)

- 0o).

[lO(t)l[2 I_2

-]lO(t

4

-

e(t) 2 c+~b(tI) TqS(t- 1)"

(22)

N o w since c > 0, we have

V2(t) = ( O ( t ) - 0o) T P ( t - 1 ) - ~ ( 0 ( t ) - 0o). (14) As a concrete example of the convergence analysis we give the details for the gradient scheme. 1980) For the algorithm (9) applied to the system (1) it follows that 1 (Goodwin, Ramadge

1)]l 2 =

qS(t - 1)T~b(t -- 1) ] c + ~(t - l) ~ ~(t - 1)

(13)

Least squares

Lemma

t)

and Caines,

-2 + c

~b(t- 1 ) r ~ b ( t - 1) ] -li
(23)

and then (15) follows from (22). (ii) We observe that I[0(t)[[2 is a bounded nonincreasing function, and by summing (22) we have

522

G . C . GOODWIN, D. J, HILL and M. PALANISWAM| (ii)

II•(t)ll 2 = IIO(O)1t~ +

•

N

~(t-

l)r G ( t - 2 ) 0 ( t - 1)e(t):

j--1

I

-2+

oh(J- 1) ~b(j--1)

126t

]

Off) lira II0(t) -

etj) 2

O(t -

k)ll = 0 for any finite k. (27)

t~

c ÷ OIJ - lIT O i j -

1)

Since 110(0112 is non-negative, and since (23) holds,. we can conclude (16). (a) (17) follows immediately from (16). (b) Noting that e(t) 2

The above results follow very easily once the algorithm has been expressed in the error model form: O(t) = O(t -

1) +

GO(t

-

l)L{q~(t -

l)rO(t)l

(28) where

e + qS(t -- 1) T ~b(t - - 1) [c + ~ t t -

1) T ~ ( t -

1)3e(t) 2

[c + ~b(t -- 1)r~b(t - 1)3 2 we establish (18) using (16). (c) (18) immediately implies (19) by noting the form of the algorithm (9). (d) It is clear that: IlO(t) -

O(t -

l) -

+ O(t -

k)l]z = I]0(t) 2)...

O(t -

O(t -

O(t -

1) 1) -

k +

O(t -

k)llL

Then, using the Schwarz inequality II0(t) -

O(t -

k)llz <

k(llO(t) -

O(t -

1)112 + ...

+ II0(t - k + 1) -

O(t -

k)ll2).

Then, the result follows immediately from (19) since k is finite. (e) Equation (21) follows immediately from (20). [] By a similar argument, corresponding properties can be derived for the least squares algorithm. In summary, we have the following key properties: Parameter

estimator

properties

N

e(t)2

lim ~, <~ N.... ~=1 c + ~b(t - 1)rG(t -- 2)~b(t - 1) where c = 1 and

G(t -

2) =

c > 0 and

G(t -

2) = I

e(t) = y(t) -

49(t - -

P(t -

(24)

2) for least squares for gradient.

1)r0(t - 1).

Equation (24), in turn, implies (i) there exists c1, lim

t~C

cz

> 0 such that e(t) 2

1 + ezflp(t-

l)r~(t-

1)

= 0

(25)

0(t) = 0(t) - 0o.

(29)

For the above algorithms, Lis a positive scalar. In addition, G is a positive scalar for the gradient scheme and G is a time-varying positive definite matrix for the least squares scheme. These two cases cover many of the algorithms that have been shown to be convergent in the literature. A third and final variant is where L is a positive operator (i.e. satisfies a positive real or passivity condition). The latter form is useful in output error algorithms (Landau, 1979), stochastic algorithms (Ljung, 1977) and model reference control algorithms involving filtering (Narendra and Lin, 1980). The corresponding continuous-time parameter updates (Morse, 1980; Narendra, Lin and Valavani, 1980) have a similar structure to (9) and (11) except that O(t) - O(t - 1) is replaced by O(t). To be more precise about how these algorithms fall into the above error model format, we have: • The discrete-time algorithms of Goodwin, Ramadge and Caines (1980) where L is a positive scalar and G is a positive scalar or matrix. • The continuous-time algorithm of Morse (1980) where G and L are positive scalars. (This is achieved by pre-filtering the regression vector.l • The continuous-time algorithm of Narendra, Lin and Valavani (1980) where G is a positive scalar and L is a positive operator. (Thus a positive real condition on a filter of the users choice appears as a condition for convergence.) • The output error algorithms of Landau (1979) where L i s a positive operator which turns out to be the system denominator polynomial. (Again a positive real condition appears.) • The self-tuning regulator algorithm of Astr6m and Wittenmark (1973) which basically has the same format as the output error algorithm but with L related to the Kalman Filter for the system; see Egardt (1980a) for a discussion of the relationship between these algorithms.

Convergence of adaptive control algorithms To show how the properties of (15)-(18) are related to the convergence of adaptive control algorithms, we need to add the determination of the feedback control law to the overall algorithm. Here the Certainty Equivalence Principle (Bar-Shalom -and Tse, 1974) is employed, i.e. O(t) is used to synthesize the control law as if it were the true parameter vector. For the control law (5), (8), this gives: E(q -~) = f(q-a,t)~4(q-~,t) + q-dG(q-~,t)

(30)

f ( q - 1, t)13'(q- 1, t)u(t) + G(q- 1, t)y(t) = n ( q - 1)r(t)

(31) where ,4(q-1, t),/~(q-1, t) are obtained from O(t). Our next step is to investigate the combination of (21 ) and (22), together with the estimator (9) or ( 11 ). It can be seen from (25) that, if the system variables are bounded, then so is {¢(t)} and convergence of e(t) to zero follows immediately. Thus we see that the question of boundedness and convergence are intimately connected. One way of relating the questions ofboundedness and convergence has been through the following Key Technical Lemma. Lemma 2 If the following conditions are satisfied for some given sequences {s(t)}, {a(t)}, {bl(t)}, {bz(t)}: (i)

lim b

,~

t

s(t):

x( ) + b2(t)a(t)r a(t)

= 0

and

(ii) lim [y(t) - y*(t)] = 0

(iii) Linear boundedness condition

Ila(t)ll _< Ca + C2 max Is(z)[

(35)

t~o0

where y*(t) is the output of the reference model. Proof To manipulate the time-varying operators it is convenient to define A/3 and A.B as follows: AB = ~ ~ (li(t)bs(t)q-' -J =/3.4

(36)

i )

(37)

A'B=~fi(t)bs(t-i)q-'-J~B'A. ij

We also define /~ =/~(t - 1, q- ')

(38)

/3 =/~(t + d - 1,q-a).

(39)

The model prediction error is given by

(40)

= ~ y ( t ) - ~ ' u ( t - d).

for all t _> 1.

(34)

O~<'r_
<

(i) {u(t)}, {y(t)} are bounded for all time

O < bz(t) < K <

(33)

where 0 that

Theorem 1 Provided the projection or least squares algorithm is used to generate O(t) and provided the system is stably invertible, then the indirect model reference adaptive control algorithm is globally convergent in the sense that

e(t) = y ( t ) - ¢(t - 1)rO(t - 1)

(ii) Uniform boundedness condition O < b~(t) < K < ~

The utility of the above Lemma becomes evident when we compare equation (32) with (25). To show the application of the above Lemma to convergence of adaptive control algorithms we consider the indirect model reference adaptive controller defined by equations (9) [or (I1)] and (31).

(32)

where {bl(t)}, {bz(t)} and {s(t)} are real scalar sequences and {a(t)} is a real (p x 1) vector sequence.

523

C 1 < (x], 0 < C2 < oo, then it follows

We shall now derive a model for the closed loop system. Recall that when the system parameters are known then the closed loop system is given by Ey(t + d) = Ey*(t + d) and EB'u(t) = EAy*(t + d). Substituting (31) and (30) into (1) and using (40) we immediately obtain e + [f~

- f A]

A ' [ F A - FA]

(a) l i m s ( t ) = 0 t~oC

- [f.B' - fB']

EB' - A . [ f . B ' - fl~ y(t + d)]

,

]J

u(t) J

and =[

(b) {ll~r(t)ll} is bounded. Proof See Goodwin, Ramadge and Caines (1980). []

Ey*(t+d)+Fe(t+d) I EAy*(t + d) + A.Fe(t + d) "

(41)

Equation (41) can be regarded as a linear timevarying system having inputs {e(t)}, {y*(t + d)} and

524

G . C . GOODWlN, D. J. HILL and M. PALANISWAMI

outputs {u(t)l, {y(t)}. The terms in e(t) arise due to the modelling error and the terms in square brackets, e.g. [ F . / ~ ' - F/3'] etc. arise due to the time-varying nature of the parameter estimates. Now the standard parameter estimation algorithms have the following properties [see (24) to (27)]: (i)

,4,/~ bounded for all t.

(ii) H0(t + k) - 0(t)ll 2 ~ 0 for all finite k. e(t) 2

(iii) c + 4~(t - 1)rqS(t - 1) ~ 0 . Property (i) also implies that/~, (~ are bounded since (30) is solvable for a n y / t . Property (ii) ensures that the model (41) is asymptotically time-invariant and convergent toward a stable system provided E-~ and B'-~ are both stable. This implies, using (41) that, {u(t - d)} and {y(t)~ are asymptotically bounded by {e(t)}. [This is heuristically reasonable and a full proof can be found in Goodwin, Hill and Xie (1984). Related results are also given in Fuchs (1982). ] Thus we can apply the Key Technical Lemma to show using property (iii) that e(t) converges to zero and that {~b(t)} is bounded. This establishes the theorem. [] The above theorem applies to a particular algorithm but it turns out (Goodwin and Sin, 1984) that the general approach used is applicable to a wide range of algorithms. The basic idea is to exploit the properties of the parameter estimation algorithm via the Key Technical Lemma together with the equations for the closed loop to prove convergence. Thus the proof of Theorem 1 provides a paradigm for convergence proofs of many algorithms. It is also possible to see why the model reference control algorithm is so simple. The basic point is that the control law depends only on solving equation (30) which can always be done without difficulty (by simple back substitution). The above algorithm is called an indirect adaptive control law. One finds the control law (indirectly)by first estimating the system parameters and then using these to evaluate the feedback control law. It is also possible to substitute (5) into the system equations and estimate the coefficients in the feedback law (8) directly. This gives a direct model reference adaptive control law. Again global convergence can be established by a straightforward argument (see, for example, Egardt, 1980b; Goodwin, Ramadge and Caines, 1980; Narendra and Lin, 1980). The above arguments extend in a relatively straightforward fashion in the multi-input multioutput case save that some purely system theoretic issues have to be resolved. In particular, the

generalization of the concept of relative degree (delay in the discrete case) turns out to be a matrix known as the interactor matrix (Wolovich and Falb4 1976). This matrix contains non-integer parameters and some current research is aimed at showing how these parameters can be estimated alongside the other system parameters (Johansson, 1983 : Dugard~ Goodwin and de Souza, 1983). Another direction of current research has been to combine discrete parameter estimation with continuous control. This leads to the so-called hybrid model reference adaptive control schemes (Elliott, 1982; Narendra, Khalifa and Annaswamy, 1983; Gawthrop, 1980). Again, global convergence can be established along the lines outlined above. Before concluding this section, we make some general comments on the convergence proofs found in the literature. The stability results reported in Landau (1979) use feedback system stability theory based on concepts of hyperstability (Popov, 1973) to show asymptotic zero tracking error if system inputs and outputs are bounded. Following these results, Narendra and Valavani (1978) had given a global convergence result in the simple case of unity pole-zero excess. The globally convergent algorithms of Feuer and Morse (1978), Egardt (1980b), Goodwin, Ramadge and Caines (1980), Narandra, Lin, Valavani (1980) and Morse (1980) are all based on a study from first principles of the equations for the parameter estimator and controller. However, recent work has shed some light on the role which conventional stability theory can play in convergence results. Kosut and Friedlander (1982) have used the input-output stability theory setting (Desoer and Vidyasagar, 1975) to give a general stability result for model-reference control systems in terms of passivity restrictions. Johansson (1983) uses Lyapunov stability theory (Hahn, 1963) to show uniform global stability. That hyperstability, Lyapunov stability, and input output properties of passivity and finite-gain should all play some role in convergence theory is not surprising in view of their general close inter-relationships (Hill and Moylan, 1980) and roles in the general theory of feedback system stability (Vidyasagar, 1978). The above discussion has been based on rather idealized assumptions (known upper bound on model order, no disturbances, time invariant systems, etc.). In the next few sections, we will briefly discuss the convergence of the algorithms under more realistic conditions. 3. C O N V E R G E N C E IN THE PRESENCE OF NOISE

The basic convergence analysis described in Section 2 can be readily extended to include external disturbance or noise. In particular, we discuss deterministic disturbances (such as periodic signals), bounded noise and random noise.

Convergence of adaptive control algorithms 3.1. Deterministic disturbances One way of modelling a deterministic disturbance is as a sum of a finite number of sine waves, e.g. l

d(t) = ~ G i sin (pit + ¢~).

(42)

i=1

We next note that the disturbance (42) can be modelled by an observable state space model having 21 uncontrollable roots on the unit circle (at cosp~ +jsinpi). The corresponding DARMA model is

D(q- 1)d(t) = 0

(43)

to be predictable. In some cases, it may be desirable to have the algorithm of Section 2 handle disturbances of quite arbitrary waveform. This can be readily done provided the disturbance is bounded. The idea is to simply turn the parameter estimator off when it is likely to be overly 'confused' by the noise. Inspection of the proof in Section 2, shows that if the noise satisfies sup Id(t)[ < A

O(t) = O(t - 1) +

D(q -1) = H (1 - (2cospi)q-' + q-2). (44)

(46)

where

A ( q - ' ) = A ( q - ' ) O ( q - ' ) ; B ( q - ' ) = B(q-1)D(q-'). (47) The algorithms of Section 2 and the corresponding convergence theory applies without change to this new situation. The key point to note is that, subject to the usual minimum phase assumption on B(q- 1), the growth of {u(t)} is still bounded by the growth of {y(t)}. The polynomial D(q -x) simply introduces uncontrollable modes on the unit circle of Jordan block size I (Willems, 1970). The only change necessary in the algorithm is to increase the dimension of the parameter vector to accommodate ,~(q- 1) and B(q - 1). An interesting interpretation of the model reference controller in this case is that the feedback renders the uncontrollable disturbance modes unobservable at the output. Global convergence can be established exactly as in Section 2 (Goodwin and Chan, 1983). 3.2 Bounded noise The convergence analysis of Section 3.1 showed that disturbances can be removed from the output. This follows because the disturbance was assumed AUTO 20:5-C

O(t -

1)TO(t

-

1)3,

(49)

where

a ( t - I ) = {10 if ly(t)otherwise.-¢(t- 1 ) T o ( t - 1)l > 2A

(50)

(45)

Now, suppose the system output in (1) is corrupted by a disturbance of this type. Then the input-output behaviour is describable by the following (observable, but uncontrollable) model:

A(q- 1)y(t) = B(q- 1)u(t)

a(t1 ) q ~ ( t - 1) c + q~(t - 1)rq~(t - 1) [.V(t) -

i=1

D(q -1) = 1 - q-P.

(48)

then the appropriate modification to the algorithm is to replace (9) by

where

A special case of (42) is when the disturbance has period p, but is otherwise arbitrary. In this case, the model is as in (43) with

525

Subject to the usual minimum phase assumption, the algorithm (49) can be easily shown (Egardt, 1979; Samson, 1983; Goodwin, Long and Mclnnes, 1980; Peterson and Narendra, 1982; Kreisselmeier and Narendra, 1981; Martin-Sanchez, 1982) to be globally convergent by the same basic argument as that used in Section 2. The final tracking error, e(t), satisfies limsupe(t) < 2A.

(51)

3.3 Random noise It is also possible to model the noise as a stochastic process. The key idea used here is that the noise is predictable up to an unpredictable residual (the innovations). The deterministic predictor (1) should now be replaced by the optimal stochastic predictor based on the Kalman Filter (Anderson and Moore, 1979). This predictor gives the input-output (ARMAX) form:

A(q- 1)y(t) = B(q- 1)u(t) + C(q- a)w(t),

(52)

where {w(t)} is the innovations sequence ('white noise') and C(q -1) denotes the denominator polynomial of the steady state Kalman Filter and therefore satisfies (Anderson and Moore, 1979): IC(z-a)[# 0

for

Izl_> 1

(53)

where z is the z-transform variable. A difficulty with the model (52) is that, in the adaptive case, both the parameters and the system states (past optimal predictions) will be unknown.

526

G . C . G O O D W I N , D. J. HILL and M. PALANISWAMI

Thus one must combine both parameter and state estimation. One method of doing this is to base the state estimate on the current parameter estimates and vice versa. This kind of'boot-strap' procedure is commonly called a Pseudo Linear Regression. An identical procedure is used in the output error algorithm which was briefly mentioned in Section 2. For the gradient algorithm, the algorithm takes the form:

C ( q - ~ ) arises as part of the convergence condition. The need for this condition was clarified in the comprehensive studies of Ljung (1977). Various extensions of the above result are also possible. For example, it turns out that the positive real condition can be slightly weakened by replacing the (a p r i o r i ) state estimates given in (58) by the following (a p o s t e r i o r i ) state estimates rl(t) = y ( t ) - 49(t -

O(t) = O(t -

1) +

4,(t - r(t

1~Ev(t)~ -4(t1)

1t ~ 1)3

O(t-

(54)

where 1)=r(t-2)+ck(t-

r(t-

1)r~b(t-- 1) (55)

0(t)= [61 ..... a,,/~o ..... /~,.,01.... ,0,].

(56)

The regression vector 4~(t - 1) is now a function of past outputs, past inputs and past state estimates, i.e. 0(t

-

1) r =

[-y(t

-

1) ....

. . . . u(t - d - m), e(t -

,-y(t

-

n),

1). . . . , e ( t - n)

(57)

where e(t) denotes the estimate of w ( t ) based on the estimated parameters, i.e. e(t) = y ( t ) -

1)w 0(t - 1).

O(t -

(58)

A very simple calculation shows that the prediction error e ( t ) is related to the parameter error O(t) = O(t) - Oo as follows: C(q-l)e(t)

= -q6(t

-

1 ) T 0 ( t - 1).

(59)

The convergence analysis follows similar lines to that in the deterministic case but is more difficult due to the need to do combined parameter and state estimation. Important insights and tools for dealing with this problem have been developed by many authors, in particular Tsypskin (1971), Lj ung (1977) and Solo (1979). As in the deterministic case, convergence is intimately related to boundedness of system variables. The Lyapunov type argument in the deterministic proof is replaced by the corresponding stochastic argument using the Martingale Convergence Theorem (Hall and Heyde, 1980). Also a stochastic counterpart of the Key Technical Lemma is used (Goodwin, Ramadge and Caines, 1981). The presence of the filter C ( q - 1 ) in (59) means that the convergence proof has much in common with the deterministic proofs discussed in Section 2 which incorporate some form of data filtering. In particular, a positive real condition on

1)r 0(t).

(60)

It is also possible to develop a least squares form of the algorithm. Convergence of this latter algorithm has also been established subject to a minor modification to ensure that the condition number of P ( t ) remains bounded (Sin and Goodwin, 1982). It has recently been shown that the convergence properties are retained if the steady state model (52) is replaced by the time-varying true Kalman Filter model (Goodwin, Hill and Xie, 1984). The key observation here is that the Kalman Filter converges exponentially fast to the steady state solution. 4, ROBUSTNESS TO MODELLING ASSUMPTIONS

A feature of all the results described above is the assumption that the system is modelled exactly by the model used in the adaptive algorithm. Of course, all real systems are essentially infinite dimensional. Thus it is usually impractical to use the true system model. Straightforward application of the algorithms in Section 2 can lead to difficulties if the model order is underestimated (Rhors e t at., 1982). It appears that the parameter estimator can drift into unacceptable regions leading to difficulties in convergence. This problem has led to recent interest in the question of the robustness of the algorithms to the modelling assumptions. Once it is realized that an adaptive control algorithm is nothing more than a rather special nonlinear feedback controller, then it is clear that one ought to be able to analyse some of the robustness questions formulated above in terms of recent robust stability results including the work of Doyle and Stein (1981), Safonov, Laub and Hartmann (1981), Cruz, Freudenburg and Looze (1981), etc. which appeared in a recent special issue of the I E E E Transactions.

Various approaches suggest themselves including: • Robust input-output results (passivity, small gain and conic sector results). • Robustness based on exponential convergence (Lyapunov stability). • Singular perturbation analysis.

Convergence of adaptive control algorithms Each of these methods is currently under study though the analysis appears to be in a relatively early stage. Kosut and Friedlander (1982) have introduced the concept of a tuned parametric model. This is the best simplified model that can be matched to the input-output data. The normal stability conditions are imposed on the tuned parametric model and certain restrictions are imposed on the uncertainty. An additional assumption is made which effectively demands that the tuned system be robust. Anderson and Johnson (1982) take the view that if the control system can be made exponentially stable then robustness is guaranteed. Their results are dependent on an assumption of persistent excitation. An interesting result relevant to this approach is given by Johansson (1983) for model reference control. By restricting the plant zeros and closed loop poles to be well-damped, exponential convergence is achieved without persistent excitation. Ioannou and Kokotovic (1982) suggested that a modification to the parameter update law can give tracking to within a uniform bound in the presence of unmodelled dynamics. One difficulty is that the analysis to date has only been applied to the simplest model reference adaptive control algorithm using a gradient parameter estimator. Inherent limitations are present. The model reference controller is fundamentally non-robust (Francis and Wonham, 1976) (even when the true parameters are known) since it incorporates feedforward terms. Also, the gradient procedure is known to converge slowly. Thus it would seem that more work is needed on the robustness question from two points of view. Firstly, attention should be given to adaptive (versions of) robust control for the algorithms. Then these algorithms could be subjected to robustness analysis along the lines mentioned above. 5. TIME-VARYINGPLANTS It can be argued that every system is timeinvariant if only one could find a suitably complex non-linear model. However, from a practical point of view it is often convenient to work with a simplified model. If this is done, then the 'best' parameters in the simplified model will change with operating condition. This suggests that adaptive control might be helpful in updating the model and hence the feedback control law. Indeed this probably was the principal motivation for initial work in adaptive control. It is therefore of considerable importance to investigate the performance of adaptive controllers when the model is time-varying. The gradient algorithm of Section 2 is, in principle, suitable for time-varying systems since its gain does not go to

527

zero. However, in practice the algorithm can converge slowly since it takes no account of the curvature of the sum-of-squares error surface. On the other hand, least squares is essentially a Newton algorithm and uses curvature information. However, the basic algorithm as given in (11) and (12) has the undesirable property that the gain goes to zero and thus the algorithm effectively turns itself off. Thus it is clear that some modifications of the algorithms of Section 2 are necessary for practical use on time-varying systems. Also, it seems that it may be better to concentrate on the least squares algorithm since it uses curvature information. Two possible strategies for stopping the gain going to zero have been described in the literature: (1) discard past data exponentially or (2) periodically reset the P matrix to either a fixed initial value or a value depending on the latest P matrix. The first of these suggestions has been frequently used. The resulting algorithm is very simple. All that is necessary is to replace (12) by 1 P(t - 1) = ~ [P(t - 2)

2 ) ~ ( t - 1)~b(t _- 1)r_P(t Z_2)] 1 + ~-(f~ 1 - ) r ~ - 2)~b(t - 1) J

P(t-

where

0<2<

(61)

1.

However, this modified algorithm has a potential difficulty for practical use. As soon as the system settles down, P(t - 1) begins to grow exponentially and thus the gain of the parameter estimator becomes very large. This can be seen by putting ~b(t - 1) = 0 in (61). When the gain of the estimator grows, the parameters drift and ultimately the adaptive control law will fail. One way of avoiding this is to place an upper limit on the P matrix. However, a better suggestion seems to be to make 2 a function of the data. This leads to the so called 'variable forgetting factor' algorithms (Fortescue, Kershenbaum and Ydstie, 1981). The key idea here is to keep 2 near unity save when the prediction error is large indicating that a significant change in parameters has occurred. Then 2 is pulled down for 1 to 2 steps. This has a similar effect to simply resetting the P matrix as suggested earlier. An alternative approach is to keep the trace of P constant by adjusting 2 as discussed in Lozano Leal and Goodwin (1984). Having thus motivated various algorithms which seem potentially useful in the time-varying case, the next question is to ask, "What can be said about their convergence properties?"

528

G . C . GOODWIN, O. J. HiLL and M. PALANISWAMI

An initial step would seem to be to show that the algorithms, designed with time-varying systems in mind, would at least converge in the time-invariant case. This has been verified for the variable forgetting factor algorithm (Cordero and Mayne, 1981 ), the resetting algorithm (Goodwin, Elliott and Teoh, 1983) and the constant trace algorithm (Lozano Leal and Goodwin, 1984). The tools used to perform the analysis are exactly those described in Section 2 because the system is presumed to be timeinvariant for the purpose of the analysis. To make progress on the genuine time-varying case, it seems necessary to postulate a model for the nature of the time variations. In fact, Caines and Chen (1982) have rigorously shown that arbitrary parameter variations cannot be dealt with. In developing appropriate models for time-varying parameters, it should be borne in mind that an implicit assumption to date in adaptive control has been that the parameter estimates may be used as if they are the true time-invariant system parameters (certainly equivalence). This philosophy implies that the bandwidth of the parameter variations should be less than the desired bandwidth of the closed loop system. To develop convergence results, it is clear that the parameters should have some pseudo-time-invariance property. This suggests the following two dual models for the parameter time variations: (1) Large changes infrequently ('jump parameters'). (2) Small changes frequently ('drift parameters'). Some preliminary results are available on the convergence of adaptive control algorithms with this type of parameter variation. For example, the robustness results described in Section 4 immediately give insights on the 'drift parameter' case. In particular, Anderson and Johnstone (1983a) have argued that an external set-point perturbation gives exponential parameter convergence which makes the controller inherently robust to drifting parameters. A related idea is described in Goodwin and Teoh (1983) where it is shown that an external signal helps to keep the parameter estimates close to the true system parameters. Another useful idea that has emerged from recent convergence studies is that there is an advantage in updating the control law less frequently than the sampling rate (Anderson and Johnstone, 1983; Elliott, Cristi and Das, 1982; Goodwin and Teoh, 1983i. Apart from the convergence analysis question, this seems to have the desirable heuristic advantage that it separates the bandwidth of the control law up-date from the bandwidth of the system. The stochastic time-varying case is, of course,

technically difficult. However, important preliminary insights have been obtained by several authors. For example, Chen and Caines (1983b) establish global convergence for a stochastic adaptive control algorithm when the parameter variations are modelled as a bounded martingale difference sequence. A related result is given in Goodwin, Hill and Xie (1984) where convergent deterministic parameter variations are considered. 6. ADAPTIVE POLE ASSIGNMENT So far we have limited our discussion to the model reference type control algorithms. The key point about these algorithms is that the control system design is very easy [see for example (5) and (6)1. However, the price one pays for this is inherent lack of robustness and a requirement that the plant be minimum phase. In this section, we will therefore turn to alternative design procedures which overcome these limitations. The two algorithms which suggest themselves are pole-assignment and linear quadratic optimal control. Each of these has very similar properties from a convergence viewpoint. We will therefore illustrate the ideas by reference only to the pole-assignment case. If we assume that A(q ~), B(q -1) are relatively prime then the closed loop poles can be arbitrarily assigned by solving:

A(q-1)L(q 1) + B(q-1)p(q-~)= A,(q-~)(62) where A*(q-l) is an arbitrary stable polynomial. The feedback control law can then be implemented as

L(q-~)u(t) = P(q l)Ey*(t) - y(t)l.

(63)

In fact, model reference control is a special case of this algorithm when A*(q -1) is chosen as

E(q-1)B,(q- 1). An indirect adaptive form of the above control law can be readily conceived by simply replacing A(q-1), B(q -1) in (62) by the estimators obtained from (9) or (11). Convergence for this algorithm can be established along exactly the same lines as the argument used in Section 2 with one major difference. The problem is that, unlike (5) which is always solvable, (62) only has a solution when A(q-1), B(q-1) are relatively prime, i.e. the convergence analysis depends upon showing that all limit points of the parameter estimator correspond to models in which A(q -1) and B(q-1) are relatively prime. One way out of the problem is to presume that one knows a closed convex set C having the following properties Oo~C C ~ {O:A(®,q 1), B(®,q 1) are relatively prime~,. (64)

Convergence of adaptive control algorithms If C is known, then the parameter estimation algorithm can be modified so as to ensure ~ C. Then the convergence properties, (15)-(18), are retained. In the case of the gradient algorithm, all that is needed is to orthogonally project ~ onto the surface of C if ~ is outside C. A similar procedure applies to the least squares algorithm. With this modification it is straightforward to establish convergence of the adaptive pole assignment algorithm (Goodwin and Sin, 1984). Note, however, that the result is implicitly local in nature since sufficient a priori knowledge about Go is necessary to be able to construct a suitable region C. In the next section, we will suggest an alternative strategy which leads to global convergence of the adaptive pole assignment algorithm. This requires the addition of a 'persistently exciting' external input to ensure that the parameters converge to their true values and thus to asymptotically avoid pole-zero cancellations. There still remains some interest in the conceptual question of whether or not global convergence can be achieved without persistent excitation. One interesting approach pursued by Praly (1982, 1984) is based on simultaneous input and output prediction models and leads to a bilinear parameter estimation problem. An alternative approach avoids pole-zero cancellation by using a finite search in a particular subspace defined by the parameter estimator (Lozano Leal and Goodwin, 1984). 7. PARAMETERCONVERGENCE Note that the convergence proofs given earlier said nothing about ~(t) converging to G0. In fact, it has not been claimed that {~(t) converges to anything. In general it is an open question to state conditions under which parameter convergence occurs, An interesting result in this direction has been described by Kumar (1984) who shows that for regulation about a zero desired output the parameter estimates converge to a fixed multiple of the true parameter values. In the deterministic case, once boundedness of the system input and output has been established, then it is possible to ensure that {~(t) does indeed converge to Go by applying an external set point perturbation (Anderson and Johnson, 1982). The stochastic case is more difficult, but parameter convergence has been established by Chen (1983) (for the least squares algorithm) and Chen and Caines (1983) (for the stochastic gradient algorithm). As mentioned previously, one side advantage of having parameter convergence is that this can improve the robustness of the algorithm to unmodelled disturbances and time variations. Unfortunately, the above analysis does not allow one to estalish convergence of the pole-assignment

529

adaptive control algorithm. The problem is that boundedness of inputs and outputs must be shown first. Recent research has shown that this problem can be resolved by establishing parameter convergence without requiring a priori bounds on the system variables. Four recent reportings of results of this nature are: (1) Anderson and Johnstone (1983b) who use a complex sequence of control law settings to ensure convergence. (2) Elliott, Cristi and Das (1982) who show that parameter convergence occurs for a direct pole assignment algorithm by making the set point sufficiently rich. (3) Goodwin and Teoh (1983b) who establish a general convergence result for parameter estimates in the presence of possibly unbounded signals. (4) Trulsson (1983) who analyses an instrumental variable scheme and establishes parameter convergence with possibly unbounded noisy signals. Once the potential circular argument between boundedness and parameter convergence has been broken, then it is possible to guarantee that the poleassignment adaptive control algorithm does, in fact, ensure boundedness of the system variables. 8. SOME OPEN PROBLEMS In the previous sections, we have given an overview of the current status of convergence of adaptive control algorithms. Some remaining interesting problems appear to be: • Further robustness results as indicated in Section 4. • Algorithms which apply to systems which are nonlinear in G. This problem occurs quite frequently. For example, when prior knowledge dictates a particular model structure. Even a linear transfer function is nonlinear in Go. EIn the stochastic case, an appropriate algorithm would appear to be Recursive Prediction Error Methods (Ljung and S6derstriSm, 1983) since this algorithm does not require linearity in G. ] • Further analysis of stochastic time-varying problems. (Note that most of the existing stochastic analyses are unsuitable because the gain of the parameter estimator goes to zero. This ensures almost sure convergence in the presence of random disturbances but this is clearly unattainable in the time-varying case.) • Extension of the adaptive pole-assignment algorithm to include deterministic disturbances as was done in Section 3.1 for the model reference algorithm. • Design of simple controllers, e.g. PID, for high

530

G . C . GOODWIN, D. J. HILL and M. PALANISWAMI

order plants. (Some preliminary results in this direction are given in Ramadge and Goodwin, 1979; Ljung and Trulsson, 1981.) • Adaptive controllers for decentralized situations where each controller has only partial information. (Some interesting preliminary work is described in loannou and Kokotovic, 1983.) • Adaptive solution of nonlinear optimal control problems including two point boundary value problems. (A practical case of this was recently brought to the authors' attention by Harrison, 1983. The problem involved the control of a Basic Oxygen Furnace in steelmaking to achieve a desired end product over a fixed time interval with unknown initial state and unknown and variable plant parameters.) • Rates of convergence. (Very little is known about the rate of convergence in terms of system properties.) • Adaptive control in Markov decision processes. (The paper has not so far referred to work done in this area. Certainty Equivalence Control has been discussed in Borkar and Varaiya, 1979; Kumar, 1983.) There would seem to be scope for further developments in the convergence theory of these algorithms including interconnecting the theory with that described in Section 2 for ARMA type models. • Adaptive Control algorithms which use nonlinear dynamic models, e.g. bilinear models. (Some preliminary results in this direction are given by Goodwin, McInnis and Long, 1982.) • The incorporation of a priori plant knowledge into the algorithm form rather than just via its initialization. 9. C O N C L U S I O N S

In this paper, we have concentrated on those aspects of the convergence question which have a personal interest for us. This is not meant to imply that those aspects which have not been treated are any less interesting or important. We feel it is worth pointing out that convergence of adaptive controllers is only one aspect of the design philosophy. To give an analogy, the Principal of the Argument was crucial in developing the Nyquist stability criterion and the associated classical frequency domain design methods. However, real design using these methods also relies upon a vast body of experience. In the same way, convergence of adaptive control algorithms can, at best, suggest simple prototype strategies. The rest is up to the ingenuity of the designer and is almost certainly problem specific. REFERENCES Anderson, B. D. O. and C. R. Johnson, Jr. (1982). Exponential convergence of adaptive identification and control algorithms. Automat(ca, 18, 1-14.

Anderson, B. D. O. and R. M. Johnstone (1983a). Adaptive systems and time varying plants. Int. J. Control, 37, 367 377. Anderson, B. D. O. and R. M. Johnstone (1983bj. Global adaptive pole positioning. Tech. Report, Department of Systems Engineering, Research School of Physical Sciences, Inst. of Advanced Studies, Australian National University. Anderson, B. D. O. and J. B. Moore 11979). Optimal Filtering, Prentice-Hall, New Jersey. AstriSm, K. J., U. Borisson, L. Ljung and B. Wittenmark 119771. Theory and application of self-tuning regulators. A ulomatica, 13, 457 476. Astr/Sm, K. J. and B. J. Winenmark (1973). On self-tuning regulators. Automatic& 9, 195 199. Bar-Shalom, Y. and E. Tse (1974). Dual effort, certainty equivalence and separation in stochastic control. IEEE Trans Aut. Control, AC-19, 494 500. Bellman, R. (1961). Adaptive Control Processes: A Guided Tour. Princeton University Press, New Jersey. Borkar, V. and P. garaiya (1979). Adaptive Control of Markov Chains, I: Finite parameter set. IEEE Trans Aut. Control, AC24, 953 957. Caines, P. E. and H. F Chen (1982). On the adaptive control of stochastic systems with random parameters: A counter example. ISI Workshop on Adaptive Control, Florence, Italy. Chen, H. F. (1983). Recurs(re system identification and adaptive control by use of the modified least squares algorithm. Research Report, Dept. of Electrical Engineering, McGill University. Chen, H. F. and P. E. Caines (1983a). The strong consistency of the stochastic gradient algorithm of adaptive control. Tech. Report, Department of Electrical Engineering, McGill University. Chen, H. F. and P. E Caines (1983b). On the adaptive control of a class of systems with random parameters and disturbances. IFAC Workshop on Adaptive Systems and Signal Processing, San Francisco, California. Cordero, A. O. and D. Q. Mayne (19811. Deterministic convergence of a self-tuning regulator with variable forgetting factor. Proc. lEE, 128, 19 23. Cruz, J. B., J. S. Freudenberg and D. P. Looze 11981). A relationship between sensitivity and stability of multivariable feedback systems. IEEE Trans Aut. Control, AC-26, 66 74. Desoer, C. A. and M. Vidyasagar (1975). Feedback Systems: Input and Output Properties. Academic Press, New York. Doyle, J. C. and G. Stein (1981). Multivariable feedback design: Concepts for a classical/modern synthesis. IEEE Trans Aut. Control, AC-26, 4 16. Dugard, L., G. C. Goodwin and C. E. de Souza (1983). Prior knowledge in model reference adaptive control of multi-input multi-output systems. 22nd CDC and IEEE Tran.s Aut. Control, to appear. Egardt, B. (1979). Stability of Adaptive Controllers. SpringerVerlag, New York. Egardt, B, (1980a). Unification of some discrete time adaptive control schemes. IEEE Trans Aut. Control, AC-25, 693-697. Egardt, B. (1980b). Stability analysis of discrete-time adaptive control schemes. IEEE Trans Aut. Control, AC-25, 710-717. Elliott, H, (1982). Hybrid adaptive control of continuous time systems. IEEE Trans. Aut Control,, AC-27, 419 426. EIliott, H., R. Cristi and M. Das (1982). Global stability of a direct adaptive pole placement algorithm. Tech. Report UMASS-ECE-No. 82-1, Dept. of Electrical and Computer Engng, Univ. of Massachusetts. Felbaum, A. A. (1965). Optimal Control Systems. Academic Press, New York. Feuer, A. and S. Morse 11978). Adaptive control of single-input single-output linear systems. I EEE Trans Aut. Control. AC-23, 557 570. Fortescue, T. R., L. S. Kershenbaum and B. E. Ydstie (1981). Implementation of self tuning regulators with wit(able forgetting factors. Automat(ca, 17, 831-835. Francis, B. A, and W. M. Wonham (1976). The internal model principle of control theory. Automat(ca, 12, 457-465. Fuchs, J. J. J. (1982t. On the good use of the spectral radius of a matrix. IEEE Trans Aut. Control, AC-27, 1134-1135. Gawthrop, P. J. (1980). On the stability and convergence of a selftuning controller. Int. J. Control, 31,973-998.

Convergence of adaptive control algorithms Goodwin, G. C. and S. W. Chan (1983). Model reference adaptive control of systems have purely deterministic disturbances. IEEE Trans Aut. Control, AC-2g, 855-858. Goodwin, G. C., H. Elliott and E. K. Teoh (1983). Deterministic convergence of a self tuning regulator with covariance resetting. Proc. IEE, 130, 6-8. Goodwin, G. C., D. J. Hill and Xie Xianya (1984). Stochastic adaptive control for exponentially convergent time-varying systems. Technical Report EE8407, Dept. of Electrical and Computer Engng, Univ. of Newcastle, N.S.W. Goodwin, G. C., B. Mclnnis and R. S. Long (1982). Adaptive control algorithms for waste water treatment and pH neutralization. Optimal Control Appl. & Meth., 3, 443-459. Goodwin, G. C., P. J. Ramadge and P. E. Caines (1980). Discretetime multi-variable adaptive control. IEEE Trans Aut. Control, AC-25, 449-456. Goodwin, G. C., P. J. Ramadge and P. E. Caines (1981). Discretetime stochastic adaptive control. S l A M J. Control Optimiz., 19, 829-456. Goodwin, G. C. and K. S. Sin (1981). Adaptive control of nonminimum phase systems. IEEE Trans Aut. Control, 26, 478-483. Goodwin, G. C. and K. S. Sin (1984). Adaptive Filtering Prediction and Control. Prentice-Hall, New Jersey. Goodwin, G. C. and E. K. Teoh (1983a). Adaptive control of a class of linear time-varying systems. Presented at IFAC Workshop Adaptive Systems in Control and Signal Processing, San Francisco. Goodwin, G. C. and E. K. Teoh (1983b). Persistency of excitation in the presence of possible unbounded signals. IEEE Trans Aut. Control. Gregory, P. C. (Editor) (1959). Proceedings of the Self Adaptive Flight Control Systems Symposium. WADC Tech. Report 59 49, Wright Patterson Air Force Base, Ohio. Hahn, W. (1963). Theory and Application of Lyapunov' s Direct Method. Prentice-Hall, New Jersey. Hall, P. and C. C. Heyde (1980). Martingale Limit Theory and its Applications. Academic Press, New York. Harris, C. J. and S. A. Billings (1981). Self-tuning and Adaptive Control: Theory and Applications. IEE Publication. Harrison, C. (1983). Private communication, B.H.P., Newcastle. Hill, D. J. and P. J. Moylan (1980). Dissipative dynamical systems: Basic input-output and state properties. J. Franklin Inst., 309, 327-357. Ioannou, P. A. and P. V. Kokotovic (1982). Singular perturbations and robust redesign of adaptive control. 21st CDC, Orlando, Florida. loannou, P. A. and P. V. Kokotovic (1983). Adaptive systems with reduced models. Lecture Notes in Control and Information Sciences, Springer-Verlag, New York. Johansson, R. (1983). Multivariable adaptive control. Ph.D. thesis, Dept. of Automatic Control, Lund Institute of Technology. Jeanneau, J. L. and Ph. de Larminat (1975), Metodo de regulacion adaptiva para sistemas de "fase no minima". Congreso Nacional Informatica y Automatica, Madrid. Kailath, T. (1980). Linear Systems. Prentice-Hall, New Jersey. Kosut, R. L. and B. Friedlander (1982). Performance robustness properties of adaptive control systems. 21st IEEE Conference on Decision and Control, Orlando, Florida, pp. 18-23. Kreisselmeier, G. and K. S. Narendra (1982). Stable model reference adaptive control in the presence of bounded disturbances. IEEE Trans Aut. Control, AC-27, 1169-1175. Kumar, P. R. (1983). Simultaneous identification and adaptive control of unknown systems over finite parameter sets. IEEE Trans Aut. Control, AC-28, 68-76. Kumar, P. R. (1984). Adaptive control with the stochastic approximation algorithm: Geometry and convergence. IEEE Trans Aut. Control, to appear. Landau, 1. D. (1979). Adaptive Control--The Model Reference Approach. Marcel-Dekker, New York. Ljung, L. (1977). On positive real functions and the convergence

531

of some recursive schemes. IEEE Trans Aut. Control, AC-22, 539-551. Ljung, L. and T. S6derstriSm (1983). Theory and Practice of Recursive Identification. M.I.T. Press, Cambridge, Mass. Ljung, L. and E. Trulsson (1981). Adaptive control based on explicit criterion minimization. Proc. 8th IFAC Worm Congress, Kyoto, Japan, VII, paper 31.1, pp. 1-6. Lozano Leal, R. and G. C. Goodwin (1984). A globally convergent adaptive pole placement algorithm without a persistency of excitation requirement. Technical Report EE8405, Dept. of Electrical and Computer Engng, University of Newcastle, N.S.W. Martin-Sanchez, J. M. (1984). A globally stable APCS in the presence of bounded noise and disturbances. IEEE Trans Aut. Control, to appear. Moore, J. B. (1983). Persistency of excitation in extended least squares. IEEE Trans Aut. Control, AC-28, 60 68. Morse, A. S. (1980). Global stability of parameter-adaptive control systems. 1EEE Trans Aut. Control, AC-25, 433 439. Narendra, K. S., I. H. Khalifa and A. M. Annaswamy (1983). Error models for stable hybrid adaptive systems. Tech. Report, Center for Systems Science, Yale University. Narendra, K. S. and Y. H. Lin (1980). Stable discrete adaptive control. IEEE Trans Aut. Control, AC-25, 456-461. Narendra, K. S., Y. H. Lin and L. S. Valavani (1980). Stable adaptive controller design, Part II: Proof of stability. IEEE Trans Aut. Control, AC-25, 440-449. Narendra, K. S. and R. V. Monopoli (Editors) (1980). Applications of Adaptive Control. Academic Press, New York. Narendra, K. S. and L. S. Valavani (1978). Stable adaptive controller design-direct control. IEEE Trans Aut. Control, AC23, 570-583. Peterson, B. B. and K. S. Narendra (1982). Bounded error adaptive control. IEEE Trans Aut. Control, AC-27, 1161-1167. Popov, V. M. (1973). Hyperstability of Automatic Control Systems. Springer Verlag, New York. Praly, L. (1982). Towards a direct adaptive control scheme for general disturbed MIMO systems. Analysis and optimization of systems. Lecture Notes in Control and Information Sciences, 44. Springer Verlag, Berlin. Praly, L. (1984). Towards a globally stable direct adaptive control scheme for not necessarily minimum phase systems. IEEE Trans Aut. Control, to appear. Ramadge, P. J. and G. C. Goodwin (1979). Design of restricted complexity adaptive control systems. IEEE Trans Aut. Control, AC-24, 584-588. Rohrs, C., L. Valavani, M. Athans and G. Stein (1982). Robustness of adaptive control algorithms in the presence of unmodelled dynamics. 21st CDC, Orlando. Safonov, M. G., A, J. Laub and G. L. Hartman (1981). Feedback properties of multivariable systems: The use of the return difference matrix. IEEE Trans Aut. Control, AC-26, 47-65. Samson, C. (1983). Stability analysis of adaptively controlled systems subject to bounded disturbances. Automatica, 19, 81-86. Sin, K. S. and G. C. Goodwin (1982). Stochastic adaptive control using modified least squares algorithms. Automatica, 18, 315-321. Solo, V. (1979). The convertence of AML. IEEE Trans Aut. Control, AC-24, 958-962. Trulsson, E. (1983). Adaptive control based on explicit criterion minimization. Dissertation No. 106, Linkoping University, Sweden. Tsypskin, Y. Z. (1971). Adaptution and Learning in Automatic Systems. Academic Press, New York. Vidyasagar, M. (1978). Nonlinear System Analysis. Prentice-Hall, New Jersey. Willems, J. L. (1970). Stability Theory of Dynamical Systems. Nelson, London. Wolovich, W. A. and P. L. Falb i1976), lnvariants and canonical forms under dynamic compensation. S l A M J. Control Optimiz., 14, 996-1008.