- Email: [email protected]
Theory of System Identification and Adaptive Control for Stochastic Systems*
Copyright © IFAC Identi ficat ion and System Parameter Estimation, Beijing, PRC 1988
THEORY OF SYSTEM IDENTIFICATION AND ADAPTIVE CONTROL FOR STOCHASTIC SYSTEMS* Han·Fu Chen I nstitute of Systems Science, Academia Sinica, Beijing, PRC
Abstract. This paper presents a series of theoretical results on parameter identification and adaptive control for discrete - time stochastic feedback control systems . Without a use of persistent excitation condition the unknown coefficients of the system are consistently estimated by use of the extended least squares (ELS) and the stochastic gradient (SG) algorithms . Then a strong consistent estimate for orders of the system is also given by minimizing an information criterion designed for feedback control systems . In order to apply results of parameter estimation to adaptive control systems an attenuating excitation technique is introduced so that for strong consistency of the parameter estimate the conditions imposed on eigenvalues are replaced by the growth rate restriction on the input - output of the system . Further , optimal controls are designed respectively for tracking a given reference signal and for minimizing a quadratic loss fun ction so that the parameter estimates are strongly consistent and , simul taneously , the performance indeces are minimized. When the system contains an unmodeled dynamics but the adaptive control is designed on the basis of the mode led part of the system a robust analysis is presented by esti mating the error in parameter estimation and in control performances . Keywords . Identification ; adaptive control ; strong consistency ; adaptive t r acking ; adaptive LQG problem;order estimation;robustness . INTRODUCTION The task of system identification is to con st ruct a mathematical model of a system by use of i t s input - output data . Normally , the s y s tem inp ut is not arbitra r y but is de s ign e d to satisfy a certain control purpose whil e identifying the system . This is the topic of ada ptive control . In this pa per we only concern feedback cont r ol systems in time domain and leave ex per ime n t d esign (Z a rrop , 1979) , identification in frequency domain (Ljung , 1985) and a da ptive control for Markov chains (Kumar , 1 985) al one . To be specific , we discuss the di sc rete - time deterministic or stochastic input - output model called ARMAX process . A(z)y =B(z)u +C(z)w , y .= O, u. =O, w.=O,i
Assume {wn , Tn } is a martingale dif f erenc e sequence with respect to a nondecreasing f a mily {T n} of o - algebras E ( wn / T n _ 1 ) = 0 , ... n~ 0 •
The control u depends upon the past mea surement (y o . \Jn .yo u " ' U n - 1) , i.e. u n is T n measurable . The feedback nature of u makes our identification problem differen£ from time series analysis (Box & Jenkins , 1970) . In this paper we first estimate the unknown matrix coefficient 8 by the ELS algo r ithm and give its convergence rate assuming upper bounds for system orders a r e a vailabl e. Then we discuss the simp l er comput a ble SG algorithm to estimate 8 and the projection algorithm for deterministic system s (w =0) . All these results are characte r i z ed byntha t the persistent excit a tion condition , which is usually used in literature (Moo r e , 1978 ; Solo , 1979 ; Ljung & Soderstrom ,1 983 ; Chen . 1985 and Soderstrom & Stoica , 1985) is not i mposed here . However , for strong consistence of parameter estimation there are still so me conditions on re a lizations have to be assu1 2 q med.In order these not easily ver if i a bl e r C(z)=ItC zt ••• tC z , r~O (4) conditions to be satisfied we app l y the a t 1 r tenuating excitation method which mea n s wit h unknown or ders p , q , r and unknown mathat the desired control is distur bed by a tr i x coe ff icient dither tending to zero as time tends to in finity . 8 T = ( - A ••• AB ••• BC · •• C ) (5 ) 1 p1 q1 r For order estimation we introduce a n inf or {y } is the m- output and {u } is the ~ - in mation criterion designed fo r feedback con pu£. When w =0 the system n(l) degenerates trol systems and by minimizing it we ge t to the d e t~ rm inistic one , otherwi s e , C(z)w the consistent estimate for orders . Th us, r ep r esents the dynamic nois e of the syst~m . strongly consistent estim a tes for both the *Wo r k supported by the National Natur a l Science Foundation of China and by the TWAS Re sear ch Gra nt No . 87-43 . 51
52
Han-Fu Chen
system coefficients and the orders can be obtained simultaneously. By using the estimation results we design optimal adaptive controls for tracking a given reference signal and for minimizing a quadratic loss function so that the control performances are minimized and the parameter estimates are strongly consistent. Finally,we consider the influence of the unmodeled dynamics possibly cont ained in the system upon the parameter estimation and upon the adaptive cont r ol which are designed on the basis of the modeled part of the system.It is shown that errors arising in parameter estimation and in control performances are of order E where e is the multipIer of the dominator for unmodeled dynamics of the system.
follow i ng estimate (G uo,198 7 ; Chen & Guo, 1987a) for the weighted sum of a martingale difference sequence (wn,r n ) with
~~bE(llwn+l W/'n)<= a.s. a£[l,2)
L Mowo+ =0(S n (a) loga i=O 1 1 1
By use of Condition 2 it is proved that td8-8 n +1 )Tp~ :l (8-8 n + l )
(6)
with 8 0 arbitrarily chosen,Po=dI,d=mp+~q+mr. It is easy to verify -1
n
T
1
n
n
L 1I(8-8 1o+1 )'$oW-2 L w\1(8i=O 1 i=O 1 n
P +l=P -a P $ $T p , a =(1+$T p $ )-1(7) n n n n n n n n n n n $T=(yT ••• yT uT"'U T T_~T 8 n-p+l n n-q+1 Yn wn _l n n n ···y~-r+l-$~-r8n-r+l) (8)
P +1= L $o$o+-d I n i=l 1 1
(13)
-8i+l)T$ o~0(1) + 0( L ao$:po$ollwo+ 1W) a.s. 1 i=O 1 1 1 1 1
In this section the system orders are not estimated and their upper bounds p,q and r are assumed to be known. One of commonly used methods for parameter estimation is the ELS algorithm which is the extention of the least squares method to the correlated noise and is defined as follows
8n+l=8n+anPn$n(Y~+1-$~8n)
(Sa(a) +e) ) n
a.s."'v'"n >O ,where M is any' -measurable random matrix of ncompatibl~ dimension and S (a) = ( E IIMolla )l/a n i=O 1
~O(l)-ko PARAMETER ESTIMATION FOR STOCHASTIC SYSTEMS
~+n
n
(9)
By A~~X and A~in denote the largest and the smallest eigenvalues of Pn-1+l respectively. For stron~ consistency of 8 it is usually required (Ljung & Soderstro~,1983;Moore, 1978;Solo,1979 and Chen,1985)that An /Anol ,where 8n is given by (6)-(8) and 8.S
6(x)={l,x=0 O,Xl' O Proof.The proof is essentially based on the
for any n~O.Then the conclusion of the theorem is separately verified for two ca ses n ==.The key point limA n <= and n~oo limA max n~oo max is to show that n
o+ W=o(logdet(P-: ) L ao$:Po$ollw 1 1 1 1 1 1 n l
i =O
(10glogdet(P~:1))C6(S-2) •
For details we refer to (Chen & Guo,1986d).
# This theorem says that 8 is strongly consistent with convergencenrate indicated by (12) if Condition (11) is fulfilled and i f S>2. We note that An and Ano are estimatemax mln dependent hence so is the right-hand side of (12).We now remove this undesirable dependence by using
$~T=(y~",y~_P+1u~ ••• u~_q+lw~"'w~_r+1) $~=O,i
a.s.
if 10gA on (loglogA on )c6(8 -2 ) =0(Ao~ ) for max max mln some c>l. Proof.The proof consists in verifying logAn (lo glog An )c6(S-2) max max =O(logA on (loglogAon )c6(S-2)) # max max • We now introduce
w~=(y~,··y~_P+lu~ ••• u~_q+1Y~-W~_len_l ••• yT _WT 8 ) n-r+1 n-r n-r where for estimating wn the aposteriori
Identification and Adaptive Control for Stochastic Systems estima te yn - S~ _ 1~n _ 1 r ather than the po s t er iori e s t ima te yn - S~~n _ 1 a s in (8) is us ed . The SG alg or ithm fo r estim a ting S is defi ned as f ollows Wn T T Sn +1=Sn+ r (Yn+1 - WnSn) (16) n n
r =1+ L Ilw. n i=1 l
W,
( 17)
r =1 0
with So a rbitrarily chosen . condition 3. C( z) - ~I i s strictly positive r eal . Theo r em 3 . If Conditions 1 (with 8=2) and 3 are s a tisfied , r ~oo and if there are a E( O,iJ a nd N nand M possibly dep e nding on w such tha~ ( 18) r n +1 / r n ~M ( logr)a a.s. n n
~ max
/
n
~min~
(19)
n ~No - 1 ,then 8 n n 100 8 a . s . where 8 n is defined by ( 16) ( 17) and ~~ax and ~~in respectively den ote the maximum and minim um eigenvalues of ~L (W .W.T +aI 1 ). i =1 l l The conc l usi on of the theorem remains valid if ~n , ~n . and r are replaced by ~on , max mln nn max ~o~ and r O =1 + L 11 ~~ W respectively . i=1 l mln n
Proof . The theorem is proved by investig at ing beha v iors of th e mat r ix froduct WnW~ WoW o 'I'(n , O)=(I -- - ) '" (1 - - ) . rn ro It can be shown that under conditions of the theorem 'I'(n , O) ~ 0 and this ~uarantees stron g consistencen~f 8 . DetailsOare given i n (Chen & Guo , 1985a , n 1 987b) . From Theorem 3 we see that for st r on g con sistence of 8 g iven by SG the persistent excitation i snn ot hequired since Condit i on (19) al l ows ~n /~ . to d i verge . Howeve r, max ITan the ord er i- a in (19) indicating the diver g ence rat e , in g eneral , cannot be enlarged to more tha n 1 as shown by an counterex a mp l e in (Chen & Gu o , 1986a ) . PARAMETER ESTIMATI ON FOR DETERMINISTIC SYSTEMS Set wn =O in (1) . Then it t urns to a determi nistic system.Thus , we consider d (2 0 ) A( z)Yn=z B( z)u n , d~ O whe r e A(z) a nd B( z) a r e define d by (2) (3) . By use of the projec t ion a l g orithm we esti mat e th e unknown parameter T 8 =(- A1 '" - Ap B1 '" Bq) with fi xe d (p , q ) as fo l lows
~n n
where 6n =8 - Sn . Then the conve r gence ra te of Sn to zero i s defined by the behav io r of the mat r i x ~(n , i) given rec ur sively by ~
~T
~(n+1 , i) = (I - 1~1I~ W)( n, i) , ~( i, i) = I
(24)
n
Theo r em 4 (Anderson & J ohnson , 1982) . Ass um e that !y ) is bo unded , A(z) and B(z) are l eft coprime nw i t h A being of f u l l r an k an d tha t th e input is Psufficie nt l y r ich , i . e . t h ere are const a nts 8 >0 , 6 >0 and an integer N>O 1 2 such that n +N S1I~ L V.U: ( S2 I ,'f n , (25 ) i=n +1 l l T T T whe r e (26) _ U.=( l u l. ···u l. - mp - q +1 ) Then Sn exponentially tends to O, i . e . Is - Sn "=O(y n) for some YE(O,1) . (27)
#
a.s.
fo r all
8 n +1 =8 n +1+11 ~
53
T) w(T y n +1 - ~ n Sn
(21 )
We now extend this well - known result in se ve r al asp e cts , namely , we will allow y to diverge , the low e r bound of (25) to gB to zero and the number of sum mands in (25) to increase unbo undedly . Such an extention not on l y is interest i ng by itsself but a l so is a s i gnificant step towards solving the ad aptive control pr oblem . Theore m 5 . Suppose that A(z) and B(z) are left - coprime with A being of full r ank and T
n
-1
P
L V.U:).6I ,V n ~ 1 (28) i =T +mp l l n n-1 for so~e sequence of non - negative numbers 6 >c / n , 6>O , possib l y tendin g to ze r o and n integers iT } with T =1 , d =T -T 1~mp a nd n 0 \ n n nthat.lI~ II=O(nv) , d =O(n ) for some constants n n v).O , Ap O, 6p O and c >O satisfyin g 4( 1+\)vt26 +5 A< 1 . The n 1 - 2 6- 5\ - 4(1+\) v 118 - 8 II=O( e xp( - an n
1+\
))
(29)
where a is a positive const a nt . Proof . We point out the key steps of the proof.First , it is established that N c N- d+1 A . ( I. ~·~:)~N k 0+1 \ . ( L V.V: ) mln i=k l l - - mp ml n i=ktmp -d +1l l fo r any N9k +mp and k~O , whe r e c is a cons tant and \ . ( X) denotes the °min im um ei g env a lue mln of a mat r ix X. Th e n , it is shown that 6 2 , ,\, 11 ~ ( N , k) lid 1 - 4TN"=kT r N- 1 ~ . ~ : where
.L
1+tl~~W~ 6 I , 6>0 .
l=k l Finally , it is concluded tha t n
6~
--t5)
11~(T n+ d - 1 , O) lI~exp( - c1.~ l- 1M d i i =0 (
exp
(_
c2
( nr1)1~ 6 - 5 \ - 4 v (1+\ ) )
1 - 26 - 5\ - 4v(1 +\}
SlJ P II~ . W+1 , c1>O,c2>0 f r om T.1- 1~j~ T.-1 J 1 wh i ch (29) f oll ows if we note T ~.F o r det a ils we r efe r to (Che n & Gu o , n1987 c ) . # Remar k . Theo r em 5 g iv e s a ne ar exponenti a l r a t e of conve rg enc e. Se tt ing v =O, \ =O an d 6 =0 in (29) we g et an e xp on ent ia l convergen c e with Mi=
54
Han -Fu Chen
rate wh ich coincides with the one given by Theorem 4 . PARAMET ER IDENTIFICATION WITH ATTENUATELY EXCITED CONTROL From Th eor ems 1 - 5 we see that for consisten cy of parameter estimates some conditions are imposed on eigenvalues depending on re alizations (see (11) , (12) ,(1 5) , (18) , (19) , (25) and (28» . These conditions are diffi cult to verif y and are often not satisfied f or adaptive control systems as shown in (Chen & Guo,1987a) . In order to get consis tent estimate a dither with const a nt vari an c e is adde d to control (Caines & Lafortu ne , 1984;Chen,1984 ; Chen & Caines , 1985) to excite the system , but , at the same time , it hurts the control performance . In the subse quent sections we change the abovementioned dither to the attenuating excitation (Chen & Guo ,1 986d,1987a,b) which sufficiently ex cites the system to yield consistent esti mate s but does worsen the control performa ce . When the attenuating excitation cont r ol is used we show that the difficu l t condi tions impo sed on eigenvalues can be replaced by the easier verifiable ones imposed on the growth ra te of the system input - out put . To be specific, let { £ } be a sequence of £ -di mensional iid rand8m vect ors with cont inu ous distr ibution and let it be i ndependent of wn and such that E£ n =O,£E n E'n =UI , liE n Iku 0
a . s ., u>O
(30)
Without loss of generality assume 'n=o{wi ' £i , h n }, '~ = o{wi , Ei _ 1 ,i~n } Let " -measurable Uo be the desired control . n n The attenuately excited control un applied to the system is def ined by (31) u =uo+v n
where v =E
n
n
~E
1 , £ f[ O '~ t+1 ) , t=max(p , q , r)+mp-1 ~\vT I) (32) Con d ition 4 . There is a posit i ve definite matr i x R>O possibly depe~ding on realiza tions so that n
n
In
n
lim2 ! w.w:=R n ni~1 1 1
a.s.
(33)
Condi ti on 5 . A(z) , B(z) and C(z) ha ve no common left factor and A i s of full rank. p Theorem 6. Suppose that Conditions 1 , 2 , 4 :~~ ~h:~e a~U~::~ led and un is g iven by (31)
2
6 Y. (lIy. W t llu?W)=0(n ) , a . s .
ni~1
1
The similar t heorem is t rue for SG algorithm . Theorem 7 . Assu me that Conditions 1 (6=2) , 3 ,4 and 5 are satisfied and un is given by (31) but with vn defined by
v1=o,vn=En/(10gn)~£ ' E E (O ' 4s(m+2»,n>2(36) s=max(p , q,r+1) instead of (32) .If
2 T.
n i =1
(1Iy.W+!Iu?W) =0(10 g6 n ) a.s. 1
1
for some 6E(O,(i - (m+2)sE! / (2 +(m+lls) as n+'n_1 : . L Ui U~~Il 1='n_1+ mp , = n k- 1 if A • ( { U. U:)<1 ,'1" k>, 1 +mpl 1 nmln i =, n-1 It can be sh own that 2(m +q )(E +V)+£ - V 1 - 2 mD+q)(£+V) - EtV) • O( T - T = n" n n- 1 Then the conclusion follows from Th e orem 5 if in which we set 6 =0 and A_ 2 (m +q )(E t V) t E- V 1 -2 mp+q)(£+v) - £+v For detail s we refer to (Ch en & Guo , 1987c) .
r
r
#
(34)
1.
1-2E{t +1) co for any 6f ( 0 , 2t+3 ) . Then .or en given by ELS '(3 2' lie _ eW= O((logn)(log lo gn ) C, - ) , a . s . (35) n na for any c>1 and any a E ( '( 1t 5) , 1- (t t1 )(£ t 6» where 6( · ) is defined in lheorem 1 . Proof. Condition (34) i~plies An =0( n 1 +6 ) , max so i t suffices to show An. >c n a , c >O ,V n~n mln 0 0 0 for some n . This can be proved by that a converse ~ssu mption leads to a contradic tion.For details we refer to (Chen & Guo , 1986d ) . #
(3 7)
ORDER ESTIMATION Fo r system (1) u is defined by {y. , i~n}, so , in general , itnis not stationary1or ergodic.In this case the order estimation prob le m is more complicat ed than that for time series for which the orders are esti mated by minimi zi ng AI C (Akaike,1969),BIC ( Akaike,1979;Rissanen,1978 ;S chwarz , 1978) and ct IC (Hannan & Qu in n ,19 78) . In this section we assum e that true orders for A(z),B(z) and C(z) are p , q a nd r respectively. 0 0 0 Cond i tion 6 . The tr u e orders (po , qo ,r ) o belong to a known set M: M={(p , q,r):O,p,p·,O,q~q*,O~r'r*}.
Identification and Adaptive Control for Stochastic Systems
55
Condition 7. A sequence of posi tive numbers {an} can be found such that (logr o ) (loglogr O )O o(8 - 2 ) n n +0 a.s. (42) an n -+00
ted noise case. Theo re m 9 . If Conditions l, 2,6and 7 hold , then (Pn , qn,r n ) n!oo (po , qo , r o ) a . s .
for some 0>1 and a h (!? ,q,r) (n) +0 n+«> n m~n for any (p,q,r )€M* consisting of three points: M*={ (po,q*,r*),(p*,qo,r*),(p*,q*,r o ))
where (Pn ,q n , r n ) is given by (55) . Proof . The first step is to show n- 1 ( I. 11c;.+1 J12=O((logr o ) (loglogr o)c O B- 2))
where A(!?,q,r)(n) den otes the minimum eigen-
i~O
m~n
value of n-1
l
o( ~n
~? ( p,q,r )~? ' ( p,q,r)+~I,d=mp *+ lq * + m r * ~
(44)
p,q,r )_(" - Yn"' Yn -p+1
u "n ",u n _q +1
w'n ... w'n-r+1 l' and r
O
n
-~n= (' , , , l' Yn"'Yn_p *+1un",un_q *+1wn ",wn_r *t1 ~,~,
(46) where the estim ate ~ for wn is recursively defined as follows: n ~n=yn-8~~n-1' n~O, wn=O,n
(47)
8n+1=8n tanFn~n (Y~+ 1-~~8n)
(48) 1
F n +1=F n -a nF n~ n~'F n n ,a=(1+~'F n n n~ n r (4 9) with arbitrary 8 0 and Fo=dI. For any (p,q,r)€M s et e(p,q,r) = (-A1···-ApB1···BqC1···Crl"t J
IJ2 ,li B 1J2 ,IIC
Po
qo
ro
1J2} >O
and
(S , t , A)=(p po , q qo , rvr o ) ' Finally we show that any limit point of (Pn,qn , rn)coinc i des with (po , qo , ro) . For de tails we refer to (Chen,Guo & Zhang , 1988).# We now specify { a \ used in Condition 7 when the attenuat~ng excitation contro l is applied. Theorem 10 . If Condit.i.ons 1 , 2 , 4 a nd 6 ho ld, A(z),B(z) and C(z)h ave no common left-factor and A , B and C are of full rank and Po qo qo if applying control (31)(32) with t=(mt 1 )p* +q *+r* -l leads to 1 n -ni=O l (Jly 1· lJ2 t llu 0~. W)=O(n 0 )
a.s.
(57)
for some oEl O, (1 - 2E(t+1)/(2tt3)1 , then (50)
where by definition A.= O,B. =O , C =0 for i>po n
j>qo and bro' At time n e(p,q,r) is estimated by ELS n-1 , 1 -1 e (p, q ,rl = ( l ~. ( p, q, r) ~. (p , q , rl +ar) n i=O ~ ~ n-1 . i l=0 ~.(p,q,r)Y~+1 (51) 1. ~ where ~ n ( p,q, r )= ( Yn' ••• Yn-p+1 ' u'n ... u'n-q+1
w'n .. ·~'n-r+1 l' ~n
o =min[JIA
o
L 11~ ?( p * ,q * ,r *)IJ2 . i=O ~
~
"4
m~n
for any (p , q , r)EM ,w here
( 45)
Denote the regr ession vector co rresp ondin g to the system of the l argest possib l e or ders by
and
n
a n (p+q+ r- po - qo -r o t o(1)) a . s . if (S , t , A) >{ ( ) 0 =(p , q ,r ) A ~ , t ' A (n)( °to( 1 )) a . s . otherwise
n-1 =1+
n
CIC(p , q , r)n - CIC(po , qo , ro)n
i=1 ~ with
~
a.s. for any c>1 . where C;n=wn - wn . Then we pr ove that
(52)
is g ive n by (47) .
We now introduce a new information criterion CIC(p,q,r) where the first " CH means that the criterion is designed for feedback control systems: CIC(p,q,r ) n=on(p,q,r ) +( p+qtr)an (53)
and a is given in Condition 7.The estimate (pn,q~,rn) for ( po,qo,r o ) is g iv en by minimizing CIC(p,q,r)n (Pn,qn,rn)=arg min CIC(p,q,r)n (55) (p,q, rlE M The following theorem generalizes result given in (Chen & Guo,1987d) to the correla-
(Pn , qn ,r n ) n!oo(po , qo ,r o ) a.s. where (Pn , qn , r n ) is given sat.isfying (logn)(loglogn)a6(B - 2) a
and
n
a /n 1 - (t+1)(r:tO) n
n++0oo for some 0>1 ( 58)
+0 n-?oo
(59 )
where E is defined by (32) and t by (56) . Proof. Noticing 1 - ( tt1)(E+6);:d1 - E)/2>O ,w e see that a =nu with any uE(O , l - (l+t)(E +O)) meets re qu~rements (58)(5 9).By (526 and 1to Condition 4 it is easy to see rn=O(n ). So the critical point i s to show liminf n - aA(J? , q , r)(n)=O a . s . mln n+ oo for any (p , q , rl EM* where 0=1 - (tt1)(£ tS). Then the conclusion of the theorem follows from Theorem 9 . # SIMULTANEOUS LY IDENTIFYING SYSTEM PARAMETER AND TRACKING A REFERENCE SIGNAL Since sevent i es there has been a lot of research made on adaptive control such as seIf-tuning reg ulator (Astrom & Wittenm a r k, 1973,1985;Clarke,1984),model r eference adaptive control (La ndau,1979) and adaptive pole-assignment ( Johnston & Anderson , 1983) .
56
Han-Fu Chen
We now de s ign control u in or der that the output y of system (1)n with known orders p , q an d nr follows a given reference signal . In (Goodwin , Ramadge and Caines , 1981) the control un is selected to satisfy
8~Wn=y~+1 (60) where 8 is given by SG algo r ithm . There it is p r ov~d that 1 n
L
l~!ll~ u~
n i =1
(lly . W+lluiW)
a . s.
(61)
a .s .
(62)
and
(66) where B1 is the estim a te for B1 g iven by le f t - cop 8 • 2) n1f A(z) and B(z) are n rime with A being of full tank , d =1 a nd zer os of p det(B(z)(I 0 ) ) li e outside the closed unit disk and mthe cont r ol u is defined by (31)(38)(66) , then (y) a nd (U) n n are bounded a . s . and Ily - y * ll=o( ~ ) 118 - 8 IFO(exp(-an1 - 12(mp+q)e:~u)) n!'1
where R is the limit matrix in (33) . I n th i s well - known result there are two pro blems which remain open:Is (60) solvable with respect to u ?Is 8 strongly consistent un der such design~d adaBtive control?In fact , the control designed by (60) , in genera l , d oes not lead to consistency of the parame te r estimate . For example , if y*~O , then T T T 8nw n ~O , 8 n (8 n +1 - 8 n )=8 n w n y~+1/r n n =0 a n d hence T T 8 n 8 n =8 n - 18 n- 1+( 8 n - 8 n- 1?(8 n - 8 n - 1) n
= 8~80+ i~ 1 (8 i - 8 i _ 1 ?(8 i - 8 i _ 1 h8~80 ,"" n>1 . does not converge to 8 if 8'8 n o 0 The following theorem solves the problems raised above . The solvability of (60) is al so discussed in (Meyn & Caines , 1986) . Theorem 11. 1) If for system (1) m",£ , u . , i
£:
n
'
n a . s . (67) Proof . It is easy to see ooI18:ilJ .W ,_ L _ _1_1_
Hence , Ile'ilJ. W=0(1+llilJ.112) . By (66) we have 111
-,
.
yn+1 - y~+1=8nilJn+B1nEn
and by boundedness of
(69)
(y~l
IlilJ n +1 W=O(1)+o(s~p IlilJ' t1 W) . 0" J ~n J This implies boundedness of {y } and {u } , and the theorem f ollows from n(69) and n (40) with v=O . The details of the proof are giv en in (Chen & Guo , 1987c) .
So 8
B1nu~+(8~Wn - B1nun)=y~+1
(63)
is so l vab l e with respect to uO , where B1 denotes the estimate for B1 ngiven by n8 • 2)If Conditions 1(~=2) , 3 , 4 , 5 hold , m~£ , zer8s of det(B(z) ( I 0)') lie out side the closed un it disk andmthe control u is defined by (3 1 )(36) and (63) , then (6 1 )n(62) hold and 8 n n~oo8 a.s . Proof . For proof of 1) we refer to (Chen & Gu o , 1986c) . 2) We fi,st show (Chen & Caines , 1985a) that LIIC+ W/r.
The adaptive control problem with q uad r atic loss function has been considered for vari ous cases:for the case where the un kno wn e takes value only in a finite set in (K umar , 1983 ; Hijab , 1983 ; Caines & Chen , 1985) , for the case where the availability of cons i stent parameter estimates is assumed in (Chen & Caines , 1985b ; Chen , 1985) an d fo r other cases in (Samson , 1983 ; Chen & Guo , 1986b) . For the case where the noise is correlated both consistency of parameter estimates an d minimality of the loss function are achie ved in (Che n & Guo , 1986d , 1987b) . Assume that p , q,r for system (1) a re known and the unknown 8 defined by (5) is esti mated by ELS . We want to design adaptive con t r ol to :ninimize
(64)
n
J(u)=limsupl
,
~n+1 = yn+1 - wn+1 - 8nWn '
n~=
(65 )
Then by (31)(63) we have y n t1 - y~+1=~nt1twnt 1+B1nvn
from which , afte r showing liPl ~MP r n/n
SIMULTANEOUS LY IDENTIFYING SYSTEM PARAM ETER AND MINIMIZING A QUADRATIC COST
L
n i =1
((y, _y'!)TQ1 (y · -y'.')tu:Q2u. ) il
l
1
1
1
where ~130 , Q~ 0 and (yrl is a bounded de terministic Signal . ~e present system (1) in the st a te space form ( 70) Xk+1 =AxktBuktCwk+1
Yk=~xk ' whe re - A1 10 '" : 0 A= - As 0 " "
0• :
x~ = (y~O "' O)
~1
I
Urn '
, C= ?1 , H' (71) I ' B= B 0 C s s-1 m s=~ax(p , q , r - 1) , A.=O , B .= O ' Ck=0 for i>p , j>q a nd k>r . 1 J Condition 8 . (A , B, D) is cont r oll a bl e a nd observable where D is any matrix satisfying d e composition D'D =H'Q1H .
Ide ntifica tio n a nd Adaptive Control fo r SLOchastic Systems
It is wel l - known (Ande rs on & Moore , 1971) that under Conditio n 8 in the class of non negative matrices there is an unique posi tive defin it e matrix 5> 0 sat is fying and
S =A' SA _A' SB(Q2+B' SB) - 1 2, SA +H' Q H 1 1 F=A- B(Q2 +B' SB) - B'SA
1
'"
F( n) =A ( n) - BT (n) (B T (n) S B( n) tQ T\ilW S,.p.( n ) n Z .. (85)
estimate for u * proposed by the cer tainty equivalencenprinciple is
T ~e
(72)
is sta bl e , Th en (b ) is bo unded where i b.= -
57
.
L FJ'H'Qly*+· = Fb · +l - J.:'Q1Y ~ j= O 1 J 1 1
(73)
It is not difficult to see that under Con dition s 1 (S=2),4 ,8 J (u ) can be expressed as n- l J(wtrSCR C' +1 imsu~ L (y ~T. Qp".' - B( Q + BT S E rl 1 2 n+'" ni= O 1 1 1
t:
(86) but wh i ch , in general , does not lead to con sistency of B , and hence J(Qa) may be far from the ~in i~a l value J o ' We will a p oly the randomly varying tra nca tic n te~hnique which consists in tr uncating G~ at stopp ing t imes Ilk) and (ok ) defined as follows : 1 =T 1 <01
cl
0k =sup lt >Tk : .L IIY i W~(j - 1 )log-(j - 1) l =Tk
(87)
(Q 2 +B' S B) (u . + (Q2+ BT S B) - 1 BT (SAx. +b. + 1 ») 1 1 1 (74) for any uEU,w here n -l U={ u :
L
i~O
(1Il LW ~ I x. W )=O ( n ) ,1 11J II "O(n) , a . s . 1
where
1
Hence , when e is known t he optimal control is u*=Lx +d n (75) n n with L= - (Q +BTSB) - l BTSA d =- (Q +S' SB) -l B' b 2 ' n 2 n+l ( 76) and the minimal value of J (u) is J =minJ(u) =t r SC RC+
u nO =B-1 n! (Y n·" +1- (8\[, n n - B1 n u n ») with ~.= m . ~e
now define the desir e d co ntr ol n E[ Tk , ok)n Ac for some k
IO ,if
(90)
u~ , if n~[ok " k+l ) for s o me k
i' Q1 Yi +b ~ +1 B ( Q2
t
BT S B) - 1 B'b.J:+l )
(77 )
and adaptive control ua=uo+v n
n
as (91 )
n
with vn given by (36) ,
T
en =( - A1 n " ' - Apn B1 n " · Bq n C1 n " ' Crn )
be given by the SG algorithm and set -A IO .. · O , 1n
•
A(n)=
I
3
,
1
nj
: I'
B(n)= :
",'1
C(n )=
(78)
n
C _ 0 · .. · 0 l ::·sn sn s 1n (79 ) s=max (p ,q, r - l) . Given any S 30 the S is recursively esti ma ted by 0 -A
S =A'(n ) S
n-
+B'(n )S
1 AT( n) - A' ( ~. ) s l B (n» - l
n-
s T(n)S
1A(~)+::'~1~
and L,d,b ly by n n
i = (y'O .. , J) '
(81) o 0 and F are estimated res pec tiv e n
uE :J
L =_ (ST(n)S S(n ) +~2 ) - 13' (n)S A(n )
( 82)
an =- u (~'(n)S . n 3(n)tQ2 ) -1 s T(n)b n +'
(83)
n
bn =- i~I O FJT ( n - l)H ' ~ 1 y n+J " .
n
I
(84)
0
Proof . Th e f i rst step is to sh ow n
I
iktl=A (n)in+S(n)untC (n)( Yn+l-SA(n)in - H3(n ) u )
Theore::J 1 3 . If for s ys t e m ( 1 ) ~= ~ , Condi t.ions 1 (8 =2) , 3 , 4 , 8 hold , zeros of detB(z, ) l ie outside the closed unit disk and if A i s of full. rank with p~q , th e n the adactivi contro 1 iUoi) d efined by (36)(90) and t9 1) 1eads to n consistency ' oj ~n given hy (16) (1 7) a nd ~inimizes J(~) J(·l a ) =J:",inJ ( u)=J (9 2)
1 3 ( n ) (~2 +
nn( 8 0) The state xn is estimated by an adaptive fil ter
n
(89)
and denote by UOn the so1ution of (63) , i . e .
In the pre sent case both e and x, are un avai l ab l e , and we will us e their ~est i mates.
n
u'1.' Ik i log\\ , 1). 1 )
A={ i :11
u~ =l n~ ' if nE [Tk, ok)nA for some k
o uE: U
Let
i s given in (37) .
<5
Def i ne a set A of integers :
Uif:'j,"f i>O }
n- l +1 i ms u ~): (y n+ex> FO
(88)
11 0'; W= J( nl og~n) .
i:;:: 1
.,L.
:he consis~e nc ~ of S tn en f o l lows fr om :heore~ 7 . : he ~econdnstep is t: prove S
+
n n -+ oo
S
and tte :tird step is to establish that :here exi st s sone k fer '.... h i cr. T, <~ and KO ::= ;:x) . ::ence
(93) ?inally , (92) can be established by using consistency of and (93) . For details we refer to (then &n Cuo , 1987 ) , #
e
When sult
we ca n show a more crecise re lbased on the ~L S algorithm .
y~ S C
58
Han-Fu Chen
We retain notations introdu ced by (78) - (82) (85) and (86) but where e shoul d be under stood a s the estimate ngiven by (6) - (8) . We redefin e {T k } and {ok}:
where the modeled part is the same as system (1) , while ~ is T - measurable and is do~inated by n n n
1=T 1 <01
-\fj~ (Tk , tJ
°k - 1
} (94)
<-7-' 1 +6
Tk+1=inf{t>ok:, LIIL i S
(95) (96)
t =max (p ,q,r ) +m p -1. Set L , if nE[Tk,ok) for some k L0 ={ n
(97)
O, if nE[ok,T k +1 ) for some k Finally,the adaptive contro l (u a } now is defined as n ua=Lox +v (98) n n n n wher e (v } is given by (32) . n Theor em 1 4 . If Conditions 1 , 2 , 5 , 8 hold and A(z) is stable with A being of full rank (A =1) and if as n+oo p n
o
11 ~
ni
n
L w,w~ - RII=O(n - P) =1 l l
a . s . P>O ,
then control {u~} defined by (98) leads to c6(6 - 2) lI e - e W =O ( ( logn ( loglogn ) ) a . s .'I c>1 n a n and u
1)
l ims uI=1l en - ell~co/KE
2)
k~4k o /(1 - c 1 (1+2k 0 )E2)
a.s. n+oo where c >0 is a constant independent of k and E aRd 'K= 1 ' n / n , lmsupA max Amin n +oo a . s . provided on k =limsupA hor: <00 and E<1 /(c (1+2k ) )~ o n +00 max mln 1 0 where c is a constant independent of k , k o and E . 1 Fer proof we refer to (Chen & Guo , 1987e) . This theorem means that the estimate e gi ven by ELS is robust in the sense thatnas the multiplier E in the dom i nator f or ~ tends to zero the estimation erro r also n goes to zero , if persistent excitation con dition is satisfied . When we disturb a desired control by a di.ther with constant va riance (Caines & Lafortune , 1984;Chen , 1984 ; Chen & Caines , 1985a) , then we can remove the boundedness assumption on k ' o Let the control u applied to the system (99) - (101) be n ( 102 ) where and E
U
n
O is T'- measurable desired control n is giOen by (30) .
Theo r em 16 . If Conditions 1(S=2) , 2 , 4 and 5 hold , control (102) is applied to system (99) - (101) and i f n
L
U
i=O
n- 1
where 6( x) =(6 ,x= ~ , J (u)=~ I (Y:Q1y,+u~Q2 11.)' , x~ n ni~ O l l l l
(11y, W+llu,112)~Mn , Vn >O , l
l
then the Sr.S algorithm is robust in the sense that there exists a consta nt E*>O such that
limsupJ (u)=J(u) and minJ(u)=trSCR CT • n+oo n uEU
limsu~len - elkcE ,
The proof of this theorem is similar to that for Theorem 13 , and the convergence rat e follows f r om Theorem 6 . # ROBUST IDENTIFICAT ION Since a rea l system is rare to be exactly modeled by a linear deterministic or sto chastic system it is of great importance to analy ze t he influence of the unmode l ed dy namics upon the behavior of an adaptive control ~ystem,such as stabilit y , c~ntrol perf ormance and accuracy of the parameter esti mation and so on (Sga rdt , 1980 ; P.ohrs , 19 82;Riedle & Kokotovic, 1984 ; Kreisselmeier & Anderson, 1986 ; Goodwin , Eill , Mayne & ~iddle ton 1986;Ioannou & Kokotovic , 1984 ; Ioannou & Tsaklis , 1985 ; Bitmead & Johnson Jr ., 1986 ; Chen & Guo , 1987e ) . Con sider the foll owing stochastic system with unmodeled dynamics ~n : A(z)Yn+1 = B(z)un+C(z)wn+1+~n,n~ O
(10 1 )
with a E: (O , 1) , OO. Theorem 15 . If for system (99) - ( 1 01 ) Condi tions 1(6=2) , 2 and 4 hold , then for en given by ELS
11 J (ua) - minJ(u) II=O(n - (P A d) a . s . n
,
I1 ~ Ik E L an - l (i I y, 11+11 u, 11+11 w 11+ 1 ) l l 0 n i=O
(99) (100 )
whe r e
~
VEE( O, E*) n +oo and c are constants independent of
E.
The proof consists in verifying finitene s s of k . ? o r details we refer to ~hen & Guo , 1 988 0 ) • We now design a robust stochastic controller . Conditi on 9 . m=~ , zeros of detA(z) and det B(z) lie outside the closed unit 1isk , and the up per bou:1d s for 11 e I1 , trR ,11 B:; 11 , s u !) 1I A- 1 ( e i lII -s up 11 A- \ e i \: (i ~ 11 oE:[ :J , 2"1'J
I! B- \eic)A(i
o ~ lO , 27Tl
are ava il able and they are den ote d by a S, Y, KAB , KAC . K3A and Kec respect iv ely . Let Iy~ } be the reference signal ,lly*II<£ ,¥ n . Defi ne u ' fro~ n 0 n B u ' =(3 u _eTe )+y* 1n n 1n n n n n+1 if detB ~ O , oth erw is e u' =O , whe r e B i s the 1 1n est i ~atenfer B given b9 e • Choose constants Mo and M1nlarge enough so
Identification and Adaptive Control for Stochastic Systems
that Mo~ (32(Ct+l
'f y2 (p+q+r)+2) ( (4Kh+1
)M~
+(4Klc+1 )6+2(\.l0+1)) M1,32(Ct+l )2y2 (p+q+r) ( ( 4K
1B+l) (lJ2+1)
+(4Klc+1 )6+2 ) where 2 2 2 M'=2lJ o 0 +8K BC 6+1+16K BA M"0 ' M"=6£2+6(Ct+l )2lJ2+66+8K 2 (lJ 2+1 )+4K2 6+~ o 0 0 AB 0 AC' a We design adaptive control u as n
Ua=Uo+E n n n where En is given by (30) .
(103)
o u~,if nE[Tk' Ok) for some k un={ O,if nE[ok,T + ) for some k k 1 and stopping times {T k} , {ok} are defined as follows j -1
1
1
0k=SUp{t>Tk:.L IluiW~M o(j -l)+llukW,VjE (Tk't]l l=Tk °k- 1
Tk +1 =inf{ t>ok: 2k. L 11 ui W",t,11 l=T k
u~ W:SM 1 t}
Theorem 17.If for system (99)-(101) Condi tions 1(6 =2) ,2,4,5 and 9 are satisfied and the control is given by (103) ,then applying ELS algorithm to the modeled part of (99) leads to 1 n 1) limsup::;- L Oly.W+llu.112)
2) limsup::;- 'i. Ily . -y'.'W ,,0(E2) +trR+IIBlI12lJ2 n+oo ni=O l l 0 3) limsu~le-e II= O(d n+oo n
a.s.
where E,R and lJ are given by (101),(33) and (30) respec£ively. For Proof we refer to (Chen & Guo,1988). # This theorem means that ELS estimation al gorithm and the adaptive control given by (103) is robust with respect to the unmodeled dynamics. For SISO system assuming the driven nois8 to be a stationary ergodic process and restricted to the reference signal satisfying some equation Guo & Ch en (1987) have succeeded in designing a stochastic adaptive tracking control motivated by a po18-assignment method so that the results similar to conclusions of Theorem 17 remain valid,but the minimum phase condition for the modeled part of the syste m and the upper bound restrictions required in Condition 9 are no longer needed and the SPR condition is assumed for 1 D(z)C- (z) -~ with D(z) not necessarily being 1.To completely remove SPR condition Guo & Moore (1987) proposed a spectral factorization approach for a class of linear models. CONCLUSION We have given convergence analysis of recursive estimation algorithm for coefficients of ARMAX model and have shown that conditions for consistency are satisfied if a dither tending to zero is added to con-
59
trol.Then we heve given a consistent estimate for orders of the feedback control system.When the upper bounds for orders are available we have designed optimal adapti ve controls which lead to consistency of parameter estimates and to minimality of control performances.Finally,we have given the robustness analysis of parameter estimation and adaptive control when the system contains the unmodeled dynamics.The emphasis of the paper is on stochastic systems,but results for deterministic systems are also demon strated . In the field of parameter identification and adaptive control there many problems belonging to further study : 1) Design robust adaptive controller for stochastic systems under conditions more general than those used in Theorem 17. 2) Considering adaptive control problems for time-varying systems is of great impor tance in both theory and application. 3) It is desirab18 to extend order estimation result to the case where upper bounds for true orders are unavailable and to weaken the SPR condition imposed on the noise model. 4) For continuous -t ime stochastic systems there some results given in (Chen & Moore, 1987:Van Schuppen,1983:Chen & Guo,1987g), but many problems are still open,for example ,the existence of the strong solution for the system of stochastic differentia l equations arising from identification and adaptive control :t he continuous - time version of attenuating excitation method and so on. REFERENC ES Akaike,H.(1969) .Fitt ing autoregressive models for prediction ,Ann. Inst .Statist. Math. ,21,243-247. Akaike,H.(1977).On entropy maximization principle,in Applications of Statist i cs Ed . P. R. Krishaiah, 27- 41 , Amsterdam, NorthHolland. Anderson,B. D. O.& C.R.Jo hnson J r., Exponenti al convergence of adaptive identification and control algorithm,Automatica, 18,1-13,(1982). Anderson,B.D. O.& J. B.M oore (1971) , Linear Optima l Control , Prentice Hall .-----Astrom,K.J.& B.Wittenmark (1973) , On selftuning regulators,Au tomat ica,9,185-195. Astrom,K.J.& B. Wittenmark (1985),The selftuning regulators revisited, Proceedings of the 1985 IFAC Symposium on Identificat io n and System Parameter Estimation . Bitmead,R.& C.R.J ohnson Jr ., Discrete averaging and robust ident i fication,in Advances in Control & Dynamic System8:Ed . C. T.Leondes,Vol.24. Box,G.E.P.& G.M.Jenkins,Time Series Analysis,Forecasting and Cont ro l ,Holde n - Day, San Francisco,(1970). Caines ,P. E.& S.Lafortune (1984) ,Adaptive control with recursive identification for stochastic linear systems, IEEE Trans .Autom.Contr ol ,A C-29,312-321. Caines ,P. E.& H.F.Chen (1985), Optimal adaptive LQG control for systems with finite state process parameters IEEE Tran s. Autom.Control,AC-30,185-189. Chen,H.F.(1984J.Recursive system identification and adaptive control by use of the modified least squares algorithm, SIAM J.Control & Optimiz, ,22,758-776. Chen,H.F.(1985J.Recursive Estimation and Control for Stochastic Systems,New ~ York,John Wiley. Chen,H.F.& P .E.Caines (1985) .Adaptive li-
60
Han-Fu Chen
near quadratic control for stochastic discrete-time systems,IMA J.Control & Information,2,319-334. Chen,H.F.& P.E.Caines(1985).Strong consistency of the stochastic gradient algorithm of adaptive control,IEEE Trans. Autom.Control,AC-30,189-192. Chen,H.F.& L.Guo(1985a).Strong consistency of para~eter estimates for discretetime stochastic systems,J.Syst.Sci.& Math.Scis.5,81-93. Chen,H.F.& L.Guo(1985b).Strong consistency of recursive identification by no use of persistent excitation condition,Acta Math.Appl.Sinica, (English Series) 2,133-145. Chen,H.F.& L.Guo(1986a).The limit of stochastic gradient algorithm for identifying systems excited not persistently,Kexue Tongbao (Science Bulletin),1986,6-9. Chen,H.F.& L.Guo(1986b).Optimal adaptive control with quadratic index,Int.J.Control,43,869-881. Chen,H.F.& L.Guo(1986c).Strong conSistency of parameter estimates in t he optimal adaptive tracking systems,Scientia Sinica (Series A),29,1145-1156. Chen,H.F.& L.Guo(1986d).Convergence rate of least squares identificat i on and adaptive control for stochastic systems, Int.J.Control,44,1459-1476. Chen,H.F.& L.Guo(1987a).Asymptotically optimal adaptive control with consistent parameter estimates,SIAM J.Control & Optim.,25,558-575. Chen,H.F.& L.Guo(1987b).Optimal adaptive control and consistent parameter estimates for ARMAX model with quadratic cost, SIAM J.Control and Optim.,25,845-867. Chen,H.F.& L.Guo(1987c).Adaptive control via consistent estimation for deterministic systems,Int.J.Control,45,21832202. Chen,H.F.& L.Guo(1987d).Consistent estimation for the order of the stochastic control systems,IEEE Trans.Autom.Control,AC 32,531-535. Chen,H.F.& L~Guo(1987e).Robustness analysis of identification and adaptive control for stochastic systems,Systems & Control Letters,19,131-140. Chen,H.F.& L.Guo(1987f),Stochastic adaptive control for a general quadratic cost, J.Syst.Sci.& Math.Scis,7,289-302. Chen,H.F.& L.Guo(1987g).Continuous-time stochastic adaptive track i ng:robustness and asymptotic properties, Technical Report,Institute of Systems Science, Academia Si nica. Chen,H.F.& L.Guo(1988).A robust stochastic adaptive controller,IEEE Trans.Autom. Control,to appear. Chen,H.~.,L.Guo & J.F.Zhang(1988).ConBistent order estimation for linear stochastic feedback control systems,Automatica, to appear. Chen,H.F.& J.B.Moore(1987).Convergence rate of continuous-time ELS parameter estimation,IEE E Trans.Autom.Control,AC-32, 267-269. Clarke,D.W.(1984).Self-tuning control of non-minimum phase systems,Automatica, 20,501-517. Egardt,B.(1980).Stability analysis of adaptive control systems with disturbances, Proc.JACC,San Francisco,CA. Goodwin,G.C. ,D.J.Hill,D.Q.Mayne and R.H.Middleton(1986).Adaptive robust control (Convergence,Stability and Performance) Proc.25th IEEE Conf.on Decision & Control,Athens,468-473. Goodwin,G.C.,P.J.Ramadge & P.E.Caines(1981) Discrete-time stochastic adaptive con-
trol,SlAM .T.Cor.ltrol & Optim.19,829-853. Goodwin,G.C.,P.J.Ramadge & P.E.Caines,Discrete-time multivariable adaptive control,IEEE Trans.Autom.Control,(1980), AC-25,584. Guo,L.(1987).Identification and Adaptive Control of Dynamic Systems,Dissertation. Guo,L.& H.F.Chen(1987).Robust stochastic adaptive control for non-minimum phase systems,Systems Science & Math.Sciences,to appear. Guo,L~ J.B.Mcore(1987).Adaptive spectral factorization with applications to system identification,Technical Report, Dept.Syst.Eng.,Australian National UniverSity. Hannan,E.J.& B.G.Quinn(1979).The determination of the order of an autoregression, J.Royal Stat. Society,B.41,190-195. Hijab,O.B.(1983).The adaptive LQG problem -Part 1 ,IEEE Trans.Autom.Control,CA-28 171-178. Ioannou,P.A.& P.V.Kokotovic(1984).Robust design of adaptive control,IEEE Trans. Autom.Control,AC-29,2-2-211. Ioannou,P.A.& K.Tsaklis(1985).A robust direct adapt.ive controller,Report 85-071 ,Electrical Engineering Systems,University of Southern California. Johnston,R.M.& B.D.O.Anderson(1983).Global adaptive pole placement:Detailed analysis of a first order system,IEEE Trans.Autom.Control,AC-28,853-855. Kreisselmeier,G.& B.D.O.Anderson(1986).Robust model reference adapt.ive control, IEEE Trans.Autom.Control,AC-31,127-133. Kumar,P.R.(1983).Optimal adaptive control of linear-quadratic-gaussian systems SIAM J.Control & Optim.,21,163-178. Lai,T.L.& C.Z.Wei(1986).Extended least squares and their application to adaptive control and prediction in linear sys~ tems,IEEE Trans.Autom.Control,AC-31, 898-904. Landau,I.D.(1979).Adaptive Control:The Model Reference Approach,Dekker,New York. Ljung,L.(1985).On estimation of transfer functions,Automatica,21 ,677-696. Ljung,L.& T.Soderstrom(1983).Theory and Practice of Recursive Estimation,MIT Press. Meyn,S.P.& P.E.Caines(1985).The zero divisor of multivariable stochastic adaptive control systems,Systems & Control Letters,6,235-238. Moore,J.B.(1978).On strong consistency of least squares identification algorithm, Automatica,14,505-509. Riedle,B.D.& P.V.Kokotivic(1984).Disturbance instabilities in an adaptive system, IEEE Trans.Autom.Control,AC-29,822-824. Rissanen,J.(1978).Modeling by shortest data description,Automatica,14,467-471. Rohrs,C.E •• L.Valavani;M.Athans & C.Stein (1982).Robustness of adaptive control algorithms in the presence of unmodeled dynamics,Preprints of 21st IEEE CDC,Orlando,FL. Samson:G.(1983).Stability analysis of adapt.ively controlled systems subject to bounded disturbances,Automatica,19,8186. Schwarz,G.(1978).Estimating the dimension of a model,Annals of Statistics,6,461464. Van Schuppen,J.H.(1983).Convergence results for continuous time adaptive stochastic filtering algorithm,J.Math.Anal. Appl.96,209-225. Soderstrom,T.& P.Stoica(1985).Instrumental methods for identification of dynamic system, Proceedings of 1985 IFAC Sym-
Identification and Adaptive Control for Stochastic Systems posium on Identification and System Parameter Estimation. Solo , V.(1979) .The convergence of AML ,I EEE
Trans.Autom.Control,AC-24,958-96~
Zarrop,M.B.(1979J.Optimal Experiment Desi gn for Dynamic System Identification, Springer,Berlin.
61
THEORY OF SYSTEM IDENTIFICATION AND ADAPTIVE CONTROL FOR STOCHASTIC SYSTEMS* Han·Fu Chen I nstitute of Systems Science, Academia Sinica, Beijing, PRC
Abstract. This paper presents a series of theoretical results on parameter identification and adaptive control for discrete - time stochastic feedback control systems . Without a use of persistent excitation condition the unknown coefficients of the system are consistently estimated by use of the extended least squares (ELS) and the stochastic gradient (SG) algorithms . Then a strong consistent estimate for orders of the system is also given by minimizing an information criterion designed for feedback control systems . In order to apply results of parameter estimation to adaptive control systems an attenuating excitation technique is introduced so that for strong consistency of the parameter estimate the conditions imposed on eigenvalues are replaced by the growth rate restriction on the input - output of the system . Further , optimal controls are designed respectively for tracking a given reference signal and for minimizing a quadratic loss fun ction so that the parameter estimates are strongly consistent and , simul taneously , the performance indeces are minimized. When the system contains an unmodeled dynamics but the adaptive control is designed on the basis of the mode led part of the system a robust analysis is presented by esti mating the error in parameter estimation and in control performances . Keywords . Identification ; adaptive control ; strong consistency ; adaptive t r acking ; adaptive LQG problem;order estimation;robustness . INTRODUCTION The task of system identification is to con st ruct a mathematical model of a system by use of i t s input - output data . Normally , the s y s tem inp ut is not arbitra r y but is de s ign e d to satisfy a certain control purpose whil e identifying the system . This is the topic of ada ptive control . In this pa per we only concern feedback cont r ol systems in time domain and leave ex per ime n t d esign (Z a rrop , 1979) , identification in frequency domain (Ljung , 1985) and a da ptive control for Markov chains (Kumar , 1 985) al one . To be specific , we discuss the di sc rete - time deterministic or stochastic input - output model called ARMAX process . A(z)y =B(z)u +C(z)w , y .= O, u. =O, w.=O,i
Assume {wn , Tn } is a martingale dif f erenc e sequence with respect to a nondecreasing f a mily {T n} of o - algebras E ( wn / T n _ 1 ) = 0 , ... n~ 0 •
The control u depends upon the past mea surement (y o . \Jn .yo u " ' U n - 1) , i.e. u n is T n measurable . The feedback nature of u makes our identification problem differen£ from time series analysis (Box & Jenkins , 1970) . In this paper we first estimate the unknown matrix coefficient 8 by the ELS algo r ithm and give its convergence rate assuming upper bounds for system orders a r e a vailabl e. Then we discuss the simp l er comput a ble SG algorithm to estimate 8 and the projection algorithm for deterministic system s (w =0) . All these results are characte r i z ed byntha t the persistent excit a tion condition , which is usually used in literature (Moo r e , 1978 ; Solo , 1979 ; Ljung & Soderstrom ,1 983 ; Chen . 1985 and Soderstrom & Stoica , 1985) is not i mposed here . However , for strong consistence of parameter estimation there are still so me conditions on re a lizations have to be assu1 2 q med.In order these not easily ver if i a bl e r C(z)=ItC zt ••• tC z , r~O (4) conditions to be satisfied we app l y the a t 1 r tenuating excitation method which mea n s wit h unknown or ders p , q , r and unknown mathat the desired control is distur bed by a tr i x coe ff icient dither tending to zero as time tends to in finity . 8 T = ( - A ••• AB ••• BC · •• C ) (5 ) 1 p1 q1 r For order estimation we introduce a n inf or {y } is the m- output and {u } is the ~ - in mation criterion designed fo r feedback con pu£. When w =0 the system n(l) degenerates trol systems and by minimizing it we ge t to the d e t~ rm inistic one , otherwi s e , C(z)w the consistent estimate for orders . Th us, r ep r esents the dynamic nois e of the syst~m . strongly consistent estim a tes for both the *Wo r k supported by the National Natur a l Science Foundation of China and by the TWAS Re sear ch Gra nt No . 87-43 . 51
52
Han-Fu Chen
system coefficients and the orders can be obtained simultaneously. By using the estimation results we design optimal adaptive controls for tracking a given reference signal and for minimizing a quadratic loss function so that the control performances are minimized and the parameter estimates are strongly consistent. Finally,we consider the influence of the unmodeled dynamics possibly cont ained in the system upon the parameter estimation and upon the adaptive cont r ol which are designed on the basis of the modeled part of the system.It is shown that errors arising in parameter estimation and in control performances are of order E where e is the multipIer of the dominator for unmodeled dynamics of the system.
follow i ng estimate (G uo,198 7 ; Chen & Guo, 1987a) for the weighted sum of a martingale difference sequence (wn,r n ) with
~~bE(llwn+l W/'n)<= a.s. a£[l,2)
L Mowo+ =0(S n (a) loga i=O 1 1 1
By use of Condition 2 it is proved that td8-8 n +1 )Tp~ :l (8-8 n + l )
(6)
with 8 0 arbitrarily chosen,Po=dI,d=mp+~q+mr. It is easy to verify -1
n
T
1
n
n
L 1I(8-8 1o+1 )'$oW-2 L w\1(8i=O 1 i=O 1 n
P +l=P -a P $ $T p , a =(1+$T p $ )-1(7) n n n n n n n n n n n $T=(yT ••• yT uT"'U T T_~T 8 n-p+l n n-q+1 Yn wn _l n n n ···y~-r+l-$~-r8n-r+l) (8)
P +1= L $o$o+-d I n i=l 1 1
(13)
-8i+l)T$ o~0(1) + 0( L ao$:po$ollwo+ 1W) a.s. 1 i=O 1 1 1 1 1
In this section the system orders are not estimated and their upper bounds p,q and r are assumed to be known. One of commonly used methods for parameter estimation is the ELS algorithm which is the extention of the least squares method to the correlated noise and is defined as follows
8n+l=8n+anPn$n(Y~+1-$~8n)
(Sa(a) +e) ) n
a.s."'v'"n >O ,where M is any' -measurable random matrix of ncompatibl~ dimension and S (a) = ( E IIMolla )l/a n i=O 1
~O(l)-ko PARAMETER ESTIMATION FOR STOCHASTIC SYSTEMS
~+n
n
(9)
By A~~X and A~in denote the largest and the smallest eigenvalues of Pn-1+l respectively. For stron~ consistency of 8 it is usually required (Ljung & Soderstro~,1983;Moore, 1978;Solo,1979 and Chen,1985)that An /Ano
6(x)={l,x=0 O,Xl' O Proof.The proof is essentially based on the
for any n~O.Then the conclusion of the theorem is separately verified for two ca ses n ==.The key point limA n <= and n~oo limA max n~oo max is to show that n
o+ W=o(logdet(P-: ) L ao$:Po$ollw 1 1 1 1 1 1 n l
i =O
(10glogdet(P~:1))C6(S-2) •
For details we refer to (Chen & Guo,1986d).
# This theorem says that 8 is strongly consistent with convergencenrate indicated by (12) if Condition (11) is fulfilled and i f S>2. We note that An and Ano are estimatemax mln dependent hence so is the right-hand side of (12).We now remove this undesirable dependence by using
$~T=(y~",y~_P+1u~ ••• u~_q+lw~"'w~_r+1) $~=O,i
a.s.
if 10gA on (loglogA on )c6(8 -2 ) =0(Ao~ ) for max max mln some c>l. Proof.The proof consists in verifying logAn (lo glog An )c6(S-2) max max =O(logA on (loglogAon )c6(S-2)) # max max • We now introduce
w~=(y~,··y~_P+lu~ ••• u~_q+1Y~-W~_len_l ••• yT _WT 8 ) n-r+1 n-r n-r where for estimating wn the aposteriori
Identification and Adaptive Control for Stochastic Systems estima te yn - S~ _ 1~n _ 1 r ather than the po s t er iori e s t ima te yn - S~~n _ 1 a s in (8) is us ed . The SG alg or ithm fo r estim a ting S is defi ned as f ollows Wn T T Sn +1=Sn+ r (Yn+1 - WnSn) (16) n n
r =1+ L Ilw. n i=1 l
W,
( 17)
r =1 0
with So a rbitrarily chosen . condition 3. C( z) - ~I i s strictly positive r eal . Theo r em 3 . If Conditions 1 (with 8=2) and 3 are s a tisfied , r ~oo and if there are a E( O,iJ a nd N nand M possibly dep e nding on w such tha~ ( 18) r n +1 / r n ~M ( logr)a a.s. n n
~ max
/
n
~min~
(19)
n ~No - 1 ,then 8 n n 100 8 a . s . where 8 n is defined by ( 16) ( 17) and ~~ax and ~~in respectively den ote the maximum and minim um eigenvalues of ~L (W .W.T +aI 1 ). i =1 l l The conc l usi on of the theorem remains valid if ~n , ~n . and r are replaced by ~on , max mln nn max ~o~ and r O =1 + L 11 ~~ W respectively . i=1 l mln n
Proof . The theorem is proved by investig at ing beha v iors of th e mat r ix froduct WnW~ WoW o 'I'(n , O)=(I -- - ) '" (1 - - ) . rn ro It can be shown that under conditions of the theorem 'I'(n , O) ~ 0 and this ~uarantees stron g consistencen~f 8 . DetailsOare given i n (Chen & Guo , 1985a , n 1 987b) . From Theorem 3 we see that for st r on g con sistence of 8 g iven by SG the persistent excitation i snn ot hequired since Condit i on (19) al l ows ~n /~ . to d i verge . Howeve r, max ITan the ord er i- a in (19) indicating the diver g ence rat e , in g eneral , cannot be enlarged to more tha n 1 as shown by an counterex a mp l e in (Chen & Gu o , 1986a ) . PARAMETER ESTIMATI ON FOR DETERMINISTIC SYSTEMS Set wn =O in (1) . Then it t urns to a determi nistic system.Thus , we consider d (2 0 ) A( z)Yn=z B( z)u n , d~ O whe r e A(z) a nd B( z) a r e define d by (2) (3) . By use of the projec t ion a l g orithm we esti mat e th e unknown parameter T 8 =(- A1 '" - Ap B1 '" Bq) with fi xe d (p , q ) as fo l lows
~n n
where 6n =8 - Sn . Then the conve r gence ra te of Sn to zero i s defined by the behav io r of the mat r i x ~(n , i) given rec ur sively by ~
~T
~(n+1 , i) = (I - 1~1I~ W)( n, i) , ~( i, i) = I
(24)
n
Theo r em 4 (Anderson & J ohnson , 1982) . Ass um e that !y ) is bo unded , A(z) and B(z) are l eft coprime nw i t h A being of f u l l r an k an d tha t th e input is Psufficie nt l y r ich , i . e . t h ere are const a nts 8 >0 , 6 >0 and an integer N>O 1 2 such that n +N S1I~ L V.U: ( S2 I ,'f n , (25 ) i=n +1 l l T T T whe r e (26) _ U.=( l u l. ···u l. - mp - q +1 ) Then Sn exponentially tends to O, i . e . Is - Sn "=O(y n) for some YE(O,1) . (27)
#
a.s.
fo r all
8 n +1 =8 n +1+11 ~
53
T) w(T y n +1 - ~ n Sn
(21 )
We now extend this well - known result in se ve r al asp e cts , namely , we will allow y to diverge , the low e r bound of (25) to gB to zero and the number of sum mands in (25) to increase unbo undedly . Such an extention not on l y is interest i ng by itsself but a l so is a s i gnificant step towards solving the ad aptive control pr oblem . Theore m 5 . Suppose that A(z) and B(z) are left - coprime with A being of full r ank and T
n
-1
P
L V.U:).6I ,V n ~ 1 (28) i =T +mp l l n n-1 for so~e sequence of non - negative numbers 6 >c / n , 6>O , possib l y tendin g to ze r o and n integers iT } with T =1 , d =T -T 1~mp a nd n 0 \ n n nthat.lI~ II=O(nv) , d =O(n ) for some constants n n v).O , Ap O, 6p O and c >O satisfyin g 4( 1+\)vt26 +5 A< 1 . The n 1 - 2 6- 5\ - 4(1+\) v 118 - 8 II=O( e xp( - an n
1+\
))
(29)
where a is a positive const a nt . Proof . We point out the key steps of the proof.First , it is established that N c N- d+1 A . ( I. ~·~:)~N k 0+1 \ . ( L V.V: ) mln i=k l l - - mp ml n i=ktmp -d +1l l fo r any N9k +mp and k~O , whe r e c is a cons tant and \ . ( X) denotes the °min im um ei g env a lue mln of a mat r ix X. Th e n , it is shown that 6 2 , ,\, 11 ~ ( N , k) lid 1 - 4TN"=kT r N- 1 ~ . ~ : where
.L
1+tl~~W~ 6 I , 6>0 .
l=k l Finally , it is concluded tha t n
6~
--t5)
11~(T n+ d - 1 , O) lI~exp( - c1.~ l- 1M d i i =0 (
exp
(_
c2
( nr1)1~ 6 - 5 \ - 4 v (1+\ ) )
1 - 26 - 5\ - 4v(1 +\}
SlJ P II~ . W+1 , c1>O,c2>0 f r om T.1- 1~j~ T.-1 J 1 wh i ch (29) f oll ows if we note T ~.F o r det a ils we r efe r to (Che n & Gu o , n1987 c ) . # Remar k . Theo r em 5 g iv e s a ne ar exponenti a l r a t e of conve rg enc e. Se tt ing v =O, \ =O an d 6 =0 in (29) we g et an e xp on ent ia l convergen c e with Mi=
54
Han -Fu Chen
rate wh ich coincides with the one given by Theorem 4 . PARAMET ER IDENTIFICATION WITH ATTENUATELY EXCITED CONTROL From Th eor ems 1 - 5 we see that for consisten cy of parameter estimates some conditions are imposed on eigenvalues depending on re alizations (see (11) , (12) ,(1 5) , (18) , (19) , (25) and (28» . These conditions are diffi cult to verif y and are often not satisfied f or adaptive control systems as shown in (Chen & Guo,1987a) . In order to get consis tent estimate a dither with const a nt vari an c e is adde d to control (Caines & Lafortu ne , 1984;Chen,1984 ; Chen & Caines , 1985) to excite the system , but , at the same time , it hurts the control performance . In the subse quent sections we change the abovementioned dither to the attenuating excitation (Chen & Guo ,1 986d,1987a,b) which sufficiently ex cites the system to yield consistent esti mate s but does worsen the control performa ce . When the attenuating excitation cont r ol is used we show that the difficu l t condi tions impo sed on eigenvalues can be replaced by the easier verifiable ones imposed on the growth ra te of the system input - out put . To be specific, let { £ } be a sequence of £ -di mensional iid rand8m vect ors with cont inu ous distr ibution and let it be i ndependent of wn and such that E£ n =O,£E n E'n =UI , liE n Iku 0
a . s ., u>O
(30)
Without loss of generality assume 'n=o{wi ' £i , h n }, '~ = o{wi , Ei _ 1 ,i~n } Let " -measurable Uo be the desired control . n n The attenuately excited control un applied to the system is def ined by (31) u =uo+v n
where v =E
n
n
~E
1 , £ f[ O '~ t+1 ) , t=max(p , q , r)+mp-1 ~\vT I) (32) Con d ition 4 . There is a posit i ve definite matr i x R>O possibly depe~ding on realiza tions so that n
n
In
n
lim2 ! w.w:=R n ni~1 1 1
a.s.
(33)
Condi ti on 5 . A(z) , B(z) and C(z) ha ve no common left factor and A i s of full rank. p Theorem 6. Suppose that Conditions 1 , 2 , 4 :~~ ~h:~e a~U~::~ led and un is g iven by (31)
2
6 Y. (lIy. W t llu?W)=0(n ) , a . s .
ni~1
1
The similar t heorem is t rue for SG algorithm . Theorem 7 . Assu me that Conditions 1 (6=2) , 3 ,4 and 5 are satisfied and un is given by (31) but with vn defined by
v1=o,vn=En/(10gn)~£ ' E E (O ' 4s(m+2»,n>2(36) s=max(p , q,r+1) instead of (32) .If
2 T.
n i =1
(1Iy.W+!Iu?W) =0(10 g6 n ) a.s. 1
1
for some 6E(O,(i - (m+2)sE! / (2 +(m+lls) as n+
r
r
#
(34)
1.
1-2E{t +1) co for any 6f ( 0 , 2t+3 ) . Then .or en given by ELS '(3 2' lie _ eW= O((logn)(log lo gn ) C, - ) , a . s . (35) n na for any c>1 and any a E ( '( 1t 5) , 1- (t t1 )(£ t 6» where 6( · ) is defined in lheorem 1 . Proof. Condition (34) i~plies An =0( n 1 +6 ) , max so i t suffices to show An. >c n a , c >O ,V n~n mln 0 0 0 for some n . This can be proved by that a converse ~ssu mption leads to a contradic tion.For details we refer to (Chen & Guo , 1986d ) . #
(3 7)
ORDER ESTIMATION Fo r system (1) u is defined by {y. , i~n}, so , in general , itnis not stationary1or ergodic.In this case the order estimation prob le m is more complicat ed than that for time series for which the orders are esti mated by minimi zi ng AI C (Akaike,1969),BIC ( Akaike,1979;Rissanen,1978 ;S chwarz , 1978) and ct IC (Hannan & Qu in n ,19 78) . In this section we assum e that true orders for A(z),B(z) and C(z) are p , q a nd r respectively. 0 0 0 Cond i tion 6 . The tr u e orders (po , qo ,r ) o belong to a known set M: M={(p , q,r):O,p,p·,O,q~q*,O~r'r*}.
Identification and Adaptive Control for Stochastic Systems
55
Condition 7. A sequence of posi tive numbers {an} can be found such that (logr o ) (loglogr O )O o(8 - 2 ) n n +0 a.s. (42) an n -+00
ted noise case. Theo re m 9 . If Conditions l, 2,6and 7 hold , then (Pn , qn,r n ) n!oo (po , qo , r o ) a . s .
for some 0>1 and a h (!? ,q,r) (n) +0 n+«> n m~n for any (p,q,r )€M* consisting of three points: M*={ (po,q*,r*),(p*,qo,r*),(p*,q*,r o ))
where (Pn ,q n , r n ) is given by (55) . Proof . The first step is to show n- 1 ( I. 11c;.+1 J12=O((logr o ) (loglogr o)c O B- 2))
where A(!?,q,r)(n) den otes the minimum eigen-
i~O
m~n
value of n-1
l
o( ~n
~? ( p,q,r )~? ' ( p,q,r)+~I,d=mp *+ lq * + m r * ~
(44)
p,q,r )_(" - Yn"' Yn -p+1
u "n ",u n _q +1
w'n ... w'n-r+1 l' and r
O
n
-~n= (' , , , l' Yn"'Yn_p *+1un",un_q *+1wn ",wn_r *t1 ~,~,
(46) where the estim ate ~ for wn is recursively defined as follows: n ~n=yn-8~~n-1' n~O, wn=O,n
(47)
8n+1=8n tanFn~n (Y~+ 1-~~8n)
(48) 1
F n +1=F n -a nF n~ n~'F n n ,a=(1+~'F n n n~ n r (4 9) with arbitrary 8 0 and Fo=dI. For any (p,q,r)€M s et e(p,q,r) = (-A1···-ApB1···BqC1···Crl"t J
IJ2 ,li B 1J2 ,IIC
Po
qo
ro
1J2} >O
and
(S , t , A)=(p po , q qo , rvr o ) ' Finally we show that any limit point of (Pn,qn , rn)coinc i des with (po , qo , ro) . For de tails we refer to (Chen,Guo & Zhang , 1988).# We now specify { a \ used in Condition 7 when the attenuat~ng excitation contro l is applied. Theorem 10 . If Condit.i.ons 1 , 2 , 4 a nd 6 ho ld, A(z),B(z) and C(z)h ave no common left-factor and A , B and C are of full rank and Po qo qo if applying control (31)(32) with t=(mt 1 )p* +q *+r* -l leads to 1 n -ni=O l (Jly 1· lJ2 t llu 0~. W)=O(n 0 )
a.s.
(57)
for some oEl O, (1 - 2E(t+1)/(2tt3)1 , then (50)
where by definition A.= O,B. =O , C =0 for i>po n
j>qo and bro' At time n e(p,q,r) is estimated by ELS n-1 , 1 -1 e (p, q ,rl = ( l ~. ( p, q, r) ~. (p , q , rl +ar) n i=O ~ ~ n-1 . i l=0 ~.(p,q,r)Y~+1 (51) 1. ~ where ~ n ( p,q, r )= ( Yn' ••• Yn-p+1 ' u'n ... u'n-q+1
w'n .. ·~'n-r+1 l' ~n
o =min[JIA
o
L 11~ ?( p * ,q * ,r *)IJ2 . i=O ~
~
"4
m~n
for any (p , q , r)EM ,w here
( 45)
Denote the regr ession vector co rresp ondin g to the system of the l argest possib l e or ders by
and
n
a n (p+q+ r- po - qo -r o t o(1)) a . s . if (S , t , A) >{ ( ) 0 =(p , q ,r ) A ~ , t ' A (n)( °to( 1 )) a . s . otherwise
n-1 =1+
n
CIC(p , q , r)n - CIC(po , qo , ro)n
i=1 ~ with
~
a.s. for any c>1 . where C;n=wn - wn . Then we pr ove that
(52)
is g ive n by (47) .
We now introduce a new information criterion CIC(p,q,r) where the first " CH means that the criterion is designed for feedback control systems: CIC(p,q,r ) n=on(p,q,r ) +( p+qtr)an (53)
and a is given in Condition 7.The estimate (pn,q~,rn) for ( po,qo,r o ) is g iv en by minimizing CIC(p,q,r)n (Pn,qn,rn)=arg min CIC(p,q,r)n (55) (p,q, rlE M The following theorem generalizes result given in (Chen & Guo,1987d) to the correla-
(Pn , qn ,r n ) n!oo(po , qo ,r o ) a.s. where (Pn , qn , r n ) is given sat.isfying (logn)(loglogn)a6(B - 2) a
and
n
a /n 1 - (t+1)(r:tO) n
n++0oo for some 0>1 ( 58)
+0 n-?oo
(59 )
where E is defined by (32) and t by (56) . Proof. Noticing 1 - ( tt1)(E+6);:d1 - E)/2>O ,w e see that a =nu with any uE(O , l - (l+t)(E +O)) meets re qu~rements (58)(5 9).By (526 and 1to Condition 4 it is easy to see rn=O(n ). So the critical point i s to show liminf n - aA(J? , q , r)(n)=O a . s . mln n+ oo for any (p , q , rl EM* where 0=1 - (tt1)(£ tS). Then the conclusion of the theorem follows from Theorem 9 . # SIMULTANEOUS LY IDENTIFYING SYSTEM PARAMETER AND TRACKING A REFERENCE SIGNAL Since sevent i es there has been a lot of research made on adaptive control such as seIf-tuning reg ulator (Astrom & Wittenm a r k, 1973,1985;Clarke,1984),model r eference adaptive control (La ndau,1979) and adaptive pole-assignment ( Johnston & Anderson , 1983) .
56
Han-Fu Chen
We now de s ign control u in or der that the output y of system (1)n with known orders p , q an d nr follows a given reference signal . In (Goodwin , Ramadge and Caines , 1981) the control un is selected to satisfy
8~Wn=y~+1 (60) where 8 is given by SG algo r ithm . There it is p r ov~d that 1 n
L
l~!ll~ u~
n i =1
(lly . W+lluiW)
a . s.
(61)
a .s .
(62)
and
(66) where B1 is the estim a te for B1 g iven by le f t - cop 8 • 2) n1f A(z) and B(z) are n rime with A being of full tank , d =1 a nd zer os of p det(B(z)(I 0 ) ) li e outside the closed unit disk and mthe cont r ol u is defined by (31)(38)(66) , then (y) a nd (U) n n are bounded a . s . and Ily - y * ll=o( ~ ) 118 - 8 IFO(exp(-an1 - 12(mp+q)e:~u)) n!'1
where R is the limit matrix in (33) . I n th i s well - known result there are two pro blems which remain open:Is (60) solvable with respect to u ?Is 8 strongly consistent un der such design~d adaBtive control?In fact , the control designed by (60) , in genera l , d oes not lead to consistency of the parame te r estimate . For example , if y*~O , then T T T 8nw n ~O , 8 n (8 n +1 - 8 n )=8 n w n y~+1/r n n =0 a n d hence T T 8 n 8 n =8 n - 18 n- 1+( 8 n - 8 n- 1?(8 n - 8 n - 1) n
= 8~80+ i~ 1 (8 i - 8 i _ 1 ?(8 i - 8 i _ 1 h8~80 ,"" n>1 . does not converge to 8 if 8'8 n o 0 The following theorem solves the problems raised above . The solvability of (60) is al so discussed in (Meyn & Caines , 1986) . Theorem 11. 1) If for system (1) m",£ , u . , i
£:
n
'
n a . s . (67) Proof . It is easy to see ooI18:ilJ .W ,_ L _ _1_1_
Hence , Ile'ilJ. W=0(1+llilJ.112) . By (66) we have 111
-,
.
yn+1 - y~+1=8nilJn+B1nEn
and by boundedness of
(69)
(y~l
IlilJ n +1 W=O(1)+o(s~p IlilJ' t1 W) . 0" J ~n J This implies boundedness of {y } and {u } , and the theorem f ollows from n(69) and n (40) with v=O . The details of the proof are giv en in (Chen & Guo , 1987c) .
So 8
B1nu~+(8~Wn - B1nun)=y~+1
(63)
is so l vab l e with respect to uO , where B1 denotes the estimate for B1 ngiven by n8 • 2)If Conditions 1(~=2) , 3 , 4 , 5 hold , m~£ , zer8s of det(B(z) ( I 0)') lie out side the closed un it disk andmthe control u is defined by (3 1 )(36) and (63) , then (6 1 )n(62) hold and 8 n n~oo8 a.s . Proof . For proof of 1) we refer to (Chen & Gu o , 1986c) . 2) We fi,st show (Chen & Caines , 1985a) that LIIC+ W/r.
The adaptive control problem with q uad r atic loss function has been considered for vari ous cases:for the case where the un kno wn e takes value only in a finite set in (K umar , 1983 ; Hijab , 1983 ; Caines & Chen , 1985) , for the case where the availability of cons i stent parameter estimates is assumed in (Chen & Caines , 1985b ; Chen , 1985) an d fo r other cases in (Samson , 1983 ; Chen & Guo , 1986b) . For the case where the noise is correlated both consistency of parameter estimates an d minimality of the loss function are achie ved in (Che n & Guo , 1986d , 1987b) . Assume that p , q,r for system (1) a re known and the unknown 8 defined by (5) is esti mated by ELS . We want to design adaptive con t r ol to :ninimize
(64)
n
J(u)=limsupl
,
~n+1 = yn+1 - wn+1 - 8nWn '
n~=
(65 )
Then by (31)(63) we have y n t1 - y~+1=~nt1twnt 1+B1nvn
from which , afte r showing liPl ~MP r n/n
SIMULTANEOUS LY IDENTIFYING SYSTEM PARAM ETER AND MINIMIZING A QUADRATIC COST
L
n i =1
((y, _y'!)TQ1 (y · -y'.')tu:Q2u. ) il
l
1
1
1
where ~130 , Q~ 0 and (yrl is a bounded de terministic Signal . ~e present system (1) in the st a te space form ( 70) Xk+1 =AxktBuktCwk+1
Yk=~xk ' whe re - A1 10 '" : 0 A= - As 0 " "
0• :
x~ = (y~O "' O)
~1
I
Urn '
, C= ?1 , H' (71) I ' B= B 0 C s s-1 m s=~ax(p , q , r - 1) , A.=O , B .= O ' Ck=0 for i>p , j>q a nd k>r . 1 J Condition 8 . (A , B, D) is cont r oll a bl e a nd observable where D is any matrix satisfying d e composition D'D =H'Q1H .
Ide ntifica tio n a nd Adaptive Control fo r SLOchastic Systems
It is wel l - known (Ande rs on & Moore , 1971) that under Conditio n 8 in the class of non negative matrices there is an unique posi tive defin it e matrix 5> 0 sat is fying and
S =A' SA _A' SB(Q2+B' SB) - 1 2, SA +H' Q H 1 1 F=A- B(Q2 +B' SB) - B'SA
1
'"
F( n) =A ( n) - BT (n) (B T (n) S B( n) tQ T\ilW S,.p.( n ) n Z .. (85)
estimate for u * proposed by the cer tainty equivalencenprinciple is
T ~e
(72)
is sta bl e , Th en (b ) is bo unded where i b.= -
57
.
L FJ'H'Qly*+· = Fb · +l - J.:'Q1Y ~ j= O 1 J 1 1
(73)
It is not difficult to see that under Con dition s 1 (S=2),4 ,8 J (u ) can be expressed as n- l J(wtrSCR C' +1 imsu~ L (y ~T. Qp".' - B( Q + BT S E rl 1 2 n+'" ni= O 1 1 1
t:
(86) but wh i ch , in general , does not lead to con sistency of B , and hence J(Qa) may be far from the ~in i~a l value J o ' We will a p oly the randomly varying tra nca tic n te~hnique which consists in tr uncating G~ at stopp ing t imes Ilk) and (ok ) defined as follows : 1 =T 1 <01
cl
0k =sup lt >Tk : .L IIY i W~(j - 1 )log-(j - 1) l =Tk
(87)
(Q 2 +B' S B) (u . + (Q2+ BT S B) - 1 BT (SAx. +b. + 1 ») 1 1 1 (74) for any uEU,w here n -l U={ u :
L
i~O
(1Il LW ~ I x. W )=O ( n ) ,1 11J II "O(n) , a . s . 1
where
1
Hence , when e is known t he optimal control is u*=Lx +d n (75) n n with L= - (Q +BTSB) - l BTSA d =- (Q +S' SB) -l B' b 2 ' n 2 n+l ( 76) and the minimal value of J (u) is J =minJ(u) =t r SC RC+
u nO =B-1 n! (Y n·" +1- (8\[, n n - B1 n u n ») with ~.= m . ~e
now define the desir e d co ntr ol n E[ Tk , ok)n Ac for some k
IO ,if
(90)
u~ , if n~[ok " k+l ) for s o me k
i' Q1 Yi +b ~ +1 B ( Q2
t
BT S B) - 1 B'b.J:+l )
(77 )
and adaptive control ua=uo+v n
n
as (91 )
n
with vn given by (36) ,
T
en =( - A1 n " ' - Apn B1 n " · Bq n C1 n " ' Crn )
be given by the SG algorithm and set -A IO .. · O , 1n
•
A(n)=
I
3
,
1
nj
: I'
B(n)= :
",'1
C(n )=
(78)
n
C _ 0 · .. · 0 l ::·sn sn s 1n (79 ) s=max (p ,q, r - l) . Given any S 30 the S is recursively esti ma ted by 0 -A
S =A'(n ) S
n-
+B'(n )S
1 AT( n) - A' ( ~. ) s l B (n» - l
n-
s T(n)S
1A(~)+::'~1~
and L,d,b ly by n n
i = (y'O .. , J) '
(81) o 0 and F are estimated res pec tiv e n
uE :J
L =_ (ST(n)S S(n ) +~2 ) - 13' (n)S A(n )
( 82)
an =- u (~'(n)S . n 3(n)tQ2 ) -1 s T(n)b n +'
(83)
n
bn =- i~I O FJT ( n - l)H ' ~ 1 y n+J " .
n
I
(84)
0
Proof . Th e f i rst step is to sh ow n
I
iktl=A (n)in+S(n)untC (n)( Yn+l-SA(n)in - H3(n ) u )
Theore::J 1 3 . If for s ys t e m ( 1 ) ~= ~ , Condi t.ions 1 (8 =2) , 3 , 4 , 8 hold , zeros of detB(z, ) l ie outside the closed unit disk and if A i s of full. rank with p~q , th e n the adactivi contro 1 iUoi) d efined by (36)(90) and t9 1) 1eads to n consistency ' oj ~n given hy (16) (1 7) a nd ~inimizes J(~) J(·l a ) =J:",inJ ( u)=J (9 2)
1 3 ( n ) (~2 +
nn( 8 0) The state xn is estimated by an adaptive fil ter
n
(89)
and denote by UOn the so1ution of (63) , i . e .
In the pre sent case both e and x, are un avai l ab l e , and we will us e their ~est i mates.
n
u'1.' Ik i log\\ , 1). 1 )
A={ i :11
u~ =l n~ ' if nE [Tk, ok)nA for some k
o uE: U
Let
i s given in (37) .
<5
Def i ne a set A of integers :
Uif:'j,"f i>O }
n- l +1 i ms u ~): (y n+ex> FO
(88)
11 0'; W= J( nl og~n) .
i:;:: 1
.,L.
:he consis~e nc ~ of S tn en f o l lows fr om :heore~ 7 . : he ~econdnstep is t: prove S
+
n n -+ oo
S
and tte :tird step is to establish that :here exi st s sone k fer '.... h i cr. T, <~ and KO ::= ;:x) . ::ence
(93) ?inally , (92) can be established by using consistency of and (93) . For details we refer to (then &n Cuo , 1987 ) , #
e
When sult
we ca n show a more crecise re lbased on the ~L S algorithm .
y~ S C
58
Han-Fu Chen
We retain notations introdu ced by (78) - (82) (85) and (86) but where e shoul d be under stood a s the estimate ngiven by (6) - (8) . We redefin e {T k } and {ok}:
where the modeled part is the same as system (1) , while ~ is T - measurable and is do~inated by n n n
1=T 1 <01
-\fj~ (Tk , tJ
°k - 1
} (94)
<-7-' 1 +6
Tk+1=inf{t>ok:, LIIL i S
(95) (96)
t =max (p ,q,r ) +m p -1. Set L , if nE[Tk,ok) for some k L0 ={ n
(97)
O, if nE[ok,T k +1 ) for some k Finally,the adaptive contro l (u a } now is defined as n ua=Lox +v (98) n n n n wher e (v } is given by (32) . n Theor em 1 4 . If Conditions 1 , 2 , 5 , 8 hold and A(z) is stable with A being of full rank (A =1) and if as n+oo p n
o
11 ~
ni
n
L w,w~ - RII=O(n - P) =1 l l
a . s . P>O ,
then control {u~} defined by (98) leads to c6(6 - 2) lI e - e W =O ( ( logn ( loglogn ) ) a . s .'I c>1 n a n and u
1)
l ims uI=1l en - ell~co/KE
2)
k~4k o /(1 - c 1 (1+2k 0 )E2)
a.s. n+oo where c >0 is a constant independent of k and E aRd 'K= 1 ' n / n , lmsupA max Amin n +oo a . s . provided on k =limsupA hor: <00 and E<1 /(c (1+2k ) )~ o n +00 max mln 1 0 where c is a constant independent of k , k o and E . 1 Fer proof we refer to (Chen & Guo , 1987e) . This theorem means that the estimate e gi ven by ELS is robust in the sense thatnas the multiplier E in the dom i nator f or ~ tends to zero the estimation erro r also n goes to zero , if persistent excitation con dition is satisfied . When we disturb a desired control by a di.ther with constant va riance (Caines & Lafortune , 1984;Chen , 1984 ; Chen & Caines , 1985a) , then we can remove the boundedness assumption on k ' o Let the control u applied to the system (99) - (101) be n ( 102 ) where and E
U
n
O is T'- measurable desired control n is giOen by (30) .
Theo r em 16 . If Conditions 1(S=2) , 2 , 4 and 5 hold , control (102) is applied to system (99) - (101) and i f n
L
U
i=O
n- 1
where 6( x) =(6 ,x= ~ , J (u)=~ I (Y:Q1y,+u~Q2 11.)' , x~ n ni~ O l l l l
(11y, W+llu,112)~Mn , Vn >O , l
l
then the Sr.S algorithm is robust in the sense that there exists a consta nt E*>O such that
limsupJ (u)=J(u) and minJ(u)=trSCR CT • n+oo n uEU
limsu~len - elkcE ,
The proof of this theorem is similar to that for Theorem 13 , and the convergence rat e follows f r om Theorem 6 . # ROBUST IDENTIFICAT ION Since a rea l system is rare to be exactly modeled by a linear deterministic or sto chastic system it is of great importance to analy ze t he influence of the unmode l ed dy namics upon the behavior of an adaptive control ~ystem,such as stabilit y , c~ntrol perf ormance and accuracy of the parameter esti mation and so on (Sga rdt , 1980 ; P.ohrs , 19 82;Riedle & Kokotovic, 1984 ; Kreisselmeier & Anderson, 1986 ; Goodwin , Eill , Mayne & ~iddle ton 1986;Ioannou & Kokotovic , 1984 ; Ioannou & Tsaklis , 1985 ; Bitmead & Johnson Jr ., 1986 ; Chen & Guo , 1987e ) . Con sider the foll owing stochastic system with unmodeled dynamics ~n : A(z)Yn+1 = B(z)un+C(z)wn+1+~n,n~ O
(10 1 )
with a E: (O , 1) , OO. Theorem 15 . If for system (99) - ( 1 01 ) Condi tions 1(6=2) , 2 and 4 hold , then for en given by ELS
11 J (ua) - minJ(u) II=O(n - (P A d) a . s . n
,
I1 ~ Ik E L an - l (i I y, 11+11 u, 11+11 w 11+ 1 ) l l 0 n i=O
(99) (100 )
whe r e
~
VEE( O, E*) n +oo and c are constants independent of
E.
The proof consists in verifying finitene s s of k . ? o r details we refer to ~hen & Guo , 1 988 0 ) • We now design a robust stochastic controller . Conditi on 9 . m=~ , zeros of detA(z) and det B(z) lie outside the closed unit 1isk , and the up per bou:1d s for 11 e I1 , trR ,11 B:; 11 , s u !) 1I A- 1 ( e i
I! B- \eic)A(i
o ~ lO , 27Tl
are ava il able and they are den ote d by a S, Y, KAB , KAC . K3A and Kec respect iv ely . Let Iy~ } be the reference signal ,lly*II<£ ,¥ n . Defi ne u ' fro~ n 0 n B u ' =(3 u _eTe )+y* 1n n 1n n n n n+1 if detB ~ O , oth erw is e u' =O , whe r e B i s the 1 1n est i ~atenfer B given b9 e • Choose constants Mo and M1nlarge enough so
Identification and Adaptive Control for Stochastic Systems
that Mo~ (32(Ct+l
'f y2 (p+q+r)+2) ( (4Kh+1
)M~
+(4Klc+1 )6+2(\.l0+1)) M1,32(Ct+l )2y2 (p+q+r) ( ( 4K
1B+l) (lJ2+1)
+(4Klc+1 )6+2 ) where 2 2 2 M'=2lJ o 0 +8K BC 6+1+16K BA M"0 ' M"=6£2+6(Ct+l )2lJ2+66+8K 2 (lJ 2+1 )+4K2 6+~ o 0 0 AB 0 AC' a We design adaptive control u as n
Ua=Uo+E n n n where En is given by (30) .
(103)
o u~,if nE[Tk' Ok) for some k un={ O,if nE[ok,T + ) for some k k 1 and stopping times {T k} , {ok} are defined as follows j -1
1
1
0k=SUp{t>Tk:.L IluiW~M o(j -l)+llukW,VjE (Tk't]l l=Tk °k- 1
Tk +1 =inf{ t>ok: 2k. L 11 ui W",t,11 l=T k
u~ W:SM 1 t}
Theorem 17.If for system (99)-(101) Condi tions 1(6 =2) ,2,4,5 and 9 are satisfied and the control is given by (103) ,then applying ELS algorithm to the modeled part of (99) leads to 1 n 1) limsup::;- L Oly.W+llu.112)
2) limsup::;- 'i. Ily . -y'.'W ,,0(E2) +trR+IIBlI12lJ2 n+oo ni=O l l 0 3) limsu~le-e II= O(d n+oo n
a.s.
where E,R and lJ are given by (101),(33) and (30) respec£ively. For Proof we refer to (Chen & Guo,1988). # This theorem means that ELS estimation al gorithm and the adaptive control given by (103) is robust with respect to the unmodeled dynamics. For SISO system assuming the driven nois8 to be a stationary ergodic process and restricted to the reference signal satisfying some equation Guo & Ch en (1987) have succeeded in designing a stochastic adaptive tracking control motivated by a po18-assignment method so that the results similar to conclusions of Theorem 17 remain valid,but the minimum phase condition for the modeled part of the syste m and the upper bound restrictions required in Condition 9 are no longer needed and the SPR condition is assumed for 1 D(z)C- (z) -~ with D(z) not necessarily being 1.To completely remove SPR condition Guo & Moore (1987) proposed a spectral factorization approach for a class of linear models. CONCLUSION We have given convergence analysis of recursive estimation algorithm for coefficients of ARMAX model and have shown that conditions for consistency are satisfied if a dither tending to zero is added to con-
59
trol.Then we heve given a consistent estimate for orders of the feedback control system.When the upper bounds for orders are available we have designed optimal adapti ve controls which lead to consistency of parameter estimates and to minimality of control performances.Finally,we have given the robustness analysis of parameter estimation and adaptive control when the system contains the unmodeled dynamics.The emphasis of the paper is on stochastic systems,but results for deterministic systems are also demon strated . In the field of parameter identification and adaptive control there many problems belonging to further study : 1) Design robust adaptive controller for stochastic systems under conditions more general than those used in Theorem 17. 2) Considering adaptive control problems for time-varying systems is of great impor tance in both theory and application. 3) It is desirab18 to extend order estimation result to the case where upper bounds for true orders are unavailable and to weaken the SPR condition imposed on the noise model. 4) For continuous -t ime stochastic systems there some results given in (Chen & Moore, 1987:Van Schuppen,1983:Chen & Guo,1987g), but many problems are still open,for example ,the existence of the strong solution for the system of stochastic differentia l equations arising from identification and adaptive control :t he continuous - time version of attenuating excitation method and so on. REFERENC ES Akaike,H.(1969) .Fitt ing autoregressive models for prediction ,Ann. Inst .Statist. Math. ,21,243-247. Akaike,H.(1977).On entropy maximization principle,in Applications of Statist i cs Ed . P. R. Krishaiah, 27- 41 , Amsterdam, NorthHolland. Anderson,B. D. O.& C.R.Jo hnson J r., Exponenti al convergence of adaptive identification and control algorithm,Automatica, 18,1-13,(1982). Anderson,B.D. O.& J. B.M oore (1971) , Linear Optima l Control , Prentice Hall .-----Astrom,K.J.& B.Wittenmark (1973) , On selftuning regulators,Au tomat ica,9,185-195. Astrom,K.J.& B. Wittenmark (1985),The selftuning regulators revisited, Proceedings of the 1985 IFAC Symposium on Identificat io n and System Parameter Estimation . Bitmead,R.& C.R.J ohnson Jr ., Discrete averaging and robust ident i fication,in Advances in Control & Dynamic System8:Ed . C. T.Leondes,Vol.24. Box,G.E.P.& G.M.Jenkins,Time Series Analysis,Forecasting and Cont ro l ,Holde n - Day, San Francisco,(1970). Caines ,P. E.& S.Lafortune (1984) ,Adaptive control with recursive identification for stochastic linear systems, IEEE Trans .Autom.Contr ol ,A C-29,312-321. Caines ,P. E.& H.F.Chen (1985), Optimal adaptive LQG control for systems with finite state process parameters IEEE Tran s. Autom.Control,AC-30,185-189. Chen,H.F.(1984J.Recursive system identification and adaptive control by use of the modified least squares algorithm, SIAM J.Control & Optimiz, ,22,758-776. Chen,H.F.(1985J.Recursive Estimation and Control for Stochastic Systems,New ~ York,John Wiley. Chen,H.F.& P .E.Caines (1985) .Adaptive li-
60
Han-Fu Chen
near quadratic control for stochastic discrete-time systems,IMA J.Control & Information,2,319-334. Chen,H.F.& P.E.Caines(1985).Strong consistency of the stochastic gradient algorithm of adaptive control,IEEE Trans. Autom.Control,AC-30,189-192. Chen,H.F.& L.Guo(1985a).Strong consistency of para~eter estimates for discretetime stochastic systems,J.Syst.Sci.& Math.Scis.5,81-93. Chen,H.F.& L.Guo(1985b).Strong consistency of recursive identification by no use of persistent excitation condition,Acta Math.Appl.Sinica, (English Series) 2,133-145. Chen,H.F.& L.Guo(1986a).The limit of stochastic gradient algorithm for identifying systems excited not persistently,Kexue Tongbao (Science Bulletin),1986,6-9. Chen,H.F.& L.Guo(1986b).Optimal adaptive control with quadratic index,Int.J.Control,43,869-881. Chen,H.F.& L.Guo(1986c).Strong conSistency of parameter estimates in t he optimal adaptive tracking systems,Scientia Sinica (Series A),29,1145-1156. Chen,H.F.& L.Guo(1986d).Convergence rate of least squares identificat i on and adaptive control for stochastic systems, Int.J.Control,44,1459-1476. Chen,H.F.& L.Guo(1987a).Asymptotically optimal adaptive control with consistent parameter estimates,SIAM J.Control & Optim.,25,558-575. Chen,H.F.& L.Guo(1987b).Optimal adaptive control and consistent parameter estimates for ARMAX model with quadratic cost, SIAM J.Control and Optim.,25,845-867. Chen,H.F.& L.Guo(1987c).Adaptive control via consistent estimation for deterministic systems,Int.J.Control,45,21832202. Chen,H.F.& L.Guo(1987d).Consistent estimation for the order of the stochastic control systems,IEEE Trans.Autom.Control,AC 32,531-535. Chen,H.F.& L~Guo(1987e).Robustness analysis of identification and adaptive control for stochastic systems,Systems & Control Letters,19,131-140. Chen,H.F.& L.Guo(1987f),Stochastic adaptive control for a general quadratic cost, J.Syst.Sci.& Math.Scis,7,289-302. Chen,H.F.& L.Guo(1987g).Continuous-time stochastic adaptive track i ng:robustness and asymptotic properties, Technical Report,Institute of Systems Science, Academia Si nica. Chen,H.F.& L.Guo(1988).A robust stochastic adaptive controller,IEEE Trans.Autom. Control,to appear. Chen,H.~.,L.Guo & J.F.Zhang(1988).ConBistent order estimation for linear stochastic feedback control systems,Automatica, to appear. Chen,H.F.& J.B.Moore(1987).Convergence rate of continuous-time ELS parameter estimation,IEE E Trans.Autom.Control,AC-32, 267-269. Clarke,D.W.(1984).Self-tuning control of non-minimum phase systems,Automatica, 20,501-517. Egardt,B.(1980).Stability analysis of adaptive control systems with disturbances, Proc.JACC,San Francisco,CA. Goodwin,G.C. ,D.J.Hill,D.Q.Mayne and R.H.Middleton(1986).Adaptive robust control (Convergence,Stability and Performance) Proc.25th IEEE Conf.on Decision & Control,Athens,468-473. Goodwin,G.C.,P.J.Ramadge & P.E.Caines(1981) Discrete-time stochastic adaptive con-
trol,SlAM .T.Cor.ltrol & Optim.19,829-853. Goodwin,G.C.,P.J.Ramadge & P.E.Caines,Discrete-time multivariable adaptive control,IEEE Trans.Autom.Control,(1980), AC-25,584. Guo,L.(1987).Identification and Adaptive Control of Dynamic Systems,Dissertation. Guo,L.& H.F.Chen(1987).Robust stochastic adaptive control for non-minimum phase systems,Systems Science & Math.Sciences,to appear. Guo,L~ J.B.Mcore(1987).Adaptive spectral factorization with applications to system identification,Technical Report, Dept.Syst.Eng.,Australian National UniverSity. Hannan,E.J.& B.G.Quinn(1979).The determination of the order of an autoregression, J.Royal Stat. Society,B.41,190-195. Hijab,O.B.(1983).The adaptive LQG problem -Part 1 ,IEEE Trans.Autom.Control,CA-28 171-178. Ioannou,P.A.& P.V.Kokotovic(1984).Robust design of adaptive control,IEEE Trans. Autom.Control,AC-29,2-2-211. Ioannou,P.A.& K.Tsaklis(1985).A robust direct adapt.ive controller,Report 85-071 ,Electrical Engineering Systems,University of Southern California. Johnston,R.M.& B.D.O.Anderson(1983).Global adaptive pole placement:Detailed analysis of a first order system,IEEE Trans.Autom.Control,AC-28,853-855. Kreisselmeier,G.& B.D.O.Anderson(1986).Robust model reference adapt.ive control, IEEE Trans.Autom.Control,AC-31,127-133. Kumar,P.R.(1983).Optimal adaptive control of linear-quadratic-gaussian systems SIAM J.Control & Optim.,21,163-178. Lai,T.L.& C.Z.Wei(1986).Extended least squares and their application to adaptive control and prediction in linear sys~ tems,IEEE Trans.Autom.Control,AC-31, 898-904. Landau,I.D.(1979).Adaptive Control:The Model Reference Approach,Dekker,New York. Ljung,L.(1985).On estimation of transfer functions,Automatica,21 ,677-696. Ljung,L.& T.Soderstrom(1983).Theory and Practice of Recursive Estimation,MIT Press. Meyn,S.P.& P.E.Caines(1985).The zero divisor of multivariable stochastic adaptive control systems,Systems & Control Letters,6,235-238. Moore,J.B.(1978).On strong consistency of least squares identification algorithm, Automatica,14,505-509. Riedle,B.D.& P.V.Kokotivic(1984).Disturbance instabilities in an adaptive system, IEEE Trans.Autom.Control,AC-29,822-824. Rissanen,J.(1978).Modeling by shortest data description,Automatica,14,467-471. Rohrs,C.E •• L.Valavani;M.Athans & C.Stein (1982).Robustness of adaptive control algorithms in the presence of unmodeled dynamics,Preprints of 21st IEEE CDC,Orlando,FL. Samson:G.(1983).Stability analysis of adapt.ively controlled systems subject to bounded disturbances,Automatica,19,8186. Schwarz,G.(1978).Estimating the dimension of a model,Annals of Statistics,6,461464. Van Schuppen,J.H.(1983).Convergence results for continuous time adaptive stochastic filtering algorithm,J.Math.Anal. Appl.96,209-225. Soderstrom,T.& P.Stoica(1985).Instrumental methods for identification of dynamic system, Proceedings of 1985 IFAC Sym-
Identification and Adaptive Control for Stochastic Systems posium on Identification and System Parameter Estimation. Solo , V.(1979) .The convergence of AML ,I EEE
Trans.Autom.Control,AC-24,958-96~
Zarrop,M.B.(1979J.Optimal Experiment Desi gn for Dynamic System Identification, Springer,Berlin.
61