- Email: [email protected]
On the Asymtotic Behaviour of an Adaptive Pole-Placement Algorithm
© IF.-\C .-\dapti'e S,stellls ill COlltrol and Signal Processing. l.und. Sweden. 19K1i
Cop~Tight
ON THE ASYMPTOTIC BEHAVIOUR OF AN ADAPTIVE POLE-PLACEMENT ALGORITHM S. S. Stankovic and M. S. Radenkovic Fawl!.\· vf Elretriml ElIgillrnillg. L'lIit'nsit,l' vf Bl'lgradl'. Bdgmdl'. l'lIgu.\lm'ia
Abstract. In this paper convergence of an adaptive pole-placement algorithm based on the minimization of a predefined criterion is analysed. It is supposed that an external random excitation with possibly decreasing variance is injected into the system. The main result is the proof of the convergence w.p. 1 of the re gulator parameter estimates to the optimal ones, i . e . convergence w.p.1 of the closed loop poles to the desired ones, derived in two methodologically different ways. Convergence is ensured under mild conditions concerning the number of estimated parameters and requiring irreducibility of the optimal regulator transfer function. It is demonstrated that the algorithm converges even in the case of an external random excitation whose variance tends to zero,
despite the fact that the persistence of excitation is not satisfied. Cons isten cy proof for the parameter estimates in minimum variance self-tuning control algorithms can be derived as a special case . Keywords. Adaptive systems; pole-placement, stochastic approximation; global stability; convergence of the parameter estimates. INTRODUCTION
probability one can be guaranteed under mild conditions related to the number of esti mated parameters and the irreducibility of the optima l regulator transfer function. It will be suppose d that an external excitation signal is fed to the system, but we shall show that the strong consis tency of the parameter estimates can be ensured even in the case when this excitation tends to zero in the
Self-tuning control algorithms and their applications have been an objec t of study within control engineering for many yea rs, e.g . I 1, 2 I . Asymptotic properties of these algorithms have been analysed usin g different methodologies, e.g. 13,4,5,6,7 I. Goodwin, Ramadge and Caines 141 made an important step by proving asymptotic optimality and mean-square boundedness of the input-output variables for the self-tuning algorithms of minimum variance type. However, the convergence of the regulator parameter estimates themselves has still remained only partially understood I 5, 6, 7 I.
mean-square sense.
PROBLEM FORMULATION Let a time-invariant single-input single-output discre te-time sys tern be represented by an ARMAX model
The class of the self-tuning control algorithms based on pole-placement have attracted the attention of many researchers,e.g. 12,8 , 9,10,11,121. However, in spite of very valuable achievements, many important aspects of their convergence are still incompletely clarified. Most of the approaches to this problem are devoted to deterministic systems, with the focus on the role of the persistent excitation . 19, 10 I. In the stochastic case, the lack of a general methOdology for the parameter convergence analysis prevents drawing precise conclusions related t o the asymptotic pole positioning.
A(q -1) y(i)=q-
=b +b 1q
-1
o
+ ••• +c
nc
+ ••• +b n q -nc B
-n
B (b ;0) and C(q 0
-1 A
-1
)=1+c q' + 1
q
We shall adopt the following assumptions about the process characteristics: (AI) C(z) and B(z) are stable polynomials; (A2) C(z)-(~/2) is a strictly positive real function for some a>O.
In this paper we shall analyse a pole-placement self-tuning control algorithm based on the indirect approach, i.e. on the definition of a criterion function whose minimisation leads to the desired pole positioning. Similar ideas can be found in 11O,11,12!, but, however, analysis of the asymptotic properties of the described aliorithms are completely lacking in the literature 131. We shall prove first the global stability of the given algorithm including stochastic boundedness of the regulator parameter estimates, and then their almost sure convergence to the optimal regulator parameters in two different ways. In the first we shall analyse asymptotic properties of the closed-loop transfer function, and in the second conditions related to the persistent excitation. It is important to stress that the convergence of the c losed-loop poles to the desired positions with ASC-p
(1)
where (y(i)}, (u(i)} and (w(i)} are output , input and disturbance sequences , respectively, q-l stands for the unit delay operato r, d for the pure process time-delay, while A(q-l)=I+al q- 1+ ••• +a n q-nA,B(q-1)=
All random variables are defined on the underlying probability space {n,F,p} . We shall take xo={y(O), ••• ,y(l-k),u(l-d), ••• ,u(1-k),w(O), ••• ,w(l-k)}, where k=max{nA,nB+d,nC}' to be a random variable, and (w(i) ;i.::l} to be a stochastic process on {n,F,P}. We shall assume that a reference sequence (y*(i)} is given, satisfying: (A3) (y*(i)} is a bounded deterministic sequence (Iy*(i) I~m*), defined for i.::1 (y*(i)=O for i<1). (A4) For allNpairs of integers k.£ lim
1
I
y*(i -k) y*(i -l)
N-- N i=1
exists, and depends upon k-£. 149
150
S, S, StallkO\'ic alld \1. S, RadellkO\'ic
Following the methodology of Caines and co workers 15, 7 I we shall assume that an exogenous signal £(i) is added to the reference y*(i) (i>I). Its role is to ensure a sufficiently rich excitation of the process. Let {£(i)} be a stochastic process on {n,F,p}. Introduce the following assumptions: (AS) All finite dimensional distributions of x '
o (w( i)} and {£(i)} are absolutely continuous with respect to the Lebesgue measure; (A6) If Fo is the a -algebra generated by x o , and Fi that generated for i~1 by (Xo,w(I), ••• ,w(i), £(1), ••• , £ (i)}, then w.p.1 y(i)=u(i)=w(i)=£(i)=O for i
r~~~)lrW(i): £ (i)J IFi_I}=[a2 £(i)
0
0
~2/s(i)
controller (the cha racteristic equation is B(q-I) C(q-I)A*(q-I)=O, but B(q-I) and C(q-I) are ca ncelled in (9». The optimal regulator (B) depends explicitely on the process parameters . We shall suppose that these parameters are not known to the designer,i.e. that u(i) can only implicitely depend on them through observations. We shall construct an adaptive control algorithm based on the following assumptions: (A7) time-delay d is known; (AB) upper limits of nA' nB and nC are known. This algorithm, derived using the methodology of Goodwin and co-workers 12 ,41 and Caines and Lafortune 1 7 ; is defin~d by :
J
8 (i)= 8( i-d)+ -~- ~( i-d) [A*(q -I)y(i)_y*(i)_ r(1-d)
E{W(i)4I F _l; I,::k <00 w i E{s (i) 2£ (i) lE I }
-
r(i-d)=r(i-d-I)+. (i-d)T~(i-d); r(O)=1
Our control objective is to position the closed loop system poles at the zeros of a predefined stable polynomial A*(q-I) of degree n*. We shall attempt to achieve this goal , coping at the same time with disturbances, by minimizing the following J=lim
1
N
I'
N-N i=1
.
E{(A*(q-l)y(i+d)-y*(i+d»2 IF , }
(2)
1
If polynomials F(q-I) and ~(q-I) are_The minimum degree solutions with respect to F(q ) de ); F(q-I) < ~ nF=d-l, deg G(q-I) $nG~ax(nA-I,nCTn*-d» of the following diophantine equation C(q-I)A*(q -1)=A(q-I)F(q -I)+q -clC(q -I) (3) C(q-I) (A*(q-I)y(i+d)-y*(i+d)-£(i)-F(q-I)w(i+d»= -I -I -I -I =B(q )F(q )u(i)+G(q )y(i)-(C(q )-1) (y*(i+d)+£(i»-y*(i+d)-£(i)
GLOBAL STAB ILITY In this paragraph we shall analyse global stability and asymptotic optimality of the algorithm (10). (1 1) usin g the methodology presented in 141. We shall pay a special attention to the properties of the regulator parameter estimates, since they represent a prereq uisite for the consistency analysis. We shall also formulate a techni cal lemma crucial for the subsequent derivations.
THEOREM 1. Let assumptions (AI)-(AB) hold. Then for algorithm (l0), (11), with probability one:
'" i +id) 11 = 8 ~ <00 a) lim 118( o
o
10
8T~ (i)=y*(i+d)+£(i) 22
Jmin=a + ~ lim
N-
IN
P { s~p ll~ (i0+id) 112~£l'::
(6) I
N i~1 s(i)
L
c)
i=1
2 Ile(i 0 +id)- e (io+(i-1)d) 11 <00; (io=O, •.•• d-I)
z(i) 2 e) i t r(i)-
f)
1
Urn
N- N 1=
-I
)w(i»
2
I
E{ (A*(q-I)y(i+d)-y*(i+d)) IF, I 2
1
2
fo r s(i)=1 for s(i)=k 1ni s -T [
wher: 18 (i) ~~ (i - 8 o , 8 0 = go ••••• gnG'O ••••• 0, coe ff (B( z )F(z )} .0 •.••• O.cl' •••• c n ,0 •..•• O).the number of inserted zeros dependi~g on the chosen dimensions n I' n 2 and n3 in (10). (11). '"
2
00
( a I+ ~ <, 2 l al
(7)
s (i) =ks 1 ni
<
N
I
, I
s (i) = I
=0, where a l=E {( F(q
c£1 ; (OO)
00
(5)
0
where z(i)=A*(q-l)y(i+d)-y*(i+d)-£(i)-F(q-l)w(i+d), 8 =coeff{C(q-l) ,B(q-I)F(q-l) ,C(q-I)-I } and j0(i)T= =ly(i), ••• ,u(i), ••• ,-(y*(i+d-I)-di-1), •.•• The optimal control law is defined by 00
b)
(4)
C(q-l)z(i)= 8T~ (i)-(y*(i+d)+£(i»
(11)
where, in general, ~ (i) T= [y(i) , ••. , y(i -n ) ,u(i), ..• , l u (i~n2) , y*( i+d-I) +£ (i-I) , ••• ,y~(i+d-n )+£ (i-n 1 ») , nl~nG' n2~nB+nF' n3~nC' while a satisties (A2). The estimation algorithm (10) consists of d-interlaced stochast i c pr oce dure s with a.s. finite initial co nditions 8 (0)= 8 0 , 8 (1)=8 1 , .•• , 8 (d-I)=6 d_I'
i-+
one can wri te
or
(10)
e(i) T. (i) =y *( i+d) H(i)
E:
In (A6) (s(i) } is a deterministic sequence. We shall consider two cases: a) s(i)=I, which cor resp onds to continually disturbed controls discussed in I 7 I ; and b) s(i)=k 1ni (O
cri terion:
(a >O, i~d)
- £ (i-d)]
A
f
Proof. Statements a). c), d) and e) can be proven The optimal three-term controller following f rom (6) is given by
After straightforward manipulations. One can obtain from (10). for each of the re c ursions
B(q-I)F(q-l)u(i)=-G(q-I)Y(i)+c(q-I)(y*(i+d)+ +£ (i» while the output tern satisfies
using directly the results presented in 12.4. 7J. We shall. therefore. focus our attention to b) and f).
(B)
E{IIS( io+id) 112 IFi_d}.:: of
the
closed-loop
A*(q-l)y(i)=y*(i)+£(i-d)+F(q-l)w(i)
IIS( io+(i-l)d) 112+
sys+
Cl( io+(i-I)d) -I
(9)
Obviously, the system dynamics is defined by A*(q-I) so that (7) can be considered as a pole-placement
2
+ where Cl ( J')= ~2.W.L r(j) From
(3)
(12) -2 2 2 + a IIHj) 11 al r(j) 2
:\dapti\'e Pole-Placement Algorithm (13) Put for brevity io=O. Define the sequence (W(id)} i
~
2 W(id)= 116(id) 11 +E{) a(U -l)d) IF id }- ) c>(U -l)d) J=1 J=1 (14) Obviously W(id)~O, i~I, and (15)
E{W(id) IF( i_l)d }.::W(id) i.e. (W(id) ,F(i-l)d } bye)
is a supermartinga le. Since
) aCid) < ~ (16) i=1 '\, 2 and E{ 116 11 }<~)it follows th at E{W(O) }<~ a.s.and 0 E{W(id) }.::E{W«i-l)d) }.:: ... .::E{W(O)} < ~ (17)
• -1 ' . ( .) -n2 R(6(i) ,q )= 6 n1 + 1 (1)+ •• • +6 nl+n~1 1 q and •
L(6(i) ,q
-1
~ El .:: ~ E{W(i1d) } ; d~
W(id)~ll e( id)11
as a consequence of the fact that
Statement f) is proven in 110 I for A*(q -l ) =1 a nd s(i)=I. When s(i)=k £ni one can start from th e fundamental rel a ti o~ lim
~ ~
+£ (i) ) +C(q -I ) R( S( i) ,q -1)w(i+d)
1 ~
E{(A*(q-l) y (i+d)- y *(i+d»2 IF i ) N-- N i=1 2 . 1 ~ £ 2(i) 2 k £ni = 0 1 (20) = ° l+l1m N I. N-i=1 s after applying the lemma in Neveu 1151, pp.148-ls0. ( see also 12, 71).
Hiz' (i)=(A*BLi_ £-H _ £) (y*(i+d)+£ (i»+ i + (GRi_ £-BFSi_ £)w(i) =
Lemma 1. Let assumptions (Al)-(A8) hold. Then with probability one ( 8 (i o+(i-Od)T q,(i o+id» 2 <~ ;(io=O,_.,d-l) (21) i";o r(io+id) where £ is any finite integer.
r
z ' (i)=O
(i~l)
Ci.::O)
where £ is a finite integer, and ~ _. nQ _j P.= p.(i)q J a nd Q. = I qJ.( i)q . 1 j =0 1 j=o J
L
Lemma 2. Let assumptions
I
(Al~-(A8)
hold. Then w. p. 1
(Hi n+( i_ £)dz'(io+id»
( 28)
where £ i s any finite integer, while the argument w denotes that c 4 (w) is a random cons tant, possibly depen den t upon r ea li za ti on. Pr'oof. Pu t i =0. It follows from the statement d) in Theorem 1. thgt there exists a finite integer i 1 such that f or i >i l with probability one kr( w )r(i)~i (kr( w)<~ ). Rewrite (2R) in the following way
tl
- 8« (i- £)d» T;(id»2
(H(i_ £)dz'(id»2.::(np+nQ+2)
(29)
[.f' Pj«i_ £)d)2 J=O
(y *(i_d_j)2+£ (i_j)2)+ IQ q .« i_ £) d)2 w(i_j)2] j=o J (30) or, having in mind b) (H(i_ £)dz'(id»2.::
(23)
(a.s)
(H(i_~)dZ'(id»
n
(22)
Ile(id)-e(i-£)d)112<~
I
i=i +1 1
The second term in (29) is, thus, finite, having in mind Lemma 1 and a.s. boundedness of the parameter estimates (statement b), Theorem 1). We shall demonstrate boundedness of the first term starting from (26), which leads t o
(6«i_£)d)Tq,(id»2 <2( 6 (id)Tq,(id»2+2«8(id)one obtains (21) af ter using statements c) and e) of Theorem I, i.e. the fac t that 2 < 2 L (6 (id)T q, (id»2 + I (6(U'£)d)T
(H(i_ £)f'(id»2 +
i=1
Pr'oo f: Assume th a t i o=O. Starting from the obvi ous inequality
+
~Q
k~(w )
[I
P
(m*2+£(i_j»2+
j=o
w(i_j)2]
(31)
j=o CONS ISTENCY OF THE REGCLATOR PARAMETER ESTIMATES
We shall focus the analysis presented in this paragraph on the time-varying dynamic models of both the self-tuning three-term regulator and the whole closed-loop system. The proof of the consistency of the parameter estimates with be derived from the asymptotic behaviour of these models. Using (4), (8) and (10) the time-varying adaptive regulator model can be expressed as R( a( i) ,q-l)u(i)=-S( 8(i) ,q -1)y(i)+L( 6( i) ,q -1) (y*(i+d)+£(i» .
(27)
=Pi_ £(y *(i+d)+£(i»+Qi_ £w( i)
The following lemma is fundamental for all the sub sequent derivations concernin g the conve r gence of the regulator parameter estimates.
+ 2 I i=1
(25)
i=1
which can be derived fr om d) and e), and obtain lim
•
H. z(i)=(A*BL.-H.)( y*(i+d)+£( i»+( GR.-BF5.)w(i) 1 1 1 1 1 (26) Define the sequence (z '(i) } in the following way
(19)
(a.s) .
z(i)2 = 0
) -n3
e
i=1
N--
'
whe re H( El (i) ,q - 1) =A( q - 1) R( (i) ,q - 1) +q - dB (q - 1) S(6(i),q-l) is the cha rac teristic polynomial. Denoti~g sho:-tly H(6(i) ,q~I)=Hi' S(8(i)8- 1)=5 i , R( 6(i) ,q 1)=Ri and L( 6( i) ,q-l)=L i , it follows from (4) th a t
(18)
.
l
H( 6 (i) ,q -1) y( i+d )=B(q -1)L( 6(i) ,q -1 ) (y*(i+d)+
i~i 1
which implies
·'
)=1+6 n1 + n2 + 2( i)q- + ••• +6nl+n2+n 3+1 (i q
The closed-loop syste m is described then by
This means that (W(id) , F li-lld } is a co nvergen t supermartingale. The app11cat10n of the Kolmogo r ov inequality for semimartingales 114 I gives p{sup W(id)
151
where S(6(i) ,q
-1
'
(24) .
)= 60 (i)+6 1(i)q
-1
+ ••. +6
nl
(i)q
-n 1
where
k~ (w)
is a finite random constant.
Analyse now the sequence i wU)2 (). .1+6 } , where 6>0 . It is easy to see j=1 J that {t , F _ } is a convergent supermartingale, hai i 1 ving in mind that E{t. } is finite by virtue of (A6). Therefore, lim t.=t( w Y <~, and, consequently t < ii~ 1 2 6 1 .:: t(w)<~ for all i , implying w (i)
Thus
£
£
S. S. Stankovic and M. S. Radenko\'ic
152
( 32) implying boundedness of the first term at the right-hand side of (29). The lemma is, thus, proved. The main result of this section is formulated in the following theorem.
I ~ q.«i-£)d)qk«i-£)d)w(id-j)w(id-k)
Theorem 2. Let assumptions (A1)-(A8) hold, together with
n
(A9) At least one of the following equalities holds: n =fic=max (nA-1,n C+n*-d), n2=nB+nF=nB+d-1 and 1 n3=nC (A10)The optimal regulator transfer function G~(z)= =[C(z):G(z)F(B(z)F(z»-l is irreducible 1161.
E(id_j)2+ IQ qk«i-Od)2w(id-k) 2]<00 (a.sX40) k=o The first sum in (40) is finite as a consequence of (A2) and b). Consider the sequence
f
lim 8(i) =
e
i....-oo
(33) 0
j °,
T_ [ whe re eo-go"' " gn ' 0, •.. c oe f f { B( z-1 ) F ( z-l ) }T , 0, ..• ,0, cl' ... ,C ne , ,0 .
8, ...
I.~
lip }=V n
and, since
{V }
n
is L2_
rr-
I
i=l
f' P . «i_£)d)2 1J 2 < oo(a.s)
-,--J.-('d) 1S
j=o
1
)
i
IQ qk«i-£)d)2 0 2 < 00 k=o
i= 1 p . «i-Od)Pk«i-Od)y*«i+1)d-j).
j ,k J
Since
E(id-k)+2 I p.«i-£)d)Pk«i-£)d) E( id-j)E(id-k) j,k J ifj
(a.s)
(43)
~
-,--J.-(. d) for both considered choices i=l 1S 1 of s(id) (see (A6», it follows from (4~) gnd o (43) that there exists such a subsequence {t ;t .::t .:: ••• i 1 2
-
r p.«i-£)d)qk«i- £)d)(y*«i+1)d-j)+ j ,k
+ 2
1
lim p. «t~-£)d)=O; lim qk«t~-£)d)=O
J
i-+m
+ E(id-j»w(id-k)+2 I q . «i-£)d)qk«i-£)d) i,j J ifj
(34)
~
J
i-+
(44)
1
(j=O, .• • ,np;k=O, ••• ,n ) (a.s) Q Since by Theorem 1 the sequence{6(t?d)} is a boun1 ded sequence, it follows that these exists a subsequence {s?:sol
--
-11
(45)
Consic.ic r the sum
S
n
~ ~ 4p. «i-Od)p 1 '_1
(42)
J
Analogously, one obtains from (40)
n
,,(id-j)w(id-k)
n
-bounded one obtains lim Vn=Voo
P . «i_£)d)2 E(id_j)2+ IQ q.«i-Od)2 j =0 J j =0 J
I
n+
I'
(H(i_£)dz'(id»2 = (P(i_£)d Y*«i+1)d»2+
w(i-d-j) 2+2
p . «i-£)d)2(E(id-j)2_// s (id-j»} (41)
1
Obviously, E{V
Proof. Put again io=O. It follows from (27) that
+
~
{V }={ n i=l
Then with probability one
J
1-
k
«i-£)d)y*«i+1)d-:j)E(id- k) (35)
Therefore, according to (44) and (2/), (46)
Obviously
S~+l=S~+ n~l
Sn P j«n+1-£)d)Pk(n+1-£)d)
y*«n+2)d-j)E«n+1)d-k)+ _1_-2 p.«n+l-£)d)2 (n+1) J Pk «n+1-£) d) 2y *( (n+2) d-j) 2E «n+1) d-k) 2
(36)
Taking into account (A6) one obtains 2 2 1J 2 E{Sn+11 Pnd-k},::Sn+c 5 (w) ---2;:-...!:...--(n+1) s( (n+1) d-k)
(37)
(a.s) Moreover li m S2=S <00 n 00
I i=l
implying
fPj«i-£)d)Pk«i-£)d)y*«i+1)d- j )E(id-k)<00 (j ,k=O, ••• ,~)
(38)
Having in mind that Do can divid~ n~ither C (by (Ala» nor Bo ' it follows that Do =D o D0 2'
Oneoocan prove in a similar way that a.s. I ~ p . «i-£)d)Pk«i-£)d)E(id-j)E(id-k)
'_1 1
J
I i=1
Consequently, D~=Do1 and L =k'C, where k' is, in general, an arbitrary cons~ant. However, the lead~ ing coefficients of both C and L are equal to 1 by assumption (see (24\), so that k'=l, and, therefore, So=G, Ro=BF and Lo=C,
(j ,k=O, ••• ,~,jfk)
1-
i Pj«i-£)d)qk«i- £)d)(y*«i+1)d- j )+ +E(id-k»w(id-k)< 00 (j=0, ••.
,~;k=O,
(39)
••• ,n ) Q
Suppose that n =n , n2=n B+n F and n3=nC' Let D be 1 G the greatest common divisor of the polynomial~ G and BF (we shall ommit the arguments for simplicity) i.e. G=D Go and BF=(Do1Bo) (D02 Fo), Do1D02=Do' It follows ~hat (47) is satisfied for So=DbGo and Ra= =D~BoFo, where D~ is any polynomial satisfying deg D~,::deg Do' After replacing in (46) and using (3) one obtains D'B A*BL o =H 0 =AB 0 F0 D'+q-dBG D' = ~ (48) 0 0 0 D02 A*C
It is easy to check that the same conclusions hold in the more general case when at least one of the equalities: n =n , n =n +n F and n3=nC holds. 1 G 2 B Therefore, lilt A(s~d)=eo' i.e. limIIS(s?d) 11 =0. i-+-OD
1
i-+-oo
1
153
Adaptive Pole-Placement Algorithm
proving the a.s. convergence of (10), (11) is given.
" id) II} converges, i t Since by Theorerr. l", { Ile( follows that lim ; iO( id) 11 =0. In the same way one
Theor em 3 . Let assumptions (AI)-(AIO) hold. Then
i-
"" T R =lim 1 N ~ s(i)~(i)4>(i) ~ csI s N-- N i=1
concludes that for io=O, ... ,d-1 lim lle (io+id) 11 =0. i-
The theorem is, thus, proved.
(O
PERSISTENCE OF EXCITATION
and, moreover, 8(i) converges to eo with probability one.
It has been sho"n that the persistence of excitation condition plays one of the fundamental roles in ensuring convergence of stochastic recursive estimation algorithms 15 ,7,~. In this section we shall prove the almost sure convergence of the regulator parameter estimates using a methodology closely related t o the analysis of the persistence of excitation of the re gressor vector ~( i) in (10), (11). The proofs will be presented concisely because of the lack of space. We shall start the analysis by expressing the following way, using (4) and (I) y(i)
Hi)=
A*
z(i-d)
A = A*B z (i)
u(i) y*(i+d-I)+'(i-I)
(53)
~(i)
We shall analyse matrix RI obtained from Rs replacing ~(i) by ~defined as ~(i)=A*(q-I)B(q-l) 4>(i)=A*(q-I)B(q-I)~*(i)+A*(q-I)B(q-I)~ (i)+A*(q-1) B(q-I) 4>,(i)=~*(i)+~ (i)+$,(i), sincewRI>O implies Rs>O,having in mind t~at for any nonzero vector A
Froof;
R,Y
N
N
i= I
i= I
~ s(i) (AT'$(i)/ .:: c~ ~ s(i) (AT$(i» 2 (O <~)
in
l
f1A* y*(i) + ~ y*(Hd) A*B
(54)
~~eR:r~~: ~~~~~c~~ ~:~:~e~o~;~~~~ :~(!~ ::~m:l~i~he sequence (xi)={s(i)y*(i-j)w(i-k)} (j,k <~ ). The application of the lemma in Neve u 1 2,15 1 leeds to the conclusion that N
lim 1 2. Xi =0, N--N i=1
<
since
~
y*(Hd-l)
0
a consequence, RI is positive definite irrespective of {y*( i)J if the condition
As
N
~*
1
w(i)
A*
lim 1 Y (AT~ (i»2 =0; lim N--N i";l w N--
,(i-d)
A "., 1') = ~z(i)+~*(i)+ + A*B
A w(i + - A*B
o
(49)
N
T
2
N I, s(i)(A $,(i»)=O i=1
(55) implies A=O. By virtue of (A6) random sequences (w(i)} and (IS(i),(i)} are ergodic, and, therefore, the He rglotz' s theorem can be applied to (55)., After straightforward calculations, similar to those in 1 7 I, (55) results into the following pOlynomial equations
, (i-I)
(56) (57)
Lemma 3. Let assumptions (AI)-(A8) hold. Then the parameter estimates generated by the adaptive control algorithm (10), (11) converge to of these exists a sequence {Vi} of finite ingegers satisfying vi ~ Amin{ ~ ~(i +(i-j»d)~(i +(i-j)d)T J j=o 0 0
e
~
i=l
r(io+id) (i
0
(50)
=0, ... ,d- 1)
PrOOf; Put io=1. Lemma I .implies, having in mind (49), b) and e) that
v · " , . .,
.L1(e(id)~«i-j)d» \' J =0 i~1 r(id)
2 < ~
(5 I)
for any finite j and any sequence {v,} of finite integers. Obviously, (51) implies 1 Vi " " A . {~ ~« i-j)d)~«i-j)d) T } ffi1n j=o 2 Y Ile(id)11 r(id) i=l (52) If it is possible to find a sequence {Vi } ensuring (50), Ile(id) 11 converges to zero w.p.1 as a consequence of a). In the following theorem an alternative way of
where polynomials AI(z) , A2 (Z) and A3(z) of degrees n1,n2 and n3-1, respectively, are defined by the corresponding components of vector A in (55). Suppose first that n1=nG' n 2=n B+o F and n3=nC' as well as that Do is tne greatest common divisor of G and BF, i.e.: G=DoG o and BF=(D o1 F?) (DOZFQ ), DQ= =D oI D0 2. Then AI=GoD~ and A2=BoFoDo sat1sty (50), where D~ is any polynomial of degree not greater than deg D . Suppose also that the greatest common divisor ofoC and BF is DI' i.e. C1=D 1C1 and BF= =(D1lB1)(DI2Fl)' D1=D11D12. Since Do and D1 are copnme by (A10). one concludes that Fo=F*Dll' F 1= =F*D o1 ' Bo =B*D 12 and B1=B*D 0 2. ConsequentlY,one obtains (58) A 3zB*D02F*D01=B*D12F*D11D~C1 From (58) D =Db and A3 zC, since Do cannot divide D. The last e q8a1ity contradicts, however, the assumption that C is a monic polynomial. Therefore, AI(z)= =0, A (Z)=0 and Ajz)=Ois the only solution of (56), 2 (57), i.e. A=O is (54). Therefore (53) holds, i.e. for A10 (59) (0<'1<~ )
(a.s)
implying that for every i large enough there exists such a finite integer Vi satisfying Vi T" 2 2 ) s«i-j)d)(A 4>«i-j)d» ~£21IAII j=o (a.s) Therefore,
(60)
IS4
S, S, StankO\ic and 1\1. S, RadenkO\'ic )I' '\,
'\,
T
The correspondin g conver gence proofs can be obtaine d as s pec i a l cases by putting simply A*(q-1)=1.
A , ( 1. 1 (~« i- j:P)( i-j)d) }
ml.n . __
I
, .-l~
_ __ _ _ _ ,. _ _ _
~
r(id)
i=l
(a . s)
(61) REFERENCES
The applica ti on of Lemma 3 gi ves the des ired result. The extension to values of n1,n2 and n3 satisfying (A9) i s st rai ghtforward .
III Rs trom, K.J . and B.Wittenmark (1973). "On self-tuni ng re gulat ors ", Automatica, 9, 185-199.
In the following th eorem we shal l test t he persistence of excitation of the vecto r (i) in (10) ,(11).
121 Goodwin,G.C. and K.S . Si n (1984). "Adaptive filtering , prediction a nd control" ,Prentice-Hall.
Theorem 4 . Let assumptions (A1)-(A10) hold. The n
13 1 Ljun g L. (1977) . " Analysis of recursive stochasti c algo rithms ",IEEE Trans.on Aut.Control, vol. AC- 22, No.4 .
fo r s(i)=l the persistence of exc it a ti on is sa tisfie d for vector ~ ( i) in (10), (11), L e . ~
R=lim
N )' i
N-+
~(iH(i) T::CRI
(O
(62)
=1
whi te for s(i)=ks £ni det R=O, i.e ,t he pe r sistence of exc itation is not sa ti sfied .
Proof: The proof ca n be derived following the lines of the proof of Theorem 3 . Denote onl y that the concl usion det R=O in the case whe n s(i)= =ks£ni fo ll ows from the fac t that N
N
lim~ ,2: (AT~£ (i»2.?lim~ ~ I I AI 12 11 ~£ (i) 11 2=0 N-+
1=1
N-+
fo r a ll finite A, s i nce lim N--=
i=l 2 k ~N s
(63)
N
I
£rrl =0.
i=l
The application of the Herglotz's th eorem leads to onl y one equation (59) , which has at least one nonze r o so l ution . This implies singularity o f R. CONCLUSION In thi s paper convergence of a self- tun ing con trol a l go rithm based on pole-placement has been analysed in the case of a s t ochas ti c discrete-time process. The algorithm is const ruc ted s t a rting from a mean- squa r e c rite ri on whose op ti miza ti on l eads t o the desired pole positions,and conta ins a stochastic appr oximation go rithm gene r a tin g the regulator par ame ter es tima t es . It is supposed that an externa l stochastic excitation is fed to the sys te~ its variance being either constant o r decreasing with time. In the first pa rt of the paper both t he global stability and asymp t o ti c op timalit y are proved. Properties of th e parameter estimates important for the main der i va tions are explored with more de tails. A key technica l lemma represents a basis for the subsequent consistency proofs. The ma in contribution of the pape r consists of the proof of the almost sure conve r ge nce of the re gulator parameter es timate s t o th e op ti ma l ones, i.e. of the c losed loop poles t o the desired ones, derived in two di ffere nt ways. In the second part of the paper conve r ge nce of the parameter esti ma tes is proved by analysing dire c tl y asymptotic propertie s of the closed-loop transfer function, for both t ypes of the external excitation. The conditions ensuring strong consisten cy are related to the dimension of the parameter vec tor and require irr e ducibility of the multivariable transfer function of the optimal re gulator . In the third part the same re s ults are obtaine d in a methodologically different way. The a nalysis is f ocus ed on the properties closely related t o the persistence of exc itation. I~ addition, the persistence of excitation of the re g ressor vec tor in t he algorithm are directly checked. It is proved that in the case of a decreasing excitation the strong consistency is ob tained despite t he fa ct that the persi s tence of e xc itation condition is not sa tisfied. This is an important result, since the asymptotic variance of the output is not affected by the excitation si gnal. It is t o be stressed that the given results provide the solution to the open problems of the convergence of the parameter estimates in minimum-variance self-tuning re gulators (see constatations in I2 ,13 1) .
141 Goo dwin, G. C. ,P.J.Ramadge a nd P.E.Caines (1981). "Discre te - time stochastic adaptive control",
SI AM J.Control a nd Optimization,Vol.19,No.6. 151 Chen,H.F.(1984) . "Recu rsive system identification and adap ti ve con trol by use of the modified least sq uare s algorithm" ,SIAM J .Cont rol and Optimiza ti on, Vol . 22 , No.5 . 161 Becker A.H . , P.R.Kumar and C. Z.Wei (1985) ."Adaptive Con trol with th e Stochastic Approximation Al go rith m: Ge omet r y a nd Conver ge nce" ,IEEE Trans. on Aut .Con trol , Vol . AC- 30 , No .4. 171 Caines,P.E. and S.Lafortune (1984). "Adaptive Control wit h Rec ursive Identi f icati on f o r Stochastic Linear Systems ", Vol. AC-29,No.4. 181 Rstrorr.,K.J. a nd B.Wittenmark (1980) ."Self-tuning con trollers based on pole-zero placement", lEE Proc ., 127, 120- 130. 19 1 Anderson, B. D.O. an d R.M.Johnston (1985) ."Globa l adapt i ve po le posi tioning", IEEE Trans. on Aut . Control, Vol . AC- 30 , No .1. 110 I Elliot H., R. Cristi a nd M.Das (1985) ."Global s tability of Adap tive Po le Placement Algorithms", IEEE Trans .on Au t. Control,Vo l.AC-30,No.4. 1111 Allidina, A. Y. and Huges, F.M. (1980) ,"Generalised self-tuning controlle r with pole assignment", lEE Proc ., Vo l.127, Ph.D., No .l. 1121 Trulsson E. (1983). "Adaptive control based on explici t cri t e rion minimization",Ph.D .Disse rtati on, Dept.of EI.Eng.,Linkoping University, Sweden. 1131 Goodwin,G.C.,Hill.D.J. and Palaniswami,M. (1984) . "A perspective on convergence of adaptive control algorithms", Au t omati ca , Vol.20, No.5,pp.519-531. 114 1 Gikhma n, 1. 1. and Skorokhod A. V., "Introduction into the the o r y of random processes", Nauka, Moscow, (in Russian). 115 1 Neveu J. ( 1965). "Mathematical Foundations of the Ca l culus of Probability", San Francis co , CA: Holden-Day. 116 1 Kailath, T. ( 1980). "Linear Systems", Prentice-Hall , Englewood Cl iffs, New Jersey.
Cop~Tight
ON THE ASYMPTOTIC BEHAVIOUR OF AN ADAPTIVE POLE-PLACEMENT ALGORITHM S. S. Stankovic and M. S. Radenkovic Fawl!.\· vf Elretriml ElIgillrnillg. L'lIit'nsit,l' vf Bl'lgradl'. Bdgmdl'. l'lIgu.\lm'ia
Abstract. In this paper convergence of an adaptive pole-placement algorithm based on the minimization of a predefined criterion is analysed. It is supposed that an external random excitation with possibly decreasing variance is injected into the system. The main result is the proof of the convergence w.p. 1 of the re gulator parameter estimates to the optimal ones, i . e . convergence w.p.1 of the closed loop poles to the desired ones, derived in two methodologically different ways. Convergence is ensured under mild conditions concerning the number of estimated parameters and requiring irreducibility of the optimal regulator transfer function. It is demonstrated that the algorithm converges even in the case of an external random excitation whose variance tends to zero,
despite the fact that the persistence of excitation is not satisfied. Cons isten cy proof for the parameter estimates in minimum variance self-tuning control algorithms can be derived as a special case . Keywords. Adaptive systems; pole-placement, stochastic approximation; global stability; convergence of the parameter estimates. INTRODUCTION
probability one can be guaranteed under mild conditions related to the number of esti mated parameters and the irreducibility of the optima l regulator transfer function. It will be suppose d that an external excitation signal is fed to the system, but we shall show that the strong consis tency of the parameter estimates can be ensured even in the case when this excitation tends to zero in the
Self-tuning control algorithms and their applications have been an objec t of study within control engineering for many yea rs, e.g . I 1, 2 I . Asymptotic properties of these algorithms have been analysed usin g different methodologies, e.g. 13,4,5,6,7 I. Goodwin, Ramadge and Caines 141 made an important step by proving asymptotic optimality and mean-square boundedness of the input-output variables for the self-tuning algorithms of minimum variance type. However, the convergence of the regulator parameter estimates themselves has still remained only partially understood I 5, 6, 7 I.
mean-square sense.
PROBLEM FORMULATION Let a time-invariant single-input single-output discre te-time sys tern be represented by an ARMAX model
The class of the self-tuning control algorithms based on pole-placement have attracted the attention of many researchers,e.g. 12,8 , 9,10,11,121. However, in spite of very valuable achievements, many important aspects of their convergence are still incompletely clarified. Most of the approaches to this problem are devoted to deterministic systems, with the focus on the role of the persistent excitation . 19, 10 I. In the stochastic case, the lack of a general methOdology for the parameter convergence analysis prevents drawing precise conclusions related t o the asymptotic pole positioning.
A(q -1) y(i)=q-
=b +b 1q
-1
o
+ ••• +c
nc
+ ••• +b n q -nc B
-n
B (b ;0) and C(q 0
-1 A
-1
)=1+c q' + 1
q
We shall adopt the following assumptions about the process characteristics: (AI) C(z) and B(z) are stable polynomials; (A2) C(z)-(~/2) is a strictly positive real function for some a>O.
In this paper we shall analyse a pole-placement self-tuning control algorithm based on the indirect approach, i.e. on the definition of a criterion function whose minimisation leads to the desired pole positioning. Similar ideas can be found in 11O,11,12!, but, however, analysis of the asymptotic properties of the described aliorithms are completely lacking in the literature 131. We shall prove first the global stability of the given algorithm including stochastic boundedness of the regulator parameter estimates, and then their almost sure convergence to the optimal regulator parameters in two different ways. In the first we shall analyse asymptotic properties of the closed-loop transfer function, and in the second conditions related to the persistent excitation. It is important to stress that the convergence of the c losed-loop poles to the desired positions with ASC-p
(1)
where (y(i)}, (u(i)} and (w(i)} are output , input and disturbance sequences , respectively, q-l stands for the unit delay operato r, d for the pure process time-delay, while A(q-l)=I+al q- 1+ ••• +a n q-nA,B(q-1)=
All random variables are defined on the underlying probability space {n,F,p} . We shall take xo={y(O), ••• ,y(l-k),u(l-d), ••• ,u(1-k),w(O), ••• ,w(l-k)}, where k=max{nA,nB+d,nC}' to be a random variable, and (w(i) ;i.::l} to be a stochastic process on {n,F,P}. We shall assume that a reference sequence (y*(i)} is given, satisfying: (A3) (y*(i)} is a bounded deterministic sequence (Iy*(i) I~m*), defined for i.::1 (y*(i)=O for i<1). (A4) For allNpairs of integers k.£ lim
1
I
y*(i -k) y*(i -l)
N-- N i=1
exists, and depends upon k-£. 149
150
S, S, StallkO\'ic alld \1. S, RadellkO\'ic
Following the methodology of Caines and co workers 15, 7 I we shall assume that an exogenous signal £(i) is added to the reference y*(i) (i>I). Its role is to ensure a sufficiently rich excitation of the process. Let {£(i)} be a stochastic process on {n,F,p}. Introduce the following assumptions: (AS) All finite dimensional distributions of x '
o (w( i)} and {£(i)} are absolutely continuous with respect to the Lebesgue measure; (A6) If Fo is the a -algebra generated by x o , and Fi that generated for i~1 by (Xo,w(I), ••• ,w(i), £(1), ••• , £ (i)}, then w.p.1 y(i)=u(i)=w(i)=£(i)=O for i
r~~~)lrW(i): £ (i)J IFi_I}=[a2 £(i)
0
0
~2/s(i)
controller (the cha racteristic equation is B(q-I) C(q-I)A*(q-I)=O, but B(q-I) and C(q-I) are ca ncelled in (9». The optimal regulator (B) depends explicitely on the process parameters . We shall suppose that these parameters are not known to the designer,i.e. that u(i) can only implicitely depend on them through observations. We shall construct an adaptive control algorithm based on the following assumptions: (A7) time-delay d is known; (AB) upper limits of nA' nB and nC are known. This algorithm, derived using the methodology of Goodwin and co-workers 12 ,41 and Caines and Lafortune 1 7 ; is defin~d by :
J
8 (i)= 8( i-d)+ -~- ~( i-d) [A*(q -I)y(i)_y*(i)_ r(1-d)
E{W(i)4I F _l; I,::k <00 w i E{s (i) 2£ (i) lE I }
-
r(i-d)=r(i-d-I)+. (i-d)T~(i-d); r(O)=1
Our control objective is to position the closed loop system poles at the zeros of a predefined stable polynomial A*(q-I) of degree n*. We shall attempt to achieve this goal , coping at the same time with disturbances, by minimizing the following J=lim
1
N
I'
N-N i=1
.
E{(A*(q-l)y(i+d)-y*(i+d»2 IF , }
(2)
1
If polynomials F(q-I) and ~(q-I) are_The minimum degree solutions with respect to F(q ) de ); F(q-I) < ~ nF=d-l, deg G(q-I) $nG~ax(nA-I,nCTn*-d» of the following diophantine equation C(q-I)A*(q -1)=A(q-I)F(q -I)+q -clC(q -I) (3) C(q-I) (A*(q-I)y(i+d)-y*(i+d)-£(i)-F(q-I)w(i+d»= -I -I -I -I =B(q )F(q )u(i)+G(q )y(i)-(C(q )-1) (y*(i+d)+£(i»-y*(i+d)-£(i)
GLOBAL STAB ILITY In this paragraph we shall analyse global stability and asymptotic optimality of the algorithm (10). (1 1) usin g the methodology presented in 141. We shall pay a special attention to the properties of the regulator parameter estimates, since they represent a prereq uisite for the consistency analysis. We shall also formulate a techni cal lemma crucial for the subsequent derivations.
THEOREM 1. Let assumptions (AI)-(AB) hold. Then for algorithm (l0), (11), with probability one:
'" i +id) 11 = 8 ~ <00 a) lim 118( o
o
10
8T~ (i)=y*(i+d)+£(i) 22
Jmin=a + ~ lim
N-
IN
P { s~p ll~ (i0+id) 112~£l'::
(6) I
N i~1 s(i)
L
c)
i=1
2 Ile(i 0 +id)- e (io+(i-1)d) 11 <00; (io=O, •.•• d-I)
z(i) 2 e) i t r(i)-
f)
1
Urn
N- N 1=
-I
)w(i»
2
I
E{ (A*(q-I)y(i+d)-y*(i+d)) IF, I 2
1
2
fo r s(i)=1 for s(i)=k 1ni s -T [
wher: 18 (i) ~~ (i - 8 o , 8 0 = go ••••• gnG'O ••••• 0, coe ff (B( z )F(z )} .0 •.••• O.cl' •••• c n ,0 •..•• O).the number of inserted zeros dependi~g on the chosen dimensions n I' n 2 and n3 in (10). (11). '"
2
00
( a I+ ~ <, 2 l al
(7)
s (i) =ks 1 ni
<
N
I
, I
s (i) = I
=0, where a l=E {( F(q
c£1 ; (O
00
(5)
0
where z(i)=A*(q-l)y(i+d)-y*(i+d)-£(i)-F(q-l)w(i+d), 8 =coeff{C(q-l) ,B(q-I)F(q-l) ,C(q-I)-I } and j0(i)T= =ly(i), ••• ,u(i), ••• ,-(y*(i+d-I)-di-1), •.•• The optimal control law is defined by 00
b)
(4)
C(q-l)z(i)= 8T~ (i)-(y*(i+d)+£(i»
(11)
where, in general, ~ (i) T= [y(i) , ••. , y(i -n ) ,u(i), ..• , l u (i~n2) , y*( i+d-I) +£ (i-I) , ••• ,y~(i+d-n )+£ (i-n 1 ») , nl~nG' n2~nB+nF' n3~nC' while a satisties (A2). The estimation algorithm (10) consists of d-interlaced stochast i c pr oce dure s with a.s. finite initial co nditions 8 (0)= 8 0 , 8 (1)=8 1 , .•• , 8 (d-I)=6 d_I'
i-+
one can wri te
or
(10)
e(i) T. (i) =y *( i+d) H(i)
E:
In (A6) (s(i) } is a deterministic sequence. We shall consider two cases: a) s(i)=I, which cor resp onds to continually disturbed controls discussed in I 7 I ; and b) s(i)=k 1ni (O
cri terion:
(a >O, i~d)
- £ (i-d)]
A
f
Proof. Statements a). c), d) and e) can be proven The optimal three-term controller following f rom (6) is given by
After straightforward manipulations. One can obtain from (10). for each of the re c ursions
B(q-I)F(q-l)u(i)=-G(q-I)Y(i)+c(q-I)(y*(i+d)+ +£ (i» while the output tern satisfies
using directly the results presented in 12.4. 7J. We shall. therefore. focus our attention to b) and f).
(B)
E{IIS( io+id) 112 IFi_d}.:: of
the
closed-loop
A*(q-l)y(i)=y*(i)+£(i-d)+F(q-l)w(i)
IIS( io+(i-l)d) 112+
sys+
Cl( io+(i-I)d) -I
(9)
Obviously, the system dynamics is defined by A*(q-I) so that (7) can be considered as a pole-placement
2
+ where Cl ( J')= ~2.W.L r(j) From
(3)
(12) -2 2 2 + a IIHj) 11 al r(j) 2
:\dapti\'e Pole-Placement Algorithm (13) Put for brevity io=O. Define the sequence (W(id)} i
~
2 W(id)= 116(id) 11 +E{) a(U -l)d) IF id }- ) c>(U -l)d) J=1 J=1 (14) Obviously W(id)~O, i~I, and (15)
E{W(id) IF( i_l)d }.::W(id) i.e. (W(id) ,F(i-l)d } bye)
is a supermartinga le. Since
) aCid) < ~ (16) i=1 '\, 2 and E{ 116 11 }<~)it follows th at E{W(O) }<~ a.s.and 0 E{W(id) }.::E{W«i-l)d) }.:: ... .::E{W(O)} < ~ (17)
• -1 ' . ( .) -n2 R(6(i) ,q )= 6 n1 + 1 (1)+ •• • +6 nl+n~1 1 q and •
L(6(i) ,q
-1
~ El .:: ~ E{W(i1d) } ; d~
W(id)~ll e( id)11
as a consequence of the fact that
Statement f) is proven in 110 I for A*(q -l ) =1 a nd s(i)=I. When s(i)=k £ni one can start from th e fundamental rel a ti o~ lim
~ ~
+£ (i) ) +C(q -I ) R( S( i) ,q -1)w(i+d)
1 ~
E{(A*(q-l) y (i+d)- y *(i+d»2 IF i ) N-- N i=1 2 . 1 ~ £ 2(i) 2 k £ni = 0 1 (20) = ° l+l1m N I. N-i=1 s after applying the lemma in Neveu 1151, pp.148-ls0. ( see also 12, 71).
Hiz' (i)=(A*BLi_ £-H _ £) (y*(i+d)+£ (i»+ i + (GRi_ £-BFSi_ £)w(i) =
Lemma 1. Let assumptions (Al)-(A8) hold. Then with probability one ( 8 (i o+(i-Od)T q,(i o+id» 2 <~ ;(io=O,_.,d-l) (21) i";o r(io+id) where £ is any finite integer.
r
z ' (i)=O
(i~l)
Ci.::O)
where £ is a finite integer, and ~ _. nQ _j P.= p.(i)q J a nd Q. = I qJ.( i)q . 1 j =0 1 j=o J
L
Lemma 2. Let assumptions
I
(Al~-(A8)
hold. Then w. p. 1
(Hi n+( i_ £)dz'(io+id»
( 28)
where £ i s any finite integer, while the argument w denotes that c 4 (w) is a random cons tant, possibly depen den t upon r ea li za ti on. Pr'oof. Pu t i =0. It follows from the statement d) in Theorem 1. thgt there exists a finite integer i 1 such that f or i >i l with probability one kr( w )r(i)~i (kr( w)<~ ). Rewrite (2R) in the following way
tl
- 8« (i- £)d» T;(id»2
(H(i_ £)dz'(id»2.::(np+nQ+2)
(29)
[.f' Pj«i_ £)d)2 J=O
(y *(i_d_j)2+£ (i_j)2)+ IQ q .« i_ £) d)2 w(i_j)2] j=o J (30) or, having in mind b) (H(i_ £)dz'(id»2.::
(23)
(a.s)
(H(i_~)dZ'(id»
n
(22)
Ile(id)-e(i-£)d)112<~
I
i=i +1 1
The second term in (29) is, thus, finite, having in mind Lemma 1 and a.s. boundedness of the parameter estimates (statement b), Theorem 1). We shall demonstrate boundedness of the first term starting from (26), which leads t o
(6«i_£)d)Tq,(id»2 <2( 6 (id)Tq,(id»2+2«8(id)one obtains (21) af ter using statements c) and e) of Theorem I, i.e. the fac t that 2 < 2 L (6 (id)T q, (id»2 + I (6(U'£)d)T
(H(i_ £)f'(id»2 +
i=1
Pr'oo f: Assume th a t i o=O. Starting from the obvi ous inequality
+
~Q
k~(w )
[I
P
(m*2+£(i_j»2+
j=o
w(i_j)2]
(31)
j=o CONS ISTENCY OF THE REGCLATOR PARAMETER ESTIMATES
We shall focus the analysis presented in this paragraph on the time-varying dynamic models of both the self-tuning three-term regulator and the whole closed-loop system. The proof of the consistency of the parameter estimates with be derived from the asymptotic behaviour of these models. Using (4), (8) and (10) the time-varying adaptive regulator model can be expressed as R( a( i) ,q-l)u(i)=-S( 8(i) ,q -1)y(i)+L( 6( i) ,q -1) (y*(i+d)+£(i» .
(27)
=Pi_ £(y *(i+d)+£(i»+Qi_ £w( i)
The following lemma is fundamental for all the sub sequent derivations concernin g the conve r gence of the regulator parameter estimates.
+ 2 I i=1
(25)
i=1
which can be derived fr om d) and e), and obtain lim
•
H. z(i)=(A*BL.-H.)( y*(i+d)+£( i»+( GR.-BF5.)w(i) 1 1 1 1 1 (26) Define the sequence (z '(i) } in the following way
(19)
(a.s) .
z(i)2 = 0
) -n3
e
i=1
N--
'
whe re H( El (i) ,q - 1) =A( q - 1) R( (i) ,q - 1) +q - dB (q - 1) S(6(i),q-l) is the cha rac teristic polynomial. Denoti~g sho:-tly H(6(i) ,q~I)=Hi' S(8(i)8- 1)=5 i , R( 6(i) ,q 1)=Ri and L( 6( i) ,q-l)=L i , it follows from (4) th a t
(18)
.
l
H( 6 (i) ,q -1) y( i+d )=B(q -1)L( 6(i) ,q -1 ) (y*(i+d)+
i~i 1
which implies
·'
)=1+6 n1 + n2 + 2( i)q- + ••• +6nl+n2+n 3+1 (i q
The closed-loop syste m is described then by
This means that (W(id) , F li-lld } is a co nvergen t supermartingale. The app11cat10n of the Kolmogo r ov inequality for semimartingales 114 I gives p{sup W(id)
151
where S(6(i) ,q
-1
'
(24) .
)= 60 (i)+6 1(i)q
-1
+ ••. +6
nl
(i)q
-n 1
where
k~ (w)
is a finite random constant.
Analyse now the sequence i wU)2 (). .1+6 } , where 6>0 . It is easy to see j=1 J that {t , F _ } is a convergent supermartingale, hai i 1 ving in mind that E{t. } is finite by virtue of (A6). Therefore, lim t.=t( w Y <~, and, consequently t < ii~ 1 2 6 1 .:: t(w)<~ for all i , implying w (i)
Thus
£
£
S. S. Stankovic and M. S. Radenko\'ic
152
( 32) implying boundedness of the first term at the right-hand side of (29). The lemma is, thus, proved. The main result of this section is formulated in the following theorem.
I ~ q.«i-£)d)qk«i-£)d)w(id-j)w(id-k)
Theorem 2. Let assumptions (A1)-(A8) hold, together with
n
(A9) At least one of the following equalities holds: n =fic=max (nA-1,n C+n*-d), n2=nB+nF=nB+d-1 and 1 n3=nC (A10)The optimal regulator transfer function G~(z)= =[C(z):G(z)F(B(z)F(z»-l is irreducible 1161.
E(id_j)2+ IQ qk«i-Od)2w(id-k) 2]<00 (a.sX40) k=o The first sum in (40) is finite as a consequence of (A2) and b). Consider the sequence
f
lim 8(i) =
e
i....-oo
(33) 0
j °,
T_ [ whe re eo-go"' " gn ' 0, •.. c oe f f { B( z-1 ) F ( z-l ) }T , 0, ..• ,0, cl' ... ,C ne , ,0 .
8, ...
I.~
lip }=V n
and, since
{V }
n
is L2_
rr-
I
i=l
f' P . «i_£)d)2 1J 2 < oo(a.s)
-,--J.-('d) 1S
j=o
1
)
i
IQ qk«i-£)d)2 0 2 < 00 k=o
i= 1 p . «i-Od)Pk«i-Od)y*«i+1)d-j).
j ,k J
Since
E(id-k)+2 I p.«i-£)d)Pk«i-£)d) E( id-j)E(id-k) j,k J ifj
(a.s)
(43)
~
-,--J.-(. d) for both considered choices i=l 1S 1 of s(id) (see (A6», it follows from (4~) gnd o (43) that there exists such a subsequence {t ;t .::t .:: ••• i 1 2
-
r p.«i-£)d)qk«i- £)d)(y*«i+1)d-j)+ j ,k
+ 2
1
lim p. «t~-£)d)=O; lim qk«t~-£)d)=O
J
i-+m
+ E(id-j»w(id-k)+2 I q . «i-£)d)qk«i-£)d) i,j J ifj
(34)
~
J
i-+
(44)
1
(j=O, .• • ,np;k=O, ••• ,n ) (a.s) Q Since by Theorem 1 the sequence{6(t?d)} is a boun1 ded sequence, it follows that these exists a subsequence {s?:sol
--
-11
(45)
Consic.ic r the sum
S
n
~ ~ 4p. «i-Od)p 1 '_1
(42)
J
Analogously, one obtains from (40)
n
,,(id-j)w(id-k)
n
-bounded one obtains lim Vn=Voo
P . «i_£)d)2 E(id_j)2+ IQ q.«i-Od)2 j =0 J j =0 J
I
n+
I'
(H(i_£)dz'(id»2 = (P(i_£)d Y*«i+1)d»2+
w(i-d-j) 2+2
p . «i-£)d)2(E(id-j)2_// s (id-j»} (41)
1
Obviously, E{V
Proof. Put again io=O. It follows from (27) that
+
~
{V }={ n i=l
Then with probability one
J
1-
k
«i-£)d)y*«i+1)d-:j)E(id- k) (35)
Therefore, according to (44) and (2/), (46)
Obviously
S~+l=S~+ n~l
Sn P j«n+1-£)d)Pk(n+1-£)d)
y*«n+2)d-j)E«n+1)d-k)+ _1_-2 p.«n+l-£)d)2 (n+1) J Pk «n+1-£) d) 2y *( (n+2) d-j) 2E «n+1) d-k) 2
(36)
Taking into account (A6) one obtains 2 2 1J 2 E{Sn+11 Pnd-k},::Sn+c 5 (w) ---2;:-...!:...--(n+1) s( (n+1) d-k)
(37)
(a.s) Moreover li m S2=S <00 n 00
I i=l
implying
fPj«i-£)d)Pk«i-£)d)y*«i+1)d- j )E(id-k)<00 (j ,k=O, ••• ,~)
(38)
Having in mind that Do can divid~ n~ither C (by (Ala» nor Bo ' it follows that Do =D o D0 2'
Oneoocan prove in a similar way that a.s. I ~ p . «i-£)d)Pk«i-£)d)E(id-j)E(id-k)
'_1 1
J
I i=1
Consequently, D~=Do1 and L =k'C, where k' is, in general, an arbitrary cons~ant. However, the lead~ ing coefficients of both C and L are equal to 1 by assumption (see (24\), so that k'=l, and, therefore, So=G, Ro=BF and Lo=C,
(j ,k=O, ••• ,~,jfk)
1-
i Pj«i-£)d)qk«i- £)d)(y*«i+1)d- j )+ +E(id-k»w(id-k)< 00 (j=0, ••.
,~;k=O,
(39)
••• ,n ) Q
Suppose that n =n , n2=n B+n F and n3=nC' Let D be 1 G the greatest common divisor of the polynomial~ G and BF (we shall ommit the arguments for simplicity) i.e. G=D Go and BF=(Do1Bo) (D02 Fo), Do1D02=Do' It follows ~hat (47) is satisfied for So=DbGo and Ra= =D~BoFo, where D~ is any polynomial satisfying deg D~,::deg Do' After replacing in (46) and using (3) one obtains D'B A*BL o =H 0 =AB 0 F0 D'+q-dBG D' = ~ (48) 0 0 0 D02 A*C
It is easy to check that the same conclusions hold in the more general case when at least one of the equalities: n =n , n =n +n F and n3=nC holds. 1 G 2 B Therefore, lilt A(s~d)=eo' i.e. limIIS(s?d) 11 =0. i-+-OD
1
i-+-oo
1
153
Adaptive Pole-Placement Algorithm
proving the a.s. convergence of (10), (11) is given.
" id) II} converges, i t Since by Theorerr. l", { Ile( follows that lim ; iO( id) 11 =0. In the same way one
Theor em 3 . Let assumptions (AI)-(AIO) hold. Then
i-
"" T R =lim 1 N ~ s(i)~(i)4>(i) ~ csI s N-- N i=1
concludes that for io=O, ... ,d-1 lim lle (io+id) 11 =0. i-
The theorem is, thus, proved.
(O
PERSISTENCE OF EXCITATION
and, moreover, 8(i) converges to eo with probability one.
It has been sho"n that the persistence of excitation condition plays one of the fundamental roles in ensuring convergence of stochastic recursive estimation algorithms 15 ,7,~. In this section we shall prove the almost sure convergence of the regulator parameter estimates using a methodology closely related t o the analysis of the persistence of excitation of the re gressor vector ~( i) in (10), (11). The proofs will be presented concisely because of the lack of space. We shall start the analysis by expressing the following way, using (4) and (I) y(i)
Hi)=
A*
z(i-d)
A = A*B z (i)
u(i) y*(i+d-I)+'(i-I)
(53)
~(i)
We shall analyse matrix RI obtained from Rs replacing ~(i) by ~defined as ~(i)=A*(q-I)B(q-l) 4>(i)=A*(q-I)B(q-I)~*(i)+A*(q-I)B(q-I)~ (i)+A*(q-1) B(q-I) 4>,(i)=~*(i)+~ (i)+$,(i), sincewRI>O implies Rs>O,having in mind t~at for any nonzero vector A
Froof;
R,Y
N
N
i= I
i= I
~ s(i) (AT'$(i)/ .:: c~ ~ s(i) (AT$(i» 2 (O
in
l
f1A* y*(i) + ~ y*(Hd) A*B
(54)
~~eR:r~~: ~~~~~c~~ ~:~:~e~o~;~~~~ :~(!~ ::~m:l~i~he sequence (xi)={s(i)y*(i-j)w(i-k)} (j,k <~ ). The application of the lemma in Neve u 1 2,15 1 leeds to the conclusion that N
lim 1 2. Xi =0, N--N i=1
<
since
~
y*(Hd-l)
0
a consequence, RI is positive definite irrespective of {y*( i)J if the condition
As
N
~*
1
w(i)
A*
lim 1 Y (AT~ (i»2 =0; lim N--N i";l w N--
,(i-d)
A "., 1') = ~z(i)+~*(i)+ + A*B
A w(i + - A*B
o
(49)
N
T
2
N I, s(i)(A $,(i»)=O i=1
(55) implies A=O. By virtue of (A6) random sequences (w(i)} and (IS(i),(i)} are ergodic, and, therefore, the He rglotz' s theorem can be applied to (55)., After straightforward calculations, similar to those in 1 7 I, (55) results into the following pOlynomial equations
, (i-I)
(56) (57)
Lemma 3. Let assumptions (AI)-(A8) hold. Then the parameter estimates generated by the adaptive control algorithm (10), (11) converge to of these exists a sequence {Vi} of finite ingegers satisfying vi ~ Amin{ ~ ~(i +(i-j»d)~(i +(i-j)d)T J j=o 0 0
e
~
i=l
r(io+id) (i
0
(50)
=0, ... ,d- 1)
PrOOf; Put io=1. Lemma I .implies, having in mind (49), b) and e) that
v · " , . .,
.L1(e(id)~«i-j)d» \' J =0 i~1 r(id)
2 < ~
(5 I)
for any finite j and any sequence {v,} of finite integers. Obviously, (51) implies 1 Vi " " A . {~ ~« i-j)d)~«i-j)d) T } ffi1n j=o 2 Y Ile(id)11 r(id) i=l (52) If it is possible to find a sequence {Vi } ensuring (50), Ile(id) 11 converges to zero w.p.1 as a consequence of a). In the following theorem an alternative way of
where polynomials AI(z) , A2 (Z) and A3(z) of degrees n1,n2 and n3-1, respectively, are defined by the corresponding components of vector A in (55). Suppose first that n1=nG' n 2=n B+o F and n3=nC' as well as that Do is tne greatest common divisor of G and BF, i.e.: G=DoG o and BF=(D o1 F?) (DOZFQ ), DQ= =D oI D0 2. Then AI=GoD~ and A2=BoFoDo sat1sty (50), where D~ is any polynomial of degree not greater than deg D . Suppose also that the greatest common divisor ofoC and BF is DI' i.e. C1=D 1C1 and BF= =(D1lB1)(DI2Fl)' D1=D11D12. Since Do and D1 are copnme by (A10). one concludes that Fo=F*Dll' F 1= =F*D o1 ' Bo =B*D 12 and B1=B*D 0 2. ConsequentlY,one obtains (58) A 3zB*D02F*D01=B*D12F*D11D~C1 From (58) D =Db and A3 zC, since Do cannot divide D. The last e q8a1ity contradicts, however, the assumption that C is a monic polynomial. Therefore, AI(z)= =0, A (Z)=0 and Ajz)=Ois the only solution of (56), 2 (57), i.e. A=O is (54). Therefore (53) holds, i.e. for A10 (59) (0<'1<~ )
(a.s)
implying that for every i large enough there exists such a finite integer Vi satisfying Vi T" 2 2 ) s«i-j)d)(A 4>«i-j)d» ~£21IAII j=o (a.s) Therefore,
(60)
IS4
S, S, StankO\ic and 1\1. S, RadenkO\'ic )I' '\,
'\,
T
The correspondin g conver gence proofs can be obtaine d as s pec i a l cases by putting simply A*(q-1)=1.
A , ( 1. 1 (~« i- j:P)( i-j)d) }
ml.n . __
I
, .-l~
_ __ _ _ _ ,. _ _ _
~
r(id)
i=l
(a . s)
(61) REFERENCES
The applica ti on of Lemma 3 gi ves the des ired result. The extension to values of n1,n2 and n3 satisfying (A9) i s st rai ghtforward .
III Rs trom, K.J . and B.Wittenmark (1973). "On self-tuni ng re gulat ors ", Automatica, 9, 185-199.
In the following th eorem we shal l test t he persistence of excitation of the vecto r (i) in (10) ,(11).
121 Goodwin,G.C. and K.S . Si n (1984). "Adaptive filtering , prediction a nd control" ,Prentice-Hall.
Theorem 4 . Let assumptions (A1)-(A10) hold. The n
13 1 Ljun g L. (1977) . " Analysis of recursive stochasti c algo rithms ",IEEE Trans.on Aut.Control, vol. AC- 22, No.4 .
fo r s(i)=l the persistence of exc it a ti on is sa tisfie d for vector ~ ( i) in (10), (11), L e . ~
R=lim
N )' i
N-+
~(iH(i) T::CRI
(O
(62)
=1
whi te for s(i)=ks £ni det R=O, i.e ,t he pe r sistence of exc itation is not sa ti sfied .
Proof: The proof ca n be derived following the lines of the proof of Theorem 3 . Denote onl y that the concl usion det R=O in the case whe n s(i)= =ks£ni fo ll ows from the fac t that N
N
lim~ ,2: (AT~£ (i»2.?lim~ ~ I I AI 12 11 ~£ (i) 11 2=0 N-+
1=1
N-+
fo r a ll finite A, s i nce lim N--=
i=l 2 k ~N s
(63)
N
I
£rrl =0.
i=l
The application of the Herglotz's th eorem leads to onl y one equation (59) , which has at least one nonze r o so l ution . This implies singularity o f R. CONCLUSION In thi s paper convergence of a self- tun ing con trol a l go rithm based on pole-placement has been analysed in the case of a s t ochas ti c discrete-time process. The algorithm is const ruc ted s t a rting from a mean- squa r e c rite ri on whose op ti miza ti on l eads t o the desired pole positions,and conta ins a stochastic appr oximation go rithm gene r a tin g the regulator par ame ter es tima t es . It is supposed that an externa l stochastic excitation is fed to the sys te~ its variance being either constant o r decreasing with time. In the first pa rt of the paper both t he global stability and asymp t o ti c op timalit y are proved. Properties of th e parameter estimates important for the main der i va tions are explored with more de tails. A key technica l lemma represents a basis for the subsequent consistency proofs. The ma in contribution of the pape r consists of the proof of the almost sure conve r ge nce of the re gulator parameter es timate s t o th e op ti ma l ones, i.e. of the c losed loop poles t o the desired ones, derived in two di ffere nt ways. In the second part of the paper conve r ge nce of the parameter esti ma tes is proved by analysing dire c tl y asymptotic propertie s of the closed-loop transfer function, for both t ypes of the external excitation. The conditions ensuring strong consisten cy are related to the dimension of the parameter vec tor and require irr e ducibility of the multivariable transfer function of the optimal re gulator . In the third part the same re s ults are obtaine d in a methodologically different way. The a nalysis is f ocus ed on the properties closely related t o the persistence of exc itation. I~ addition, the persistence of excitation of the re g ressor vec tor in t he algorithm are directly checked. It is proved that in the case of a decreasing excitation the strong consistency is ob tained despite t he fa ct that the persi s tence of e xc itation condition is not sa tisfied. This is an important result, since the asymptotic variance of the output is not affected by the excitation si gnal. It is t o be stressed that the given results provide the solution to the open problems of the convergence of the parameter estimates in minimum-variance self-tuning re gulators (see constatations in I2 ,13 1) .
141 Goo dwin, G. C. ,P.J.Ramadge a nd P.E.Caines (1981). "Discre te - time stochastic adaptive control",
SI AM J.Control a nd Optimization,Vol.19,No.6. 151 Chen,H.F.(1984) . "Recu rsive system identification and adap ti ve con trol by use of the modified least sq uare s algorithm" ,SIAM J .Cont rol and Optimiza ti on, Vol . 22 , No.5 . 161 Becker A.H . , P.R.Kumar and C. Z.Wei (1985) ."Adaptive Con trol with th e Stochastic Approximation Al go rith m: Ge omet r y a nd Conver ge nce" ,IEEE Trans. on Aut .Con trol , Vol . AC- 30 , No .4. 171 Caines,P.E. and S.Lafortune (1984). "Adaptive Control wit h Rec ursive Identi f icati on f o r Stochastic Linear Systems ", Vol. AC-29,No.4. 181 Rstrorr.,K.J. a nd B.Wittenmark (1980) ."Self-tuning con trollers based on pole-zero placement", lEE Proc ., 127, 120- 130. 19 1 Anderson, B. D.O. an d R.M.Johnston (1985) ."Globa l adapt i ve po le posi tioning", IEEE Trans. on Aut . Control, Vol . AC- 30 , No .1. 110 I Elliot H., R. Cristi a nd M.Das (1985) ."Global s tability of Adap tive Po le Placement Algorithms", IEEE Trans .on Au t. Control,Vo l.AC-30,No.4. 1111 Allidina, A. Y. and Huges, F.M. (1980) ,"Generalised self-tuning controlle r with pole assignment", lEE Proc ., Vo l.127, Ph.D., No .l. 1121 Trulsson E. (1983). "Adaptive control based on explici t cri t e rion minimization",Ph.D .Disse rtati on, Dept.of EI.Eng.,Linkoping University, Sweden. 1131 Goodwin,G.C.,Hill.D.J. and Palaniswami,M. (1984) . "A perspective on convergence of adaptive control algorithms", Au t omati ca , Vol.20, No.5,pp.519-531. 114 1 Gikhma n, 1. 1. and Skorokhod A. V., "Introduction into the the o r y of random processes", Nauka, Moscow, (in Russian). 115 1 Neveu J. ( 1965). "Mathematical Foundations of the Ca l culus of Probability", San Francis co , CA: Holden-Day. 116 1 Kailath, T. ( 1980). "Linear Systems", Prentice-Hall , Englewood Cl iffs, New Jersey.