Necessary and Sufficient Conditions for Strong Consistency of Recursive Identification Algorithm

C()p~right

©

11-":\( . Idt'lltii'itatitJll ;tIld S~ .. tt'1ll P;tr;llllt:ll'j

Estimatioll 1~I!'G , York, t"K. IQX:I

NECESSARY AND SUFFICIENT CONDITIONS FOR STRONG CONSISTENCY OF RECURSIVE IDENTIFICATION ALGORITHM H. F. Chen and L. Guo /mllllllr "/

S\' III'III'\

.)(iPlIlI'. Amdl'lllill Sinilll. Beijing. Thl' PI'''ple'l RI'p"bli(

"j Chilla

Abstract. For strong consistency of the parameter estimates given by the stochastic gradient algorithm this paper gives 1) a necessary and sufficient condition for systems with noise belonging to a class of practical i mportance and 2) some sufficient conditions for systems with ARMA type noise, these conditions are essentially weaker than the persistent excitation-like conditions, so that the optimal adaptive control law ca n be given for adaptive tracking with results that both the minimality of the tracking ,'error and the strong consistency of the parameter estimates are achieved simultaneously. Keywords. Stochastic systems; identification; parameter estimation; strong consistency; .; adaptive control. INTRODUCTION There is strong practical motivation and theoretical interest to estimate the unknown parameters of the linear stochastic system and to consider the problem whether the estimates converge to their true values. As a result of the recent intense research activity, there is now a vast literature on this topic. By using the persistent excitationlike condition Ljung (1976) and Moore (1978) studied the least squares method for system with uncorrelated noise. Afterwards, for the correlated noise case, the sufficient conditions guaranteeing the strong consistency are given by Solo (1g79) for the approximate maximum likelihood algorithm, by Chen (1982) for least squares algorithm, by Chen (1981) and Chen and Caines (198+) for the stochastic gradient algorithm and by Chen (1984) for the modified least squares algorithm. In this paper for the stochastic gradient algorithm we give a necessary and sufficient condition for strong consistency of the parameter estimates for the discrete-time system with noise which may be the martingale difference sequence and other correlated random sequences and show the continuous-time analogue. Then this condition is modified for the case when the system noise is an ARMA sequence. It is shown that this modified condition is essentially weaker than any persistent excitation-like condition and is sufficient for strong consistency of the estimates for unknown parameters appearing in the system as well as in the noise model. By use of this improved result for tracking problem we give an adaptive control which is optimal in the sense that the lorig run average of the tracking error is minimal and the parameter estimates are strongly consistent (Chen and Guo 1985a, 1985b,1985c). Consider the multi-input multi-output discrete(continuous-) time systems

m-dimensional system noise. z (s) denotes the shift-back (intrgral) operator t

zYn= Yn-1

r

Given any initial value 1. and stochastic gradient algorithm

( d

0

we consider the (2)

et

(2')

=

r~11t(dy~ -~t"etdt)

where ~ ( ~) is the ~

)

(4 ) measurable

regre-

ssion vector which will be given later on by (8) ((19» or (27) and It.

r

n

+

L

l=1

z

11 'f..1I ,

r

(3)

0

t

r

t

1+Jn~tcIs,ro

1)

(3')

o

Clearly, (2) is a difference equation which can be calculated recursively, wh:te (2') is a stochastic differential equation which is assumed to have the strong solution. A NECESSARY AND SUFFICIENT CONDITION Unlike the next section the unknown parameters in the noise model are not estimated here.

(4) Discrete-time case.

Let the system noise two ra~dom sequences:

E,nU:t ) is the

8

e,..,= e" + r~1~ (Y~+c ~T.e,,)

Set

where Yn (Yt) is the m-dimensional output, unCUt)

J YA d»,)

o

Let (11,!F, P) be the underlying probability space and lJ;;] ( {.1;) ) be a family of non-decreasing sub-cr--algebras of and let EnU: ) be:fi. (ft )-measurable. t

1.

is the i-dimensi onal input, and

(sYt=

En

be the superposition of (5)

1249

12,,()

H. F. Chcll alld L.

where (w n ':;:"') is a martingale difference sequence with E[", n /7",]

s

0, E[llw n 11'/1,;"]

=

c rf. 1

(6)

n-

0

where Then summation by par ts Yields the deslfed results

for some constants c > 0 and e E: [0, I), and { v } is an arbitrary random ~equence satisfying n a.s.

(;1I()

11

r"n (t~=~ '" r~w,:, )11

r~ ~ S ~+I (

= 11 S"n

The regression vector II~

If"

'f..t"

0(1) + o(l)r"

(8)

To consider the strong consistency of parameter estimates, we introduce matrix
er -

~ 11 t:.

-b - -'r )1/ ~... !

i.

rfcn,i), q;(i,i)=I.

1: ( -

0(1), n -oo

r/

n C='I

Finally for estimating the last two terms in (13) we have

(9)

"

II~..:F.( . ) y; T. 11 11 L.. '!" n+I, J+I ---w. III j=o J+

=li~ c? (n+I , j+I)(S-S

Ij

",

J

J =0

J-/

iD

1'\

~ [(n+I,j+l) - if' (n+I ,j)] S _11I

=I\s Theorem I. Let the system noise ll~} be defined by (5). Then for any initial value 6. the estimate 9.. given by (2) converges to a.s. i f and only

e

if

S'- ) 1 +1

0:>

" , " ] Yn···Yn-p+1 un···u n _ q+ 1


l:"

here is defined by

[~

J~=

1

Q)

(7)

r"n t~~I\ (S " -

= lJ

t-.co 0

j~

n

J

1\

= 11 S

)=0

(10)

a.8.


"

e-

Proof. Set ~ = e" . It follows from (1),(4) and (8) that yn+1 = e"1.. + t:"'I' This together with (2) and (5) gives

n

I

+ "I" n+ ,

+ cC

(11)

_

+
-s n

t

O)sll

+ c

r:1I.p(n+l,jt/)~'1/ ~ /=c r.>i r.1+6'

IIt(h".jtl)~·II~/i( ~

j=N

t

J

J

l-4f-.?

t='"

7

1 +2.0

'

which tends to zero by (14) and the fact that (12)

f JJ£..
, if we first let n

/:.0 tj'"''

or

~ 1.. e: = cP (n+I,O) e- -1'=0 ~(n+l,j+I)--4...w~ 1 J+ L...

lit!

-t

j=o

..li... v~ 1 rj' J+

q>(n+l,j+l)

and

For the last term in (13) we have

~

0

-00

then N - 0 1 )

(13)

Notice that the last two terms on the right-hand side of (13) is free of so the necessary part is obvious.

e.,

n .... Q>

For sufficiency we first establish the following inequali ty

..

a.s.

O.

"'--<>0 This completes the proof of the theorem.

(14)

which can be seen by noting

2.

Continuous-time case.

For the continuous-time case let the noise (et) be expressed by an Ito process: (16)

£ [ei> 11-1

~ tr

(n,i+t)p"(n,i+I) -cp(n,i).p'cn,i)]

n,

.

~ y;T.

'<

"-I

.

~ tr ~ p(n,Ht) r.i'(n,HI)=?; L=O

11 ~(n,i+f)tlJt

,...

•

tr Fl t"

Then we prove that there is a c >0 such that n '' .... Il10

Sn- IH

~,..~

-

S~

~'f.

n

~

l:"~ ' '11.. (aO

re:

and

are convergent martingales, hence for

a~; fix~d ~

we have

S -0 n " .. ~

lIS"1I n

0(1),

n--ClO

cortE

""'t~O

(17)

1

CX)

n '

In fact, it is easy to verify that

~

with constants co > 0 and E E [0, I ), and

~ cr-S

where S = S-S =~ --- w. I' S = ~ ~T. S_I=O. n n i:n+, rL 1+ n ~~ r~ 1+1'

~ 'f.1',., "'i+' f:o

processes satisfying

L-D.

,

S---+-O , 1/

where (w ,~ ) is an m-dimensi ona l standard Wiener ptocess, Ht and Ft are ~ -measurable

o

ds
The regression vector UI"

~

1t

is defined by

p-l ~"t q-I " ] Yt Ut'" s Ut

It = [y t ... s

and the matrix

(18)

a.s.

rs

f

'a 1>(t,5) at -_

(19)

(t,s) by

- 'Itr;'ft' t

.t; (

T

)

"' (s ,s ) =I.

t,s , T

(20)

Stroll g CO ll sistt' lln'

o ll{ cc llr s i \"(~

Id e ntif icat io ll ;\I gor it hm

Theorem 2 . If the system noise E~ is defined by (16), then for anv initial value 6. the estimate given' by (2 " ) converges to a.s. if and only i f -f (t , o) ~ O. a . s .

et

125 1

SUFFICIENT CON DI TIONS FOR

e

THE MA NOISE CASE

, .... a>

Let {f.} be a MA sequence Proof .

See-

-

et

e - et- '

t~

Then from (1') , (2 ' ) , (16), (19) and (20) we have tet- =
e: -

S

I)

S

driven by a martingale diffe r ence sequence I w t ' satisfying (6) wi th unknown matrices C, (i= l .~ . r) . I

5

Set C(z) = I + C1z + ... + Cr z

'fs Jot p et,s) -r: H ds s s

(24 )

= wn + Cl wn _ l +· . . + Cr wn- r

r

(25)

't

+

(2 1)

The continuous- time ana l ogue of ( 14) is

J-:- II 'f(U)~II" r: ds S

o

Since have

t --'lL dw's F"s •

~

'f.~= 't

d/2, a . s . (d=mp+l q) . (22)

(26)

e"C-= [Al" . Ap Bl . . . Bq Cl " . Crl

e •(

~ 't , .. 1,' ",' e' ,,' .•t 7) [ Yn"·Yn-p+l un .. · un_ q+lo-J•• ...;" /.."j":;. ... "....J 2

and w~_ r +ll

(28)

is a convergent martingale we

~

Let l' ( n, i) still be give n by t he recursive eq uation (9) but with '!~ defined by (27). Simila r ly , r ecursivel y define

E{_I- E[lIj(~dw'tF.'
J Q)

~

o

~

~

t-

1';1' Cl:> J I/~II ~ dsd t It t ~l.-t

~ L±l

CIl

coE

o

l} dt



wit h

and this implies Cl:>

fJ 11 fl (~/'S)c:lws'rJl~ d -'--"--"---'---~'-

rt

I)

t


a.s .

Y~;~O<) 'f'(n,i), f'(i'i)=I

tf>°(n +l ,i) = (I -

t

S

J

( 23)

'1,,0

•

given by (28) and r = 1 + n

L:" 11 'f.°lf J,

(:.,

The follow in g condition HI is crucial for the a na l ysis of rec ur s i ve , i dentification algorithm in the ARMA noise case . HI'

C(z) -

t

I is strictly positive real.

Appl yi ng I to ' s form ula we obta i n of: 1#

P (O ,t ) J ~ dw"'F't = o

rs

s s

t,t,( OTO, S)

J

"oS

0

oS

Theorem 3. For the sys tem desc r i bed by (1) Bnd (2 4) and the algorithm gi ven by ( 2) , ( 3 ) , (26) and (27) if Cond ition HI holds, then

J 0

t-

b 'f (O , s)

+(

~ ~

t

' J t= Thus, settIng and

J = Jt+]lt

J~ r. dw' F', o.s ss

dw'F" s s

re .iL dw'F"t Ij ss

-J = ) t t

J ep (O , s) ~r. dw' F '1I o 5 ss t

11 J - J + 'f (t,O)J -

t

T

"..,.00

a ny initial val ue &. .

(i i ) .

a,,-e "~

for

1> ( n,O) .,-:;;:

0 if a nd only i f -"o( n, O) ___ O. T

h-CD

we have

t

1/ ep (t,O)

~ (n , O) -;:--+ O implies

(i ) .

Proof.

J,,= Yn - wn - e~11f...

d


Set

1j~1= [ 0 .. . 00 ... 0

(O , s»( J s- J)ds ft

~T
J,," . .. 3,,-:(.. t l~

By using Condi t ion HI and the su pe r marti ngale convergence t heorem it is known tha t

~ n "'I

JI

'f~11l1 rl'l.


(29)

a.s.

which tends to zero by (22) and (23), i f we first let t ---+- ex> and then T ---+- ex>

Then an arg ument s i milar to that used i n Theo r em leads to Assertion ( 1). By using (14),(29) and t he following expression

By using (18) and (22) it is easy to prove that the last term in (21) a l so tends to ze r o .

4' (n+l,O) =i' (n+ l, O) + L

This proves the sufficiency part of the theo r em while the necessary part is trivial .

it can be shown that cp (n , O) ~ 0 implies

•

I\.

'f! 'f.' 'f."'f;#t • p (n+l,j +l )( !L!l. -~Jf(j , 0)

f:()

q

7'

p'(n,O) ~ O. The converse assertio n can be proved in a similar way .

1252

H. F. Chl'1I alld L.

(;1I0

Conditions a) or b) are usually called the pers i stent excita t ion condition. be the maximum and minimum eigenvalue of matrix 1\

1\

~ ~ f+;tI

(,,?'f:"~'~ +

cool

,::;

wi th d ~ mp+ 1 q+mr •

-f- I)

respectively

It can be shoWftthat a) implies b) which in its t urn implies c) (Chen 1984) and c ) leads to pCn,o) ~ 0 (Chen and Guo 1985b). Theorem 4 shows that the boundedness of the

Theorem 4. For the system described by (1) and(24) and the algorithm given by (2),(3),(26) and (27) i f Condition HI holds and i f for -'If' n ~No 'I."

~

/,,,

(or

iTni r / r n- 1<""

M(log rnft, rn-et) ,

/\ma¥ "min '

lA"min

ratio /'max .,." tency of the

urn

,

,,~.,

r'/r' 1<00

nI

J

0-

with some positive quantities N and M possibly depending on W , then for any ~nitial value 9.

In this section we continue to deal with the system described by (1) and (24) and the algorithm given by (2),(3),C26) and (27). Let

I + A z + ... + A zP 1 P

A(z)

8.S.

B(z) Proof. The proof consists of three steps. First we have to establish the following inequality

~

B + B z + •.. + B zq-l 1 2 q

We need the following conditions: + + B A(z) and B B(z) are left coprime and 1 1

B+B

a.s. -V-k~l where 0(,

estimates.

APPLICATIONS TO ADAPTIVE TRACKING

h..,.. n

;(1. v/),· .. ~M(log rc~,r'-+<2) ma~l l'mln n n

is not necessar y for consis-

~arameter

is of full rank.

1 q

Zeros of det[B1B(z)l lie outside the closed

fo '

N are positive quantities and

met) ~ max[n: t

n

p~I, q~1

unit disk, 'and

(. tl

p, q, m and

and

m~l

where

1 are defined in (1).

l

w } is an independent sequence of .random n vectors with properties: Then we need to show that there exists a positive quantity cl free of k such that

IIpCm(N+k
j

EWi~

i~

sup Ellw.II'~"'G
-.L' "tk~l. C,I(

1 -

0, for

.;

1

wi~

0;

0, for i< 0

for some 0;;0

and W,W."1 1

By using this estimation we can conclude that

R

>0

.:p(n,O) ~ 0 and then the convergence Theorem 3.

follows from

tI --.,. &I "'lltI_ v

We select the adaptive control Discussion:

For the strong consistency of recur-

satisfy

(uo~ ul~

law

u

n

to

0) :

sive identification algorithms the following conditions are usually used

(n ~ 2)

(30)

1"1

a) •

+~

'!,;'fi.<'

c~1

--+

R >0. (Ljung 1976).

tt.-+Q:)

where

"t n ~O and r n

{y~}

is a bounded deterministic reference

sequence and {vnJ --+0)

(Maore 1978).

c).

is an artificially introduced

i.i.d. random vector

l wn } with

sequence independent of

properties:

and for some

,to

0";;0.

where 0( and Tare pos1t1ve quant1t1es wh1ch may depend on cv and met)

~

max[n: t

n-,

~

n

tl,

t

n

~

'""

11

'fd/2.

L.. -r;-

,:=0

i.

(Chen 1981,1984; Chen and Caines 1984).

Under some mild conditions the existence of control satisfying (30) can be proved «(hen and Guo 1985a). By use of Theorems 3 and 4

~e

can establish the

Strong ConsistenC\' of Recursi\'e Identificatioll ;\\gorithlll following theorem: Theorem 5.

It for the system described by (1) and

(24) and the algorithm given by (2),(3),(26) and (27) Conditions H -- H4 are fulfilled, and the 1 control u i~ selected to satisfy (30), then the n

adaptive tracking system has the following properties:

(i). Stability lim 11--

(ii).

f;;"

1/

YiJ(~OO

(31)

4.5.

(32)

Optimality lim

r'~GD

(iii).

~

...!...

t JI

n (=-1

y*n21

y.-

1

tr R

Consistency

e.

Discussion:

(33)

4.S.

By taking the adaptive control

u

n

satisfying

e,,~ J'" = y~+ 1 Goodwin et al (1981) established properties (31) and (32), but in this case, generally speaking, (33) does not hold as is shown by Becker et al(1984). For strong consistency of parameter estimates in adaptive tracking systems, Caines and Lafortun,1984), Chen (1984) and Chen and Caines (1984) introduced a disturbance vn added to y~+l with EVnv;

being

a constant matrix and define un from: +

v

n

Then they established properties (31) and (33),but (32) can not take place anymore because of the non-attenuating disturbance v ' In these results n the fact that the covariance matrix of the disturbance does not go to zero is substantial, since there the persistent excitation condition is applied. In co ntrast to this, Theorem 4 guarantees the strong consistency of the estimates by not invoking the persistent excitation condition , namely,

11"ma j"n. /\mln

is allowed to increase unboun-

dedly. This is the principal reason why optimality in both estimation and control can be obtained in Theorem 5.

REFERENCES Becker,A. ,P.R.Kumar,an d C.Z.Wei,(1984). Adaptive control with the stochastic approximation algorithm --- Geometry and convergence, to appear in IEEE Trans.Autom.Control. Caines,P.E. and S.Lafortune,( 1984). Adaptive control with recursive identification for stochastic linear systems. IEEE Trans.Autom.Control,AC-29.

Chen,H.F.(1981). Quasi-least-squares identification and its strong consistency,Int.J.of Control,Vol.34,No.S,921-936. Chen~982). Strong consistency and covergence rate of least squares identification,Scientia Sinica,(Series A),Vol.2S,No.7,771-784. Chen~1984). Recursive system identification and adaptive con trol by use of the modified lea st squares algorithm, SIAM J. on Control and Optimization,Vol.22,No.S. Chen,H.F.3nd P.E.Caines,(1984). Strong consistency of the stochastic gradient algorithm of adaptive control,Proceedings of Conference on Decision and Control,Las Vegas,Nevada,December 1984,also accepted by IEEE Trans.Autom.Control. Chen,H.F. and L.Guo,(1985a) . Adaptive control with recursive identification for stochastic linear systems,to appear in C.T.Leondes (Ed.) Advances in Control a nd Dynamic Systems, Vol. 24, Academic Press. Chen,H.F. and L.Guo,(1985b). Strong consistency of parameter estimates for discrete-time stochastic systems, J.of Systems Science and Mathematical Sciences, Vol.5,No.2. Chen,H.F.3nd L.Guo,(198Sc). Strong consistency of recursive identification by no use of persistent excitation condition,Acta Mathematicae Applicatae Sinica (English Series) ,Vol.2,No.l. Goodwin,G.C.,P.J.Ramadge,and P.E.Caines,(1981) Discrete time stochastic adaptive control, SIAM J.on Control and Optimization, Vol.19, 829-853. Ljung,L.(1976). Consistency of the least squares identification method,IEEE Trans.Autom.Control, AC-21,No.5,779-781. Moore,J.B.(1978). On strong consist ency of least squares identification algorithm,Automatica, Vol.14,No.S,505-S09. Solo,V.(1979). The convergence of AML, IEEE Trans. Autom.Control,AC-24,No.6,958-962.