- Email: [email protected]
On Tracking Linear Plants Under Uncertainty
ON TRACKING LINEAR PLANTS
UNDER UNCERTAINTY
Max Mintz U n i v e r s i t y of I l l i n o i s Urbana, I l l i n o i s
I.
INTRODUCTION
I n t h i s paper we c o n s i d e r a minimax t e r m i n a l s t a t e e s t i m a t i o n problem which can be viewed a s a mathem a t i c a l model f o r a n e s t i m a t i o n problem i n which one i s a t t e m p t i n g t o t r a c k an uncooperative t a r g e t w i t h b a s i c a l l y unknown maneuvering c a p a b i l i t i e s i n a r e l a t i v e l y n o i s y environment. The problem can a l s o be thought of a s a s t o c h a s t i c two-person zerosum game i n which t h e p l a y e r s a r e t h e t r a c k e r and t h e evader. The t r a c k e r wishes t o e s t i m a t e t h e s t a t e of t h e t a r g e t a t t h e end of a predetermined time p e r i o d , by making use of t h e n o i s e c o r r u p t e d r a d a r d a t a . He knows t h e dynamics of t h e t a r g e t , t h e s t a t i s t i c a l d e s c r i p t i o n of t h e i n i t i a l s t a t e of t h e t a r g e t , and t h e s t a t i s t i c s of t h e a d d i t i v e obs e r v a t i o n n o i s e . The evader knows t h e same s t r u c t u r a l i n f o r m a t i o n and a l s o makes n o i s y measurements of t h e s t a t e of t h e t a r g e t - - h i s a d d i t i v e observat i o n n o i s e being s t a t i s t i c a l l y independent of t h a t of t h e t r a c k e r . The evader wants t o maximize t h e e s t i m a t i o n e r r o r of t h e t r a c k e r and a t t h e same time t o minimize t h e q u a d r a t i c c o s t of h i s c o n t r o l . Hence he i s faced w i t h making a compromise between two a l t e r n a t i v e s , which i s r e f l e c t e d through our g e n e r a l i z e d q u a d r a t i c l o s s f u n c t i o n t h a t weights t h e c o s t of t a r g e t c o n t r o l e x p l i c i t l y . We model t h e t a r g e t dynamics by a l i n e a r v e c t o r d i f f e r e n c e e q u a t i o n , a n d t h e i n i t i a l s t a t e of t h e t a r g e t by a gaussian random v e c t o r w i t h z e r o mean and known covariance. This random v e c t o r i s s t a t i s t i c a l l y independent of t h e a d d i t i v e o b s e r v a t i o n n o i s e s of t h e t r a c k e r and t h e evader--which a r e assumed t o be g a u s s i a n w h i t e n o i s e p r o c e s s e s w i t h known s t a t i s t i c s . A s i m i l a r f o r m u l a t i o n h a s been considered i n [ l ] f o r a problem i n which t h e evader does not make any o b s e r v a t i o n s of h i s own s t a t e . That i s , i n [I] t h e p e r m i s s i b l e c o n t r o l p o l i c i e s of t h e evader have been f o r c e d t o be open-loop c o n t r o l p o l i c i e s t h a t do not depend f u n c t i o n a l l y on t h e i n i t i a l s t a t e . A thorough t r e a t m e n t of t h i s problem i s a l s o given i n [2] where s e v e r a l o t h e r r e l a t e d s t o c h a s t i c game problems a r e considered. We have shown i n [ 11 and [2] t h a t f o r t h e c a s e when t h e evader has a c c e s s t o no s t a t e measurements, a minimax e s t i m a t e f o r t h e t e r m i n a l s t a t e of t h e t a r g e t can be r e a l i z e d by a l i n e a r e s t i m a t e which i s bayes with respect t o a gaussian l e a s t favorable prior d i s t r i b u t i o n f o r t h e c o n t r o l f u n c t i o n s of t h e evadex I n t h i s p a p e r , however, we extend t h e s e r e s u l t s i n
Tamer Basar Harvard U n i v e r s i t y Cambridge, Massachusetts
o b t a i n i n g a s o l u t i o n t o a more g e n e r a l problem-t h e minimax e s t i m a t i o n problem i n which t h e evader can a l s o make n o i s y s t a t e measurements. We a d d r e s s o u r s e l v e s , mainly, t o t h e e x i s t e n c e of a minimax t e r m i n a l s t a t e e s t i m a t e f o r t h e problem r a t h e r than t o t h e a c t u a l d e r i v a t i o n and computation of t h e opt i m a l p o l i c i e s of e i t h e r t h e t r a c k e r or t h e e v a d e r . We f i r s t o u t l i n e a proof f o r t h e e x i s t e n c e of l i n e a r minimax e s t i m a t e s f o r a sequence of r e l a t e d e s t i m a t i o n problems and then show t h a t a minimax t e r m i n a l s t a t e e s t i m a t e f o r t h e o r i g i n a l problem can be r e a l i z e d a s t h e l i m i t of a convergent sequence formed from t h e s e l i n e a r e s t i m a t e s . 11. MATHEMATICAL FORMULATION We assume t h a t t h e f o l l o w i n g information i s known by both p a r t i e s ( t h e t r a c k e r and t h e e v a d e r ) : The t a r g e t dynamics a r e d e s c r i b e d by a vector d i f f e r e n c e e q u a t i o n d e f i n e d by where x ( n ) r e p r e s e n t s a s t a t e v e c t o r of known dimens i o n m, u(n) r e p r e s e n t s a c o n t r o l v e c t o r of known dimension r , t h e m a t r i c e s f ( n ) and g ( n ) have t h e dimensions mxm and mxr r e s p e c t i v e l y , a n d n i s a n element of t h e d i s c r e t e time s e t 8 = ( 0 , 1 , . . ,N-11. The i n i t i a l s t a t e x ( 0 ) i s a z e r o mean g a u s s i a n r a n dom v e c t o r w i t h a known covariance m a t r i x denoted by q .
.
The t r a c k e r makes t h e f o l l o w i n g o b s e r v a t i o n s i n t h e extended time s e t 8 = ( 1 , 2 , N};
...,
where z (n) i s a n o b s e r v a t i o n v e c t o r of known dimens i o n m,lh fn) i s a known mxm o b s e r v a t i o n m a t r i x 1 d e f i n e d on 6 , where h (N) i s a nonsingular m a t r i x , v l ( n ) i s a n m v e c i o r z e r o mean gaussian white and n o i s e p r o c e s s w i t h a known covariance m a t r i x d e f i n e d by: c o d vl (n) ,vl ( k ) l = r l (n)+,k'
(3)
where r ( n ) - i s a n mxm p o s i t i v e d e f i n i t e m a t r i x 1 d e f i n e d on 8 . F u r t h e r , t h e normal random v e c t o r s {x(O) , v (n) ;nee] a r e assumed t o be s t a t i s t i c a l l y independenk. The evader a l s o makes n o i s y o b s e r v a t i o n s of t h e s t a t e v e c t o r a s i t e v o l v e s . These o b s e r v a t i o n s a r e
x = "F + E L , -0
d e f i n e d by t h e s e t z 2 (n) = h2 ( n ) x ( n ) + v2 (n) : nee
,
(4)
where x8[xt(1)1 -
where dim(z )=m; v ( n ) r e p r e s e n t s a z e r o mean gaus2 2 s i a n white noise pfocess s t a t i s t i c a l l y independent of { x ( 0 ) , v ( n ) ; nee] , w i t h a known c o v a r i a n c e m a t r i x d e f i n e d by:
where r (n) i s a n rnxm p o s i t i v e d e f i n i t e m a t r i x 2 d e f i n e d on 8 . F u r t h e r , h (n) i s a known mxm o b s e r v a t i o n m a t r i x d e f i n e d on
i.
The e v a d e r h a s complete c o n t r o l o v e r { u ( n ) ; nee] and i s p e r m i t t e d t o choose t h i s c o n t r o l sequence t o be a n y mapping of 8 i n t o E~ which i s m e a s u r a b l e and n o n a n t l c l a t o r w i t h r e s p e c t t o t h e o b s e r v a tionThe n o n a n t i c i p a t i v e p r o p e r t y of t h e c o n t r o l sequence { u ( n ) ] i m p l i e s t h a t f o r a l l n e e , u ( n ) d o e s n o t depend f u n c t i o n a l l y on t h e d a t a s e t {z2(k); ~ n The rp e r m i s s i b l e c o n t r o l s f o r t h e e v a d e r w i l l a l s o have t o be s t a t i s t i c a l l y independ e n t of t h e t r a c k e r ' s o b s e r v a t i o n n o i s e { v l ( n ) ; We d e n o t e t h e c l a s s of c o n t r o l s f o r t h e nee]. e v a d e r , which p o s s e s s e s t h e above p r o p e r t i e s b y % . Given t h i s p r i o r i n f o r m a t i o n , t h e t r a c k e r ' s o b j e c t i v e i s t h e d e t e r m i n a t i o n of a minimax e s t i m a t e f o r t h e t e r m i n a l s t a t e x(N) o f t h e t a r g e t w i t h r e s p e c t t o t h e f o l l o w i n g g e n e r a l i z e d q u a d r a t i c l o s s funct i o n L(6 ( z l ) , u ) :
x -0
uses
The t r a c k e r ' s o b j e c t i v e c a n now be d e f i n e d i n t e r m s of t h e c o n d i t i o n a l r i s k f u n c t i o n r ( 6 , u ) which i s g i v e n f o r f i x e d heA, us% by
4 [x'
...I
x'(N)]',anNmvector,
.. . I
(0)l
-u 4 [ u ' (0)l . . . I
' , an
x ' (o)]
u ' (N-1)]
,
(9b)
Nm v e c t o r ,
(9c)
a n Nr v e c t o r ,
(9d)
a n NrnxNrn m a t r i x d i a g o n a l i n b l o c k s of rnxm m a t r i c e s ; t h e i i ' t h b l o c k b e i n g g i v e n by
-
1 4 [ (fg)il . . .I ( f g ) $ ' , a n
... I
NmxNr m a t r i x ( 9 f )
. . .Ig ;
g(n-1)l~I
(9g)
a n mxNr m a t r i x w i t h N p a r t i t i o n s f o r e a c h n e e . The o b s e r v a t i o n d a t a (2) of t h e t r a c k e r c a n be written as:
z1 =
H15 +
x17
(10a)
where
where F.(z ) d e n o t e s a n a f b i t r a r y mapping o f t h e 1 d a t a sequence { z ( n ) ; nee] i n t o a n m-dimensional v e c t o r t o be a s an estimate f o r x(N)--this mapping may depend o n a l l of t h e p r i o r i n f o r m a t i o n known t o t h e t r a c k e r . We w i l l h e n c e f o r t h d e n o t e t h e c l a s s of s u c h maps by A. C i s a n rnxm non-negat i v e d e f i n i t e w e i g h t i n g m a t r i x , and d ( n ) i s a n r x r p o s i t i v e d e f i n i t e weighting matrix f o r each nee.
(9a)
z b[zt(l)l -1 A = [v;(l)I
x1
...I z ' ( N ) ] ' ,
a n Nmvector,
(lob)
. . . I v;(N)] ' ,
a n Nm v e c t o r ,
(10c)
H I , a n NmxNm m a t r i x d i a g o n a l i n b l o c k s of rnxm m a t r i c e s ; t h e i i ' t h b l o c k b e i n g g i v e n by:
-
mllii
= hl(i).
(10d)
Further define:
-
Q, a n NmxNm m a t r i x h a v i n g i d e n t i c a l b l o c k s of rnxm m a t r i c e s g i v e n by:
-
"Q1 3.
= q, for a l l i , jag,
(lla)
R1; a n NmxNm m a t r i x d i a g o n a l i n b l o c k s of rnxm m a t r i c e s ; t h e i i ' t h b l o c k b e i n g g i v e n by:
[.I
where E denotes the expectation with respect t o t h e s t a t i s t i c s of { x ( O ) , v (n) , v ( k ) ; na6, k e e l . 1 2 D e f i n i t i o n of a minimax e s t i m a t e f o r x(N): I n t h e c o n t e x t of t h e p r e v i o u s n o t a t i o n , a n e s t i mate C " ~ A i s s a i d t o be minimax i f f o r a l l 6 e A , sup r ( r 4 , u ) 5 us%
SUP
r(5 , u ) .
(8
~ € 3
We now g i v e a n a l t e r n a t i v e f o r m u l a t i o n f o r t h i s problem i n t h e n e x t s e c t i o n , which w i l l a i d u s i n p r o v i n g t h e e x i s t e n c e of a minimax e s t i m a t e f o r t h e problem posed i n t h i s s e c t i o n .
t h a t i s , T h a s N p a r t i t i o n s w i t h (N-1) z e r o m a t r i c e s and one mxm i d e n t i t y m a t r i x . C = T'CT, (12b) t h a t i s , C i s a n NmxNm m a t r i x h a v i n g b l o c k s of rnxm m a t r i c e s a l l of which a r e e x c e p t t h e NN'th b l o c k which i s g i v e n by CNN = C. F u r t h e r , d e f i n e D t o be a n NrxNr p o s i t i v e d e f i n i t e m a t r i x which i s d i a g o n a l i n b l o c k s of r x r m a t r i c e s , t h e i i ' t h b l o c k b e i n g g i v e n by: Dii
111. AN ALTERNATIVE FORMULATION We now o b s e r v e t h a t t h e s e t of l i n e a r d i f f e r e n c e e g n s ( 1 ) c a n a l s o be w r i t t e n a s :
= d(i-1).
(13)
Using t h e p r e c e d i n g n o t a t i o n , we now pose a minimax e s t i m a t i o n problem which w i l l h e n c e f o r t h be r e f e r r e d t o a s Problem A i n t h e p a p e r .
i i i z 2 ( n ) = H (n)xo
Problem A F i n d a minimax e s t i m a t e n*(zl) f o r t h e v e c t o r 2 d e f i n e d by ( 9 a ) and u s i n g t h e o b s e r v a t i o n given by (10a),-under t h e g e n e r a l i z e d q u a d r a t i c l o s s f u n c t i o n L(ll ( z l ) ,E) d e f i n e d by:
zl
where C a n d D a r e a s d e f i n e d by (12b) and (13) The v e c t o r E i s a l l o w e d t o depend respectively. on t h e o b s e r v a t i o n s {z ( n ) ; nee] d e f i n e d by (4) i n 2 a nonanticipatory fashlon i n the sense t h a t the f i r s t n r e l e m e n t s o f 2 d o n o t depend f u n c t i o n a l l y 3 x f o r a l l nee. T h i s on t h e d a t a s e t { z 2 ( k ) ; 1 i s i n a c c o r d a n c e w l t h t h e d e f i n i t i o n of t h e p e r m i s s i b l e c l a s s of c o n t r o l s q f o r t h e e v a d e r g i v e g i n ' with Q ' , and s e c t i o n 11. For Problem A we r e p l a c e & t o stand f o r t h e perf u r t h e r we r e p l a c e A w i t h m i s s i b l e c l a s s of e s t i m a t e s f o r t h e v e c t o r 2 . Denoting t h e c o n d i t i o n a l r i s k f u n c t i o n a s s o c i a t e d w i t h t h e l o s s f u n c t i g n d e f i n e d by (14) by Y((n,g), we n o t e t h a t n*(z )eA i s a minimax e s t i m a t e f o r -1 x if -
+
i W ( n ) , nee,
(19a)
where z l ( n ) i s a n Nm-dimensional v e c t o r , H1(n) i s a n Nmxd m a t r i x d i a g o n a l i n b l o c k s of mxm m a t r i c e s ; t h e j j ' t h b l o c k b e i n g g i v e n by n-1
f o r a l l nee. i W ( n ) r e p r e s e n t s a z e r o mean g a u s s i a n w h i t e n o i s e process s t a t i s t i c a l l y independent of {xi,v1], with a c o v a r i a n c e m a t r i x d e f i n e d by:
a
sup
"rn*
-uc'l%
,g) Isup ue'ci
where R(n) i s a n NmxNm p o s i t i v e d e f i n i t e b l o c k d i a g o n a l m a t r i x , t h e j j ' t h b l o c k b e i n g g i v e n by
"rn ,u).
for a l l n(.)ei. We now o b s e r v e t h e f o l l o w i n g p r o p e r t y r e l a t i n g t o t h e s o l u t i o n s of t h e problem of s e c t i o n I1 a n d Problem A: Property 1
,
j = 1
,
otherwise
Denoting by T~ t h e ( p r o j e c t i o n ) f o r a l l nee. t r u n c a t i o n map t h a t r e s t r i c t s a n y Nm d i m e n s i o n a l v e c t o r t o i t s f i r s t m e l e m e n t s , we f u r t h e r r e q u i r e wl(n) t o p o s s e s s t h e p r o p e r t y
where v ( n ) was d e f i n e d by ( 5 ) . 2
The e x i s t e n c e o f a minimax e s t i m a t e n*(iz,)eZ t o Problem A i m p l i e s t h e e x i s t e n c e o f a minimax e s t i mate 6*(z ) € A t o t h e problem posed i n s e c t i o n 11. 1 F u r t h e r m o r e , t h e s e two e s t i m a t e s have t h e s i m p l e relation
T h i s , t o g e t h e r w i t h t h e d e f i n i t i o n of Problem A i n s e c t i o n 111, c o n c l u d e s t h e f o r m u l a t i o n o f Problem B. w i t h t h e e x c e p t i o n of t h e i n t e g e r i which w i l l be d e f i n e d a s f o l l o w s :
where T i s d e f i n e d by ( 1 2 a ) .
( i ) Denote by i t h e s m a l l e s t p o s i t i v e i n t e g e r s u c h t h a t for a l ? integers i > i n , the matrix
Hence, i n t h e r e m a i n i n g p a r t s of t h e p a p e r we w i l l b e d e a l i n g m a i n l y w i t h Problem A and p r o v e t h e e x i s t e n c e of a minimax e s t i m a t e n*(zl) a s a l i m i t i n g c a s e of t h e s o l u t i o n of a n o t h e r e s t i m a t i o n problem (Problem B) which i s posed i n t h e n e x t section. IV.
A RELATED MINIMAX ESTlMATION PROBLEM-PROBLEM B
The f o r m u l a t i o n of Problem B i s s i m i l a r t o t h a t o f Problem A w i t h t h e f o l l o w i n g m o d i f i c a t i o n s : (1) Replace
byF1
which i s d e f i n e d by
1 = F + - I (17) i Nm' where i i s a p o s i t i v e i n t e g e r t o be d e f i n e d i n t h e s e q u e l by (ZO), and IN, s t a n d s f o r t h e Nm-dimensional identity matrix.
F
i
(2) Replace x v e c t o r x i su$
by a n o t h e r zero-mean g a u s s i a n Nm that i i' i 1 E [ x ~ x Q ~ ~= Q + y I N m . (18)
4
( 3 ) R e p l a c e t h e o b s e r v a t i o n z ( n ) , d e f i n e d by ( 4 ) 2 by
[
n- 1 f(k) k= 0
n
1
+ TIm]i s n o n s i n g u l a r .
( i i ) Denote by in t h e s m a l l e s t p o s i t i v e i n t e g e r s u c h t h a t for a l l i n t e g e r s i > in, the matrix [h2(n)
n- 1 T! f ( k ) k= 0
(iii) Let i max
+
1
i s nonsingular.
4 - arg
max { i n , i n } . ne6
Then p i c k i n g i 2 imax c o n c l u d e s t h e f o r m u l a t i o n of Problem B . We p a r e n t h e t i c a l l y n o t e t h a t f o r a l l f i n i t e i 2 i m a x , F1 a n d H1 (?) a r e n o n s i n g u l a r f o r a l l n e 6 , and f u r t h e r t h a t Q~ i s p o s i t i v e d e f i n i t e f o r a l l f i n i t e i. T h i s p a r t i c u l a r c h o i c e of i h a s been made i n o r d e r t o i n s u r e t h e e x i s t e n c e of a s o l u t i o n t o Problem B, which c a n t h e n be g e n e r a l i z e d t o p r o v e t h e e x i s t e n c e of a minimax e s t i m a t e t o Problem A . Before g o i n g i n t o a d i s c u s s i o n of t h e s o l u t i o n o f Problem B we want t o remark on t h e a p p a r e n t s i m i l a r i t y between t h e f o r m u l a t i o n s of Problems A and B , i n t h e l i m i t a $ i g e t s a r b i t r a r i l y l a r g e . The two r e l a t i o n s F'IF and ~ l 1 h r oe b v i o u s from ( 1 7 ) a n d . (18) r e s p e c t i v e l y . However, t h e f a c t t h a t z $ ( n ) d e f i n e d
by (19a) p r o v i d e s t h e same i n f o r m a t i o n , i n t h e l i m i t , t o t h e maximizing p a r t y ( e v a d e r ) a s z (n) d e f i n e d by (4) d o e s , f o l l o w s from t h e n o n a n t g c i p a t i v e property of the c o n t r o l functions of t h e maximizing p a r t y . The r e a s o n why we have i n t r o duced a d d i t i o n a l a d d i t i v e n o i s e i n r e l a t i o n (19a) i s a g a i n t o i n s u r e t h e e x i s t e n c e of a s o l u t i o n t o Problem B. I n summary, Problem B becomes i d e n t i c a l t o Problem A i n the l i m i t a s i gets a r b i t r a r i l y large. I n what f o l l o w s we w i l l f i r s t g i v e t h e s o l u t i o n t o Problem B f o r e a c h f i n i t e i and t h e n c o n s t r u c t a l i m i t i n g scheme t h a t w i l l i n s u r e t h e e x i s t e n c e o f a minimax e s t i m a t e f o r Problem A. V.
SOLUTION TO PROBLEM B
The s o l u t i o n t o Problem B i s p r o v i d e d i n Theorem 1 below. P r e l i m i n a r y N o t a t i o n : - - D e f i n i t i o n of m a t r i c e s which a p p e a r i n Theorem 1. H I ; a n N2mxN2m m a t r i x , diagonal i n blocks of N G M m a~t r i c e s , t h e j j ' t h b l o c k b e i n g g i v e n by
f o r any given A' with t h e property r(Ai)
< 0.
(27b)
The s o l u t i o n t o t h i s m a x i m i z a t i o n problem w i l l , i n g e n e r a l , depend on t h e c h o i c e of A1. We emphasize t h i s dependence by w r i t i n g as pi = @ i ( ~ ~ )
(27~)
where Oi i s a m a t r i x v a l u e d f u n c t i o n o f a p p r o p r i a t e dimension d e t e r m i n e d t h r o u g h t h e p r e c e d i n g maximizat i o n problem. We now s e t
A'
. iH;[ A
i-iSii;+ ill
(28a)
i where A depends on pi t h r o u g h (24) w i t h . P r e p l a c e d We i n d i c a t e t h i s dependence of A' on pi by by pi. i i i A = cp ( P ),
(28b)
where mi c a n r e a d i l y be d e t e r m i n e d t h r o u g h (24) and (28a). We now s e e k a s o l u t i o n t o
-
JJ
2 2 where ~ ~ ( ~ i-s 1d e) f i n e d by (19b). R; a n N mxN m p o s i t i v e d e f i n i t e block diagonal matrix, t h e j j ' t h b l o c k b e i n g g i v e n by
[iljj
= ~(j-11,
P. = 0 for j < k. (22) ~k We d e n o t e t h e s p a c e o f a l l s u c h m a t r i c e s P b y R and n o t e t h a t e a c h e l e m e n t o f 0 i s u n i q u e l y deFerP ! l k ! e! nh t r i e s , i . e . , np i s mined by s p e c i f y i n g & 2
-
dimensional euclidean isomorphic t o the 2 space. A~ ; a n ~ m x ~ rmna t r i x (23)
P
wJ7i(~i>)<
0
Theorem 1
(
i ) For e a c h f i n i t e i n t e g e r i 2 i e i s t s a t l e a s t one m a t r i x - v a ~ r$ (.) s u c h t h a t pi g i v e n by (27c) J ~ ( P )o v e r R f o r e a c h Ai s u b j e c t P
H
( i i ) For e a c h f i n i t e i n t e g e r i 2 i a t l e a s t one m a t r i x why:$ (29a) and (29b).
there ~f ui n c t i o n maximizes t o (27b). there e x i s t s satisfies
( i i i ) A minimax e s t i m a t e *nl(z ) f o r Problem B i s given f o r each f i n i t e inkeger i 2 i max by
'i ; a n NmxNm m a t r i x d e f i n e d f o r by
where.*Ai i s g j v e n by (28a) w i t h h i r e p l a c e d by *A', and *A' i s d e f i n e d by (24a) w!th P r e p l a c e d by *P: o r e q u i v a l e n t l y *A1 i s g i v e n by a i ( * p l ) .
--
i
r(A ) ; a n NrxNr m a t r i x g i v e n by
i J (P) i s a s c a l a r f u n c t i o n of P a p d e f i n e d f o r e a c h g i v e n A' by
where
subject t o the condition t h a t
(21b)
2 where R ( j - 1 ) i s d e f i n e d by (19d). P; a n NrxN m m a t r i x , lower t r i a n g u l a r i n b l o c k s o f rxNm m a t r i c e s , i . e . , t h e j k ' t h b l o c k i s g i v e n by
e a c h Pd?
i i i p i = @ (a ( P I ) ,
~ r C(H i e H i ' + z ) ~ ' ~ ~ ] ,
depends on P t h r o u g h (24).
denote any s o l u t i o n t o Now, l e t t h e m a t r i x pi& t h e f o l l o w i n g m a x i m i z a t i g n problem f o r a n y f i n i t e i n t e g e r i 2 imax'. max J ~ ( P ) , (27a)
PC
P
A proof o f p a r t s ( i ) a n d ( i i ) of Theorem 1 w i l l n o t be g i v e n h e r e b e c a u s e o f i t s l e n g t h . It c a n be found i n o u r r e l a t e d l o n g e r p a p e r [3] . A proof of p a r t ( i i i ) o f Theorem 1 w i l l be p r o v i d e d h e r e by showing t h a t *ni(El) and ui=*pizi s a t i s f y the -2 f o l l o w s n g saddl_e;point i n e q u a l i t y ( 3 1 ) f o r a l l n l ( . ) e ~ ~z(.)eQ1 , and f i x e d i 2 imax:
-
I n the preceding i n e q u a l i t y , r i ( - , * ) denotes the c o n d i t i o n a l r i s k f u n c t i o n ass_ociated-with Problem B and f o r a n y f i x e d i; f u r t h e r A1 and ?A1 d e n o t e t h e p e r m i s s i b l e c l a s s of s t r a t e g i e s f o r Problem-B d e f i n e d i n a way s i m i l a r t o t h o s e of A and Q r e s p e c t i v e l y f o r Problem A. The v e c t o r z1 i s ~ ~ m -2 d i m e n s i o n a l and i s d e f i n e d by
-
where z i ( n ) i s g i v e n by (19a).
p r o v i d e s a minimax e s t i m a t e f o r Problem A and hence c o m p l e t e t h e proof of t h e e x i s t e n c e o f a s o l u t i o n t o Problem A.
We n o t e p a r e n t h e t i c a l l y t h a t i n e q u a l i t y (31) implies t h a t
For t h i s p u r p o s e e d e f i n e a m e t r i c ~ ( A , B ) :n -Ey by P P 2 Nr N m 2 $ P (A,B) = C C Z (A - B ~ I~ ) k j k=l j=l
and hence *lli
Using e x p r e s s i o n s . ( 2 6 ) and ( 2 8 a ) , i t c a n be shown t h a t [ s e e [3]] *pl d e f i n e d i n p a r t ( i i ) of Theorem 1 i s bounded i n norm by a p o s i t i v e number 7 g i v e n by
i s minimax.
Proof o f p a r t ( i i i ) o f Theorem 1:
-
BY s u b s t i t u t i n g 2i *piz$ i n t o t h e c o n d i t i o n a l r i s k f u n c t i o n Zi(ni,u) a s s o c i a t e d w i t h Problem B we o b t a i n :
*
( [ . I m . , d e n o t e s t h e minimum e i g e n v a l u e o f [.I>?.) , where t h e norm i s induced by t h e for a t 1 i 2 i m e t r i c (35a).ma3e n o t e t h a t 7 i s i n d e p e n d e n t o f i a n d hence *pidl for a l l i 2 i where R i s 7 7 max' d e f i n e d by
where
It f o l l o w s from (33b) t h a t yi i s a normql random v e c t o r w i t h mean z e r o and c o v a r i a n c e *A' which i s But, a? g i v e n by (24) w i t h P r e p l a c e d by *P1. e s t i m a t e t h a t m i n i m i z e s < 3 3 a ) , when y i e ~ @ , * ~ l ) , i s t h e b a y e s e s t i m a t e * l l l ( ~ ~g)i v e n by (30). T h i s completes t h e proof of t h e right-hand-side i n e q u a l i t y o f (31).
where
11 -11
i s t h e norm i n d u c e d by t h e m e t r i c ( 3 5 a ) .
We now o b s e r v e t h a t R i s a c l o s e d and bounded subs e t of Rp , and hence P h e m e t r i c s p a c e (& , p ) i s e l e m e n t of R p , t h e
To p r o v e t h e l e f t - h a n d - s i d e i n e q u a l i t y of ( 3 1 ) , we s u b s t i t u t e *ni(gl) g i v e n by. (30) i n t o t h e c o n d i t i o n a l r i s k f u n c t i o n Ti(nl,u): Proposition 1 T h e r e e x i s t s a l i n e a r minimax e s t i m a t e f o r Problem A, which i s g i v e n by
where A* i s g i v e n by
where t h e e x p e c t a t i o n . o p e r a t i o n i g t a k e n o v e r t h e p r i o r s t a t i s t i c s o f x i , v and {w1(n);ne8]. gut s i n c e r(*A1) < 2, t h e maxkmum o f (34) o v e r ueQ1 e x i s t s and, without any l o s s of g e n e r a l i t y , t h i s maximum may be s o u g h t o v e r t h e s p a c e of l i n e a r policies. Hence, maximizing (34) o v e r Gi. i s e q u i v a l e n t t o maximizing (34) w i t h 2 = PZ' o v e r -2 Rp. But t h e maximum i s a c h i e v e d by *pi a s g i v e n by p a r t s ( i ) and ( i i ) of Theorem 1.
w i t h pk b e i n g computed t h r o u g h t h e l i m i t i n g procedure o u t l i n e s i n t h i s s e c t i o n p r i o r t o the p r o p p s i t i o n , a n d H b e i n g d e f i n e d t o be t h e l i m i t of i1a s i g e t s a r b i t r a r i . 1 ~l a r g e .
Hence t h i s c o m p l e t e s t h e p r o o f o f p a r t ( i i i ) of Theorem 1.
We w i l l prove t h e p r o p o s i t i o n by showing t h a t t h e as e s t i m a t e n*(zl) and t h e p o l i c y p = pkz 2: defined above, s a t i s f y t h e saddle-point I n e q u a l i t y
VI.
Proof:
EXISTENCE OF A SOLUTION OF PROBLEM A
We have i n d i c a t e d p r e v i o u s l y i n t h e p a p e r t h a t Problem A o f s e c t i o n 111 c a n be c o n s i d e r e d a s a l i m i t i n g c a s e of Problem B a s i g e t s a r b i t r a r i l y l a r g e . I n s e c t i o n V , we have shown t h e e x i s t e n c e o f a minimax e s t i m a t e *ni(zl) f o r Problem B f o r any f i n i t e i 2 i max' We w i l l o u t l i n e a proof t h a t i n t h e l i m j t a s i g e t s a r b i t r a r i l y l a r g e t h e sequence {*n1(21)]
f o r a l l n ( . ) c i , u(.)eG. To p r o v e t h e r i g h t - h a n d - s i d e i n e q u a l i t y o f ( 3 7 ) we s u b s t i t ~ t efi* = pkz i n t o ? ( l l , u ) and minimize 2 over n ( . ) d . It f o l l o w s from a n argument s i m i l a r t o t h e one g i v e n i n t h e p r o o f of p a r t ( i i i ) of Theorem 1 t h a t t h e m i n i m i z i n g b a y e s e s t i m a t e i s g i v e n by ( 3 6 a ) .
The v a l i d i t y o f t h e l e f t - h a n d - s i d e i n e q u a l i t y of (37) w i l l be e s t a b l i s h e d i n t h e s e q u e l v i a a proof by c o n t r a d i c t i o n : Assume t h a t t h e i n e q u a l i t y d o e s n o t h o l d . Then t h i s i m p l i e s t h a t t h e r e e x i s t s a s c a l a r c > 0 and a p o l i c y such t h a t
z
and L(n*,g) i s Now, s i n c e n* i s l i n e a r i n q u a d r a t i c i n 2 , w i t h o u t a n y t o s s of g e n e r a l i t y t h e p e r m i s s i b l e c l a s s of c o n t r o l s f o r t h e e v a d e r c a n be r e s t r i c t e d t o t h e form
Denote t h e c l a s s of c o n t r o l s of t h e form (38b) 5 y Note t h a t ?,( c and t h a t t h e r i s k %(lll,g) a s s o c i a t e d w i t h &roblem B d e f i n e s a c o n v e r g e n t sequence a s a f u n c t i o n o f i when ni = n* and ueGE This implies t h a t , given an c > 0 , there e x i s t s a n M1 > i such t h a t f o r a l l i > M max 1
GA.
where u ef4 i s t h e c o n t r o l u s e d i n i n e q u a l i t y -O a (38a). We now n o t e t h a t , f o r f i x e d ucQA, n c a n be r e s t r i c t e d t o l i n e a r e s t i m a t e s of t h e form n = Az , which f o l l o w s from a r e a s o n i n g s i m i l a r t o t h a k u s e d p r i o r t o e x p r e s s i o n (38b) i n o r d e r t o r e s t r i c t I! t o ?,( f o r II(-) = n*(z ) . Denote a -1 t h i s c l a s s of l i n e a r e s t i m a t e s by a a' Now, s i n c e ;(n,u) a r e y . ( n , g ) a r e continuous functions of n(.)eaA-for fixkd u(.)c$ a and v i c e v e r s a , and s i n c e t h e r e e x i s t s a t l e a s t one c o n v e r g e n t subp?;, i t f o l l o w s t h a t g i v e n a n sequence {*pi] - C c 0, there e x i s t s an M > i such-that f o r 2 max a l l i > M2
-
Using (38a), ( 3 9 a ) - (39e) we now have
f o r a l l i > M = max(M1 ,M2). But c was a r b i t r a r y . We now p i c k c = 5 and o b s e r v e t h a t i n e q u a l i t y 6 i i ( 3 9 f ) c o n t r a d i c t s t h e a s s u m p t i o n t h a t {*nl(gl),*~ z ] form a s a d d l e - p o i n t s o l u t i o n f o r Problem B; t h i s p r o v e s t h e r i g h t - h a n d - s i d e i n e q u a l i t y of (37). Hence we have shown t h a t (36a) i s a minimax e s t i m a t e f o r Problem A.
ACKNOWLEDGEMENT: The a u t h o r s w i s h t o e x p r e s s t h e i r g r a t i t u d e f o r t h e f o l l o w i n g s u p p o r t . T h i s work was s u p p o r t e d i n p a r t by t h e f o l l o w i n g a g e n c i e s and i n s t i t u t i o n s : By t h e U. S. A i r F o r c e under G r a n t AFOSR-68-1579D, and i n p a r t by t h e J o i n t S e r v i c e s E l e c t r o n i c s Program under C o n t r a c t WB-07-67-C-0199 w i t h t h e C o o r d i n a t e d S c i e n c e L a b o r a t o r y , U n i v e r s i t y of I l l i n o i s , Urbana, I l l i n o i s and by t h e N a t i o n a l S c i e n c e F o u n d a t i o n Grant GK 31511 and t h e U. S O f f i c e of Naval Research under t h e J o i n t S e r v i c e s E l e c t r o n i c s Program by C o n t r a c t N00014-67-A0298-0006 t h r o u g h t h e D i v i s i o n of E n g i n e e r i n g and A p p l i e d P h y s i c s , Harvard U n i v e r s i t y . REFERENCES [I]
T. Basar and M. M i n t z , "Minimax Terminal S t a t e E s t i m a t i o n F o r L i n e a r P l a n t s With Unknown F o r c i n g F u n c t i o n s " I n t e r n a t i o n a l J o u r n a l of C o n t r o l , Vol. 1 6 , No. 1 ( 4 9 - 7 0 ) , 1972.
[2]
T. B a s a r , On a C l a s s of Minimax S t a t e E s t i m a t o r s For L i n e a r Systems With Unknown F o r c i n g F u n c t i o n s , Ph.D. T h e s i s , Department o f E n g i n e e r i n g and A p p l i e d S c i e n c e , Yale U n i v e r s i t y , F e b r u a r y 1972.
[3]
M . M i n t z and T. B a s a r , "On t h e E x i s t e n c e o f L i n e a r Saddle P o i n t S t r a t e g i e s f o r A Minimax Terminal S t a t e E s t i m a t i o n Problem." To be s u b m i t t e d f o r p u b l i c a t i o n .
UNDER UNCERTAINTY
Max Mintz U n i v e r s i t y of I l l i n o i s Urbana, I l l i n o i s
I.
INTRODUCTION
I n t h i s paper we c o n s i d e r a minimax t e r m i n a l s t a t e e s t i m a t i o n problem which can be viewed a s a mathem a t i c a l model f o r a n e s t i m a t i o n problem i n which one i s a t t e m p t i n g t o t r a c k an uncooperative t a r g e t w i t h b a s i c a l l y unknown maneuvering c a p a b i l i t i e s i n a r e l a t i v e l y n o i s y environment. The problem can a l s o be thought of a s a s t o c h a s t i c two-person zerosum game i n which t h e p l a y e r s a r e t h e t r a c k e r and t h e evader. The t r a c k e r wishes t o e s t i m a t e t h e s t a t e of t h e t a r g e t a t t h e end of a predetermined time p e r i o d , by making use of t h e n o i s e c o r r u p t e d r a d a r d a t a . He knows t h e dynamics of t h e t a r g e t , t h e s t a t i s t i c a l d e s c r i p t i o n of t h e i n i t i a l s t a t e of t h e t a r g e t , and t h e s t a t i s t i c s of t h e a d d i t i v e obs e r v a t i o n n o i s e . The evader knows t h e same s t r u c t u r a l i n f o r m a t i o n and a l s o makes n o i s y measurements of t h e s t a t e of t h e t a r g e t - - h i s a d d i t i v e observat i o n n o i s e being s t a t i s t i c a l l y independent of t h a t of t h e t r a c k e r . The evader wants t o maximize t h e e s t i m a t i o n e r r o r of t h e t r a c k e r and a t t h e same time t o minimize t h e q u a d r a t i c c o s t of h i s c o n t r o l . Hence he i s faced w i t h making a compromise between two a l t e r n a t i v e s , which i s r e f l e c t e d through our g e n e r a l i z e d q u a d r a t i c l o s s f u n c t i o n t h a t weights t h e c o s t of t a r g e t c o n t r o l e x p l i c i t l y . We model t h e t a r g e t dynamics by a l i n e a r v e c t o r d i f f e r e n c e e q u a t i o n , a n d t h e i n i t i a l s t a t e of t h e t a r g e t by a gaussian random v e c t o r w i t h z e r o mean and known covariance. This random v e c t o r i s s t a t i s t i c a l l y independent of t h e a d d i t i v e o b s e r v a t i o n n o i s e s of t h e t r a c k e r and t h e evader--which a r e assumed t o be g a u s s i a n w h i t e n o i s e p r o c e s s e s w i t h known s t a t i s t i c s . A s i m i l a r f o r m u l a t i o n h a s been considered i n [ l ] f o r a problem i n which t h e evader does not make any o b s e r v a t i o n s of h i s own s t a t e . That i s , i n [I] t h e p e r m i s s i b l e c o n t r o l p o l i c i e s of t h e evader have been f o r c e d t o be open-loop c o n t r o l p o l i c i e s t h a t do not depend f u n c t i o n a l l y on t h e i n i t i a l s t a t e . A thorough t r e a t m e n t of t h i s problem i s a l s o given i n [2] where s e v e r a l o t h e r r e l a t e d s t o c h a s t i c game problems a r e considered. We have shown i n [ 11 and [2] t h a t f o r t h e c a s e when t h e evader has a c c e s s t o no s t a t e measurements, a minimax e s t i m a t e f o r t h e t e r m i n a l s t a t e of t h e t a r g e t can be r e a l i z e d by a l i n e a r e s t i m a t e which i s bayes with respect t o a gaussian l e a s t favorable prior d i s t r i b u t i o n f o r t h e c o n t r o l f u n c t i o n s of t h e evadex I n t h i s p a p e r , however, we extend t h e s e r e s u l t s i n
Tamer Basar Harvard U n i v e r s i t y Cambridge, Massachusetts
o b t a i n i n g a s o l u t i o n t o a more g e n e r a l problem-t h e minimax e s t i m a t i o n problem i n which t h e evader can a l s o make n o i s y s t a t e measurements. We a d d r e s s o u r s e l v e s , mainly, t o t h e e x i s t e n c e of a minimax t e r m i n a l s t a t e e s t i m a t e f o r t h e problem r a t h e r than t o t h e a c t u a l d e r i v a t i o n and computation of t h e opt i m a l p o l i c i e s of e i t h e r t h e t r a c k e r or t h e e v a d e r . We f i r s t o u t l i n e a proof f o r t h e e x i s t e n c e of l i n e a r minimax e s t i m a t e s f o r a sequence of r e l a t e d e s t i m a t i o n problems and then show t h a t a minimax t e r m i n a l s t a t e e s t i m a t e f o r t h e o r i g i n a l problem can be r e a l i z e d a s t h e l i m i t of a convergent sequence formed from t h e s e l i n e a r e s t i m a t e s . 11. MATHEMATICAL FORMULATION We assume t h a t t h e f o l l o w i n g information i s known by both p a r t i e s ( t h e t r a c k e r and t h e e v a d e r ) : The t a r g e t dynamics a r e d e s c r i b e d by a vector d i f f e r e n c e e q u a t i o n d e f i n e d by where x ( n ) r e p r e s e n t s a s t a t e v e c t o r of known dimens i o n m, u(n) r e p r e s e n t s a c o n t r o l v e c t o r of known dimension r , t h e m a t r i c e s f ( n ) and g ( n ) have t h e dimensions mxm and mxr r e s p e c t i v e l y , a n d n i s a n element of t h e d i s c r e t e time s e t 8 = ( 0 , 1 , . . ,N-11. The i n i t i a l s t a t e x ( 0 ) i s a z e r o mean g a u s s i a n r a n dom v e c t o r w i t h a known covariance m a t r i x denoted by q .
.
The t r a c k e r makes t h e f o l l o w i n g o b s e r v a t i o n s i n t h e extended time s e t 8 = ( 1 , 2 , N};
...,
where z (n) i s a n o b s e r v a t i o n v e c t o r of known dimens i o n m,lh fn) i s a known mxm o b s e r v a t i o n m a t r i x 1 d e f i n e d on 6 , where h (N) i s a nonsingular m a t r i x , v l ( n ) i s a n m v e c i o r z e r o mean gaussian white and n o i s e p r o c e s s w i t h a known covariance m a t r i x d e f i n e d by: c o d vl (n) ,vl ( k ) l = r l (n)+,k'
(3)
where r ( n ) - i s a n mxm p o s i t i v e d e f i n i t e m a t r i x 1 d e f i n e d on 8 . F u r t h e r , t h e normal random v e c t o r s {x(O) , v (n) ;nee] a r e assumed t o be s t a t i s t i c a l l y independenk. The evader a l s o makes n o i s y o b s e r v a t i o n s of t h e s t a t e v e c t o r a s i t e v o l v e s . These o b s e r v a t i o n s a r e
x = "F + E L , -0
d e f i n e d by t h e s e t z 2 (n) = h2 ( n ) x ( n ) + v2 (n) : nee
,
(4)
where x8[xt(1)1 -
where dim(z )=m; v ( n ) r e p r e s e n t s a z e r o mean gaus2 2 s i a n white noise pfocess s t a t i s t i c a l l y independent of { x ( 0 ) , v ( n ) ; nee] , w i t h a known c o v a r i a n c e m a t r i x d e f i n e d by:
where r (n) i s a n rnxm p o s i t i v e d e f i n i t e m a t r i x 2 d e f i n e d on 8 . F u r t h e r , h (n) i s a known mxm o b s e r v a t i o n m a t r i x d e f i n e d on
i.
The e v a d e r h a s complete c o n t r o l o v e r { u ( n ) ; nee] and i s p e r m i t t e d t o choose t h i s c o n t r o l sequence t o be a n y mapping of 8 i n t o E~ which i s m e a s u r a b l e and n o n a n t l c l a t o r w i t h r e s p e c t t o t h e o b s e r v a tionThe n o n a n t i c i p a t i v e p r o p e r t y of t h e c o n t r o l sequence { u ( n ) ] i m p l i e s t h a t f o r a l l n e e , u ( n ) d o e s n o t depend f u n c t i o n a l l y on t h e d a t a s e t {z2(k); ~ n The rp e r m i s s i b l e c o n t r o l s f o r t h e e v a d e r w i l l a l s o have t o be s t a t i s t i c a l l y independ e n t of t h e t r a c k e r ' s o b s e r v a t i o n n o i s e { v l ( n ) ; We d e n o t e t h e c l a s s of c o n t r o l s f o r t h e nee]. e v a d e r , which p o s s e s s e s t h e above p r o p e r t i e s b y % . Given t h i s p r i o r i n f o r m a t i o n , t h e t r a c k e r ' s o b j e c t i v e i s t h e d e t e r m i n a t i o n of a minimax e s t i m a t e f o r t h e t e r m i n a l s t a t e x(N) o f t h e t a r g e t w i t h r e s p e c t t o t h e f o l l o w i n g g e n e r a l i z e d q u a d r a t i c l o s s funct i o n L(6 ( z l ) , u ) :
x -0
uses
The t r a c k e r ' s o b j e c t i v e c a n now be d e f i n e d i n t e r m s of t h e c o n d i t i o n a l r i s k f u n c t i o n r ( 6 , u ) which i s g i v e n f o r f i x e d heA, us% by
4 [x'
...I
x'(N)]',anNmvector,
.. . I
(0)l
-u 4 [ u ' (0)l . . . I
' , an
x ' (o)]
u ' (N-1)]
,
(9b)
Nm v e c t o r ,
(9c)
a n Nr v e c t o r ,
(9d)
a n NrnxNrn m a t r i x d i a g o n a l i n b l o c k s of rnxm m a t r i c e s ; t h e i i ' t h b l o c k b e i n g g i v e n by
-
1 4 [ (fg)il . . .I ( f g ) $ ' , a n
... I
NmxNr m a t r i x ( 9 f )
. . .Ig ;
g(n-1)l~I
(9g)
a n mxNr m a t r i x w i t h N p a r t i t i o n s f o r e a c h n e e . The o b s e r v a t i o n d a t a (2) of t h e t r a c k e r c a n be written as:
z1 =
H15 +
x17
(10a)
where
where F.(z ) d e n o t e s a n a f b i t r a r y mapping o f t h e 1 d a t a sequence { z ( n ) ; nee] i n t o a n m-dimensional v e c t o r t o be a s an estimate f o r x(N)--this mapping may depend o n a l l of t h e p r i o r i n f o r m a t i o n known t o t h e t r a c k e r . We w i l l h e n c e f o r t h d e n o t e t h e c l a s s of s u c h maps by A. C i s a n rnxm non-negat i v e d e f i n i t e w e i g h t i n g m a t r i x , and d ( n ) i s a n r x r p o s i t i v e d e f i n i t e weighting matrix f o r each nee.
(9a)
z b[zt(l)l -1 A = [v;(l)I
x1
...I z ' ( N ) ] ' ,
a n Nmvector,
(lob)
. . . I v;(N)] ' ,
a n Nm v e c t o r ,
(10c)
H I , a n NmxNm m a t r i x d i a g o n a l i n b l o c k s of rnxm m a t r i c e s ; t h e i i ' t h b l o c k b e i n g g i v e n by:
-
mllii
= hl(i).
(10d)
Further define:
-
Q, a n NmxNm m a t r i x h a v i n g i d e n t i c a l b l o c k s of rnxm m a t r i c e s g i v e n by:
-
"Q1 3.
= q, for a l l i , jag,
(lla)
R1; a n NmxNm m a t r i x d i a g o n a l i n b l o c k s of rnxm m a t r i c e s ; t h e i i ' t h b l o c k b e i n g g i v e n by:
[.I
where E denotes the expectation with respect t o t h e s t a t i s t i c s of { x ( O ) , v (n) , v ( k ) ; na6, k e e l . 1 2 D e f i n i t i o n of a minimax e s t i m a t e f o r x(N): I n t h e c o n t e x t of t h e p r e v i o u s n o t a t i o n , a n e s t i mate C " ~ A i s s a i d t o be minimax i f f o r a l l 6 e A , sup r ( r 4 , u ) 5 us%
SUP
r(5 , u ) .
(8
~ € 3
We now g i v e a n a l t e r n a t i v e f o r m u l a t i o n f o r t h i s problem i n t h e n e x t s e c t i o n , which w i l l a i d u s i n p r o v i n g t h e e x i s t e n c e of a minimax e s t i m a t e f o r t h e problem posed i n t h i s s e c t i o n .
t h a t i s , T h a s N p a r t i t i o n s w i t h (N-1) z e r o m a t r i c e s and one mxm i d e n t i t y m a t r i x . C = T'CT, (12b) t h a t i s , C i s a n NmxNm m a t r i x h a v i n g b l o c k s of rnxm m a t r i c e s a l l of which a r e e x c e p t t h e NN'th b l o c k which i s g i v e n by CNN = C. F u r t h e r , d e f i n e D t o be a n NrxNr p o s i t i v e d e f i n i t e m a t r i x which i s d i a g o n a l i n b l o c k s of r x r m a t r i c e s , t h e i i ' t h b l o c k b e i n g g i v e n by: Dii
111. AN ALTERNATIVE FORMULATION We now o b s e r v e t h a t t h e s e t of l i n e a r d i f f e r e n c e e g n s ( 1 ) c a n a l s o be w r i t t e n a s :
= d(i-1).
(13)
Using t h e p r e c e d i n g n o t a t i o n , we now pose a minimax e s t i m a t i o n problem which w i l l h e n c e f o r t h be r e f e r r e d t o a s Problem A i n t h e p a p e r .
i i i z 2 ( n ) = H (n)xo
Problem A F i n d a minimax e s t i m a t e n*(zl) f o r t h e v e c t o r 2 d e f i n e d by ( 9 a ) and u s i n g t h e o b s e r v a t i o n given by (10a),-under t h e g e n e r a l i z e d q u a d r a t i c l o s s f u n c t i o n L(ll ( z l ) ,E) d e f i n e d by:
zl
where C a n d D a r e a s d e f i n e d by (12b) and (13) The v e c t o r E i s a l l o w e d t o depend respectively. on t h e o b s e r v a t i o n s {z ( n ) ; nee] d e f i n e d by (4) i n 2 a nonanticipatory fashlon i n the sense t h a t the f i r s t n r e l e m e n t s o f 2 d o n o t depend f u n c t i o n a l l y 3 x f o r a l l nee. T h i s on t h e d a t a s e t { z 2 ( k ) ; 1 i s i n a c c o r d a n c e w l t h t h e d e f i n i t i o n of t h e p e r m i s s i b l e c l a s s of c o n t r o l s q f o r t h e e v a d e r g i v e g i n ' with Q ' , and s e c t i o n 11. For Problem A we r e p l a c e & t o stand f o r t h e perf u r t h e r we r e p l a c e A w i t h m i s s i b l e c l a s s of e s t i m a t e s f o r t h e v e c t o r 2 . Denoting t h e c o n d i t i o n a l r i s k f u n c t i o n a s s o c i a t e d w i t h t h e l o s s f u n c t i g n d e f i n e d by (14) by Y((n,g), we n o t e t h a t n*(z )eA i s a minimax e s t i m a t e f o r -1 x if -
+
i W ( n ) , nee,
(19a)
where z l ( n ) i s a n Nm-dimensional v e c t o r , H1(n) i s a n Nmxd m a t r i x d i a g o n a l i n b l o c k s of mxm m a t r i c e s ; t h e j j ' t h b l o c k b e i n g g i v e n by n-1
f o r a l l nee. i W ( n ) r e p r e s e n t s a z e r o mean g a u s s i a n w h i t e n o i s e process s t a t i s t i c a l l y independent of {xi,v1], with a c o v a r i a n c e m a t r i x d e f i n e d by:
a
sup
"rn*
-uc'l%
,g) Isup ue'ci
where R(n) i s a n NmxNm p o s i t i v e d e f i n i t e b l o c k d i a g o n a l m a t r i x , t h e j j ' t h b l o c k b e i n g g i v e n by
"rn ,u).
for a l l n(.)ei. We now o b s e r v e t h e f o l l o w i n g p r o p e r t y r e l a t i n g t o t h e s o l u t i o n s of t h e problem of s e c t i o n I1 a n d Problem A: Property 1
,
j = 1
,
otherwise
Denoting by T~ t h e ( p r o j e c t i o n ) f o r a l l nee. t r u n c a t i o n map t h a t r e s t r i c t s a n y Nm d i m e n s i o n a l v e c t o r t o i t s f i r s t m e l e m e n t s , we f u r t h e r r e q u i r e wl(n) t o p o s s e s s t h e p r o p e r t y
where v ( n ) was d e f i n e d by ( 5 ) . 2
The e x i s t e n c e o f a minimax e s t i m a t e n*(iz,)eZ t o Problem A i m p l i e s t h e e x i s t e n c e o f a minimax e s t i mate 6*(z ) € A t o t h e problem posed i n s e c t i o n 11. 1 F u r t h e r m o r e , t h e s e two e s t i m a t e s have t h e s i m p l e relation
T h i s , t o g e t h e r w i t h t h e d e f i n i t i o n of Problem A i n s e c t i o n 111, c o n c l u d e s t h e f o r m u l a t i o n o f Problem B. w i t h t h e e x c e p t i o n of t h e i n t e g e r i which w i l l be d e f i n e d a s f o l l o w s :
where T i s d e f i n e d by ( 1 2 a ) .
( i ) Denote by i t h e s m a l l e s t p o s i t i v e i n t e g e r s u c h t h a t for a l ? integers i > i n , the matrix
Hence, i n t h e r e m a i n i n g p a r t s of t h e p a p e r we w i l l b e d e a l i n g m a i n l y w i t h Problem A and p r o v e t h e e x i s t e n c e of a minimax e s t i m a t e n*(zl) a s a l i m i t i n g c a s e of t h e s o l u t i o n of a n o t h e r e s t i m a t i o n problem (Problem B) which i s posed i n t h e n e x t section. IV.
A RELATED MINIMAX ESTlMATION PROBLEM-PROBLEM B
The f o r m u l a t i o n of Problem B i s s i m i l a r t o t h a t o f Problem A w i t h t h e f o l l o w i n g m o d i f i c a t i o n s : (1) Replace
byF1
which i s d e f i n e d by
1 = F + - I (17) i Nm' where i i s a p o s i t i v e i n t e g e r t o be d e f i n e d i n t h e s e q u e l by (ZO), and IN, s t a n d s f o r t h e Nm-dimensional identity matrix.
F
i
(2) Replace x v e c t o r x i su$
by a n o t h e r zero-mean g a u s s i a n Nm that i i' i 1 E [ x ~ x Q ~ ~= Q + y I N m . (18)
4
( 3 ) R e p l a c e t h e o b s e r v a t i o n z ( n ) , d e f i n e d by ( 4 ) 2 by
[
n- 1 f(k) k= 0
n
1
+ TIm]i s n o n s i n g u l a r .
( i i ) Denote by in t h e s m a l l e s t p o s i t i v e i n t e g e r s u c h t h a t for a l l i n t e g e r s i > in, the matrix [h2(n)
n- 1 T! f ( k ) k= 0
(iii) Let i max
+
1
i s nonsingular.
4 - arg
max { i n , i n } . ne6
Then p i c k i n g i 2 imax c o n c l u d e s t h e f o r m u l a t i o n of Problem B . We p a r e n t h e t i c a l l y n o t e t h a t f o r a l l f i n i t e i 2 i m a x , F1 a n d H1 (?) a r e n o n s i n g u l a r f o r a l l n e 6 , and f u r t h e r t h a t Q~ i s p o s i t i v e d e f i n i t e f o r a l l f i n i t e i. T h i s p a r t i c u l a r c h o i c e of i h a s been made i n o r d e r t o i n s u r e t h e e x i s t e n c e of a s o l u t i o n t o Problem B, which c a n t h e n be g e n e r a l i z e d t o p r o v e t h e e x i s t e n c e of a minimax e s t i m a t e t o Problem A . Before g o i n g i n t o a d i s c u s s i o n of t h e s o l u t i o n o f Problem B we want t o remark on t h e a p p a r e n t s i m i l a r i t y between t h e f o r m u l a t i o n s of Problems A and B , i n t h e l i m i t a $ i g e t s a r b i t r a r i l y l a r g e . The two r e l a t i o n s F'IF and ~ l 1 h r oe b v i o u s from ( 1 7 ) a n d . (18) r e s p e c t i v e l y . However, t h e f a c t t h a t z $ ( n ) d e f i n e d
by (19a) p r o v i d e s t h e same i n f o r m a t i o n , i n t h e l i m i t , t o t h e maximizing p a r t y ( e v a d e r ) a s z (n) d e f i n e d by (4) d o e s , f o l l o w s from t h e n o n a n t g c i p a t i v e property of the c o n t r o l functions of t h e maximizing p a r t y . The r e a s o n why we have i n t r o duced a d d i t i o n a l a d d i t i v e n o i s e i n r e l a t i o n (19a) i s a g a i n t o i n s u r e t h e e x i s t e n c e of a s o l u t i o n t o Problem B. I n summary, Problem B becomes i d e n t i c a l t o Problem A i n the l i m i t a s i gets a r b i t r a r i l y large. I n what f o l l o w s we w i l l f i r s t g i v e t h e s o l u t i o n t o Problem B f o r e a c h f i n i t e i and t h e n c o n s t r u c t a l i m i t i n g scheme t h a t w i l l i n s u r e t h e e x i s t e n c e o f a minimax e s t i m a t e f o r Problem A. V.
SOLUTION TO PROBLEM B
The s o l u t i o n t o Problem B i s p r o v i d e d i n Theorem 1 below. P r e l i m i n a r y N o t a t i o n : - - D e f i n i t i o n of m a t r i c e s which a p p e a r i n Theorem 1. H I ; a n N2mxN2m m a t r i x , diagonal i n blocks of N G M m a~t r i c e s , t h e j j ' t h b l o c k b e i n g g i v e n by
f o r any given A' with t h e property r(Ai)
< 0.
(27b)
The s o l u t i o n t o t h i s m a x i m i z a t i o n problem w i l l , i n g e n e r a l , depend on t h e c h o i c e of A1. We emphasize t h i s dependence by w r i t i n g as pi = @ i ( ~ ~ )
(27~)
where Oi i s a m a t r i x v a l u e d f u n c t i o n o f a p p r o p r i a t e dimension d e t e r m i n e d t h r o u g h t h e p r e c e d i n g maximizat i o n problem. We now s e t
A'
. iH;[ A
i-iSii;+ ill
(28a)
i where A depends on pi t h r o u g h (24) w i t h . P r e p l a c e d We i n d i c a t e t h i s dependence of A' on pi by by pi. i i i A = cp ( P ),
(28b)
where mi c a n r e a d i l y be d e t e r m i n e d t h r o u g h (24) and (28a). We now s e e k a s o l u t i o n t o
-
JJ
2 2 where ~ ~ ( ~ i-s 1d e) f i n e d by (19b). R; a n N mxN m p o s i t i v e d e f i n i t e block diagonal matrix, t h e j j ' t h b l o c k b e i n g g i v e n by
[iljj
= ~(j-11,
P. = 0 for j < k. (22) ~k We d e n o t e t h e s p a c e o f a l l s u c h m a t r i c e s P b y R and n o t e t h a t e a c h e l e m e n t o f 0 i s u n i q u e l y deFerP ! l k ! e! nh t r i e s , i . e . , np i s mined by s p e c i f y i n g & 2
-
dimensional euclidean isomorphic t o the 2 space. A~ ; a n ~ m x ~ rmna t r i x (23)
P
wJ7i(~i>)<
0
Theorem 1
(
i ) For e a c h f i n i t e i n t e g e r i 2 i e i s t s a t l e a s t one m a t r i x - v a ~ r$ (.) s u c h t h a t pi g i v e n by (27c) J ~ ( P )o v e r R f o r e a c h Ai s u b j e c t P
H
( i i ) For e a c h f i n i t e i n t e g e r i 2 i a t l e a s t one m a t r i x why:$ (29a) and (29b).
there ~f ui n c t i o n maximizes t o (27b). there e x i s t s satisfies
( i i i ) A minimax e s t i m a t e *nl(z ) f o r Problem B i s given f o r each f i n i t e inkeger i 2 i max by
'i ; a n NmxNm m a t r i x d e f i n e d f o r by
where.*Ai i s g j v e n by (28a) w i t h h i r e p l a c e d by *A', and *A' i s d e f i n e d by (24a) w!th P r e p l a c e d by *P: o r e q u i v a l e n t l y *A1 i s g i v e n by a i ( * p l ) .
--
i
r(A ) ; a n NrxNr m a t r i x g i v e n by
i J (P) i s a s c a l a r f u n c t i o n of P a p d e f i n e d f o r e a c h g i v e n A' by
where
subject t o the condition t h a t
(21b)
2 where R ( j - 1 ) i s d e f i n e d by (19d). P; a n NrxN m m a t r i x , lower t r i a n g u l a r i n b l o c k s o f rxNm m a t r i c e s , i . e . , t h e j k ' t h b l o c k i s g i v e n by
e a c h Pd?
i i i p i = @ (a ( P I ) ,
~ r C(H i e H i ' + z ) ~ ' ~ ~ ] ,
depends on P t h r o u g h (24).
denote any s o l u t i o n t o Now, l e t t h e m a t r i x pi& t h e f o l l o w i n g m a x i m i z a t i g n problem f o r a n y f i n i t e i n t e g e r i 2 imax'. max J ~ ( P ) , (27a)
PC
P
A proof o f p a r t s ( i ) a n d ( i i ) of Theorem 1 w i l l n o t be g i v e n h e r e b e c a u s e o f i t s l e n g t h . It c a n be found i n o u r r e l a t e d l o n g e r p a p e r [3] . A proof of p a r t ( i i i ) o f Theorem 1 w i l l be p r o v i d e d h e r e by showing t h a t *ni(El) and ui=*pizi s a t i s f y the -2 f o l l o w s n g saddl_e;point i n e q u a l i t y ( 3 1 ) f o r a l l n l ( . ) e ~ ~z(.)eQ1 , and f i x e d i 2 imax:
-
I n the preceding i n e q u a l i t y , r i ( - , * ) denotes the c o n d i t i o n a l r i s k f u n c t i o n ass_ociated-with Problem B and f o r a n y f i x e d i; f u r t h e r A1 and ?A1 d e n o t e t h e p e r m i s s i b l e c l a s s of s t r a t e g i e s f o r Problem-B d e f i n e d i n a way s i m i l a r t o t h o s e of A and Q r e s p e c t i v e l y f o r Problem A. The v e c t o r z1 i s ~ ~ m -2 d i m e n s i o n a l and i s d e f i n e d by
-
where z i ( n ) i s g i v e n by (19a).
p r o v i d e s a minimax e s t i m a t e f o r Problem A and hence c o m p l e t e t h e proof of t h e e x i s t e n c e o f a s o l u t i o n t o Problem A.
We n o t e p a r e n t h e t i c a l l y t h a t i n e q u a l i t y (31) implies t h a t
For t h i s p u r p o s e e d e f i n e a m e t r i c ~ ( A , B ) :n -Ey by P P 2 Nr N m 2 $ P (A,B) = C C Z (A - B ~ I~ ) k j k=l j=l
and hence *lli
Using e x p r e s s i o n s . ( 2 6 ) and ( 2 8 a ) , i t c a n be shown t h a t [ s e e [3]] *pl d e f i n e d i n p a r t ( i i ) of Theorem 1 i s bounded i n norm by a p o s i t i v e number 7 g i v e n by
i s minimax.
Proof o f p a r t ( i i i ) o f Theorem 1:
-
BY s u b s t i t u t i n g 2i *piz$ i n t o t h e c o n d i t i o n a l r i s k f u n c t i o n Zi(ni,u) a s s o c i a t e d w i t h Problem B we o b t a i n :
*
( [ . I m . , d e n o t e s t h e minimum e i g e n v a l u e o f [.I>?.) , where t h e norm i s induced by t h e for a t 1 i 2 i m e t r i c (35a).ma3e n o t e t h a t 7 i s i n d e p e n d e n t o f i a n d hence *pidl for a l l i 2 i where R i s 7 7 max' d e f i n e d by
where
It f o l l o w s from (33b) t h a t yi i s a normql random v e c t o r w i t h mean z e r o and c o v a r i a n c e *A' which i s But, a? g i v e n by (24) w i t h P r e p l a c e d by *P1. e s t i m a t e t h a t m i n i m i z e s < 3 3 a ) , when y i e ~ @ , * ~ l ) , i s t h e b a y e s e s t i m a t e * l l l ( ~ ~g)i v e n by (30). T h i s completes t h e proof of t h e right-hand-side i n e q u a l i t y o f (31).
where
11 -11
i s t h e norm i n d u c e d by t h e m e t r i c ( 3 5 a ) .
We now o b s e r v e t h a t R i s a c l o s e d and bounded subs e t of Rp , and hence P h e m e t r i c s p a c e (& , p ) i s e l e m e n t of R p , t h e
To p r o v e t h e l e f t - h a n d - s i d e i n e q u a l i t y of ( 3 1 ) , we s u b s t i t u t e *ni(gl) g i v e n by. (30) i n t o t h e c o n d i t i o n a l r i s k f u n c t i o n Ti(nl,u): Proposition 1 T h e r e e x i s t s a l i n e a r minimax e s t i m a t e f o r Problem A, which i s g i v e n by
where A* i s g i v e n by
where t h e e x p e c t a t i o n . o p e r a t i o n i g t a k e n o v e r t h e p r i o r s t a t i s t i c s o f x i , v and {w1(n);ne8]. gut s i n c e r(*A1) < 2, t h e maxkmum o f (34) o v e r ueQ1 e x i s t s and, without any l o s s of g e n e r a l i t y , t h i s maximum may be s o u g h t o v e r t h e s p a c e of l i n e a r policies. Hence, maximizing (34) o v e r Gi. i s e q u i v a l e n t t o maximizing (34) w i t h 2 = PZ' o v e r -2 Rp. But t h e maximum i s a c h i e v e d by *pi a s g i v e n by p a r t s ( i ) and ( i i ) of Theorem 1.
w i t h pk b e i n g computed t h r o u g h t h e l i m i t i n g procedure o u t l i n e s i n t h i s s e c t i o n p r i o r t o the p r o p p s i t i o n , a n d H b e i n g d e f i n e d t o be t h e l i m i t of i1a s i g e t s a r b i t r a r i . 1 ~l a r g e .
Hence t h i s c o m p l e t e s t h e p r o o f o f p a r t ( i i i ) of Theorem 1.
We w i l l prove t h e p r o p o s i t i o n by showing t h a t t h e as e s t i m a t e n*(zl) and t h e p o l i c y p = pkz 2: defined above, s a t i s f y t h e saddle-point I n e q u a l i t y
VI.
Proof:
EXISTENCE OF A SOLUTION OF PROBLEM A
We have i n d i c a t e d p r e v i o u s l y i n t h e p a p e r t h a t Problem A o f s e c t i o n 111 c a n be c o n s i d e r e d a s a l i m i t i n g c a s e of Problem B a s i g e t s a r b i t r a r i l y l a r g e . I n s e c t i o n V , we have shown t h e e x i s t e n c e o f a minimax e s t i m a t e *ni(zl) f o r Problem B f o r any f i n i t e i 2 i max' We w i l l o u t l i n e a proof t h a t i n t h e l i m j t a s i g e t s a r b i t r a r i l y l a r g e t h e sequence {*n1(21)]
f o r a l l n ( . ) c i , u(.)eG. To p r o v e t h e r i g h t - h a n d - s i d e i n e q u a l i t y o f ( 3 7 ) we s u b s t i t ~ t efi* = pkz i n t o ? ( l l , u ) and minimize 2 over n ( . ) d . It f o l l o w s from a n argument s i m i l a r t o t h e one g i v e n i n t h e p r o o f of p a r t ( i i i ) of Theorem 1 t h a t t h e m i n i m i z i n g b a y e s e s t i m a t e i s g i v e n by ( 3 6 a ) .
The v a l i d i t y o f t h e l e f t - h a n d - s i d e i n e q u a l i t y of (37) w i l l be e s t a b l i s h e d i n t h e s e q u e l v i a a proof by c o n t r a d i c t i o n : Assume t h a t t h e i n e q u a l i t y d o e s n o t h o l d . Then t h i s i m p l i e s t h a t t h e r e e x i s t s a s c a l a r c > 0 and a p o l i c y such t h a t
z
and L(n*,g) i s Now, s i n c e n* i s l i n e a r i n q u a d r a t i c i n 2 , w i t h o u t a n y t o s s of g e n e r a l i t y t h e p e r m i s s i b l e c l a s s of c o n t r o l s f o r t h e e v a d e r c a n be r e s t r i c t e d t o t h e form
Denote t h e c l a s s of c o n t r o l s of t h e form (38b) 5 y Note t h a t ?,( c and t h a t t h e r i s k %(lll,g) a s s o c i a t e d w i t h &roblem B d e f i n e s a c o n v e r g e n t sequence a s a f u n c t i o n o f i when ni = n* and ueGE This implies t h a t , given an c > 0 , there e x i s t s a n M1 > i such t h a t f o r a l l i > M max 1
GA.
where u ef4 i s t h e c o n t r o l u s e d i n i n e q u a l i t y -O a (38a). We now n o t e t h a t , f o r f i x e d ucQA, n c a n be r e s t r i c t e d t o l i n e a r e s t i m a t e s of t h e form n = Az , which f o l l o w s from a r e a s o n i n g s i m i l a r t o t h a k u s e d p r i o r t o e x p r e s s i o n (38b) i n o r d e r t o r e s t r i c t I! t o ?,( f o r II(-) = n*(z ) . Denote a -1 t h i s c l a s s of l i n e a r e s t i m a t e s by a a' Now, s i n c e ;(n,u) a r e y . ( n , g ) a r e continuous functions of n(.)eaA-for fixkd u(.)c$ a and v i c e v e r s a , and s i n c e t h e r e e x i s t s a t l e a s t one c o n v e r g e n t subp?;, i t f o l l o w s t h a t g i v e n a n sequence {*pi] - C c 0, there e x i s t s an M > i such-that f o r 2 max a l l i > M2
-
Using (38a), ( 3 9 a ) - (39e) we now have
f o r a l l i > M = max(M1 ,M2). But c was a r b i t r a r y . We now p i c k c = 5 and o b s e r v e t h a t i n e q u a l i t y 6 i i ( 3 9 f ) c o n t r a d i c t s t h e a s s u m p t i o n t h a t {*nl(gl),*~ z ] form a s a d d l e - p o i n t s o l u t i o n f o r Problem B; t h i s p r o v e s t h e r i g h t - h a n d - s i d e i n e q u a l i t y of (37). Hence we have shown t h a t (36a) i s a minimax e s t i m a t e f o r Problem A.
ACKNOWLEDGEMENT: The a u t h o r s w i s h t o e x p r e s s t h e i r g r a t i t u d e f o r t h e f o l l o w i n g s u p p o r t . T h i s work was s u p p o r t e d i n p a r t by t h e f o l l o w i n g a g e n c i e s and i n s t i t u t i o n s : By t h e U. S. A i r F o r c e under G r a n t AFOSR-68-1579D, and i n p a r t by t h e J o i n t S e r v i c e s E l e c t r o n i c s Program under C o n t r a c t WB-07-67-C-0199 w i t h t h e C o o r d i n a t e d S c i e n c e L a b o r a t o r y , U n i v e r s i t y of I l l i n o i s , Urbana, I l l i n o i s and by t h e N a t i o n a l S c i e n c e F o u n d a t i o n Grant GK 31511 and t h e U. S O f f i c e of Naval Research under t h e J o i n t S e r v i c e s E l e c t r o n i c s Program by C o n t r a c t N00014-67-A0298-0006 t h r o u g h t h e D i v i s i o n of E n g i n e e r i n g and A p p l i e d P h y s i c s , Harvard U n i v e r s i t y . REFERENCES [I]
T. Basar and M. M i n t z , "Minimax Terminal S t a t e E s t i m a t i o n F o r L i n e a r P l a n t s With Unknown F o r c i n g F u n c t i o n s " I n t e r n a t i o n a l J o u r n a l of C o n t r o l , Vol. 1 6 , No. 1 ( 4 9 - 7 0 ) , 1972.
[2]
T. B a s a r , On a C l a s s of Minimax S t a t e E s t i m a t o r s For L i n e a r Systems With Unknown F o r c i n g F u n c t i o n s , Ph.D. T h e s i s , Department o f E n g i n e e r i n g and A p p l i e d S c i e n c e , Yale U n i v e r s i t y , F e b r u a r y 1972.
[3]
M . M i n t z and T. B a s a r , "On t h e E x i s t e n c e o f L i n e a r Saddle P o i n t S t r a t e g i e s f o r A Minimax Terminal S t a t e E s t i m a t i o n Problem." To be s u b m i t t e d f o r p u b l i c a t i o n .