Invariant system description of the stochastic realization

0005 1098 79 0901-049~ ~,2 00/0

4utomatl¢a Vol 15 pp 4 9 3 ~ 9 5 Pergamon Press Lid 1979 Printed in Great Britain © International Federation of Automatic Control

Technical Commumque Invariant System Description of the Stochastic Realization*t J V CANDY:~, T E BULLOCK§ and M E WARREN§ Key Wnrd Index--Realization, stochastic reahzatlon, canonical forms, identification, stochastic systems, systems theory Abstract--In this article it is shown that the class of all reahzatlons possessing the same power spectral density can be uniquely characterized by giving an lnvarlant system description A new transformation group Is introduced and shown to leave the spectral density unchanged The action of th~s group must be considered when attempting to specify a stochastic realization from spectral densmes or equivalently covarlance sequences 1 Introduction THE PROBLEM considered in this communique is the determination of a unique model for the power spectral density (PSD) or equivalently the measurement covarlance sequence of a constant linear system driven by white noise An lnvarlant system description under suitable transformation groups for this model is specified It is shown that the general linear group used in the deterministic case is not sufficient to describe this model uniquely so that more than a direct extension of realization theory results previously developed (Denham, 1974, Candy, 1978) Is required In this section the problem is defined and some of the necessary background results are presented The algebraic structure of a group acting on a set to obtain lnvarlant system descriptions is given in the following section It is shown that this description can be used to uniquely specify a stochastic realization The following notation is used throughout GL(n) denotes the general linear group R IS a field, A > 0 denotes a positive definite matrix A, dim( ) is the dimension of , 2(A) is the set elgenvalues of A Recall that the stochastic reahzatlon problem asks that a linear system driven by white noise be found such that the covarlance of the measurement sequence matches a given covarlance sequence for a stationary process The given infinite sequences of p x p covarlance matrices Cj will be ~" denoted by ~Cj~_~ with C_j=CJ for a real stationary process An alternate description is the power spectral density q~z(Z) (PSD Matrix) obtained by taking the z-transform of the covarlance sequence We assume that ~Cj} has been generated by the white-noise model given by usual equations Xt, + I = F x k q- W,~

z~=Hxk+v ~

(1)

where x~,wk~R n, zk, vkffR v, and wk and vk are zero-mean white Gausslan noises with covarlances Q and R respectively and cross-covarlance S Typical constraints for the stochastic reahzatlon problem can be summarized by defining the set

~' = [(F, H, Q, R, S)IF stable and nonsmgular (F,H) observable,

A>01

where

Analysis of the white noise model shows that the stationary state covarlance FI satisfies a Lyapunov equation (2)

11 = FFIFT + Q

F r o m this model it IS easily shown that Cj = Cov(za +j, zk) = HF IJI- l (FFIH T + S) y ~ 0,

/3)

Co = Cov(zk, zk) = H H H T + R

The PSD matrix ~b'z(Z) is obtained by taking the bilateral ztransform of (3) ~b'z(z) = H (Iz - F)- 1Q ( l z - t _ F r)- t H T + H (Iz - F)- 1S (4)

+STIz I - - F T ) - t H T + R

Therefore if (F, H, Q, R, S) is a stochastic realization for the given sequences, we must have q~.(z) equal to ~b'z(Z)in (4) The dellnltlOn ot 5 ~ requires that the Q, R, S matrices satisfy the covartance condition, Cov[(~)(wrvX)]=A6, , e ' A >- - 0 If t J J J' this were not necessary, it would be an easy matter to specify Q, R, and S Here we study the algebraic problem of speclfymg an lnvarlant system description for the stochastic realization Such a development is not possible without relaxing the constraints on A 2 lnvarlant s)stem descmptlon of the stochastic reahzatlon In this section a characterlzauon of the class of all realizations possessing the same PSD is developed by approaching the problem from a purely algebraic viewpoint We define a set of quintuplets more general than the set 5~ and consider only those transformation groups acting on the set which leave the qS(z) or equivalently {Cj] unaltered We then specify mvarlant system descriptions under these groups which prove useful in specifying a stochastic reahzatlon algorithm (Candy, 1977) The groups employed were originally introduced by Popov (1973) O u r results also parallel some results obtained in the quadratic optimization problem (Wtllems, 1971) Define the set X1 = [ ( F , H , Q , R , S ) I FER"×",H~RV×",Q~R"X", RERp~p,

*Received 18 January 1977, revised 31 August 1977, revised 7 November 1978, revised 17 January 1979 The original version of this article was not presented at any IFAC Meeting This paper was recommended for publication in revised form by associate editor B D O Anderson 1"Partial support was provided by U S Department of Energy by the Lawrence Llvermore Laboratory under contract # W-7405-ENG-48 :~Unlverslty of California, Lawrence Llvermore Laboratory, P O Box 5504, L-156, Llvermore, CA 94550, U S A §Electronics Engineering Department, University of Florida, Galnesvllle FL 32601, U S A 493

S~R "~,

IFl~0,I~(F)l
and consider the following transformation group specified by the set G R . = ~(T,L)ITeGL(n), L 6 R "×", L symmetric} and the group operation (~ L)°(T,L)=(7"T,L + T - t L T -

T)

Let the action of GR. on XI be defined by 'J.' and

494

l'echmcal Commumque (T,L)+(I. H Q,R,S) =(TI~ F - I , t t T F ( Q + F L F X - L ) T t , R + H L H ~ 71S+ILHT))

(~,)

An element (F,H,Q,R,S) ol the set gl is said to be eqm~alent to the element (F,H,Q,R,S) of ~t~ if there exists a (T,L)eGR. such that (F,H Q,R S)=(T,L)J,(F,H,Q,R,S) fhls relaUon is reflexive, symmetric and transitive, therefore, GR. induces an equivalence relation on ¥i which we denote by ETa and (5) defines the partitioning of \ t into classes The following lemma shows that ErL-eqmvalent quintuplets possess properties which are necessary from the realization viewpoint, ~e, they have ~dent~cal PSD's Lemmu 1 Let (P,f/,~,/~,~), (F,H,Q,R,S)~)t~ with completely observable pairs (/7',/~) (F H) and respective PSD's q~,lz), and ~b'z(z) Then ~z(z~=~,(z) ,ff ( f /1 0 /~ S)

Et~(F,H Q R 51 ProoJ Necessity follows d~rectly from Proposltmn (2 6 3) ol Popov (1973) Sufficiency follows by direct calculation [ ] The measurement covanance sequence ~s mvarmnt under the actmn of GR. on Yt because the PSD is also gwen by q ~ z ( z ) = ~ ] - _ . C~z ~ We will call any two systems represented by the quintuplet of X t tovar~ame equwalent, ff they are E~-equlvalent Note that since X~ ~ '/ an element of Xt may not correspond to a system w~th real noise covarmnces due to the fact that A is not posmve semldefinlte In order to uniquely characterize the class of covarlance eqmvalent quintuplets we must determine an lnvarlant system description for X~ under the actmn of GR. The number of lnvarmnts may be found by counting the parameters If we define, M~ =dlm(F,H,Q,R S) and ME =&m(T,L), then there are

(k,H,Q R S)E+L(F H Q R, 5) then there exists lhre,, unique matrices L~ L 2 and La such that Q=Q+kLtFr-LI Q=Q+bI 2FI-12 Q=Q+kL~br-L3 From these equations, O = F ( L t + L s - L 2 ) F ~ - I L t + L + - L 2 1 and hence L t + L 3 = I 2 Similarly, from E~L-equi~alence it Is easdy shown that/~1 =/~2 and $1 = S z [] Further canonical forms may be generated by specffy,ng other elements m (Q,R,S) however one must be careful to insure that these elements are independent An interesting second choice ts S = 0 e ~ . described m the following lemma Lemma 2 If 0e,/4, Q, R S) is a canomcal form for A 1 under the action of GR. with S = 0 v ~ . , then 0 and /~ together contain (n + 1 )p mvarlants ProoJ The first column of LH 1 speohes a ro~ ot L which also specifies elements of each of the other rows ot L by symmetry Proceeding in this fashion, the second column ot H T speofies n - 1 elements in another row of L and continuing ~e ha~e n+(n-1)~-

+(n-p+l)=np-½p
elements of L The remaining 1/2n(n + l ) - t l p + 1/2p(p- 1) elements m L determine the same number of elements in 0 a n d / ~ The remaining

1/2n(n + 1 ) + 1~2pip + 1 ) - 1/2n(n + 1 ) + n p -

1 / 2 p ( p - 1) = ( n + 1)p

M+ =n 2 +np+ 1/2n(n+ 1)+ 1/2p(p+ l)+np parameters specifying this quintuplet and GR. acts on M2 = n 2 + I/2n(n + 1) of them, thus, there exist M~ - M z = 2np + l / 2 p ( p + l ) mvarlants If we conmder any transformatmns (T,0.)e GR. such that the pmr (F,H) is umquely speofied by np lnvarlants,* then we need only conmder the remaining action of L on Q,R and S Henceforth, we assume that (F,H) has been transformed to this form Recall that after the action of T has been performed, L acts on Q R ~; as

Q=Q+FLF i -L

(6)

1~= R + HLH ~

(7)

= S + FLH T

(8)

The transformatmn L acts on 1/2n(n+ 1) parameters of the total n p + l / 2 n ( n + l ) + l / 2 p ( p + l ) parameters available in Q , R , S Once this actmn ~s completed, the remaining parameters are m v a n a n t s There are several ways one might choose to specify these free parameters For example, an arbttrary choice of Q (symmetric) fixes L and hence the remaining parameters One m a y make other arbitrary choices of parameters (Q,R,S) to determine L. but only 1/2n(n+l) parameters are free to be specified Using these concepts, ~t is posmble to specify a canomcal form for ErL-equivalence on X~ Theorem 1 Any symmetric n x n matrix 0 defines a set ol canonical forms (F, H, (), /~, ~) for E~L-equl~alence on X~ Proof Since F ~s a nonsmgular stabd~ty matrix, (6) has a u m q u e solutmn for L for arbitrary symmetric Q=(~ and Q (see Gantmacher, 1959) Therefore R = / ~ and g = S are uniquely specified through (7) and (8) Furthermore since F and H specify a canomcal pmr under GL(n) it is sufficient to consider the (Q,R,S) matrices only Hence, any element ( F , H , Q , R , S ) e X l is ErL-equlvalent to a unique element (F. H, (~,/~, oe) e X 1 It is easy to show by direct calculation that ff (F,H,Q,R,S)ErL(F,H,Q.,II.g), then (F,H,Q,R,S) and (F, H, I~, R, $) have the same canomcal form For ff (~ defines a canomcal form for (F,H,Q,R,S) and (F,H,Q R,N) with *For example, if T = TR, then (F, H)~(FR, H~), where the R subscript represents for Luenberger row coordinates which is a canonical form (see Popov. 1972, for details)

elements m Q and /~ not determined by a speoficatlon of L are l n v a n a n t s EI If we let all of the 1/2p(p+ l) elements of R be mvarlants, then () has (n + 1 ) p - 1/2p(p +1} = n p - 1/2p(p- I) m v a n a n t s as noted by Luo and Bullock, 1975, Majumdar, 1975 In the context of system identification and modeling this means that tf R is completely identified from covarlance data, only np 1/2p(p- 1) parameters of Q may be identified and schemes which use a larger parameterlzatlon are likely to run into trouble In addmon, the parameters chosen must constitute a complete set of lnvarlants If one considers S specified+ another constraint is important From (8) it can be shown that H F - I ( S - S ) = H L H T and therefore HF I(e,-S) is s~mmetric which constrains S since then -

HF I($--S)--(S-S)T(FT)-IHT=O

(9)

Although further posmble specml cases may be of some interest, we now turn to an example which illustrates some of the eqmvalent elements of X 1 Note that the odd numbers occur in order to obtain rumple transformations Example Suppose we are g~ven the stochast;c reahzatton (I',H,Q,R,S) as

+__i I ":Io I._ 4

24

Q=

3]

121

i°oj f°iJ 1

0

0

3667 28

24

with (F,H) m Luenberger row form (Popov, 19721

Technical Commumque

as the solution of a Rlccatl equation as in Denham (1975). Tse (1975), Faurre (1976), Gevers (1973), Kalman (1974) These results lead to minimal rank spectral factors, however, other non-minimal spectral factors are possible as pointed out in Candy (1977) The tradeoff here of course is the computational burden of the RIccati equation

(i) First, select a Q as

O=

2

0

0

1

13

1

24

24 1 12

3 Conclusions This communique has shown that when considering the class of stochastic realizations which possess the same PSD's, the transformation group required is richer than the general linear group of the corresponding deterministic problem An lnvanant system description under this group was specified and it was sho~n that these descriptions can lead to unique stochastic realizations

77

E-I,I 12

180

then solving (6), we obtain L = 13 and therefore

/~=

g=

7

Acknowledgements The authors would like to express their thanks to Dr V M Popov and Dr A Malumdar for many stimulating discussions

(ll) If we choose to select an S instead, we must first satisfy the constraint imposed in (9) The choice S = 0 trivially satisfies this constraint, thus, we solve (8) for the (np-1/2p (p - 1 )) elements of L

L=

-

-1

1

13

and therefore

(~=

;11

( 25 l / ' ~ \ - - 2 4 - - ~ 33J [

(2-133)

[

25

1

\

4L~5

/'913

143

\ [

Note that (F,H,O_,R,S)E~(F,I-S,~,~,S)for T = I , L=L-L and that various choices of 133 lead to reahzations that are not in 5e This example illustrates two methods of specifying an invariant system description of the given stochastic reahzatlon It also points out that selecting Q in (l) uniquely specifies R and S, however, selecting S in (n) uniquely fixes R, but not Q Thus, there is an entire family of Q's which have the same R and S and each particular Q specifies a canonical form for iF, H, Q,R, S) on X1 under the action of GR. Unfortunately the mvariant system description just developed does not completely solve the stochastic realization problem In order for an element (F,H,Q,R,S) of XI to be a stochastic realization, it must be a member of 5e The description of the degrees of freedom available in (Q, R, S) is helpful In finding a reahzatlon which satisfies the covariance property, see Candy (1977) for details One well known technique to guarantee that the Q,R,S specified also satisfies the covariance condition is to select L

References Candy, J V , T E Bullock and M E Warren (1977) Stochastic realization via Invanant system descriptions Proc of J A C C, pp 1200-1222 Candy, J V, M E Warren and T E Bullock (1978) Realization of lnvarlant system descriptions from infinite Markov sequences IEEE Trans Aut Control AC-23, 93 Denham, M J (1974) Canonical forms for the identification of multlvariable hnear systems IEEE Trans Aut Control AC-19, 646 Denham, M J (1975) On the factonzatlon of discrete-time rational spectral density matrices IEEE Trans Aut Control AC-20, 535 Faurre, P (1976) Stochastic reahzatlon algorithms In Systems Identification Advances and Case Studies (Edited by Mehra and Lainlotis) Academic Press, New York Gantmacher, F R (1959) The Theor)of Matrices Vols 1-2 Chelsea Publishing C o , New York Gevers, M R and T Kallath (1973) An innovations approach to least squares estimation--part IV discrete-time innovations representation and recursive estimation IEEE Trans Aut Control AC-18, 588 Kalman, R E (1974) Class notes on systems theory, course of R E Kalman at the Univ of Florida, Gamesville, Florida Luo, Z and T E Bullock (1975) Discrete Kalman filtering using a generalized companion form IEEE Trans Aut Control AC-20, 227 Majumdar, A (1975) Private communication Popov, V M (1972) Invanant description of linear timeinvarlant controllable systems SIAM J Control 10, 252 Popov, V M (1973) Hyperstabllltv of Control Systems Springer-Verlag, New York (Initially pubhshed in Romanian in 1966 ) Tse, E and H L Weinert (1975) Structure determination and parameter Identification for multivariable stochastic linear systems IEEE Trans Aut Control AC-20, 596 Willems, J C (1971) Least squares stationary optimal control and the algebraic Riccatl equation IEEE Trans Aut Control AC-16, 621