Computer Aided Design of the Stochastic Controller for a Large Scale River System

COMPUTER AIDED DESIGN OF THE STOCHASTIC CONTROLLER FOR A LARGE SCALE RIVER SYSTEM M. F. Hassan*, M. G. Singh** and A. Titli*** *Cairo University, Faculty of Engineering Electronic and Comm. Dept., Giza, Egypt **Laboratoire d'Automatique, E.N.S.M., 1 rue de la Noii, 44072 Nantes Cedex, France ***Laboratoire d'Automatique et d'Analyse des Systemes du C.N.R.S., 7 Avenue du Colonel Roche, 31400 Toulouse, France

Abstract. In a recent paper Hassan et al have developed a new algorithm for solving the large scale systems filter design problem for the case of linear interconnected subsystems with gaussian white noise disturbance sequences. Their algorithm has desirable numerical properties which makes it superior to the global Kalman filter approach for large scale systems. Parallel to this work, Singh has developed a hierarchical algorithm for computing the feedback gains for large scale linear interconnected dynamical systems with quadratic cost functions. Since both the filter of Hassan et al and the controller of Singh provide the optimal solution and use a classical information pattern (in the sense of Witsenhausen) it should be possible to combine the two structures in order to produce stoch8stic optimal control. In this paper~e see hm; this isdone for a large river pollution control example. The overall system is of order 52 and it is divided into 6 subsystems. Each subsystem problem can be treated as a Linear-Quadratic-Gaussian subproblem. He use the controller algori thI'l as \;ell as the filter algorithm in order to provide simulation results for stochastic optimal control for this practical large scale problem. Keywords.

Hierarchical control, stochastic control, large scale systems

I. INTRODUCTION

linear interconnected dynamical systems with gaussian white noise disturbances. Their algorithm has desirable numerical properties which makes it superior to t~e global Kalman filter approach for large scale systems. Parallel to this '"ork, Singh [" 2J has developed a hierarchical algorithm for computing the feedback gains for large scale linear interconnected dynamical systems with Guadratic cost functions. Since both the filter of Hassan et al [IJ and the controller of Singh [2J provide the optimal solution and use a classical information pattern (in the sense of Witsenhausen [3J), it should be possible to combine the two structures in order to produce stochastic optimal control. In this paper we see how this is done for a large scale river pollution control ~roble m . The overall system is of order 52 and it is divided into 6 subsystems. Each subsystem problem can be treated as a linear quadratic gaussian subproblem. We use the controller algorithm as well as the filter algorithm in order to provide simulation results for stochastic optimal control for this practical large scale problem.

Optimal stochastic control theory provides one framework for the calculation of the parameters of the feedback controls for linear dynamical systems with quadratic performance indices and subj ec t to Gauss ian \Jhi te no ise disturbances. However, since it is difficult to assign a priori weights in the cost function, the design is best carried out using an interactive computing facility since simulations with different weighting matrices enable the designer to find a set of weighting matrices which yield those optimal control. That give a satisfactory response. However, as the size of the problem becomes bigger, the computing facility is unable to handle it if standard algorithms are used. But, taking into account that most large scale systems in fact comprise interconnected subsystems, it may be possible to use a hierarchical calculation structure to get around the computational difficulties. Recently, Hassan et al (I] have developed a new algorithm for solving the large scale systems filter design problem for the case of

93

94

M. F. Hassan, M. G. Singh a nd A. Titli

The rest of this paper is divided into 3 parts. In section 2 we describe the stochastic control problem and we outline the new filter calculation structure. In section 3 we describe the hierarchical controller for optimal stochastic control. Finally, in section 3 we solve the 52 order river pollution control problem. 2. THE STOCHASTIC CONTROL PROBLEM AND THE NEW FILTER CALCULATION

saving results using this successive orthogonalisation procedure since at each stage only low order subspaces are manipulated. The actual orthogonalisation procedure that is performed in the Kalman filter is based on the following theorem (cf. Luenberger [4 ] p. 92). Th eorem I : Let ~ be a member of a H~lbert space H of random variables and let ~ I, denote its orthogonal projection on a closed subspace Y I of H (thus I is the best estimate of ~ in YI ). Let Y2 be an m vector of random variables genera ting a subspace Y2 of H and let Y2 denote the rn-dimensional vector of the projections of the components of Y2 on to YI (thus Y2 is the vector of best estimates of Y2 in YI)' Let Y2 = Y2 - Y2'

13'

The problem of interest is to minimise the cost function kC I

J=

E{~ L

11 x(k+l)11

k=kO

~

+ lIu(k) 11 I

~ l

(I)

If

(where E is the expected value operator and II.I/~ = .IS.), subject to the constraints x(k+l) =cpx(k) +'f"u(k)+Cz(k)+w(k)

Th en the projection of ~ on t o the subspace YI El Y2 , denotedJ is

(2)

where z(k)

LMx(k) + LNu(k)

(3)

l4J

For proof cf. Luenbe r ge r p. 92 . The above equa ti on can be interpreted as :

and y(k) = Hx(k) + v(k)

where E is the expected val ue.

(4)

Here v, ware uncorrelated zero mean Gaussian random white noise vectors of known covariance . QI' RI' 0, 'I-' , c , n, N, Il are all block diagonal matrices with blocks corresponding t o a distinct subsystem structure whilst L is a full matrix . Thus essentially we have a system comprising N linear interconnected dynamical systems whose outputs are corrupted by noise as shown by equation (4) and whose interaction inputs Z are formed by a linear combina ti on of the -;tates and controls of all th e other subsystems.

~ is ~ I plu~ the best estim,ete of J?> the subs pace YZ generated by Y2

10

Consider next the system comprising N interconnected linear dynamical subsystems defined by N

x.(k+l) 1

011 .. x.(k) 1

+

L.

0 .. x. (k)+W. (k)

j= I; i# j

lJ J

1

i=I,2, ... ,N

(5)

with the output s g iven by Yi(k +l) = Hixi(k+l) + Vi(k+l) i=I,2, ... , N

Next we describe a new eff ici ent algorithm for solving the state estimation in this case . 2 . I. The basis of the filter algori thm [I] The most appealing property of the g l oba l Kalman filter from a practical point of view is its recursive nature. Essentially, this recursive property of the filter arises from the fact that if an estimate exists based on measurements up t o that instant, then when receiving another set of measurements, one could subtract ou t from these measurements that part which could be anticipated from the results of the first measurements i.e. the updating is based on that part of the new data which is orthogonal to the old data. In the decentralised computation of the filter for systems comprising lower o rd er interconnected subsystems this orthogonalisation is ? e rformed sybs::s ten by subsys tern i. e. the optimal state of the subsystem one is obtained by successively orthogona lising the error based on a new measurement for subsystems I, 2, 3 , ... , K w.r .t. the
(6)

;!. i' ~i ~re uncorre la te~ zero me~n gaus Slan white nOise sequences With covar lances Qi' Ri respectively. Consider the Hilbert space Y formed by the measureme nts of the overall systems. At the instant k+l, this space is denoted by Y(k+I). The optimal mi nimum variance estimate ~Ck+1 \ k+ I) is given by

w~ere

E{ XCk +l)

I YCk+I)}=Elx(k+I)\ YCkt

ElxCk+I)IYCk+1

! k)}

(7)

This equat i on states algebraically the geometrical result of theorem I. The idea of the new filter is to decompose the second term i. e: E { x Ck+ I) \ (k+ I I k)} such tha t the optlmal estimate x(k+ 1 I k+l ) is given using the two terms by considering the estimate as the orthogonal projec ti on of ~iCk+l) taken on the Hilbert space generated by

1.

YCk)

CV

YICk+llk)

CV

2 Y Ck+1 3

CV Y~Ck+llk+l )

I k+l ) 0 .... 0

"':i- I

Y~

(k+J/k+l)

Compute r Ai ded Design ",i -I (k+ 1 Ik+ I ) is th e s ub space ge ner awhere Y. t e d by 1the s ub space of meas ureme nt s Yi(k +l ) and t he pr ojec t ion of it on t he subspaces genera t ed by Y(k) (£) Y1(k+ l ) (£) YZ(k +l ) 0. . . CV Y.1- 1 (k+ I ) whic h leads t o theo r em Z.

95

Theorem 3 : The op t ima l es t ima t e of t he s t at e of the ith subsys t em a t t he Ith iteration is g i ve n by

Theorem Z : The opt i ma l estimate ~. (+11 k+l ) ", Y1-1 (k +1 I k+l ) (8) o f t he ith s ubsys t em is given by tfie prol whe r e j ec t io n of xi (k+l ) on the space ge ne r a t ed by a ll mea sur ement s up t o k (y ;k» and t he pro1-1 -I 1 \1 (k +1 )= P~ ",1 - 1{k +1 k+1 )P?< l - I ~ l -I (k +1 k+ l ) j ectio n of x i (k+l ) on the subspace generax.{ r Y l ted by Y1 (k+I) k) (t) Yi(k+l l k+ l) GY~-I (k +ll k+ l ) i l l (9)

I

8 ..

Pr oof : Rew r i t e equa t ion (7) as

~ i (k + l l k +I ) =E f Xi(k+ I I Y(k) , YI(k + I) , YZ(k+I ) , .. . ( 10)

+ R (k+ I)

1

Yi(k+ I), Yi +l (k + I), ... , YN(k + l)f Et xi(k+I ) I Y(k) , yl(k+I ), YZ(k + I) .. . Yi(k+ I ) , Yi +l (k+ I), .... , yN _1 (k+I )} +

E { Xi(k+I ) ly~-I (k+I/ k+I ) }

where

{I

vY~N-I (k+1 I k+I ) =yN(k+ I) - ELyK(k+ l ) Y(k), y l (k +I ), .... , YN_1 (k+I) } (dere we have essen t ially split up th e measurement vector at the instant k+1 into the vec t ors for t he components subsysteus) or

~i (k+ll k+ l) = E{xi (k+II Y(k) } + + Ei xi(k +II Y1(k+II l:)}+ +

t

r =

E { xi (k+ I )\~~-I (k+ 11 k+ I )

Z

which proves the assertion. This theo r em yields the basic ca l cu l ation structure in t he ne,.] filter algori t hm. Essentially , we consider the ith subsystem at t he ins t ant k and then using the measuremen t of t he whole of the output up t o k-I we project the state vector xi(k) on the space gene r a t ed by a l l the measurements up to k-I. Then xi(k) is projected on the space YI (k \k-I ) and then successively on yr - I (k l k) for r = 2 to N and all these are add~d together t o give the optimal estima t e ~i(k l k) . Clearly this process of successive o r thogonalisation can be done independently for each subsystem so that we could envisage this being performed on a multiprocessor sys t em where at any instant k, the measurements are collected and sent by the coordina t i ng compu ter to t he computer for subsys t em i which uses t he s e measurements and then successively t he new measurements at time k+ 1 for subsys tems I , 2 etc . (which are all provided by t he coordinator) in order to compute the ne" e3timate ;i(k+l l k+l) which is used until a new measurement arrives and the "ho l e process is repeated. The actual orthogonalisation ope rations are defined in the following theorem which is given without proof . (For proof cf. Hassan et al . [IJ ).

(I I)

Hassan et a l [IJ have shown t hat such an algori t hm gives substant i a l savings i n computation time compared to the standard Kalman filter algorithm . In addition since low order matrices are manipulated at each itera tion , the fi l ter appears to be comp u tational l y stable for even high order systems . This fact is il l ustrated by Hassan et al [IJ by solving t he filter ~roblem for a 20th order mul t imachine example where the new filter is computationally stable whilst the global Ka l man fi lt er diverges . Having briefly described the decen tr alised fil t er ca l culation we are now in a position to examine how we can incorporate this fil ter hierarchy into the determinis t ic control hierarchy in order to compute optimal stochas t ic control . 3 . THE HIERARCHICAL CONTROLLER FOR 5TOCHA5TIC CONTROL

OPTI~~L

5ince here we have a classical information pattern , the stochastic optimal controller can be developed using the fo l lowinl t~o theorems whose proof is standard [ 5J . Theorem 4 : For large s c ale linear in t er connected dynamical systems with quadratic cost functions of the type kf - I 1 \ 2 2 ~Iin J '2 L 11 x(k+I ) II + lI u(k) 11 RI ( 12) Q k=kO s.t .

x(k +l ) = 0x(k) +'tJu(k) + Cz(k) Z(k) =

L~!x(k)

+ L:\u(k)

the opt i mum control law is given by : u(kf - k) = 5 1 (kf - k)x(kf - k) +52(kf - k)x(kf- k) ( I 3)

where 5 1 is a time varying block diagonal matrix "'hilst 52 is a full matrix .

96

M. F. Hassan, M. G. Singh and A. Titli

3.1. Remark The calculation of SI and S2 can be done using the deterministic control hierarchy of Singh L2J . Essentially, SI is calculated from local Riccati equations whilst S2 is computed off-line by storing the states and controls at certain points of the trajectories obtained using a standard hierarchical method and then inverting a matrix.

~(k+l)

Ax(k) + Bu(k) + C +

y (k+ I)

Dx(k+l) + ~ (k+l)

For large scale linear stochastic interconnected dynamical systems with an average quadratic cost function of the type kC I

min J ; Ef

~ L

k;kO

IIx(k+I)U~

+lIu(k)1i I

~f I

and subject to the constraints given by equations (2)-(4) the optimum control law is given by SI(kf-k) ~(kf-k/kf-k)

u(kf-k)

A

+ S2(k f -k) x(kf-k/kf-k) where SI and S2 are identical to those in theorem 4 whilst ~(kf-k/kf-k) is the optimal filtered estimate of x(kf-k). It is easy to prove this theorem by noting that on substituting (3) in (2), we have the classical LQG problem [5J whose solution is given by theorem 5. 3.3. Remarks In theorem 5 we have a separation theorem i.e. we see that it is possible to compute optimal stochastic control by superposing the deterministic control hierarchy on the decentralised filter hierarchy as shown in Fig. I. Here SI is computed by the local deterministic controllers whilst S2 is determined using the 2 level method of Singh [2J The estimate ~(kf-k/kf-k) is obtained from the filter hierarchy using the equations defined by theorem 3. The superposition of the two hierarchi=s yields optimal stochastic control. This superposition is shown in Fig. I. Next we examine the river pollution control problem. 4.

STOCHASTIC CONTROL FOR THE 6 REACH RIVER SYSTEH

OPTI~~L

The distributed delay multiple reach river pollution control problem is described by Tamura [6J . Using his discrete time river Cam model but for 6 reaches with distributed delays, we arrive at a 52nd order model of the form

(k)

where ~ and ~ are uncorrelated zero mean Gaussian random vectors. x ~ R52 essentially describes the dyna;ic behaviour of B.O.D. and D.O. in each of the 6 reaches We note that B.O.D. is not measurable whilst D.O. is. The control is given by

Next we note that the optimal stochastic controller uses the same "gains" SI and S2 and the filtered estimate obtained by using the decentralised filter equation given by theorem 3. 3.2. Theorem 5

1

! I (k) ] ~(k);

, [

k;O, I, ... ,K-I

TC 6 (k)

wherelf l , ... /( 6 are the maximum fractions of B.O.D. removed from the effluent in reaches I to 6. A suitable cost function is of the form

f

+ Min E ~ 11 x(400) 112 152 u (k) 399 I 2 d 2 + :L _ ("~(k)-~ 11 I +lIu(k) 11 k;O 2 1001 52 2 where x d are the desired values which are taken to be the steady state values of B.O.D. and D.O. for each reach. 152 is the 52nd order identity matrix, 12 is the 2nd order one. J

4.1. Simulation results The filter and the control hierarchies for this 52 order river pollution control problem were simulated on an IBM 370/165 digital computer at LAAS Toulouse over a period of 4 days divided into 100 sampling periods per day. Fig. 2-7 show the real and estimated B.O.D. and D.O. in each of the 6 reaches. Fig. 8 shows the 6 controls. As expected, the estimation of B.O.D. is not very good since it is not measured. We also see that in each case the state approaches its desired value although as we go further downstream, it takes much longer to reach the steady state value. 5. CONCLUSIONS In this paper we have superposed the hierarchical filter calculation structure of Hassan et al upon the hierarchical discrete time control structure of Singh in order to provide a computer aided design procedure for stochastic optimal control. This procedure is essentially applicable to large scale linear interconnected dynamical systems with quadratic cost functions and where the subsystems are subject to Gaussian white noise disturbances. The procedure has been illustrated on a large scale example i.e. that of 52 order river pollution control problem. This shows that: a) A hierarchical structure of the kind developed here is very efficient for solving such large scale problems

Computer Aided Design b) The procedure can be efficiently used to provide computer aided design for the stochastic optimal control for such systems. REFERENCES

[IJ

Hassan M., Salut G., Singh M.G. and Titli A. "A decentralised computa tional algorithm for the global Kalman filter". IEEE Trans. AC 23, 2, 262268, 1978

[2J

Singh M.G. "A feedback solution for the large scale infinite stage discrete time reculator and servomechanism problem". Comput. and Elect. Eng. 3, 93-99, 1976

[3J

Wi tsenhausen H. S. "A counter examp le in stochastic optimum control" SIAM J. Contr., vol. 6,1,131-147,1968.

[4J

Luenberger D. "Optimisation by vector space methods" New York, lVi ley, 1969

[5J

Meditch J. "Stochastic optimal linear estimation and control" McGraw Hill Book Co., 1969.

[6J

Tamura H. "A discrete dynamical model with transport delays and its hierarchical optimisation to preserve stream quality". IEEE Trans. SHC, 4,424-429, 1974.

Fi g . 1

Filter and control structure

97

98

M. F. Hassa n, M. G. Singh and

Fig. 2

A. Titli

Concen t ra t'10ns of BOD and DO (actual and es t1. mated) for reach

States

Fig. 3

Conc ent r at i ons of BOD and DO (real and es o . mated) fo r reach 2

I

Computer A"d 1. ed Design

Fig" 4 " Concentrat'1.ons of BOD and DO f or reach 3

•

States

2

rncl26t-Jit~--"--tihme 340 3SQ

. Fig. 5 : Concent rat1.onS of BOD and DO f or reach 4

99

M. F . Hassan , M.

100

G• S ~ngh ' and A. Tltli .

Fi g . 6 : Con centrat ~ons ' of BOD and DO f or subsystem 5

t

StattS

3.4 f

3.0

~

Fig. 7 . Concent rations . of BOD and DO f or reach 6

' l. ed DeSl.gn Comput er A'd

101

Controls

0.24

0.2 0

0.16

' t,Ime

o

7r1~...l....-....o_- '

20

60

140

100

180

220 ~;:;; 300 -- ' 260 ----' 340

Fig. g , Th,

0

.

38

ptl.mal st oc h as tic contro ls f or the 6 reaches

0