Hierarchical optimisation and control – an applications orientated survey

HIERARCHICAL OPTIMISATION AND CONTROL AN APPLICATIONS ORIENTATED SURVEY Madan G. Singh, A. Titli and M. Hassan Laboratoire d'Automatique et d'Analyse des Systemes du C.N.R.S. , 7, Avenue du Colonel Roche, 31400 Toulouse, France

ABSTRACT

ring and stochastic optimal control for linear Gaussian interconnected dynamical systems. Our previous surveys (Ref. 6,7) as well as that of Smith and Sage (Ref. 5) have concentrated mainly on part (a). However subse~uent to the publication of these works, a number of new and powerful me thods have been developed for computing open l oop co ntrol for non-linear systems using a hierarchical structure and we will concentrate on these in our treatmen t of part (a). In each part we wi ll examine the practical applicability of the methods by studying con crete non-trivial examples.

In this paper a survey is given of the "state of the art" of dynamical hierarchical control from a practical applications point of view. The subject is essentially divided into three main parts. In the first part hierarchical optimisation techniques which lead to open loop control are studied. Here the recent work on a l gori thms for non-linear problems is particularly emphasised. In the second part, hierarchical calculation of feedback contro l for both linear quadratic as well as non linear systems is described. In the final par t, the s tochas tic control prob lem is considered and it is shown how a deterministic controller hierarchy can be superposed on that of a new decentralised filter to achieve stochastic optimal control. In each case the a l gor ithms are illustrated on practical examples t aken from diverse fields like river pollution control, power systems, etc. I.

2. OPEN LOOP HIERARCHICAL METHODS

I NTRODUCTION

In the context of the utilisation of systems theory for solving some of the problems facing the developing countries it is recognised that the area of hierarchical multilevel control will provide some of the most useful tools. In this paper we describe recent developme nts in dynamical hierarchical control from a practical applications point of view. In the last few years a number of algorithms have been proposed for the optimisation and control of large scale dynamical systems using hierarchical techniques ('<.ef. 1-8). However, it is felt that the field of hierarchical control does not really have an image of its own and that it comprises essentially of a number of adhoc methods. In this paper we attempt to provide some kind of a unifying structure in the field of hierarchical control from an applications point of view.

The hierarchical optimisation methods for dynamical systems which l ead to open loop control have been extensively described for linear-quadratic problems by Pears on (Ref.2), Singh (Re£. 6, 7), Takahara (Ref. 9) and the interested reader is refferred to their work. Here we will essentially discuss the methods for non-linear problems and in one case how the non-linear algorithm leads to a good one for linear problems. 2. I. Open Loop Hierarchical Methods for Non-Linear Systems Since the standard goal coordination method for linear quadratic problems (Ref. 2) could fail for the non-linear case because of possible duality gaps (cf. Javdan (Re£. 12» we will treat only those methods which do not rely on the strong duality theorem for their essential justification. We will describe the more practical prediction approaches of Hassan and Singh (Re£. 10), Singh and Hassan (Re£. 11), the costate coordination method of Hassan and Singh (Ref. 13) and the three level method of Hassan et a1 (Ref. 14). 2.2. The Prediction Method of Hassan and Singh (Ref. 10)

It is convenient to divide the subject matter into 3 main parts i.e. (a) optimisation of linear and non-linear dynamical systems using hierarchical techniques and leading to open loop control, (b) the use of hierarchical techniques to compute closed loop control for both linear and non linear dynamical systems and (c) optimal decentralised filte-

The basic problem is to minimise a cost function of the type J =

N

1 [T

i=l

0

L: 2"

where

11

11.11

2

2

2

( 11 x. 11 Q + 11 u. I1 R ~ T

i

"-

) dt

(I)

i

S = . S.; and there are N inter-

12

M.C. Singh, A. Titli and M. Hassan

connected dynamical subsystems with separable quadrati c cost functions.

-I 1\ x 1\ 2 + -I \\ u 11 2 +

The minimisation of (I) has to be performed subject to the non-linear dynamical constraints.

x f (x, - --

-

2'J 11

~-~

where x is an n dimensional state vector and u is a~ m dimensional control vector. Let the equilibrium point of the system be at the origin i.e.

Q

°\H +

°

~ = Ax + Bu + D (x,u)

d HO d~ =

Using

(3)

To solve this problem of minimising J in equation (I) subject to equation (2) let us assume that a higher level provides ~ = ~O and u = uO. If these fixed trajectories are used-in ~uation (3), it fixes D(x O, uO) and since A, B are block d iagona 1, th; pr;;-b lem reduces to solving N independent minimisation problems the ith of which is given by Min J

=)

°

OO ~T lA~+B!:!.+D (2! ,~ )

~ L::-~

] +

°]

~HO

g, -

cl HO

•

:<,"A + ~

'C);:\

d HO - cl HO d f.l = Q, ~ =

0,

~

x

9,

~,

d HO ~ =

2"

all

[\l~.

~

dt

B

-Q

~

1:

!-A

u

°

(7)

T,

~

+ S r;.0 -'1\ ",here Q1:

Q

+

S (8)

= x

(9) (J 0)

u

°° 1:-1 ° 1:H,: -R

_xO)+dDC,J''\!)l d xO

°

H~ -H(-R

1:-1 T") B ~+R

,cJD(x,lJ) °° d ° Basically, we have +

subject to

(6)

H";J

°

-;\ (T) =

O-

x

(4 )

Q.

T,

B ~ + H ~ -~ where R1: =R+H

~(O) = ~o

2

11

I-

° 1 ~=Ax + BR1:-1 I- T, °-~J+ 1 .P(~ °,:,:°) 0+ 1:-1

~= R

where A, B are block diagonal with N blocks corresponding to the subsystems and D contai ns the non-linearities and the off-diagona l terms.

1 -2

2 + S

the stationarity conditions could be ,vri tten as :

Now it is possible to rewrite the constraint equation (2) as

T

°

I -11 x-x \I 2 - -

where ~ is the adjoint vector and 'Jr, ~ are appropriate Lagrange multiplier vectors.

C>'ir_1

at

11 2

R

-

TOT

d HO f(x,u,t)

2

~ [~-~

+

(2)

u, t)

=

2

(I I)

I

~)

(12)

I!..

°°

i.=A.x.+B.u. + D. (x.,u.)

-~

~-~

~-~

-~

-~

(5)

-~

where Ai ' Bi , Di are respectively the ith blocks in A, Band D. However, to ensure that ultimately we are solving the original problem it is useful to introduce certain penalty terms in the cost function. Let us modify the cost function ~n equation (I) to J

I

°fT (\\ ~ IIQ2 +II~-~°IIH2 )dt ="2I

+

I\ llllR2

+ 1\

°

to ensure that the stationarity conditions are iteratively satisfied. This can be done within a two level structure as follows : ~

: start with some guessed values for the traj ec tor ies x O, uO, '\( and 0 and send to level I. It is-pos~ibl~ to start for example with the initial guess xO=O, uO=O; '1(=00=0 ----

-

-'

-

~

2

~-2: 1\ S

(6)

where S, H are block diagonal matrices. These extra convexifying terms have been added in to give the designer an additional degree of liberty. Now if xO could be forced to equal x and uO to eq ual u then the modified cost f:-nctio; JI in equ-:rtion (6) will be identical to J in equation (5) and the resulting ~, ~ will be the optimal ones. To ensure this, consider first the overall stationarity conditions for this modified problem. Write the Hamiltonian of the overall system as

: At level I, solve the N independent minimisation problems for the given xC, uO,'il' ,(3 by noting that (a) the two pointbo:nd;ry value problems of equations (7), (8) splits into N inde pendent blocks because of the block diagonal nature of all the relevant matrices and (b) each of these low order blocks represents a linear two point boundary value problem ,,,hich can be solved easily by integrating an appropriate Riccati type equation. The resulting locally optimal trajectories for ~, ~, -;\ are collated and sent to level t'Vo. Step 3 : Produce new prediction for xO ,uO, 'iY ,(3 by substituting the x, ll, ?-. obtained f~om level one into the R.H-:-S. - of - eq uations (9)-(12). Iterations are stopped ",hen the ., Iy norms of ~ , ~ , "D: ' f? b ecome suff~c~ent

° °

Hierarchical Optimisation and Control close in successive iterations. Remarks : (I) Hassan and Singh (Ref. 10) have given a convergence condition for this algorithm (2) The jobs of level one are very simple since we merely have to solve a Riccati type equation for a low order system. On level 2 simple substitution is all that is requir ed . (3) There are clearly savings in computer storage compared to solving the problem using standard single level techniques like quasilinearisation since here only lower order subsystem matrices are stored. (4) Singh and Hassan (Ref . 15) have shown that it is possible to use a similar approach to deal with the case of non-linear, non separable cost functions . (5) The calculation structure is ideally suited for implementation on a multiprocessor sys tern (7) Singh (Ref. 17) has tested this algorithm on a system of two coupled synchronous machines.

Next,we give an example to illustrate that there are savings not only in computer storage but also in computation time even if a single computer is used to do the cal culation serially. 2 . 3 . Example : Control of a synchronous machine A model for this system is given by MUkhopadhyay (Rei. 16) as B2 Y2 =BI-AIY2 - A2Y3sin y - sin 2y YI Y2 I 2

.

.

u

Y3

l

- C Y3 + C cos YI l 2

and the cost function is I

J =

2

d L (Yl-Yel) 2 + Y22 + O. I (Y3- Ye3) 2 + (2r

100 (u I -u e I) 2 ]

dt

13

to exclude th e penalty terms in equation (6). Singh and Hassan (Ref. I I) have derived a convergence condition for this case and tests on the above machine examp l e setting S = show that convergence now is slightly faster and convergence still takes place even if the period of opt imis ation is subs tantially increased. The latter is not the case with the previous method.

°

2.4. Other methods for non-linear problems, (Ref. 13, 14) We saw with the above two algorithms that the stationarity conditions gave US a vector equation each for u, x , ~ , x O, uO, 'I! ,f}. and we iteratively ;ttemPted t o satis fy the equations for u, x, ~ at level I and xO, uO,m-, f:> at level 2-:- This led to the solutior:of Riccati type equations a t the lowest level. Now, there is no real reason IVhy we should not consider alternative combinations of variables to satisfy at different levels as pointed out by Mahoud et al (Ref. 22). The new three level method of Hassan et a1. (Ref. 14) does precisely this. The algori thm in this case is : Step I : Set the iteration index of the 3rd level i.e. k = I and provide a guess for the initial trajectories for xO u O, to the lm.,er levels Step 2 : At level 2, guess the initial trajectories ~ and and sent these to level I and set the iteration index of level 2 i.e .

r:.

L = I. ~

: At level I, integrate equation (7) forward in time with !(O) = ~O and calculate ~(t) from equation (6) and send ~, ~ to l evel 2.

Step 4 : At level 2 integrate equation (8) backIVard s in time from (\ (T) =0 and labe 1 the resul ting new (\. traj-ectory obtained by the index L + I i.;: ~ (t)L+I. Also calculate ~ (t) from equation (12) and label this tra'ector (t)L+I. If

. ;\ (t)-(\.L (t) ] 2 dt~E:(\.~s fT~L+1 -

not sa-

D

The parameter values and the details of the computation using the above algorithm are given by Singh (Ref . 17). Results: Convergence to the optimum (with an error of 10- 5 ) took place in 11 second level iterations using the best choice of S

° ° took 6.16 seconds on the 0. I

and this IBM 370/ 16 5 digital computer . Against, this, the optimal solution using quasilinearisation took 15 seconds on the same computer. Thus there were substantial savings in both computer storage and computation time using even a single computer . Next we consider what happens if we modify the me thod of Hassan and Singh (Ref. 10)

tisfied, where €~ is a small prechosen positive number, go to step 3. Els e

°

Step 5 : At level 3, use ~, ~ obtained from the fir st level to produce a ne'.' ~O '!:'. by using equations (9), (10) and label these new trajectories ,!O(t)k+l, ~O(t)k+l. T

If

f L~? k+1 (t)

° and

f [~

Ok+1

~Ol(t) J 2

(t)-:~

Ok

J2

(t) _

dt

dt ~E.x

~ E.u. where

E:x:.' f..u. are also small prechosen positive numbers, stop and record x, u as the opti mal state and control tr;Jectories. Other wise send .!O, u O to level 2 and go to step

2. 2.4. I. Remarks (I) This method

w

ouId appear to give even

14

M.C. Singh, A. Titli and M. Hassan

greater savings ~n computer storage since now only low order vector equations are manipulated at the lowest level instead of having to solve the matrix Riccati equation (2) Hassan et al (Re£. 14) have proved that the algorithm will converge (3) The method extends directly to the case of linear-quadratic problems (Ref. 30) (4) Hassan and Singh (Re£. 15) have also developed a discrete time version of the same algorithm and have apPlied it to solve avantageously a high order turbogenerator system problem (Ref. 18) previously treated by Jamshedi (Ref. 19) (5) The nel" method does appear to provide savings in computation time even using a single computer serially. This is illustrated next on the machine example as \VeIl as for the linear-quadratic case on the 22nd order river pollution example previously treated by Singh et al. (Ref. 20). 2.5. Examples

3 - CLOSED LOOP CONTROL The main idea used here is to solve both the linear-quadratic as well as the nonlinear problems off-line within a hierarchical structure for a given initial condition and use these to compute the feedback control. For the linear-quadratic case, this control is independent of the initial conditions (Ref. 20) whilst for the non-linear case it is not (Ref. 17,21). In the latter it may be possible to make the control relatively insensitive to initial state perturbat ions (Ref. 17). We begin our analysis in this second part by examining a method for computing closed loop control using a hierarchical structure for linear-quadratic problems. 3-1 - Closed loop hierarchical control for L-Q problems The dynamic optimisation problem for N linearquadratic interconnected dynamical systems can be written as tf

N

2.5.1. ~~~~£1§_1 : Q£!~~~1_£2~!E21_2i_~y~£bE2nous machine excitation The first example treated is the machine example considered above. Here the new algorithm was programmed on the IBH 370/165 digital computer and convergence to the optimum took place in 1.67 seconds compared to 6.26 seconds using the prediction method of Hassan and Singh (Re£. 10) on the same computer and 15 seconds using quasilinearisation globally.

The second example is of a large scale linear quadratic river pollution control problem with 22 state variables. The model and the control problem are fully described by Singh and Hassan (Ref. 20). This problem was solved by the new method on the same digital computer as used by Singh and Hassan (Ref. 20). With the new method, convergence to the optimum took place in 54.83 seconds compared to 85.58 seconds with the best previous hierarchical method (Ref. 20). 2.6. Remarks

Min J

2 Q. ~

subject to A. x. + B. u. + C. z.

x.

-~

~

x. (0)

z.

(15)

L .. x.

f=1

-~

(14 )

-~

~

~iO

-~

N

-~

~

-~

-]

~]

This problem can be solved within a tl"O level hierarchical computational structure using one of the several methods available for this purpose (Ref. 17). In each case, I"hen convergence to the optimum has taken place, the final control lil,,, for the ith subsystem can be written as ( 16) where

K.+K.A.+A.TK.-K.B.R~1 B:K. + Q. ~

~

~

~

~

~

~

~

~

~

~

o

(17)

and s.

-~

=

rLK.B.R.-1 ~

N

So far in this paper we have studied a number of new hierarchical algorithms for computing optimal open loop control for large scale interconnected dynamical systems and we have seen that the methods appear to give substantial savings in both storage and computation time. This is not surprising since unlike for the case of linear-quadratic problems both the global and hierarchical solutions are iterative and the latter manipulates lower order equations. An extra bonus which arises from this fact is that the hierarchical solution is substantially more accurate than the global solution since in the former, numerical inaccuracies build up more slowly than in the latter.

1

J (2 il.~)\ i= 1 to

L.

=

~ J=1

~

~

[?-. -J

T TJ s.-K. C.s. + B.-A. ~

T

~

1 F J

-

T

~

~

~ -~

(18 )

L ..

Thus the optimal control for each subsystem consists of two parts -a local feedback control obtained by solving a Riccati type equation for the subsystem plus a term which accounts for the interaction with the other subsystem. This latter term makes the control open loop. In order to make the control a closed loop one independent of the initial conditions, it has been shown by Singh et al (ref. 20) that ~(t) = T (tf,t) x(t) (19)

Hierarchical Optimisation and Control

and for the case where t ~oo, and A, B, f C, Q, R are time invariant,T becomes a constant matrix. Thus for this infinite time regulator, we can use the fact that ~ Tx to compute T as follows : Since near t = 0, T is constant whilst ~ and s are not, record sampled values of x and s ; t the first n po~nts (for an overall system of order n) and fOlm the matrices S

[~ (to)

~ (t I)

S

lx(t ) o

x( t I)

............ ............

J x(t ) J n

s (t ) n

-I

then T = S X

This inversion of X should n o t pose much of a problem for even high order systems since it is to be done off -line*. If it is desired to calculate the time varying T (i. e . if the horizon must be considered finite) it is possible to do so by solving the problem n times for n different initial conditions and then formirgthe n x n matrices . S

[~I(t)

X

b

l

(t)

2 s (t)

l:

2

(t)

.......... .2 n (t) J .......... !n(t) J

for each integration point and determini£~ each va lu e of T by the relationship T=SX 3-2 - Reoarks (I) The above approach enables one to solve the large int erco nnected systems regulator problem within a completely decentralised calculation structure (2) The resulting gains are independent of the initial conditions (3) The method has been used by Singh and Hassan (Ref. 23) to solve a 22nd order river

*

It should be noted that the approach hinges on one's ability to invert X which basically depends on the linear independence of the chosen record. In practical cases , if a large enough perturbation is given it will be possible to obtain an invertible X. Otherwise we can solve the problem off-line n times success ively for the initial conditions I 0 0

-------

0 0 Then T vert X.

pollution control problem, by Hassan and Titli (Ref. 24) to solve a problem of ll cou pled synch ronous machines and by Hassan et a!. (Ref. 30) to solve a 52nd order river pollution control problem. (4) Singh (Ref. 25) has developed a discrete time version of the above continuous time regulator and used it to solve a discrete version of the river pollution control.

Next we consider the developme nt of closed loop control for non-linear systems .

4 - CLOSED LOOP CONTROL FOR NON - LINEAR SYSTEMS The problem of computing optimal feedback control for large scale non-linear dynamical systems has received very little attention. This ~s not surprising since at the present time it is not possible to compu t e th e f ee d back co ntrol law for even the centralised case for non-lin ear systems if it is desired that the feedback parame ters be independ e nt of the initial conditions. Thus the best that we could hope to achieve is to compute the feedback controls which a re relatively insensitive to initial st a te variations. The basic approach that we will use r elies on the fact that if for a give n init ial condition, the problem is solved using o n e of the methods described in section 2 . Th en from the resulting state-costate traj ecto ries it will b e possibl e to ob tain a co ntrol tvhich is a function of the current state. To make the control relatively insensitive to variations in the initial stat e ( or to perturbations) it is possible to derive a sensitivity relationship which could enable us to test a priori the relative sensitivity of th e various states to initial state perturbations. If it appears that the system is too sensitive, it is possibl e to correct the feedback gain matricES on-lin e using a two level computation structure. We begin our analysis by considering the computation of feedback control starting from the stationarity cond iti ons given by equations (6-12) in section 2. Here at the lower level, it is necessary to solve the linear two point bound ary va lue problem in x, ~ , given by equations (7) and (8) for-gi-;;en l;.0, ~O,!! ' Let us assume a solution of the form

fl .

~ = P x + s \ihere P i~ a blo ck diagonal matrix. Then A = Px + Px + s.Substituting this into equ;tion; ,8)- and-( 7) we obtain after minor manipulations the feedback con trol

0 0

0 I 0

15

0 S and it is not necessary to

(20) ~n-

where

P+ P (T) and

AT P + PA - PB R- I BT P + Q*=O 0

(21)

16

M.G. Singh, A. Titli and M. Hassan T

S + A s - PBR PBR

- I

x-I p.,

I~ + PD (2:

wi t h

.§

(T) =

x

T

B s + BPR 0

I

H u

0

the problem was solved and the optimum

0

,~ ) -

(22 )

cost J was found to be J

1.1 658

The time varying optimal control law for this initial condition was of th e form

Q

To compute the feedback control law given by equation (20), it is necessary to in t egrate the N blocks of the Riccati equation (2 1) independent l y, bachvards in time from peT) = 0 and equation (22) also backwards from !(T) = Q using the given values of the trajectories x O, u ,1'"1', ~ supplied by the second leve 1. - \-1he; x cr, utl", 1"1' , 0 converge to their optimal values ~ these - are u sed to compute the optimal feedback con t rol u from equation (20). Note that this control is dependent on the initial state.

HI

HII YI + HI2 Y2 + HI3Y3 + q l

M2

H2 1YI + H22 Y2 + 1123 Y3 + C;z

Next the intial conditions were changed to /(0) = [ 1, 0.1,10, 1, 0 . 1, 8 . 69 J Note th a t these new initial conditions ar e far from the old ones . Here the optimal C(l ntrol gave a C(lst of J = 78 . 5048 .

Next we consider an eX2mple to illustrate th e approach .

Using the previously calculated gains etc . the contro l was recalculat ed a nd a simulati on of th e problem with this control gave a cost of

4- 3 - Example : Control of tW(l coupled sYJ'clnonous machines (R ef . 17,26)

80 .1 3021 only 2 %.

The problem is to contro l the excitation vol tages of two coupled synchronous machines optimally . A model for the system is given by the 6 non- linear differential equations

4- 4 - Sensitivity with r espec t to initi a l conditions

YI=Y2' Y2=n l- Al Y2 - A2Y3sinYI - B2/2 sin 2YI ;3=M I - C l Y3+ C2 cos YI ' ;4 = yS ;

, J-

1. e . a loss of opt imality of

Calvet (Ref. 26) has d eveloped an expres sion to relate variations in initial c(1ndi tions to changes in the cos t and this ex pression is of th e form

YS=B4 - A4YS - ASY6 sin Y4 - BS/2 sin 2Y4 where S is a sensitivity matrix .

16 = M2 -C 4 Y6 - Cs cos Y4

For the above two machine example, the sen-

where YI-Y6 are the 6 state variables and MI , M2 are the controls. The parameters are AI

A4

0 . 2703; A = 12.0 12 , A =1 4 . 4 144 2 5

:it~r~~ matr~: ] was found to be

39 .1 892, B =- 48.048, B =- 57 . 6S76 2 S 0.3444, C = 1. 9, Cs = 2 . 28 2 and the system comprises two subsystems coupled by YI = Y4 '

SI

S2 =

It is desired to minimise J where J =

[

o

2 I

-2

2

2

[\I y-y p \\ Q + \I M-M c 1\ R

]

[ 0 . 7461

M~

[ 1.1

R

~

Q

1S

0

7.7438

0 . 746 1

- 7.2 4.9 - 1.7

29.5 ] - 1. 7 5.0

- 7. 2 3 1. 0

-7.2 4. 5 - 1.9

- 1.9

[",5.,

31 . 0

J

5. 7

dt

These mat r ices show that th e feedback control is rather sensitive to variations 1n Y1(0) andy{,(O) .

07.7438 J

Next we examine how we could modify th e feed back parameters on- line in order to reduce this sens i t i vity to initial state perturbations.

where

Y~

[ 26 1. ' - 7. 2 29 . 5

I. I ]

12 the second order id e n ti t y matrix and a diagonal matrix wi th t he diagona l give

by 20

2

20

20

3- 3-1 - Simula t io n resu lt s Usi ng the ini ti a l condit ion s /(0) =[ 0 . 7374 - 0 . 2 15 1

- 0.2 151

7.7443

6 . 9483 ]

4- 5 - Improvement of the feedback control From equation (20) i t is clear that the op timal c l osed loop control is of the form u

=Hx

+ q

(23 )

uhere C] is not independent of the initial c~nditions. One nossible way of making the system l ess sensitive to initial state variations is to modify q o n-l ine as the "in i tia l " sta t es changes- as a result of unknown per turbations. This can be done within

Hierarchical Optimisation a nd Control a t wo l eve l struc tu re as follows Le t the original problem be t f Mi n f(~,~, t) dt

(24 )

::: 0 subject to x

(25 )

J

g(~,';!.,t)

_x(tO) ~O

and assume that the optimal closed loop co ntr ol of the form of eq uat i on (23) has been determ ined using the method described previously. Suppose nmv that at inst ant &" the s t ate changes to x( ?;') as a result of some perturbation a nd ;e wish to determine the new op tim a l ~ quickly on - line. In orde r to do this, rewrite equations (24) and (25) as

(J

Min

t

f

f ('i, Hx + ,!,t) dt

to

q

s . t.

ic

= g(!5 , H:: + q, t) ; ~(tO) =! O

In order to perform this minimisation to ob tain a new q , writ e the Hamiltonian

and minimise it w. r .t. q. From th e necessary conditions for optimali t y

_W = d H* 1 d~ ..:c( t

d H* 2I q

0 and

:;.

ii(:: ,H~+3,t)

o)=,9

We wish basically to solve this two point boundary value problem iteratively to obtain a new q. This can in fact be done u sing any of the- standard hierarchical methods des cribed previously (Ref . 10, 11, 13, IS) using q i ns tead of u. The idea is that since in practice initial-conditi ons do not change substantially , the precalculated value uf q could serve as an excellent initial guess and thus convergence would take place rapidly. Numerica l studies by Calvet (R ef . 26) show that this is indeed so. Having described the n ew approaches to closed l oop control for l a rg e scale systems we are now i n a position to go on finally to the prob l em of optimal stochastic co ntr ol. 5 - OPTIMAL STOCHIISTIC CONTROL FOR LARGE SCALE L- Q- G PROBLEMS

In this final part we consider th e stochas ti c case and give some results that have been obtained for systems comprising Linear Quadratic Gaussian sllbsyst ems . Th e problem is to minimise the cos t function k -I

J-

C-

2

\I.:s(k + l) 11

k=kO

QI

subject to the constraints 0~(k) + 'P .t,!(k) + C~ (k) + n(k)

~(k+ l)

(27 )

\vhere ~(k)

LM~(k)

+ LN u(k)

(28 )

and y(k)

Hx (k) + v(k)

OSAFD-D

(29)

17

Here n, v are uncorrelated zero mean Gaussian random whit e noise vectors of knmvn covarian ce . QI' RI' 0, If, C, M, N, H are all block diagonal a nd L is a full matrix . Thus essential l y we hav e a system comprising N i nterconnected subsystems whose outputs are corrupted by n oise as shown by equ~tion (29) and whose interaction inputs z are formed by a linear comb inat i on of th; states and controls of all th e other subs ystems . Now , for the deterministic version of the discrete dynamical problem described above , Singh (Ref. 25) has previously obta ined a n optimal co ntroller the ga ins of which can be calculated wit hin a hierarchi cal compu tation al struc tur e. Here we describe how this con troller hierarchy can be superposed on the new decentralised filter hierarchy recently developed by Hdssan et al (Ref. 27) in order to provide optimal stochastic feedback con tr ol. The validity of this sunerposition reli es on the separation principle which has been established for such systems by Hassan et al. (Ref. 28). We begin our ana lysis by describing the new decent r al i sed f ilter a nd then we will go on t o describe the stochastic controller . 5 -1 - The basis of th e new decent ru lised filter

The most appealing property of the global Kalman filter from a practical point of view is its re curs iv e nature . Essent i ally, this recursiv e property o f the filter ar i ses from the fact th at i f an estimate exi sts b ased on measurements upto that instunt th e n when r eceiv ing a noth e r set of measu r eme nts, one could subs tract out from these mea sur e ments that part which could be anticipated from the results of the first measurements i . e . the updating is based on that part of th e new data which is orthogonal to the old data. In th e new filter for systems compri sing l ower order interconnect ed subsystems this orthogonal i sation is perfo rm ed sllbsys terns by subsystem i. e . the opt im al estimate of th e s t ate of subsystem one is obtJ i ned by sUCCEssively or t hogo nnli sing th e error based on a new measur e ment for subsyst ems 1, 2 , 3 , •.. N H.r. t. the Hilbert space for med by all measurements of al l th e subsys tems upto that inst art . Much computational saving r esults usin g this successive orthogonalisation procedure si,ce at eac h stag e onl y low order subspaces are manipulated . The actual or thogonalisation procedure t hat is performed in th e Kalman f i lter is based on the following theorem (cf. Luenb e rger (Ref. 2:; , p . 92). THEOREH I ; Let (j be a member of a Hil~ ert space H of a random variables and let r3 de note its orthogonal projecticn on a clo sed subspace Y 1 of H ( t hus (6 is th e best est imate of 0 ~n Y1). Let Y2 be an m v ector of random var i ables ge neratin g a subspaceYz of H and let denote the m-d ime.,sional vector of the projections of the compone nts of Y2 on to Y I (thus Y2 is th e vector of bes t es timates of Y2 in YI ) ' Le t Y2 = Y2 - Y2

92

M.G. Singh, A. Titli and M. Hassan

18

Then the projectiontPf ~ on to the subspace YI (i) Y2, denot ed (~is ,...

f->

'"

'V

({?l'h)

f.>A, + E

=

T

r

N

'"

T

J-

I

L E (y 2 Y 2)

oJ

N

0 .. x.(t)

x.(k+l)

11 - 1

1

+

L j=l,iFj

0 .. x. (k)+H. (k) IJ - J

(30)

with the outpu ts given by (3 1)

y. (k+l) = H.x. (k+ l ) + V . (k+l)

_1

i

1-1

-1

I, ... , N

=

wh e r e

Vi, W· a r e unc orrelate d zero mean

gaussia~ whi~ e noise sequences wi th cova-

r~ances Qi' Ri respectively . Consider th e Hl1bert space Y formed by the measure~ents of th e overall systems. At th e instant k+ I, t his space is deno t ed by Y(k+ I) . Th e opti~a l min i mum variance es t ima te ~(k+ I \ k+ I) is give n by

£(k+1

I k+I)=E {.~[(k+l)

E k(k+l)

I Y(k+I) }=

I Y(k )} + E t~ (k+ l ) \ ~(k+ ll k) }

(32)

Th is eq ua tion states algebraically the geo me tri ca l result of t heorem I. The idea of the net" f ilt e r is to decom po se the second term i.e. E { x ( k+ l ) 1 r(k+1 I k) } s uch that the optima l es tima t e ~.ck+1 I k+ l) is give n using the two t erms by considering the estimate as the or th ogonal projection of ~i( k +l) ta ken on the Hilbert space ge nerat ed by Y(k)

(i)

8YI( k+llk)8Y~( k+l) k+ l)

I

2 N N-I (k+ 11 k+l) Y (k+1 k+I )G), .. G)Y 3 N

i-I t"h ere Yi (k+1 k+ l ) is th e s ubspa ce ge ner ated by the subspace of measurement s Y.(k+ l ) and the projection of it on the sub s~acesAenera t ed by YCk)G)Y j (k+I )G)Y 2 (k+ I) \:0 .... G) Yi -I (k+ I) tvh 1Ch l eads t o theorem 2.

2

THEOREH 2 : The optimal state i (k+l\k+l) of the i"-subsystem i s given by th e projec tion of ~i(k+l) on the space generated by al l measurements upto k (Y(k» and the projec,Sion of ::'i (k::~) on the subspace", &enerated by Y I (k+ 1 \ k) G Y2 (k +II1:+I) + .. G YN(k+1 1k+l)

i

Proof : Rewrite equ atio n (32)as

~i(k+l l k+l)

Ep' :I( k+I)\y (1:) ,y I (H I), ... 'YN-I ( k+I)} Here we have essentially split up the measurement vector at the instant k+1 int o the vectors for the component subsystems. or gi (k+ I \ k+ I) = E c.~\ (k+ 1\ Y(k) + +

E[xi(k+I)\~I(k+l\k)}

+

~

- 1

1, 2 , ... , N

=

_N

- cl

y2

where E is the expec teo value. For proof cf . Luenberger (Ref. L9 p. 92) The above eq uation can be interpreted as ~ is~ plus the best estimate of f., in the subspace Y2 generated by 9'2' Consider next the system comprisi ng N int erco nnected li near dynamical subsystems defin ed by - 1

where y ; -I(I:.+I\k+l) = y (k+I)-

r=2

E (x.(k+I) 1 ~r-I t 1 -r

(k+llk+I)}

which proves the assertion This theorem gives us the basi c struc ture of the new filter. Essentia ll y , we consider th e ith sub system at the instant k a~ then using the meas ureme nt of th e whole of th e o utput upto k+ I He project the state vector !i(k) on the space ge ner a t ed by all the measurements upto k-L. Then ~i( k ) is proJ.ected on the space YI(k\k-l) and then 'lrr-I (k \ k ) for r = 2 t o N and all these are added t oge ther to give the op timal estimat e gi(k \ k) . Clear l y this process of succes~ive orthogonalisation can be done independently for each subsystem so that we could envisage this beirg perfo rmed on a multiprocessor syst ems where a t a ny instant k , the measurements are col l a t ed and sent by the coo rdinating computer to the computer for subsys tem i tvhich uses th ese measurements and then successively th e netv measurements at ti~ k+1 for subsystems I, 2 etc. (which are all provided by the coordinator) in order t o compute th e new es tima t e !i(k+1 \ k+l) which is used until a new meas urement arrives and the whole pr oces s is repeated. The actual orthogonalisation operations are def ined in the following theorem which is gi ven without proof (Por proof cf . Hassan et al (Ref. 27 ). THEORD! 3 : The optimal estimate of the sta te of the ith subsystem at th e -tt h i t eration is give n by

e.1

!iCk+l\ k+l)e N

e.i

Xe

= ~(k+llk+l)e_1 + Ki e (k+l) (33 )

(k+ I ! k+ I )

\Vhe r e

\ee·1 (k+l)

p", ",e·1 (k +1 \ k+l) p;:'L..t'ik+Ijk+ l ) .! il ! i e (34 )

e

p"'e.1 ,,~.~(k+1 ( k+ l ) ~e

]e

+ Re. (k+l)

e

He p", g.1",e.1(k+l!k+) HTe

!- e.

~

e

(35 )

= Eki(k+I!Y(k), II( k+I )'l2(k+ I),

.... _.1 y.(k +I), -y 1. + l (k+ I )""5·N (k+I)}= E{Xi

(k+ II Y(k)'~ 1 (k+ I), ~2(k+I)+ ... +"

:t.i(k+l),

~i +l( k+I),

E{x.(k+I) I;Al~ -I (k+ t\ 1

· .. ·Y:.; _I (k+I)}+ k+I)}

If \Ve examine the algebraic structure of the f ilter equations we see that they give substantial savings in computer storage as compare d to the global solution using a standard Kalman filter.Hassan et al (Ref.27 ) have shown that in addition ther e are also

Hie r arch i ca l Op timisat i on a nd Con trol savings , in computa ti on time as compared to the standard so lu tion using even seria l opera tions on a single digital computer . The savings in computa t ion time and storage are particularly significant for lar ge scale problems . In addition, since on l y l ow order matrices are manipulated at eac h i t erat i on, the filter is computational l y s t able for even high order systems . This fact is illust r ated by Hassan et al. (Ref. 24) b~' solving the filter probl e~ for a 20th order multimachi ne example \vhere the new fi l ter is 'computationally stabl e whilst the global Ka l man filte r diverbes . Having briefly described the new dece n tralis ed filter we are now in a position to examine how we can incorporate t his filter hierarchy into the deterministic control hie rarchy in order tocompute optima l stochastic con trol. 5- 2 - The hierarchical controller fo r optimal stochastic control

19

where S I and S2 are i dentical to t hose in t heorem 4 whilst g(kf- k l kf - k) i s the optimal fil t ered est i ma t e of !(kf- k) . Remarks : With t h i s theorem which is also proved by Hassan e t a l (Ref . 28) we have established that it is possible to compute optimal s t ochastic control by superposing the deterministic control hierarchy on the decentralised filter hierarchy as shown in Fi g . I . Here SI is computed by the local determi nis tic co n trollers whilst S2 is determined using the 2 leve l method of Singh (Ref . 25) . The estima t e ~(kCk I kf - k) is obtained from the fil t er hierarchy us i ng the equations defined by Theorem 3 . 5- 3 - The multi - level stochas t ~controller exa~ple : stochastic contro l for a r iver problem The dynamic be haviou r of a two reach river system is given by

The stochastic optimal controller can be de veloped using the following two theorems which are proved by Hassan et al (Ref' 28) .

!(k+l )

A~(k)

y (k+ I)

Dx (k+ I ) +:1 (k+ I )

THEOREM 4 : For l arge scale linear i n terconnected ~ynamical systems with quadratic cos t function of the type . I kf -I 11 ~(k+ l ) 11 ~ + \l;: (k) lI ~ (37) M1n J = "2 ~ I I k=kO

where :J are uncorrelated zero mean Gaus sian ra ndom vectors, x I ' x3 are the concentrations of B 0 D (mg/l) in reaches I and 2 of the stream and x 2 ' x4 are the concentrat i ons of D 0 in reaches I and 2 of the

and subject to the constraints given by equations (27 - 29) but without the noise terms , th e optimum contro l la\v is give n by ~( kf - k) = SI(kf-k) ~(kfk) + S2(k - k) ~(kf - k)

f

(3 8) whe re S I is a time va r ying block diagonal matrix whilst S2 is a ful l matrix . Re~ark

: The calculation of S I and S2 can be done using the determinist1c con tro l hierarchy of Singh (Ref. 25).E s sential l y, S I is ca lculated from local Ri ccati equations whilst S2 is computed off- line by storing the states and controls at certain po int s of the trajectories obtained using a sta ndard hierarchical method and then i nverting a matrix . Nex t we show that the optimal stochastic controller uses the same "gai ns " S I and S2 a nd the filter ed e stima t e obtained by using the decentralised fi l te r equatio ns g i ven by Thoerem 3 .

J=E { ±

Z:\I~ (k+I ) II ~

k=kO

+1I:;( k )lI i ]} ]

the op t imum contr o l I mv is given by u(k - k) = S (k - k ) ~ ( k - k _ f I f f S2(k f - k)

B (kf - k I kf- k)

I k f - k)

+

B~ (k) +

C + )" (k)

:5 '

:::~a: T[h;1~~~t]rOl iSkg:v:~ I ~~ .. k- I -

'iYZ (k)

where 11~, 1\2 ar e the maximum fractions of B 0 D removed from the effl uent in reaches I and 2 . For river Cam near Cambridge, A, B, C, D are g i ve n by

A=

[ ~:)i __ ~'~7_1_ t 0.55

0 0.55

I I

-~.o]

C=

o

B=

[r

D • [

THEOREM 5 : For l arge scale l ine ar i nt er conne c ted stochastic dynamical systems with an average quadra t ic cost function of the t ype kf-I

+

:

0

__ J_ ]

0. 18 - 0.25

0 0 . 27

[4.5J ~. 15

2 . 65 0

0

0

2

J

i .e . on l y D. O.i s measure ~ (B.O .Rbeing ra t he r difficu l t t o meas ure) A ~uitab l e cost f unc t ion is K-1 d 22 J:i'1inE [ t-11~(K)1I14+~ t(II~(K)-~ IlIL 2 ~_o ... +I~ (k) roo 1 2 )} \vhere E i s t he expected value operator

Il.

1\ . 11 S

,..

T

S .,

14 i s the fO 'Jr th

order

20

M.G. Singh, A. Titli and M. Hassan

identity matrix and 12 is the second order identity matrix x d are desired values and these are given by

~d r ~ 1

of direct interest to developing nations whi lst the tools that have been developed cou ld be used advantageously t o solve many other problems facing the developing nations.

=

'-lJ

REFERENCES

This problem can be physically interpreted as :it is desired on average to maintain B 0 D around 5 mg/l, D 0 around 7 mg/l whilst minimising the treatment at the sewage works . Simulation results For k = 159, the system was simulated on the IBM 370/ 165 digital computer. The controller hierarchy g ave a constant gain matrix i . e. '1.

=

2

Pears on J.D ., "Dynamic optimisation tech niques" in "Optimisation methods for large scale systems" D.A. Wismer (editor) McGraw Hil1, 1971.

3

Titli A. "Contribution a l' etude des structures de commande hierarchisees en vue de l'optimisation des processus complexes " These d'Etat nO 495, Toulouse France , 1972. Published by Dunod, Paris , 1975.

4

Cohen G., Benveniste A. and Bernarhd P. " Cormnande hierarchisee avec coordination en ligne d 'u n systeme dynamique" Revue Fran~aise d'Automatique, Informatique et Recherche Operationne ll e, J4, 77-101, 1972.

5

Smith N.J. and Sage A.P. ' ~n introduction to hierarchical systems theory". Computers and Electrical Engineering, I, 55 -71,1 973

6

Singh M.G. "Practical methods for the control and stat e estimation of large interconnected dynamical systems" Revue Fran~aise d'Automatique, Informatique et Recherche Operationnelle, J3, 5- 45 ,1 974.

7

Singh M.G., Drew S, and Coales J.F. "Comparisons of practical hierarchical control methods for interconnected dynamical systems" Automatica, 11, 33 1-350, 197 5.

Gx + s!.

where G

d

Mesarovic M.D ., Maco D. and Takahara Y. "Th eory of hierarchical multi -l evel control" Academic Press, 1970.

=[

0 . 0074

-0.0011

0.0006

- 0.0 126

-0.0025

0.0042

- O. OOO IJ -0. 0004

0 . 05449 J [ 0 . 00668

Using this gain in the filter hierarchy, the filter was simulated. Figs. 2-5 show the states and estimates of the four states while fig. 6 and 7 show the control. Note that the e stimation of xI' x3 is poorer than that of x2' x since there is no measurement of BOD 4 Note that here a lmv order problem \vas used essentially to illustra t e th e approach . In fact, for the filter a 11 machine problem has been solved previously (Ref. 27). Th e d e c entralised stoch~stic control is quite appealing since it gives savings in computer storage, computation time and also since only low order problems are solved successively, numerical errors are reduced so that the contro ll er is computationally stable even for high order systems. 6 - CONCLUSIONS In this paper we have given the "state of the art" of dynamical hierarchical control. We have updated previous surveys on the subject in the light of the wealth of new results that have been obtained subsequent to the publication of previous surveys. Essentially , we have divided up the subject into three ma1n parts. In the first part we have dea lt with hierarchical optimisation methods which lead to open loop control . In this area, most of the recent advances that have been made are in the field of non-linear systems and this work has been particularly emphasised. In the second part we have dealt with recent work on computing closed loop control for bo th linear and non-linear systems whilst in the final part the stochastic case has· been treated. In sum, it is shown that the methods are quite practical and references have been given to the applications that have been made. The applications that have been made cou ld be

8

Chong C. Y. and Athans M. "On the perio dic coordination of linear stochastic systems" Proc. 6th IFAC World Congress, Boston, 1975.

9

Takahara Y "A multi-level structure for a class of dynamical optimisation problems " M.S. Thesis, Case Western Reserve University, Cleveland, USA, 1965.

10 Hassan M. and Singh M.C . "Optimisation of non-linear systems using a nEW two level method " Automatica 12, 3, July 1976. 11 Singh M.G. and Hassan M. "A two level prediction algorithm for non-linear syst ems" ,Automatica, Janv. 1977. 12 Javdan M.R. "On the use of Lagrange duality in multi-level control " Proc. IFAC Symposium on Large Scale Systems,Udine, June 1976. 13 Hassan M. and Singh M.G. '~ two level costate prediction method for non-linear

Hierarchical Optimisation and Control systems" Cambridge University Enginee27 ring Dept. Reprt n° CUED/F CAMS TR124, 197 6. Submitted for publication to Automatica. 14

15

16

Hassan X., Hurteau R., Singh M.C. and Titli A. " A three level costate prediction method for cont inu ous dynamical systems"T o be presented at the IFAC MVTS Fredericton 1977 . Singh U. C. and Ha ssan M. "Hi erarchical optimisation for non-linear dynamical systems with non-sep a rable cost functions". Submitted to Automatica. Mukhopadhyay B.K . and Malik O.P. "Optimal control of synchronous machine excita ti on by quasilinearisation" Proc. IEEvol. 119, I, p. 91 -9 8,1972

17

Singh M.C. "Dynamical hierarchical control" North Holland Publishing Co . 1977 (to appear)

18

Hassan M. and Singh M.C. "Optimisation of turbogenera tor transient performances by co stat e coordination " Cambridge University Engineering Dept . Report CUED/F CAMS TR125 (1976).

19

Jamshedi M. "Optimal control of non linear power systems by imbedding method" Automatica, 11, 633-6 36, 197 5 .

20

Singh M.C., Hassan M. and Titli A "Multi-level feedback control for interconnected dynamical systems using the prediction principle" IEEE Trans. SMC 6, 233-239, 1976.

21

Singh M. C. and Ti tl i A. "Closed l oop hierarchical control for non-linear systems using quasi-linearisation" Proc. 6th IFAC World Congress, Boston, 1975. Also Automatica, 11,541-546, 1975.

22

Mahmoud M., Vogt W. and Mickle M. "Multi-level control and optimisation using generalised gradients techniques" IJC (to appear)

23

Singh M.C. and Hassan M. "A closed loop hierarchical solution for the continuous time river pollution control problem". Automatica, 12, 261-264,1976.

24

Hassan M. and Titli A. "Closed loop hierarchical control for practical large scale ~stems using the prediction principle" ¥roc. of the Conference on Statistics and Computing, Cairo, 1976.

25

Singh M.C. "A feedback solution for the large scale infinite stage discrete time regulator and servomechanism problem " Computers and Electrical Engineering 3, 93-99, 1976.

26

Calvet J.L. "0pt imisation par calcul hierarchise et coordi nation en ligne des systemes dynamiques" 3rd Cycle Doctoral Thesis, Toulouse, 1976.

21

Hassan M. , Salut C., Singh M. C. and Titli A. "A ne'." decentralised filter for l arge scale linear interconnected dynami ca l systems " submitted for publication t o the IEEE Trans.AC

28

Hassan M. , Titli A., Singh M. C. "Multi-l evel feedback con trol for large scal e lin ear s t ochas tic systems " submitted for publication to the IEEE Trans. A. C.

29

Luenberger D. "Optimi sation by vector space methods" Wiley 19 69 .

30

Hassan M., Hurteau R., Singh M.C. and Titli A. "A netJ thr ee level al gorithm for riv e r pollution contr ol ". Submitted for presentation at th e I FAC Conferenc e "Syst eMS Approaches to Developin g Countries\~ Cairo , 1977.

_FIG . 1 _

M.G. Singh , A . T1tli . and M. Hassan

I 75

I 100

I 125

ITIME 150 ~ _FIG.5

_FIG .2

002,

0.016 TIME 125

150

0.008

0 _4

0

25

~

50 _

25 _FIG . ..;

50

mo

125

FIG. 6

75 _FIG.7

TIME

150

100

125

TIME 150