Decentralized, Stochastic Control for a LQ-Static Problem

Copyright © IFAC Distributed Intelligence Systems, Varna. Bulgaria, J 988

DECENTRALIZED, STOCHASTIC CONTROL FOR A LQ-STATIC PROBLEM R. Gessing and Z. Duda Institute of Automatic Control, Silesian Technical University, Pstrowskiego 16, 44-101 Gliwice, Poland

Abstract. An original problem statement for the optimal control laws design is presented. A large scale system composed of M linear static subsystems wi th an interac tion and q U3.dra tic peri'orrrance index are considered. A two-le v e l hi era rchical control structure is assumed, in which a coordinator and local controllers have access to different infornation. The so called elastic constraint (Gessing, 1985) is used for coordination. For the problem the possibility of partial decompositicn of calculations and a dec e ntralization of the control, as well as an anal1tical form of the o ptimal laws are obtainec. . An influence of the particular subsystem on the control quality is investigated • Keywords. Large-scale systems ; optimal control; stochastic control control system synthesis; liuear-qUBdratic problem. INTRO CUCTION

MODEL OF A SYSTEM

The p roblems with d ifferent controllers a nd differe nt available inforrration are stud ied in the t eam dec ision theory, as well as in the hi e rarchical con t rol (Chong ann. Athans , 1979 ; Chu, 1972 I Ho , 1980 ; Ho a nd Chu, 1972 , Sande ll a nd othe rs, 1978) An e ffe ctivene ss of s olutio n d epen~s on the mode l of the system a nd the informat ion structure • Of co urs e an e sse ntial role plays a lso a problem stat ement.

Let us consider a static linear, larg e scale system composed of M interacting subsystems and described by the equations Xi

= Bi i ui

+

vi + w;,

i = 1,2, ••• M

(1)

M

v* i

L.A* x j=1 ij j

i '" 1,2, ••• M

j!i

In the pres ent paper a stochastic optimal control probl e m is co n s i de r ed for a system composed of the stat ic linear inte racting s ubsyst e ms in the cas e of qU3. c;ratic perfo rmance index and a two-level information struct ure . The primary problem statement ",as present ed a t the 4-th Symposium on "Stochastic i''J :1trol'' in Vilnius a nd the more a d vanced version is actU3.lly printe d (G essing , 1987 ) • The present paper diffe rs from that by the way of inte raction of the s ubsyst e ms. a:o r f! the s ubsystems interact by means of th.; o utput variables, while the r e by means of the control variabl '~ 5 .

where xi'Ui'v~,w~ denote th~ output,control, interaction and d isturl::e.nce vector ~riables of the i-th subsystem, respectively ; Bi i , A~j' i,j = 1,2, ••• M , i ~ j are appropriate matrices. The sum appearing in (2) will be denote d below by ~ • j"i The mo del of the measurements has the form (3) (4)

In the paper the optimal c ontrol laws in a two-level, hierarchical, partially dec e ntralized control structure are consiQe red • It is assumed that the coordinator, as well as the local controll e rs mve d ifferent info rmat ion. The so-called elastic constraint of the control variable is used for coordination, similarly as in (Gessing, 1987) •

whe re Yi'€i are the vectors of the measurements and measurement errors ; m. is the l vector of the transformed measurement of a low d imension towards that of Y ; Ci,F i i are appropriate matrices.

•

The two f ol d int e rpretation of the control variabl e is utiliz ed 1 uring the de rivation of the control laws. The same control variable i s treated as the decision variable for the local controller and as a ran ciom varia bl e for the coordinator. Owing to this and to the elas tic constraint, the solution of the problem ms the anali tical, linear form.

We assume that the noises wi,e are indei pendent of w;, e j , i"j, i,j - 1,2, ••• M and 0 f u l ' 1 - 1,2, ••• M • The primary performance index of the whole syste m, which we would like to minimize ras the form

175

176

R. Gessing and Z. Duda

M T TT) I • ~(XiQixi + 2Xi Gi u i + uiHiui 1-1

CONTROL AND COMMUNICATION STRUCTURES

(5)

where the augmented matrix

[Of Gij is Gi Hi nonnegative and Hi is positive definite.

Let us consider a two-level control structure with the coordinator on the higher level and the local controllers of the lower level Fig. 1 •

Let us introduce the augmented vectors T Of x ,",[x 1 ,x2 ,

•••

'* *T *T w -[w1 ,w2 '

TT "M] ,

[T T u • u1'~

*T T

•••

WM

COORDINATOR

TJT '1-1,

1

(6 )

and the ma trices

B".. d i ag [B" ~ 11' B22' •••B"MM] '

~l

H .. diag [H 1 ,H 2 , where for

2> ij .. 0

for i-j

'

AY< ..

[ A ~j .

and

6 ij

6 ij ] (7)

.. 1

1-TH LOCAL

M-TU LOCAL

CONTROLLER

CONTROLLER

y~

i,lj.

YM

Then (1) and (5) may be written in the form

x _ Blf U + A* x + wit;

T

T

(8)

T

I = x Ox + 2x Gu + u Hu

(9) A priori in fo rma tion for the problem consists of the model (1)-(5) or (14), (13), (3), (4), (5), as well as the appropriate probability distribution •

After denoting by B .. AB* , w .. Aw"'" , A .. (1 _ A*)-1

(where 1 is a diagon:ll unit matrix)and assuming that the inverse matrix exists we have ( 11) It is easy to notice that the i-th subsystem may be described by the equation

(12)

Let us assume that the i-th local controller receives from the appropriate subsystem the measurement y i' which is transformed and transmitted to the coordinator. The coordinator collects the transformed measurement m from all the local controi llers and in return transmits to them the values of coordinating variables Pi' The i-th local controller transfers the decision on u to its subsystem. i We see that all decision-makers have different a posteriori information. The measurement Yi with the coordinating variable

~n~ information defined by m .. [m;,~, •• ~l represent a posteriori information of the i-th local controller and the coordinator, respectively. Owing to the low dimensions of the vectors m , i=1,2 •• M, i the amount of in fo rma tion transmitted and converted by coordinator may be decreased.

Pi

where the matrices B ' B ij , A 11 , Aij U result from (11) and (10). The formula (12) after introducing the vectors

..

(13) can be written in the form (14)

Let us notice that the noises are independent •

Fig. 1. The control struc ture

As the admissible control laws of the i-th local controller and the coordinator we mean the functions u i = ait Yi'P ) and i Pi • bi(m), i = 1,2, ••• M, respectively, each of which maps the set of the vectors Yi' P 1 and m to the sets of the vectors u and Pi' respectively. 1

Decentralized, Stochastic Control

Add1t1oaaUy we assume t~t the !unct1ons a (-) and b (-) fulfil the constt1!int 1 i E [a i (Yi' P1)! m] • P1

(15)

where E( -Im) denotes the conditional mean, g1vea m • The dependence (15) defines the so-called elastic constI9.1nt of the control (Gessing, 1985, 1987) • Let us define the secondary performance iRdex, which for the considered structure can be minimized r(a,b).

E

~(xTQ x +2xTG u +uT1H u) i.1 i i i i i i 1 i

177

The optimal control law a~ results from solving the min1mization problem (17) with the constI1lint (15). This can be done by using the method described by Gessing ( 1985). Using the method of the LagI1inge multipliers we can take into account the constt1!int (15) in (17). Then the problem may be tI1insformed to the form

-S1(m,Pi,li)· M;n Elm [(Baai+vi+wi+zi) T 1

• 0i(B a

&

1+v i +w i+z( +2 (Baa i+ v i+wi+Zi) T.

(16) (18)

where u i • ai[Yi,bi(m»),a .[a ,a , •• aM]' 1 2 b .. [b 1 , b 2 , •• b J • M

where

li

is the LagI1lQge multiplier •

The fulfilment of the constraint (15) is ensured by an appropriate choice of the constant multiplier li •

Problem formulation For the considered system, assumed control and information structures, among admissible control laws, the optimal control laws o 0 u i • ai(Yi'Pi) and Pi· bi(mJ,i .. 1,2, •• M are to be found for which the perfor1l'/3.nce index (16) is minimized. SOLtJrION OF THE PROBLEM

In order to solve the minimization problem in (18) with respect to the function a i we may tt1!nsform the expression (18) by replacing Min E aad a with E MinE 1 11 lm Im u I y l' m i i and ui' respectively in the bI1icket Next, after performing the El m Opel1lYl' tion we obtain

[-1.

In connection with the available information for the i-th controller and the coordinator the twofold treate ment of the con trol va ria ble Ut is conseQ uent (G,a ssing. 1987). nam.&ly , in th~ courSe of the control realization -the variable u i • a i ey i'P ) is the decision variable i (dec u ) but it is the I1lndoDl variable i (I1ln uJ for the coordinator and for the j-th (j~i)controller. This must be taken Into account during the synthesis of the o optimal control law ai. Thus,duning the derivation of a~ the variable u appear_ i ing in (14) plays the dec u role, while i the VEl riables u ' j"i, appearing in vi j the I9.n u role. So, the minimization only j with respect to dec u is performed. This is equivalent to the ~erformance of a local optimization of the i-th subsystem by the i-th local controller, and only then the derived a i takes the admissible form.

o Derivation of the Optimal Law a 1 Let us denote Si(m,pJ- :1n E!m[(Baai+vi+wi+zJT i • (B 11a 1+v i +w i +z i) +

.°1 •

(19) where Vi

-

zi A

E(vi!m)

(20)

E(zi lm). E(zi/Yi,m)

(21)

wi • E(w i !Yl'm) .. E(wi/y i )

(22)

Diffeventiating in (19) with respect to u and equating to zero, we obtain after i some transformations

Substitutillg(24)into (15), calculatiJllg li and subst1tut~ this into (24)we obtain the optimal control law a in the form i ( 25)

178

R. Gessing and Z. Duda

where Wi - E(wi'm) - E(wil mi ) Substituting (25) and, resulting from (13) and (20) Vi -

Lj,li

(26)

BijP j

into (19), we obtain after transformations Si(m,P i )- piv iPi+2piCBiictGiHL BijP j )+ j,l1

i)'

+(311 B1lj)TC i (JiiB 1j Pj )+ 2Pi(Bi i Ci +G o

(wt.Zf • 2(£ BijPj) TCi(Wi+Zi )+

The miniasl value of the perfOrDliRCe index (16), which for the two-level control structure we denote by 12 , results from (33) aDd is determined by IO 2

si (V)

where

(28)

_

E

S

(m)

( 34)

DISCUSSION OF THE USEFULNESS OF LOCAL CONTROLLERS Let us consider the case of a one level control structure with one central decision-~ker having the same info~tion as the coordinator in two level structure, considered abpve. In this case the admisi_ ble control law has the form I u - ai(m). i Using (19),(25), (V), (28) and the ~ct that vi - Vi' ~i .. wi we can see that the control law in one level structure is determined by (32) with p replaced by u and the mini~l value of the appropriate perforDlince index is determined by

( 29)

The second term in (28) after averaging is equal zero, because w is independent i of wj(j,li). Also si is independent of Pj' j - 1,2, ••• M • Derivation of the Optimal Law boi Now, we use the notation applied in (9), TT (11).1 We denote additionallyp - [ P1,P2' •• ~]T - - T - T •• ~ , W - [(W1+W1) • ("2 +Z2) , ••• - +-z )T] T aad •• Cw M M

Thus we have

where in accordance with (25), (13), (26)

M

sCm,p) -

L

(30)

Si (m, Pi)

i-1 ( 37)

Substituting (V) into (30) we obtain T T T T S(m,p)- p (H+B QB+G B+B G)p + T

T

T

From the equation (37) and the assumption of the independence of the noises w and i wj , i,lj , results that (36) ~y be written in the form

M

+ 2p (B C+G ) w + L St i-1 After differentia ting (31) with respect to p and eqUiting to zero we obtain the O control law b in the form

T

T

T -1

p - _(H+B QB+G B+B G)

T

T

CB C+G )w

Substituting (32) into (31) we have

(32) where (39)

Decentralized, Stochastic Control

It ls worthwhile to apply the consldered two-level control structure lf 11-12~ o. It ls posslble, that the l-th local contra: Her should not appear ln the two-level structure. Now, we w111 try to answer the questlon of in which case the l-th local controller should appear ln the two-level structure • Let us choose from (36) all the cymponents conta inl~ t.. u 1 and denoted by AI • The term AI deteraines the value by which the performance lndex ls decreased owing to the control action of the i-th local controller. The apperaJ\ce of the l-th local controller i O. caD be justified HilI>

179

rate i-th subsystea wlth 1\0 1ateractloD (Yi • 0) in the aOdel (1), the controls of a declsioa.-aker having 1nfo~tlo11 Y or i a haYe the form, respectively 1 U

1 -1 T T A 1 - - Vi (B l1Q1+G 1 ) v 1

1,1· -; a nd

where

1

(46)

lit • w;

The value t.. u 1 • ui-Pi - uCPi deflaes the correctioa of the coordinator declsloa determ1Aed by the 1-th local controller •

One can notice, that Exa!lJ)le AI

T

i

T

• E [AUi(VC r,ri Bj1Qj Bji)

Il

ui -

TT'" ] - 2 AU i J1iBjiQjWf

(40)

Afier transforlDi tions, using (Yl), we obtain the condition Ali

tr [Ki
B~iQljJKi

-

A computer program has been constructed to test the problem mentioned above. It ls written in TURBO B\SCAL on IBM/PC. This program _s used to solve an exalllple below. Let us conslder a simple system composed of two subsystems for which t

rx 11 ' ~ J T

• w1

)1:1 -

1 2 JT [Y1' Y1

1 2 JT ' e1 • [81 , e1

.

P~i

- E

(Wl-WJ(~l-Wi)

T

.

1113.-

( 42)

(43)

"'w Since PH> 0 then the sufficient conditlon for the fulfilment of (41) is T( ~ T Q B ) K 2K T ) T Q Kl V1 - j~ Bji j ji C i :r,r/ji j > 0

44 The condltion (44) can be fulfilled it the influeace of the control u on the output i Xi is suffieiently strong in comparison to that on other outputs x ' (j,il) • j

G2 •

0

-It

, A21 • [-1

, H1 •

:

2

1

[ ~ 1' C2 •

F

0

2

•

, G1 •

, H2 •

C1 -

1

2"J T w1

3

-It

-1 ] 1 ' Q2 •

·L:

•

• B22 •

-1 J T

A12 - [-1 Q 1

J,o

1] T

B11 - [2 where tr ( .) denotes the trace of the trix and

rw1110'

x1 •

-1) [0 0] T , (48)

2

, F1 • [ 1

¥

...

1

J

.....

The d1sturb:iaces w1 and w2 ....ve gaussiaa distributlons defined by t

Ew;. [:] Ew;.

2 (49)

DISCUSS ION OF THE SOLUTION

Let us consider the case of a oae level structure w1 th one central dec ision-Blker havlng complete informa tion. The optiBll control law results frolll solvlng the m1almlza tiOIl problem (9) vl th the constraint (11). Thell we obtain the control law in the fOrlll T T T -1 T T u - -(H+B QB+G B+B G) (B Q+G ) w

(45)

Ito ls easy to noticed t~t the control law (b ) of the coordinator descrlbed by (32) resulting from (45) by replacing u aJ\d w by p and Vi • Thus the optl_1 control law of the coordinator fulfils the cOnveatlOnlll certainty equlvalence principle. Let us note that in the case of the sepe.-

,

The distYrbances 8 and e 2 1 ss1an distributloa defined by

(50)

The optllal control laws of the local controllers in accordaJlce w1 th (25) and (13)

180

R. Gessing and Z. Duda

REFERENCES

blve the form :

AstrOm, K.J. (1970). In troduc tion to Stochastic Control Theory,AcademIc Press, (51)

(52) The optimal control laws of the coordinator ha ve the fo rm (53) (54) hot

h*

_11

_ ."

The estimates w , w , w and w can be 2 2 1 1 determined by using the conventional formulae for the calculation of the conditional mean. i Let us determine AI, i = 1,2. Us ing the fonnula (41' we obtain

.

AI~ .. 5.4

(55)

Thus, the 1-th local controller should no t appear in the two-level control structure. while the second one should appear indeed . Conclusions The contribution of the paper is partially in the problem statement in which the twofold treatement of the control variables, as well as the elastic constraint has an essential meaning a nd pa rtially in its s olution. All control laws take the form of linear functions of appropriate estimate!';. The optimal control law of the coord inator is de tennined by the ordinary certainty equivalence principle. The correction of the coordinator decision determined by the local controller is defined by the d ifference between the controls resulting from the solution of the separate subsystem problem with no interaction, for a de c ision-maker having the in fo rma tion of the local controller and coordinator, respectively. It is possible that one or several local controllers should not appear in the tv,Qlevel structure • Then the local optimization action is not justiHed. The cond itions derived in the paper allow to investigate this problem without a simulation of the system • Acknowledgement The paper ",as supported by the departmental program No. RP.I.02 coordinated by the Institute of Automatic Control of the Warsaw Technical University.

New York.

Chong, C.Y., and M.Athans (1971). On the Stochastic Control of Linear Systems with Different Information Sets. I.E.E~. Trans. a utom.Control..16, 15,

423-4

•

Chu, K.C. ( 1972). Team decision theory and information structures in optimal control problems, Part. 11. I.E.E.E. Trans. automatic Control, 17, 1, 22-28. GessIng, R. (1985). TWO-level hierarchical control for stochastic optimal resourceallocation. Int.J.Control. 41,1, 161-175 • Gessing, R. (1987). Two-level stochastic control for the linear quadratic problem related to a static system. l!U.d.. Control 46, 4, 1251-1259. Ho, Y.C. (1~8()). Team Decision Theory and Information Structures. Proc.lnt.elect. ~rs •• 68~ 6, 644-654 • and .C. Chu (1972). Team de ciHo, sion theory and information structures in optimal control problems, Part.I. I.E.~E. Trans. automatic Control, 17,

roe.

1, 1

21.

Sandell , N.R " P. Vara iya, M.A thans and M.G. Safonov ( 1978). Survey of Decentralized Control Methods for Large Scale Systems. I.E.EaE, Trans. a utomatic. Control, 23, 2, 1 8:1 28 •