Control Strategies in A Stochastic System With Nonclassical Information Structure

Copyright © IFAC Large Scale Systems: Theory and Applications, Osaka, Japan, 2004

ELSEVIER

IFAC PUBLICATIONS www.elsevier.comllocatelifac

CONTROL STRATEGIES IN A STOCHASTIC SYSTEM WITH NON CLASSICAL INFORMATION STRUCTURE Zdzislaw Duda, Witold Brandys 1

In.5i-itllte of Automatic Contol. Si/fs;af! Technical Univf.1"sity. Poland

Abstract: In t.he paper control st.rategies realized in a stochastic system are present.ed . The syst.em composed of coupled linear subsyst.ems and quadratic performance index which shonkl be minimized , are considered. The synthesis of control laws realized in a t.wo-level hierarchical control structure with nonclassical informat.ion pattern is presented. Algorithms, in which it is possible to partially decompose calculations and to realize decentralized control, are proposed. The influence of available informat.ion on control quality is investigated in a simple example. Copyright © 2004 IFAC Keywords: Stochastic systems , Large scale systems. Suboptimal control. Optimal control

1. INTRODUCTION

crease amount of information to be transmitted to al1d processed by decision makers. A conflict. between local controllers is softened by the coordinator on the upper level.

This paper deals with a synthesis of a control laws for a st.ochastic system composed of coupled subsystems and quadratic performance index, which should be minimized.

Control problems with decent.ralized measurement information are studied in a team decision theory, as well as in the hierarchical control (Aoki, 1973; Chong and Athans. 1971: Ho, 1980) . The problems may Ut> complicat.ed, especially in the case of so called nonclassical information pattern, in which controllers do not have identical information. In (\Vitsenhausen , 1968) it is shown that a linear quadrat.ic gaussian case is nontrivial when the information pattern is non classical.

The quality of control depends on assumed information and control structures. In a classical information structure decision makers determine values of controls on the basis of all (t.he same) informat.ion collected from subsystems. Sometimes (e.g. in a large scale distributed system) the process of transmission and transformat. ion of information can be difficult to realize and thel1 the decel1tralization of information and control structures are proposed .

In the present. paper, a stochastic optimal control problem for a syst.elJl composed of linear subsystems int.eracting by means of control variables is considered and the quality of some cont.rol algorithms are discussed .

Control and optimization for large scale systems are usually based 011 a decomposition of a global system into subsystems and coordil1ation so as to decrease computational requirements and de1

Noteworthy attention are control strategies realized in a two-level hierarchical control st.ructure and nonclassical information pattern.

Supported by KBN Grant No .. TllA 012 in the period

2002-20().j

289

Control is realized in a two-level structure with a coordinator on the upper level and local controllers 011 the lower level. It is assumed that the local controllers have essential illformatioll of their subsystems, while t.he coordiuator has aggregated illforlllatioll 011 t.he whole system. The problem statemellt was discussed iu (Gessing , 1987) . where so called elast.ic constraillt was iutroduced. The two-fold interpretation of the ('outrol variable was utilized during the derivation of the cOlltrol laws. The cOlltrol variable Ui was treated as the decisiou variable for the i-th local cont.roller alld as a random variable for others. Owillg to this the solutioll of the problem had au analytical, lillear for1l1. Auother way of the solution was preseuted in (Duda. 2001), where the two-fold interpretation of the control variable was not required . The solutions presented in (Gessing, 1987; Duda, 2001) are suboptimal.

Uj(Zi) is a control law for the ith subsystem wit.h an argument Zi . The argument Zi represents all available information, which will be defined later.

The output equatious and the performance iudex written for the whole system can be described b.v the equatiolL'> (.t)

x= Bu+w

(5)

where

Qd = diag[QI Q2 ... Q,\I], Hd = diag[Hl H 2·..H ,\l]' B

= [Bij],

i,j

= 1, 2, ... ,!If

2. MODEL OF THE SYSTEM

The problem is to design optimal control laws UI = Ui(Zi), i = L 2, ... , !If for which the perfonnance index (5) takes a minimal value under constraint (4). The realization of the control 'Uj results from the relation 'Ui = Uj(Zi) '

Consider a system composed of !If subsyst.ems described by the equat.ion

3. PROBLEM FORMULATION

Present paper differs in the synthesis of the controL which leads to the optimal algorithm .

AI

(1)

The complexity and the effectiveness of a solution depend on assumed information and control structures.

where Xi, Ui, 11'i denote realizations of an output. control and disturbance vector variables, respectively, ofthe ith subsystem; Bij, i,j = 1,2, ... , AI are appropriate matrices. The sum appearing in (1) will be denoted by LN;'

In a one level structure with classical information pattern J = {y = [Yi , ..., y~y} admissible control law of the i-th decision maker is described as Ui = ai(Y) '

The model of the measurements has the form

Assume that the measurements Yi , i are aggregated to the form

:ri

= Bii'Ui + J~Bij'Uj + 11'; j

Yi

i- ;

=
(2)

where Yi and el are the vectors of the measurements and measurement errors. It is assumed that Wi aud ei are random variables with givell probability distn bution functions and independent of Wj , ej , i"# j , i,j = 1, 2, ... ,AI; I/>i( ' )' i = L 2, .... !If . is a given functioll . The realization of the ralldom variable Yi will be denoted by Yi . The performance index of t.he whole system has the form .\I

1= E[L(xrQ;Xi

+ ur HiUi)u,=a;(zt\]

(3)

i=1

where E denotes mean operat.ion, Xi is a random variable with realizations described by (1) and

111i

= L 2, ... , 11.[

= DiYi

where mi is the vector of lower dimension than Dj is an appropriate mat.rix.

(6)

Y,:

In the one level structure with classical information pattern J = {m = [mi, ... ,m~lT} admissible control law of the i-th decision maker is described as Ui = bi(m) .

•

Owing to low dimension of the vector 111j. I = 1, 2, .. . , A[ , the amount of information transmitted and converted by the decision maker may be decreased . This structure is justified for large scale system distributed system, iu which transmission of iuformation Yi , i = L .. . , !I[, to all decision makers (or to one central controller) is difficult to realize .

Reduction of inforInation makes the qualit~, of control worse. v.,'e lIIay improve the qualit.y of control in a two-level structure with the coordinator on t.he upper level and the local cont.rollers on the lower one.

5. SUBOPTIUAL SOLUTION TO THE PROBLE~I IN THE TWO-LEVEL STRUCTURE Denote

Assume t.hat the ith local controller receives from the appropriate suhs~'stem the measurement y, which is aggregated to the forlll (6) .

AI

Vi =

(12)

LBijUj J = 1 J;f:i

Inserting (1) and (12) into (3) gives

TII{' coordinator collects the transformed measurementlll; from all local controllers and in return t.ransmits to them t.he values of coordinating variahles Pi, The ith local controller transfers the decision ILi to it.s subsyst.em.

.\I

1= EL[urFiUi +2(Vi +Wi)TQiBiiUi + i=l

By the admissible control laws of the coordinator and the ith local controller are meant. the functions Pi = Mm) and Ui = a,[Yi.bi(m)]. i = L 2 ..... AI, respectively. For the realization of the random variables In and Yi the realized cont.rols detennined by the coordinator and the local cont.rollers take values Pi = b,(m) and lIi = ai(Yi,Pi), respectively.

(13)

where Vi

= B~QiBii + Hi.

The prohlem is to find optimal control laws Ui = aflYi,b?(m)] and Pi = b?(m). i = 1.2 ..... AI. which minimize the performance index (13) .

For the system considered with the assumed control and information structures, among the admissihle control laws, the optimal cont.rollaws are to be found for which the performance index (5) under constraint (4) is minimized.

5.1 Syntht.sis of local control law.:<- algor'ithm 1

In (Gessing . 1987) t.he problell1 was solved under assumption t.hat t.he functions o,j[Yi.bj(m)] and b;(m) fulfil the constraint E1ma;[(Yi,bi(m)] = E1mCLi(Yi,P;) = Pi , i = L. ... M

4. OPTI1IAL SOLUTION TO THE PROBLE~I WITH CLASSICAL INFORMATION PATTERN

(14)

where Elm denot.es t.he condit.ionaimean. given m.

Consider two structures with t.he classical information pattern and the admissible control laws described by Ui = airy) and Ui = bi(m), respectively.

The local control laws aj(.) were found by local minimizat.ioll of t.he performance index

The performance index (5) with (4) can be written in the form

under the constraints (1) and (14).

r = E(XrQiXi + ur HjUi)u,=a;(.)

This approach was justified as follows. In connection wit.h the inforlllat.ioll available for decisioll makers the variable Ui plays the decision role (dt<; lti) for t.he ith local cont.roller alld the ralldom role (mu Uj) for the ot.her cOllt.rollers.

1= E(uTVu+2uTBTQdW+wTQdW) (7) where V

= Hd + BTQdB

The opt.imal cont.rol laws in the control structures result from t.he minimization of the performance index (7) and have the form UO

= aryl = [(w

(8)

UO

= a(m) =

(9)

[(w

where [( = -\,--IBTQd and w=Elmw.

Because t.he lIIinimizatioll is perforll1ed with respect to (du: ILi), then it. is equivalent to the 10CAlI minimizat.ion. Using the Lagrange multipliers lIlC'thod it was shown that the local cont.rol laws have the form

w

(16)

where Ki = _1/;-1 B~Q, and U-'j

Inselting (8) or (9) into (7) gives

E im,Wi

If = E(WTQdW - wT [(T\/I\w) 1;°

=

E(WTQdW -wTKTVKw)

(15)

= E ly, Wi .

iDi =

o

The yalue of u? is t.he realization of control determined by the ith controller for given Pi (transmitted from the coordinator) and given .l/i necessary for determillatioll of the estimates li'i and Il'i'

(10)

(11)

respectively.

291

5.2 Synthtsis of local control laws- algorithm

Inselt.ing (24) iuto (20) gives the value of the perfom8nce index (3) resulting frOI11 strategies (16) and (24) applied to the system (1) and realized in a two-level control structure:

;2

In (Duda , 2001) it was assumed t hat the ith suhsyst.em was described by· the equation

xi = BiiUj +vi

( 17)

+Wj

(25)

where 5. 4 Discussion of thf IISfflllne"" of local controllers

(18) It is possible that. t.he ith local controller should not realize the control accordi ng to (16). but t.o realize the cOlltrol u7 = pi·

The realizat.ion of the nllldolll variable vi is t.he hest. est.imat.e of the interaction , hased on the inforlllation of the coordinator.

Choose from (13) all the cOlnponents containing Uj and denote this expression by [i .

In (Duda , 2001) it was shown that the local control laws have the forlll (16).

Performing some transformations produces

5.3 Synthesis of optimal control la·ws for the

[i

= E[U;Wi + L

c007-dinator

+2u; L(BJ;Qj L Bj.Uk)+ Ni .4i.j

The equation (16) can he written for the whole system in the form

+2 L

where p = [pI· ... pir]T and

[(cl

= diaY[[(l ...... K M

+ B!JQjBji)Ui+

+2u; L BJ;Qjwj + 2W;QiBiiU;] j#i

].

Inserting (19) and (4) into (5) gives

= E[(pTVp + 2pT BTQdW)p=b(m)] +

u;(B5QiBii

j#i

( 19)

hSllb

BIQjBj i)Uj+

j#i

(26)

In the one-level structure the control has the fOrlll ui = pi and

s(20)

Ii = E[piT(V; +

where

LBJ,QjBjj)pi+ Ni

oS

= E[(w -

w)T [(JVKd(w - w)

+

T

+2Pi L(B};Qj L BjI,Uk)+ Ni k#i.j

+wTQdW-2(w-wfKJBTQdW] (21) +2

The problem of the coordinator is to det.ermine the optimal control laws balm) which minimize the performance index (20) writ.ten in the form

L u;(BT/JiB;i + B!JQjBj;)pi+ N i

T +2Pi L

BIQjWj + 2W;Qi B i;pi]

(27)

Ni h s ub

= E{E1m[(pT y ' p +

2pT BTQctw)p=b(m,]} + s In t.he two-level strneture the local control has the (22) form ui = pi + I"'-i(Wi - Wi) , and

It can be transformed to the minimizat.ion of the expression (for given m)

I~ = E[uiT(V,

+L

BJ;QjBj,)ui+

#i

T +2ui L(BJ;Qj L Bjk-Uk)+ #i k#i.j

(23) with respect. to the variable p.

+2

L u;(B5QiBii + B!JQjBji)ui+ j#i

Taking the derivative in (23) with respect to p and setting t.o zero gives

•

+2uiT L

BJ;Qjwj + 2w;Q;B"ui]

(28)

Ni

(24)

Then

where K was defined in the section 4.

t.l

The value of pi is t.he realization of control determined by the coordinator and transmitted to t.he ith local controller.

+L #i

292

i

= I;

-

14 = -t1"[I\; (" i+

BIQjBj.)KiPw,] - 2f1·(Q,B ii K,Pw.)

where PW1

=

wd T .

E(Wi - Wd(Wi -

+ L(B~QiBji

If tll < 0 t.hen it.h local decision maker should not appear in t.he two-level structure. In this caSE' the local control law has the form (16) with Ki = O.

+ LB~Qjfi.lj Then

(33)

where

The optimal control laws of the coordinator have the form (24).

It is easy to show that the optimal control law results from the minimization of the performance index min

E 1y,.m[ut(V;

[Y I .b, ("')1

+ 2ut L(B~Qj L BjkUk) Ni k#i ,j

The performallce index will be delloted by the numerical example) .

j#i

+ 2ut L B~QjWj+ j#i

Consider a simple system composed of two subsystems and described by the equations Xl =

= min[uT(V; + LB~QjBj ;)Ui + 211,;Qi B ii Ui+

X2

j#i

+2uT L(B~Qj L BjkPk) Ni k#i .j +2 LP;(BJ;QiBii

(in

7. EXAMPLE

Ni

Il,

12

+ LB~QjBj;)Ui+

UJ(B~QiBii + BLQjBji)Ui + 2wtQiBiiUd

+2 L

= -Ri B~Qi'

According to (33) the ith controller should know the matrices Bji, (j #- i) which determine the interaction between subsystems.

Consider the equation (26) and find the optimal control law (Li(Yi ,Pi), which minimizes the performance index [i.

II I

Ki

l

Notice, that (33) has the same form as (16) , where Ki is replaced by Ki·

6. OPTIMAL SOLUTION TO THE PROBLEM IN THE TWO-LEVEL STRUCTURE

=

+ B~Q i ((:-; l

j#i

The perforlllance index in the modified structure will be denoted by J:t. ub (in the nUlllerical example).

S iO

+ B~QjBjj)pj+

Ni

i

Bll Ul

B12U2

+

+ Wl

= B22U2 + B21 Ul + W2

(34)

+ 2uT L B~QjWj+ Yl=CIWI+el

Ni

+ B'LQjBji)Ui]

= C2W2 + ~

Y2

(29)

(35)

Ni

where

ml = DIYl

(36)

m2 = D 2 Y2

(30)

for which Taking the derivative in (29) with respect to and setting to zero gives

uf = _R;l[LB~Qj j#i

+ L(B~QiBji

Ui

B~

= [2 1], Bi; = [3 1) , BI; = [5 1]. B~ = [1 4] ,

L Bjk'Pk'+ k#i .j

+ B~QjBjj)pj+

Cl =

N i

+ LB~QjWj +B~QiWd where

EWl

+ L B~QjBji

(32)

Pw ,

Ni

Using (30) gives Eel

Pi

= -Ril[LB~Qj Ni

n,

The disturbances WI . W2, distributions defined by

(31)

Ni

Ri = V;

[~

L BjkPd k#i ,j

Pel

293

= [1 =

= [1 1] T

= [~

=

el

and

~],

pw •

,

=

Ee2 Peo

~

have gaussian

= [1

2 ( . EW2

[i ~].

[~ ~] ,

C2

1( ,

[i ~] .

= [ 1 0] T

= [~

n'

,

8. CONCLUSIONS

The performance index has the form 2

1= EI)xTQ;xj

+ uT H,uil

In t he paper are presented the subopt.imal and optimal control strategies of the st.ochastic system realized in one and two-level hierarchical st.ructures.

(37)

;=1

where QI =

Q2 =

[3-2] -2 3

[21 11] '

= [ 1] .

. HI

H2 =

In the two-level st.ruct.ure the local controllers have decentralized a priori information and decent.ralized measurelllent.s . It is shown t.hat local control laws are linear functions of disturbance estimat.es and can be realized in decent.ralized way completely. The coordinator collects an aggregated informat.ion from local subsystems and determines some " directions" to local controllers. The cont.rol law is linear function of disturbance est.imate. Owing to aggregation, amot1l1t of information transmitted and transformed by decision makers can be decreased.

[2] ,

The estilllates tL't and U'i i = 1. 2, can be detennined by using t.he cOllvellt.ional formulae

rh

= EWj + PW,y,P;.~,(Yi

11\ = EWi

- Eyil

+ Pw,m,P;;",lm, (111; -

(38)

Emt) (39)

where P WiYi = E(Wi - EWi)(Yi - EYi)T, PYiYi = E(Yi - EYi)(Yi - EYi)T and PW1ml = E(Wi EWj)(mj - Em;)T , Pm1m , = E(mi - Emi)(mjEmif· From (10), (11) and (25) it was found 14.6944, IjO = 18.5837, h.ub = 15.8718.

The influence of ., aggregation of information" on t.he quality of control can be investigated.

If =

REFERENCES Aoki. 1\1. (1973). On decentralized linear stochastic control problems with quadratic cost. IEEE Tmns. Aut. ContrallS, 243- 250. Chong, C. Y. and t-.1. Athans (1971). 011 the stochastic coutrol of linear syst.ems wit.h differeut iuformat.ion sets. IEEE T,.(ws. Ant. Control 16, 423-430. Duda, Z. (2001). Two-level hierarchical control in a large scale ~'Ystem. Proc of the ECC. Gessing, R. (1987). Two-level hierarchical control for linear quadratic problem related to a static system. Int. J. Control 46, 1251-1259. Ho, Y.C. (1980). Team decision theory and information structures. Proc. lEE. 6S. 644-654. Witsenhausen, H.S. (1968) . A counterexample in stochastic optimum control. SIAM 1. Contral 6, 131-147.

Determine t.l i , i = 1,2. From (7) it results that !::..II = -0.6509 and !::..[2 = 3.3628. Thus, the first local controller should not appear in the two-level structure. while t.he second should. In the modified control structure the opt.imal value ofthe performance index is I!l.'sub = 15.2209. The optimal value of the performance index is 12 = 14.85. In t.he Tab. 1 and Tab. 2 are presented the values of the perfonnance index for the strategies discussed in the paper. Table 1. The quality of the suboptimal controls controls in t.he t.wo-level struct.ure. 12.,wb

Dj = 1. Dj = O. Dj = 1 , Dj = O. Dj = [1

D2 = 1 D2 = 0 D2 = 0 D2 = 1 IJ. D2 = 0

14.69 20.:3:3 15.12 19.90 1.5.87

I 2m.... ub 14.69 15.66 15.12 15.23 15.22

tJ./ 1

tJ./ 2

0 -Hi7 0 -4.67 -0.65

0 3.36 3.36 0 3.36

Table 2. The quality of control strategies in the one and t.wo-level structures. Dj

DJ DJ

= 1 . D2 = 1 = O. D2 = 0 = =

1. D2 = 0 DJ 0 , D2 = 1 DJ = [1 IJ. D2 = 0

/*0 j 14.69 19.02 18.48 1.5.23 18.59

I},

Jfn 2._"iub

14.69 J.t.87 14.82 14.75 14.85

14.W

15.66 15.12 15.23 15.22

Notice. that for D = I and D2 = I t.he values of the performance index take t.he same value for all st.ructures presented in the paper.

294