A Stochastic Aspect of Decentralized Control

A STOCHASTIC ASPECT OF DECENTRALIZED CONTROL A. Wahdan and B. Cheneveaux Laboratoire d'Automatique E.N.S.M . Nantes, France

Witsenhausen (ref. 1) has called "non-classical information pattern" in the control problem. For which the separation principle (ref. 3, 4) (which states that the regulator of the linear quadratic gaussian problem may be devided into a seperate estimation part of the state vector and another optimization part), does not hold. Due to the work of Witsenhausen (ref.1, S and 6) a decentralized dynamic structure of control began to be treated. The interaction between information and control is clear in many works in the literature (ref. 7, 8 and 9) for example. Some non classical information patterns are treated previously, such as partially nisted information structure (ref. 8 and 10) and one step or n- step delayed information (ref. 9, 11 12 and 13). We must take into account the impact on the cost of communications (ref.14) due to this delaying of informations. Another point of view (ref.1S) considers that each control center makes its estimation of the complete system state vector using its own measurements, then generates its control in co-operation with the other centers to attain certain performance of the total system. But we remark that the estimators have the same dimension of the original system and this may pose a heavy restriction. The linear quadratic gaussian problem can be treated differently, (ref. 16, 17, 18 and 19) in a centralized manner but without estimation, the control signal is restricted to be a linear transformation of the observations at actual time. Also a deterministic output feedback problem is treated (ref. 20, 21 and 22). From these studies we can remark that in either the deterministic or the stochastic cases, the output feedback control must obey an asymptotic stability condition and that it results in a suboptimal performance index. In the present paper we treat the large linear quadratic gaussian problem by considering a decentralized scheme of control in which the dynamic controllers have a smaller dimensions than the original system and their structure can be fixed a priori. The ' approach of the dynamic output feedback is adopted to calculate the decentralized regulators parameters in a suboptimal

ABSTRACT. In this paper we aim to calculate a decentralized dynamic output feedback regulator for the linear multi variable stochastic process. Effectively the classical seperation principle does not hold to this structure of control, so we use a direct search method to verify our aim. With the decentralized control structure we try to get a satisfactory suboptimal performance which may represent a manner to avoid some of the criticisms mentioned by Variya (ref.2) about the hierarchical control. Also we avoid the construction of decentralized estimators in which the control agents make complete estimations of the 'state vector. The dynamic output feedback regulators fed by partial information of the output vector are constructed by a direct search algorithm which defines the values of the rpgulator parameters in the finite time also in the infinite time cases. The noise-free output and the static regulators are special cases of the considered structure. Finally a numerical example is treated to present the utility of the algorithm. INTRODUCTION. There has been an increasing amount of interest in recent years in the study of decentralized control. The concept is to achieve the control by a set of controllers (decision makers) where each of them acts on a different data bases and affects different aspects of the total system performance, with or without comunication between agents. This form of control configuration arises when the system is large, in this case the centralized scheme of control results in prohibtive data handling and computational requirements especially when the memory size is limited, also the decentralized form is reliable and much easy to implement for instance by using parallel small computers. The use of this decentralized form or more generally the incomplete form (in the sense of incomplete information to the decision makers) in the control of larc;J-:. stochastic systems arises what .

113

114

A. Wahdan and B. Cheneveaux

fashion (minimizing a quadratic performance criterion in respecting the given structure of control) . To do that, the original stochastic problem is transformed into a deterministic one which depends on the original system parameters, criterion weighting matrices and the sufficient statistical charactristics of the state and output noises (in our case the first two moments are considered sufficient to charactrise the noise). This deterministic problem is a parametric optimization one which can be solved using a dynamic programming algorithm results in a two-point boundary value problem which can be solved iteratively in the finite time case. For the steady state case we adopt a nonlinear programming algorithm (ref. 23) to solve the parametric optimization problem directly, under the condition of asymptotic stability, which gives the regulators parameters. This paper is organized as follows : section 2 treats the problem formulation, section 3 represents the mathematical procedure to solve our problem with some emphasis to the form of the criterion used. In section 4, we give some numerical remarks, useful in the numerical solution, then section 5 treats an example of the literature (stirred tank problem (Ref. 3» and some numerical results are given to illustrate our approach. PROBLEM FORMULATION The system shown in fig. represents the frame of our dynamic decentralized control scheme. Here we consider two controllers but the generalization to more decision centers is straightforward.

u

ml 1 u

Dynamic

1 l

Irontroller 1 (si)

Each center us e s a part of the noisy outputs as its inputs t o e laborate its decisions (control) in c o - o p e rat .. o n with the other centers to make th e total system satisfactory. In mathematical terms, the nth order system i s descri be d by the difference equations : x ( k+l)=A x ( k)+B U (k ) +B U (k )+GW(k) 1 1 2 2 Yl(k ) =C Y2(k)=C

l

X(k ) +H v (k ) l l

(2 )

2

x ( k)+H v (k) 2 2

(3 )

where : x(k) : is n- dimensional state vector, u (k) and u (k) : are m and m dimensional l 2 l 2 control vectors, m = m + m , 2 l w(k) : is £ - dimens i onal white random vector which represents the state noise, with zero mean values and covariance matrix W(k) x(o) : is considered as a white random vector mean value x and covariance matrix P , o 0 Yl(k) and Y2(k) are . ub-vectors of measurements with dimensions r and r respectively l 2 r = r + r 2 l v (k) and v (k) are PI and P dimensional 2 l 2 white random vectors with zero mean values and covariance matrices V (k) and V (k) l 2 respectively, P = P I + P 2 k : represents the time. A, B , B , G, Cl' C ' HI and H2 are constant l 2 2 parameters known matrices with appropriate dimensions and the random variables are independent . We consider the following dynamic regulator for the control station i Zi (k+l) = Di (k)Zi (k) + Si (k) Yi (k)

(4)

u. (k)

(5)

~

-- - - --. -

(1)

T. (k)Z . (k)+F . (k) y . (k) ~

~

~

~

where Zi(k) is the regulator state vector, with dimension

-~

n

th

1

order linE'ar

- -

-. Ylrl y

Stochastic dynamic ~

.

system

.

-

1 yl

.

2 y

r2

2 u

1 2

m2

Dynamic Controller 2 (s2)

Fig. 1.

. .

s .~ ~

n

The problem is to find Di(k), Si (k) T . (k) and F . (k) for k = 1, .... N.[N is the ~

~

number of sampling period~, which minimize the following criterion : J

~

N-l E{ 1: x' (k)Qx(k)+u (k)R

1

k=O

R

U

2 2

(k) }

U

1 l

2

(k)+u (k) (6)

where Q and R are weighting matrices with appropriate dimensions. We remark that the static controllers and the no ise free output situations are special cases of the previous formulated problem and ( ') is the transpose.

115

A stochastic aspect of decentralized control

MATHEMATICAL PROCEDURE :

\~

Equationsl1), l2) andl3) may be written in a compact form as x(k+1 )

Ax(k) + Bu(k) + Gw(k)

( 7)

y(k)

Cx(k) + Hv(k)

(8)

o

__ [WO(k )

(k)

u(k)=

B =[B 1 , m1·

I 1

J

B2

m2,

0] ~

[I

where y is a

"

( 13)

matrix

rr. J s. /

L

and equation (11) will be N-1 L [~ ( k)Q~ ( k)+ ( ~~(k ) J = l E{ N k=O ;(kh 'Ry;(k)

n

is the ide ntity s x s matrix

[~~ ( k ) +~ ~ ( k)J

and C

"

I

1~ terms of the new state vector x (k) equation (10) will be

u(k) = y F(k)

where

"] and

V(k )

+~~ ( k »

(~~(k)+~~(k» l

,

(14)

We define the variance matrix

Equations (4) and\5) may be written also as D(k) Z(k) + S(k) y(k)

Z (k+1)

( 10)

T(k) Z(k) + F(k) y(k)

u(k)

=

(G+BF(k)H) W(k ) (G+BF(k)H) ,

[Zl(k)j with dimension s = sl+ s 2' Z2(k)

:1

D(k) = [

D,:k']

also S(k), T(k) and F(k)

~ L[tr{Q+~';'(k)Y'Ry;(k)~)

J

+ tr{

(~,;,

where

The performance criterion (6) will be

Q

N-1

l

r

E{

(x'(k)Qx(k)+u(k)Ru(k)}

(11)

N k=O with R is a block-diagonal matrix of RI and

R2 · Consider the augmented state vector X(k)] which is

x(k)

~

= n+s dimensional

[ Z(k) (10) we

: 0]

- [B , B= 000

( 12)

0] it • =[c 0] ;r ,~=[G I

~

0

I

,s

tln_ . s .,

,.~

,L-

0

0

2.

~ p...

~'F(k)=

and w(k) =

S(k)

[w (k)]L-

D(k)

-

(16)

s-/

-

- -1

. -

,.

.

. .

-

. .

-

B' S(k) (AP(k)C' '

-1

(CP(k)C'+HW(k)H')

(17)

where P(k) is the solution of equation (15) with Pto) = P , =

o

Ip 0

0

01

0

,

oJ

and Blk) is the solution of

__ p..l

. [0 H] ~ _ [F (k) T(k) }~ H=

n4

F(k)= -(y'Ry+B'S(k) B)

P

0

'Ry;(k)~)~(k)}]

l: :] ~

.

t. C

P(k) }

The original problem is transformed to a deterministic one in which equation ( 15) is the state difference equation and we want to find the sequence F(k), k = 0, .... N - 1 which minimizes the performance criterion (16). Using a standard dynamic pro gramming algorithm we can find that

+ G';Hk)H')

x(k+1)=(A+BF(k)C)x(k)+(G+BF(k)H)w(k)

~he[r:

(k)y

•

vector. Then from equation (7) have :

A=

( 15)

also (k)

are block diagonal matrices as D(k).

J =

E ( x(k) x'(k) }

in e q uation (12) we have P(k+1)=(A+BF(k)C) "P(k) (A+BF(k)C)' +

where Z(k)

~(k)~

(9)

S (k) = (A+BF(k+l)C)' S(k+1) (A+BF(k+l)C) + Q+C'F'(k+l)y'RyF(k+1)C

~

r...... s -"

v(k)

We consider that v(k) and w(k) are completely independent, then w(k) is a zero mean new random variable with covariance matrix

with

( 18)

S (N-l) = 0

which represents a two-point boundary value problem. Equations (15) and (18) can be solved iteratively, one forward and the other backward to find the sequence F(k),K=O, . . . N-l Now we consider the steady state case for which we use a time invariant decentralized regulator, i.e. the matrices in equa-

116

A. Wahdan and B. Chenevf>aux

quations (4) and (5) are independent on the time k, then equations (9) and (10) will be Z(k+l)

DZ(k) + Sy(k)

(9' )

u(k)

TZ(k) + Fy(k)

(10' )

the limit of J in equation (11) when

t~

We

N-> '" i.e

N-l

[2.N E <•

= lim

J

1: k=O

N ... '"

(x' (k)Qx(k)+u' (k)Ru(k»}l

(11' )

equation (12) will be : x(k+l)=(A+BFC)x(k)+(G+BFH)w(k)

(12' )

also (15) will be the steady state Liapunov matrix equation :

P=

(A+BFC) P(A+BFC) '+(G+BFH)W(G+BFH)'

(15')

charactristic values have moduli strictly less than 1, we prefere to carry out the minimization problem directly i.e. treating the parametric optimization problem itself (equations 15' and 19) using the quasiNewton method of nonlinear programming to find the wanted parameters numerically but in a suboptimal manner. Some comments on the numerical procedure. - Tb carry out the previous algorithm, the ql1asi-Newton method (Ref .23) is used to minimize J in equation (19). - In each iteration of the program we need to solve the equation (15') and get new values of J. We solve this equation into two steps : firstly it is transformed to an equivalent steady state continious equation using the bilinear transformation (ref . 25), briefly speaking the equation

finally J in (11') and using (16) we have 1 N-l ,,' 1: {tr(Q+C'F'y'R yFC)P + N ... '" k=O

= lim

J

[N

(a)

A'MA + M = - C is t.ransforrned to _1/

B'M + MB

2 by using the formula W is the steady state covariance matrix of ~ (k). In (16') the terms between brackets -are independant on the time k, then J

+

{tr(Q+~';'y 'Ry;~)P

+

tr(~';'y 'Rrmw}

tr{[(~+~;~)P(~+~;~) ,+(~+~~)W

- FlA'}

where A is the lagrange multiplier vector. Due to the matrix minimum principle (Ref.24) the conditions of optimality are:

'01 1= a P ..

0

and

al(F,p,A) a F 1.J . .

where [

After simple analysis we get B' A(APC'+GWH')

(ci~, + ~~') -1

y

( 20 )

where P is the solution of (IS') and A is the solution of A = (A+BFC)' A(A+BFC)+Q+C'F'y'RyFC (21) The solution of these last equations i.eo (15',20, 21) represents a numerical diffic~l!¥.till now. Actually, if the matrix (A+BFC) is stabilizable one i.e its

inverse

(B-1) C(B '-I)

Secondly the continious equation (b) is solved using the algorithm proposed in (ref. 26) . ILLUSTRATIVE EXAMPLE Fig. 2 represents-_ the stirred tank example (ref. 3), which is fed with two incoming flows having two corresponding concentrations which are considered as ex~nentially correlated noises with known mean and covariance. It is assumed that the tank is st-i rred well so that the outgoing flow conoentration equals the con~entration in the tank. The discrete difference equation is caractrised by the following constant matrices 0.9512

A=

, -1'

F = -(y'Ry+B'AB)

-1

where I is the identity matrix, the exist and

tr(Q+C'F'y'RyFC)P + tr(H'F'y'RYFH)W

(19) The lagrangian for this minimization problem (15' and 19) is

J=

B = (A+I) (A-I)

(b)

Y

0

0

0 0.02262

0

0.9048

0.0669

0

0

0.8825

0

0

0

0

0.9048

[4.'"

B'= 4.877

::d[ 0.01 0

-1.1895

0

0

3.569

0

0

0

0

0

1.0

0

0

] 1

117

A stochastic aspect of decentralized control

The quadratic criterion has the followi n g Q and R constant matrices :

o

Q{T

0.02

o o

o

o

o o o

o o o

0_ 10 6

o

-j

we get

-0 . 054659 y (k) 2

and

4 The criterion valup is J = 0.478236xl0and the performance degr~dation equals 9.3286 % When using a dynamic regulator for the last case we get the following results :

0

ZI(k+1)~.2045547

0

0

o .4886xl0 -3 o .9375xl0 -4 o .lxl0 -3 o .9375xl0 -4 0.2212xl0 -2 0

0

3 0.1 x 10-

0 W

0

=[ 10-6

-2 . 775536 Y1(k)

The steady s t e atp c o va ri ance matrix is

0

v

0

0.7252xl0

-2

u (k) 1

= 0.03293255

ZI ( k ) -o.124 23 17 Yl(k) Z1 ( k) - 5.91654 Yl (k)

Z2(k+l)=O.2286461 Z2(k) - 0 . 099646 Y2(k) u (k) = 0.127604 Z2(k) - 0 . 0481845 y (k) 2 2

\'~, \

feed 1

feed 2

the criterion value is J = 0.458529 x 10and the performance degr~dation equals 4.8234 %. 8 But if the V matrix is 10diagonal elements we will have : 0.253441 ZI(k)-0.1536577 y (k) 1

concentration 2

Concentration

Z2(k+l)

- - --

---------- ....... - -- Concentration C

4

0.029288 Z2(k)-0.242724 Y2(k )

u (k) 1

0.143676 ZI(k)-6.117344 Y (k) 1

u (k) 2

-0.042535 Z2(k)-0.0683387 y (k) 2

The criterion value is J = 0.4518751xl 0 and the performance degr5dation = 3.297 ~ For comparison we give here the optimization part of the centralized optimal regulator (using the seperati o n principle) '\,

4

'\,

u(k) = F x(k ) '\,

where x ~, s the estimation of the state vector and F is the f o llowing gain matrix outgoin flow c o ncent ation C

Fig. 2. A STIRRED TANK. Firstly we consider the noise free output case with static decentralized regulators the resul ts are as follows - 7.063 Y1 ( k) -0.057247 Y2(k) The criterion value is J = 0.44109xl0- 4 and the performance degridation with respect to the optimal centralized case equals 0,8 % For t he same case but with noisy outputs which have a diagonal covariance matrix

0 . 071314

-0.070 231

[ 0.013549

0.045496

-0 . 00977/)4

-0 . C1C 3 3 8~~

0.008687

0. 0 0 3058 8 ]

and the criterion value is: 0 .4 3 74 3 x 10 -

4

CONCLUSIONS The decentrali z ed stochastic control problem is treated, in adopting the idea that the engineer wants to implement the simpler control system which satisfies in a reasonable way its needs. So we consider here the direct search algorithm, which may represent a heavy off-line computational effort, but we can use the minimum number of control parameters for implementation. Also it results in a suboptimal performance criterion, under the condition that the regulator structure and parameters satisfy an asymptotic stability condition, which may

A. Wahdan and B. Cheneveaux

118

be satisfactory from engineering point of view. The performance degradation, less thun 10 % in all cases considered, represents in our opinion, a reasonabl e numerical result which shows some validty of our algorithm. We also remark that we can avoid the construction of state estimators by using this approach. We hope to modify our algorithm to simplify the computational effort especially for large systems. ACKNOWLEDGMENT. We wish to thank Profe s sor MEZENCEV and Professor de LARMINAT, for their encouragrnent and useful discussions, also Mr HANEN for his valuabl~ aide in programming. REFERENCES. 1.- WITSENHAUSEN H.S. "A countpr example in stochastic optimal control".SIAM J. Control. Vol. 6 N° I (1968). 2.- VAAAIYA P.P."Revipw of (Theory of hierarchical, Multilpvpl systpms) by M. D. Mesarovic and al. I.E.E.E. Trans. Automatic Control. AC - 17.2 (1972) 3.- KWAKERNAAK H. and SIVAN R.l', "Linear optimal control systems" Wilpy. New York (1972).

° 4.- ASTROM K.,T · (19 70) "Introduction to stochastic control theory" AcadPmic prpss. 5.- WITSENHAUSEN H.S. "Spppration of estimation and con trol for discretp time systems" Proc. I.E.E.E. 59 , nO 11 (1971). 6.- WITSEN'dAUSE N H.S. "On information structures . f pedback and causa lit y ". SIAM J. Cont r el vol. 9.2 . (1971) 7.- BISMlJT.J. "An example of interaction between information and control" I.E.E.E. Trans. Automatic Control AC.18 Oct. (1973). 8.- HO Y.C. and aw K. "Tpam decision theory and information structures in optimal control problem~ Part.l. I . E.E.E trans. Automatic Control. AC. 17 nO 1 (1972).

12.- KURTAAAN B.Z. and SIVAN R, "Linear quadratic gaussian control with one step delay sharing pattern". I.E.E.E. trans. Auto . Control. AC.19 Oct (1974) 13. - KURTAAAN B. 2. "Decentralized stochastic control with delayed sharing information pattern". I.E.E.E. trans. Auto. Control. AC. 21 Aug. (1976) 14.- MAYNE D.Q. "Decentralized control and large scale system~in the~directions in Large-scale systems" by HO and MITTER. Plenum press. New York (1975). 15.- CHONG C . Y. and ATHANS M. "On thp stochastic control of linear systems with different information sets" I.E.E.E. trans. AC - 16 nO 5 Oct. (1971). 16. - AXSATER S. "Sub-optimal time variable feedback control of linear dynamic systems with random inputs" Int. J. Control Vol. 4,6. ( 1966) . 17. - MC - LANE P. J. "Linear optimal stochastic control using instantaneous output-feedback" Int. J. Control. vol. 13, 2 Feb. ( 1971 ) . 18.- KURTAAAN B.2, SIDAR M. "Optimal instantaneous output-feedback controllers for linear stochastic systems". Int.J. Control. Vol.1 9 . 4 April (1974). 19.- ERMER C.M. and VANDELINDE V.D. "Output feedback gains for a linear. Discrete stochastic control problem" IEEE Trans. AC. 18> 2 ('1973). 20 . - LEVlNE W. S. and ATHANS M. "On the determination of the optimal constant output-fepdback gains for linear multivariabJp systems" IEEE. AC.15,Feb. (1970) 21. - WAHDAN A. "Contribution a l'etude des commandes decentralisees de systemes dynamiques" D.E.A. Laboratoire d'Automar.ique E.N.S.M. NANTES (1974). 22.- DAVISON E. "Decentralized stabilization and regulation in multivariable systems" in *directions in large scale systems". Plenum press. New york (1975). 23.- GILL. P.E. and MURRAY W "QuasiNewton methods for U'C "cnar3ined optimization" J. of Maht. and applications. Vol. 9 (1972).

9.- SANDELL N. and ATHANS M. "Solution of some non classical LQG stochastic decision problems" I.E.E.E . Trans . Auto. Control. vol. AC.19 N° 2 (1974).

24.- ATHANS and F. SCHWEPPE "Gradient matrices and matrix calculations" MIT Tech . note Nov. (1965).

10.- CHU K. "Team decision theory and information structures in optimal control problems" part.II: I.E.E.E. trans. Auto. Control AC. 17. N° 1 (1972).

25.- BARTELS R.H . and STEWART G.W. "Solution of the matrix equation AX+XB=C" Communications of ACM vol. 15. N°9. Sept. (1972) .

11.- YOSHlKAWA T. "Dynamic programming approach to decentralized stochastic control problems". I.E.E.E. Trans. A. C. 20 Dec . (1975).

26.- POWER H. "A note on the matrix pquation A'LA-L = -K" IEEE. Trans. AC.14, 4, (1969).