Solutions of the problem of stochastic optimal control of hydro-thermal power systems

Digi tal Computer Applications to Process Con trol, Van © IFAC and orth-Holland Publishing Company (1977)

SOL TIO S OF THE P OBLE

a ta Lemke, ed.

A3-3

F ST CHASTIC OP I AL CO TROL OF HY arti

o~

e art en ,0

as

Cla to,

. S.

O-THE

.AL PO ER SYSTE S

illo

Electr'ca E g~ eeri g i versi t ictoria, Australia

he pro le of long-term regulatio of large seasonal storage reservoirs is examined using a stochastic represe tatio of inflows and loads. Several solution ap roaches are investigated i cluding open loop and feedback solutions, emplo ing variants of the Differential Dynamic Progra ming ethod and a worst case solution using a zero-su Differential Game as its basis. Application of these methods to a sample system provides an interesting interpretation of these d'fferent solutions.

1. I TROD C 10

For short-term opti ization proble the water inflows can be considered to be deterministic and hence the ap lication of the deterministic ethods of opti al co trol previousl established would give good solutions.

The i creasing size and co plexity of odern ower systems as created the necessi y for the develop e t of sophisticated 0 erational and control procedures. By the term opti izatio of a power system we ean securing all the uses of electrical energy in the required quality and quantity at the minimu cost, whilst fUlfilling all the physical limitations of the system.

In recent years, the solution of this deter inistic problem has een attempted by a number of research workers who have applied such methods as classical lagrange mUltiplier theory [13,16, 7J dynamic programming [11,16J, gradient methods [13J and the maximum principle as formulated by Pontryagin [2,4,17,18J. Their methods, however, are likely to give rise to unstable and unrealizable solutions and show several other seriou eaknesses as noted in Dillo and orszt n [5,6J. Dete i istic solution ethods t at do not show these weaknesses are re orte i references [5,6,3J.

There are several complex aspects to the econo ic control and operation of hydro-thermal power s ste s which each give rise to an associated proble. hese proble s include: (a)

(b)

Long ter control of h dro-ther al syste s, which deals wit the q estion of sage of seaso al storage reservoirs. he 0 ti izatio eriod here is 0 eater ear or longer.

er

opti ization

roble

a

Sort ter 0 ti a co trol of dro-ther al ower systems, w ich rovides the ode of usage of eaki g and short ter storage reservoirs. Here the ti e period of i teres varies fro a week to a da

( c) e, s case is

(

)

of f

o~

ot

a give

e oa d stoc astic

d

g

257

258

SOLUTIO S OF THE PROBLE

OF

A3-3

TOCHASTIC OPTI·mL CO TROL OF HYDRO-THERMAL PO ER SYSTE S

e distr' io of ower ydro a d ther al ower ith etwork constraints,

s t

e

Ge era ion Pi a d t e dro t r i es ··i· xi t .e rese voir of t e is chosen as e sta e

f(P. )

L

reduced to t e ich ens re stable etween odes. his

al i. e.

e f el costs for 0 eratio of la ts are f ct'ons of t e outp t e cost for he i ther al f(P i ). he cost of ge eratio ther .al stations is

u i

(1)

1, .... ,£

l

erval

he cost of ge eratio o er the ti e t ) is (t 0' f tf L( i) dt to he o ti alit to e

,

criterio

(2)

therefore, is taken

(3 )

ower were E is t e ex ected states of the s s e .

alue of lover all t e

T e differential equations of state corres onding to t e h dro stations on independent rivers and the top storage of a cascade are l

s t e

(

i = 1, .... ,

- R.

i

X.

~e'

ru off i to

s ora e a dro-

.~ t e tur . e d:scharge at t e i h

)

i

statio . ~or cascade H S's t ese of s a e are

di~ferential

l

ere t e s .... a io i ~o e i

s

d~rectl

F'.

s~ca

~c

ar':'ab es ... ':'c .t'i .. =..

(5 )

1, .... ,

X.

".-J

eq a io s

< :::

.r..ax

l

(6 )

, •••• ,Q.

(7 )

i

is

ce

"'-':-:e ;;c...: er :--r e~ 2-:~0

rol

s, also, ca. e ? .::-.ax

-i

. S' s feedi g

0 er al HPS.

g~

e

t: Ta .

.e forr.

of

(

.... (x.)(. .-c.)

l

.ere an

w

::::i •.ce c ce s='s e;;-.

"

-

="-l

are of

~.e

e-

L

.e

sys er:;

~m"er

s a a_a .ce

.e

)

A3-3

SOLUTIO S OF THE PROBLE

£

I

S) +

P. (

i=l

l

. =1

(9)

j( :?S)

o es a d ca e calI coef:=ice ts I . reg latio o se a fixed wit load co su.ed. Therefore, i a di io , to t e ine_ualities constraints (6) and (7), t e control variables ust also satisfy the equality constraints (8) and (9).

The storage reservoirs ave fi ite vol e, t erefore there are li its on testate variables of the for X.min l

<

x. l

259

OF STOCHASTIC OPTIMAL CO TROL OF HYDRO-THERMAL PO ER SYSTE S

(10)

X. ax l

In ad itio t ere will e a e d condition on t e final volu e of later in t e reservoir of t e for l , .... ,n

o e is e iffere t ra sitio are ostula ed over o acco for sease

A more recent study b t e asis of the load paper.

ear

Phua and illon [20J, is odel e 10 ed in this

It sold be oted, owever, that the fo of distri tic c osen is to so e exte t dictated b t e solutio et od availa le a d the ost accurate re_rese tctio. €ed ot 01 a s e the est a ro ria e. SALIE T FEATURES OF THE STOCHASTIC OPTI. CO TEO PROBLE,

(11)

Both the River Inflow iCt) and t e load Ci(t) are stochastic variables.

i ue s are ical well eca

Ca) C )

Cc)

..a

e

is e

a

o ze

r i .. e;:e. e.. ~

SOLUTIO S OF THE PROBLEM OF STOCHASTIC OPTIffiL CO TROL OF HYDRO-THERMAL POWER SYSTE S

260

HI river HPS's. ence the atrix is singular and the seco d order u u ethod can ot be employed. As a result, the ethods used are first or er variants of the D P approach [ 9]. A question that arises in t is roblern relates to the equality constraints on the control variables. Theoretical a proaches to the problem of stochastic optimal control normally use an expected value of any equality constraint present. However this poses some difficulties in the case of the power problem, as it could include trajectories that do not meet the power balance constraint, even though the expected value is zero. A way around this problem could be to maintain the expected value of the square of the constraint at zero. Fortunately in the feedback case, this is avoided by simply ensuring that the constraints are met for each realization. 4

OPTI AL FEEDBACK SOL TIO

ETHOD

The opti al cost for the stochastic comes (x,d,t)

=

roble

A3-3

be-

(V(x,d,t)

E

(x,d) The adjoint vector

is given by

(16) The equations for the backward integration in time are -a

6H

-v x

H + x

(17) (18)

G

x

where a is an estimate of the deviation of the cost from opti al. G is the equality constraint and the active inequality constraints. a is the associated Lagrange multiplier. Here ,t,

For the realizations, it is assumed that the stochastic disturbances are discrete values i.e. have k fixed discrete values, and occur only at discrete times T, 2T .... T. The transition probability of the first order arkov Chain is specified by PIj, where i and j are discrete levels of the rando variable. At the instants of ti e T, 2T, .... therefore t e trajectory undergoes a decomposition, a new trajectory being generated for each possible transition. There are k such ossi le trajectories. This means that co trol and state varia les undergo a si ilar deco position corres 0 ing to eac trajectory.

6H = H(x,d,u",Vx,t) -H(x,d,u,Vx,t) where the bar denotes nominal quantities. The boundar conditions on the differential equations (17), (18) are a(x,d,t ) = 0 f

(20)

x(x,d,t ) = f

(21)

In additio t e deco posed branches for a and x coalesce at eac tra sition oi t td and t e .atc ing conditio s are a(x,d , r

Eac i di vid al traj ectory 0 ti. izatio s carried 0 t us' g P. e coincide ce of t e trajectories over the different regions can o ever, be utilized to red ce the er of egra io s to a i .u.

':(x,d, , , t) = ere d

a

s

is

.e ad' 0'

t.e H dro- ,er al s s e ) +

( -

variab e

is

(22)

F

(23)

rs

atching co d'+'o s in trod ce the effect i~~ere.t realizatio s ca sed b the ector over the re a' i g i e

'='. e

asic algori ... ::or fol ows:

e sol t'on 's as

( 12)

)

efi e

(16)

e s oc astic dist race vector.

he 0 ti al cost at t'me t 0 realization for the fixed e d

-)

J

a lor series ex a.sio e neig 0 rood of t .e con rols. ~or

(19)

a y individual oint roble is

2 I tegra e ti e e coefficie ts a, x of the first or er ex a sio of t e cost a d no inal trajec ory (17,18) usi g a co rol u~t: W • c ;ini izes the Hamiltonian H ~tp.

A3-3

SOLUTIO S OF THE PROBLE

OF STOCHASTIC OPTIMAL CO TROL OF HYDRO-THEffi1AL POWER SYSTEMS

give b (12). This inimization of H is erfor ed using t e Sequential Linearly Constrained ini ization Technique (SLC T) [6J. Ste

3

Set

u(t)

u(t)

t£:[to,taJ

(24)

u(t)

u:': ( t )

t £: [ ta' t fJ

(25)

and adjust t a until an acceptance criterion is satisfied. Step 4

Set

u(t)

x( t)

and

u(t)

(26)

= x( t)

(27)

Step 6 Adjust the end point Lagrange multipliers b, using --1 -V b b

6b

o ti al control proble defined in section 2. The second layer is chosen to be the vector of stochastic variables d i.e. the inflows and loads. The first player seeks to inimize the payoff function I, given by eqn.(3) whilst the second player seeks to aximize it. The state of the s ste evolves according to the differential equations specified by eqn. (4) and the inequality and equality constraints. In addition to the bounds on the control variables and stochastic variables specified by (6) to (7), it is also necessar to place upper and lower limits on the stochastic variables d, of the form J.,

Step 5 Repeat steps (1) to (4) until the value a, which measures the deviation from the optimal is below a prespecified tolerance.

(28)

C.

5 OPTI A

0 E

is

0

to

=1

U

he first chosen to

APPROACH

SI G

max

(32)

l

max

min (H)

(d)

(u)

(33)

(34)

is the adjoint variable.

This is in contrast to the optimal control vers ion where u~: is selected by solving a nonlinear progra ming pro lem. 7 APPLICATIO

dro

GA ES

layer i t is differential ga e is e the control vector u of the

LE SYSTE

~ P T:r ~l

V

S a io

oad (3 )

TO SA

The sample s stern deals with the 3 node system sown e ow. At node 1 there is a Hydro power station, at node 2 a t er al power station and node 3 is a ure load.

ze

]

IFFERE TIA

C,

l

The essential difference between these is that in the case of the differential games versi9.n ~ n the method chooses the saddle point pair (un,d ) by solution of the static game

where

were i de otes t e pro ab'lit condit'o al 0 (x,d) of t e 't value of the eco posed tra'ectory a t e associated Hamilto 'a , H·. 6 S

... C,

(31)

The present authors have produced an extension of the DDP method to solve this two player game.

T·l q.H,

l

H(x,u,d,t) = L(P) + AT (J - R)

LOOP SOL TIO

c ose

min

l

Here H is given by

he algor't for sol tio is esse tiall the sa e as t e fee ack co trol g'ven e revious sectio , with the followi g exce tion: (3) u

J, max

J.

Subject to the equality constraint (8) and (9) and inequality constraints (6),(7),(31),(32).

In the feedback case co troIs were obtained for each deco positio of the state trajectory. For the 0 e loo case t e control is only a single f nction of ti e as no infor ation relating to the i divid al state tra'ectories is fed ack.

ste

min

l

(29)

and bb is computed numerically. ote that there is one such end point multiplier for each trajectory. Repeat steps (1) to (3) until the end point condition is met to within a prespecified tolerance for each trajectory. To include state variable constraints in the above algorithm an extension of the D P method as given in ref. [31J has to be employed.

261

The syste constan s

x (3) 1 1.6xl 9 2 A ( ) l 7 6.1xl0

3 L (P ,t) = 2.5 2(t) + 0.05P;(t) 2 2 s characterized b the followi g

X (3) 1 0.0 1 ( 3/ sec ) 10 .0

Base Head H

0

200.m

1

(

3/ sec ) 2.0

262

SOLUTIO S OF THE PROBLE

q.(.3/ sec )

p

2 "(

l

.. (

OF STOCHAST C OPTI

3)

3)

1.2xlO 9

1.2xI0 9

.~

0

O.

280.0

seaso s eac

1

2

3

1

50.0 60.0 70.0

70.0 80.0 90.0

50.0 0.0 70.

3

pl =

~

0.3

0.2

O. 4

O.

O. 2

0.3

O.

0.3

2

0'5

~

0.5

0.3 0.1

0.2] 0.4

0.2

0.3

0.5

The load C has 3 equal seasons a d 3 levels. T e followi g table defi es C

IS

1

2

3

200. 220.0 240.

25 .0 270.0 290.0

190.0 210.0 230.0

Level 1 2 3

1

etwee

roba ility matrix

The tra sitio seaso s 0.5

0.2

.2

0.4

.4

.2

.5

.3

[0.2

0.1

0.4

O.

0.6

0.3

.1

e ~nit'al probabilities (?O) for are

A3-3

.e i~~ere solution ( E t e ex_ecte ..... e .ree seaso s.

g of

:ig res (1) a d ( ) Sf:O t e 0 for t e 0 en loo ,differe ia a e and de er inistic cases. I figures (3) and (4) t e feed ack co trols are de.ic ed. HOJever eca se of t. e large u er (729) o~ co trols, 0 e associa ed with each trajec or_ 0 1 t e of co trol at eac ti e insta t as een ed. Thus an 0 ti al feedback control us wit i t e crosshatched area. A ical is shoo as a solid line.

The tra sitio ro a ilit atrix etwee seasons 1 and 2 ( 1), a d seaso s 2 a d 3 2) ( are

0'5

CO TROL OF HYDRO-THERMAL PO ER SYSTE S

it

3

Le el

~L

.5

:or the ifferential Ga e the deterministic state trajectory is shown as a solid line in fig. (5). W en this co trol is a lied to he stochastic sJs e the state trajectories occu _ the crosshatched and indicated in fig. (5). Si ilarl the opti al trajectory for the deterinistic case using ex ected load a d i floH is t e solid line in fig. (6) whilst he band defines t e ra ge of he trajectories whe this deterministic control is a lied to the stochastic roblem. Fig.(7) shows the range of the feedbac state trajectories and one t pical trajector. Tre 0 en loo state trajec ories have a si ilar s read to that shown for the deter inistic case in fig.(6). An im ortant feature or the state trajectories for the deter inistic and open loo solutio s t e s read of the e d poi t. e deter inic ro le 's solved to eet e end oi dition 0 1 for t e ex ected load and r noLf. So t is c ntrol Hhen a lied to the stochastic ro le akes no acco t of as or fu ure le els o~ .... e stoc ast'c ara e ers .

e 'nfloHs o r .. a e ~.

y a .

.... ~.e case

...... e ass r::e

o~

.ds

'.-lit'.

Seas J

?or co. ariso fee ac~ (r. B . ) ,

es e 0 ti"al loo ( .L.) a

_ 3-3

SOL TIO. 'S OF THE PROBLE

U

cv

OF STOCHASTIC OPTI·

100

100

I/)

~

E Q:

90

90

w

80

80

C)

I/)

~ E

J

LL LL

0 Z

(/)

0

::>

W

0 ci

0::

z

CD

a:

2

::>

4

.....

6

8

10

160 140

260

120

220

w

3: 0 a.. ...J


a.L. ~

100---7--=

~.::::::::

180

...

DET.

80

140

"""' ~ ~

u

100 2

4

6

8

10

)(

~

::> ...J

(j)

2

4

6

8

10

12

FIG.3

TIME (mths)

~

Cl..

a:

4

8

160

w

0

...J


F.B. 200

a..

~

~

a: W

I

.....

2

6

10

12

TIME (mths)

FIG.4

~

::>

1.4

...J

0

> W

19 0:

40

CD 0::

.. w

w

~

w

~

X

1.6

>

0

0

g

0


(j)

0)><

~

w

U

"'e

1.8

......., X


I

TIME (mths)

FIG.2 '-'E 0>0

0::

~ 0

0 0

.....

w

(!)

0

...J

I

et:

3:

0::

60

F.B. 80

240

~

w

.§

300

C

Cl.. 0::

~

L PO ER SYSTE S

90

I/)

.....

3:

~

cv

ROL OF HYDRO-THEm

::>

TIME (mths)

FIG.1

u

Ucv

0::


L CO

C)

1.0


4

6

8

a::

0

2

4

6

8

10

12

~

FIG.5

TIME (mths)

<.f)

FIG.6

TIME (mths)

263

264

SOLUTIO S OF THE PROBLE

OF STOCHASTIC

sho s at t e reser oir e 0' t If t e 0 er e ow he re uired value. was ai tained additio al ther al ower t 's discharge sched le wo Id e er red ce t e reservoir vol e ear 0 ear. Howe er this solu io is the ost ex e sive in ter s of e s ste cost functio and will usually lead to wastage of water, ote t at in the case the trajectories were axi reservoir vol the wastage of water t use of this control.

The ex ected costs associated with the differe t controls together with the expected end point error and the range of the end oint are abulated elow. Expected cost

ET.

2.29xl0

O.

F,B.

Expected End oint Error

10

CO TROL OF HYDRO-THERMAL PO ER SYSTEI S eet

e e

oi

A3-3

co stra'

8 CO CL SIO S Several

ere ado ed for i al con rol

of e differential ga e allowed to exceed the e i order to indicate a would occur fro t e

Only in t e case of the feedback solution are the s ste constraints co pletely satisfied. Both the power balance and end point constraints are et ever trajectory and control. It is worth oti g that t e feedback control ro erly takes account of all future disturbances through matching conditions (22),(23).

ethod

OPTI~L

9 9 0.93xl0 -1.57xl0 9 9 0.93xl0 -1.57xl0

0

2.29xl 10 10 3.0 xl0

0.26xl0

2.32xl 10

0

Range of End oint

0 9

1.20xl 9-1.82xl09 0

_t ca e seen the end oi t co dition is et for ever realizatio o 1 the feedback case.

REFER.E CES [lJ AGARWAL, S.K., Proc. lEE, 120,674-678(1973). [2J BELAE , L.S" Isves. Acad., auk, U.S.S.R. Energet. & Trans. 0.5, 13-22 (1965). [3J BO AERT, A.P., EL-ABAID, A.H. and KOI O,A.J. IEEE Trans. pwr. Ap ar. and Syst. 91, pp. 268-270, 1972. [ J DAHLI , E. B. and SHE, . . C., IEEE Trans. Pwr. Appar. Syst., 85, 437-458 (1966). [5J DILLO , T.S, and ORSZTY , K., 5th I.F.A.C. World Cong., Paris, June 1972. [6J ILLO , T.S. and ORSZTY , K., Int. J. Control, 13, 833-851, (1971). [7J DOPAZO, J.G., KLITI , O.A., STAGG, G.W. and ATSO., ., Proc. IEEE 54, 1877-1885, (1967) . [ 8J G EDE KO, B., Invest. uz. "Elektrornek. o. 1, 1961. [9J ACOBSO, .H. and A E, D.G., A erican Elsevier, ew York, (1970). [lOJ KA T ELISH ILl, .A., Israel Prog. Scien. Trans. erusale, 1969. [llJ KECKLE_, . G. a LA SO , .. F . , ..at . Anal. A lic., 2 , 80-109, (1968). [12J KI CH .AYER, L.F., oh Wiley & Sons, Y,1958. [13J U , L.A., nvest. cad. auk, Energ.&Trans. [1 J LAZAREV,I.A. Electr'chestvo, 06,35- 0,1969. [15J ITT E, .Op. es. Soc. 1955, 187-197. [16J OS A E , A.G., Elek richestvo,12,2 -33 1963 [1 J T.I S., Elect E g. i a a , 85, 23-33, (196 [ 8J OH, a an,87,17-28,1967. [19J PE E ort 6. (1966) [2 J S it ed

1.8 [21J

1.6

x w

~

F.B.

[22J

1.4

[23J [2 J

::>

--I

o

>

~

3.

[2

w (!)

Trans.

1.0

o .

[ 6J

a::

o

~

(j)

FIG.7

TIME (mths)

[27J J [ [ 9J [3 J [3 J

~