On the solution of optimum problems in non-linear automatic control systems

ON TllE SOLUTIO;YOF OPTIWU~4PROXEW IN NON-LINEAR AUTOVATICCONTROLSYSTEMS’ V. F. DFM’YANOV Leningrad 9 necember

(Received

1954)

Introduction A .!F!‘UF?of practically important engineering problems can be reduced to tne problem of minimizing some functional. In [II, for problems described by s.vstems of 0rdi:lary differential eqllations (or systems of differential equacio;ls with constant lag) with functionals of integral form, necessary conditions are introduced in the form of Pontryagin’s To fin1 the optimum control various numerical “maximumprinciple”. metnods are used [T!, 11, whose co:lverge:~ce, however, even in the case when a unique minimum exists, is hot guaranteed. Relow we consikr the problem of finding optimum controls by objects wnose state can bz .lcscribe:i 113y 3 sysr;zm of R non-linezr ordinary differential eqtrations or ti systeal of Jifferential equations witn constant or variable lag. A necessary conditiou is also deduced wnicn the optimum control must satisfy. For the case of sysiems of orlinary differential equations this condition is a linearization of Pontryagin’s “maximumprinciple” and is applicable to classes of equations of a form which is more general tnan that of [Xl,

1. Formulation of the problem Yrob lem .4

We are given a non-linear *

Zh.

vychisl.

Mat.

mat.

Fiz.

system of ordinary differential 5,

2,

32

218 - 228, 19536.

equations

Non-linear

with initial

automatic

control

33

systems

conditions =

X(O)

(1.2)

XQ.

Here X = .dX / dt; X(t) = (z’(t), . . . , z”(t)) is an n-dimensional and vector, f = (f’, . . . , j“) is an n-dimensional vector function, . . . . u’(t)) is an r-dimensional vector control function u(t) = (u’(t), subject to choice from some class of controls 3, described below. The functions on the right-hand side of (1.1) are assumed to be continuously differentiable with respect to xi and uj, i = 1, . . ., 11, j = 1, . . ., r (in the region of permissible values of xi and uj, defixied by the class of controls 11and system (1.1))) and continuous with respect to t on [O, ~1, T > 0 being fixed.

We denote by X(t, II) the solution of system (1.1) tions (1.2) with choice of the control u E 0’. Let us consider

with initial

condi-

the functional T

Spw,4,

I(u)=

u(t), t)&

where the function g(x, u, t) is continuous with respect to t 0x1 [O, rl and continuously differentiable with respect to xi and uj, i = 1, . . . , II, j = 1, . . . . r, in the region of permissible values of xi and uj, defined by the class of controls II and system (1.1). It is required

to choose

a control

u(t)

E

;i such that

I(u) = minF(v). vtzu Problem

(1.3)

R

In [O, rl equations

(T > 0 is fixed)

we are given

a(t) ~ dt

f(X(t),

= x(t) = X(t)

= K(t)

if

x(t

-

a system of differential

h,(t)),

u(t),

t E [--h,(O),

01.

t),

(1.4) (1.5)

V.F.

34

differentiable

in

Dem ‘yanov

[O, 1’1, is real

O
03

tE[o,

if

a;

min h,(t) > tE LO,Tl

0.

Then an inverse function t = Fl(V) exists, w.?ic!l is also a strictly increasing, continuously differentiable, real function on [- ;11(0), T -

~l~ml. x”(t)) is an n-dimensional vector function, x(t) = (*l(t), . is an r-dimensional vector control function u(t) = (u’(t), . . . . ur(t)) belonging to tne class ji described below, f(X, y, U, t) is a real IIdimensional vector function which is continuous with respect to xi, yj, uk, t and continuously differentiable with respect to xi, yj, uk (i, j = values of xi, .A 1, . . . . n, k=l, . . . . F) in the region of permissible uk, defined tiy class I,! system (1.4) and the initial vector function (1.5) given in [- h,(O), 01. The region of permissible values of t is the segment [O, 1’1, the initial vector function x,(t) being given and continuous in [ - ii,(O), 01. ..)

We shall denote by X(t, conditions (1.5) and fixed

u) the solution 11 E li.

Suppose that we are given

of system (1;4)

with initial

the functional

(1.6)

I(u)=i8(X(t,u),X(I-hz(t),u),u(t),t)dt, 0

where v(t) = t - h,(t) is a strictly increasing continuously differeatiable real function in [O, ~1,0 <‘:?(t) < ~0 with t E [o, ~1, and let t = F~(v) be the function which is the inverse of v(t) (it is also strictly increasing and continuously differentiable in [ - &(O), ‘r i12(r) I); g(X, y, u, t) is a scalar real function which is continuous in differentiable with respect to xi, yj, xi, yj, uk, t and continuously values .k (L, ; = I, . . . . n, k = I, . . . . F) in the region of permissible ,k, defined oy the class of controls II, the system (I. 4) with 0f xi) yj, the initial vector function (1.5) and t E LO, ~1. Te shall assu,ne that if h,(O) > ill(r)) then X(t) is given and continuous in [- i12(9). - /x1(0)1 (in [-/11(O), 01 the continuous initial vector function is given by tne relation (1. !i)). ?roa COiltrOl

the class of controls function (control) U

!J it E

9

is required that

to find

such a vector

Son-linear

automatic

control

35

systems

Z(u) = minZ(v). VEU Tile control II E opt imum control.

, which satisfies

(1.7)

(1.3)

or (1.7)

If hi(t) = ii,, h,(t) = h2, where h,, h, are constant then prob lein ? is a constant lag problem.

is said to be the

posit

The vector control function u(t) = (u’(t), . . . . u’(t)) can be ally o.~e from the class of controls U; U is a convex, bounded and weakly closed class of real r-dimensional vector functions, whicn are summable in [o, rl and satisfy one of the following constraints in [Q, 7’1:

p(t) 1 d m(t),

(1 1

(1.X)

i = l,...,r,

ai(t.)>O, (i = 1, . . . . r) are non-negative, bounder.; i‘u;lctions in LO, ~1;

piecewise-continuous,

2(t) 2 o is a piecewise-continuous bounded function in [o, '~1, :C(t) is an I‘ x r matrix, which is positive definite in lo, ~1 with piecewisecontinuous bounded elements in [O, ~1; the index * denotes tile transpose;

e ui2 0t dt < Ci,

i=l,...,r,

t)
(1.10)

b T c 0

u*(t)N(t)u(t)dt.,(

C,

‘v(t) is an r x F matrix which is positive mable elements in [O, ~1.

(1.11)

c > 0, definite

in

[0, T] with sum-

For problem A and classes of controls of the form (1.8) - (1.9) the necessary conditions for optimality are derived in [II. In 1 ‘z 1 the necessary conditions for optimality in problem B for classes of controls of the form (1.8) - (1.9) have been investigated. Since

we shall

problem A is a particular case of problem R with h, only consider problem R from now on.

t) e h,(t)

-0

35

V.F.

Dem’yanov

2. The necessary condition for optimality If t:ie conditions . ^. snt1sr1eu

. . thzn,

as

imposed on the functional is well known, tae relations

I(u)

an4 system (1.4)

are

‘
is proved

in

[Al:

Tn or;ler that tne functional .f(u), which is given and bounded on U and has a continuous gradient there, may reach a minimum (with controls from class li on tile control u E iI, it is necessary, and in the case of convexity of the functional 1Cdl also sufficient, that min \G:(~)(v(.c)-U(T))& = 0. VISUO We IIOVIapply ‘beorem

(2.3)

1 to problem R.

Calculating G”(T) and substitutin g it in (2.1) (these calculations are given in the appndix) from Theorem ! we obtain the following theorem. I’lkore

:; 2

(1.6) In order that the control 91 G 2 may furnish the functional witn a aiiiiwlln it is OCCBSSilr’:, uii in tne case of the convexity of the futrct’ional (1.5) also sufficient, that

EC!;: [(~)'~"(,)+~](Vi(Z)--lli(~))dZ=O, 0 i=l

(2.4)

Yen-linear

control

automatic

systems

where

x (f - hl (a u), u (z), ‘CL u), x @-ha V), u), u (f), r),

(z) = f (X w,u),

t

gu@) = g (X (t,

( _ (fgj

(z) -

4”

(at.y)))*

W” 6)

il

*if

--z-=

I I

T-h(T)],

zE[O,

(a!$)j~.(2)-c(Z) if TE[T-~~(T),

-

(z)

(z) t$” (r1 (r)) -c

(2.5) ~1, (2.6)

df

ax= 0)) %” 0) + a& @a

I

3X

c (t) =

( ag

-g-p..vj---$

For classes of controls tmt the followin% theorem I’heorein

if

tEI0,

T--h(T)l, (2.7)

if

L3X

ar

ag

(q

867”V)

== ax=

&

aY

(

af’

af"

~“‘.?dui

ag

> ,

tE

[T--z(T),

1 ( ag

T],

;

ag

Y&y=

of the Porn (1.3) is valid.

ay’l”‘.9y”

ag

*

- (I. Q), from (2.4)

we see

3

In order that the control u E (// wy give a minimum for (1.5) (for classes of controls of the Pow (l.?) - (1.9)), sary tilat for almos-t all t E [n, ?I the relation

vg~. should

)

the functional it

is neces-

$ (~~jgu(t)+~)(v’(t)-u*(t)) =o.

i

1

be satisfied.

For problem A, Prom Theorem ? we obtain

r‘leorem 4.

(2.8)

V.F.

Dee’yanou

In order that tne conirol ti E :I may give to the functional I(u) (in Problem 9) the mirlirnm possible value frm the control class I/, it is ndcessary t.lat

I$& [(y&“(T) _ta~](v’(t)-~~(~))dz=O,(2.9) a

i=l

w,iere

lj?,(Q

=

(5$)’

-

%lm

qu (r)

ax

ag -

( axi

.‘,_**,----

ag

1 7

ax72

(2.10)

=o,

(2.11)

w ag -=

!E$ ,

-

aff’

s7”‘1&iT

af

a$ =

(2.12)

ap , ’ * * * ’ axn

afn ad

&l(c) = f(X(T, 4, 44,

4,

gu(z) = 9(X(% 4, uw, 4.. It i.5 easily

S@ell

that y"(T) in 'f:ieorerd 4 is the s~:II(? its :I(?)oi

[I,

pp. a? - ?51 apart :‘rOn tile sig. For yroOlea condition

4 a:l,iclrssc3 or' controls

:u,3 38v iL+.~[KJCt tllil,t r‘0: ti?e “~~r2ximl:.il priaciglc” minj

vtzu0

(1.q) - (1.0) L?lenx35s;r.v

('I.(J)is 3 lin3LrizLIti0n 9f.?ontrJ~giI1 5 "mximm

[f”(x(~

u),v(x),

where y(~) satisfies

shsszs

rvill

of

priilciple".

cont.rols (?.lO) - (1.71) i:l prooierir1

.iavt3 the for.2

T)Su(4+g(X(T,

4,

v(z),

(2.13)

~.)ld~=

(2.10) witi1 initial conJitioiis (2.11). Qere we

shall &lot coiIcer:lourselves witn the proof or conditiou The control wnic.1 satisiies

(7.11).

(2.4) or (2.3) will be called stczzzorccrx.

Yon-linear

nutomatic

Any optimum control is stationary

control

s,ystems

out tne converse

3s

is not true.

To obtain a stationary control we can use tie method oi successive epproximations

given in [il.

In applying these metilods it is necessary followin< r&t

u

to know the solution 01 tile

linear problem. E

’ ,‘. Ye shall denote 0y v[Ul the control v(t) E

i'.iVniCi; satis-

fies tile co&ition

To find Y"(T) it is necessary to solve tile system of linear ,iiffarential equations

(2.5) wit:] initial conditions

(3.5), and to find tile

control vi31 we must minimize the linear functional (for classes of coutrols of the form (1.8) - (1.11)). The

solution of this linear prculem is

given in the appendix.

3. Appendix

Be find the i-th coordinate of the vector function C%(T)

s;(t)

=5(~g)*q(t,T)dt+ ,B(Jy)* *

wiiere g(t) = L{(X(C, II),

ag -= ax

oi(t,T)

iIere olsCt,

q(t---ha(t),

3,

z)dtf

t

X(t

ag (,-,...,---ad =

- h,(t), ag . axn

(Ui’(t,T),

U),

1 )

....

_=

u(t)*

ag ay

Oi”(drT))

tj,

c ‘--G-

ag w ti=

ag

I7

*.*I

1’

r)*

1) (s = 1, . . . . n) is the gradient of tne functional P(t,ui) = ~S(t,ui,

.. ..

ui, . . . . 2~')

(3.1)

V.F.

40

Dem’yanov

(for fixed t and fixed functions ul, . . ., ul-l, uitl, . . ., u’), calwThe function Oi( t, T) is given for T < t_ < I’. We lated. at the point ui. transform the second integral on the right-hand side of (Ii. 1). We make the change of variables v = t-

h,(t),

dt =

t = b(v),

iz(v)

dv,

WV)

h(v)=

-.

(3.2)

dV

Then ..

+7(t)

l

_-__

ii T,W)

8Y

w(t

-

h,(t),

7)dt

=

T-h2(T)i9g(r2(t))

>

We noilr substitute

(Y.31

. rc

(

3Y

l

oi(t,

(3.3)

r)iz(t)dt.

>

in (3.1) T

G,'(r)=S

(3.4)

c*(t)co&r)dt+T, T

where

ag(t) __~ _ + Wr2(4 1 f2(t) if

1 ---i)X

c(t)=

as(t)

if

8X

all

Oi(t, Thus, in (7.1) only the I1cl:l!:LiOil t;iat from (1. 1)

v - h(v), T!ien

tc

at(v)

Wi

(t, T) =

3y-

0:

.

(v, T) dv

-

dt

.-axa,(t,

T)

T)

if

t -cQ

at(v)

-Ui(Y-hi(V),

t E[t,

initial

(3.5) T]

is UilkiloWIl. '4ots

u), u(v), v)dv +

first

xow.

t)dV +

of

(3.6)

af(4 a(L’-

’

r,(t)],

(3.7)

.- m* (t, ,T) =

wit!;

h2(T)],

~E[T-/z~(T),

T,;o

T

d

t E [O, T -

dY

co:i*Litions wi (e, z) =-j-J-

,

(3.8)

#on-linear

To fi:l-i differential

d

Y(t,

Tit-

tiie solution equatio$l

T) =

afw Y(t, __ 3X

T)f

t E b,

systems

41

we consider

tile matrix

rf(t)l,

f!.E-J’(t ay

(3.9) h,(t), T) if

t~Iri(t),

yl

conditions Y(qr)

(:’ is a unit

control

of s.vstcm (3. 7) - (3.5)

af(t) _ -_ If (4 7) if ax

I-

wit;1 iliitial

automatic

(3.10)

= E

fl x n-matrix).

Then Oi(t,T) = Y(44The relation (3.11) is proved witbout with respect to t akid tasting the initial

‘Qmtituting

(3.11)

in (3.1)

G;(r)=jq)*

!Jere c(t)

is calculated

af(f)

(3.11)

do’ difficulty data at

by Jiffereiltiatiag point t = T.

we have

ag (7) \ y* (C z) c (t) CD, + 7 5 by fomula q(T)=

tile

(2.5).

i Y’(& z)c(t)dt,

(i=i,

Introducing

. . . ( r).

(3.12)

the notation (3.13) (3.14)

we obtain (3.15)

V.F.

12

lve :iote for all

Den’ynnov

irntnelliately that tne vector i (1 = 1, . . . . r-J.

Ce uow differentiate

function

is one aid the sane

V(T)

(3.13)

am

-

dr

Ty--g d Y*(t,r)C(t)dt

s

=

(3.16)

- c(r).

r

In (3.16) the matrix fUnCtiOn .1Y(t, T)/ri~ is UihdWil. It W&S CStab1isheJ earlier that I’( t, T) satisfies the systen of equations (3.o) witn initial conilitions (3.19). Let us consider

the systen

i(t) = -

afw ax

Z(0)

The11 if

t E

[T

t

52

i’l we

L-i(T),

(3A8)

= E.

Fi(T)] Y(h

and if

(3.27)

z(t),

z)

=

(3.19)

Z(t)Z-i(r),

have

E

Y(&r)=Z(t)Z-i(r)+

afb9 Y(Y

v ) - ay

z(t)z-l(

S r,(r)

-

hi(V), r)dv.

(320)

The trutil of (7.19) aud (3.26, is verified UY differentiating with respect to t and testing Witil 2 = T. Ttlus Ve il3Ve an exulicit furm for the matrix function Y( t, T). Ia (7.15) *e require to know tne Jerivative
t

[T,

E

Fl(T)]

2

Y(t,r)

= z(t) $z-l(r)= -

Y(t,r)

fg,

(3.21)

or

(4 L&z-l(r)=-- af ax If sides ;

t E L-1(T), rl(rl(T)jl of (1.30) with respect

Y(t, z) =

-

z(t)z-‘(r)

afw

--g-

-

(1.20) to T:

Z-l(r).

is valid;

z(t)z-‘(h(f))

we differe:ltiate

af (iw ) --Z(r)Ei(r)i1(r)+ ay

both

:‘on-linear

+

automntic

s

if

= Y(t,n(~))

t~[r~(~),

ye

now of

prove tlii?

tile trlt.1 uroperties

(‘.‘?i!)

0; Of

i. 2. if

dv,

~E[I*(z),

ri(n(7))]

aftr,(4 1n(7). .

(3.22)

-ay--

for t

E

i‘l .

[rl(rl(‘T)),

3.Y

h,(t)

v+

t = r,(v)>

‘c’p prove

Y (t, ri (2))

L&O

43

(-!!$L)

ri(r,(z))],

aft%)

;i.(t,T)=-Y(t,T)>~-

virtue

systems

1;ztv - w9)Z-‘(4 .

Z(%)Z-‘(v)

1,(‘1)

dut Z(t)Z-i(rl(~))

control

tile vtr1iuit.y of

min h(t), tE[O.rl

(3. a‘?) Dy i:iduction.

that

SUppoSe

t =[rik(r), riA+‘(r)] for!nula (3. d’?) Aolds. it’e consider Differentiating (R.20) wit,1 rejpxt to T, xe ootoin

for

%

F(*;‘F)h@)+

rig.itv-hi(v) for ,iY(v - ;bl(vj, valid. Su!xtii;uthg dY(t, T) -= dr

T)/&,

ay

(2.22)

z(t)Z-‘(T)+

Z(W1tri(r))+

E W(t9,

iii.l~lCtiVt:

iu (?.?-)

5 z(t)Z-i.(v) r1WI

asaunption, ws oxain

aft4 7

l:lll0i

side

of

rik+i(dl,

at(v) --j-y Y(v-

5Z(t)Z-l(v)

(3.23)

fornuln

h&),r)dv](

Y(v,-

h(v),

‘(?.??\

-F)

is

+

rl(r))dv

(3.24)

x 1

r,v,m

x

c -

Witd) aY

ii (4

V.F.

44

Dem’yanov

In the second integral of (3.21) rl(?), because from (3.21.) - (3.22)

af(4

-Y(v--h,(YjJj~

dY(v _--- - hi (vj 1rj = a7

the lower limit it follows that if

af (.c) hl(Y),‘Cj ax-

- Y(v-

is not equal to

v - hi(vjE tz, ~(41, af (pi (7) j

Y (F-hi (v); r’i(vj j

i\

if

ay

ii (7)

v--~(v)E[~~(z),T~

or, since v - hi (vj E t7, r’i(7jl

iL

v-h(v)

if

=h(7j,Tl

v = h(t),

ri(ri(zjj1,

vEh(ri(~)),Tl,

then

dY(v -

-

Y(v

-

Y(v-

I

hi (vj, 7) -= a7

Considering

(7.201

z(tjz-1(ri(7)j+ Substituting

dY(t, 7) -d7

ing

=

hi(V),

at (7) - ax -

h(v),4

v~[~i(7),~i(~~(7))1,

if

y

Y(v-h(v),

af (vj ay Y(v-hj(v),ri(z))dv

i;l (‘3.24)

and considering

af (7) - Y(G7) ~- ax - Y(t,

r,(z))

It follows by induction that (3.25) in (7.16) we obtain

af(ri(fjj

.-~

.

‘iz)

T

)’

if

t E [7, ri(r)l

if

t E

valid

(-Y(t,

ri(7))af(ray(r))

r,(r)

(af(r~:r))).i,(r)iY’(f,r,(~))c(t)dt--c(r),

r

or

we obtain

rk+2(7)l.

[c(7),

ay

is also (3. 36) A

c(t)dr+i

(3.25)

(3.26) ii(r)

SC =-(z)’ fY*(t,r)c(t)dt-

d7

= Y(t,ri(T)).

(?.?I?)

yjy

d4W =’ - Y(t, 7)-=

h (7)

ay

af(7)

7)

Wd7))

ri(v))

we nave

(7.25) Y(&

af(4

7)

S z(tjZ-l(vj r,(r,W

I -

-

in

h(7)y

[C,

‘fl.

Substitut-

c(t)dt-c(z)

=

Non-linear

Here C(T) is given Al(T),

2.

n),

U(T),

automatic

control

by the formula

(9.5).

systems

f(T)

45

= f(X(.r,

W,

ahci VT -

T).

Substituting

(‘i. 15) in (?. 1) we obtain

The solution

of

a

proble:a

luienr

%‘e now show how to find of the fora

for

the minima

L(u)=

T

,

1

2

Theorem 2. vnrc0u.s

c Lmszs

of a linear

0’

integral

controls

functional

(3.291

Qi(t)uf(t)dt,

0 i-1

where (D(t) = (D,(t), (1.q) - (1.11).

. . . . 3,(t))

for classes

of coAxok

wit3 canstraints

It is easy to Mow that the !ninimum of the linear functional reached when tht! equality sign ilolds in (1.3) - (I, I! 1.

is

For classes 01 controls with constraints (l,?) its solutioh is obvious, it?J for classes of controls with constraints (1.9) - (1.11) me solutioii of the linear problem is obtained by tile use of LaXrange mlcipliers: (1.1 for

(2) for functional

for

those

coritrois

witn constraWs

d(t)

= -CZi(t)Sign G*(t),

I?, S) th2 iaiuimm is reamed iSI

I * * . , f,

classes of controls with constraints (‘3.39~ is reached for

t for

whim Q(t)

= 0, ae ootain

v(t)

with

t E w, II; (7.. 4) tile minimunr of the

= II;

ditd corlstrvints

(3, for classes ol co;ltruls (3. ?rlJ is reached for

for

those

i for wnich

(4) for the linear

iI Dj [I = 0, i. 9. @i(t) = 0, we oatltin

classes of functional

v(t)=

with constraizits is: rescl1e.J for

ui(t)

3 Q);

tile iniaimum of

(1.11)

COiltr9ls

T

IF -N-l(t)@(t);

-

(1, 10) the minimum of

ll~li2

ll@ll

( w(t)N-i(t)@(t)dt;

.

=

0

hera llf.Pll z=-0, if the vector v E 1; we have L(v) = 0.

fuiictida

O(t) + 0; if

m’(t) c 0, theu for any

hy

i’ransl~ted

1.

PONTWASIN, L. S.,

F!OLTYANSKII,V.G., The mnthematicol

~ISCCX~NKO, E.“.

(Matemat icheskasa

teoriya

;AMKXCLIDX, Z.V. theory

of

oot ianl

Cleaves

‘1.F.

ard nrocesses

o3t imal’ nykh crotsessov).

Fizzlatgiz,

woscow ( 1951. 2.

KPLYLOV,I. A. and C:-!ZR~‘~OI13’ YD, I7.C. 3n tile dnproxi!nations Zh. vychisl. Compllt.

3.

D-84,

‘!ath.

to

1. 33 - 37,

K4ZARTa!OV, vu.!‘.

snlution

mat.

math.

subject

functional

2,

Fiz.

P,hys.

of 5,

- 1139,

1371,

aaproximation input

nethod

proble~,~s of 1132

I!?%%‘,;: 5,

Trans.

vat.

Yekh.,

T!le maxiwm

principle 104.

of

Leningrad

State

on a convex Astron.),

1952.

control. (l’ransl.

for

onti.w: Ifin.

control Engrs.

1952.

variable

functional

successive

4m. Inst.

for

In 6ymp.

naxirtizing Yetohy

IJniversity,

19,

set.

Vest.

5 - 17,

1954.

Leninqr.

an jntc,:ra;

vychislenii,

34 - 47,

DC&l’YANOV, V. P. and RUI(INOV, A.?il. The #ninimization vex

of

optimal

1953).

technique

saturation,

ivitn

Publication 5.

Vat.

tl:e

YU-C!IJ HO, A successive systems

4.

for

of
vo.

“J.

1956. a sjnootn Ilniv.

con-

(Series