Gradient methods for solving problems of terminal control in linear automatic control systems

GRADIENT METHODS FOR SOLVING TERMINAL CONTROL IN LINEAR CONTROL SYSTEMS

~RORL~~S OF AUTO~ATlC

A.Yu. BARANOV, Yu.F. KAZARINOV and V.V.

KHOMENYUK

Leningrad (Received

15

October

1964)

TI+Zproblem of minimizing the strictly convex functional of the final state of an entity whose motion is described by a linear system of ordinary differential equations is discussed. The possible employment of gradient methods for solving this problem is investigated.

1. Formulation

of

the problem

Let the motion of the controlled entity be defined by the linear system of ordinary differential equations

dxft)

____ dt

with the initial

=

A (~)X(~)+ 2 Bj(tf uj(t) + f(t)

(d?

j=i

conditions X(0) = x0,

(21

where X, Ej 0 is the given control period. We denote by L’ the set of real measurable r-dimensional vector-functions u(t) = (ul(t!, . . . . u,(t) (called the controls). satisfying the l

Zh.

vjrchisl.

Mut.

mat.

Fiz.,

5. 5, 894 - 902. 1965.

155

158

A.Yu.

Baranov

et

al.

condition

I”j(t) 1< 1

(j = 1, . . r),

t

.)

E

[O, T].

(3)

It is required to find the control u E U (which we shall term optimal), minimizing the functional J(u) = gW(T, u))

(4)

In (41, g(X) is a real, twice continuously differentiable, convex function such that

strictly

where n is a constant > 0, and X( t, u) denotes the solution of system (1) with initial conditions (21, corresponding to the control u E c’. The existence theorem for an optimal control control problems in [Il.

is proved for terminal

2. Application of the "maximum principle" Observe that the functional

J(U)=

j [ Vg(X(t,u)),

I(u) can be written as

A(tJX(t,u)+

ZBj(t)uj(t)+f(t)]dt,

(5)

j=l

0

where X(t, u) is the solution of system (1) with initial conditions (2), corresponding to the control u E II; vg(X(t, u)) is the vector with components ag(x(t, w)/bl, . . . , ag(x(t, u))/ann; and

ix, yl = 2

w/i

i=i

is the scalar product of the vectors X = (~1, . . . , Xn) and Y = (~1,. . . ,y,,). The necessary condition (see [211 for u(t) to be optimal is that the function

x=

[q(t)_Vg(X(t)),

A(t)X(t)+

iBj(tluj(t)+f(t) ] 7

(6?

j=i

have its maximumwith respect to u E 0 on 6, where the vector y(t)

is

Solving

the solution

problems

157

control

terminal

of the conjugate system a33

d$i(t) --=-dt

satisfying

of

the “transversality

conditions” (i = 1, . . ., n).

4%(T) = 0 Thus the optimal control u(t) iij(t)

=

sign [g(t)

-

(7)

(i=l,...,n),

dxi

(8)

has the form

Vg(X(t)),

(j=l,

Bj(t)]

.,

*,

4,

(9)

where X(t) is the optimal trajectory. The solution

of system (7) under conditions

i(t)

*(t) = V&NW))

-

(8) is

(~,-‘(t))*~~((T)Vg(X(T)),

(IO).

where O(t) is the fundamental matrix of the system dY(t) -=A(t)Y(t), dt

Substituting

(JO) in (q),

Ej(l) = -sign

(11)

aqO)=E.

we get (j =

Vg(X(T))J

[Bj(t),

1, . . .) T),

(12)

where Bj(t)

=

(i

Q(T)W’(t)Bj(t)

The optimal control will therefore

=

1,

. . .,

(13)

T).

be found, provided we can find the

vector X(T) representing the end of the optimal trajectory. Theorem

1

The end XC;) of the optimal trajectory

L(Z)eZ--R+

i

is the solution Q(Z)]

fBj(t)Si@l[Bj(t),

of the equation

dt = 0,

(14)

j=iO

where I? = CD(T)XO+

@((T) j

@-‘(t)f(t)dt. 0

(15)

A.Yu. Baranov

158

et

al.

The proof of the theorem follows from the form of the control iC t) and the representation of the solutions of system (1) by Cauchy’ s formula. We multiply

(15)

by the matrix

h(Z)=D(Z)(Z-RR)+ We define

D(Z)

=

(9g(Z)

/dzidzk)

Vg(Z)]dt.

$J jo(z)Bj(t)sign[Bjo,

j=i 0

the functional

F(Z)

and put

(16)

in En

~~(~o=EVR(Z),Z--Rl-g(Z)fR(R)+

i’s j=i

(17)

Imp), Vg(qlp. 0

It can be shown after straightforward working that the gradient of $((z) is the same as h(Z). Since the matrix D(z) is positive definite, equation (14) is equivalent to the vanishing condition for the gradient h(Z) of 5’(Z). The gradient methods of [31 might be applied to find the stationary points of 9 (Z), except that 5 (2) is not sufficiently smooth. Since in practice optimal controls in autotiatic control systems are found by taking as optimal the control which gives a value close to optimal for the optimality criterion, it is worth posing the following problem: To find

the control

u E 0 such that

I

(ii) <

where E > 0 is any previously trol e-optimal.

min I (u) + E, UEU

assigned

quantity.

We shall

call

this

con-

It is clear from the statement of the problem that the a-optimal control is not unique. This fact can be used to find a continuous e-optimal control, whereas the optimal control itself may be a piecewise continuous function.

3. We introduce v(t) = (u1(t), such that

the set --.*

ur(

The

E-problem

V of piecewise t)), and denote

&Z(t) + uj2(t) <

1

continuous vector-functions by w the subset of the set

(j = 1, . . . , r).

0 x V

(18)

Solving

problems

terminal

of

159

control

We state the E-problem as follows: given the system (1) with initial conditions (2); it is required to find tne w = (6, $1 among all the w E ?I’such that the functional le(w)=l(n)+--$

_ j uj(t)dt I=1 0

is minimized for w = G. Lemma 1

Let w = (us) control. Proof.

minimize the functional

(19). Then ii is an e-optimal

Obviously,

We obtain from fl9)

so that

which means that a is e-optimal. Lemma 2

Let 2 = @,-vi) and k2 = Then X(2’,<‘) =X(2’,?). Proof. Suppose that convexity of g(X) I, (awl + (1 -a) i_ -$

i

X(T,uf)

@,F2)

be optimal for the E-problem.

We now have, by the strict

# X(2’, iz).

7”) f= g (ax (T, ii’) + (1 -a)

x (T, 2)) +

i (uvi”(t) + (1 -a)~$yb))dt
(T, 2)) +

3=10

+

(1 -a)

g (X (T,

ii”)) +

-$- .i { (c&l (t) )==l

0

+ (I-

a)vt (t)) dt =

160

A.Yu.

= al,(2)

_t (1 -a)

I,

Boranov

(2) =

et

al.

I, (G),

i.e. wa = awi f (1 -a)%” gives the functional I,(w) a value less than the optimal, which is impossible. The contradiction proves the lemma. Notice that the functional

re(W)=

I,(w)

can be written as

i[vP(x(tvu)), A(t)x(t,U)+fl

Bj(t)Uj(t)+f(t)ldt+

j=i

0

(20)

J

where X(t, u) is the solution of system (1) under initial conditions (2), corresponding to the control u E U; Vg(X(t, u)) is the vector with components ~g(~(~,u))

f&r%,. . ., ~g(X(~,u))

/ax,.

Applying Pontryagin’ s maximumprinciple tion of the e-problem is given by

i23, we find that the solu-

[Bj(% wwN1

6(t)= - ll[Bj@), Vg(X(T))]2 @j(t) = where X(T) Theorem

+

E/rT

fi[Bj@), Vg(X(T))]*+

(211

s2/rT7

(j= **LssTr)y

&2/rT

(22)

is the end of the e-optimal trajectory.

2

The necessary and sufficient condition for x(T) to be the end of the e-optimal trajectory is that it be a solution of the equation Z_fi+

i

7

3’=10

~~(~1

_

LBj(t),Vg(z)ldt

1I[Bj(Q, Vg(Z)l”

+

E2/rT

= **

(23)

This theorem follows from the Cauchy representation of the solution of system (1) with initial conditions (2) in the case of the control ii
(23) by the matrix D(Z)

=

(aZg(Z) /&c&k)

and putting

Solving

problems

I

of

terninal

pjw,w~)1~~

.T.

2 1www)

w~)=w)(~---R)+

(24)

Vg(Z)12 + S2/9T2 ’

fiBj(t),

j=i 0

we find that h,(Z)

161

control

is the same as the gradient of tne functional r ,T

Fe(z)=IVg(z),

z-RI-g(Z)+g(R)+

2

5 flEj(t),

f=i

0

Vg(Z)]2+c2/rzT2di.

Since the matrix D(z) is positive definite, equation (23) is equivalent to the vanishing condition for the gradient h,(Z) of se(Z). Properties

of the functional

F-,(Z),

1. By Lemma2 and the sufficiency in our case of the maximum principle, equation (23) has a unique solution, i.e. s,(Z) has a unique stationary point. 2. By hypothesis, g(z) is twicejcontinuously differentiable, g(R) =gjZ) + [Vg(Z),R-21 +~[D(“+e(R-i))(R’--),

so that

where 06

=

8 < 1, i.e.

-g(Z)

+ g(R)

+

[Vs(Z),

2 -RI

R--Z],

$[D(Z+B(R-z))(R--Z), R-2) 20. It follows from this and (25) that Be(Z) is bounded from below, and in view of the uniqueness of the stationary point of ge(Z), its stationary point is a minimum. 3. Since the function g(Z) is strictly exists such that [D(Z3_ we have B,(Z)

0(%-Z))

(R---.2),

convex and a constant m > 0

(R-Z)]

> mllR-22112,

(26)

+ 00 as IlZll * 00.

4.

Iterational

gradient

methods

We find the minimumof the functional s;(Z) by employing the gradient method of [31. which in our case amounts to forming the minimizing sequenceXO, Z1, . . . , Zk, . . . in accordance with the formula 2h.Q = Zk - Ukhe(Zk), where a variety of methods is avilable

for choosing the interval

(27) ok>O.

162

fn

A.Yu. Baranov

Simple

descent

et

al.

iteration we put ak = const., while Uk is determined from the condition

$e(Zk -

= ;E

akhe(Zk))

in the method of steepest

C,(Zk - ahe(Z

(2s)

Lemma 3

Whatever the c >-sm&l L(Z) n the set

S =

(2: a,(Z)

<

C}

is compact.

Proof. To prove that S is compact, we only need to show that it is bounded, since the former is a consequence of the latter in the space En. But the boundedness of s follows from property 3 of the previous section. Theorem

3

In the method of steepest descent,Zk minimum of the functional s,(Z).

-. Z* as k -. co, where Z’

Proof. It follows from Theorem 2 of h,(Zk) -) 0 as k -, co. est descent,

[31 that,

is the

in the method of steep-

In view of the compactness of the set ,?, whatever the initial Z” of the sequence {Zk), a convergent subsequence {Zkj) can be extracted, such that

Xki

3

2”

as

i-+ 00.

It was mentioned

Since

in Section

h,(Z)

is continuous,

3 that the equation

we have h,(Z*)

h,(Z)

=O.

= 0 has a

i.e. all the convergent subsequences {Zkj) are conunique solution, vergent to the same limit Z’. Using this, we can show that the sequence suppose the contrary to be {Zk) is itself convergent to Z*. In fact, true, i.e. that a number p > 0 exists such that, whatever K > 0, there k> K exist such that I(Zk -Z*jl 2 p. There must now exist an infinite subsequence other

{Zkj)

hand, since

{Zkj 1) , i .e.

such that

llZkj -

S is compact,

we can find

Z”II >

a K > 0 such that,

Z’ (1 < p. This contradiction

p.

for

we can extract

proves

for

our theorem.

all

its

terms.

a convergent kj l>K.

On the

subsequence

we have

11Z kj l_

Solving

Theorem

problems

of

terminal

control

163

4

If

h,(Z)

satisfies

the condition

llhe(Z + Y) - he(Z) II < CIIYII,

(2%

is convergent tne sequence {Zk) in the method of simple iteration (the minimum point of se(Z) J with 0 < a < 2/C and any z”, ge(Zo) We find

Proof.

from Theorem 3 of

$s(Zk+i)-

00.

[31 that

u($

bs(.q<

to .Z* <

- 1 ) Ilhe(Z) II2< 0

and

Ilk(Z) II2Q

irep) - JL(Zk+i) a(1 - aC/2)

Hence s,(Zk) while

monotonically

--t F*

non-increasing,

follows

from this,

k-t

CO,

(30)

and

h(Zk) +O It

as

as

k-too.

using the compactness

(31)

of S =

{Z: s,(Z)

<

and the uniqueness of the solution of the equation h,(!) (on the basis of the ssme arguments as in the proof of Theorem 3),

9, (ZO))

Zk+Z* and this

Z* is the minimum point

as

that (32)

of 5;(Z).

,Votk. The existence of bounded third cient for condition (29) to be satisfied. Theorem

k-too

= 0

derivatives

of g(Z)

is suffi-

5

If h,(Z) satisfies condition (29), the sequence {Zk) in the method of iterations is convergent to Z* (the minimum point of .Fe(Z) 1, however the ck is chosen from a (previously assigned) interval [m, VI, containing (0, 2/c) as an Llterior interval, for any initial .P,

l e(ZO)

At the same time, .Fe(ZA) + $*(Z*) < 00. being monotonically non-increasing.

as

k+

00,

while

164

A.Yu. Baranov

et al.

The proof of Theorem 5 follows from the fact that he(@) + 0 as k + 03, and from the com~ctness of the set 8 = (2: a,(Z) < $,(Z*)). If extra restrictions (as e.g. in [S]) are imposed on the functional an assessment of the rate of convergence of these methods may 8, (2) be obtained.

5. Continuous

gradient

aethods

In continuous gradient methods, instead of forming a sequence of the type (27)‘ we consider the solution ?(T) of

--Wz) = - cz(T)h&(~)),

(33)

dz

lwhere a(T) is a summablescalar non-negative Theorem

function.

6

In continuous gradient methods:

2) Z(r) -) z* as r-,co,where

2‘ is the minims point of F-,(Z);

3) $8 (Z(r)) + 88 (Z’) as T + co, while monotonically decreasing, whatever the choice of function al-r) (a(T) > a0 > 0 for r 20) and initial approximation z(O) = .z*, S8(z”) < O”. Proof.

Since

and a,(Z)

is bounded from below, a real number 8”

exists such that (35)

while monotonically decreasing. -

We find by integration

{ a(6)ll~e(~(~))l12~~= Se(Z(Z)) 0

and, since,

- Se(Z*)) 3

for any o(T) 2 ao > 0, and arbitrary

that

9’ - 8e(z”),

1.3 0, we nave

Solving

prablens

II~s(q~))l12~~<

i 0

the integral on the left-hand + 0 as ~-,a. h&(T))

of

control

terminal

~&q-

%*< + a0

165

o.

(361

,

side of (3e) is convergent as

T

-f Co,

i.e.

We now show that Z(T) is convergent as T -) 03. We take an arbitrary sequence of -rk>O such that T&-, 0;)as k -, ~0, and consider the corresponding sequence @(T,)). Since ~~(z(~)) is monotonic in T for any k>,O, we have Z(Q)*E~ = (2: $“e(Z) < ~~(Z~)~, and it follows from the compactness of s and the uniqueness of the solution of the equation h,(Z) = 0 (repeating similar arguments to those used in Theorem 3) that z(‘r,) + Z* as k+m, where h,(Z*) = 0. Since the sequence {ok) is chosen arbitrarily, T + a. The proof of our theorem is complete.

we have T(T) -) Z* as

By selecting a(7) in different ways, realizations gradient methods may be obtained.

of continuous

6. A quadratic functional In the important case of the quadratic functional g(Z) =

the functional

(with Fe(R)

Jr,(z)

$ $,z,,

t=i

(37)

has the form

= 0).

The functional (37) is twice continuously differentiable with respect to 21, . ..) z, and strictly convex. It is also obvious that, for sufficiently small E > 0.

166

A.Ya. Baranov

et

al.

fh*(Z + Y) -k%(Z), Y] 2 (I- 62)llY112, llk(Z + Y) - h(Z) II < (2 - 62)IIYII, where 6 # 0 is an arbitrary

constant (0 < 6’ < 1).

It follows from this that, no matter how ok is selected from the interval tm, Wl E (0, 2/(3 - S2)). discrete gradient methods are convergent for any initial Z*, 11Z* 11< a), while 131II Zk - 2' II < Coqk, where q = I - df(l - ~3~)2, 0 < q < 1, CO > 0 being a constant dependent on Z*.

7. Remarks 1. Theorem 6 also holds for E = 0. It follows from this that continuous gradient methods can be used to find the optimal control. 2. Generally speaking, the determination of ok in the method of steepest descent requires an infinite computational procedure. This defect is not apparent in iteration methods, whose convergence follows

t InPa

cc,6. z 0)

1

from condition (29) being satisfied. To satisfy (291, it is sufficient for the operator h,(Z) to be smooth; this is the case when e-optimal controls are considered and when bounded third derivatives of the function g(Z) exist.

Solving

3.

f’(aL(Zl

problems

of

terminal

16‘7

control

The operator equation L(Z) = 0 is equivalent to the equation = 0, where P(Z) is a continuous non-singular n x n

case continuously able with respect to X, we find on employing,the gradient methods described that the values of the functional Fe(Zk) are monotonically decreasing and he(@) + 0 as k -, co. These methods therefore enable us to find the stationary points of the functional Fe(z). The controls based on these have to be further examined for e-optimality. 5. It must be mentioned that, when the proposed methods are used to solve a problem by computer, the main computer time is spent in evaluating h,(Z), since it is necessary to evaluate at each-step the integral on the right-hand side of (24). while to determine Ej(t) (j = 1,. . . r) we have to compute the matrix (@( t,@-l(t),. A block diagram of the computer solution of a problem, using discrete methods, is shown in the figure.

6. The results obtained are readily extended striction on the control u(t), as for instance: (a) (b)

(j=l,

Iui (t) / < 9j (t)

~,%(+j2(t) G

cp’(t),

ajr

to other

types

of re-

. . . . r),

g,=J52(0,

T),

aj

>

0;

j=l T (cj

s

aj

(t)

Uj2(

t) Lit <

(j =

pj2

1, . . . ? r),

0

Uj > 0,

aj ELz(O,T),

(d)

~CW,

QWWW

<

p2,

p =

pj=

COIlSt;

const,

0

where ‘2(t)

is a positive-definite

r x r matrix.

7. The problem of minimizing the functional where 3 is a real symmetric positive definite in [41. Using the change of variable

g(X(?)) = [X(T), ?/X(7)1, n x n matrix, is considered

this functional may be reduced to that considered in Section 6. As distinct from the method of [41. not only the convergence of discrete

168

A.Yu.

gradient methods, Section 5.

but their

The authors express rest and comments.

their

Baranou

rates

et

al.

of convergence,

sincere

are obtained

thanks to V.I.

in

Vuzbov for his

Translated

by

inte-

Brown

D.E.

REFERENCES 1.

KIRILLOVA, L.S. Problem of the optimization of the final state controlled system, Avtomatika Telemekh. 23, 12, 1564 - 1594,

of a 1962.

2.

PONTRYAGIN. L.S., BOLTYANSKII, V.G., GAMKRELIDZE, R.V. and MISHCHENKO, E.F. Mathematical theory of optimal processes (Matematicheskaya teoriya optimal’ nykh protsessov), Moscow, Fizmatgiz, 1961 (Transl. by Pergamon Press).

3.

POLYAK, B.T. Gradient method of minimizing functionals, Zh. Mat. mat. Fiz. 3, 4, 643 - 653, 1963.

4.

DEM'YANOV, V.F. Contruction of an optimal programme in a linear 25, 1, 3 - 11, 1964. system, Automatika Telemekh.

vphisl.