Nonlinear optimal controls

JOURNAL OF COMPUTER AND SYSTEM SCIENCES: 4, 29--37 (1970)

Nonlinear Optimal Controls A. A. GOLDSTEIN*

University of Washington and Boeing Scientific Research Labs AND J. S. ~IEDITCH

Boeing Scientific Research Labs Received November 22, 1968

ABSTRACT

In this paper we state a necessary condition for optimality for unconstrained controls. For certain systems a constructive proof is given showing the existence of controls satisfying this condition.

I. STATIONARY CONTROLS

For the system

~(t) = f(x(t), u(t), t),

(t)

on the fixed time interval [0, T] with fixed boundary conditions x(O) and x(T) E E~, assume that there exists a continuous control u, u(t) ~ E r , such that T

fo g(u(t)) dt is minimized. Assume further that f has partial Jacobians with respect to x and u, denoted respectively by Jx(x(t), u(t), t) and J~(x(t), u(t), t), and let Xu(t ) denote the fundamental matrix of the system S0(t ) = Jx(x(t), u(t), t) x0(t),

(2)

with u fixed, where x(t) is the corresponding solution of (1), and xo(t ) the solution of (2). We denote by U the Hilbert space of r-tuples of square-integrable functions u on [0, T], with norm I1u ]i2 = f0rl u(t)l 2 dt, where I u(t)] * = [u(t), u(t)]. By L we denote the Hilbert space of n-tuples of such functions normed as above. 9 S u p p o r t e d by G r a n t A F - A F O S R - 9 3 7 - 6 5 .

29

30

GOLDSTEIN AND MEDITCH

In the sequel, terms such as J~(x(t), u(t), t) will often be abbreviated as J~(t) or Jx, etc., depending on the context. If P : U • L -~L, then the abstract partial of P with respect to u, written P~', is the Fr~chet derivative of P with respect to u with x fixed. We define P~' similarly. For the differential o f f at x in the direction h, we shall use either f'(x, h) or f'(x)h. The same symbol f[ [[ will be used for norms in various spaces. Let g be a Fr6chet-differentiable function from U to L2[0, T] and f a continuous function from E, • Er • R to En with continuous partial Jacobians with respect to x and u, respectively. Note that g'(u) is a bounded linear operator from U into L2[O, T]. r Since g'(u, h) = ~i=1 (~g/~u,) hi, we shall identify g'(u) and (~g/~u x .... , Og/~u~). THEOREM. A point u E U minimizes frog(U(t)) dt -~ ~(u), subject to the satisfaction

of the system ~(t) = f (x(t), u(t), t),

(I)

x(0), x( T) fixed in E, , only if for some point ~ E E, , the foUowing equations are satisfied: P T

j f(x(t), u(t), t) dt -~ x(T) -- x(0), 0

g'(u)(t) + 1*X~(T)

(II)

Xul(t) J~(x(t),

u(t), t) = O,

a.e. on [0, T], where * denotes the transpose. We give two proofs of this result, the first via the maximum (minimum) principle [1], the second a direct proof by means of the multiplier rule [2], utilizing the implicit function theorem [3]. Because the usual sufficient conditions for the multiplier rule are rather strong, the second proof is less general than the first. The inclusion is justified however because formulae are developed in the proof which are needed in the sequel.

Proof 1.

From [1], the Hamiltonian for the above system is

H(% x, u, t) : ~o*f(x, u, t) + g(u), where ~0EL is subject to the system

~(t) = -J~*(x(t), u(t), t) ~o(t). That H is a minimum for u E U with 9~ and x fixed implies the condition ~*(t) J~(x(t), u(t), t) + g'(u)(t) : 0 a.e. in [0, T].

(IiI)

NONLINEAR OPTIMAL CONTROLS

31

For any A ~ E . , it is easily verified that ~(t) = [ X ( T ) X~I(t)I*A is a solution of (III). It now follows from the reference that a necessary condition for u ~ U to be an optimal control is that there exist A 6 E n for which

g'(u)(t) + A * X ( T ) X~l(t) J,(x(t), u(t), t) = 0, a.e. in [0, T]. Finally, satisfaction of the given boundary conditions requires that i * 7"

J f(x(t), u(t), t) dt = x ( T ) -- x(0). 0

Proof 2.

Set

P(u, x)(t) = x(t) -- x(O) --

f(x(s), u(s)) ds.

(IV)

0

Given u, any x satisfying the equation P(u, x) = 0, satisfies the differential equation of(I). Here, P : U • L--~ L. For fixed u, let F(u)(t) satisfy i~(u) = f ( x , u) and F(u)(0) : - x(0). In this notation, a function u o satisfies the given boundary conditions if

F(uo)(T ) = x(T). By a theorem of Liusternik [2], if (/) has a local extremum subject to the condition F(u, T) -- x ( T ) = O, and k ---~F'(u, k)(T) is onto En, there exists a point l ~ E n such that qS'(u)k -- l*F'(u, k)(T) = O, for all k ~ U. Our aim below is the computation of

F'(u). For our setting, the implicit function theorem of Graves (see Kantorovich [3]) states that if for some u o , x0, P(u o , Xo) = O, Pff and Pv' exist and are continuous at (u 0 , Xo), and Px' has an inverse, then for some neighborhood N of Uo, there exists an operator F : N - - ~ L such that F ( u o ) : Xo, P ( u , F ( u ) ) : O, for all u ~ N , and F'(u) -- --[Px'(u, x)] -1 P,'(u, x). Differentiating (IV), with respect to x we get the formula

h = Px'(,, x)k = k(.) -

f(

.)

L(x(,), u(s)) k(s) d,.

0

Clearly Pff is a bounded linear operator from L to L and, moreover, Px' is continuous on U @ L . Differentiating the above result with respect to t, assuming u and k smooth, we get

~(t) = L ( t ) k(t) + ~(t).

32

GOLDSTEIN AND MEDITCH

This has the solution (taking k(0) = 0),

k(t) = x ( t )

f' x~(s) ~(s) as.

(v)

0

We may write this as the operator equation k = [G'(~, x)]-~ h,

where p, [ x (U, x)]-lh = X ( . )

fo) o

dh .Xul(s ) ~-$ ds.

Integrating (V) by parts, we get k(t) = h(t) -- X (t) f ' o dsd S~l(s) h(s) ds~

Differentiating X[, ~X,, = I and using Xu -----Jx(t) X . gives ds

x~l(s) = - x ~ ( s ) L(s).

Thus, t

k(t) = h(t) + x.(t) f o x;'(,) L(,) h(,) as. Clearly the above holds for arbitrary u and h in U, whence .)

[Px'(x, ,,)]-lh = h(.) + x,(-)

fl0 x~l(s) L(s) h(s):as,

showing that [Px'(x, u)] -1 is a hounded linear operator from L to L. Next we calculate that l') P.'(x, u) ,~(') = -J,,(s) ~(s) ds, 0

f

(vI)

and infer that the hypotheses of the implicit function theorem have been satisfied. We conclude then that

F'(u) = - [ G ' ( x , ,,)]-'[P.'(x, u)]. Setting k -- [Px'(x, u)]-lh with h = P,,'(x, u), we get, using (V) and (VI), that k = - x . ( . ) f'o" x ; ' ( s ) ~

= &(.)

(')

fo

h(,) ds

(VII) x~-l(s) L(s) ~(s) as = F'(u)

33

NONLINEAR OPTIMAL CONTROLS

If our system is quasi-controllable (see definition below in part II), the map F'(u)(T) will be onto E,~ and we can use the multiplier rule, viz: T

T

f g'(u)(t) k(t) dt + A*Xu(t ) fo Xua(S) Ju(s) k(s) ds = O, 0

for all k e U. Because this equation holds for all k e U, we conclude that

g'(u)(t) + A*Xu(T ) X~1(t) ]u(t) = O.

(VIII) Q.E.D.

Condition V I I I can be written as + ~ Aia/ = 0, ~Uj

j = 1,..., r,

(VIIIa)

i=1

where c~fl(t) is the ijth element of the n • r matrix a ( t ) = X~(T)X'~I(t)J~(t). We are thus seeking an r-tuple of functions u, so that the r-tuple of functions (Og/Ou1 ,..., Og/~u~) belongs to the linear span of the rows of a and such that the boundary conditions (II) are satisfied. Finally, we remark that the hypotheses of continuity, differentiability, etc., could of course be restricted to small neighborhoods containing the extremal u.

II. EXISTENCE OF STATIONARY POINTS Our motivation is the foPowing. Given a mapping G from an inner product space E to itself, we seek a point z ~ E, if such exists, for which G(z) = O. Suppose G has a derivative G'(x, .) which is onto E. T h e n if(x) = (G(x), G(x)} has the derivative ff'(x) = 2(G(x), G'(x, .)), and (G(z), G'(z, ")) = 0, implies that G(z) = 0. If we can generate a sequence {xk} converging to z such that ~'(z) = 0, then G(z) = O. Consider the Hilbert space H = U @ En, and define the inner product {., .} on H as follows. I f p l = (u I , Aa) andp2 = (u 2 , A2), then @1, P2) = J'~ [Ul(t), u2(t)] dt + [A1, A2], where [.,.] is the inner product on E n . We define a mapping G : H - ~ H by

j=l T

,

LEMMA. Let U' C U be such that fo a(u, s) c~ (u, s) ds has rank n, o~is differentiable

and g is twice differentiable, for all u ~ U'. Then G'(u, ~) is onto H, for all A ~ E,~ . 571141I-3

34

GOLDSTEIN AND MEDITCH

Proof.

We compute that

a'(u, a)(zJu, zJa) =

(,

F. a::~(u, zJu),F'(u, zJu)(r

"(u, ., zlu) -

--

zla,.~(u),

o) ,

(x) where T

e'(u, Au)(T) = x ~ ( r ) f o X;I(s) Jr(s) Zu(,) as

= fr ~(s) Zu(s) as, 0

by (VIIIa) and (VII). Take (% ~) in E and (u, A) ~ U' • E~. We show that for some point (Au, AA), G'(u, A)(Au, A A ) - (~, ~). Stated otherwise, we seek (Au, AA) to satisfy g#(u, .Au) -- a*'(u, Au)A -- a*(u) AA = 9(u) (a)

f~"~,(s)ztu(s) as

= ~

(h)

0

To satisfy (b), set Au = a'g, and find it z E,~ from (h). From (a) we have, after premultiplying by a(s) and integrating, that

fr

~(s) ~*(s) .4a as =

~,(s)[e'(~) - ~(s) - ~*'(~, ~u)]

0

as,

0

which yields a unique solution for AL DEFImTION. A point (u*, A*) in E, for which system (II) is satisfied, will be called a stationary point. DEFINITION. Any system (I) for which T

rank f a(s) a*(s) as

is n,

0

for all u ~ U' will be called quasi-controllable on U'. THEOREM. ChoosePo = (uo , Ao) arbitrarily in E. Let S -----{p ~ n : qC(p) = (G(p), G(p)> ~< f#(Po)}. Assume S is bounded in H and that on S, g'(u), X-~t, J~(u), Xu and Ju(u) exist and

35

NONLINEAR OPTIMAL CONTROLS

are uniformly bounded. Assume that the system (I)/~ quasi-controllable on S. Then a sequence {p~} can be recursively defined with the following properties: (a) {G(p~)} --} 0 (b) I f the mapping u--} g'(u) has a continuous inverse on S, then there exists a stationary point ~ such that {Pk} --~ P.

Proof. Set A(p, ~,) = &(p) _ ~ ( p _ ~,Vf~(p)), and g(p, ~,) = A(p,y)[y It v&(P)[]~. Select a, 0 < o ~< 89 Set P~+I = P~ if ff~(p~) = 0; otherwise, choose )'k so that o <~g(p~, y~) ~< 1 -- o, when g(p~, 1) < a, or ~ : 1, when g(p~, 1) >/o, and set Pk+l = P k - )'kV~(Pk) 9 By the result in [4], we have, provided that V~(p) is uniformly continuous on S, that {ff&(p~)}--* 0. With this hypothesis, we have that {89

= {(a(p~), V'(p~, " ) ) ) - , 0.

Assume temporarily that c, is differentiable on S. We also havep, e S, so that G'(p,, ") is onto H. Then, choose h, so that G'(p,, h,) = G(p,) to obtain (a). To get (b), take a subsequence {Pk) so that {A~,} converges to A~. Then,

j=l

j=l

where {~} ~ 0. Therefore,

= (e')

("•

\j=l

+

j=l

+,

b

and lim{u~) : u -----(g')-1(~=l ~ J ) . Clearly (u, ~) is a stationary control. We now show that V~(p) is Lip-continuous in S. The following facts are useful. Suppose f is a mapping between normed linear spaces E and F such that the G~teaux differential G' exists on some subset S orE, and that sup{llf'(x q- tk, h)ll/ll k I1: t ~ (0, 1)} is finite, for all x E S and k =/~ 0 in E. Then by the generalized mean-value theorem, f is Lip-continuous. I f f and g are Lipschitzian mappings defined on S, and the range o f f and g is bounded in norm on S, then fg (if this makes sense) and ~f + fig (for all real scalars ~ and fl) are Lipschitzian. A computation shows &'(p, .) is Lip-continuous on S if II G II and II G'(p, ")ll are uniformly bounded on S, and G'(p, .) is Lip-continuous on S. We turn then to this latter question. Using (X), we calculate that II G'(Ul , ~1) -- G'(u2 , ~m)ll

~< II g"(ul) -- g~(u~)ll +

I[ ~'(u011" II )il -- a211 "-]- II o~'(Ul) -- o~'(u2)ll 9 1] )t2 ]l

+ iT (a(ul, t) -- a(u,, t)) dt q- [1 a(u~) -- a(u~))]l. dO

36

GOLDSTEIN AND MEDITCH

Thus, if g'(u) and o~'(u) are uniformly bounded on S, say by fl and 7, respectively, and o~'(u) is Lip-continuous on S with constant 3, and ~ and ~r bound ]] A ][ and ]] ~(u)]], respectively, on S, then II G'(ua, Ax) -- G'(u~, A2)II ~< (/~ § T 9 + ,~p. + ~)ll ut -- u~ [I + II Ax - 'X~I1 ~< M [[(u, -- u2), (At -- A')][,

(XI)

where M = max{3 + T9 + 3~ + ~r, ,~}. Formula (XI) will be verified therefore if we check that ~'(u) is Lip-continuous. Since ~(t) = X.(T) X~*(t) J.(t), we examine first the ingredients X.(T) and X=X(t). Set X(u, t) = X.(t). Since all varied trajectories must pass through x(0), we must have that X,,'(u, h, 0) = 0, for all h e U. Differentiating X(u, t) X-l(u, t) = I, we conclude that X-~a'(u, h, 0) = 0. Again differentiating the differential equation for X, we get

~.'(u, h) = L(x, u) x.'(u, h) + L'(x, u, h) X(u, h), whence

X~,'(u(t), h(t), t) = X(u(t), t) f X-l(u(s), s) ]~'(x(s), u(s), h(s)) X(u, s) ds. (XII) 0

Since 2 f f - 1 = - X - t J x ( x , u) and X *-t is the fundamental matrix for the system = --]~*x, we get X U ( u , h) = -x*-~O,, 0

f'o L ' ( x, u, h) a,.

(xln)

Thus X.'(u) and X~V(u) are bounded linear operators from L to B(L, L) (bounded linear operators from L to L). Since [[ X(u)l [, II X-l(u)ll and II J~'(x, u, ")11are uniformly bounded on S, it follows from (XII) and (XIII) that I] Xu'(u, ")][and I[ X~t'(u)H are also uniformly bounded on S. Thus, X(u, .) and X-t(u, .) are Lip-continuous on S. Since ]] J~(x, u,., .)[[ is uniformly bounded on S, ]~'(x, u, .) is also Lip-continuous on S. It follows therefore from (XII) and (XIII) that )~,'(u, .) and X~l'(u, .) are Lipcontinuous on S. Since [I ]:(x, u,., ")l[ is uniformly bounded on S, J~'(x, u, .) and J(x, u) are Lip-continuous. Therefore all the ingredients of c~' are Lip-continuous so that ~' is itself Lip-continuous on S.

Remark. Consider the problem of minimizing

f

rF ~

1~

subject to

9 (t) = A(t) x(t) + B(t) u(t),

x(O), x(T)

given.

NONLINEAR OPTIMAL CONTROLS

37

I f the system is controllable, then an application of the above theorem asserts the existence of stationary points, while, by convexity considerations, a stationary point is an extremal.

REFERENCES

1. L. S. PONTRYAGIN,V. G. BOLTYANSKn,R. V. GAMKRELIDZE,AND E. F. MISHCHENKO."The Mathematical Theory of Optimal Processes." Wiley (Interscience), New York, 1962. 2. L. LIUSTERN1KAND V. SOBOLEV, "Elements of Functional Analysis." Ungar, New York, 1961. 3. L. V. KANTOROVICH,"Functional Analysis in Normed Spaces." Pergamon Press, New York, 1964. 4. A. A. GOLDSTEIN. "Constructive Real Analysis," p. 151. Harper, New York, 1967.