Necessary conditions for Chebyshev-Bolza optimal control problems

Necessary, Conditions for Chebyshev-Bolla Optimal Control Problems by FREDRIC

W. HEMMING

Bell Telephone Laboratories, Whippa?y, New Jersey andv.

DAVID

Inc.

VANDELINDE

Department of Electrical Engineering The Johns Hopkins University, Baltimore,

A performance

ABSTRACT:

plus

and integral

terminal

First-order

necessary

index

consisting

Maryland

of a Chebyshev

cost is appl,ied to the optimal

conditions

necessary

conditions,

analytic

properties

of this class of qstems.

are

derived for a

exa,rnples

are

of dynamical

class of q&ems.

large

worked

absolute maximum control

in

functional systems.

Utilizing

the

demonstrating many of the

I. Introduction

Consider

the absolute

maximum

functional,

on the interval [t,, $1, given by

This functional was first used as a performance criterion by Chebyshev in analyzing certain dynamical systems (1). Such a performance criterion, which is of interest in modern control theory, can be contrasted to the usual Bolza type performance index which consists of terminal plus integral cost (2,3). Generally, these two types of control problems have been treated separately in the control theory literature. In this paper, a synthesis is investigated which results from adding a Chebyshev absolute maximum functional to the Bolza performance index. The resulting “ChebyshevBolza” class of optimal control problems is herein called the main problem or Problem I. Minimax control problems constitute an important subclass of the main problem. A number of approaches (2-10) currently exists for various classes of minimax optimal control problems [also see the extensive reference list in Ref. (2)]. In particular, Warga (8) extended the calculus of variations in proving existence theorems and deriving relaxed necessary conditions as well as utilizing the duality between minimax control problems and control problems in a restricted or bounded state space (9,lO). More recently, Barry (3) demonstrated an equivalence between a class of minimax optimal control problems and the problem of Mayer in the calculus of variations. This equivalence, and thus first-order necessary conditions, hold for those performance criteria for which g is unimodal along the optimal trajectory x*.

123

Fredric W. Hemming and V. David Vandelinde Chebyshev-Bolza optimal control problems have been defined in Lee and Markus [(ll), p. 2601, where an existence theorem is proved. If the dynamical equations are linear in Problem I and the performance index takes the form of a Banach space norm, then one has a minimum effort control problem which reduces to finding Chebyshev sets (12) in a Banach space (13-16). In this paper classical variational methods are coupled with recent results from optimal control theory with bounded state variables (17) to derive a set of first-order necessary conditions for Problem I. The main difficulty with the classical technique is calculating the first variation of the absolute maximum functional. However, once this is accomplished, this approach gives a unified treatment of Chebyshev-Bolza optimal control problems and a new set of necessary conditions. A number of analytic examples are worked to illustrate the results.

ZZ. Statement

of the Problem

Let z(t) represent the state of the system at time point t. It is assumed z(t) E En, n-dimensional real Euclidean space. The control a is selected from the set U, which consists of all m-dimensional, piecewise continuous, real valued functions on [to, $1. Problem I. Minimize over the class of controls

U the performance

J1 = +(x(tr)) + max cg(x(t)) + %(x, u) dt f&t=& s to

index (2)

subject to the constraint W

= f(x(% W)

(3)

given the initial conditions t, and x(t,) = x0. The constant c > 0 is a weighting parameter. The terminal cost function 4 need only be a function of the final state, while g and L are defined for all values of the state and control. A solution to Problem I is assumed to exist. By a solution to an optimal control problem we mean the minimizing control u* and the corresponding trajectory x* such that the cost is finite. Conditions on the real-valued functions +, g, L andf which guarantee exact solutions to this problem are given in Ref. (ll), p. 260.There are two auxiliary problems, defined in the following development, which relate to the structure of the main problem. 111. An Zmbedding

Property

An optimal control problem with bounded state variables and fixed weighting constant k is now defined. Minimize over the class of controls U, the performance index J, = &x(t& +

124

“L(x, u) dt s !-a

(4)

Journal of The Franklin Institute

Necessary Conditions for Chebyshev-Bolza

Optimal Control Problems

subject to the constraints g(t)

=f(x(t), u(t)), x&l) = x0,

(5)

g(x(t)) < k,

(6)

t E [&I,tfl,

where U, = {U 1u E U

and

g@(t)) 6 k,

(7)

t E [to, $1).

There are two regions of state space which are of interest in bounded state variable problems. These regions are the free region and the boundary region. The boundary region is given by M) The optimal weighting

constant

I g(m)

(8)

= 6

k* for Problem

I is defined by (‘3)

k” = ~n~s(z*(t)). Theorem I If the main problem has a solution then there exists an optimal problem with bounded state variables which has the same solution.

control

Proof: The proof is by contradiction. Consider the bounded state variable problem defined above with k = k*. Assume there exists UOE U,, such that e&.(UO)
(10)

It follows that max cs(xO(t)) + J&O) < pGE;eg(z*(t)) +J&*) loQ
(11)

Or 4(u0)

which contradicts

the optimality


(12)

of u*. Therefore

Js(u*)
u E u,..

(13)

It follows from Theorem I that the optimal trajectory for the main problem region, which will be called the will have free regions and a “boundary” absolute maximum region. The absolute maximum region is given by W)

I dx(t))

(14)

= k*$

If one is given k* then Problem I is reduced to a bounded problem. In general, however, the value of k* is not known.

state variable

Corollary 1 If the main problem has a solution then the optimal cost Jf is given by

J: = ~J&,), where uk is the optimal corresponding to k.

Vol.290,

No. 2,

August 1973

control

for the bounded

(15)

state variable

problem

125

Fredric W. Hemming and V. David Vandelinde IV.

Derivation

of the Necessary

Conditions

The second auxiliary problem is an optimal control problem with bounded state variables in which the fixed weight k is replaced by a floating weight g(x(t,)), tl~ [&,,$I. Minimize over the class of controls U the performance index J = $(x($)) + cg(x(t,)) +

“I&, u) dt s lo

(16)

subject to the constraints W) = f@(t), u(t)), g(W)

@Cl) = x0,

(17)

(18)

$1,

t E PO9

G MU),

(to- tJ (tf- tl) < 0.

(19)

Note that t, depends on the choice of control and may not be unique. Lemma 1 The optimal control problem with bounded state variables and floating weight is an alternative definition of the main problem. The necessary conditions are derived using this second definition of the main problem. Initially, assume a single absolute maximum occurs for t E Vl, GJ,

t, < t, < t, < tf.

Then, using the dynamical equations, the constraint given by Eq. (18) can be differentiated p-times where p is the first integer to yield an explicit function of the control, i.e. ;g(z*(t))

= g’qr”(t),

u*(t)) = 0.

(20)

For Eq. (20) to imply Eq. (18) it is required that the following point equality constraint be satisfied for i = j = 1

y[x*(ti),ti,t*]

=[

(l-x@;?“11

=O,

(21)

where x(tz, tj) = The constraints (17)-(21) give the Lagrangian*

1,

0,

ti=tj, otherwise.

(22)

can be adjoined to the performance index (16) to

* Evaluated along the minimizing trajectory. Asterisks are omitted for convenience.

126

Journal of The Branklln Institute

Necessary Conditions for Chebyshev-Bolxa

J’ = I@ +

Optimal Control Problems

+B(tcl-tl) (tf - tl) + cgb&)) +pT(t,) Y[x(tJ, t,, 41

s

ll-[L+hT(f-i)]dt+

lo

+

r”

s

t2-[L+XT(f-i-i)+yg(~)]dt

h+

[L+hT(f--)]dt,

J 1st

where X, y, to and ,6 are Lagrange multipliers and T denotes transpose. The plus and minus signs on t, and t, indicate that the limit is taken from the right and left, respectively. Breaking up the integral in Eq. (23) allows for possible discontinuities in the Lagrange multipliers. Assuming the required differentiability properties on +, g, L and j’, then along the minimizing solution a necessary condition is that the first variation of J’ must vanish (18). Since the details of this calculation are straightforward and are performed in Ref. (19), they are not duplicated here. In the general case of N absolute maxima the point equality constraint given by Eq. (21) is adjoined to the Lagrangian at time points t,, i = 1,3, . . ., 2N - 1, with j = 1. Setting the first variation of J’ equal to zero gives first-order necessary conditions for Problem I. The development to this point gives results which are identical with the bounded state variable necessary conditions of Bryson et al. (20) with the exception that their Lagrange multiplier pl(tl) is replaced here by the weighting parameter c. Note that the function g in Eq. (16) could have been evaluated at tj instead of t, for any j, j = 3,5, . . . . 2N - 1. Calculating the first variation of J’ for each of these cases yields the correspondence with Ref. (20) that i = 1,3, . . . . 2N-

/+(ti) = c,

A stronger set of necessary conditions If the integral term

1.

(24)

can be obtained.

tit1 yg’P’ dt ti

I

(25)

in the Lagrangian is integrated by parts, p-times, before calculating the first variation, then this yields the bounded state variable necessary conditions of Speyer and Bryson (21) where their Lagrange multiplier v is given by

p2 + v(Q

( -

l)P-1

y’P-2’

=

,

i=1,3

,..., 2N-1,

(26)

2N.

(27)

t=ti

v(&)

Vol. 206, No. 2, August

=

1973

i=2,4

,...,

127

Fred&

W. Hemming and V. Da&d Vandelinde

By using a generalized version of the Kuhn-Tucker Theorem in a Banach space, Jacobson et al. (17) improved upon existing bounded state variable necessary conditions. It follows from Theorem I that their necessary conditions hold for Problem I. This gives v&)

= Y&) = . . . = VP(&)= 0,

i = 1,2, . ..) 2iV.

Also, if p > 1 is odd then the optimal trajectory maximum region only at isolated points.

(23)

can strike the absolute

The necessary conditions Letting

71= (- l)Py(P--l)

then the variational

Hamiltonian

for g = k*,

(29)

is given by

H = XTf+L+7jg,

(30)

where 20 = 0

*= The Euler-Lagrange

ifg=k*, ifg < km.

(31)

equations are aH “fT,; Z = aTc

ah-0

(32)

au

and

where

8% [:I aH

aH

z=

-

.

*

aH

(35)

au,,

If an absolute maximum occurs transversality condition is

at just a single time point ti, then the

W,_) = A@,+) +c

(21

(36)

I=I.

1

If absolute maxima occur only on time intervals of nonzero the transversality conditions are x(t,-) =X(f,,)+(~+-?(t,))(~)~=~,

Ir

128

t

length, then

i = 1,3,...,2N-1,

i = 2,4 ,...,

2N.

(33)

Journal of The Franklin Institute

Necessary Conditions for Chebyshev-Bolza

Optimal Control Problems

At terminal time t,

Comparing these results to the bounded state variable necessary conditions in Ref. (17), it can be seen that there is some a priori knowledge of the Lagrange multiplier vl(ti). In particular, vl(ti) and T(t) are related along the optimal trajectory. The assumption on g in this approach is that g is 1, + l-times continuously differentiable with respect to time. Thus the above first-order minimax necessary conditions enlarges the class of performance criteria obtained in Ref. (3).

V. An Example

of Minimum

E$ort

Control

It follows from a theorem in Ref. (16) that the first example has a unique solution. An analogous bounded state variable problem is solved in Ref. (19). Example 1 (40) subject to the constraint (41) with the boundary conditions x,(O) = r,(l) The solution for 0 < c < c1 is given by

= 0, x,(O) = -z,(l)

= 1.

tE[O,*]: W)

24c = (192+c)

-24~ h2(t) = (192+c)

@)

’ t+ 8(48 + c) (192+c)’

24c 8(48 + c) = (192+c)t-(192+c)’ 12c

(42)

8(48 + c)

x2(t) = (192+c)t2-(192+c)t+1’

4c 4(48 + c) X1@) = (192+c)t3-(192+c)t2+t’

Vol.

298, No.

2, August

1973

129

Fredric W. Hemming’and V. David Vandelinde tE[Q, 11: - 24~ h(t) = (192+c)’

24c p+g$, Mt) = (192+c)

(43)

Now consider c > cl, in which case t, # t,. The boundary conditions and the dynamics give t, = (12/c),, t, = 1 - (12/c)*, (44)

c1 = 96, T(h) =

?lV,)

=

-c/2.

The complete solution for c > 96 is t E [O, (12/c)&) : Al(t) = 2(c/12)#, x,(t) = - 2(c/12)8 t + 2(c/l2)B, u(t) = 2(c/12)‘t

- 2(c/12)*,

x2(t) = (c/12)JtZz#)

2(c/12)*t+

(45) 1,

= (c/12)Q3/3-(c/12)*tz+t,

1

t E [(12/c)*, 1 - (12/c)*] : h,(t) = A2(t) = q(t) = 0,

u(t) =o,

xz(t) = 0,

)

xl(t) = (12/~)*/3,

(46)

:1-(12/c)*,l]: x,(t) = - 2(c/12)5, X,(t) = - 2(c/12)H(1 -t) + 2(c/l2)*, (47)

u(t) = 2(c/l2)~ (1 -t) - 2(c/l2)6, x2(t) = -(c/12)q1-t)2+2(c/12)q1-t)-1, xl(t) = (c/12)1 (1 -t)3/3-(C/12)ql

130

-tp+

(l--t).

Journalof

The Franklii Institute

Necessary Conditions for Chebyshev-Bolxa

Optimal Control Problems

A plot of the graph of x1 for various values of c is given in Fig. 1.

$I_ 6

x, 0)

t FIG. 1. VI. A Class

of Minimax

Control

Consider the following

Problems

with Free

Endpoints

subclass of systems contained

in Problem

I,

min max XT(t) x(t) u to
(48)

k(t) = A(t) x(t) + B(t) u(t),

(49)

subject to the constraint where A(t) and B(t) are matrices of conformable dimensions and x(t,) = x,,. The terminal time endpoint is free. Note that the minimax control for this problem is not, in general, unique. For example, any control would be minimax which makes XT(t) z(t) less than $z,,. Thus we look for conditions in Eq. (49) which guarantee Ic* = XT+ Assume that the necessary condition given by Eq. (20) holds for p = 1, thus XT(t) B(t) u(t) = -XT(t) A(t) x(t). (50) If u(t) E En, this expression bY

defines a linear transformation h(t)

l? : En + E given

= G(t) B(t) u(t).

(51)

Theorem II (i) Assume B(t) is an n x n matrix which is nonsingular

for each

tE[t,,tfl, Vol.

296

No. 2, August

1973

131

Fred&

W. Hemming and V. David Vandelinde

then a control which solves the above minimax zc.W), t) = &$% where the scalar-valued

function

problem

is given by

t) BT@) r(t),

(52)

01is defined by

XT(t)A @)XV)

a(x@L t, =-XT(t) B(t)

w(t)

x(t)

(53)

’

(ii) Assume a solution to Eq. (49) exists with Eq. (52) as the input u. Proof: Since B(t) is nonsingular, xT(t) B(t) # 0 for x(t) # 0 and XT(~)x(t) is of order p = 1. By the Schwartz inequality I is continuous. Therefore a u(t) which satisfies Eq. (50), actually the u(t) E En which has minimum norm, is obtained from Ref. (22), p. 192 as u(t) = - ryrP]--IXT(t)

A(t) x(t).

(54)

Substituting the definition of I’ into Eq. (54) gives Eq. (52). The control given by Eq. (52) “constrains” the state variable in such a way that if a solution to Eq. (49) exists, then it must lie on the absolute maximum region with k* = x:x0. The control problem is thus reduced to finding a solution to the dynamical equation k(t) = A@) z(t) + &3(t) P(t)

x(t),

(55)

where 01is defined by (53). Example 2 min max [x:(t) + xi(t)] u OGwf

(56)

subject to the constraint

with the initial conditions x,(O) = 0 and x,(O) = 1. The aii may be functions of time. Using the control given in Theorem II, consider (55) with n = 2 and with B(t) equal to the identity matrix. Assume a solution of the form (58) This gives &J(t)cos w(t) all+a d(t) sin w(t) 1 = ( i “21

(5g)

where 01= -a,,

sin2 w(t) + (aI2 + azl) sinw(t) cos w(t) -I-aa2cos2 w(t).

(60)

Using sin2w(t) + cos2u(t) = 1, the two differential equations in (59) reduce to a single differential equation for the angular frequency w(t), G(t) = (ai, - a& cos w(t) sin w(t) - (aI2 + a& sin2 w(t) + a12.

132

Journal of The

(61)

Franklin Institute

Necessary Conditions for Chebyshev-Bolza

Optimal Control Problems

For B(t) an arbitrary n x m matrix, the control given by (52) holds provided xT(t)z(t) is of first order and (50) remains well defined, i.e. XT(t) B(t) = 0 only if XT(t) A(t)x(t) = 0. This condition holds for the following example which gives a minimax control policy for the linear mass-spring system. Example 3 min max [x:(t) + xi(t)] u lOGl
(62)

subject to the constraint (63) where w > 0 and the initial conditions are x,(O) = 10, x,(O) = 0. The free response of this system is given by (64) A plot in the phase plane for w = i, w = 1 and w = z is given in Fig. 2.

FIG. 2.

A minimax

control for this problem is given by (52) which reduces to ?&c(t), t) = (d-

1) q(t).

(65)

The solution to (63), using the control (65), corresponds to the w = 1 curve in Fig. 2. Note that minimax control changes both the frequency and amplitude of the x2 component of the state vector. The results of this section are easily extended to hold for the plant W) = f(W)

+ WW)

(66)

u(t)

and any g function of first order such that the set is null.

k(t) I gW))

Vol. 296, No. 2, August 1973

= s&J)

n W)

I (WW’B(~(~))

= 01

(67)

133

Fred& VII.

W. Hemming

and V. David Vandelinde

Conclusions

An approach has been given to the optimal control of dynamical systems with a Chebyshev-Bolza performance index. Pu’ecessary conditions are derived which generalize the well-known Euler-Lagrange equations for the Bolza-type control problem. This approach also gives constructive, first-order necessary conditions for a large class of minimax control problems. Acknowledgement The authors

would

like to thank

Professor

R. Isaacs for his interest

and help.

References (1) P. L. Tchebycheff, “CEuvres de P. L. Tchebycheff, A. Markoff et N. Sonin”, St * Petersburg, 1899. (2) C. D. Johnson, “Optimal control with Chebyshev minimax performance index”, Trans. ASME, Series D, Vol. 89, pp. 251-262, 1967. (3) P. E. Barry, “Optimal control with minimax cost”, IEEE Trans., Vol. AC-16, pp. 354-357, 1971. (4) R. E. Bellman, I. Glicksberg and 0. Gross, “Some non-classical problems in the calculus of variations”, Proc. Am. Math. Sot., Vol. 7, No. 1, pp. 87-94, 1956. (5) R. E. Bellman, “Notes on control processes-I. On the minimum of maximum deviation”, Quart. appl. Math., Vol. 14, pp. 419-423, 1957. (6) E. Sevin, “Min-max solutions for the linear mass-spring system”, Trans. ASME, Series E, Vol. 79, pp. 131-136, 1957. problems for linear (7) A. Ya. Dubovitskii and A. A. Milyutin, “Certain optimality systems”, Automn. Remote Control, Vol. 24, No. 12, pp. 1471-1481, 1963. (8) J. Warga, “On a class of minimax problems in the calculus of variations”, Mich. Math. J., Vol. 12, No. 3, pp. 289-311, 1965. (9) J. Warga, “Minimizing variational curves restricted to a preassigned set”, Tralzs. Am. Math. Sot., Vol. 112, pp. 432-455, 1964. (10) J. Warga, “Minimax problems and unilateral curves in the calculus of varia.tions”, SIAM J. on Control, Ser. A, Vol. 3, No. 1, pp. 91-105, 1965. (11) E. B. Lee and L. Markus, “Foundations of Optimal Control Theory”, Wiley, New York, 1967. (12) D. F. Cudia, “Proceedings of Symposia in Pure Mathematics”, Am. Math. SOL, Vol. VIII, pp. 73-97, 1963. (13) L. W. Neustadt, “Minimum effort control”, SIAM J. on Control, Vol. 1, pp. 16-31, 1962. (14) W. A. Porter and J. P. Williams, “A note on the minimum effort control problem”, J. Math. Analyt. AppE., Vol. 13, pp. 251-264, 1966. (15) W. A. Porter and J. P. Williams, “Extensions of the minimum effort control problem”, J. Math. Analyt. Appl., Vol. 13, pp. 536-549, 1966. “An optimal control problem in Banach (16) F. W. Hemming and V. D. VandeLinde, space”, J. Math. Analyt. Appl., Vol. 39, pp. 647-654, 1972. (17) D. H. Jacobson, M. M. Lele and J. L. Speyer, “New necessary conditions of optimality for control problems with state variable inequality constraints”, J. Math. Analyt. Appl., Vol. 35, pp. 255-284, 1971. (18) A. E. Bryson, Jr. and Y. C. Ho, “Applied Optimal Control”, Blaisdell, Waltham, Mass., 1969. (19) F. W. Hemming, “Absolute maximum control”, Ph.D. diss., The Johns Hopkins Univ., Baltimore, Maryland, 1971.

134

Journal

of The Franklin

Institute

Necessary Conditions for Chebyshev-Bolza

Optimal Control Problems

(20) A. E. Bryson, Jr., W. F. Denham and S. E. Dreyfus, “Optimal programming problems with inequality constraints I: Necessary conditions for extremal solutions”, AlAA J., Vol. 1, No. 11, pp. 2544-2550, 1963. (21) J. L. Speyer and A. E. Bryson, Jr., “Optimal programming problems with a bounded state space”, A1AA J., Vol. 6, pp. 1488-1491, 1968. (22) W. A. Porter, “Modern Foundations of Systems Engineering”, Macmillan, New York, 1967.

Vol.296,

No. 2,

August 1973

135