Interior equilibrium solutions of optimal control problems

Interior Equilibrium Solutions of Optimal Control Problems FRED A. MASSEY

R. E. WORTH Department

of Mathematics,

Georgia State

Universiw,

University

Plaza, Atlanta,

Georgia

30303

Received 31 March 1978; revised 10 May 1978

ABSTRACr Many authors consider the following special optimal-control problem [see, e.g., Silvert and Smith, Math. Biosci. 33: 121-134 (1977)]: Find a control function u and a state function x which m aximke J$eep’R(x,u)dt subject to a!x/dt=g(x,u), with x and u constrained to certain sets X and CJ, and such that x and u are equilibrium (constant) solutions which are interior to X and U. We show that (x,,,uO) is an interior equilibrium solution of the problem if and only if (x,,,uJ is an interior global maximum for R(x,u) which satisfies g(x,,, u,,) = 0.

INTRODUCTION In a recent article [l], Silvert and Smith have considered the following problem. Consider a multi-species ecosystem, with the populations of the n species at time t represented by a vector x(t) [all vectors henceforth will be column vectors, unless transposed, with uT being the transpose of a]. At time t the rates of harvest of these species is represented by the n-vector h(t). The system dynamics are given in the differential equation x?=f(x) - h, (1) wheref is twice continuously differentiable. Also x(t) and h(t) are assumed constrained separately to two parallelopipeds in R”, usually xi z 0 and Ui
V(x,h)=J?

-p’R(x(t),h(t))dr,

0

MATHEMATICAL

BIOSCIENCES

OElsevier North-Holland,

Inc., 1978

42, 31-35 (1978)

31 0025-5564/78/090031-5$02.25

32

FRED A. MASSEY

AND

R. E. WORTH

over all choices of x(t) and h(t) satisfying the constraints and Eq. (1) where p is a constant discount rate. The authors then treat this as an optimal-control problem and apply the Pontryagin maximum principle to obtain the usual necessary conditions. They then assume that the optimal policy (x*(t),h*(t)) is constant and interior to the constraint sets; that is, x*(t) is an equilibrium solution of (1) which lies in the interior of the constraint set. From these assumptions they find that a necessary condition for (x*,h*) to be an optimal interior equilibrium solution is that they satisfy

i=l ,..-, n. In this paper we present a slight generalization of the above system. We then show that the problem can be treated as a problem of maximizing a function of several variables using simple calculus techniques. We finally show how to determine the existence of interior equilibrium solutions, actually showing how to derive these solutions. In what follows if T is a real-valued function of the vector x, then dT/tlx denotes the column vector whose entries are CJT/tlxi, i = 1,. . . , n, and if T is a vector-valued function of the vector x, then aT/Clx denotes the matrix whose columns are dTi/Clx, i=l,..., n. This leads to considerably simplified notation. A GENERALIZATION Let the system dynamics be given by

of the ecosystem

described

in the introduction

i=g(x,h), where g is twice continuously differentiable non-singular for each h. Let the Hamiltonian of the system be

(2) and

the matrix

ag/Clh

is

H(x,h,z,t)=e-P’R(x,h)+[g(x,h)]‘z.

Then, by the maximum principle, necessary conditions for (x*(r),h*(t)) to maximize V(x, h) subject to the constraints and satisfying (2) are that there

33

INTERIOR EQUILIBRIUM SOLUTIONS

exists z such that

2

i=

&i=__=e-Pl_-

Pa)

=g(x,h), 3H ax

aR ax

aR -aH =e -%&+ ah

ag ah C-1

Tz, - ag ( ax 1 ==o

Pb) PC)

.

Assume x* and h* constant. Then x* = h’*=O, so (x*, h*) lies on the level surface g(x, h)=O. Since x* and h* are interior points and @g/ah)-’ exists, we can solve locally for h as a function of x, that is, there is a neighborhood of x* and a function G(x) defined there such that g(x, G(x)) = 0. We need an expression obtaining

for aG/ax;

(4)

thus, differentiate

(4) with respect to x,

ag aG -ag ax +ahaX=O’ so that -1 ag -ag -. ( ah 1 ax

aG -=ax Also, solve (3~) for z, and obtain z=

K )1-* ag

__e_P’

ah

T -1aR ah

Now, following Silvert and Smith, differentiate (3~) with respect to 1, assuming that x* and h* are constant, so that aR/ah and (ag/ah)T are also constant. Thus -pep@@R/ah) +(Elg/i3h)Ti =O, and using (3b) and substituting for z,

-ptg_ ag Tz ( 115 ( )L -p,g+ ag T -pe -p,g+ -peah

agT

__e-Pf_+e-Pr

ah

Canceling

(

ah

=o

Zieeax

ax

( 25;x)T[

E

(~)‘]-‘~)

=o.

)(

- eep’ and substituting

(aG/axy

for -(ag/ax)T[(ag/ah)T]-l,

FRED A. MASSEY AND R. E. WORTH

34 we have

This can be written

(ahag1=am,axG(x)) =. *

aR

wl+ This last equation A CALCULUS

is a generalization APPROACH

(5)

of (9a) and (9b) of [l].

TO THE PROBLEM

Since the interior equilibrium (x*,/r*) maximizes V(x,h) over all choices of x and h, it certainly maximizes V(x,h) over all constant choices of x and h. Now note that if x and h are constant, we have V(x,h)=(l/p)R(x,h). We see then that (x*,h*) maximizes (l/p)R(x,h) [and hence &x,/r), for p is positive] subject to the constraint g(x,h)=O. Using G(x) obtained above, we have that (x*,/z*) is a local maximum of R(x,G(x)). Then a+, G(x))/ax =0 at x *. But this with (5) gives

Also, since

aR(x,

aR

G(x))

aG TaR

=ax+ax ( 1ah’

ax we must have, using (6) that

g(x*,h*)=o.

g(x*.G(x*))=

(7)

It is now apparent that a necessary condition for an optimal path to be an interior equilibrium solution is that it is a critical point of R(x,h) which lies on the surface g(x,h)= 0 and yields a global maximum of R over the constraint sets. Suppose conversely that (x,,,h,,) is a global maximum of R(x,h) which lies on g(x,h) =O, whether interior or not. Then we have, for any choice of (x(t),h(t)), R(x(t),h(t))
/0

me-P’R(x(t),b(t))dt

Hence, (xo,ho) is an equilibrium

G

solution

~me-@R(x,,hO)dt. of our original

problem.

INTERIOR EQUILIBRIUM

SOLUTIONS

35

In conclusion, we have that a necessary and sufficient condition for the existence of a constant interior solution to our optimal control problem is that there exists an interior global maximum for R(x,h)which lies on g(x,h)=O. This shows, for example, that such models in which R is non-trivially linear in x or h cannot have interior equilibrium optimal paths. In [ 11, Silvert and Smith study the “quadratic” case, in which g(x, h) = f(x)- h, with f(x) quadratic, R(x,h)= PTh,and there are no harvest constraints. They develop necessary conditions for a solution, derive certain consequences (one “surprising”), and discuss their economic interpretations. Our work shows that, unless P inzero (all prices are zero), we cannot have i3R/ah=O, so that no solution can exist and any analysis of necessary conditions is wasted. Once again we see the danger inherent in studying necessary conditions without knowing existence. Of course, given the scarcity of existence theory for optimal control, we must usually study necessary conditions if we are to do anything at all. REFERENCES 1 William Silvert and William R. Smith, Optimal exploitation of a multi-species community, M&h. Biosci. 33:121-134 (1977).