Optimal control with an unbounded horizon

Journal of Economic Dynamics and Control 9 (1985) 291-316. North-Holland

OF’TIMAL

CONTROL

WITH AN UNBOUNDED

HORIZON

Michael A. TOMAN* Resources

/or the Fwure,

Washington,

DC 20036.

USA

Received April 1984, final version received August 1985 This paper extends existence results for finite and infinite horizon control problems to ‘unbounded horizon’ problems where both finite and infinite terminal times are feasible and there are non-trivial payoff implications (in particular, because of a non-zero terminal valuation) in making this choice. The classical approach used in the paper leads to simple and fairly intuitive conditions for existence involving direct assumptions about the objective function, state dynamics, and control constraints which are accessible to applied users of control techniques. In addition, the results are proved under fairly weak concavity assumptions and thus have potential applicability to ‘increasing returns’ problems.

1. Introduction The extension of existence theorems for continuous time deterministic control problems from a 6nite horizon to an infinite horizon setting has received much attention in both the mathematics and economics literatures. Some of these studies, such as Baum (1976) and Balder (1983), have used classical arguments similar to those employed in the finite horizon case [see, e.g., Cesari (1966,1983)], where the conditions for existence directly involve properties of the basic problem structure - objective function, state dynamics, and control constraints. The work by Magill (1981) also can be placed in this category, though it involves substantially more complex mathematical machinery. A second approach, exemplified by Brock and Haurie (1976) and Haurie (1980), involves application of the dual space/Hamiltonian methods pioneered by Rockafeller (1973,1975).’ In these studies of the infinite horizon existence question the generic problem considered is a ‘Lagrange’ control problem, where the objective is an *This paper is adapted from the author’s PbD dissertation in economics at the University of Rochester. Helpful discussions with Robert Becker, Larry Benveniste, Swapan Dasgupta. James Friedman, Makoto Yano and Ken Yesuda are gratefully acknowledged, as are the useful comments of anonymous referees on earlier drafts. Responsibility for errors and opinions is the author’s alone. ‘See also the bibliography in Magi11 (1981) for an excellent summary of the existence literature. More recently, work by Balder (1983) and Carlson (1984% b) has yielded results which provide a basis for integrating the two approaches by highlighting the connections between tbem. 0165-1889/85/S3.3001985,

Elsevier Science Publishers B.V. (North-Holland)

292

M.A.

Totnan.

Oprimal

cotrlrol

with an urlhouttded

horicott

integral cumulation of an instantaneous net benefit flow. Thus, these existence theorems are not directly applicable to what might be called ‘unbounded horizon’ control problems, where finite as well as infinite terminal times are feasible and the choice of finite versus infinite terminal time has non-trivial implications for the value of the program. Problems of this type arise in particular if there is a non-zero state-dependent ‘terminal valuation’ associated with a finite stopping time.2 One example of such a problem is the choice of a neoclassical firm between carrying out production and investment activities over an infinite horizon or liquidating the capital stock in finite time. Another example is the optimization problem of a non-renewable resource producer whose operating horizon may be finite or infinite, and where the choice of horizon can depend on the ‘site value’ of the resource-bearing land.3 This paper presents existence theorems for a class of unbounded horizon problems such as those described above. Both classical finite horizon results and infinite horizon extensions of these results emerge as special cases of the theorems proved in this paper. In particular, the theorems cover cases where the objective function depends on the initial state and time because of, for example, ‘start-up costs’, thus extending infinite horizon results which do not incorporate this feature. The methods of proof in the paper are very similar to Baum’s (1976) approach, which is itself an extension of the classical method employed by Cesari (1966)? In particular, he imposes an integrable bound condition on the instantaneous net benefit flow [hypothesis (i), Theorem 5.1, p. 971 which is identical to Assumption A.10 in this paper. Moreover, the ‘diagonalization’ process used in proving Proposition 2 below is almost the same as the technique used by Baum in proving his main result (Theorem 6.1, pp. 102-103): However, Baum considers only infinite horizon (versus unbounded horizon) problems with the strong optimality criterion [Haurie (1980, pp. 518-519)], where improper integrals of the instantaneous net benefit flow are assumed to converge. In this paper a slightly weaker criterion is assumed, as discussed further in the next section. ‘In the finite horizon control literature these problems are known as ‘Bolza’ problems. See Fleming and Rishel (1975, pp. 23-26) for further discussion of this problem classification. ‘The former example is an extension of Treadway’s (1970) infinite horizon firm model; the latter example is an extension of standard resource supply models suchas Peterson (1978) and Pindyck (1978) and is treated in Toman (1985a, b). 41 am grateful to a referee for drawing my attention to the connections between my results and Baum’s. s Baum’s results also go further than those presented here by including the possibility of additional constraints on the limit of the state (assuming the limit exists), analogous to finite horizon terminal transversality conditions. To incorporate these constraints, Baum must impose a certain ‘Property (P)’ on the class of admissible programs (p. 96). For the ‘free endpoint’ problems considered in this paper, it can be shown that this condition is satisfied, given the other assumptions imposed on the problem.

M.A.

Toman,

Optimal

conrrol

with an unbounded

horizon

293

There are two important advantages to the classical approach used here. First, as noted above, the conditions for existence involve fairly simple (ad usually intuitive) properties of the basic problem structure, rather than properties of a Hamiltonian dynamic system. Some of the assumptions underlying the results in this paper can be weakened, as discussed subsequently. However, one aim of the paper is to present conditions (and arguments) perta,hhg to existence in as intuitive a manner as possible to make them accessible to a broader group of non-specialists, so that existence questions can be addressed more frequently in applied work. A second advantage to the classical approach is that relatively weak concavity assumptions are required. In particular, the instantaneous net benefit flow is required to be a concave function of the controls but need not be jointly concave in states and controls. Thus, the theorems proved here are potentially applicable to ‘increasing returns’ problems. The balance of the paper consists of six parts. In section 2, the generic unbounded horizon control problem studied in the paper is set up and other preliminary concepts and notation are introduced. Section 3 states and proves two general existence theorems for this problem under somewhat abstract assumptions. Section 4 shows how these abstract assumptions can be replaced by stronger but more intuitive hypotheses. In section 5, existence theorems for the corresponding infinite horizon control problem are established as special cases of the results in sections 3 and 4. Section 6 briefly notes how the basic assumptions introduced in section 3 can be weakened to extend the theorems. The seventh and final section of the paper briefly summarizes its conclusions and contains a discussion of their applicability. 2. The unbounded

horizon

control

problem

The control problems studied in this paper are of the general Bolza type, in which the initial and terminal times and the initial states are choice variables along with the paths of controls, and the objective function typically depends on all of these decisions. Lagrange problems, in which the objective function consists only of a benefit flow integral, and problems in which the initial time and states are fixed represent special cases. To fix notation, let x be an element of R”, t a non-negative scalar, u an element of R”‘, A a subset of R”+l, B a subset of A, and U(t, x) a variable subset of R”. A program is a pair of functions (x, U) defined on a common connected subset of R!+ =’ [0, 00); x(t) is the state trajectory and u(t) is the control. The initial time, the left endpoint of the domain of (x, u), is denoted by s, and y = x(s) is the initial state; the pair (s, y) will be referred to as the initial data. The set A is the state space and represents constraints on state variables such as non-negativities or a priori upper bounds; the subset B represents any additional constraints on the initial data.

The evolution

of the state trajectory

over time is governed by a state

dynamics function denoted f(t, .x, u), so that k =f(r, X, 14). The set u(t, s) is the conrrol set correspondence and represents a priori constraints on controls

such as capacity constraints or non-negativities. These constraints may vary with both the state x and the time index r. The objective function, or perjornrance index, is denoted by V(s. to and consists of three parts. The first part is the cumulative value of an insranruneous net benejr function &(r, X, u), which can be interpreted as utility or profiL6 The other two parts are R(s. y). the initial oahtarion function. and W( T, x(T)). the terminal valuation function. For example, R CM be interpreted as start-up cost and W as resale or scrap value. The overall performmce index V( x, u) then is given by V(s, u) = R(s, y) + W(s, y), = Rb. Y) +lTf,(

s=T
r,x,u)dr+

= R(s, y) + Iilii r>Jr,x. P-m 8

W(T.x(T)),

s
u)dr,

(1)

s
where the time indexes of it(r) and u(r) in fO(r, X, u) are suppressed for notational convenience and where T 2 00 represents the terminal rime. The ‘limsup’ objective for infinite horizon programs is used to avoid problems associated with the possible non-integrability of r -) fb over M infinite horizon. Under Assumptions A.10 and A.11 listed in the next section, it is fairly straightforward to show that the function V in (1) is well defined and uniformly bounded on the set of feasible programs defined below.’ Given these conditions, maximization of V over the feasible set is similar to but stronger than the weak overtaking optimality criterion used in Brock and Haurie (1976, p. 338) and Haurie (1980, p. 519). The weak overtaking optimality criterion for infinite horizon programs states that (x*, IC*) is optimal if

for all other feasible programs (x, u). Under A.10 below, it is easy to show that “In pmblenrs involving discounting of future returns, the discount factor have been absorbed into lo. In other words. with a constant discount e”/u( r. s, 14) represents undiscounted returns. ‘Proof Iquest.

of these

and other

teehnicol

points

not elabomted

upon

can be understood to mtc r the function

in the paper

am wailable

upon

ALA.

Tonm.

Optimal

control

with ao tmhomtded

hori:ott

295

this criterion is implied by V(x*, u*) 2 V(x, 10, where V is given by the third line of (1) above. As noted previously, Baum (1976) uses the strong optimality criterion lVmfOdt for infinite horizon programs, where the feasible set is restricted so that this improper integral is well defined. In Balder (1983) the criterion takes the same form, but the problem of the integral being well defined is avoided in the following way. Let $ = max(f,,O) and & = max( -fU, 0). so that lo =/;: -6. Then one can write

with both integrals on the right-hand side of the equality being well defined since G 2 0 and & 2 0. With the additional convention that 00 - cc = - co, /:‘a d t also is well defined. The results in this paper also can be established using this approach in lieu of the ‘lim sup’ formulation in (1) above. The ‘lim sup’ formulation was chosen here to avoid the convention that 00 - 00 = - co. Note that despite its resemblance to the weak overtaking optimality criterion, the objective (1) - like the objectives in Baum (1976), Magi11 (1981) and Balder (1983) - is a complete ordering of programs, whereas weak overtaking optimality provides only a partial 0rdering.s As also noted above, one of the hypotheses imposed here in proving the existence theorems (Assumption A.10 below) is that

along any admissible program, where g(t) 2 0 is a finitely integrable function. Construction of such an integrable dominating function from more basic hypotheses is considered in section 4 of the paper. If the instantaneous return also satisfies f0 2 g(r) for all admissible programs, where g(t) is a finitely integrable function, then t -+!a necessarily is finitely integrable and the distinction among the criterion in (1) Baum’s objective, and Balder’s objective (as well as the weak overtaking optimality criterion) disappear. Such a lower bound often arises in applications. For example, if f0 is a utility function which is necessarily non-negative, then B(r) = 0. The specification d = 0 also can be used in cases where f0 is a profit function with zero fixed costs.’ ‘Sm Mogill (1981, p. 682) for further discussion of this point. ‘See Toman (19R5a, b) for illustrations of this point.

296

The unbounded horizon control problem is to choose a program which maximizes the performance index (1) subject to the following conditions: u(t) is measurable.

(0 (ii)

x(t) is absolutely continuous (AC) on all bounded subsets of its domain.

(iii)

u(l) is an element of U(t, x(t)) almost everywhere (a.e.).

(iv)

(I. x(r)) is contained in A for all t.

(VI (vi)

i(r) =f(r,

(vii)

(s, r) is an element of I!!.”

x(t), u(r)) a.e.

t +j,(t, x(t), u(r)) is (finitely) integrable on all bounded subsets of the domain of (x, u).

Conditions (i)-(vii) define the class of feasible or admissible programs, denoted by S. Henceforth, the unbounded horizon problem will be abbreviated by the pair (V, S). Note that admissible programs may have an infinite horizon or a finite horizon. If s = T, then the program (x, u) is interpreted to be the initial data pair (s, JJ) and is called degenerate. Such programs are admissible, provided condition (vii) holds; conditions (i)-(vi) are understood to hold vacuously in this case. By incorporating degenerate programs as well as non-degenerate paths, the analysis integrates static and dynamic decision making. To state and prove the existence theorems, some additional notation is useful. First, for a given program (x, u), define the auxiliary state x0( 1) and the auxiliary control uo(t) by

xob’) = /‘kc 5

x, u) dt,

(2)

u&) =fo(t, x, 4. Then xo(t’) = /,“u,(!)dt, qx,

(3) and the performance index (1) can be rewritten as

u) = R(s, y) + W(& y), = R(s, y) +x,(T) =R(s,y)+

+ W(T, x(T)),

El x,(v), I” m

s=T
(1’)

s
“Throughout the paper, ‘measurable’ and ‘ae.’ are with respect to complete regular Bore1 measure on the Bore1 subsets of [0, co) and integrals are Lcbesgue integrals with respect IO this measure: ‘integrable’ is taken to mean that the integral is well defined and finite. Vector valued functions are called integrable, measurable or absolutely continuous il and only if their components have these properties. ‘Absolute continuity’ implies that the derivative (componentwise. for vectors) exists a.e. and is integrable. See Hewitt and Stromberg (1965) for further discussion of these concepts.

M.A.

Totnm.

Oprimul

control

wirlt UII unhouded

horizotl

291

Next, for t 2 0, let A(t) denote the set

A(t)=

(XER”: (t&=4),

(4)

and let M denote the set M=

{(t,x,u):

UE u(t,x),(t,x)EA}.

Thus, A(t) is a cross-section 1. M represents the set of compact subset A, of A, let to (4). In addition, for (I, x)

Q(t.x)=

(5)

of A - the set of all feasible state vectors at time all possible times, states and controls. For any M,, be the corresponding subset of M analogous E A let Q(f, x) denote the correspondence

{(z,,,z)Q?"+~: ~~~Sf&,x,u), z=f(f,x,u),

UE U(t,x)}.

(6)

Thus, each element in Q( 1, x) consists of a state velocity vector f( t, x, U) and a number less than or equal to the value of the instantaneous net benefit function fa(t. x, u) for time t, feasible state vector x, and feasible control u. The economic significance of this correspondence is not immediately clear. However, it plays a vital technical role in the existence arguments, as indicated below. Finally, let p be the maximal value of the performance index: (7)

As noted above, under the assumptions given in the next section it can be shown that 8~ co. An optimalprogram (x*, a*) is an element of S such that v(s*, u*) = v. Proof that an optimal program exists is accomplished by ‘constructing’ an optimum from a maximizing sequence, a colIection of programs {(x,, u,), i = 1,2,. . . ) in S such that V(x,, ui) + v as i + co. It is easy to show that at least one maximizing sequence always exists for the unbounded horizon control problem unless the admissible set S is finite, in which case the existence of an optimal program is trivial. The assumptions listed in the next section are sufficient for an optimal program to exist. 3. Existence

of an optimal

program

In this section, existence theorems are proved for the problem (V, S) both with and without a priori bounds (such as ‘capacity constraints’) on the set of admissible controls, in other words, for bounded and unbounded control sets U(t, x). Assumptions A.l-A.9 and A.12 below are drawn directly from the

298

M.A.

Toman,

Optimal

control

with an unbounded

horizon

classical existence theory for finite horizon programs, while A.lO-A.11 are added to extend this theory to the infinite horizon case. Alternative assumptions with greater intuitive content are developed in the next section. A.l.

A is a closed subset of R$X R” with non-void interior, and A(t) Z 0 for all t L 0. B is a non-void compact subset of A. There exists at least one (s’, JJ’) E B such that y’ E int[A(s’)].

A.2.

U(t, x) is defined and uppersemicontinuous closed image sets in R”.ll

A.3.

f(t, x, u) and fc(t, x, u) are defined and continuous on the set A4 in (5) and take values in R” and R’, respectively; R(s, y) is a real valued continuous function defined on B.

A.4

W(t, x) is a real valued continuous function defined on A.

AS.

The correspondence Q(t, x) in (6) has convex image sets.

A.6

The correspondence Q(r, x) defined in (6) is USC.

A.I.

For every compact subset A, of A there exists a continuous function h: R: --) R’, satisfying h(a)/a + - cc as Q + cc, such that fo(t, x, u) < h(luj) for all (t, x, u) in M,, where Ma is the subset of M corresponding to A,.

A.8.

For every compact subset A, of A there exist non-negative constants C and D such that If(t,

A.9.

x, u)ls

C+ Dlul,

(USC) on A, with non-void

for all (t, x, u) in MO.

(8)

Either (a) for each T > 0 the sets A( t ), 0 I t I T, are uniformly bounded, or (b) there exists K, r 0 such that the inner product xf( t, x, u) satisfies xf(t,x,u)
forall(t,x,u)in

M.

(9)

A-10. There exists a (finitely) integrable function g: R: + R: such that for all (x, u) in S and almost all t in the domain of (x, u), fo(6 407

U(I)) s g(t).

“See Cesari (1966, pp. 372) and Hildenbrand and properties of USC correspondences.

00) and Kirman (1976, pp. 187-195) for definitions

M.A.

A.ll.

Toman,

Optimal

control

with at, unbounded

horizon

299

For any a > 0 there exists N = N(a) > 0 such that, if (x, u) is any finite horizon admissible program with terminal time T 2 N, then IWT, x(T))1 < 0.

A.12. For each (t, x) E A, the control set U(t, x) is bounded. In Assumption A.l, compactness of B implies a priori bounds on the choices of initial time and initial state variables. This assumption is taken up again in the concluding section of the paper. Note that, since B # 0 and each (s, v) E B is a (degenerate) admissible program, the set of admissible programs S also is non-void. Moreover, the last condition in A.1 and continuity off imply that S contains at least one non-degenerate program.12 A(t) # 0 ensures that nondegenerate programs with connected domains can be defined. Without further assumptions, S may contain no infinite horizon programs, except perhaps for ‘trivial’ extensions of finite horizon programs.13 The focus of this paper is on problems wherethis is not the case, so that the choice of an infinite horizon program is a meaningful option. Sufficient conditions for the existence of (non-trivial) inhnite horizon programs can be gleaned from the differential equations literature and will not be explored in detail here. However, to indicate briefly what is involved, suppose that f(t, x, 0) = 0 for all (t, x) in A and that, for all t L 0, U(t, x) is a convex set containing the origin which contracts to (0) as x approaches the boundary of A(t). Then using arguments for existence ‘in the large’ like those in Hale (1969, ch. l), it can be shown that an admissible control u(l) exists which is not identically zero and which satisfies lu(t)l +O as t + co, such that the associated state trajectory x(l) is defined on [0, co) and is admissible. This pair (x, u) would constitute a non-trivial infinite horizon program. Returning to the discussion of assumptions, uppersemicontinuity of U( t, x) is a stability condition on how the set of feasible decisions varies with the passage of time or changes in the states. This condition will be satisfied in particular if lJ(t, x) satisfies any of the following specifications: U(t, x) = {UE R”‘: lul~H,(t,x)}, U(r,x)=

{uERm:

U(r,x)={uER”‘:

dsusH(r,x)}, usH(r,x)},

“See Hale (1969, ch. 1) for existence theorems in differential verify this claim. 13A trivial extension would arise, for example, if /(f, x.0) = 0 some T c CQ such that u(r) = 0 and x(r) =x(T) for all I 5. T. have this property for some ftxed T, then the unbounded horizon to a finite horizon problem.

equationsthat can be used to for all (f, x) E A and if there is If all infinite horizon programs control problem can be reduced

300

M.A.

Toman,

Optimal

control

with an unbounded

horizorl

where u’ is a fixed vector in R” and the ‘capacity constraint’ functions H: A + R” or J-Z,: A + R: are continuous. Note also that the control sets in (lla) and (llb) are bounded under these assumptions and thus satisfy A.12. Assumptions A.3 and A.4 require little comment. Assumptions A.5 and A.6 are technical conditions which are difficult to verify in general and which have no apparent economic content. As indicated below, A.6 can be dropped if A.12 holds, so that U(t, x) is bounded. Assumption A.5 is more difficult to dispose of.14 However, it is shown in the next section that A.5 can be dropped in an important set of cases. It is also shown there that A.7 can be interpreted as a concavity and curvature condition on the instantaneous net benefit function fa. Assumption A.8 asserts that locally the velocity of the state vector is dominated by a linear function of the control vector, while both A.9(a) and A.9(b) posit local uniform bounds on the state trajectory, as indicated below. Both of these assumptions also can be replaced by alternative hypotheses, as indicated in the next section. Finally, A.10 posits an integrable upper bound on the instantaneous net benefit function along any admissible program, while A.11 is a ‘uniform asymptotic negligibility’ condition on the terminal valuation. The role played by these assumptions in extending the classical existence theory for finite horizon problems to the infinite horizon case is brought out below. Note that both of these conditions are stated as properties of arbitrary admissible programs and thus are not immediately verifiable. In the next section, both of these conditions are derived from more basic assumptions on the underlying problem structure. Note also that for A.10 to hold, some dampening of the instantaneous benefit flow like a positive intertemporal discount rate is required. Discounting is explicitly introduced in the next section of the paper. The first existence theorem concerns problems (V, S) in the more general case of no a priori bounds on the controls. Theorem 1. Assume A.l-A.11. exists.

Then an optimal program (x*, u*) for (V, S)

The proof of Theorem 1 is broken down into a series of pieces corresponding to different types of maximizing sequences {(xi, ui)). First, however, a preliminary technical result is required. Lemma 1. Assume A.1 and A.9. Then for every T > 0 there exists a number K = K(T) > 0 such that, for any admissible program (x, u), t I T implies Ix(t)1 s K. 14Cesari (1966, pp. 378-381) presents necessary and sufficient conditions under which the image sets of Q(r, x) are convex, but these conditions are so stringent that they have little practical value.

M.A.

Toman,

Optimal

control

wilh an unbounded

horizon

301

Proof. The result is immediate if A.9(a) is true. Cesari (1966, p. 395) provides the proof under A.g(b). 0 Proposition 1. Suppose th_at ((xi, ui)} is a maximizing horizon programs with q. s T < COfor all i. Assume A.I-A.9. (V, S) exists.

sequence of finite Then a solution to

Proposition 1 follows directly from classical existence theorems for finite horizon problems proved in Cesari (1966, pp. 390-395; 1983, pp. 386-387) and Fleming and Rishel(l975, pp. 68-74). Thus, complete proof of this result will not be given here. However, certain key elements of the proof will be highlighted because they play an important role in subsequent arguments. (a) Note first that a subsequence of a maximizing sequence also is a maximizing sequence. Let (si, n) and (Ti, xi(Ti)) be the initial data and terminal point of (xi, ui). By A.l, the sequence {(si, y,)} is bounded, and by Lemma 1, the sequence {(q, xi(q))} also is bounded. Therefore, no generality is lost by assuming that (si, yi) + (s*, y*) for some (s*, y*) E B for some (T*,x*(T*)) E A. Let and that (& xi(q)) + (T*, x*(T*)) Ri = R(si, vi), R* = R(s*, v*), Wi = W(T, x,(q)) and W* = W(T*, x*(T*)). Then Ri + R* and Wi+ W* by A.3-A.4. (b) Let xoi(t) be the auxiliary state for (xi, ui) as in (2). Then there exist an R”-valued function x*(t) and functions x,(t) E R’ and x,(t) E R:, such that along a subsequence (xi, ui), xi +x* uniformly on their common domair$ x* is AC, xai --) x, - x, pointwise, x, is AC, x,(s*) = 0, and x,(t) 2 0 for all 1. (c) Moreover, there exists a measurable function u*(t) such that (x*, u*) is an admissible program and f, 5 u$ a.e., where u:(t) is the auxiliary control for (x*, u*) as in (3).16 Note that since i, I, u$ and x,(s*) = 0, it follows from (2) that x,(T*) I x,*(T*). (d) It follows from the definition of the objective function (1’) and from (7) that x,,~(T~) + p-R* “In a rigorous proof, of s’ = inf si < I < T’ degenerate - is allowed 161t is at this point in

- W*.

01)

the functions x, and x0, are extended continuously to a common domain sup q;; the possibility that s* = T* - so that the optimal program is for; and a somewhat more roundabout method of proof is used. the proof that Assumptions AS-A.6 on Q(r, x) play a crucial role.

302

M.A.

Toman,

Optimal

control

with an unbounded

horizon

But from parts (a)-(c) above, it follows that xOi(t)

+

xf2(t)

-xs(f)

5 x,(t),

for each t, and that T + T *. Combining shown (in a rigorous treatment) that h

i+m

X,i(T)

I; x,(T*)

S xo(T*).

(12)

these observations, it can be

(13)

Finally, by combining (11) and (13) with (1’) it follows that-V(x*, u*) 2 v. But v is the maximal value of V over S, so V(x*, u*) I V as well. Thus, (x*, u*) is an optimal program. 0 The next case involves a maximkin g sequence of finite horizon programs whose terminal times increase without bound. Proposition 2. Suppose that {(xi, ui)} is a maximizing sequence with terminal times satisfying Ti + CO. Assume A. 1-A. Il. Then (V, S) has a solution. Proof. The demonstration of this proposition uses a ‘diagonahzation’ argument. Note first that, as in paragraph (a) following Proposition 1, convergence of the initial data (si, yi) to some point (s*, r*) can be assumed, so that Ri + R*. In addition, A.11 implies that Wi = W(q, x,(q)) + 0. Finally, since q + cc, it can be assumed without loss of generality that the T converge monotonically to + cc with q > si for all i. To begin the diagonalization let (xi, ui, TJ denote the (admissible) program obtained by truncating (xi, ui) to t s T,, and consider the sequence {(xi, ui, T1)} formed by this truncation. Then arguing as in paragraphs (b) and (c) following Proposition 1, there exists an index set Ii and functions xr, u:, x,i and x,i such that xi + xi’ uniformly for t I Tl as i + oo through II, and x: is AC; the auxiliary state xgi + x,i - x,i pointwise for f I Tl as i + co through Ii, with x,i being AC, x,i r0 and x,i(s*) =O; (x,*, uf) is an admissible program with initial data (s*, r*) and terminal time T,; and R,, I u;fi a.e., where u& is the auxiliary control for (XT, u:), so that x,i(t) I x&(t) for all t s T,, where x& is the auxiliary state for (x,*, u:). Next, repeat the above argument using the sequence of truncated programs {
M.A.

Toman,

Optimal

control

with an unbounded

horizon

303

u; still generates the trajectory x; since x; is an AC extension of x:. Thus, it may be assumed without loss of generality that U$ is a measurable extension of u:. With this modification of at, u&(t) = u&(t) and x&(t) = x&(t) for all ts T,. Continue in this fashion to obtain for each integer j an index set Ii contained in Ii-t and functions x7, ~7, xoj and xsj which inherit the properties of their corresponding predecessors. Now set To = s* and define the program (x*, u*) by (x*(r),

u*(r))

=(x;(r),

TimI I r < 7J.

u-fO)j,

(14)

Note that, since (x7, x7) is an extension of (XT-,, uy-,) for each j, the auxiliary state and control for (x*, u*) satisfy x,*(r) = xgj(r) and u,*(r) = utj(r) for q-r < r -c q. The program (x*, u*) is readily shown to be admissible. The claim is that (x*, u*) also is optimal. To see this, fix an integer i and let i E Ii, so that i >i. Then (2) and condition (10) in A.10 imply that +

Ri+xoi(I&)

+ Wi+fTg(r)dr. I

WisRi+xoi(q)

(15)

Letting i --) co through Ii in (15) and recalling that Ri -+ R* and y + 0, it follows from (12) and the definition (1’) of the performance index that ~~limx,,(7j)+R*+~~wg(r)dr i

,

+ R* -t/Tmg(r)dr J

=xai(7j)

-xsj(q)

sxgj(l;)

+ R* +/“g(r)dr T,

=~$(7J)+R*+/~g(r)dr. 7

(16)

Finally, taking the limit superior as i --, co in (16) and invoking (1’) and A.lO, Since (x*, u*)ES, it follows that j;g(r) dr +O and vl V(x*,u*). V(x*, u*) 5 v as well. Therefore, (x*, u*) is optimal for (V, S). 0

304

M. A. Tomon,

Oprind

conrrol

wifh UII wlhounded

horizotl

The last case to be considered involves a maximizing sequence of infinite horizon programs. Proposition 3. Suppose that ((x,. u,)) is a maximizing sequence of injinite horizon programs. Assume A.1 -A. 1I. Then a solution to (V, S) exists.

Let (q} be any sequence of numbers converging to + cc with T, > s, for all i, and let (xi, u,, ?J) be the admissible program obtained from (x,, u,) by truncating it to [si, T,]. If the sequence ((x,, u,, T,)] can be shown to be a maximizing sequence, then Proposition 2 implies the existence of a solution and the proof is complete. To this end, note that eqs. (1’) and (10) imply that Proof.

V(X,vUit’I;)rV(x,vU,)-/Twg(~)d~+ W(T,,x,(T,))* ,

As i+ oo in (17). /Fg(g(,)dl -rO by A.10 and W(T,,x,(T,))+O Therefore, fiV(X,,Ui,qmiiv(x,,u,)=i? ,

(17) by A.ll.

i

But since (Xi, u,, T,) ES for all i, V(X,,Ui,T,)SV,

so

V(X,,Ui,7;)-,V.

Thus, ((xi. u,, q)) is a maximizing sequence. 0 Propositions l-3 can be combined to complete the proof of Theorem 1. To see this, consider any maximizing sequence {(x,, u,)) for (V, S). If all but finitely many of the (xi, u,) are degenerate programs (s,, y,), then any cluster point (se, y*) of the (s,, y,) - of which there is at least one, in view of A.1 - satisfies V(P,

y*) = R(s*, y*) + H+*,

Y’)

= lim V( si, yi) = F, I

and thus is a (degenerate) optimal program. All other possible maximizing sequences can be covered by Propositions l-3. Cl

M.A.

Tonrun.

Opfinwl

conrrol

witA un unlwunded

305

ltorirou

To this point, A.12 - boundedness of the control sets - has not been imposed. Note that under A.2 and A.12, U(f, x) is compact for each (1, x). This observation can be used to drop other hypotheses from Theorem 1. Lemma USC.

2.

Assume

A.2

and A.12.

Then the correspondence

Q(r,

x) in (6)

is

This lemma is proved in the appendix. Theorem A.6-A.8.

2.

The

conclusion

of

Theorem

I remains

valid

if A. 12 replaces

Proo/. It follows from Lemma 2 that A.12 can replace A.6. To see that A.2 and A.12 imply A.7, let A,, be any compact subset of A and let Ma be the associated subset of M. Cesari (1966, p. 375) shows that Ma is compact if U( 1. X) is compact-valued and A.9 holds. Consequently, /a( I, x, U) is bounded above on M,,, so that A.7 holds for any function h that has the properties listed in A.7 and that exceeds this bound. A similar argument shows that A.2 and A.12 imply A.8. 0

Theorem 2 shows that the hypotheses of the more general Theorem 1 can be simplified considerably if there are a priori bounds on the controls. The next section of the paper further modifies the assumptions of Theorem 1 in the direction of greater intuitive content and ease of verification. 4. Simplification

of assumptions

The following assumptions can be used to replace all of the hypotheses in Theorem 1 except for the basic continuity and closure conditions A.l-A.4. A.13. The control set correspondence U(r, x) is convex-valued. dynamics f(f, x, u) are linear in u; that is,

The state

f(t, x, u) = C(r, x) + D(t,x)u,

(18)

for some n x m matrix D(r, X) and n x 1 vector C(r, x). The instantaneous net benefit function ja(l, x, u) is concave in u. A.14. There exist constants b,, 6,, b, and c, with b, > 0 and c > 1, such that fO(~,x,u)~h(~u~)=bo+b,~u[-b,~u~‘,

forall(r,x,u)inM.

(19)

A.15. There exist non-negative constants C,,, C, and C,, such that ~/(r,x,u)l~C,+C,lxl+C,lul,

forall(r,x,u)in

M.

(20)

306

M.A.

Tottmtr.

Optintol

control

wirh an uttbouttded

horizon

A.16. For some r > 0, the net benefit function f0 satisfies fo( 1, x, u) = e-“F( t, x, u),

(21)

and there exist non-negative numbers F,, Fr and F2 such that F( t, x, u) s F, + FJxl+

FJul,

for all (t, x, u) in M.

(22)

A.17. The terminal valuation function W satisfies W(t, x) = e-“w(f,

x),

(23)

and there exist non-negative numbers wa and w1 such that Iw(f,x)I
forall(t,x)inA.

(24)

A.18. There exist non-negative numbers U, and U, such that lul
UI1xI,

foralluinU(t,x)and(t,x)in

A.

(25)

A.19. The discount factor r satisfies r > C, + CJJ,,

(26)

where C, and C, are as in (20) and VI is as in (25). Assumption A.14 specifies a particular choice of the function h in A.7 and strengthens this assumption by requiring it to hold globally on M, not just on an arbitrary compact subset Ma. The function h in (19) indicates the sense in which both A.7 and A.14 are concavity and curvature conditions on fo. Assumption A.15 strengthens A.8 by positing a uniform Lipschitz condition in both x and u on f. This condition holds in particular if f is differentiable with respect to x and u with uniformly bounded derivatives; both (18) and (20) are satisfied automatically if f is linear in x and u with constant or uniformly bounded coefficients. Assumptions A.16 and A.17 explicitly introduce a discount factor e-” into the valuation functions f. and W, and they specify uniform Lips&i& conditions on the undiscounted valuations F and w. Similarly, A.18 specifies an upper bound on u E U(t, x) that is independent of t and linear in x. Assumptions A.15-A.18 are particularly applicable in stationary problems, where f=f(x, u), F= F(x, u), W= W(x), U= U(x), and time enters explicitly only through the discount factor. They are also applicable in problems where the functions or control sets are dominated by stationary functions or control sets; for example, where If(t, x, u)I 5 lf(x, u)I or U(t, x) is contained

M.A.

Toman,

Optimal

control

with an unbounded

horizon

301

in o(x) for some f or 0. As indicated subsequently, these stationarity conditions can be relaxed in several ways, and A.18 can be derived from more basic hypotheses on the control sets. Finally, A.19 is a growth condition whose role is brought out below. The first result retains the asymptotic conditions A.lO-A.11 in Theorem 1, but replaces some of its other hypotheses, particularly the difficult-to-verify condition AS. Theorem 3. The conclusions of Theorem 1 hold if (i) A.13 replaces A.5 and A.8, (ii) A.14 replaces A.7, and (iii) A.14-A.15 replace A.7-A.8. Moreover, if A.12 also is satisfied then A.6 also can be dropped. Proof. (i) Fleming and &she1 (1975, p. 201) show that Q(t, x) in (6) is convex-valued under A.13, so A.13 implies A.5. To see that A.13 also implies A.8, let A, be a compact subset of A, and let

where IlXll denotes the matrix norm of an array X. These numbers are well defined since f is continuous by A.3, and these choices of C and D satisfy (8). The proof of (ii) is trivial. To prove (iii), it suffices to show that Proposition 1 - the finite horizon existence result - is valid when A.14-A.15 replace A.7-A.8. See Fleming and Rishel (1975, pp. 69-71) for a demonstration of this. Finally, the last statement of the theorem follows directly from Theorem 2.

cl

As noted above, a key feature of A.13 is that it provides an alternative to A.5. Models with state dynamics that are linear in the controls and instantaneous net benefit functions that are concave in the controls are more special than the general cases treated in Theorems 1 and 2. Fortunately, however, these conditions arise quite frequently in economic applications. On the other hand, part (i) of Theorem 3 can be interpreted as a note of caution concerning existence of solutions in problems where the state dynamics are non-linear in the controls, since there are few practical alternatives to A.13 in ensuring A.5.” The balance of this section is devoted to showing that the abstract conditions A.lO-A.11 can be replaced by the more intuitive and verifiable conditions A.15-A.19. To this end, a growth condition on admissible trajectories is required. “Assumption A.5 also can be verified in certain special cases where both the states and control vectors are one-dimensional (that is, n = m = 1). For example, if fO is concave in u, then it can be shown that Q( t, X) is convex-valued if /c also is monotonically increasing in u and if / is either increasing and convex in u or decreasing and concave in u. However, these assumptions have limited practical applicability.

M.A.

308

Lemma 3.

Toman,

Optimal

control

with an unbounded

borirotr

Define non-negative numbers J, D, and D, by

Y=max{lyl:

(s,Y)

E B},

D,-,=Co+C&

Di=C,+C,U,,

(27)

where C,, C,, C, are as in (20) and U,, VI are as in (25). Define X(t), O
D,=O, (28)

= (j + D,/D,)(evl’-

l),

D, # 0.

Assume A.1, A.15 and A.18, and let x(t) be any admissible trajectory. Then A.9(b) holds, and Ix(t)1
and A.19 or A.9(a).

Proof. Note first that A.18 implies A.12, so that A.6-A.8 are valid in view of Theorem 2. By Theorem 3, A.13 can replace A.5. Finally, Lemma 3 implies that A.9(b) holds. Thus, to complete the proof of Theorem 4 it suffices to show that A.lO-A.11 are satisfied under its hypotheses and to call upon Theorem 1. To verify A.lO, combine (21), (22) and (25) to obtain

f,(t, x(t), u(t)) s em”{F,+F,U,+(F,+F*U,)Ix(t)l},

(29)

along any admissible program (x, u). If A.9(a) holds, then there is a fixed number X such that Ix(t)1 s X for all t. In this case, the right-hand side of (29) with Ix(t)1 replaced by X is a suitable candidate for the function g(t) in (10). Otherwise Ix(t)1 s Z(t) for all t, where F(t) is given by (28), and condition (26) in A.19 implies that e-E?(t) is integrable on R:. In this case the right-hand side of (29) with 1x( t)l replaced by X(t) is a suitable candidate for s(t)* To verify A.ll, combine (23) and (24) to obtain

for all admissible trajectories x(t) with finite terminal date T. Arguing as in the verification of A.10 above, it follows that A.9(a) or A.19 imply A.ll. 0 Note that in eqs. (20), (22), (24) and (25), the terms Ci, 4, W;. and q are assumed to be constant, thus ruling out exogenous shifts in tastes and

M.A.

Toman,

Optimal

control

with an unbounded

horizon

309

technology. These assumptions are made only for ease of exposition and can be easily relaxed. To see this, note first that A.15-A.18 can be trivially extended by replacing the Ci, Fi, Wi and Vi with bounded functions of t. More generally, suppose that C,, C, and lJ, are constant (or bounded), but that the other terms in A.15-A.18 depend on t. Then arguing as in the proof of Lemma 3, it follows that (28’)

for some constant K. Consequently, A.10 is satisfied in view of (28’) and (29) if the following functions are integrable on [0, co):

e-r’{F,(t) + F&w&))~ ewr’{ F,(t)

+ F2(t)Ul}[jbo,(t’)dr’].DI’.

(31)

Similarly, A.11 is satisfied in view of (28’) and (30) if e-“wo( t) + 0,

e

po(

t’) dt’]eDl’ + 0.

(32)

Both (31) and (32) hold if (26) is satisfied and if the functions F,(t), F,(t), F2(t), U,(t) and Do(t) are dominated by polynomial functions of t (rather than being bounded or constant), since /OWtNe-(r-Dl)tdt < cc and tNe-(r-Dl)’ ---)0 as t + co for any integer N. Conditions (31) and (32) also will hold if the functions listed above grow exponentially, provided A.19 is suitably strengthened. The argument in the previous two paragraphs rests on the assumptions that C, and C, in (20) and U, in (25) are bounded. As noted at the beginning of this section, A.15 holds with constant C, and C, in (20) if the state dynamics function f(t, x, U) is differentiable in x and u with uniformly bounded derivatives. Assumption A.18 also holds with U, bounded in (25) if the control set correspondence satisfies (lla) and the ‘capacity constraint’ function H,,( t, x) is differentiable in x with uniformly bounded derivatives. Similarly, A.18 can be inferred if U( t, x) satisfies (llb) with analogous assumptions on H( t, x). A more complicated argument is required if U( t, x) is not bounded a priori, as in (11~). However, in some cases bounds on controls like the functions H,(t,x) and H(t, x) in (lla) and (llb) can be derived from other characteristics of the problem, so that Theorem 4 can be made to apply. Moreover, condition (26) in A.19 also can be relaxed. These issues are discussed further in the concluding section of the paper.

310

M.A.

Toman,

Optimal

conrrol

with an unbounded

horizon

5. The infinite horizon problem

An existence theorem for the infinite horizon control problem corresponding to (V, S) can be derived directly from Proposition 3. Formally, let So be the collection of admissible programs for (V, S) that have an infinite horizon; in other words, (x, u) E So if (x, U) E S and T= cc. Define the performance index V”(x, u) by vO(x,

u) = R(s,

y) + iii6 j’j&, f’-+ca s

x, 4 dt (33)

= R(s,

y) + lim x,(t’), I’--. m

analogous to (1) and (l’), where x0 is the auxiliary state as in (2). Then (V’, So) is a well-defined infinite horizon control problem provided So Z 0 .18 As before, the problem with initial time s and fixed initial state y - so that R(s, v) can be ignored in (33) - is a special case. Theorem 5. Assume A.l-A.3 and A.S-A.11. Then provided So+ 0, an optimal program for (V’, So) exists. A solution to (I/‘, So) also exists under the modifications of assumptions enumerated in Theorems 2-4 (omitting A.1 7). Proof. Define W(t, x) = 0 for all (t, x) E A. Then Proposition 3 implies that a solution exists under A.l-A.3 and AS-A.11, since A.4 holds trivially. The second assertion follows directly from the proofs of Theorems 2-4. •1 The hypotheses of Theorem 5 are fairly intuitive and easily verified in specific applications. Moreover, joint concavity of the instantaneous net benefit function fo(t, x, U) in (x, u) is not imposed, so Theorem 5 may be applicable to problems with some degree of ‘increasing returns’. 6. Weakening of assumptions

As pointed out, for example, by Magill (1981) and Balder (1983), the key to existence results for control problems lies in the simple fact that a continuous (or, for that matter, uppersemicontinuous) function on a compact set achieves a maximum value on that set. Thus, the essence of the argument is in specifying conditions under which the set of admissible programs is compact in some appropriate topology and the performance index is continuous (or at least uppersemicontinuous) in that topology. Achieving this involves some ‘tightness’ conditions on the velocities of the state variables (e.g., uniform lRSufiicient conditions for So + 0 were discussed briefly in the third section of the paper.

MA.

Toman,

Optimal

control

with an unbounded

horizon

311

boundedness) which imply a ‘regularity’ of the state trajectomy, and corresponding restrictions on the instantaneous benefit function. The continuity, concavity and growth conditions specified in section 3 of the paper (and their more intuitive analogues introduced in section 4) are one set of hypotheses to this end. As indicated below, these hypotheses can be relaxed in several directions. Since one aim of the paper is to present material in as intuitive a matter as possible, as noted in the Introduction, the discussion which follows is neither formal nor exhaustive; readers are referred to the cited references for details and additional material. The most straight-forward extension is that, since only uppersemicontinuity of the performance index over feasible programs is required, the continuity assumptions on elements of the performance index (R, f0 and W) can be relaxed to uppersemicontinuity [Magill (1981, pp. 691-692), Balder (1983, pp. 198-199), Cesari (1983, chs. 9-ll)].” The USC conditions on the correspondences IY(~, x) and Q(f, x) in A.2 and A.6 also can be relaxed; see Cesari (1966, pp. 372-376; 1983, chs. 8,13) and Balder (1983). The tightness conditions on the derivative of the state vector alluded to above are achieved in the present paper through growth and boundedness conditions like A.8, A.9, A.12, A.13, A.15, A.18 and A.19. As pointed out by Balder (1983, p. 197), such conditions reduce to an essential requirement of uniform integrability of velocity vectors. A variety of other growth conditions can be used to achieve the same result [Magill (1981, sect. 5), Balder (1983, p. 205), Cesari (1983, chs. 11, 13)]. Similarly, tightness conditions on the instantaneous net benefit function over an infinite horizon reduce to an essential requirement of strong uniform integrabihty. This is achieved by the bound condition A.10 in the present paper but can also be achieved in other ways [Magill (1981, sect. 7) Balder (1983, p. 205)]. Finally, requirements that growth conditions hold globally can be relaxed by allowing for a ‘slender’ subset of the set A where such conditions are not satisfied [Cesari (1983, ch. 12)]. 7. Concluding remarks

This paper has developed a series of existence theorems for both ‘ unbounded horizon’ and infinite horizon control problems under successively more intuitive and verifiable assumptions. As noted above, the assumptions underlying these results are not the most general hypotheses imaginable. Nevertheless, the theorems are applicable to a variety of economic control problems. For example, at several points in the paper it has been noted that the theorems presented here involve fairly weak concavity conditions and thus can “For

minimization

problems

the weakening

is to lowersemicontinuity.

M.A.

312

Toman.

Optimal

control

with au unhouuded

Irori:ou

be applied to problems with some degree of ‘increasing returns’. To briefly illustrate this point, consider a simplified version of Treadway’s (1970) neoclassical competitive firm model, with a discounted profit function given by f,(t,

K, L, I) =e-“{

p,9(K,

L) - w,L--r,K--m,Z},

where (K, L) are capital and labor, Z is gross investment, 9 is the production function, p, is the price of output, and w,, T, and m, are unit factor costs. Suppose that p, I j < cc for all t, and suppose that 9 is concave in L but not necessarily jointly concave in (K, L). If labor is an essential input, so that ‘k( K, 0) = 0 for all K, then concavity of 9 in L implies that fo( t, K, L, I) s eer’j\r/,(

K,O)L,

where ‘kL = iW/aL is the marginal product of labor, and (22) will be satisfied if qL( K, 0) is uniformly bounded in K. This will be the case in particular if K and L are substitutes, so that qKL < 0. On the other hand, if ‘k( K, 0) > 0, then concavity of 9 in L and the mean value theorem imply that

where 0 I 5 = t(K) I 1. In this case, (22) will be satisfied if !k,J K, 0) and ‘kK( K,O) are uniformly bounded in K. The latter condition rules out globally increasing marginal returns to capital, where ‘kK --) cc as K + co. However, it is consistent, for example, with increasing marginal returns at small levels of K. Another important point involves relaxing the global condition (26) on the discount rate in A.19. To illustrate this point, as well as the concavity issues discussed above, consider briefly the standard one-sector optimal growth model treated in Burmeister and Dobell (1970, ch. ll), with k denoting per-capita capital and #(k) the rate of per-capita output. Since k I #(k), it follows that condition (20) in A.15 will hold if the marginal product of capital #‘(k) is bounded, with C, = 0 and C, equal to the bound on marginal product. Assumption A.19 then requires that r > C,, meaning that the discount rate is larger than the maximal marginal product. In this case, steady-state growth is not attainable. Note, however, that in the proof of Theorem 4, the growth conditions (26), (28) and (29) are not required to hold for all Ix(t)l. Instead, they must hold only if Ix(t)1 grows large. Thus, the constants in A.15-A.18 can be replaced with constants such that the inequalities (20), (22), (24) and (25) hold only outside some (possibly large) compact subset M,, of M. With this change in hypotheses, it may be possible to satisfy A.19 for the new constants D, = C, -t

M.A.

Tonwn.

0pfinu.d

control

with on unbounded

horizon

313

C&J,. For example, in the one-sector optimal growth model, (26) can be satisfied with the appropriate choice of M, if rC/‘(k) + 0 as k + 00. This argument does not hinge on global concavity of the production function 4, though a globally convex function with q’(k) + cc as k + cc is ruled out. Yet another concern involves the boundedness conditions on the control sets U( t, x) imposed in A.12 and A.14. As noted previously, in many problems where there are no a priori bounds on the controls, effective bounds - bounds which must be satisfied along an optimal program - can be inferred from other aspects of the problem. To illustrate the procedure, consider briefly a simple Hotelling (1931) model of non-renewable resource extraction by a competitive firm.“’ In this problem, the discounted profit function is given by

fo(t, R, 4) = em”{p(r)q- E(q, R)}, where R is the stock of extractible reserves, q is the extraction rate, k = -q with initial reserves of R(0) = x > 0, p(t) is the price of extracted output, and E(q, R) is the cost of extraction with E, = dE/dR < 0, so that extraction cost rises as reserves fall. Assume that p(t) IF < 00 for all t; the bound j can be interpreted as the ‘backstop’ price of renewable substitute resources. In addition, assume that E is continuously differentiable and strictly convex in q, with aE/dq = E, non-increasing in R (so that marginal cost also rises as reserves fall) and E,(q, R) + co as q + 00. Finally, assume that quasi-fixed extraction cost is zero, so that E(0, R) = 0 for all R.*l In the absence of a capacity constraint on the extraction rate there is no a priori bound on the control q. Nevertheless, under the assumptions listed above an effective upper bound on q can be derived so that A.18 holds. To see this, note first that j-E,(q,R)lO

or

q=O

is a necessary condition for an extraction speaking, the reason is that if this condition can be increased by setting q = 0; this increases future profits as well by lowering Now define the function H,(R), 0 I R I H,(R)={q>O:

program to be optimal. Roughly is not satisfied, then current profit also conserves reserves and thus cost, since E, < 0. R, by

jq-E(q,R)isamaximum}.

“‘This model also is discussed in Levhari and Liviatan (1977). A detailed treatment of how the results in this paper can be applied to a substantial generalization of the Hotelling model is contained in Toman (1985a. b). “This assumption is made because, if E(0, R) > 0, then the optimal terminal date necessarily is finite [Kemp and Long (1980)] and finite horizon existence theorems can be applied.

314

M.A.

Tomarl,

Optimal

control

with an unbounded

horizon

Since E is continuously differentiable and strictly convex in q with E, + 00 as --, co, Ha is a well-defined, single-valued, non-negative, continuous function. HO also can be shown to be non-decreasing since E, is non-increasing in R. Since q I H,(R) is required for an opt&~ by the remarks in the previous paragraph, A.18 is satisfied with U,, = H,(R) and Vi = 0 in (25). The argument above relies on the boundedness of the state variable R. However, a somewhat more complicated approach (with additio-naI assumptions) can be used to derive an alternative upper bound function H,(R) that is Lipschitzian in R, so that (25) is satisfied with Ur > 0. Thus, the procedure can be applied even if the state variables are not bounded a priori. Another topic in this discussion of applications concerns the assumption in A.1 that in problems with variable initial state or time, the choices of s or y are bounded a priori. This assumption could be relaxed using a dynamic programming approach. Specifically, for given (s, y) let V*(s, y) be the value of an optimal program starting from this fixed initial point. Suppose that this value function is differentiable with bounded derivatives, and that the initial valuation function R(s, y) is strictly concave with derivatives that converge to - co as s + cc or ]v] + co. Then arguing as in the previous paragraphs, effective bounds on s and y can be derived. The scope of this method is limited, however, by difficulties in verifying the required properties of the value function V *?2 A final observation concerns the application of the results in this paper to spatial or cross-section problems such as non-linear taxation [Mirrlees (1971)] and pricing [Mirman and Sibley (1980)] as well as to intertemporal problems, which have been the main focus of this paper. In spatial applications, the intertemporal discount factor e-” can be replaced by a density function C+(Z) converging supra-linearly to zero as z + co so that analogues of AJO-A.11 are valid. However, there are well-known difficulties (noted by Mirrlees) which arise in rigorously casting these models in terms of a control problem. These difficulties stem in part from the possible multivaluedness of agents’ demand or supply relations when confronted with non-linear price or tax schedules. Resolution of these issues is a prerequisite to extending the results proved here to such spatial contexts. q

Appendix

Proof of Lemma 2. Define the correspondence Q’( t, x) on A by Q/(&x)=

{(z,,,z)~R”+~: z=f(t,x,l4),

zo=fo(t,x,u), UE U(t,x)}.

“See Benveniste and Scheinkman (1983) for discussion of these issues.

M.A.

Tomun.

Optimul

control

with m unbounded

horizon

315

Given that U(t, x) is compact and f0 and f are continuous, Q’ can be shown to be USC [Cesari (1966, p. 1977)].23 Combining this fact with the definition of uppersemicontinuity and the formula

Qk x> = u [(-0)

+ Q’k x)1,

U20

where [(-u, 0) = Q’( t, x)] is the translate of Q’(t, x) by (-u, 0), it can be shown that Q is USC. 0 Proof of Lemma 3.

Combining (20) and (25) yields

~f(t,x,u)~sD,+D,~x~,

forall(t,x,u)inM.

Since i = f (t, x, u), x(s) =y, (s, v) is contained in the compact set I?, and s 2 0, it follows that _cj+D,t+D,[‘lx(r)ldt. If D, = 0, then Ix(P)] I Z(P) with Z(P) given by the first half of (28). If D, > 0, then an application of Gronwall’s Lemma [Fleming and Rishel(l975, p. 198)] to the above inequality yields the same conclusion with Z(P) given by the second half of (28). •I References Balder, E.J., 1983, An existence result for optimal economic growth problems, Journal of Mathematical Analysis and Applications 95, 195-213. Baum, R.F., 1976, Existence theorems for Lagrange control problems with unbounded time domain, Journal of Optimization Theory and Applications 19, 89-116. Benveniste. L. and J. Scheinkman, 1983, Duality theory for dynamic optimization models of economics: The continuous time case, Journal of Economic Theory 27,1-19. Brock. W.A. and A. Haurie, 1976, On existence of overtaking optimal trajectories over an infinite time horizon, Mathematics of Operations Research 1, 337-346. Burmeister, E. and A.R. Dobell, 1970, Mathematical theories of economic growth (MacMillan, New York). Carlson, D.A., 1984a. A Caratheodory-Hamilton-Jacobi theory for infinite horizon optimal control programs, Discussion paper IAM 1984-5 (Institute of Applied Mathematics, University of Missouri, Rolla, MO). Also forthcoming in the Journal of Optimization Theory and Applications. Carlson, D.A., 1984b, On the existence of catching up optimal solutions for Lagrange problems defined on unbounded intervals, Discussion paper IAM 1984-6 (Institute of Applied Mathematics, University of Missouri, Rolla, MO). Also forthcoming in the Journal of Optimization Theory and Applications. Cesari, L.. 1966, Existence theorems for weak and usual optimal solutions in Lagrange problems with unilateral constraints (I), Transactions of the American Mathematical Society 124.369-412. ‘31n Cesari’s notation. Q’ is denoted by Q.

J.E.D.C.-

C

316

M.A.

Toman,

Optimal

cotr~rol wirlt at, unbounded

horizon

Cesari. L.. 1983, Optimization theory and Applications: Problems with ordinary differential equations (Springer-Verlag. New York). Fleming, W.H. and R. Rishel, 1975. Deterministic and stochastic optimal control (Springer-Verlag, New York). Hale, J.K., 1969, Ordinary ditTerentiaJ equations (Wiley, New York). Haurie, A., 1980. Existence and global asymptotic stability of optimal trajectories for a class of infinite-horizon nonconvex systems, Journal of Optimization Theory and Applications 31, 515-533. Hewitt, E. and K. Stromberg, 1965, Real and abstract analysis (Springer-Verlag, New York). Hildenbrand, W. and A.P. Kirman, 1976, Introduction to equilibrium analysis (North-Holland, Amsterdam). Hotelling. H., 1931, The economics of exhaustible resources, Journal of Political Economy 39, 137-175. Levhari, D. and N. Liviatan, 1977, Notes on Hotelling’s economics of exhaustible resources, Canadian Journal of Economics 10, 177-192. Kemp. M. and N. Long, 1980, Toward a more general theory of the mining firm, in: Exhaustible resources, optimality, and trade (North-Holland, Amsterdam). Magill. M.J.P., 1981, Infinite horizon programs, Econometrica 49, 679-712. Mirman, L.J. and D. Sibley, 1980, Optimal nonlinear prices for multiproduct monopolies. Bell Journal of Economics 11,659-670. Mirrlees, J.A., 1971, An exploration in the theory of optimum income taxation, Review of Economic Studies 38, 175-208. Peterson, F.M., 1978, A model of mining and exploring for exhaustible resources. Journal of Environmental Economics and Management 5,236-2411. Pindyck, R.S., 1978, The optimal exploration and production of nonrenewable resources, Journal of Political Economy 86, 841-861. Rockafeller, R.T., 1978, Saddle points of Hamiltonian systems in convex problems of Lagrange, Journal of Ootimization Theorv and ADDhdOnS 12. 367-390. Rockafeller, R.T.. 1975, Existence theorems for general control problems of Bolza and Lagrange, Advances in Mathematics 15,312-333. Toman, M.A., 1985a. Existence and transversality conditions for a general infinite-horizon model of the mining firm, Discussion paper D-98 (Resources for the Future, Washington, DC). Toman, M.A., 1985b, Existence and optimality of dynamic competitive equilibria with a nonrenewable resource, Discussion paper D-99 (Resources for the Future, Washington, DC). Treadway, A.B., 1970, Adjustment costs and variable inputs in the theory of the competitive firm. Journal of Economic Theory 2,329-347.