- Email: [email protected]
The Nash solution of a maintenance-production differential game
165
The N ash solution of a maintenance-production differential game Gustav FEICHTINGER h~stitut f~r Okonometrte und Operatwns Research, Techmsche Umversitiit Wien, A-1040 Wien, A ustr:a Received May 1981 Revised August 198 !
A simple two-person nonzero-sum differential game between the maintenance crew of a firm and the production department of the same firm is considered. It is assumed that preventive maintenance decelerates the decrease in quality of a machine, whereas the production activities set by the firm reduces the quality of the machine. Due to the special structure of the game a system of two differential equations for the non-cooperative Nash solution trajectories can be derived. This provides a possibility to obtain quahtauve insights into the structure of the solution paths by phase diagram methods without solving the necessary optimality conditions explicitly. For small salvage values of the machine, it is shown that the optimal maintenance expenditures decrease over time whereas the optimal production rate increases monotonically. Moreover, another interpretation of the differential game analyzed is given as a conflict situation between a firm polluting the environment by producing an output and a go~,ernmental agency cleaning up the pollution.
1. Introduction
Until now only few classes of differential games which are solvable inasmuch as they allow for qualitative characterization of their solutions have been isolated in the ocean of nonzero-sum differential games (cp., e.g. Clemhout and Wan [5, p. 21]). For the progress of the theory of differential games and its applications it would be a valuable task to formulate dynamic competitive situations whose solutions could be either determined analytically or at least characterized qualitatively. Unfortunately, for most games describing quasi-reThe author would like to acknowledge the valuable comments by Richard Harti, Alexander Mehlmann, and two anonymous referees on this paper. North-Holland Publishing Company European Journal of Operational Research 10 (1982) 165-172
alistic competitive processes this plan is too ambitious. The following simple model, however, provides an example of a qualitatively solvable differential game in the above sense. Let us consider the department of a firm responsible for maintenance of production devices and the production department of the same firm as its competitor. The conflict situation arises from the fact that the production process carried out by the firm reduces the quality of the production devices (machines). The maintenance department is interested in a high quality of the machine and spends money to maintain this quality. The aim of the maintenance department is to balance intertemporally the instantaneous utility of mac~ne quality against the expenditures for preventive maintenance. The quality of the machine may be also conceived as production factor in the sense that a higher quality of the machine causes an increzse in production profits. Thus the production department will determine the production path with the optimal trade-off between the instantaneous production gain depending on the production rate and the quahty on the one hand and the quality reduction by this production intensity oa the other hand. As the maintenance department spends money to decelerate the decrease in quality, it would be plausible to oblige the production department to pay to the agency a certain amount per unit of its production factor 'machine quality'. The crucial point defining a game situation is that the state of the machine is influenced by the decisions of both competitors. The problem is to determine optimal intertemporal behavior of both players, where non-cooperation between them is assumeo. It is admitted that the model we are going on to analyze was selected primarily because of its mathematical convenience rather than its realistic patterns. The game belongs to that class of N-person nonzero-sum differential games whose non-cooperative Nash solution trajectories satisfy as necessary conditions a system of N differential equations being independent of both state and adjoint
0377-2217/82/0000-0000/$02.75 © 1982 North-Holland
166
G. Feichtinger / Nash solution of maintenance-production differential game
variables. The class of trilinear differential games analyzed by Clemhout, Leitmann and Wan [3], and Clemhout and Wan [4] as well as their generalized class of trilinear games (Clemhout and Wan, [5, p. 21]) has this property. See also the advertising game of Leitmann and Schmitendorf [ 13] which belong to the class of qualitative solvable differential games as defined at the beginning of this section. In all these cases the qualitative solvability of the N-person nonzero-sum differential game depends on the system of N differential equations mentioned above which is satisfied by the non-cooperative open-loop Nash equilibrium solution. In the case of two players, qualitative insights into the behaviour of these solutions may be drawn by phase diagram analysis using the resulting transversality conditions. The model is presented in Section 2. Section 3 contains the derivation of the announced system of differential equations in the control variables from the necessary conditions for optimality. In Section4 a phase diagram analysis for optimal control trajectories is carried out, whereas in Section5 some conclusions are drawn. Finally, in Section 6 we present another interpretation of our game in the framework of pollution control.
gross revenue equals px which is diminished by the maintenance expenditures u. Then we are facing the following optimal control problem:
maXfoTe-n[px(t)- u(t)] dt + e-rrS(x(T)).
(2.1) s.t..~ =
-a-kx+h(u),
x(0) = x0,
(2.2)
and u >~0, where T is a given planning horizon, r the discount rate, S(x) the salvage value of the machine and x o the initial quality of the machine. Whereas Arora and Lele [1] assume that k(t) is a given time function which cannot be influenced, Hard [8,9] introduces the working speed v(t) as a second control instrument reducing the quality by the rate k(v). Running the machine at straining intensity v and spending maintenance expenditures u provides the following system dynamics
(2.3) It does not seem unrealistic to assume the marginal 'disefficiency' of u with respect to x to be increasing, i.e. k to be convex in u. Replacing in (2.1) the given function p by the control variable v, the present value of the net revenue stream is now given by
fore-r'[v(t)x(t) - u(t)] dt + e-'rS(x(T)). (2.4) 2. The model
Our starting point is a slight extension of Thompson's linear maintenance model (Thompson [ 16]) by Arora and Lele [ 1]. Let x(t) denote the quality of a machine. The deterioration rate d(t) is made up of two components: - the obsolescence rate a(t), which is due to technological progress, and is independent of the machine quality; and - t h e depreciation rate k(t)x(t), which is due to change in the physical characteristics and the performance of the machine; it is dependent upon and is proportional to the current salvage value of the machine. Maintenance expenditure u improve the quality by the effectiveness function h(u) assumed to be linear in u by Thompson [16] and Arora and Lele [1] but concave by Hartl [8,9]. Quality of a machine means that one piece produced by this machine has the value x. Per unit time p pieces are produced (p is the production rate) so that the
The structure of this optimal control model may be summarized roughly as follows: the state of the model can be improved by means of a control variable which causes costs and is reduced by another control instrument which yields a profit rate. The aim is to determine the optimal levels of these instruments over time in a way that maximizes the present value of the revenue stream. Given the initial quality x ( 0 ) = x 0, the optimal control problem is now: maximize (2.4) subject to (2.3) and u~>0, u~>0. Let us now assume two players instead of one decision-maker. Player i is responsible for the preventive maintenance of the machine considered, whereas player 2 produces an output by running this machine. Let x(t) denote the state (quality) of the machine, u~(t) the control variable of player I (maintenance expenditures at time t), and u2(t) the control variable of the competitor (production intensity or working speed of the machine). To define the state equation, we assume concav-
G. Feichtinger / Nash solution of maintenance-production differential game
ity of the effectiveness of maintenance expenditures,
h(O) =0, h'(u~) > 0 ,
h"(ul) < 0
for ul > 0 .
(2.5a)
The rate of deterioration caused by production rate u z is a convex function, k(O) = 0 ,
k(u2)>O,
k'(u2) > 0 , k"(u2) > 0
for u2 > 0 .
(2.5b)
To guarantee the existence of a nontrivial equilibrium point in the (u~.u2)-phase diagram (see below), we assume additionally lim h'(um) = oo, WI--~0
lim h'(u~) -- 0,
(2.6a)
181-4'00
k'(O) = 0 ,
lim k'(u2)-'-
oO.
(2.6b)
U2--* OO
The following state equation describes the dynamic behavior of the game
Y¢(t ) = h(u,(t)) - k( u2( t ))x( t ).
(2.7)
Remark. Note that for simplicity we have set a - 0 without any loss of generality. (2.7) means that the higher the quality of the production device, the more quality reducing is a given production rate /d 2 •
The initial state of the machine x(O) = x o.
+ e-',rS, x( T ).
the state of the machine. Since per unit time u2(t ) pieces are produced and one piece produced by the machine has value x(t) at time t, u2(t)x(t ) is the gross revenue of player2 at period t. Let further q2 be the amount which must be payed by the producer to the maintenance crew for one unit of quality. Since player I has maintenance expenditures u~ to provide quality it seems to be rational that he reclaims this costs at least partially from his competitor. To include q2 qJ should be greater than this quantity: q~ = q2 +
8
with ~ > 0.
(2.10)
Note that in case of q2 = 0 player2 takes only profit resulting from the production process. For the following analysis, the sign of q2 remains unspecified (see Section6, where q2 may also be negative). Thus, the objective funtional of the producer is the present value of the net revenue stream given by J2 = for-r~t[U2( t )X( t ) -- q2x( t )] dt
+e-'.rS2x(T).
(2.11)
The aim of the players is to maximize (2.9) and (2.11), J ! and J2, respectively, subject to the dynamic restriction (2.7) and the nonnegativity constraints ui~> '
u2~>0.
(2.12)
(2.8)
is assumed to be given. The aim of player I is to guarantee a maximal quality of the machine taking into consideration the maintenance expenditures. We assume a linear assessment of quality, denoting with q~ > 0 the utility of player I per quality unit. The performance index of player I is given by
J!--foTe-r't[qlx(t)--Ul(t)]
167
dt
(2.9)
r i is the nonnegative discount rate of player i (i = 1,2). The game is played on the fixed time interval [0,T]. For player i (i= 1,2), S, is the salvage value of one unit of machine quality at the end of the game. The output produced by player 2 depends on
Note that player I controls the maintenance expenditures u l(t), whereas the producer maximizes •/2 with respect to the production rate u2(t). The decision dilemma arises because the production intensity together with the machine quality yields an instantaneous gain but causes on the other hand a reduction in quality. This results in a two-person nonzero-sum differential game with x(t) as state variable and the rate of expenditures for preventive maintenance and production rate as instruments of player I and 2, respectively. Since uj is absent in J, for j ~ i, the competitor's decision has no direct influence on the player's utility functional. As mentioned above, the particular structure of the state equation and of the integrands in the objective functionals guarantees the possibility for interesting qualitative insights into the behavior of the optimal solutions.
168
G. Feichtinger / Nash solution of maintenance-production differential game
3. Differential equations for the non-cooperative Nash equilibrium It is well known that the solution concept of nonzero-sum differential games is ambiguous (see, e.g. Staff and Ho [15], Case [2], Ho [10], Leitmann [12]). Since the possibility seems not unrealistic that the players don't cooperate, the Nash equilibrium would be an appropriate solution concept for our maintenance game. An open-loop Nash equilibrium solution is a pair of admissible control paths {u~(t), u~(t)}, t E [0,T] such that for the two criteria J! = Ji(ui, us), .I2 = J2(ul, u2) we have
Jl( u'~, u~ ) >~Jl( Ul , ~ ),
J2(u'~,u~)>~J2(u~,u2)
(3.1)
for all admissible control trajectories u, (i = 1,2). The Nash-type of solution is secure against any attempt by one player to unilaterally alter his control since he cannot improve his performance by such a change. Candidates for open-loop Nash-solutions are produced by the following necessary conditions formulated by means of the Hamiltonians of both players. Denote by
H'=q,x-ut+X,[h(u,)-k(u2)x ]
(3.2)
and
H2=-q2x+u2x+X2[h(u,)-k(u2)x]
(3.3)
the current-value Hamiltonian of player I and 2, respectively. The current-value adjoint variable X,(t) measures the sensitivity of profits to a small perturbation in the state. Thus the i th player imputes a shadow price to the quality of environment. X,(t) satisfies the adjoint equation X,=r,h,Hx'. For i = 1,2 we obtain
X, = Xi[rl + k ( u 2 ) ] - q , ,
(3.4)
The transversality conditions are given by 2k,(T)=S,,
i = 1,2.
(3.6)
According to the maximum principle the optimal control of player I maximizes his Hamiltonian at each instant. Under the additional assumption lim h'(u,)= oo, ul~0
(3.7)
u t - 0 may be excluded as suboptimal, and the first order condition for a maximum, H~, = 0, implies that X, = [h'(u,)] -I
(3.8)
The shadow price of machine quality imputed by the governmental agency is simply reciprocal to the marginal effectiveness of expenditures for maintenance. Analogous, the first order condition for an interior maximum of H 2,//,2, = 0, may be applied under the assumption k'(0) = 0
(3.9)
yielding the necessary condition ~2 = [k'(u2)] -!
(3.10)
for the producer. Note that the shadow prices for both players are nonnegative. To eliminate the adjoint variables, we differentiate (3.8) and (3.10) with respect to time and obtain h"
X =-h,2a I,
k"
X2= k,2a .
(3.11)
Substituting (3.11) into the adjoint equation (3.4) and (3.5), respectively, the following system of nonlinear differential equations in the control variables is obtained:
-hh,~((ul)ul) [qlh'(u,)-k(u2)-rl] ,
(3.12)
k'(u,) Ct2- k,,(u2) [(u2- q2)k'(u2) - k(u2)- r2]•
(3.5) Since we restrict our analysis to open-loop Nash solutions (given ~ functions of time only and not of the state), troublesome cross-effect terms in the adjoint equations are absent. It is the special shape of the Hamiltonians which makes the following analysis possible.
Candidates for open-loop Nash-optimal solution trajectories { u~'(t), u~(t) } must satisfy this nonlinear system of differential equations. The possibility for the derivation of a system of differential equations for the 'optimal' control variables de-
169
G. Feichtinger / Nash solution of maintenance-productwn differenttai game
pends essentially on the linearity of the problem in the state variable as well as on the special structure of the Hamiltonians. The terminal conditions for system (3.12), (3.13) are obtained from the transversality conditions (3.6) and the maximum principles (3.8), (3.9): h'(ul(T))=S?',
k'(u2(T))=Sf'.
(3.14)
Because of h ' ( u l ) > O , k ' ( u 2 ) > 0 for u 2 > 0 and conditions (3.7), (3.9), (3.14) is uniquely solvable for each pair (S t, S 2) with S, I> 0 (i = 1,2) provided we assume additionally
Proof. We have .>
* ' ( u 2) : ( u 2-- q 2 ) k " ( u 2 ) =0
2>
for u 2 ~ q~, (4.2)
and according to (2.6b) 0(0) = --r 2 <~0, lim O ( u 2 ) = e~.
MI --,, O0
lim k ' ( u 2 ) = ~ .
(4.4)
! / 2 ---~ OO
From (4.2), (4.3), and (4.4) the statements of the iemma follow. Note that a positive solution /~2 of (4. l) satisfies
t~2 > q2" lim h ' ( u l ) = 0 ,
(4.3)
(4.5)
Thus, the isocline
1#2 ---~ OO
Solutions {ul(T ), u2(t)} of (3.14) are the terminal values of the optimal control of the players. Note that u , ( T ) does not depend on uj(T) for j # i and i = 1,2. Moreover, since (3.13) does not contain u I, the optimal policy of player 2 does not depend on the behavior of his competitor. On the other hand, the optimal Nash-solution of player ! is not independent of the control of player2. Thus, player 1 chooses its non-cooperative equilibrium with regard to the producer's policy, whereas player2 produces without consideration of the efforts of the maintenance department. Since the sufficient conditions of Leitmann and Stalford [17] are satisfied the candidates for optimal solutions provided by (3.12), (3.13) are Nash equilibria. This follows also from the concavity in x of the maximized Hamiltonians.
ti 2 = 0,~, # ( u 2 ) = 0
(4.6)
of system (3.12), (3.13) is horizontal and crosse.,, the u2-axis in the point (0, t~2) where fi2 is the maximal nonnegative solution of (4. I). Lemma 2. The isocline ft I = 0 ,~ qth'( u,) - k( u2) - r I = 0
(4.7)
is downward sloping and approaches" the ordinate axis asymptotically. I f r I > 0 , it intersects the abscissa in the point ( ul,O), where ft I t s positive and provided by h ' ( u l ) = r,q; ~
(4.8)
In case r I = O, the u i-axis is an asymptote for
t~, = 0 .
Proof. According to the implicit function theorem 4. Phase diagram analysis for optimal control paths To derive structural insights into the Nash-optimal solution trajectories, we calculate the equilibrium point of the system (3.12), (3.13). First, we prove the following simple results.
we have
du21 h'(ul) < 0. dul .~,:0 = qi k'(u2"-----'~ The further statements of Lemma 2 follow in'.. mediately from (2.5b), (2.6a).
Lemma 1. There always exists a solution of the This yields the following result:
equation
(4.1)
Proposition. In the cases r2 > O, and r2 - O , q2 > 0
for nonnegative values of u 2. For r2 > 0 this solution, say u2, is positive and unique. For r 2 = 0 and q2>0, (4.1) is solved by u 2 = 0
there exists a unique equilibrium (ul, f~2) of system (3.12), (3.13) in the interior of the first quadrant of the (u l, u2)-plane being an instable node. For r2 = O, q2 ~<0 there is on& a stationary point at the ul-axis (in case r I > 0 ) or no finite equilibrium (in case r i = 0).
dP(u2)=(u2-q2)k'(u2)-k(u2)-r2=O
and u 2 = u2 > q2. For r2 = 0 and q2 ~ O, the unique solution is zero.
170
G. Feichtinger / Nash solution of mamtenance- productwn differential game
Proof. Let u 2 be the maximal nonnegative solution of (4.6) (see Lemma 1). Substituting u 2 = 3 2 in (4.7) we obtain
h'( u, ) = [ k(32) + ri] q~-'.
I
u2(T)
'
(4.9)
According to Lemma 2 there exists a unique stationary point (3 I, 32) of (3.12), (3.13) determined as crossing point of the isoclines (4.7) and (4.6) except in the case r 2 = 0, q2 <~0, rt = 0, where the isoclines approach asymptotically. For Q > 0 and r 2 = 0 , q 2 > 0 , (3~, 32 ) is in the interior of the first quadrant of the phase plane. For r 2 = 0, q2 <~0 but r~ > 0 the equilibrium lies at the abscissa. In case r t > 0, r 2 = 0, q2 > 0 two equilibria exist, namely (a I, 3 2) and (a t, 0). To determine the stability properties of the stationary point (3t, 32) belonging to the interior of the first quadrant we calculate a!ii
u2T ,,0to 01,,0 I I
x\
02
b
,
(0'01
Ulff )
~Ul lul:O
Ot~ZJ = ( u 2 -- q 2 ) k ' ( u , ) . au,~ I u2=O
(4.10)
Because of (4.10) and (4.3), the Jacobian determinant of system (3.12), (3.13) evaluated in the equilibrium (3~, 32) is positive:
a(31,3,)=
3u~
Ou2
0
3u---'-~2~a,.c,2)
aa2
=qlh'(a,)(a2-q2)k'(32)>O. (4.11) Thus, (3 I, 32) is an unstable node. As mentioned before, the equilibrium in the interior is unique. Remark. In case r i > 0, r2 = 0, q2 > 0 the determinant of the Jacobian matrix evaluated in the second equilibrium (3~, 0) vanishes:
A(a,, 0) = 0. Assuming power functions for the efficiency functions,
h(ui)=u' ~ with0
withfl>l,
01
, ~1l 02"0 ~Ul
Fig. !. Phase diagram of the non-cooperative Nash solutions.
there exist additional isoclines, namely /~1 -- UI "-" 0,
= qlh'(u I ),
--
~2"-- U 2 : 0
and thus, in the general case, three additional stationary points. It can be shown, that (0, 0) is a stable node, whereas (0, 32 ) and (31, 0) are saddle points. However, since the qualitative behavior of the Nash-solution is not influenced by these boundary equilibria, we analyze the general case of concave and convex efficiency functions. The first quadrant of the (ut, u2)-phase plane is subdivided in four regions ( I - I V ) by the isoclines (4.7), (4.6). According to (4.11) al! trajectories move away from the unstable equilibrium (31, 32 ). The behavior of the Nash-equilibrium solution is different in the four regions: whereas G ( t ) > 0 in region III for i = 1,2, both of the u,(t) decrease monotonically in region IV. In region 1 we have t i ~ < 0 but ~ 2 > 0 , and in region II ~ > 0 , but ~2<0. Using these results the (u~, u 2)-phase plane is sketched in Fig. 1.
5. Conclusions According to (3.14) the terminal values of u,(T) are provided by S, for i = 1,2. It seems natural to consider the case St = c¢, S 2 = 0 in more detail. This would yield Ul(T) = u2(T ) = oo, a terminal point which could not be reached in finite time T. However, for S! very
G. Feichtinger / Nash solution of maintenance- productwn differential game
large (but finite) and $2 very small (but greater than zero) both u,(T) are very large (but finite) with ul(T) > ul and u2(T) > u 2. The solution path lies in region Ill (see Fig. 1) making economic sense. The Nash-solution may be determined by backward integration. The length of the trajectory depends on the duration of the differential game, T. Thus, in the case where the maintenance crew is interested in a high final quality whereas the producer doesn't evaluate the final state of the machine positively, both the nash-optimal maintenance expenditures and production rates increase monotonically: u~(t)>0,
u~(t)>0
for all t E [0, T].
In case S, =c,, i= !,2, c , > 0 and sufficiently small, ul(T) is very small and u2(T) very large. Thus, the non-cooperative Nash-solution lies in region I, i.e. the u~(t) decreases monotonically whereas u$(t) increases. This trajectory is shown off in Fig. 1. Some other possible optimal control paths depending on the values S, of both players are illustrated in Fig. 1. If both players impute large values for S,, u~(t) increases but u~(t) decreases (region II). For small S~ but $2 large enough both controls decrease monotonically. This optimal behavior makes economic sense. Note that the optimal production rate is always monotonic: increasing for small $2 and decreasing for large $2. For intermediate values of $1 the rate of maintenance expenditures u'~(t) over time shows a nonmonotonic development provided T is sufficiently large. For large $2, u'~(t) increases initially until the maximal value is reached at ~m = 0 and decreases thereafter. For small $2 the decreasing part of the u~ trajectory precedes the increasing one. Refering to Hartl's optimal control model where the two control instruments are at disposal to one decision-maker (Hartl [8,9], see also Feichtinger [6]), it would be an interesting task to compare the optimal policies in the differential game situation with those for the optimal control model. For q2 = 0, r i = r2 = r and S I = S 2 " - S this is an easy excercise. Hartl [8] proves that the optimal maintenance rate decreases monotonically whereas the optimal production rate is increasing provided that the value S of one unit of the salvage value of the ending state x(T) is small enough. The result
! 71
discussed above shows that the same type of behavior is also Nash-optimal in the game situation.
6. An interpretation as pollution game
The differential game analyzed so far may also be interpreted in another way as follows. The competitors are a governmental agency for cleaning the environment (player 1) and a firm polluting the environment by producing an output (player 2). The state variable x(t) is now interpreted as quality of the environment acting a~ production factor, e.g. clean water of a lake, (cf. also Sethi [14]). This means, that the poorer the quality of environment, the lower are the production profits). Let ul(t) be the rate of expenditures, in dollars, in period t for pollution control, and u2(t ) the rate of production. Pollution control u~ improves the quality of the lake with effectiveness h( u~ ), whereas production with intensity u 2 reduces the quality by rate k(u2)x. (2.7) means that the effectiveness of a dollar spent on pollution control does not depend on the existing quality. Furthermore, the higher the quality, the more quality reducing is a given production rate, i.e. the cleaner the environment, the more decreases its quality by a given production rate. Note that the solvability of the differential game in the (u~, u2)-phase plane depends on this assumption. An example for this situation would be the maintenance of a stock of houses being exposed to pollution by sulforal oxyd. The aim of the governmental cleaning agency is to provide a high level of environment qualityl taking into consideration the expenditures for it[" cleaning. Analogous as before, a linear utilir', function of environment quality is assumed for the. agency, q~ being the utility per unit of environment quality assessed by player 1. The output produced by the firm depends on the quality of environment. For simplicity, we assume that u2(t)x(t) is the output rate pr,~duced at period t by production rate u2 and quality x. The aim of the producer is to maximize the present value of his net profit provided by (2. ! 1) with respect to the time path of the production rate. Three interpretations of the term - q 2x in (2.11 ) are now possible: - F o r q2 = 0 it is the production gain which is maximized by player 2. - F o r q 2 > 0 , q2x is an amount which must be
! 72
G. Feichtinger / Nash solution of maintenance-production differential game
payed from the firm to the cleaning agency for one unit of environmental quality. Since player I cleans up the pollution by expenditures u~ it seems plausible that he reclaims this costs. Thus, in this case (2.10) should hold. - F o r q: < O, - q 2 x may be interpreted as utility imputed to the quality by the producing firm, or, more adequately, as reward of the firm for sparing the environment. According to (2.10) 6 > - q 2 should hold to guarantee q~ > 0.
References [1] S.R. Arora and P.T. Lele, A note on optimal maintenance policy and sale date of a machine, Management Sci. 17 (1970) 170-173. [2] J.H. Case. Economics and the Competitive Process INew York University Press, New York, 1979). [3] S. Clemhout, G. Leitmann and H.Y. Wan Jr., A differential game model of oligopoly. J. Cybernetics 3 (1973) 24-39. [4] S. Clemhout and H,Y. Wan Jr., A class of trilinear differential games, J. Optimization Theory Appi. 14 (1974) 419-424. [5] S. Clemhout and H.Y. Wan Jr., Interactive economic dynamics and differenual games, J. Optimization Theory Appl. 27 (1979) 7-30. [6] G. Feichtinger, Optimierung yon lnstandhaitungsinvestit~,.,~_~en, Produktionsintensitat und Nutzungsdauer maschineller Produktionsanlagen: Anwendungen der Kontroihheone in der lnstandhaltungs- und Produktionsplanung, in: K. Brockhoff and W. Krelle, Eds., U nternehmensplanung (Springer, Berlin, i 981 ) 213-234.
[7] O. Feichtinger and R. Hartl, Ein nichtlineares Kontroilmodell der Instandhaltung, OR Spektrum 3 (198 l) 49-58. 18] R. Hartl, Optimal control of concave economic models with two control instruments, Forschungsbericht Nr. 31 des lnstituts for Unternehmensforschung der Technischen Universit~t Wien (1980). [9] R. Hartl, Optimal maintenance and straining intensity of a machine: a nonlinear economic control problem, J. Econora. Dynamics and Control to appear. [10] Y.C. Ho, Differential games, dynamic optimization, and generalized control theory, J. Optimization Theory Appl. 6 (1970) 179-209. [!1] S. Jmgensen, A survey of some differential games in advertising, Paper presented at t~ae workshop on "Dynamics of the Firm", held in Brussels. October 30, 1980. European Institute for Advanced Studies in Management. To appear in J. Econom. Dynamic:; and Control. [12] G. Leitmann, Cooperative and Non-cooperative Many Players Differential Games (Springer, Wien, 1974). [13] G. Leitmann and W. Schmitendorf, Profit maximization through advertising: a nonzero sum differential game approach, IEEE Trans. Automatic Control 23 (1978) 645650. [14] S.P. Sethi, A linear bang-bang model of firm behavior and water quality, IEEE Trans. Automatic Control 22 (1977) 706-714, [15] A.W. Starr and Y.C. Ho, Non zero-sum differential games. J. Optimization Theory Appl. 3 (1969) 184-206. [16] G.L. Thompson, Optimal maintenance and sale data of a machine, Management Sci. 14 (1968) 543-550. [17] G. Leitmann and H. Stalford. Sufficiency for optimal strategies in Nash equilibrium games, in: A.V. Balakrishnan, Ed., Techniques of Optimization (Academic Press. New York, 1972) 279-285.
The N ash solution of a maintenance-production differential game Gustav FEICHTINGER h~stitut f~r Okonometrte und Operatwns Research, Techmsche Umversitiit Wien, A-1040 Wien, A ustr:a Received May 1981 Revised August 198 !
A simple two-person nonzero-sum differential game between the maintenance crew of a firm and the production department of the same firm is considered. It is assumed that preventive maintenance decelerates the decrease in quality of a machine, whereas the production activities set by the firm reduces the quality of the machine. Due to the special structure of the game a system of two differential equations for the non-cooperative Nash solution trajectories can be derived. This provides a possibility to obtain quahtauve insights into the structure of the solution paths by phase diagram methods without solving the necessary optimality conditions explicitly. For small salvage values of the machine, it is shown that the optimal maintenance expenditures decrease over time whereas the optimal production rate increases monotonically. Moreover, another interpretation of the differential game analyzed is given as a conflict situation between a firm polluting the environment by producing an output and a go~,ernmental agency cleaning up the pollution.
1. Introduction
Until now only few classes of differential games which are solvable inasmuch as they allow for qualitative characterization of their solutions have been isolated in the ocean of nonzero-sum differential games (cp., e.g. Clemhout and Wan [5, p. 21]). For the progress of the theory of differential games and its applications it would be a valuable task to formulate dynamic competitive situations whose solutions could be either determined analytically or at least characterized qualitatively. Unfortunately, for most games describing quasi-reThe author would like to acknowledge the valuable comments by Richard Harti, Alexander Mehlmann, and two anonymous referees on this paper. North-Holland Publishing Company European Journal of Operational Research 10 (1982) 165-172
alistic competitive processes this plan is too ambitious. The following simple model, however, provides an example of a qualitatively solvable differential game in the above sense. Let us consider the department of a firm responsible for maintenance of production devices and the production department of the same firm as its competitor. The conflict situation arises from the fact that the production process carried out by the firm reduces the quality of the production devices (machines). The maintenance department is interested in a high quality of the machine and spends money to maintain this quality. The aim of the maintenance department is to balance intertemporally the instantaneous utility of mac~ne quality against the expenditures for preventive maintenance. The quality of the machine may be also conceived as production factor in the sense that a higher quality of the machine causes an increzse in production profits. Thus the production department will determine the production path with the optimal trade-off between the instantaneous production gain depending on the production rate and the quahty on the one hand and the quality reduction by this production intensity oa the other hand. As the maintenance department spends money to decelerate the decrease in quality, it would be plausible to oblige the production department to pay to the agency a certain amount per unit of its production factor 'machine quality'. The crucial point defining a game situation is that the state of the machine is influenced by the decisions of both competitors. The problem is to determine optimal intertemporal behavior of both players, where non-cooperation between them is assumeo. It is admitted that the model we are going on to analyze was selected primarily because of its mathematical convenience rather than its realistic patterns. The game belongs to that class of N-person nonzero-sum differential games whose non-cooperative Nash solution trajectories satisfy as necessary conditions a system of N differential equations being independent of both state and adjoint
0377-2217/82/0000-0000/$02.75 © 1982 North-Holland
166
G. Feichtinger / Nash solution of maintenance-production differential game
variables. The class of trilinear differential games analyzed by Clemhout, Leitmann and Wan [3], and Clemhout and Wan [4] as well as their generalized class of trilinear games (Clemhout and Wan, [5, p. 21]) has this property. See also the advertising game of Leitmann and Schmitendorf [ 13] which belong to the class of qualitative solvable differential games as defined at the beginning of this section. In all these cases the qualitative solvability of the N-person nonzero-sum differential game depends on the system of N differential equations mentioned above which is satisfied by the non-cooperative open-loop Nash equilibrium solution. In the case of two players, qualitative insights into the behaviour of these solutions may be drawn by phase diagram analysis using the resulting transversality conditions. The model is presented in Section 2. Section 3 contains the derivation of the announced system of differential equations in the control variables from the necessary conditions for optimality. In Section4 a phase diagram analysis for optimal control trajectories is carried out, whereas in Section5 some conclusions are drawn. Finally, in Section 6 we present another interpretation of our game in the framework of pollution control.
gross revenue equals px which is diminished by the maintenance expenditures u. Then we are facing the following optimal control problem:
maXfoTe-n[px(t)- u(t)] dt + e-rrS(x(T)).
(2.1) s.t..~ =
-a-kx+h(u),
x(0) = x0,
(2.2)
and u >~0, where T is a given planning horizon, r the discount rate, S(x) the salvage value of the machine and x o the initial quality of the machine. Whereas Arora and Lele [1] assume that k(t) is a given time function which cannot be influenced, Hard [8,9] introduces the working speed v(t) as a second control instrument reducing the quality by the rate k(v). Running the machine at straining intensity v and spending maintenance expenditures u provides the following system dynamics
(2.3) It does not seem unrealistic to assume the marginal 'disefficiency' of u with respect to x to be increasing, i.e. k to be convex in u. Replacing in (2.1) the given function p by the control variable v, the present value of the net revenue stream is now given by
fore-r'[v(t)x(t) - u(t)] dt + e-'rS(x(T)). (2.4) 2. The model
Our starting point is a slight extension of Thompson's linear maintenance model (Thompson [ 16]) by Arora and Lele [ 1]. Let x(t) denote the quality of a machine. The deterioration rate d(t) is made up of two components: - the obsolescence rate a(t), which is due to technological progress, and is independent of the machine quality; and - t h e depreciation rate k(t)x(t), which is due to change in the physical characteristics and the performance of the machine; it is dependent upon and is proportional to the current salvage value of the machine. Maintenance expenditure u improve the quality by the effectiveness function h(u) assumed to be linear in u by Thompson [16] and Arora and Lele [1] but concave by Hartl [8,9]. Quality of a machine means that one piece produced by this machine has the value x. Per unit time p pieces are produced (p is the production rate) so that the
The structure of this optimal control model may be summarized roughly as follows: the state of the model can be improved by means of a control variable which causes costs and is reduced by another control instrument which yields a profit rate. The aim is to determine the optimal levels of these instruments over time in a way that maximizes the present value of the revenue stream. Given the initial quality x ( 0 ) = x 0, the optimal control problem is now: maximize (2.4) subject to (2.3) and u~>0, u~>0. Let us now assume two players instead of one decision-maker. Player i is responsible for the preventive maintenance of the machine considered, whereas player 2 produces an output by running this machine. Let x(t) denote the state (quality) of the machine, u~(t) the control variable of player I (maintenance expenditures at time t), and u2(t) the control variable of the competitor (production intensity or working speed of the machine). To define the state equation, we assume concav-
G. Feichtinger / Nash solution of maintenance-production differential game
ity of the effectiveness of maintenance expenditures,
h(O) =0, h'(u~) > 0 ,
h"(ul) < 0
for ul > 0 .
(2.5a)
The rate of deterioration caused by production rate u z is a convex function, k(O) = 0 ,
k(u2)>O,
k'(u2) > 0 , k"(u2) > 0
for u2 > 0 .
(2.5b)
To guarantee the existence of a nontrivial equilibrium point in the (u~.u2)-phase diagram (see below), we assume additionally lim h'(um) = oo, WI--~0
lim h'(u~) -- 0,
(2.6a)
181-4'00
k'(O) = 0 ,
lim k'(u2)-'-
oO.
(2.6b)
U2--* OO
The following state equation describes the dynamic behavior of the game
Y¢(t ) = h(u,(t)) - k( u2( t ))x( t ).
(2.7)
Remark. Note that for simplicity we have set a - 0 without any loss of generality. (2.7) means that the higher the quality of the production device, the more quality reducing is a given production rate /d 2 •
The initial state of the machine x(O) = x o.
+ e-',rS, x( T ).
the state of the machine. Since per unit time u2(t ) pieces are produced and one piece produced by the machine has value x(t) at time t, u2(t)x(t ) is the gross revenue of player2 at period t. Let further q2 be the amount which must be payed by the producer to the maintenance crew for one unit of quality. Since player I has maintenance expenditures u~ to provide quality it seems to be rational that he reclaims this costs at least partially from his competitor. To include q2 qJ should be greater than this quantity: q~ = q2 +
8
with ~ > 0.
(2.10)
Note that in case of q2 = 0 player2 takes only profit resulting from the production process. For the following analysis, the sign of q2 remains unspecified (see Section6, where q2 may also be negative). Thus, the objective funtional of the producer is the present value of the net revenue stream given by J2 = for-r~t[U2( t )X( t ) -- q2x( t )] dt
+e-'.rS2x(T).
(2.11)
The aim of the players is to maximize (2.9) and (2.11), J ! and J2, respectively, subject to the dynamic restriction (2.7) and the nonnegativity constraints ui~> '
u2~>0.
(2.12)
(2.8)
is assumed to be given. The aim of player I is to guarantee a maximal quality of the machine taking into consideration the maintenance expenditures. We assume a linear assessment of quality, denoting with q~ > 0 the utility of player I per quality unit. The performance index of player I is given by
J!--foTe-r't[qlx(t)--Ul(t)]
167
dt
(2.9)
r i is the nonnegative discount rate of player i (i = 1,2). The game is played on the fixed time interval [0,T]. For player i (i= 1,2), S, is the salvage value of one unit of machine quality at the end of the game. The output produced by player 2 depends on
Note that player I controls the maintenance expenditures u l(t), whereas the producer maximizes •/2 with respect to the production rate u2(t). The decision dilemma arises because the production intensity together with the machine quality yields an instantaneous gain but causes on the other hand a reduction in quality. This results in a two-person nonzero-sum differential game with x(t) as state variable and the rate of expenditures for preventive maintenance and production rate as instruments of player I and 2, respectively. Since uj is absent in J, for j ~ i, the competitor's decision has no direct influence on the player's utility functional. As mentioned above, the particular structure of the state equation and of the integrands in the objective functionals guarantees the possibility for interesting qualitative insights into the behavior of the optimal solutions.
168
G. Feichtinger / Nash solution of maintenance-production differential game
3. Differential equations for the non-cooperative Nash equilibrium It is well known that the solution concept of nonzero-sum differential games is ambiguous (see, e.g. Staff and Ho [15], Case [2], Ho [10], Leitmann [12]). Since the possibility seems not unrealistic that the players don't cooperate, the Nash equilibrium would be an appropriate solution concept for our maintenance game. An open-loop Nash equilibrium solution is a pair of admissible control paths {u~(t), u~(t)}, t E [0,T] such that for the two criteria J! = Ji(ui, us), .I2 = J2(ul, u2) we have
Jl( u'~, u~ ) >~Jl( Ul , ~ ),
J2(u'~,u~)>~J2(u~,u2)
(3.1)
for all admissible control trajectories u, (i = 1,2). The Nash-type of solution is secure against any attempt by one player to unilaterally alter his control since he cannot improve his performance by such a change. Candidates for open-loop Nash-solutions are produced by the following necessary conditions formulated by means of the Hamiltonians of both players. Denote by
H'=q,x-ut+X,[h(u,)-k(u2)x ]
(3.2)
and
H2=-q2x+u2x+X2[h(u,)-k(u2)x]
(3.3)
the current-value Hamiltonian of player I and 2, respectively. The current-value adjoint variable X,(t) measures the sensitivity of profits to a small perturbation in the state. Thus the i th player imputes a shadow price to the quality of environment. X,(t) satisfies the adjoint equation X,=r,h,Hx'. For i = 1,2 we obtain
X, = Xi[rl + k ( u 2 ) ] - q , ,
(3.4)
The transversality conditions are given by 2k,(T)=S,,
i = 1,2.
(3.6)
According to the maximum principle the optimal control of player I maximizes his Hamiltonian at each instant. Under the additional assumption lim h'(u,)= oo, ul~0
(3.7)
u t - 0 may be excluded as suboptimal, and the first order condition for a maximum, H~, = 0, implies that X, = [h'(u,)] -I
(3.8)
The shadow price of machine quality imputed by the governmental agency is simply reciprocal to the marginal effectiveness of expenditures for maintenance. Analogous, the first order condition for an interior maximum of H 2,//,2, = 0, may be applied under the assumption k'(0) = 0
(3.9)
yielding the necessary condition ~2 = [k'(u2)] -!
(3.10)
for the producer. Note that the shadow prices for both players are nonnegative. To eliminate the adjoint variables, we differentiate (3.8) and (3.10) with respect to time and obtain h"
X =-h,2a I,
k"
X2= k,2a .
(3.11)
Substituting (3.11) into the adjoint equation (3.4) and (3.5), respectively, the following system of nonlinear differential equations in the control variables is obtained:
-hh,~((ul)ul) [qlh'(u,)-k(u2)-rl] ,
(3.12)
k'(u,) Ct2- k,,(u2) [(u2- q2)k'(u2) - k(u2)- r2]•
(3.5) Since we restrict our analysis to open-loop Nash solutions (given ~ functions of time only and not of the state), troublesome cross-effect terms in the adjoint equations are absent. It is the special shape of the Hamiltonians which makes the following analysis possible.
Candidates for open-loop Nash-optimal solution trajectories { u~'(t), u~(t) } must satisfy this nonlinear system of differential equations. The possibility for the derivation of a system of differential equations for the 'optimal' control variables de-
169
G. Feichtinger / Nash solution of maintenance-productwn differenttai game
pends essentially on the linearity of the problem in the state variable as well as on the special structure of the Hamiltonians. The terminal conditions for system (3.12), (3.13) are obtained from the transversality conditions (3.6) and the maximum principles (3.8), (3.9): h'(ul(T))=S?',
k'(u2(T))=Sf'.
(3.14)
Because of h ' ( u l ) > O , k ' ( u 2 ) > 0 for u 2 > 0 and conditions (3.7), (3.9), (3.14) is uniquely solvable for each pair (S t, S 2) with S, I> 0 (i = 1,2) provided we assume additionally
Proof. We have .>
* ' ( u 2) : ( u 2-- q 2 ) k " ( u 2 ) =0
2>
for u 2 ~ q~, (4.2)
and according to (2.6b) 0(0) = --r 2 <~0, lim O ( u 2 ) = e~.
MI --,, O0
lim k ' ( u 2 ) = ~ .
(4.4)
! / 2 ---~ OO
From (4.2), (4.3), and (4.4) the statements of the iemma follow. Note that a positive solution /~2 of (4. l) satisfies
t~2 > q2" lim h ' ( u l ) = 0 ,
(4.3)
(4.5)
Thus, the isocline
1#2 ---~ OO
Solutions {ul(T ), u2(t)} of (3.14) are the terminal values of the optimal control of the players. Note that u , ( T ) does not depend on uj(T) for j # i and i = 1,2. Moreover, since (3.13) does not contain u I, the optimal policy of player 2 does not depend on the behavior of his competitor. On the other hand, the optimal Nash-solution of player ! is not independent of the control of player2. Thus, player 1 chooses its non-cooperative equilibrium with regard to the producer's policy, whereas player2 produces without consideration of the efforts of the maintenance department. Since the sufficient conditions of Leitmann and Stalford [17] are satisfied the candidates for optimal solutions provided by (3.12), (3.13) are Nash equilibria. This follows also from the concavity in x of the maximized Hamiltonians.
ti 2 = 0,~, # ( u 2 ) = 0
(4.6)
of system (3.12), (3.13) is horizontal and crosse.,, the u2-axis in the point (0, t~2) where fi2 is the maximal nonnegative solution of (4. I). Lemma 2. The isocline ft I = 0 ,~ qth'( u,) - k( u2) - r I = 0
(4.7)
is downward sloping and approaches" the ordinate axis asymptotically. I f r I > 0 , it intersects the abscissa in the point ( ul,O), where ft I t s positive and provided by h ' ( u l ) = r,q; ~
(4.8)
In case r I = O, the u i-axis is an asymptote for
t~, = 0 .
Proof. According to the implicit function theorem 4. Phase diagram analysis for optimal control paths To derive structural insights into the Nash-optimal solution trajectories, we calculate the equilibrium point of the system (3.12), (3.13). First, we prove the following simple results.
we have
du21 h'(ul) < 0. dul .~,:0 = qi k'(u2"-----'~ The further statements of Lemma 2 follow in'.. mediately from (2.5b), (2.6a).
Lemma 1. There always exists a solution of the This yields the following result:
equation
(4.1)
Proposition. In the cases r2 > O, and r2 - O , q2 > 0
for nonnegative values of u 2. For r2 > 0 this solution, say u2, is positive and unique. For r 2 = 0 and q2>0, (4.1) is solved by u 2 = 0
there exists a unique equilibrium (ul, f~2) of system (3.12), (3.13) in the interior of the first quadrant of the (u l, u2)-plane being an instable node. For r2 = O, q2 ~<0 there is on& a stationary point at the ul-axis (in case r I > 0 ) or no finite equilibrium (in case r i = 0).
dP(u2)=(u2-q2)k'(u2)-k(u2)-r2=O
and u 2 = u2 > q2. For r2 = 0 and q2 ~ O, the unique solution is zero.
170
G. Feichtinger / Nash solution of mamtenance- productwn differential game
Proof. Let u 2 be the maximal nonnegative solution of (4.6) (see Lemma 1). Substituting u 2 = 3 2 in (4.7) we obtain
h'( u, ) = [ k(32) + ri] q~-'.
I
u2(T)
'
(4.9)
According to Lemma 2 there exists a unique stationary point (3 I, 32) of (3.12), (3.13) determined as crossing point of the isoclines (4.7) and (4.6) except in the case r 2 = 0, q2 <~0, rt = 0, where the isoclines approach asymptotically. For Q > 0 and r 2 = 0 , q 2 > 0 , (3~, 32 ) is in the interior of the first quadrant of the phase plane. For r 2 = 0, q2 <~0 but r~ > 0 the equilibrium lies at the abscissa. In case r t > 0, r 2 = 0, q2 > 0 two equilibria exist, namely (a I, 3 2) and (a t, 0). To determine the stability properties of the stationary point (3t, 32) belonging to the interior of the first quadrant we calculate a!ii
u2T ,,0to 01,,0 I I
x\
02
b
,
(0'01
Ulff )
~Ul lul:O
Ot~ZJ = ( u 2 -- q 2 ) k ' ( u , ) . au,~ I u2=O
(4.10)
Because of (4.10) and (4.3), the Jacobian determinant of system (3.12), (3.13) evaluated in the equilibrium (3~, 32) is positive:
a(31,3,)=
3u~
Ou2
0
3u---'-~2~a,.c,2)
aa2
=qlh'(a,)(a2-q2)k'(32)>O. (4.11) Thus, (3 I, 32) is an unstable node. As mentioned before, the equilibrium in the interior is unique. Remark. In case r i > 0, r2 = 0, q2 > 0 the determinant of the Jacobian matrix evaluated in the second equilibrium (3~, 0) vanishes:
A(a,, 0) = 0. Assuming power functions for the efficiency functions,
h(ui)=u' ~ with0
withfl>l,
01
, ~1l 02"0 ~Ul
Fig. !. Phase diagram of the non-cooperative Nash solutions.
there exist additional isoclines, namely /~1 -- UI "-" 0,
= qlh'(u I ),
--
~2"-- U 2 : 0
and thus, in the general case, three additional stationary points. It can be shown, that (0, 0) is a stable node, whereas (0, 32 ) and (31, 0) are saddle points. However, since the qualitative behavior of the Nash-solution is not influenced by these boundary equilibria, we analyze the general case of concave and convex efficiency functions. The first quadrant of the (ut, u2)-phase plane is subdivided in four regions ( I - I V ) by the isoclines (4.7), (4.6). According to (4.11) al! trajectories move away from the unstable equilibrium (31, 32 ). The behavior of the Nash-equilibrium solution is different in the four regions: whereas G ( t ) > 0 in region III for i = 1,2, both of the u,(t) decrease monotonically in region IV. In region 1 we have t i ~ < 0 but ~ 2 > 0 , and in region II ~ > 0 , but ~2<0. Using these results the (u~, u 2)-phase plane is sketched in Fig. 1.
5. Conclusions According to (3.14) the terminal values of u,(T) are provided by S, for i = 1,2. It seems natural to consider the case St = c¢, S 2 = 0 in more detail. This would yield Ul(T) = u2(T ) = oo, a terminal point which could not be reached in finite time T. However, for S! very
G. Feichtinger / Nash solution of maintenance- productwn differential game
large (but finite) and $2 very small (but greater than zero) both u,(T) are very large (but finite) with ul(T) > ul and u2(T) > u 2. The solution path lies in region Ill (see Fig. 1) making economic sense. The Nash-solution may be determined by backward integration. The length of the trajectory depends on the duration of the differential game, T. Thus, in the case where the maintenance crew is interested in a high final quality whereas the producer doesn't evaluate the final state of the machine positively, both the nash-optimal maintenance expenditures and production rates increase monotonically: u~(t)>0,
u~(t)>0
for all t E [0, T].
In case S, =c,, i= !,2, c , > 0 and sufficiently small, ul(T) is very small and u2(T) very large. Thus, the non-cooperative Nash-solution lies in region I, i.e. the u~(t) decreases monotonically whereas u$(t) increases. This trajectory is shown off in Fig. 1. Some other possible optimal control paths depending on the values S, of both players are illustrated in Fig. 1. If both players impute large values for S,, u~(t) increases but u~(t) decreases (region II). For small S~ but $2 large enough both controls decrease monotonically. This optimal behavior makes economic sense. Note that the optimal production rate is always monotonic: increasing for small $2 and decreasing for large $2. For intermediate values of $1 the rate of maintenance expenditures u'~(t) over time shows a nonmonotonic development provided T is sufficiently large. For large $2, u'~(t) increases initially until the maximal value is reached at ~m = 0 and decreases thereafter. For small $2 the decreasing part of the u~ trajectory precedes the increasing one. Refering to Hartl's optimal control model where the two control instruments are at disposal to one decision-maker (Hartl [8,9], see also Feichtinger [6]), it would be an interesting task to compare the optimal policies in the differential game situation with those for the optimal control model. For q2 = 0, r i = r2 = r and S I = S 2 " - S this is an easy excercise. Hartl [8] proves that the optimal maintenance rate decreases monotonically whereas the optimal production rate is increasing provided that the value S of one unit of the salvage value of the ending state x(T) is small enough. The result
! 71
discussed above shows that the same type of behavior is also Nash-optimal in the game situation.
6. An interpretation as pollution game
The differential game analyzed so far may also be interpreted in another way as follows. The competitors are a governmental agency for cleaning the environment (player 1) and a firm polluting the environment by producing an output (player 2). The state variable x(t) is now interpreted as quality of the environment acting a~ production factor, e.g. clean water of a lake, (cf. also Sethi [14]). This means, that the poorer the quality of environment, the lower are the production profits). Let ul(t) be the rate of expenditures, in dollars, in period t for pollution control, and u2(t ) the rate of production. Pollution control u~ improves the quality of the lake with effectiveness h( u~ ), whereas production with intensity u 2 reduces the quality by rate k(u2)x. (2.7) means that the effectiveness of a dollar spent on pollution control does not depend on the existing quality. Furthermore, the higher the quality, the more quality reducing is a given production rate, i.e. the cleaner the environment, the more decreases its quality by a given production rate. Note that the solvability of the differential game in the (u~, u2)-phase plane depends on this assumption. An example for this situation would be the maintenance of a stock of houses being exposed to pollution by sulforal oxyd. The aim of the governmental cleaning agency is to provide a high level of environment qualityl taking into consideration the expenditures for it[" cleaning. Analogous as before, a linear utilir', function of environment quality is assumed for the. agency, q~ being the utility per unit of environment quality assessed by player 1. The output produced by the firm depends on the quality of environment. For simplicity, we assume that u2(t)x(t) is the output rate pr,~duced at period t by production rate u2 and quality x. The aim of the producer is to maximize the present value of his net profit provided by (2. ! 1) with respect to the time path of the production rate. Three interpretations of the term - q 2x in (2.11 ) are now possible: - F o r q2 = 0 it is the production gain which is maximized by player 2. - F o r q 2 > 0 , q2x is an amount which must be
! 72
G. Feichtinger / Nash solution of maintenance-production differential game
payed from the firm to the cleaning agency for one unit of environmental quality. Since player I cleans up the pollution by expenditures u~ it seems plausible that he reclaims this costs. Thus, in this case (2.10) should hold. - F o r q: < O, - q 2 x may be interpreted as utility imputed to the quality by the producing firm, or, more adequately, as reward of the firm for sparing the environment. According to (2.10) 6 > - q 2 should hold to guarantee q~ > 0.
References [1] S.R. Arora and P.T. Lele, A note on optimal maintenance policy and sale date of a machine, Management Sci. 17 (1970) 170-173. [2] J.H. Case. Economics and the Competitive Process INew York University Press, New York, 1979). [3] S. Clemhout, G. Leitmann and H.Y. Wan Jr., A differential game model of oligopoly. J. Cybernetics 3 (1973) 24-39. [4] S. Clemhout and H,Y. Wan Jr., A class of trilinear differential games, J. Optimization Theory Appi. 14 (1974) 419-424. [5] S. Clemhout and H.Y. Wan Jr., Interactive economic dynamics and differenual games, J. Optimization Theory Appl. 27 (1979) 7-30. [6] G. Feichtinger, Optimierung yon lnstandhaitungsinvestit~,.,~_~en, Produktionsintensitat und Nutzungsdauer maschineller Produktionsanlagen: Anwendungen der Kontroihheone in der lnstandhaltungs- und Produktionsplanung, in: K. Brockhoff and W. Krelle, Eds., U nternehmensplanung (Springer, Berlin, i 981 ) 213-234.
[7] O. Feichtinger and R. Hartl, Ein nichtlineares Kontroilmodell der Instandhaltung, OR Spektrum 3 (198 l) 49-58. 18] R. Hartl, Optimal control of concave economic models with two control instruments, Forschungsbericht Nr. 31 des lnstituts for Unternehmensforschung der Technischen Universit~t Wien (1980). [9] R. Hartl, Optimal maintenance and straining intensity of a machine: a nonlinear economic control problem, J. Econora. Dynamics and Control to appear. [10] Y.C. Ho, Differential games, dynamic optimization, and generalized control theory, J. Optimization Theory Appl. 6 (1970) 179-209. [!1] S. Jmgensen, A survey of some differential games in advertising, Paper presented at t~ae workshop on "Dynamics of the Firm", held in Brussels. October 30, 1980. European Institute for Advanced Studies in Management. To appear in J. Econom. Dynamic:; and Control. [12] G. Leitmann, Cooperative and Non-cooperative Many Players Differential Games (Springer, Wien, 1974). [13] G. Leitmann and W. Schmitendorf, Profit maximization through advertising: a nonzero sum differential game approach, IEEE Trans. Automatic Control 23 (1978) 645650. [14] S.P. Sethi, A linear bang-bang model of firm behavior and water quality, IEEE Trans. Automatic Control 22 (1977) 706-714, [15] A.W. Starr and Y.C. Ho, Non zero-sum differential games. J. Optimization Theory Appl. 3 (1969) 184-206. [16] G.L. Thompson, Optimal maintenance and sale data of a machine, Management Sci. 14 (1968) 543-550. [17] G. Leitmann and H. Stalford. Sufficiency for optimal strategies in Nash equilibrium games, in: A.V. Balakrishnan, Ed., Techniques of Optimization (Academic Press. New York, 1972) 279-285.