- Email: [email protected]
A dynamic game of waste management
ARTICLE IN PRESS Journal of Economic Dynamics & Control 34 (2010) 258–265
Contents lists available at ScienceDirect
Journal of Economic Dynamics & Control journal homepage: www.elsevier.com/locate/jedc
A dynamic game of waste management Steffen Jørgensen Department of Business and Economics, University of Southern Denmark, Odense, Denmark
a r t i c l e i n f o
abstract
Article history: Received 4 August 2008 Accepted 2 September 2009 Available online 19 September 2009
The paper studies a differential game of waste management (disposal). Each of three neighbouring regions is endowed with a stock of waste, but no additional waste is generated in any region and waste does not decay from natural reasons. A region’s stock of waste can be reduced only by dumping on its neighbours. The model features two externalities: a strategic externality caused by the fact that the payoff of a coalition depends on the actions of players outside the coalition, and a stock externality caused by the fixed overall amount of waste. The game has a finite time horizon and it is shown that intertemporal core-theoretic cooperation can be sustained under intuitive conditions. & 2009 Elsevier B.V. All rights reserved.
JEL classification: C71 C73 Q20 Keywords: Differential games Core-theoretic cooperation Waste disposal
1. Introduction This paper suggests a three-player differential game in which each player initially has a stock of waste. At any instant of time, a player may dump a fraction of its current stock on another player. Stocks are affected by the dumping activities of the players only; no additional waste is accumulated in the course of the game and stocks do not decrease due to natural decay. Therefore, the overall amount of waste remains constant throughout the game. The negative externality in the game is ‘‘shiftable’’ because a recipient has the ability to push the externality along to other players (Baumol and Oates, 1988). The performance of a player is measured by her terminal stock of waste for which the player incurs a cost. The inspiration for the game comes from a static, n-player cooperative game which appeared in Shapley and Shubik (1969). In that game, each player has a single bag of garbage which she is obliged to dump in the yard of another player. All bags are identical. The utility of having b bags dumped in one’s yard is b. A coalition of players can make monetary sidepayments or redistribute garbage within the coalition. It is readily shown that the game has an empty core if nZ3. In recent years there has been an increasing interest in dynamic coalitional games, cast in continuous or in discrete time. Within this area of research, a particular issue is intertemporal core-theoretic cooperation. Kranich et al. (2005, p. 45) characterize the idea of such cooperation as follows: ‘‘Turning back to the definition of the core, intuitively, in this context the core should capture those situations in which at each stage the grand coalition is formed, its worth is distributed among the players and no coalition has a profitable deviation’’. Predtetchinski (2007) notes that a unifying theory of dynamic cooperation is still not available and the theory of dynamic coalitional games is still under development. Kranich et al. (2005) suggest new formulations of the core in dynamic cooperative games; the formulations differ in the way coalitions may deviate from the grand coalition. One assumption here is that if a coalition deviates, players cannot return to cooperation later on. Predtetchinski (2007) notes Tel.: þ45 33312200.
E-mail address: [email protected] 0165-1889/$ - see front matter & 2009 Elsevier B.V. All rights reserved. doi:10.1016/j.jedc.2009.09.005
ARTICLE IN PRESS S. Jørgensen / Journal of Economic Dynamics & Control 34 (2010) 258–265
259
that the ‘‘strong sequential core’’ introduced in Kranich et al. (2005) essentially is the same as the more familiar notion of time consistency. Time consistency in a dynamic cooperative game means that when the game evolves along the state trajectory generated by the cooperative solution, no player wishes to deviate from the actions prescribed by that solution. Time consistency is dealt with in, e.g., Petrosjan and Zenkevich (1996), Jørgensen and Zaccour (2001, 2002), and Petrosjan (2005). The notion of time consistency applies to any type of cooperative solution (e.g., core, Shapley value and Nash bargaining solution) and has mainly been applied in two-player games and games where subcoalitions are not allowed to form. In the area of international environmental agreements on transnational pollution control, examples of applications of coalitional dynamic game theory are Germain et al. (2003) and Bre´chet et al. (2007). Germain et al. (2003) introduce a rational expectations core concept where in each period of time players consider the alternatives ‘‘cooperate’’ and ‘‘do not cooperate’’. Thus, in each period of time players reconsider their interest in continued cooperation, taking into account the current level of a pollutant. It is shown that there exist monetary transfers which induce core-theoretic cooperation in each period and hence a sequence of one-period cooperative agreements is generated. The differential game of this paper involves intertemporal core-theoretic cooperation and identifies imputation streams that removes the incentive of any coalition to deviate from the grand coalition agreement at any instant of time. The paper proceeds as follows. Section 2 presents the differential game and Section 3 identifies a unique equilibrium in the noncooperative game which is played if no cooperative agreement can be made. Section 4 defines the coalitional differential game. Sections 5 and 6 determine characteristic functions of the grand coalition and a subcoalition, respectively. Characteristic functions are major objects of interest in cooperative games, but an explicit characterization of the strategies underlying these functions is quite often missing. (In dynamic coalitional games, notable exceptions are Germain et al., 2003; Bre´chet et al., 2007). An explicit determination of strategies and characteristic functions will be provided in this paper.1 Section 7 shows that intertemporal core-theoretic cooperation can be sustained intertemporally by applying a particular cost allocation scheme. Section 8 concludes. 2. A differential game of waste disposal Let t represent real time such that t 2 ½0; T. The horizon date T is fixed and finite. Players are three neighbouring regions, 1, 2, and 3. Each region has a fixed initial stock of waste, but no additional waste is generated in any region and stocks do not decay due to natural reasons. Thus, a stock can be reduced by dumping only. Denote the stocks by real numbers xi ðtÞZ0, i 2 f1; 2; 3g. Region 1, for instance, can at any instant of time choose to dump fractions f12 ðtÞ; f13 ðtÞ 2 ½0; 1 of its stock x1 ðtÞ on Regions 2 and 3. These fractions are the control instruments of Region 1. The stocks evolve according to the differential equations x_ 1 ðtÞ ¼ ½f12 ðtÞ þ f13 ðtÞx1 ðtÞ þ f21 ðtÞx2 ðtÞ þ f31 ðtÞx3 ðtÞ;
x10 40
x_ 2 ðtÞ ¼ ½f21 ðtÞ þ f23 ðtÞx2 ðtÞ þ f12 ðtÞx1 ðtÞ þ f32 ðtÞx3 ðtÞ;
x20 40
x_ 3 ðtÞ ¼ ½f31 ðtÞ þ f32 ðtÞx3 ðtÞ þ f13 ðtÞx1 ðtÞ þ f23 ðtÞx2 ðtÞ;
x30 40
ð1Þ
in which the initial stocks xi0 are given. A term like ½f12 þ f13 x1 can be viewed as a ‘‘yield function’’ and the assumption in (1) is that the ‘‘yield’’ per unit of effort (f12 þ f13 ) is directly proportional to the stock level x1 . Adding the equations in (1) shows that x_ 1 þ x_ 2 þ x_ 3 ¼ 0, that is, the total stock of waste in the regions is constant over time. Hence x1 ðtÞ þ x2 ðtÞ þ x3 ðtÞ ¼ x10 þ x20 þ x30 for all t. The total volume of waste is x10 þ x20 þ x30 and will be denoted by x. The dynamics in (1) are impractical. The main reason is the number of control variables, leading to a myriad of scenarios to be considered. To illustrate, even if the six controls take only their extreme values (zero or one) we end up with 64 different cases. We make the assumption that dumping is directed, in the sense that Region 1 can dump on Region 2 only, which can dump on Region 3 only, which can dump on Region 1 only. The implication of the assumption is that control variables f21 ; f32 and f13 can be omitted. Controls then are f12 9f1 ; f23 9f2 ; f31 9f3 . The assumption is helpful, but limits the strategic options. Suppose, for example, that Regions 1 and 2 form a coalition. Region 2 can dump on the left-out player (Region 3), but cannot redistribute waste to Region 1. The latter may distribute waste to Region 2, but cannot dump on the left-out player. Since x3 ¼ x ½x1 þ x2 , only two state variables are needed. Letting x1 9x and x2 9y be those variables, the state equations are _ ¼ f1 ðtÞxðtÞ þ f3 ðtÞ½x xðtÞ yðtÞ; xðtÞ _ yðtÞ ¼ f2 ðtÞyðtÞ þ f1 ðtÞxðtÞ; y0 40
x0 40 ð2Þ
and the state space is S ¼ fðx; yÞ 2 R2 jxZ0; yZ0; x þ yrxg. 1 Zaccour (2003) studies the computation of characteristic functions for the special class of ‘‘linear-state’’ differential games. See also Dockner et al. (2000) and Jørgensen et al. (2003).
ARTICLE IN PRESS 260
S. Jørgensen / Journal of Economic Dynamics & Control 34 (2010) 258–265
A region’s payoff is measured at the horizon date T. Let ci 40, i 2 f1; 2; 3g, be the cost (or penalty) of having one unit of waste in Region i at time T. The objective functionals of the regions are the terminal payoffs J1 ðf; x0 ; y0 ; TÞ ¼ c1 xðTÞ
J2 ðf; x0 ; y0 ; TÞ ¼ c2 yðTÞ
J3 ðf; x0 ; y0 ; TÞ ¼ c3 ½x xðTÞ yðTÞ
ð3Þ
where f9ðf1 ; f2 ; f3 Þ and xðTÞ; yðTÞ; x xðTÞ yðTÞ are the terminal stocks of the three regions. Dumping in itself is costless, i.e., there are no costs directly associated with the controls f1 ; f2 ; f3 .2 Now, Petrosjan (2005) shows that in cooperative differential games with terminal payoffs, only one imputation can be time-consistent. In our game this imputation is ðc1 xG ðTÞ; c2 yG ðTÞ; c3 ½x xG ðTÞ yG ðTÞÞ
ð4Þ
where ðxG ðTÞ; yG ðTÞÞ is the optimal cooperative state at the horizon date. To be time-consistent, an imputation must be in the core of every subgame emanating from a point on the optimal cooperative state trajectory. This is a very stringent requirement which cannot possibly be satisfied by the imputation in (4). The conclusion is that terminal cost functions are not useful when one studies intertemporal core-theoretic cooperation. The terminal costs Ji ðf; x0 ; y0 ; TÞ can, however, be transformed into running costs. In (3) we have Z T Z T _ dt J2 ðf; x0 ; y0 ; TÞ ¼ c2 yðTÞ ¼ c2 y0 þ c2 _ dt J3 ðf; x0 ; y0 ; TÞ xðtÞ yðtÞ J1 ðf; x0 ; y0 ; TÞ ¼ c1 xðTÞ ¼ c1 x0 þ c1 0
¼ c3 ½x xðTÞ yðTÞ ¼ c3 ½x x0 y0 c3
0
Z
T
_ þ yðtÞ _ ½xðtÞ dt 0
and we can use the running costs J~ i ðf; x0 ; y0 ; TÞ given by Z T Z _ dt; J~ 2 ðf; x0 ; y0 ; TÞ ¼ c2 xðtÞ J~ 1 ðf; x0 ; y0 ; TÞ ¼ c1 0
T
_ dt yðtÞ
0
J~ 3 ðf; x0 ; y0 ; TÞ ¼ c3
Z
T
_ þ yðtÞ _ ½xðtÞ dt 0
Since c1 x0 , c2 y0 , and c3 ½x x0 y0 are constants, minimization of J~ i is equivalent to minimization of Ji . 3. Noncooperative differential game If no cooperative agreement can be reached at time zero, a noncooperative game is played on the time interval ½0; T. In this game, the value function ViNC ðt; x; yÞ of Region i is given by ViNC ðt; x; yÞ ¼ min fJ~ i ðf; x; y; T tÞg; fi ðsÞ2½0;1
i 2 f1; 2; 3g
ð5Þ
s2½t;T
subject to (x; yÞ 2 S and the dynamics in (2). The superscript NC refers to ‘‘noncooperation’’. On the right-hand side of (5) we have Z T Z T Z T _ þ yðsÞ _ _ ds; J~ 2 ðf; x; y; T tÞ ¼ c2 _ ds J~ 3 ðf; x; y; T tÞ ¼ c3 ½xðsÞ ds ð6Þ xðsÞ yðsÞ J~ 1 ðf; x; y; T tÞ ¼ c1 t
t
t
NC
NC
NC
Regions use Markovian strategies fi ðt; x; yÞ which must generate control paths fi ðÞ such that fi ðtÞ 2 ½0; 1. For the game that starts out in position ðt; x; yÞ we have the following intuitive result. Proposition 1. The noncooperative game on the time interval ½t; T has for any tZ0 and (x; yÞ 2 S a unique Markov perfect equilibrium and each region has the dominant strategy always to dump at the maximal rate. NC
NC
Proof. When determining its strategy, Region i takes the strategies fj ðt; x; yÞ and fk ðt; x; yÞ of Regions j; kai as given. It NC suffices to consider Region 1. Given the strategies of Regions 2 and 3, Region 1 must choose a strategy f1 ðt; x; yÞ which NC ~ generates a control path f1 ðÞ that minimizes the objective J 1 ðf; x; y; T tÞ. No matter the value of the cost parameter c1 and the strategies of Regions 2 and 3, it pays for Region 1 to reduce its stock as much as possible at any instant of time. NC Thus, Region 1 has a dominating strategy of maximal dumping, generating the control path f1 ðsÞ ¼ 1 for s 2 ½t; T. The argument is valid for any tZ0 and the equilibrium is perfect because it obtains independently of the value of the state ðx; yÞ. & NC
With controls fi ðtÞ ¼ 1, i 2 f1; 2; 3g, the value functions are for t 2 ½0; T pffiffiffi c1 c1 V1NC ðt; x; yÞ ¼ ð3x xÞ þ eð3=2ÞðtTÞ ð½3x xcossðtÞ þ 3½x þ 2y xsinsðtÞÞ 3 3 2
Control-dependent dumping costs can be introduced, with the expected effect of discouraging high dumping rates.
ARTICLE IN PRESS S. Jørgensen / Journal of Economic Dynamics & Control 34 (2010) 258–265
V2NC ðt; x; yÞ ¼
261
pffiffiffi c2 c2 ð3y xÞ þ eð3=2ÞðtTÞ ð½3y xcossðtÞ þ 3½x 2x ysinsðtÞÞ 3 3
pffiffiffi c3 c3 V3NC ðt; x; yÞ ¼ ð2x 3x 3yÞ þ eð3=2ÞðtTÞ ð½2x 3x 3ycossðtÞ þ 3½x ysinsðtÞÞ 3 3 pffiffiffi in which sðtÞ ¼ 3ðt TÞ=2.
ð7Þ
4. Cooperative differential game Regions negotiate at time zero to try to reach an agreement on how to play the cooperative game on the time interval ½0; T. An agreement includes the choice of strategies and a rule to share the joint cost at any instant of time during the play of the game. If no agreement is reached, the noncooperative game is played for t 2 ½0; T and there is no possibility of renegotiation. The set of players is N ¼ f1; 2; 3g and KDN is a coalition. The value function of a coalition is a characteristic function in the sense of cooperative game theory. The value function of the grand coalition is ( ) X V G ðt; x; yÞ ¼ min J~ ðf; x; y; T tÞ fi ðsÞ2½0;1;i2N s2½t;T
i
i2N
in which J~ i ðf; x; y; T tÞ is given by (6). Denote by ðxG ðÞ; yG ðÞÞ the state trajectory generated by the optimal control paths employed by the grand coalition. To define the value function (characteristic function) of a subcoalition K N we must make clear what happens if K deviates. We follow the definition used in, e.g., Kranich et al. (2005) and Petrosjan (2005); coalitions are free to deviate at any instant of time but, following a deviation, players cannot return to cooperation.3 A deviating coalition determines its strategy so as to minimize its joint cost. In our set-up, a deviation can occur if A. A two-region coalition deviates, leaving one region out. B. A one-region coalition deviates, leaving two regions out. It also needs to be settled what will be the behaviour of left-out players. Various assumptions are available here. Two of them are: 1. A deviating coalition and the left-out players play a noncooperative game. The coalition plays its best response to the individual best responses of left-out players. The assumption is called a ‘‘partial agreement Nash equilibrium with respect to a coalition’’ or ‘‘the g- assumption’’. The latter originates from the ‘‘g- characteristic function’’, cf. Chander and Tulkens (1995, 1997). 2. If a coalition deviates, left-out players fall back to their noncooperative equilibrium strategies. The idea is that if the grand coalition does not form, left-out players choose the same strategies whether or not a smaller coalition forms. The idea originates from the ‘‘a- characteristic function’’ of Von Neumann and Morgenstern. Combining the two classifications yields four possible situations: A1, B1, A2, and B2 for which we observe the following. Consider A1 and suppose that coalition K ¼ f1; 3g forms. The coalition and the left-out Region 2 play a noncooperative game. In this game, no matter the strategy choice of K, Region 2 has a unique best response f2 1. But f2 1 is the noncooperative equilibrium strategy of Region 2 in situation A2; hence situations A1 and A2 are equivalent. Situations B1 and B2 are equivalent because in both cases a three-player noncooperative game is played. Proposition 1 has shown that this game has a unique equilibrium in which all players dump maximally for all t. Therefore, what remains is to determine the characteristic functions of two-player coalitions, playing against a single region that uses its noncooperative equilibrium strategy. We focus on coalition K ¼ f1; 3g playing against left-out Region 2.4 The value function (characteristic function) of coalition K ¼ f1; 3g is defined by ( ) X 13 ~ J ðf ; f ; f ; x; y; T tÞ V ðt; x; yÞ ¼ min fi ðsÞ2½0;1;i2K s2½t;T
i
1
2
3
i2K
in which f 2 1 is the noncooperative equilibrium strategy of Region 2. Suppose that the grand coalition has existed during the time interval ½0; t and consider the subgame of duration T t that starts out at time t in the point ðxG ðtÞ; yG ðtÞÞ9ðx; yÞ on the grand coalition’s optimal state trajectory. Let xi ðt; x; yÞ represent the share of the grand coalition’s cost-to-go, V G ðt; x; yÞ, which must be paid by Region i 2 N during the (remaining) time interval ½t; T. 3 Germain et al. (2003) advocate an alternative approach in which players in each period of time consider the alternatives ‘‘cooperation’’ and ‘‘noncooperation’’. 4 Results for the two other subcoalitions are similar due to the symmetric structure of the game.
ARTICLE IN PRESS 262
S. Jørgensen / Journal of Economic Dynamics & Control 34 (2010) 258–265
A vector x ¼ ðx1 ; x2 ; x3 Þ, where xi ¼ xi ðt; x; yÞ, is an imputation in the subgame on ½t; T if X xi ðt; x; yÞ ¼ V G ðt; x; yÞ
ð8Þ
i2N
An imputation x is in the core of the subgame if (i) the imputation satisfies individual rationality
xi ðt; x; yÞrViNC ðt; x; yÞ 8i 2 N
ð9Þ
where the right-hand side is given by (7) and represents the equilibrium cost-to-go paid by Region i in a noncooperative game starting out in position ðt; x; yÞ, and if (ii) the imputation satisfies coalitional rationality X xi ðt; x; yÞrV 13 ðt; x; yÞ ð10Þ i2K
To see if imputations can be found that are in the core in every subgame starting out in a point on the cooperative optimal state trajectory, we need to determine the value functions occurring on the right-hand sides of (8) and (10), respectively. This task is accomplished in Sections 5 and 6. 5. Grand coalition The regions solve for t 2 ½0; T an optimal control problem having the value function Z T Z _ ds þ ðc2 c3 Þ V G ðt; x; yÞ ¼ ðc1 c3 Þx þ ðc2 c3 Þy þ c3 x þ min ðc1 c3 Þ xðsÞ ji ðsÞ2½0;1;i2f1;2;3g s2½t;T
t
T
_ ds yðsÞ
t
where x_ and y_ are given by (2). When cost parameters c1 ; c2 ; c3 are not equal, the cooperative subgame on ½t; T is essential, that is, V G ðt; x; yÞo
3 X
ViNC ðt; x; yÞ
i¼1
This follows from the fact that in a cooperative solution, as we shall show and which is intuitive, the region with the largest cost does not dump. In the noncooperative game, all regions are dumping. Since the overall stock of waste remains the same, the total cost in the cooperative game must be lower than that in the noncooperative game.5 The Hamilton–Jacobi–Bellman (HJB) equation for the value function V G is @V G @V G @V G ¼ min c1 c3 þ x_ þ c2 c3 þ y_ fi ðsÞ2½0;1;i2f1;2;3g @t @x @y s2½t;T and has the boundary condition V G ðT; x; yÞ ¼ 0 8ðx; yÞ 2 S. Performing the minimizations on the right-hand side of the HJB equation yields, if x40, y40, x þ yox, 8 9 8 9 > >4> < ¼0 > = @V G < = @V G ¼ c1 þ if f1 2 ð0; 1Þ c2 þ ð11Þ > > @y > @x : ; : > ; ¼1 o 8 9 >o> @V G < = ¼ c3 c2 þ @y > ; : > 4 8 9 >4> @V G < = ¼ c3 c1 þ @x > ; : > o
9 8 > = < ¼0 > if f2 2 ð0; 1Þ > > ; : ¼1 9 8 > = < ¼0 > if f3 2 ð0; 1Þ > > ; : ¼1
ð12Þ
ð13Þ
The optimality conditions in (11)–(13) provide a set of candidate triples ðf1 ; f2 ; f3 Þ, henceforth referred to as paths. We focus on situations where control variables fi are either zero or one.6 The following proposition characterizes the eight remaining paths.7 5 If c1 ¼ c2 ¼ c3 ¼ c, the cost of the grand coalition equals cx. In the noncooperative game, the total equilibrium cost also equals cx and hence the cooperative subgame is inessential if cost parameters are equal. 6 It may occur that fi ðtÞ 2 ð0; 1Þ over a nonzero interval of time, in which case we have a singular path. It can be shown that singular paths do not occur, neither in the grand coalition’s optimization problem nor in games with a deviating two-player coalition (Section 6). The proofs are available from the author upon request. 7 The proof of the proposition is available from the author upon request.
ARTICLE IN PRESS S. Jørgensen / Journal of Economic Dynamics & Control 34 (2010) 258–265
263
Proposition 2. Path 1 ðf1 ¼ f2 ¼ f3 ¼ 0Þ for all t is suboptimal. Path 8 ðf1 ¼ f2 ¼ f3 ¼ 1Þ for all t is suboptimal. Path 2 ðf1 ¼ f3 ¼ 0; f2 ¼ 1Þ, Path 3 ðf1 ¼ f2 ¼ 0; f3 ¼ 1Þ, and Path 4 ðf2 ¼ f3 ¼ 0; f1 ¼ 1Þ, can be optimal on terminal intervals of time ½tj ; T, j 2 f2; 3; 4g, only. If c1 4c2 4c3 , Path 5 (f3 ¼ 0; f1 ¼ f2 ¼ 1) is optimal for all t. If c3 4c1 4c2 , Path 6 (f2 ¼ 0; f1 ¼ f3 ¼ 1) is optimal for all t. If c2 4c3 4c1 , Path 7 (f1 ¼ 0; f2 ¼ f3 ¼ 1) is optimal for all t. Path 1 is suboptimal because it pays to redistribute waste within the grand coalition to exploit the fact that unit costs ci are unequal. Path 8 is suboptimal because the region with the smallest unit cost should refrain from dumping.8 Paths 2–4 are discarded because we focus on situations in which the grand coalition, at least potentially, exists for all t. Three paths remain. To fix ideas assume c1 4c2 4c3 Then the only solution is Path 5, f1 ¼ f2 ¼ 1; f3 ¼ 0. The intuition of this path is clear: Region 1 has a larger cost than Region 2 and dumps maximally on that region. Region 2 has a larger cost than Region 3 and dumps maximally on that region. Region 3 has the smallest cost and does not dump. 6. Coalition K ¼ f1; 3g Coalition K ¼ f1; 3g has the value function V 13 ðt; x; yÞ ¼ ðc1 c3 Þx þ c3 ½x y þ
min
ji ðsÞ2½0;1;i2K
Z ðc1 c3 Þ
s2½t;T
T
t
_ ds c3 xðsÞ
Z
T
_ ds yðsÞ
t
Region 2 uses its noncooperative equilibrium strategy f 2 1 which the coalition takes as given. The HJB equation for the coalition’s value function V 13 is 13 @V 13 @V 13 @V ¼ min c3 y_ c1 c3 þ x_ þ f1 ;f3 2½0;1;i2K @t @x @y s2½t;T and has the boundary condition V 13 ðT; x; yÞ ¼ 0 8ðx; yÞ 2 S. Performing the minimizations on the right-hand side of the HJB equation yields, if x40; x x y40, 8 9 8 9 8 9 8 9 > > >4> < ¼0 > = @V 13 > < ¼0 > = <4> = @V 13 < = @V 13 ¼ c1 þ ¼ c3 c1 if f3 2 ð0; 1Þ if f1 2 ð0; 1Þ > > > > > > > > @y : ; @x @x : ; : ; : ; o o ¼1 ¼1 There are four candidate paths (numbered as those in the grand coalition). The results for coalition K ¼ f1; 3g are summarized in the following proposition.9 Proposition 3. Path 2 (f1 ¼ f3 ¼ 0Þ is suboptimal. Path 8 (f1 ¼ f3 ¼ 1Þ. It holds that V 13 ¼ V1NC þ V3NC . Path 5 (f1 ¼ 1; f3 ¼ 0) is optimal for t 2 ½t5ð13Þ ; T if c1 4c3 . Path 7 (f1 ¼ 0; f3 ¼ 1) is optimal for t 2 ½0; t7ð13Þ if c1 oc3 . Path 2 is suboptimal because when the left-out Region 2 dumps at the maximal rate on the coalition, it is not a best response of the coalition to refrain from dumping. For Path 8 it is expected that the coalition’s value function V 13 is the sum of the noncooperative game values V1NC and V3NC : when all regions dump maximally, we obtain the outcome of the noncooperative game. For Path 7 to be optimal it is required that c1 oc3 (to satisfy necessary optimality conditions). But the requirement cannot be satisfied when we have assumed c1 4c2 4c3 . The only candidate left is Path 5 which is optimal on the terminal interval ½t5ð13Þ ; T and hence the coalition may deviate at time t5ð13Þ ¼ T ðc1 c3 Þ=c3 , or later. 7. Core-theoretic cost allocation To summarize, under the assumption c1 4c2 4c3 we have shown that (i) the grand coalition uses the strategy ðf1 ; f2 ; f3 Þ ¼ ð1; 1; 0Þ and (ii) coalition K ¼ f1; 3g has the option to deviate at time t5ð13Þ (or later).10 If it deviates, the coalition uses the strategy (f1 ; f3 Þ ¼ ð1; 0Þ and Region 2 plays its noncooperative strategy f 2 ¼ 1. 8 9
The result for Path 8 shows that the noncooperative equilibrium is not Pareto optimal. The proof of the proposition is available from the author upon request.
ARTICLE IN PRESS 264
S. Jørgensen / Journal of Economic Dynamics & Control 34 (2010) 258–265
Our task is to find an imputation xi ðt; x; yÞ, i 2 f1; 2; 3g, such that cooperation in the grand coalition is individually and coalitionally rational in the subgame starting out at time t in a point ðx; yÞ on the optimal cooperative state trajectory. The conditions for core-theoretic cooperation in the subgame are
xi ðt; x; yÞrViNC ðt; x; yÞ;
i 2 f1; 2; 3g
13
x1 ðt; x; yÞ þ x3 ðt; x; yÞrV ðt; x; yÞ 3 X
xi ðt; x; yÞ ¼ V G ðt; x; yÞ
ð14Þ
i¼1
In (14), the value functions ViNC ðt; x; yÞ are given by (7). The value function V G can be shown to be V G ðt; x; yÞ ¼ ½1 etT ½ðc3 c1 Þx þ ðc3 c2 Þy þ ðc2 c3 ÞxetT ðT tÞ while the value function V 13 of coalition f1; 3g, valid for t 2 ½t5ð13Þ ; T, is V 13 ðt; x; yÞ ¼ ½1 etT ½ðc3 c1 Þx þ c3 y c3 etT xðT tÞ We apply the transfer scheme used in Germain et al. (2003) and define the monetary transfer paid (or received) by Region i over the time interval [t; T by
yi ðt; x; yÞ ¼ ½ViG ðt; x; yÞ ViNC ðt; x; yÞ þ mi ðt; x; yÞ½V G ðt; x; yÞ V NC ðt; x; yÞ
ð15Þ
In (15), ViG is the cost-to-go of Region i if it pays its own cost in the grand coalition, ViNC is the cost-to-go of Region i in the P P noncooperative game, V G ¼ 3i¼1 ViG is the cost-to-go of the grand coalition, and V NC ¼ 3i¼1 ViNC is the aggregate cost-to-go in the noncooperative game. P The parameters mi satisfy mi 2 ½0; 1, 3i¼1 mi ¼ 1. If mi ¼ 0, Region i will not benefit from cooperation. If mi ¼ 1, Region i monopolizes all the benefits of cooperation. If yi 40 (yi o0), Region i pays (receives) a transfer. The budget is balanced P because 3i¼1 yi ¼ 0. The game studied in Germain et al. (2003) includes for each player a pollution damage cost, depending on the aggregate stock of pollution, and a cost of pollution abatement that depends on the player’s emission rate. The authors use the general transfer rule in (15) to induce individual rationality in any period of time. To obtain coalitional rationality in any period, a more specialized transfer scheme is introduced such that monetary transfers depend directly on emission costs and indirectly (through the m- parameters) on the damage costs. The two transfer schemes are equivalent when the mparameters are chosen in a particular way. Using (15), the cooperative cost-to-go of Region i, including monetary transfers, is given by
xi ðt; x; yÞ ¼ ViG ðt; x; yÞ þ yi ðt; x; yÞ ¼ ViNC ðt; x; yÞ þ mi ðt; x; yÞ½V G ðt; x; yÞ V NC ðt; x; yÞ
ð16Þ
Since the difference V G V NC is negative, the imputation defined by (16) guarantees individual rationality at ðt; x; yÞ. We have seen that coalition f1; 3g can deviate for t 2 ½t5ð13Þ ; T. Suppose that it deviates at time t5ð13Þ . Then its value is V 13 ðt5ð13Þ ; x; yÞ and its strategy is (f1 ; f3 Þ ¼ ð1; 0Þ. Region 2 plays the noncooperative strategy f 2 ¼ 1. The after-deviation strategy profile is (f1 ; f 2 ; f3 Þ ¼ ð1; 1; 0Þ which we note is also the profile that the grand coalition would use for t 2 ½t5ð13Þ ; T.11 When the same strategy profile is used by the grand coalition and the subcoalition after it has deviated, the total costto-go of all three regions, during the time interval ½t5ð13Þ ; T, must be the same in the grand coalition and in the game between the subcoalition and Region 2. In that game, the cost-to-go of subcoalition f1; 3g is V 13 ðt; x; yÞ while the cost-to-go of Region 2 is V 2 ðt; x; yÞ (which is the value function of Region 2 in the game with the subcoalition). Therefore the following is true: Lemma 4. Value functions V G ðt; x; yÞ; V 13 ðt; x; yÞ, and V 2 ðt; x; yÞ satisfy V 13 ðt; x; yÞ þ V 2 ðt; x; yÞ ¼ V G ðt; x; yÞ The lemma can be formally proved by using the expressions for the three value functions involved. It says that the game between the subcoalition and Region 2 is a constant-sum game. This result turns out to be useful in the presentation of the following proposition. 10 It can be shown that if c2 4c1 , coalition K ¼ f1; 2g has the option to deviate at time t7ð12Þ (or later). If c3 4c2 , coalition K ¼ f2; 3g has the option to deviate at time t6ð23Þ (or later). Hence, under the assumption c1 4c2 4c3 , coalitions f1; 2g and f2; 3g cannot deviate. The proof is available from the author. 11 One may define a deviation as a case where a subcoalition ‘‘seeks an arrangement of its own’’, in the sense that it uses strategies that are different from those its members would use if they were in the grand coalition. Under such a definition, coalition K ¼ f1; 3g cannot deviate in our game.
ARTICLE IN PRESS S. Jørgensen / Journal of Economic Dynamics & Control 34 (2010) 258–265
265
Proposition 5. To achieve coalitional rationality at position ðt; x; yÞ, tZt5ð13Þ , regions must agree to choose m1 ðt; x; yÞ and m3 ðt; x; yÞ such that
x1 ðt; x; yÞ þ x3 ðt; x; yÞrV 13 ðt; x; yÞ()
m1 ðt; x; yÞ þ m3 ðt; x; yÞZ
V1NC ðt; x; yÞ þ V3NC ðt; x; yÞ V 13 ðt; x; yÞ V NC ðt; x; yÞ V 2 ðt; x; yÞ ¼ 1 2NC V NC ðt; x; yÞ V G ðt; x; yÞ V ðt; x; yÞ V G ðt; x; yÞ
ð17Þ
It holds that V1NC þ V3NC ZV 13 due to subadditivity of the characteristic function. Furthermore, V NC 4V G because the cooperative game is essential. The second line in (17) states that Regions 1 and 3 jointly must be given at least a certain share of the grand coalition’s cost saving V NC V G . The intuition is that the more, the two regions can gain ðV 13 Þ by deviating from the grand coalition, compared to their joint disagreement cost ðV1NC þ V3NC Þ, the higher a share of the grand coalition cost saving will the coalition demand as its price for not leaving the grand coalition. This is intuitive. 8. Conclusions The model suggested in this paper is a simplified structure and should not be seen as a ‘‘realistic’’ representation of the interactions between real-life neighbouring regions. The analysis has shown that it is possible, by using an intertemporal transfer scheme, to sustain core-theoretic cooperation over time. A major factor that drives this result is the differences between the regions’ cost parameters. A modification of the dynamics would be to abandon the assumption that no additional waste is generated during the play of the game. To account for this one can add a term, say, mi ðtÞ40 on the right-hand sides of the state equations. Also natural decay of waste could be included in the model, by subtracting a decay term on the right-hand side of the dynamics. The implication of such assumptions would be that the total stock of waste no longer is constant throughout the game. The effects of such extensions remain to be seen. A dramatic implication for the analysis would be to abandon the assumption of directed dumping. This would increase the number of control variables to six, leading to a myriad of feasible paths.
Acknowledgements The paper was presented at the workshop ‘‘Differential Games in Management’’, HEC, Montreal, Canada, and at the ‘‘13th International Symposium on Dynamic Games and Applications’’, Wroc"aw, Poland. I am grateful for the comments of participants in these events. I also appreciate the constructive remarks and suggestions of an editor and three reviewers. Any remaining errors are my responsibility. References Baumol, W.J., Oates, W.E., 1988. The Theory of Environmental Policy. Cambridge University Press, Cambridge, UK. Bre´chet, T., Gerard, F., Tulkens, H., 2007. Climate coalitions: a theoretical and computational appraisal. CORE Discussion Paper 2007/3. Chander, P., Tulkens, H., 1995. A core-theoretic solution for the design of cooperative agreements on transboundary pollution. International Tax and Public Finance 2, 279–293. Chander, P., Tulkens, H., 1997. The core of an economy with multilateral environmental externalities. International Journal of Game Theory 26, 379–401. Dockner, E.J., Jørgensen, S., Van Long, N., Sorger, G., 2000. Differential Games in Economics and Management Science. Cambridge University Press, Cambridge, UK. Germain, M., Toint, P., Tulkens, H., de Zeeuw, A., 2003. Transfers to sustain dynamic core-theoretic cooperation in international stock pollutant control. Journal of Economic Dynamics & Control 28, 79–99. Jørgensen, S., Zaccour, G., 2001. Time consistent side payments in a dynamic game of downstream pollution. Journal of Economic Dynamics & Control 25, 1973–1987. Jørgensen, S., Zaccour, G., 2002. Time consistency in cooperative differential games. In: Zaccour, G. (Ed.), Decision & Control in Management Science: Essays in Honor of Alain Haurie. Kluwer Academic Publishers Boston, MA, USA, pp. 349–366. Jørgensen, S., Martı´n-Herra´n, G., Zaccour, 2003. Agreeability and time consistency in linear-state differential games. Journal of Optimization Theory and Applications 119, 49–63. Kranich, L., Perea, A., Peters, H., 2005. Core concepts for dynamic TU games. International Game Theory Review 7, 43–62. Petrosjan, L.A., Zenkevich, N.A., 1996. Game Theory. World Scientific, Singapore. Petrosjan, L.A., 2005. Cooperative differential games. In: Novak, A.S., Szajowski, K. (Eds.), Advances in Dynamic Games: Applications to Economics, Finance, ¨ Optimization and Stochastic Control. Annals of the International Society of Dynamic Games, 7. Birkhauser, Boston, MA, USA, pp. 183–200. Predtetchinski, A., 2007. The strong sequential core for stationary cooperative games. Games and Economic Behavior 61, 50–66. Shapley, L.S., Shubik, M., 1969. On the core of an economic system with externalities. American Economic Review 59, 678–684. Zaccour, G., 2003. Computation of characteristic function values for linear-state differential games. Journal of Optimization Theory and Applications 117, 183–194.
Contents lists available at ScienceDirect
Journal of Economic Dynamics & Control journal homepage: www.elsevier.com/locate/jedc
A dynamic game of waste management Steffen Jørgensen Department of Business and Economics, University of Southern Denmark, Odense, Denmark
a r t i c l e i n f o
abstract
Article history: Received 4 August 2008 Accepted 2 September 2009 Available online 19 September 2009
The paper studies a differential game of waste management (disposal). Each of three neighbouring regions is endowed with a stock of waste, but no additional waste is generated in any region and waste does not decay from natural reasons. A region’s stock of waste can be reduced only by dumping on its neighbours. The model features two externalities: a strategic externality caused by the fact that the payoff of a coalition depends on the actions of players outside the coalition, and a stock externality caused by the fixed overall amount of waste. The game has a finite time horizon and it is shown that intertemporal core-theoretic cooperation can be sustained under intuitive conditions. & 2009 Elsevier B.V. All rights reserved.
JEL classification: C71 C73 Q20 Keywords: Differential games Core-theoretic cooperation Waste disposal
1. Introduction This paper suggests a three-player differential game in which each player initially has a stock of waste. At any instant of time, a player may dump a fraction of its current stock on another player. Stocks are affected by the dumping activities of the players only; no additional waste is accumulated in the course of the game and stocks do not decrease due to natural decay. Therefore, the overall amount of waste remains constant throughout the game. The negative externality in the game is ‘‘shiftable’’ because a recipient has the ability to push the externality along to other players (Baumol and Oates, 1988). The performance of a player is measured by her terminal stock of waste for which the player incurs a cost. The inspiration for the game comes from a static, n-player cooperative game which appeared in Shapley and Shubik (1969). In that game, each player has a single bag of garbage which she is obliged to dump in the yard of another player. All bags are identical. The utility of having b bags dumped in one’s yard is b. A coalition of players can make monetary sidepayments or redistribute garbage within the coalition. It is readily shown that the game has an empty core if nZ3. In recent years there has been an increasing interest in dynamic coalitional games, cast in continuous or in discrete time. Within this area of research, a particular issue is intertemporal core-theoretic cooperation. Kranich et al. (2005, p. 45) characterize the idea of such cooperation as follows: ‘‘Turning back to the definition of the core, intuitively, in this context the core should capture those situations in which at each stage the grand coalition is formed, its worth is distributed among the players and no coalition has a profitable deviation’’. Predtetchinski (2007) notes that a unifying theory of dynamic cooperation is still not available and the theory of dynamic coalitional games is still under development. Kranich et al. (2005) suggest new formulations of the core in dynamic cooperative games; the formulations differ in the way coalitions may deviate from the grand coalition. One assumption here is that if a coalition deviates, players cannot return to cooperation later on. Predtetchinski (2007) notes Tel.: þ45 33312200.
E-mail address: [email protected] 0165-1889/$ - see front matter & 2009 Elsevier B.V. All rights reserved. doi:10.1016/j.jedc.2009.09.005
ARTICLE IN PRESS S. Jørgensen / Journal of Economic Dynamics & Control 34 (2010) 258–265
259
that the ‘‘strong sequential core’’ introduced in Kranich et al. (2005) essentially is the same as the more familiar notion of time consistency. Time consistency in a dynamic cooperative game means that when the game evolves along the state trajectory generated by the cooperative solution, no player wishes to deviate from the actions prescribed by that solution. Time consistency is dealt with in, e.g., Petrosjan and Zenkevich (1996), Jørgensen and Zaccour (2001, 2002), and Petrosjan (2005). The notion of time consistency applies to any type of cooperative solution (e.g., core, Shapley value and Nash bargaining solution) and has mainly been applied in two-player games and games where subcoalitions are not allowed to form. In the area of international environmental agreements on transnational pollution control, examples of applications of coalitional dynamic game theory are Germain et al. (2003) and Bre´chet et al. (2007). Germain et al. (2003) introduce a rational expectations core concept where in each period of time players consider the alternatives ‘‘cooperate’’ and ‘‘do not cooperate’’. Thus, in each period of time players reconsider their interest in continued cooperation, taking into account the current level of a pollutant. It is shown that there exist monetary transfers which induce core-theoretic cooperation in each period and hence a sequence of one-period cooperative agreements is generated. The differential game of this paper involves intertemporal core-theoretic cooperation and identifies imputation streams that removes the incentive of any coalition to deviate from the grand coalition agreement at any instant of time. The paper proceeds as follows. Section 2 presents the differential game and Section 3 identifies a unique equilibrium in the noncooperative game which is played if no cooperative agreement can be made. Section 4 defines the coalitional differential game. Sections 5 and 6 determine characteristic functions of the grand coalition and a subcoalition, respectively. Characteristic functions are major objects of interest in cooperative games, but an explicit characterization of the strategies underlying these functions is quite often missing. (In dynamic coalitional games, notable exceptions are Germain et al., 2003; Bre´chet et al., 2007). An explicit determination of strategies and characteristic functions will be provided in this paper.1 Section 7 shows that intertemporal core-theoretic cooperation can be sustained intertemporally by applying a particular cost allocation scheme. Section 8 concludes. 2. A differential game of waste disposal Let t represent real time such that t 2 ½0; T. The horizon date T is fixed and finite. Players are three neighbouring regions, 1, 2, and 3. Each region has a fixed initial stock of waste, but no additional waste is generated in any region and stocks do not decay due to natural reasons. Thus, a stock can be reduced by dumping only. Denote the stocks by real numbers xi ðtÞZ0, i 2 f1; 2; 3g. Region 1, for instance, can at any instant of time choose to dump fractions f12 ðtÞ; f13 ðtÞ 2 ½0; 1 of its stock x1 ðtÞ on Regions 2 and 3. These fractions are the control instruments of Region 1. The stocks evolve according to the differential equations x_ 1 ðtÞ ¼ ½f12 ðtÞ þ f13 ðtÞx1 ðtÞ þ f21 ðtÞx2 ðtÞ þ f31 ðtÞx3 ðtÞ;
x10 40
x_ 2 ðtÞ ¼ ½f21 ðtÞ þ f23 ðtÞx2 ðtÞ þ f12 ðtÞx1 ðtÞ þ f32 ðtÞx3 ðtÞ;
x20 40
x_ 3 ðtÞ ¼ ½f31 ðtÞ þ f32 ðtÞx3 ðtÞ þ f13 ðtÞx1 ðtÞ þ f23 ðtÞx2 ðtÞ;
x30 40
ð1Þ
in which the initial stocks xi0 are given. A term like ½f12 þ f13 x1 can be viewed as a ‘‘yield function’’ and the assumption in (1) is that the ‘‘yield’’ per unit of effort (f12 þ f13 ) is directly proportional to the stock level x1 . Adding the equations in (1) shows that x_ 1 þ x_ 2 þ x_ 3 ¼ 0, that is, the total stock of waste in the regions is constant over time. Hence x1 ðtÞ þ x2 ðtÞ þ x3 ðtÞ ¼ x10 þ x20 þ x30 for all t. The total volume of waste is x10 þ x20 þ x30 and will be denoted by x. The dynamics in (1) are impractical. The main reason is the number of control variables, leading to a myriad of scenarios to be considered. To illustrate, even if the six controls take only their extreme values (zero or one) we end up with 64 different cases. We make the assumption that dumping is directed, in the sense that Region 1 can dump on Region 2 only, which can dump on Region 3 only, which can dump on Region 1 only. The implication of the assumption is that control variables f21 ; f32 and f13 can be omitted. Controls then are f12 9f1 ; f23 9f2 ; f31 9f3 . The assumption is helpful, but limits the strategic options. Suppose, for example, that Regions 1 and 2 form a coalition. Region 2 can dump on the left-out player (Region 3), but cannot redistribute waste to Region 1. The latter may distribute waste to Region 2, but cannot dump on the left-out player. Since x3 ¼ x ½x1 þ x2 , only two state variables are needed. Letting x1 9x and x2 9y be those variables, the state equations are _ ¼ f1 ðtÞxðtÞ þ f3 ðtÞ½x xðtÞ yðtÞ; xðtÞ _ yðtÞ ¼ f2 ðtÞyðtÞ þ f1 ðtÞxðtÞ; y0 40
x0 40 ð2Þ
and the state space is S ¼ fðx; yÞ 2 R2 jxZ0; yZ0; x þ yrxg. 1 Zaccour (2003) studies the computation of characteristic functions for the special class of ‘‘linear-state’’ differential games. See also Dockner et al. (2000) and Jørgensen et al. (2003).
ARTICLE IN PRESS 260
S. Jørgensen / Journal of Economic Dynamics & Control 34 (2010) 258–265
A region’s payoff is measured at the horizon date T. Let ci 40, i 2 f1; 2; 3g, be the cost (or penalty) of having one unit of waste in Region i at time T. The objective functionals of the regions are the terminal payoffs J1 ðf; x0 ; y0 ; TÞ ¼ c1 xðTÞ
J2 ðf; x0 ; y0 ; TÞ ¼ c2 yðTÞ
J3 ðf; x0 ; y0 ; TÞ ¼ c3 ½x xðTÞ yðTÞ
ð3Þ
where f9ðf1 ; f2 ; f3 Þ and xðTÞ; yðTÞ; x xðTÞ yðTÞ are the terminal stocks of the three regions. Dumping in itself is costless, i.e., there are no costs directly associated with the controls f1 ; f2 ; f3 .2 Now, Petrosjan (2005) shows that in cooperative differential games with terminal payoffs, only one imputation can be time-consistent. In our game this imputation is ðc1 xG ðTÞ; c2 yG ðTÞ; c3 ½x xG ðTÞ yG ðTÞÞ
ð4Þ
where ðxG ðTÞ; yG ðTÞÞ is the optimal cooperative state at the horizon date. To be time-consistent, an imputation must be in the core of every subgame emanating from a point on the optimal cooperative state trajectory. This is a very stringent requirement which cannot possibly be satisfied by the imputation in (4). The conclusion is that terminal cost functions are not useful when one studies intertemporal core-theoretic cooperation. The terminal costs Ji ðf; x0 ; y0 ; TÞ can, however, be transformed into running costs. In (3) we have Z T Z T _ dt J2 ðf; x0 ; y0 ; TÞ ¼ c2 yðTÞ ¼ c2 y0 þ c2 _ dt J3 ðf; x0 ; y0 ; TÞ xðtÞ yðtÞ J1 ðf; x0 ; y0 ; TÞ ¼ c1 xðTÞ ¼ c1 x0 þ c1 0
¼ c3 ½x xðTÞ yðTÞ ¼ c3 ½x x0 y0 c3
0
Z
T
_ þ yðtÞ _ ½xðtÞ dt 0
and we can use the running costs J~ i ðf; x0 ; y0 ; TÞ given by Z T Z _ dt; J~ 2 ðf; x0 ; y0 ; TÞ ¼ c2 xðtÞ J~ 1 ðf; x0 ; y0 ; TÞ ¼ c1 0
T
_ dt yðtÞ
0
J~ 3 ðf; x0 ; y0 ; TÞ ¼ c3
Z
T
_ þ yðtÞ _ ½xðtÞ dt 0
Since c1 x0 , c2 y0 , and c3 ½x x0 y0 are constants, minimization of J~ i is equivalent to minimization of Ji . 3. Noncooperative differential game If no cooperative agreement can be reached at time zero, a noncooperative game is played on the time interval ½0; T. In this game, the value function ViNC ðt; x; yÞ of Region i is given by ViNC ðt; x; yÞ ¼ min fJ~ i ðf; x; y; T tÞg; fi ðsÞ2½0;1
i 2 f1; 2; 3g
ð5Þ
s2½t;T
subject to (x; yÞ 2 S and the dynamics in (2). The superscript NC refers to ‘‘noncooperation’’. On the right-hand side of (5) we have Z T Z T Z T _ þ yðsÞ _ _ ds; J~ 2 ðf; x; y; T tÞ ¼ c2 _ ds J~ 3 ðf; x; y; T tÞ ¼ c3 ½xðsÞ ds ð6Þ xðsÞ yðsÞ J~ 1 ðf; x; y; T tÞ ¼ c1 t
t
t
NC
NC
NC
Regions use Markovian strategies fi ðt; x; yÞ which must generate control paths fi ðÞ such that fi ðtÞ 2 ½0; 1. For the game that starts out in position ðt; x; yÞ we have the following intuitive result. Proposition 1. The noncooperative game on the time interval ½t; T has for any tZ0 and (x; yÞ 2 S a unique Markov perfect equilibrium and each region has the dominant strategy always to dump at the maximal rate. NC
NC
Proof. When determining its strategy, Region i takes the strategies fj ðt; x; yÞ and fk ðt; x; yÞ of Regions j; kai as given. It NC suffices to consider Region 1. Given the strategies of Regions 2 and 3, Region 1 must choose a strategy f1 ðt; x; yÞ which NC ~ generates a control path f1 ðÞ that minimizes the objective J 1 ðf; x; y; T tÞ. No matter the value of the cost parameter c1 and the strategies of Regions 2 and 3, it pays for Region 1 to reduce its stock as much as possible at any instant of time. NC Thus, Region 1 has a dominating strategy of maximal dumping, generating the control path f1 ðsÞ ¼ 1 for s 2 ½t; T. The argument is valid for any tZ0 and the equilibrium is perfect because it obtains independently of the value of the state ðx; yÞ. & NC
With controls fi ðtÞ ¼ 1, i 2 f1; 2; 3g, the value functions are for t 2 ½0; T pffiffiffi c1 c1 V1NC ðt; x; yÞ ¼ ð3x xÞ þ eð3=2ÞðtTÞ ð½3x xcossðtÞ þ 3½x þ 2y xsinsðtÞÞ 3 3 2
Control-dependent dumping costs can be introduced, with the expected effect of discouraging high dumping rates.
ARTICLE IN PRESS S. Jørgensen / Journal of Economic Dynamics & Control 34 (2010) 258–265
V2NC ðt; x; yÞ ¼
261
pffiffiffi c2 c2 ð3y xÞ þ eð3=2ÞðtTÞ ð½3y xcossðtÞ þ 3½x 2x ysinsðtÞÞ 3 3
pffiffiffi c3 c3 V3NC ðt; x; yÞ ¼ ð2x 3x 3yÞ þ eð3=2ÞðtTÞ ð½2x 3x 3ycossðtÞ þ 3½x ysinsðtÞÞ 3 3 pffiffiffi in which sðtÞ ¼ 3ðt TÞ=2.
ð7Þ
4. Cooperative differential game Regions negotiate at time zero to try to reach an agreement on how to play the cooperative game on the time interval ½0; T. An agreement includes the choice of strategies and a rule to share the joint cost at any instant of time during the play of the game. If no agreement is reached, the noncooperative game is played for t 2 ½0; T and there is no possibility of renegotiation. The set of players is N ¼ f1; 2; 3g and KDN is a coalition. The value function of a coalition is a characteristic function in the sense of cooperative game theory. The value function of the grand coalition is ( ) X V G ðt; x; yÞ ¼ min J~ ðf; x; y; T tÞ fi ðsÞ2½0;1;i2N s2½t;T
i
i2N
in which J~ i ðf; x; y; T tÞ is given by (6). Denote by ðxG ðÞ; yG ðÞÞ the state trajectory generated by the optimal control paths employed by the grand coalition. To define the value function (characteristic function) of a subcoalition K N we must make clear what happens if K deviates. We follow the definition used in, e.g., Kranich et al. (2005) and Petrosjan (2005); coalitions are free to deviate at any instant of time but, following a deviation, players cannot return to cooperation.3 A deviating coalition determines its strategy so as to minimize its joint cost. In our set-up, a deviation can occur if A. A two-region coalition deviates, leaving one region out. B. A one-region coalition deviates, leaving two regions out. It also needs to be settled what will be the behaviour of left-out players. Various assumptions are available here. Two of them are: 1. A deviating coalition and the left-out players play a noncooperative game. The coalition plays its best response to the individual best responses of left-out players. The assumption is called a ‘‘partial agreement Nash equilibrium with respect to a coalition’’ or ‘‘the g- assumption’’. The latter originates from the ‘‘g- characteristic function’’, cf. Chander and Tulkens (1995, 1997). 2. If a coalition deviates, left-out players fall back to their noncooperative equilibrium strategies. The idea is that if the grand coalition does not form, left-out players choose the same strategies whether or not a smaller coalition forms. The idea originates from the ‘‘a- characteristic function’’ of Von Neumann and Morgenstern. Combining the two classifications yields four possible situations: A1, B1, A2, and B2 for which we observe the following. Consider A1 and suppose that coalition K ¼ f1; 3g forms. The coalition and the left-out Region 2 play a noncooperative game. In this game, no matter the strategy choice of K, Region 2 has a unique best response f2 1. But f2 1 is the noncooperative equilibrium strategy of Region 2 in situation A2; hence situations A1 and A2 are equivalent. Situations B1 and B2 are equivalent because in both cases a three-player noncooperative game is played. Proposition 1 has shown that this game has a unique equilibrium in which all players dump maximally for all t. Therefore, what remains is to determine the characteristic functions of two-player coalitions, playing against a single region that uses its noncooperative equilibrium strategy. We focus on coalition K ¼ f1; 3g playing against left-out Region 2.4 The value function (characteristic function) of coalition K ¼ f1; 3g is defined by ( ) X 13 ~ J ðf ; f ; f ; x; y; T tÞ V ðt; x; yÞ ¼ min fi ðsÞ2½0;1;i2K s2½t;T
i
1
2
3
i2K
in which f 2 1 is the noncooperative equilibrium strategy of Region 2. Suppose that the grand coalition has existed during the time interval ½0; t and consider the subgame of duration T t that starts out at time t in the point ðxG ðtÞ; yG ðtÞÞ9ðx; yÞ on the grand coalition’s optimal state trajectory. Let xi ðt; x; yÞ represent the share of the grand coalition’s cost-to-go, V G ðt; x; yÞ, which must be paid by Region i 2 N during the (remaining) time interval ½t; T. 3 Germain et al. (2003) advocate an alternative approach in which players in each period of time consider the alternatives ‘‘cooperation’’ and ‘‘noncooperation’’. 4 Results for the two other subcoalitions are similar due to the symmetric structure of the game.
ARTICLE IN PRESS 262
S. Jørgensen / Journal of Economic Dynamics & Control 34 (2010) 258–265
A vector x ¼ ðx1 ; x2 ; x3 Þ, where xi ¼ xi ðt; x; yÞ, is an imputation in the subgame on ½t; T if X xi ðt; x; yÞ ¼ V G ðt; x; yÞ
ð8Þ
i2N
An imputation x is in the core of the subgame if (i) the imputation satisfies individual rationality
xi ðt; x; yÞrViNC ðt; x; yÞ 8i 2 N
ð9Þ
where the right-hand side is given by (7) and represents the equilibrium cost-to-go paid by Region i in a noncooperative game starting out in position ðt; x; yÞ, and if (ii) the imputation satisfies coalitional rationality X xi ðt; x; yÞrV 13 ðt; x; yÞ ð10Þ i2K
To see if imputations can be found that are in the core in every subgame starting out in a point on the cooperative optimal state trajectory, we need to determine the value functions occurring on the right-hand sides of (8) and (10), respectively. This task is accomplished in Sections 5 and 6. 5. Grand coalition The regions solve for t 2 ½0; T an optimal control problem having the value function Z T Z _ ds þ ðc2 c3 Þ V G ðt; x; yÞ ¼ ðc1 c3 Þx þ ðc2 c3 Þy þ c3 x þ min ðc1 c3 Þ xðsÞ ji ðsÞ2½0;1;i2f1;2;3g s2½t;T
t
T
_ ds yðsÞ
t
where x_ and y_ are given by (2). When cost parameters c1 ; c2 ; c3 are not equal, the cooperative subgame on ½t; T is essential, that is, V G ðt; x; yÞo
3 X
ViNC ðt; x; yÞ
i¼1
This follows from the fact that in a cooperative solution, as we shall show and which is intuitive, the region with the largest cost does not dump. In the noncooperative game, all regions are dumping. Since the overall stock of waste remains the same, the total cost in the cooperative game must be lower than that in the noncooperative game.5 The Hamilton–Jacobi–Bellman (HJB) equation for the value function V G is @V G @V G @V G ¼ min c1 c3 þ x_ þ c2 c3 þ y_ fi ðsÞ2½0;1;i2f1;2;3g @t @x @y s2½t;T and has the boundary condition V G ðT; x; yÞ ¼ 0 8ðx; yÞ 2 S. Performing the minimizations on the right-hand side of the HJB equation yields, if x40, y40, x þ yox, 8 9 8 9 > >4> < ¼0 > = @V G < = @V G ¼ c1 þ if f1 2 ð0; 1Þ c2 þ ð11Þ > > @y > @x : ; : > ; ¼1 o 8 9 >o> @V G < = ¼ c3 c2 þ @y > ; : > 4 8 9 >4> @V G < = ¼ c3 c1 þ @x > ; : > o
9 8 > = < ¼0 > if f2 2 ð0; 1Þ > > ; : ¼1 9 8 > = < ¼0 > if f3 2 ð0; 1Þ > > ; : ¼1
ð12Þ
ð13Þ
The optimality conditions in (11)–(13) provide a set of candidate triples ðf1 ; f2 ; f3 Þ, henceforth referred to as paths. We focus on situations where control variables fi are either zero or one.6 The following proposition characterizes the eight remaining paths.7 5 If c1 ¼ c2 ¼ c3 ¼ c, the cost of the grand coalition equals cx. In the noncooperative game, the total equilibrium cost also equals cx and hence the cooperative subgame is inessential if cost parameters are equal. 6 It may occur that fi ðtÞ 2 ð0; 1Þ over a nonzero interval of time, in which case we have a singular path. It can be shown that singular paths do not occur, neither in the grand coalition’s optimization problem nor in games with a deviating two-player coalition (Section 6). The proofs are available from the author upon request. 7 The proof of the proposition is available from the author upon request.
ARTICLE IN PRESS S. Jørgensen / Journal of Economic Dynamics & Control 34 (2010) 258–265
263
Proposition 2. Path 1 ðf1 ¼ f2 ¼ f3 ¼ 0Þ for all t is suboptimal. Path 8 ðf1 ¼ f2 ¼ f3 ¼ 1Þ for all t is suboptimal. Path 2 ðf1 ¼ f3 ¼ 0; f2 ¼ 1Þ, Path 3 ðf1 ¼ f2 ¼ 0; f3 ¼ 1Þ, and Path 4 ðf2 ¼ f3 ¼ 0; f1 ¼ 1Þ, can be optimal on terminal intervals of time ½tj ; T, j 2 f2; 3; 4g, only. If c1 4c2 4c3 , Path 5 (f3 ¼ 0; f1 ¼ f2 ¼ 1) is optimal for all t. If c3 4c1 4c2 , Path 6 (f2 ¼ 0; f1 ¼ f3 ¼ 1) is optimal for all t. If c2 4c3 4c1 , Path 7 (f1 ¼ 0; f2 ¼ f3 ¼ 1) is optimal for all t. Path 1 is suboptimal because it pays to redistribute waste within the grand coalition to exploit the fact that unit costs ci are unequal. Path 8 is suboptimal because the region with the smallest unit cost should refrain from dumping.8 Paths 2–4 are discarded because we focus on situations in which the grand coalition, at least potentially, exists for all t. Three paths remain. To fix ideas assume c1 4c2 4c3 Then the only solution is Path 5, f1 ¼ f2 ¼ 1; f3 ¼ 0. The intuition of this path is clear: Region 1 has a larger cost than Region 2 and dumps maximally on that region. Region 2 has a larger cost than Region 3 and dumps maximally on that region. Region 3 has the smallest cost and does not dump. 6. Coalition K ¼ f1; 3g Coalition K ¼ f1; 3g has the value function V 13 ðt; x; yÞ ¼ ðc1 c3 Þx þ c3 ½x y þ
min
ji ðsÞ2½0;1;i2K
Z ðc1 c3 Þ
s2½t;T
T
t
_ ds c3 xðsÞ
Z
T
_ ds yðsÞ
t
Region 2 uses its noncooperative equilibrium strategy f 2 1 which the coalition takes as given. The HJB equation for the coalition’s value function V 13 is 13 @V 13 @V 13 @V ¼ min c3 y_ c1 c3 þ x_ þ f1 ;f3 2½0;1;i2K @t @x @y s2½t;T and has the boundary condition V 13 ðT; x; yÞ ¼ 0 8ðx; yÞ 2 S. Performing the minimizations on the right-hand side of the HJB equation yields, if x40; x x y40, 8 9 8 9 8 9 8 9 > > >4> < ¼0 > = @V 13 > < ¼0 > = <4> = @V 13 < = @V 13 ¼ c1 þ ¼ c3 c1 if f3 2 ð0; 1Þ if f1 2 ð0; 1Þ > > > > > > > > @y : ; @x @x : ; : ; : ; o o ¼1 ¼1 There are four candidate paths (numbered as those in the grand coalition). The results for coalition K ¼ f1; 3g are summarized in the following proposition.9 Proposition 3. Path 2 (f1 ¼ f3 ¼ 0Þ is suboptimal. Path 8 (f1 ¼ f3 ¼ 1Þ. It holds that V 13 ¼ V1NC þ V3NC . Path 5 (f1 ¼ 1; f3 ¼ 0) is optimal for t 2 ½t5ð13Þ ; T if c1 4c3 . Path 7 (f1 ¼ 0; f3 ¼ 1) is optimal for t 2 ½0; t7ð13Þ if c1 oc3 . Path 2 is suboptimal because when the left-out Region 2 dumps at the maximal rate on the coalition, it is not a best response of the coalition to refrain from dumping. For Path 8 it is expected that the coalition’s value function V 13 is the sum of the noncooperative game values V1NC and V3NC : when all regions dump maximally, we obtain the outcome of the noncooperative game. For Path 7 to be optimal it is required that c1 oc3 (to satisfy necessary optimality conditions). But the requirement cannot be satisfied when we have assumed c1 4c2 4c3 . The only candidate left is Path 5 which is optimal on the terminal interval ½t5ð13Þ ; T and hence the coalition may deviate at time t5ð13Þ ¼ T ðc1 c3 Þ=c3 , or later. 7. Core-theoretic cost allocation To summarize, under the assumption c1 4c2 4c3 we have shown that (i) the grand coalition uses the strategy ðf1 ; f2 ; f3 Þ ¼ ð1; 1; 0Þ and (ii) coalition K ¼ f1; 3g has the option to deviate at time t5ð13Þ (or later).10 If it deviates, the coalition uses the strategy (f1 ; f3 Þ ¼ ð1; 0Þ and Region 2 plays its noncooperative strategy f 2 ¼ 1. 8 9
The result for Path 8 shows that the noncooperative equilibrium is not Pareto optimal. The proof of the proposition is available from the author upon request.
ARTICLE IN PRESS 264
S. Jørgensen / Journal of Economic Dynamics & Control 34 (2010) 258–265
Our task is to find an imputation xi ðt; x; yÞ, i 2 f1; 2; 3g, such that cooperation in the grand coalition is individually and coalitionally rational in the subgame starting out at time t in a point ðx; yÞ on the optimal cooperative state trajectory. The conditions for core-theoretic cooperation in the subgame are
xi ðt; x; yÞrViNC ðt; x; yÞ;
i 2 f1; 2; 3g
13
x1 ðt; x; yÞ þ x3 ðt; x; yÞrV ðt; x; yÞ 3 X
xi ðt; x; yÞ ¼ V G ðt; x; yÞ
ð14Þ
i¼1
In (14), the value functions ViNC ðt; x; yÞ are given by (7). The value function V G can be shown to be V G ðt; x; yÞ ¼ ½1 etT ½ðc3 c1 Þx þ ðc3 c2 Þy þ ðc2 c3 ÞxetT ðT tÞ while the value function V 13 of coalition f1; 3g, valid for t 2 ½t5ð13Þ ; T, is V 13 ðt; x; yÞ ¼ ½1 etT ½ðc3 c1 Þx þ c3 y c3 etT xðT tÞ We apply the transfer scheme used in Germain et al. (2003) and define the monetary transfer paid (or received) by Region i over the time interval [t; T by
yi ðt; x; yÞ ¼ ½ViG ðt; x; yÞ ViNC ðt; x; yÞ þ mi ðt; x; yÞ½V G ðt; x; yÞ V NC ðt; x; yÞ
ð15Þ
In (15), ViG is the cost-to-go of Region i if it pays its own cost in the grand coalition, ViNC is the cost-to-go of Region i in the P P noncooperative game, V G ¼ 3i¼1 ViG is the cost-to-go of the grand coalition, and V NC ¼ 3i¼1 ViNC is the aggregate cost-to-go in the noncooperative game. P The parameters mi satisfy mi 2 ½0; 1, 3i¼1 mi ¼ 1. If mi ¼ 0, Region i will not benefit from cooperation. If mi ¼ 1, Region i monopolizes all the benefits of cooperation. If yi 40 (yi o0), Region i pays (receives) a transfer. The budget is balanced P because 3i¼1 yi ¼ 0. The game studied in Germain et al. (2003) includes for each player a pollution damage cost, depending on the aggregate stock of pollution, and a cost of pollution abatement that depends on the player’s emission rate. The authors use the general transfer rule in (15) to induce individual rationality in any period of time. To obtain coalitional rationality in any period, a more specialized transfer scheme is introduced such that monetary transfers depend directly on emission costs and indirectly (through the m- parameters) on the damage costs. The two transfer schemes are equivalent when the mparameters are chosen in a particular way. Using (15), the cooperative cost-to-go of Region i, including monetary transfers, is given by
xi ðt; x; yÞ ¼ ViG ðt; x; yÞ þ yi ðt; x; yÞ ¼ ViNC ðt; x; yÞ þ mi ðt; x; yÞ½V G ðt; x; yÞ V NC ðt; x; yÞ
ð16Þ
Since the difference V G V NC is negative, the imputation defined by (16) guarantees individual rationality at ðt; x; yÞ. We have seen that coalition f1; 3g can deviate for t 2 ½t5ð13Þ ; T. Suppose that it deviates at time t5ð13Þ . Then its value is V 13 ðt5ð13Þ ; x; yÞ and its strategy is (f1 ; f3 Þ ¼ ð1; 0Þ. Region 2 plays the noncooperative strategy f 2 ¼ 1. The after-deviation strategy profile is (f1 ; f 2 ; f3 Þ ¼ ð1; 1; 0Þ which we note is also the profile that the grand coalition would use for t 2 ½t5ð13Þ ; T.11 When the same strategy profile is used by the grand coalition and the subcoalition after it has deviated, the total costto-go of all three regions, during the time interval ½t5ð13Þ ; T, must be the same in the grand coalition and in the game between the subcoalition and Region 2. In that game, the cost-to-go of subcoalition f1; 3g is V 13 ðt; x; yÞ while the cost-to-go of Region 2 is V 2 ðt; x; yÞ (which is the value function of Region 2 in the game with the subcoalition). Therefore the following is true: Lemma 4. Value functions V G ðt; x; yÞ; V 13 ðt; x; yÞ, and V 2 ðt; x; yÞ satisfy V 13 ðt; x; yÞ þ V 2 ðt; x; yÞ ¼ V G ðt; x; yÞ The lemma can be formally proved by using the expressions for the three value functions involved. It says that the game between the subcoalition and Region 2 is a constant-sum game. This result turns out to be useful in the presentation of the following proposition. 10 It can be shown that if c2 4c1 , coalition K ¼ f1; 2g has the option to deviate at time t7ð12Þ (or later). If c3 4c2 , coalition K ¼ f2; 3g has the option to deviate at time t6ð23Þ (or later). Hence, under the assumption c1 4c2 4c3 , coalitions f1; 2g and f2; 3g cannot deviate. The proof is available from the author. 11 One may define a deviation as a case where a subcoalition ‘‘seeks an arrangement of its own’’, in the sense that it uses strategies that are different from those its members would use if they were in the grand coalition. Under such a definition, coalition K ¼ f1; 3g cannot deviate in our game.
ARTICLE IN PRESS S. Jørgensen / Journal of Economic Dynamics & Control 34 (2010) 258–265
265
Proposition 5. To achieve coalitional rationality at position ðt; x; yÞ, tZt5ð13Þ , regions must agree to choose m1 ðt; x; yÞ and m3 ðt; x; yÞ such that
x1 ðt; x; yÞ þ x3 ðt; x; yÞrV 13 ðt; x; yÞ()
m1 ðt; x; yÞ þ m3 ðt; x; yÞZ
V1NC ðt; x; yÞ þ V3NC ðt; x; yÞ V 13 ðt; x; yÞ V NC ðt; x; yÞ V 2 ðt; x; yÞ ¼ 1 2NC V NC ðt; x; yÞ V G ðt; x; yÞ V ðt; x; yÞ V G ðt; x; yÞ
ð17Þ
It holds that V1NC þ V3NC ZV 13 due to subadditivity of the characteristic function. Furthermore, V NC 4V G because the cooperative game is essential. The second line in (17) states that Regions 1 and 3 jointly must be given at least a certain share of the grand coalition’s cost saving V NC V G . The intuition is that the more, the two regions can gain ðV 13 Þ by deviating from the grand coalition, compared to their joint disagreement cost ðV1NC þ V3NC Þ, the higher a share of the grand coalition cost saving will the coalition demand as its price for not leaving the grand coalition. This is intuitive. 8. Conclusions The model suggested in this paper is a simplified structure and should not be seen as a ‘‘realistic’’ representation of the interactions between real-life neighbouring regions. The analysis has shown that it is possible, by using an intertemporal transfer scheme, to sustain core-theoretic cooperation over time. A major factor that drives this result is the differences between the regions’ cost parameters. A modification of the dynamics would be to abandon the assumption that no additional waste is generated during the play of the game. To account for this one can add a term, say, mi ðtÞ40 on the right-hand sides of the state equations. Also natural decay of waste could be included in the model, by subtracting a decay term on the right-hand side of the dynamics. The implication of such assumptions would be that the total stock of waste no longer is constant throughout the game. The effects of such extensions remain to be seen. A dramatic implication for the analysis would be to abandon the assumption of directed dumping. This would increase the number of control variables to six, leading to a myriad of feasible paths.
Acknowledgements The paper was presented at the workshop ‘‘Differential Games in Management’’, HEC, Montreal, Canada, and at the ‘‘13th International Symposium on Dynamic Games and Applications’’, Wroc"aw, Poland. I am grateful for the comments of participants in these events. I also appreciate the constructive remarks and suggestions of an editor and three reviewers. Any remaining errors are my responsibility. References Baumol, W.J., Oates, W.E., 1988. The Theory of Environmental Policy. Cambridge University Press, Cambridge, UK. Bre´chet, T., Gerard, F., Tulkens, H., 2007. Climate coalitions: a theoretical and computational appraisal. CORE Discussion Paper 2007/3. Chander, P., Tulkens, H., 1995. A core-theoretic solution for the design of cooperative agreements on transboundary pollution. International Tax and Public Finance 2, 279–293. Chander, P., Tulkens, H., 1997. The core of an economy with multilateral environmental externalities. International Journal of Game Theory 26, 379–401. Dockner, E.J., Jørgensen, S., Van Long, N., Sorger, G., 2000. Differential Games in Economics and Management Science. Cambridge University Press, Cambridge, UK. Germain, M., Toint, P., Tulkens, H., de Zeeuw, A., 2003. Transfers to sustain dynamic core-theoretic cooperation in international stock pollutant control. Journal of Economic Dynamics & Control 28, 79–99. Jørgensen, S., Zaccour, G., 2001. Time consistent side payments in a dynamic game of downstream pollution. Journal of Economic Dynamics & Control 25, 1973–1987. Jørgensen, S., Zaccour, G., 2002. Time consistency in cooperative differential games. In: Zaccour, G. (Ed.), Decision & Control in Management Science: Essays in Honor of Alain Haurie. Kluwer Academic Publishers Boston, MA, USA, pp. 349–366. Jørgensen, S., Martı´n-Herra´n, G., Zaccour, 2003. Agreeability and time consistency in linear-state differential games. Journal of Optimization Theory and Applications 119, 49–63. Kranich, L., Perea, A., Peters, H., 2005. Core concepts for dynamic TU games. International Game Theory Review 7, 43–62. Petrosjan, L.A., Zenkevich, N.A., 1996. Game Theory. World Scientific, Singapore. Petrosjan, L.A., 2005. Cooperative differential games. In: Novak, A.S., Szajowski, K. (Eds.), Advances in Dynamic Games: Applications to Economics, Finance, ¨ Optimization and Stochastic Control. Annals of the International Society of Dynamic Games, 7. Birkhauser, Boston, MA, USA, pp. 183–200. Predtetchinski, A., 2007. The strong sequential core for stationary cooperative games. Games and Economic Behavior 61, 50–66. Shapley, L.S., Shubik, M., 1969. On the core of an economic system with externalities. American Economic Review 59, 678–684. Zaccour, G., 2003. Computation of characteristic function values for linear-state differential games. Journal of Optimization Theory and Applications 117, 183–194.