Optimal management with reversible regime shifts

G Model JEBO-3791; No. of Pages 13

ARTICLE IN PRESS Journal of Economic Behavior & Organization xxx (2016) xxx–xxx

Contents lists available at ScienceDirect

Journal of Economic Behavior & Organization journal homepage: www.elsevier.com/locate/jebo

Optimal management with reversible regime shifts Michele Baggio a,∗ , Paul L. Fackler b,1 a b

Department of Economics, University of Connecticut, United States Department of Agricultural and Resource Economics, North Carolina State University, United States

a r t i c l e

i n f o

Article history: Received 19 September 2015 Received in revised form 9 March 2016 Accepted 21 April 2016 Available online xxx Keywords: Adaptive management Bioeconomic modeling Fisheries Partial observability Partially Observable Markov Decision Process Regime switching

a b s t r a c t In this paper we examine the management of a natural resource, a fishery, subject to regime shifting dynamics. A regime shift is defined as an episode in which the dynamics of the resource can switch between two alternative regimes at random times. Specifically, we study the impact of reversible regime switching, both observed and unobserved, on optimal harvesting policy. The case in which the regime is not directly observed, i.e., observational uncertainty, is addressed by using the extended POMDP approach developed in Fackler and Pacifici (2014). We illustrate the performance of the model under different assumptions on the underlying stochastic growth variability, the biological structure of the stock in different regimes, and the resilience of the regimes. When the regime is known optimal policies depend on the population level and which regime is currently active. When the regime is unobserved, on the other hand, the optimal policy depends on the population level and a belief distribution about the current regime. In general when the probability of regime change is fixed, and hence is not affected by harvesting policy, the optimal policy is of the constant-escapement variety. When the probability of switching regime is endogenous the optimal policy is no longer of the constant-escapement type. Optimal policies when the regime is uncertain are approximately equal to a weighted average of the policies when the regime is certain, with the weights equal to the beliefs in the associated regimes, but there are differences. © 2016 Elsevier B.V. All rights reserved.

1. Introduction Intrinsic environmental variability affecting stock growth, inaccurate measurement of the stock, unknown stock dynamics, and imprecise implementation of management policies are all sources of uncertainty that make the management and conservation of many renewable resources particularly challenging (e.g., Roughgarden and Smith, 1996; Sethi et al., 2005; Fackler, 2014). Depending on the source and the magnitude, uncertainty can have a large impact on strategies for the optimal exploitation of a resource. There exists a vast literature on the management of renewable resources under uncertainty. Under certain conditions the presence of only environmental variability that results in future population uncertainty leads to a constant-escapement rule for the exploitation of the resource (Reed, 1979). This result no longer applies, however, when current stock cannot be perfectly measured (Clark and Kirkwood, 1986; Sethi et al., 2005). More recent literature addresses the optimal management

∗ Corresponding author. E-mail address: [email protected] (M. Baggio). 1 Lead authorship is not assigned. We thank the editor and two anonymous reviewers for their constructive comments, which helped us to improve the manuscript. http://dx.doi.org/10.1016/j.jebo.2016.04.016 0167-2681/© 2016 Elsevier B.V. All rights reserved.

Please cite this article in press as: Baggio, M., Fackler, P.L., Optimal management with reversible regime shifts. J. Econ. Behav. Organ. (2016), http://dx.doi.org/10.1016/j.jebo.2016.04.016

G Model JEBO-3791; No. of Pages 13

2

ARTICLE IN PRESS M. Baggio, P.L. Fackler / Journal of Economic Behavior & Organization xxx (2016) xxx–xxx

of dynamic systems in which uncertainty consists of the risk of regime shifts. Specifically, this literature examines the optimal management of natural resources with potential irreversible regime shifts finding that when shifts are endogenous optimal management becomes precautionary, implying lower harvest and higher conservation (Polasky et al., 2011). Analogous results are obtained in the context of pollution control with a random irreversible shift in the damage function (De Zeeuw and Zemel, 2012). However, when strategic behavior of individuals affects the risk of a shift, precautionary management may no longer be optimal and the resource is exploited at a higher rate than for non-switching resources (Sakamoto, 2014). It is also showed that the optimal management of renewable resources can become either more aggressive or more precautionary than the no-shift case depending on the shape of the reward function, which expresses changes in value of harvesting the resource (Ren and Polasky, 2014). Because of how it influences the management, it may be important to recognize the existence of the uncertainty and take steps to reduce it. This may warrant the adoption of an adaptive/learning type of management (Hauser and Possingham, 2008). There is, however, conflicting evidence concerning whether addressing uncertainty and learning using adaptive management leads to either economic gains or higher biological conservation. Costello et al. (2001) examine shocks to the growth of fish populations and show that welfare gains can arise from incorporating predictions about future environmental conditions. Bond and Loomis (2009) find only minor benefits of active over passive learning management. Carson et al. (2009) show that even an adaptive harvest policy can lead to overfishing and lower rents when only a limited amount of past data is used to predict growth rates. Springborn and Sanchirico (2013) show that resolving structural uncertainty through learning can be beneficial relative to passive learning and non-learning strategies. In the context of the management of resources subject to regime shifts, when the current regime is known, incorporating the switching behavior in the management decisions can lead to considerable gains in both rent and conservation (Baggio, 2015). In this paper we consider a bioeconomic model that incorporates regime-switching dynamics to investigate the optimal management policy for exogenous and endogenous reversible shifts. Specifically, the dynamics of the resource can switch between two alternative regimes at random times, between a high and a low productivity regime. Furthermore we examine cases in which the switch probabilities may or may not depend on the stock level and in which the regime may or may not be directly observed. There is evidence that several commercially exploited fish populations may have experienced shifts in their dynamics that resemble the type of shifts discussed in this analysis. The Peruvian anchoveta (Engraulis ringens) has undergone large fluctuations with structural changes in the stock–recruitment relationship due to the alternating of favorable and unfavor˜ Southern Oscillation (ENSO) combined with possible overfishing (Alheit and Niquen, able phases generated by the El Nino 2004; Fréon et al., 2008; Cahuin et al., 2009). Other examples of regime shifts in a fishery are the Georges Bank haddock (Melanogrammus aeglefinus) where fishing pressure is claimed to be responsible for shifts between a low and a high mortality threshold (Collie et al., 2004), and the market squid (Loligo opalescens) whose growth rate also depends on the ENSO, with ˜ (warm) phase (Jackson and Domeier 2003). slower growth occurring during the El Nino Our paper extends the existing literature in two ways. First, it presents a model that admits reversible regime shifts, both exogenous and endogenous, and, second, it examines optimal management when uncertainty exists about the current regime. One approach to addressing observational uncertainty in resource management uses the POMDP framework in which the unobserved regime is replaced by a distribution describing the beliefs about the regime itself (see Fackler, 2014; for a review). In the current case however the standard POMDP approach is not adequate because one of the states is observed and the other is not and because information concerning the unobserved state (the regime) comes not from a monitoring variable but from the levels and change in the observed state (the population level). To address these issues we use the extended POMDP approach developed in Fackler and Pacifici (2014) and applied to an adaptive management problem in Fackler et al. (2014). Unlike the results of Polasky et al. (2011) the additional features modeled here preclude obtaining closed form solutions and comparative static results. Instead, the performance of the model is solved numerically under different assumptions on the underlying stochastic growth variability, the biological structure of the stock in different regimes, and the resilience of the regimes. Results from our illustrative numerical analysis indicate that when shifts are exogenous the optimal policy for one regime is of constant-escapement type and the optimal escapement in the low productivity regime is lower than in the high productivity regime. When shifts are endogenous (stock dependent) the optimal escapement policy for the low productivity regime is not a constant-escapement policy and its shape depends on stock volatility, the biological parameters of the stock dynamics, and on the characteristics of the shift probability. We also examine how observational uncertainty about the current regime influences the optimal escapement policy of the fishery and its outcomes. Simply put, we want to learn what can be gained from observing vs. not observing the regime. This is policy relevant because considerable resources are invested in improving stock assessments and surveys in key fisheries around the world, resources could also be allocated to resolve uncertainty about the true biological structure of exploited fish resources. The paper is organized as follows. In Section 2 the analytical model is introduced, followed by an illustrative example in Section 3. Section 4 presents the results of the numerical exercises, which are used to discuss the implications on the optimal decision rule implied by different assumptions on the uncertainty and the parameters of the model. Finally, Section 5 presents some concluding remarks.

Please cite this article in press as: Baggio, M., Fackler, P.L., Optimal management with reversible regime shifts. J. Econ. Behav. Organ. (2016), http://dx.doi.org/10.1016/j.jebo.2016.04.016

G Model

ARTICLE IN PRESS

JEBO-3791; No. of Pages 13

M. Baggio, P.L. Fackler / Journal of Economic Behavior & Organization xxx (2016) xxx–xxx

3

2. Model The fish population is assumed to evolve according to a Beverton–Holt escapement/recruitment relationship N+ =

rj E 1+

rj−1 E kj

eu−

2 /2

(1)

where N+ denotes the stock in the next period, E is the escapement (the current stock less the harvest: E = N − A), r is the intrinsic growth rate of the population, K is the carrying capacity and u is a standard normal shock (this parameterization 2 ensures that the multiplicative shock eu− /2 has mean 1). The subscript j denotes the regime, with j = 1, 2, and  is a measure of the degree of environmental uncertainty present in the population. The dynamics of the resource can switch between two alternative regimes at random times. Two models are considered. In Model 1 regime shifts are not influenced by any management decisions (exogenous regime shift). The switching is governed by a matrix of transition probabilities that are constant. In Model 2 the probability of switching is a function of the resource stock and therefore depends on management actions (endogenous regime shift). Specifically, in Model 1 the following matrix represents the transition probabilities



PR =

p1

1 − p2

1 − p1

p2



,

(2)

where pj = P (R+ = j|R = j) is the probability that the regime in the next period, R+ , is the same regime as in the current period, R. This can be interpreted as the resilience of regime itself (Perrings, 1998). In Model 2 the switching probabilities depend on the stock level:2





P R+ = j|R = j =

1

 

1 + exp − ˛j + ˇj N

 .

(3)

In addition to the two models of the switching probabilities, we consider two assumptions concerning the knowledge of the current regime. In the first case the decision maker knows the current regime of the stock dynamics: Markov Decision Process (MDP). In the second it is assumed that it must be inferred from the stock data: Partially Observable Markov Decision Process (POMDP). 2.1. Markov Decision Process (MDP) When the decision maker can directly observe the regime of the stock dynamics, the structure of biological parameters that apply at time t, rj and Kj , are known. The stock dynamics are given by Eqs. (1) and (2) for the case of exogenous regime shifts and by Eqs. (1) and (3) for endogenous shifts. The decision maker chooses the harvest level with the objective of maximizing the sum of the discounted flow of expected rewards derived by the exploitation of the resource over an infinite time horizon given subject to the stock dynamics: V (Nt , Rt ) = max A

∞ 

ı−t E [W (Nt , Rt , A (Nt , Rt )) |Nt , Rt ]

(4)

=t

where ı is a discount factor defined on [0,1] and W(N, R, A) is the immediate reward obtained from taking action A in resource state (N,R). For simplicity and to make the results comparable to many previous studies we assume that the immediate reward is simply the harvest level: W(N, R, A) = A. 2.2. Partially Observable Markov Decision Process (POMDP) When the decision maker cannot directly observe the current regime of the stock dynamics the optimal management cannot be determined using the MDP framework. Instead, the problem can be solved using the POMDP framework, which replaces the unobserved regime with a belief distribution about the current regime. Beliefs are updated based on the regime transition probabilities (time update) and on observations of variables that provide information about the unobserved state (observation update). In standard POMDPs the observed information is conditional on the action and the future state (Fackler and Haight, 2014). In the current case however, the information used to update beliefs about the current regime is both the current and the next period’s population levels, N and N+ , respectively; specifically the likelihood of a given regime depends

2 We considered the alternative assumption that the switching probabilities depend on escapement. Dependence on the stock level means that the current action does not affect the switch probabilities until the next period, whereas dependence on escapement implies that the switch probabilities are affected in the current period. Following Polasky et al. (2011) we chose to use stock dependence as our base case but also examined escapement dependence. Although there were some differences resulting from the alternative assumptions most of the qualitative results were similar.

Please cite this article in press as: Baggio, M., Fackler, P.L., Optimal management with reversible regime shifts. J. Econ. Behav. Organ. (2016), http://dx.doi.org/10.1016/j.jebo.2016.04.016

G Model

ARTICLE IN PRESS

JEBO-3791; No. of Pages 13

M. Baggio, P.L. Fackler / Journal of Economic Behavior & Organization xxx (2016) xxx–xxx

4

Fig. 1. Expected escapement–recruitment relationship (base case); E denotes escapement, N+ population in the next period.

on N + /E[N + |N, R, A]. To address this we use the extended POMDP approach developed in Fackler and Pacifici (2014) and Fackler et al. (2014); see Appendix A for a summary of the features of this approach. The extended POMDP approach uses the joint distribution of the future states and any other information, all conditioned on the current states and actions. In the current case it utilizes the joint distribution of the future regime and population conditioned on the current regime, population and harvest level. The belief B(R) concerning the regime is updated using Bayes Rule





B+ R+ = Pr(R+ |N, B, A, N + ) =

 Pr (N + , R+ |N, R, A) B (R) R   + + R+

R

Pr (N , R |N, R, A) B (R)

(5)

and the optimal harvest A is a function of the stock and the belief about the regime that maximizes V (Nt , Bt ) = max A

∞ 

ı−t E [W (Nt , Rt , A (Nt , Bt )) |Nt , Bt ]

(6)

=t

Notice that the unobservable state variable R, the current regime, is replaced by the belief in a given regime. Thus both the value function, V, and the optimal decision are functions of the population level and the beliefs rather than the population and the regime. 3. An illustrative example The parameters of the escapement/recruitment relationship depend on which of the two regimes is currently active. We assume that one of the two regimes is unfavorable as it features lower growth per individual and has a lower carrying capacity than the other regime. As an example, we use parameters r1 = 1.5 and K1 = 1 for Regime 1. For Regime 2, the unfavorable regime, r2 = 1.25 and K2 = 0.5. Fig. 1 shows the expected escapement–recruitment relationship for the two regimes in the base case. We assume that the environmental noise is the same across the two regimes with a base case value of  = 0.1 and use a discount factor of 0.98 (a discount rate of approximately 2%). We consider two models for the transitions between regimes. In Model 1 the transition probability is represented by the fixed probability matrix

 PR =

0.85

0.05

0.15 0.95

.

(7)

This implies that the resilience of each state is high and is higher in Regime 2 than in Regime 1. These probabilities are similar to those estimated in Baggio (2015) for the Peruvian anchoveta, although the switching between favorable and ˜ phenomenon. In Model 1 the shift between regimes is unfavorable regimes in Baggio (2015) is induced by the El Nino of the exogenous, whereas in Model 2 the shift probability is function   stock level. For the parameters in Eq. (3) we use ˇ1 = 6 and ˇ2 = −6 and compute the constant ˛ using ␣j = log pj /1 − pj − ˇj NjMSY , where NjMSY is the stock level associated Please cite this article in press as: Baggio, M., Fackler, P.L., Optimal management with reversible regime shifts. J. Econ. Behav. Organ. (2016), http://dx.doi.org/10.1016/j.jebo.2016.04.016

G Model

ARTICLE IN PRESS

JEBO-3791; No. of Pages 13

M. Baggio, P.L. Fackler / Journal of Economic Behavior & Organization xxx (2016) xxx–xxx

5

Fig. 2. Regime switch probabilities, Pr(R+ = j|R = j), j = 1,2. Horizontal lines are for Model 1 (exogenous), curved lines are for Model 2 (endogenous).



with the Maximum Sustainable Yield under regime j, given by NjMSY = Kj rj −

 



rj / rj − 1 .3 This ensures that the switch

probabilities are the same in Models 1 and 2 at the stock level associated with the MSY. As shown in Fig. 2, the probability of staying in Regime 2 goes to 0.99 as the stock level declines to 0 and the probability of staying in Regime 1 is higher than the probability of switching back to it from Regime 2 at all positive levels of the stock.4 In particular, low population levels increase the probability that the system will be in the low productivity regime in the next period. 4. Results In this section we examine various assumptions about the magnitude and the sources of uncertainty. For each assumption we show the graphs associated with the optimal escapement policy function. Specifically, we investigate the following cases. First, the base case of regime switching dynamics with exogenous (Model 1) and endogenous regime shifts (Model 2) and where the current regime is either (a) known or (b) unknown, as described in the previous section. Second, we derive the optimal harvest closure stock levels for the case of (i) no regime shift, (ii) irreversible shift, and (iii) reversible shift, and we investigate how the harvest closure levels change with the population volatility. Third, we do sensitivity analysis on some of the key parameters of the base model. Specifically, we investigate how the optimal escapement policy for both regimes varies with the growth rate and the carrying capacity of Regime 2 (r2 and K2 ), as well as on the probability of remaining in Regime 2, p2 , and the sensitivity of this probability to the stock level, ˇ2 . 4.1. Optimal policy with varying degrees of reversibility Fig. 3 (top row), shows the optimal policy for the base case with potentially reversible regime shifts (p1 = 0.85, p2 = 0.95) for both exogenous and endogenous shifts and when the current regime is either known or unknown. The black dotted/dashed the grey lines represent lines represent the escapement policy when there is perfect certainty concerning the regime whereas

 the escapement policy at different levels of belief that the current regime is Regime 1: B1 ∈ 0, 0.2, 0.4, 0.6, 0.8, 1 . In the perfect certainty case (labeled R1 and R2 ) the optimal decisions are always constant-escapement rules if the switch probability is independent of the stock level (Model 1). The optimal harvest closure stock levels are given in Table 1 and indicate that the harvest should be closed when the stock levels fall below 0.43 in Regime 1 and 0.23 in Regime 2. This is nearly the same strategy that is optimal when the no regime shifts are possible (shown in Fig. 3 middle row and Table 1), for which the optimal harvest closure stock levels are 0.43 and 0.21. These are also the optimal harvest closure stock levels for the case in which a shift from Regime 1 to Regime 2 is possible but is irreversible (Fig. 3 bottom row and Table 1). Differences in the optimal strategy become more pronounced as the environmental noise variability  increases (Table 1). That there

3

For the Beverton–Holt model a sustainable yield, Y , satisfies N =

r(N−Y ) . 1+(r−1/K )(N−Y )

Solving for Y leads to Y =

(1−r)N+(r−1/K )N 2

(r−1/K )N−r

. Maximizing this with respect

to N leads to the expression for NjMSY . 4

Notice that increasing ˇj increases the resilience of regime j.

Please cite this article in press as: Baggio, M., Fackler, P.L., Optimal management with reversible regime shifts. J. Econ. Behav. Organ. (2016), http://dx.doi.org/10.1016/j.jebo.2016.04.016

G Model

ARTICLE IN PRESS

JEBO-3791; No. of Pages 13

M. Baggio, P.L. Fackler / Journal of Economic Behavior & Organization xxx (2016) xxx–xxx

6

Fig. 3. Optimal escapement policies for the p = [0.85, 0.95] base case (top), p = [1, 1] no shift possible (middle) and p = [0.85, 1] irreversible shift (bottom). The left column is the exogenous switching Model 1 and the right column is the endogenous switching Model 2 (right). Dashed and dotted lines denote the case of perfect certainty for Regime 1, R1 , and for Regime 2, R2 , respectively. The gray lines (denoted RU for regime uncertainty) represent the optimal strategies for the different beliefs about which regime is active. Lines closest to R2 represent strong beliefs the Regime 2 is active. Belief values of 0, 0.2, 0.4, 0.6, 0.8 and 1 are shown. Table 1 Optimal harvest closure stock levels. Reversibility

Variability

Model 1—fixed PR R=1

R=2

Model 2—variable PR R=1

R=2

p = [0.85, 0.95] p = [0.85, 0.95] p = [0.85, 0.95]

 = 0.1  = 0.4  = 0.7

0.43 0.46 0.51

0.23 0.28 0.34

0.61 0.61 0.60

0.59 0.51 0.44

p = [1.00, 1.00] p = [1.00, 1.00] p = [1.00, 1.00]

 = 0.1  = 0.4  = 0.7

0.43 0.47 0.56

0.21 0.26 0.30

0.43 0.47 0.56

0.21 0.26 0.30

p = [0.85, 1.00] p = [0.85, 1.00] p = [0.85, 1.00]

 = 0.1  = 0.4  = 0.7

0.43 0.46 0.50

0.21 0.26 0.30

0.64 0.61 0.58

0.21 0.26 0.30

Note: fixed probability, p = [p1 , p2 ], indicates the probability of remaining in either regime for the case of an exogenous shift. For variable switch probability p denotes the probability of remaining in a regime evaluated at the stock level associated with the Maximum Sustainable Yield.

are any differences may be surprising in the light of the results are Polasky et al. (2011) but that study assumes that only the switch and not the population change is stochastic and that the population transition is linear in the harvest. In the case of Model 2 (endogenous switching probability) the optimal decision rule is quite different due to the reversibility characteristics of the system. Relative to the no shift case (p = [1, 1]), the base case value of p = [0.85, 0.95] leads to a far more precautionary strategy in Regime 1 (harvest closure stock level of 0.61 rather than 0.43) and is more proactive in Regime 2 (harvest closure stock level of 0.59 rather than 0.21). Such a strategy both reduces the probability of a switch from Please cite this article in press as: Baggio, M., Fackler, P.L., Optimal management with reversible regime shifts. J. Econ. Behav. Organ. (2016), http://dx.doi.org/10.1016/j.jebo.2016.04.016

G Model JEBO-3791; No. of Pages 13

ARTICLE IN PRESS M. Baggio, P.L. Fackler / Journal of Economic Behavior & Organization xxx (2016) xxx–xxx

7

Fig. 4. Simulation results for Model 1, exogenous switching, (left column) and Model 2, endogenous switching, (right column) starting in Regime 1 at population carrying capacity under perfect regime certainty. The vertical axis indicates harvest H and population level N; the horizontal axis time. Top panel displays a representative path for the population and harvest levels and the periods when Regime 2 was active. The middle panel shows the average behavior of population and harvest and the bottom panel shows the probability of being in Regime 2 and of a harvest closure.

the high to the low productivity regimes and increases the probability of recovery from the low productivity regime. Notice that, in the low productivity regime the harvest close stock level is above the carrying capacity, which means that harvest should essentially be closed in this regime; this provides the best chance that the stock will return to the high productivity Regime 1. Seasonal or area closures are common management strategies that are intended to reduce fishing pressure and to allow for stock recovery. This is true even in the case of a fish population that fluctuates due to the alternating between regimes like the Peruvian anchoveta. Indeed, this fishery has often been shut down when surveys indicated that stock was ˜ events. too low following the onset of extreme El Nino When the regime shift is irreversible the precautionary behavior in Regime 1 is even more pronounced (harvest closure stock level of 0.64 rather than 0.61). In this case recovery is impossible so even greater precautionary behavior is called for to reduce the chance of a shift to the low productivity regime. Thus the possibility of recovery leads to less precautionary behavior. At this point it is interesting to investigate how the model behaves over time and in the long-run. Fig. 4 displays simulation results for our base case for exogenous switching Model 1 (left column) and endogenous switching Model 2 (right column). The top row show results for a representative path. For exogenous shifts (left column), adopting the optimal management strategy leads to population and harvest levels that are lower than the case of endogenous shifts (right column). This is because the constant probability of staying in Regime 2, p2 , is very high, i.e., 0.95. When the system switches to Regime 2, the regime with low productivity, it is very hard to switch back. This is clearly indicated in the figure by the periods when Regime 2 is active, denoted by the asterisks. When the probability depends on the population level (right column) the system is almost never in the unfavorable regime. Indeed, the long-run probability of being in Regime 2 quickly moves to its long run equilibrium levels of 0.75 for Model 1 and 0.26 for Model 2 (Fig. 4, bottom row and Table 2). In the middle row of Fig. 4 the expected population and harvest levels are seen to also quickly reach their long-run equilibrium levels, which is higher for the endogenous switching case: 0.641 vs. 0.323 for the population and 0.067 vs. 0.045 for the harvest (Table 2). For the model with fixed probabilities there is also a higher probability that the fishery will close, 0.46, than in the endogenous probability case, 0.364. These results are rather different than for the no-switching case. Table 2 indicates the stark difference between the long-run equilibrium expected population level of 0.54 for Regime 1, and 0.23 for Regime 2. In addition harvest is nearly three times higher for Regime 1 and there is also a smaller probability of harvest closure, 7% vs. 42%. Fig. 5 and Table 1 also explore the relationship between reversibility and the environmental uncertainty as indicated by the standard deviation  of the noise term in the population transition. In the exogenous case (Model 1) increases in uncertainty lead to increases in optimal harvest closure stock level in both regimes and thus to decreases in the harvest Please cite this article in press as: Baggio, M., Fackler, P.L., Optimal management with reversible regime shifts. J. Econ. Behav. Organ. (2016), http://dx.doi.org/10.1016/j.jebo.2016.04.016

G Model

ARTICLE IN PRESS

JEBO-3791; No. of Pages 13

M. Baggio, P.L. Fackler / Journal of Economic Behavior & Organization xxx (2016) xxx–xxx

8

Table 2 Long-run behavior under no-switching and base case switching parameters. E[N] Regime 1 Regime 2 Model 1 (exogenous switching) Model 2 (endogenous switching)

No switching: p = [1, 1] 0.541 0.226 Base case: p = [0.85, 0.95] 0.323 0.641

E[H]

Pr(R2 )

Pr(C)

0.101 0.028

0 1

0.071 0.421

0.045 0.067

0.750 0.261

0.416 0.364

Note: E [N] expected population, E [H] expected harvest, Pr (R2 ) probability of Regime 2, Pr (C) probability of harvest closure.

Fig. 5. Optimal escapement policies: Sensitivity for reversibility, p = [p1 , p2 ], and volatility, , (model 2 endogenous switch probability only); E* denotes optimal escapement, N population. Dashed and dotted lines denote the case of perfect certainty for Regime 1, R1 , and for Regime 2, R2 , respectively. The remaining solid lines identify the cases for the different beliefs about Regime 1, i.e., B1 taking values from 0 to 1.

level. In the endogenous case (Model 2) this only holds in situations where it is not possible to affect the regime shift. When the action can affect the probability of a regime shift an increase in environmental uncertainty causes the optimal harvest stock level to fall although it is still above the level for the exogenous switch model. This goes against the intuition that increases in uncertainty should lead to more precautionary behavior. What is important here however is whether an action has an impact on outcomes and specifically on a regime shift. One explanation for increases in environmental uncertainty Please cite this article in press as: Baggio, M., Fackler, P.L., Optimal management with reversible regime shifts. J. Econ. Behav. Organ. (2016), http://dx.doi.org/10.1016/j.jebo.2016.04.016

G Model JEBO-3791; No. of Pages 13

ARTICLE IN PRESS M. Baggio, P.L. Fackler / Journal of Economic Behavior & Organization xxx (2016) xxx–xxx

9

Fig. 6. Optimal escapement policies: sensitivity for the intrinsic growth rate and the carrying capacity for Regime 2, r2 and K2 (model 2 endogenous switch probability only); E* denotes optimal escapement, N population. Dashed and dotted lines denote the case of perfect certainty for Regime 1, R1 , and for Regime 2, R2 , respectively. The remaining solid lines identify the cases for the different beliefs about Regime 1, i.e., B1 taking values from 0 to 1.

leading to less precautionary behavior is that it leads to a decrease in the controllability of the switch probability and hence increases the cost of either preventing or facilitating a switch.5 4.2. Sensitivity to the low productivity regime parameters Polasky et al. (2011) argue that the increase in the steady-stock level when shifts are endogenous depends on the difference between the growth function of the two regimes and the characteristics of the transition probability (they do not derive any comparative statics or perform numerical simulation to investigate this issue however). In this section we investigate the effect on the optimal escapement policy of changing (i) the magnitude of the biological parameters, and (ii) the responsiveness to the stock level of the endogenous probability of staying in Regime 2 and on the overall level of this probability. We focus on the case of endogenous regime shifts. The optimal policy depends on the growth rate and carrying capacity of Regime 2. Fig. 6 shows the optimal policy when the carrying capacity of Regime 2 takes the value of 25, 50, and 75% of the carrying capacity of Regime 1 (plots left to right), and the growth rate taking values of 1.15, 1.25, and 1.35 (top to bottom). This gives a total of nine plots depicting the optimal escapement policy for each case; the base case is the center plot. For the most part the optimal policy of Regime 2 is not of the constant-escapement variety and the optimal escapement strategy may decrease or increase as the stock level increases. In general, increasing the growth rate r2 always decreases the harvest closure level for Regime 2, whereas the harvest closure level for Regime 2 always increases with carrying capacity. Notably, only when the growth rate, r2, and/or the carrying capacity, K2 , are relatively high is the harvest closure level in Regime 2 below carrying capacity; in these situations the differences between the two regimes are relatively small and hence the incentive to induce a regime shift is relatively low. Summing up, the optimal policy is sensitive to differences in the biological parameters between the two regimes. In general, however, harvest closures occur most of the time in Regime 2 when there is a reasonable probability of recovery. Notice that the probability of switching back to Regime 1 is 5% at K2 = 0.25, 25% at K2 = 0.5, and 55% at K2 = 0.75. So, if carrying

5 The result that the growth uncertainty has a substantial effect on optimal harvest policy contrasts with Sethi et al. (2005) who find that even large growth uncertainty per se does not have a substantial effect on the optimal policy. This difference may be due to the assumption on the distribution of the multiplicative shock to the growth function. Sethi et al. (2005) assumes a uniform distribution; we assume a log-normal distribution. Clark and Kirkwood (1986) derive optimal escapement policies for both a uniform and log-normal distribution. For uniformly distributed shocks they find that the shut-off level is a non-monotonic function of volatility, but for the log-normally distributed shocks the shut-off level is monotonically decreasing with volatility (see Fig. 2 in Clark and Kirkwood’s paper).

Please cite this article in press as: Baggio, M., Fackler, P.L., Optimal management with reversible regime shifts. J. Econ. Behav. Organ. (2016), http://dx.doi.org/10.1016/j.jebo.2016.04.016

G Model JEBO-3791; No. of Pages 13

10

ARTICLE IN PRESS M. Baggio, P.L. Fackler / Journal of Economic Behavior & Organization xxx (2016) xxx–xxx

Fig. 7. Optimal escapement policies: Sensitivity for the level and sensitivity of the endogenous probability of being in Regime 2, p2 and ˇ2 (Model 2 endogenous switch probability only); E* denotes optimal escapement strategy, N denotes population. Dashed and dotted lines denote the case of perfect certainty for Regime 1, R1 , and for Regime 2, R2 , respectively. The remaining solid lines identify the cases for the different beliefs about Regime 1, i.e., B1 taking values from 0 to 1.

capacity of Regime 2 is a small fraction of the capacity of Regime 1 a shift to Regime 2 is almost irreversible. In such a situation there is little incentive to reduce harvest. We also analyze how the characteristics of the transition probability influence the optimal policy. Fig. 7 shows the optimal policy when the fixed probability of staying in Regime 2, p2 , takes the values of 0.92, 0.95, and 0.98, and when the Model 2 sensitivity of the probability to the fish population, ˇ2 , takes the values −3, −6 and −9. Changing the fixed probability for Regime 2 changes also the endogenous probability via ˛2 in Eq. (3). Increasing p2 increases the resilience of Regime 2. For example, in Model 2 the values of p2 = (0.92, 0.95, 0.98) correspond to P (R+ = 2|R = 2) = (0.70, 0.80, 0.91) at carrying capacity. On the other hand, increasing the absolute level of ˇ2 reduces the resilience of the regime, with P (R+ = 2|R = 2) = (0.90, 0.80, 0.64) at carrying capacity. In both cases a decreased resilience makes harvest closures more effective at restoring the system to the high productivity regime and therefore more attractive. The main message of Fig. 7 is that optimal harvests are generally higher when the probability of recovery is lower and less sensitive to the population level (moving in a southwest direction in the figure); the plot in the southwest corner indicates the case with the highest resilience for Regime 2; at ˇ2 = −3 even at carrying capacity there is only a 4% probability of switching to the more favorable regime. Hence, there is no incentive to rebuild the stock in order to induce a shift back to the favorable regime. As the resilience of Regime 2 decreases, (increases in p2 or decreases in ˇ2 ), the policy becomes more conservative and the harvest closure levels for Regime 2 are above the carrying capacity, which effectively closes the fishery when Regime 2 is active. 4.3. Regime uncertainty Thus far the discussion has focused on cases in which the current regime is known. When the current regime is unknown, so the decision rule depends on the weight of belief in a given regime, the optimal escapement policy is close to a belief weighted average of the perfect certainty case. They are not exactly equal, however, and although, in general, the belief weighted average solution would yield similar results, there are differences. Even when one is currently sure of which regime is active, e.g., (B1 = 1), the optimal decision is not the same as the optimal rule when there is no uncertainty about the regime and the system is known to be in Regime 1 (R1 ). The reason for this divergence is that, with regime uncertainty, perfect certainty that the regime is currently 1 does not imply that the regime will be known with certainty in the future. Similar remarks apply to the situation when B1 = 0. To compare optimal strategies between the belief weighted strategy and the POMDP we focus on solutions for the case with r2 = 1.15 and K2 = 0.75. This is an interesting case because different beliefs yield fairly different optimal strategies. In Fig. 6, top right corner, they seem to be almost equally spaced, suggesting that the optimal strategy is equal to a belief Please cite this article in press as: Baggio, M., Fackler, P.L., Optimal management with reversible regime shifts. J. Econ. Behav. Organ. (2016), http://dx.doi.org/10.1016/j.jebo.2016.04.016

G Model JEBO-3791; No. of Pages 13

ARTICLE IN PRESS M. Baggio, P.L. Fackler / Journal of Economic Behavior & Organization xxx (2016) xxx–xxx

11

Fig. 8. Value functions for the r2 = 1.15, K2 = 0.75 case. Upper curves for each pair are for the perfect regime certainty case; lower curves are for the regime uncertainty case when the current regime is known with certainty.

weighted average of the policies obtained when the regime is certain. Supplemental Fig. S1 in the online version at DOI: 10.1016/j.jebo.2016.04.016 shows that this is not always the case. The top left panel shows the optimal strategies for the belief weighted average of the perfect certainty strategy; the top right panel shows the POMDP solution. The bottom panel indicates the difference in optimal escapement between the POMDP solution and the weighted average strategy. Differences arise when the population level is around 0.6; below this value of N the two strategies yield the same result because in both cases the fishery is always closed. For population levels between 0.6 and 0.8 the weighted average strategy is more precautionary than POMDP, for almost all belief values. At higher levels the situation is reversed with the POMDP strategy being more precautionary. The fact that the optimal strategy when the regime in unknown is close to a belief weighted action suggests that there may be not much to be gained using the POMDP framework in this situation. This certainly reduces computational challenges. It is not altogether surprising because the information gained in a single period does not change much as a result of the harvest decision. It should be noted that the POMDP approach is necessary and valuable when monitoring decisions need to be made; this has been the focus of most of the applications of the POMDP framework to natural resource management problems (see Fackler and Haight, 2014; for a review). Although small, there is some additional value in using the optimal vs. the belief weighted strategy. Fig. 8 compares the value functions for the r2 = 1.15 and K2 = 0.75 case examined previously. The upper curves for each pair are for the perfect regime certainty case, the lower curves are for the regime uncertainty case when the current regime is known with certainty (i.e., B1 = 0 or B1 = 1). Differences of as much as 1.9% arise in the value functions, mainly at values of between 0.2 and 0.8; this interval is where the population tends to be under optimal management. As these curves represent the situation in which there is perfect knowledge of the current regime, the difference between them can be viewed as a lower bound on the value of perfect information. 5. Concluding remarks This paper examines the impact of non-irreversible regime switching, both observed and unobserved, on optimal harvesting policy. When the regime is known optimal policies depend on the population level and which regime is currently active. When the regime is unobserved, on the other hand, the optimal policy depends on the population level and a belief distribution about the current regime. Optimal policies depend importantly on the degree of difference that exists between the regimes, the degree of population variability and the probability that a regime remains the same (resilience). In general when the probability of regime change is fixed, and hence is not affected by harvesting policy, the optimal policy is of the constant-escapement variety. Furthermore the optimal escapement level in the low productivity regime is lower than in the high productivity regime. When the regime switch probability depends on the stock level, however, the optimal policies tend to no longer be of the constant-escapement variety and can display considerable variability in qualitative behavior. It is possible, for example, that optimal escapement levels are very high in the low productivity regime (essentially harvest moratoria) in an effort to restore the stock to a level that will cause a shift to the high productivity regime. This proactive behavior arises when the probability of switching out of the low probability regime increases significantly with the population level. Similarly, as the probability of switching out of the high productivity regime becomes more sensitive to the population level it is optimal to engage in more precautionary behavior by increasing the escapement level and thereby to make switching less likely. Please cite this article in press as: Baggio, M., Fackler, P.L., Optimal management with reversible regime shifts. J. Econ. Behav. Organ. (2016), http://dx.doi.org/10.1016/j.jebo.2016.04.016

G Model

ARTICLE IN PRESS

JEBO-3791; No. of Pages 13

M. Baggio, P.L. Fackler / Journal of Economic Behavior & Organization xxx (2016) xxx–xxx

12

In general, the best action to take in the low productivity regime depends on the extent to which the switch probabilities can be influenced by the harvest decision. If the switch probability is very sensitive to the stock level and/or the cost of staying in the low productivity regime is high, it is optimal to close the harvest to give the stock the highest probability of recovery. Once the stock is in the low productivity regime the optimal decision depends critically on both the possibility of recovery and the effect that harvests decisions have on it. Perhaps more important is the optimal policy when the high productivity regime is operating. Although the optimal policy is generally close to a constant escapement policy, the ability to affect the probability of a regime switch by maintaining a higher level of escapement leads to a preventative policy that increases the target stock level. As would be expected, such behavior increases as the probability of switching increases and/or the probability of recovery decreases. Optimal policies when the regime is uncertain are close, although not equal, to a weighted average of the policies when the regime is certain, with the weights equal to the beliefs in the associated regimes. For population levels close to the harvest closure level the weighted average strategy is more precautionary than POMDP, while at higher levels the situation is reversed. Although they are not equal, differences are fairly small; the maximum gain from using the optimal strategy is about 2%. The fact that a reasonable strategy can be obtained using a belief weighted average of the perfect certainty strategy is useful because this strategy can be computed without the use specialized POMDP algorithms. It is not clear, however, under what circumstances such a result will occur and it would not be expected in situations in which actions have a large impact on beliefs. In particular, in situations with decisions concerning monitoring (possibly along with other decisions) the explicit modeling of the belief updating process that occurs in the POMDP framework is essential. Appendix A. Extended Partially Observable Markov Decision Processes The extended POMDP (xPOMDP) approach used in this paper was first introduced in Fackler and Pacifici (2014) and applied in Fackler et al. (2014). This appendix serves as a brief introduction to the basic idea and follows closely the more extended development in Sections 2 & 3 of Fackler and Pacifici (2014). A discrete Markov decision processes (MDPs) is defined by a tuple where S includes the values of one or more state variables, A includes the values of one or more decision (action) variables, R is a per period reward function that depends on S and A, P is a state transition probability matrix giving the probability associated with future values of the state, S+ , conditional on S and A and ı is a discount factor. The goal is to find a decision rule A (S) in order to maximize V(S; A) =

∞ 

ıt E[R(St , A(St ))|S0 = S]

(A1)

t=0

The Partially Observable Markov Decision Process (POMDP) was developed to address situations in which the state variables can not be directly observed (Monahan, 1982; Kaelbling et al., 1998). The POMDP framework supplements an MDP with an additional observation variable Y and its probability distribution conditioned on A and S+ . Although it is tempting to define a decision rule in terms of the observable variable, the dynamic system describing Y is generally not Markov and hence such an approach will result in sub-optimal decision. Instead the POMDP approach replaces the unobserved state with belief distribution B which is updated using the information contained in the observation variable. The goal is to find a decision rule A (B) that is a function of the belief states in order to maximize V (B; A) =

∞ 

ıt E[R(St , A(Bt ))|B0 = B]

(A2)

t=0

In the standard POMDP appoach all of the state variables are unobserved. Recently the Mixed Observability Markov Decision Process (MOMDP) was introduced by Ong et al. (2009, 2010) and discussed further by Araya-Lopez et al. (2010) and Chadès et al. (2012). Here the state variables are partitioned into those that are observed, O, and those that are unobserved, U. As with standard POMDPs a belief distribution B is defined that now represents beliefs concerning U. These beliefs are updated based on O, A, O+ and Y. The observational variable Y, however, is still conditioned on A and S+ . The xPOMDP framework essentially extends the MOMDP approach to allow for an arbitrary joint conditional probability distribution for O+ , U+ and Y conditioned on O, U and A. Thus, the xPOMDP approach is defined by the tuple where P now represents the joint probability matrix defining Pr(O+ , U + , Y |O, U, A). The goal is to find a decision rule A (O, B) that is a function of the observable states O and the belief states B concerning the unobservable states U in order to maximize V (O, B; A) =

∞ 

ıt E[R(St , A(Ot , Bt ))|O0 = O, B0 = B]

(A3)

t=0

The update rule for the belief state uses Bayes Rule and conditions on O+ and Y B+ (U + ) = P(U + |O, B, A, O+ , Y ) =

 P(O+ , U + , Y |O, U, A)B(U) U   + + U+

U

P(O , U , Y |O, U, A)B(U)

(A4)

Please cite this article in press as: Baggio, M., Fackler, P.L., Optimal management with reversible regime shifts. J. Econ. Behav. Organ. (2016), http://dx.doi.org/10.1016/j.jebo.2016.04.016

G Model JEBO-3791; No. of Pages 13

ARTICLE IN PRESS M. Baggio, P.L. Fackler / Journal of Economic Behavior & Organization xxx (2016) xxx–xxx

13

The xPOMDP approach allows more flexibility in the modelling process by allowing the observational variable to provide information on probabilities concerning the changes in the state variables rather than only in their levels. In the current context, a low population level is likely to come after a low population period regardless of the regime but a low population change is much more likely in the low productivity regime. References Alheit, J., Niquen, M., 2004. Regime shifts in the Humboldt Current ecosystem. Prog. Oceanogr. 60, 201–222. Araya-Lopez, M., Thomas, V., Buffet, O., Charpillet, F., 2010. A closer look at MOMDPs, ICTAI ’10. IEEE Computer Society, Washington, DC In: Proceedings of the 2010 22nd IEEE International Conference on Tools with Artificial Intelligence, vol 2, pp. 197–204. Baggio, M., 2015. Optimal fishery management with regime shifts: an assessment of harvesting strategies. Environ. Resour. Econ., http://dx.doi.org/10.1007/s10640-015-9906-0. Bond, C.A., Loomis, J.B., 2009. Using numerical dynamic programming to compare passive and active learning in the adaptive management of nutrients in shallow lakes. Can. J. Agric. Econ. 57, 555–573. ˜ Cahuin, S.M., Cubillos, L.A., Niquen, M., Escribano, R., 2009. Climatic regimes and the recruitment rate of anchoveta, Engraulis ringens, off Peru. Estuar. Coast. Shelf Sci. 84, 591–597. Carson, R.T., Granger, C.W.J., Jackson, J.B.C., Schlenker, W., 2009. Fisheries management under cyclical population dynamics. Environ. Resour. Econ. 42, 379–410. Chadès, I., Sabbadin, R., Carwardine, J., Martin, T.G., Nicol, S., Buffet, O., 2012. MOMDPs: a solution for modelling adaptive management problems. In: Twenty-Sixth AAAI Conference of the Association for the Advancement of Artificial Intelligence (AAAI-12), July 22–26, Toronto, Canada, pp. 267–273. Collie, J.S., Richardson, K., Steele, J.H., 2004. Regime shifts: can ecological theory illuminate the mechanisms? Prog. Oceanogr. 60, 281–302. Costello, C., Polasky, S., Solow, A., 2001. Renewable resource management with environmental predictions. Can. J. Econ. 34, 196–211. Clark, C.W., Kirkwood, G.P., 1986. On uncertain renewable resource stocks: optimal harvest policies and the value of stock surveys. J. Environ. Econ. Manage. 13, 235–244. De Zeeuw, A., Zemel, A., 2012. Regime shifts and uncertainty in pollution control. J. Econ. Dyn. Control 36, 939–950. Fackler, P.L., 2014. Structural and observational uncertainty in environmental and natural resource management. Int. Rev. Environ. Resour. Econ. 7, 109–139. Fackler, P., Haight, R.G., 2014. Monitoring as a partially observable decision problem. Resour. Energy Econ. 37, 226–241. Fackler, P.L., Pacifici, K., 2014. Addressing structural and observational uncertainty in resource management. J. Environ. Manage. 133, 27–36. Fackler, P.L., Pacifici, K., Martin, J., McIntyre, C., 2014. Efficient use of information in adaptive management with an application to managing recreation near golden eagle nesting sites. PLoS One 9 (8), e102434, http://dx.doi.org/10.1371/journal.pone.0102434. ˜ Fréon, P., Bouchon, M., Mullon, C., García, C., Niquen, M., 2008. Interdecadal variability of anchoveta abundance and overcapacity of the fishery in Peru. Prog. Oceanogr. 79, 401–412. Hauser, C.E., Possingham, H.P., 2008. Experimental or precautionary? Adaptive management over a range of time horizons. J. Appl. Ecol. 45, 72–81. ˜ ˜ event on the size and growth of the squid Loligo opalescens off Southern Nina Jackson, G.D., Domeier, M.L., 2003. The effects of an extraordinary El Nino/La California. Mar. Biol. 142, 925–935. Kaelbling, L.P., Littman, M.L., Cassandra, A.R., 1998. Planning and acting in partially observable stochastic domains. Artif. Intell. 101, 99–134. Monahan, G.E., 1982. A survey of partially observable Markov decision processes: theory, models, and algorithms. Manage. Sci. 28, 1–16. Ong, S.C.W., Png, S.W., Hsu, D., Lee, W.S., 2009. POMDPs for robotic tasks with mixed observability. Proceedings of Robotics: Science and Systems V. Ong, S.C.W., Png, S.W., Hsu, D., Lee, W.S., 2010. Planning under uncertainty for robotic tasks with mixed observability. Int. J. Robot. Res. 29, 1053–1068. Polasky, S., de Zeeuw, A., Wagener, F., 2011. Optimal management with potential regime shifts. J. Environ. Econ. Manage. 62, 229–240. Perrings, C.A., 1998. Resilience in the dynamics of economy-environment systems. Environ. Resour. Econ. 11, 503–520. Reed, W.J., 1979. Optimal escapement levels in stochastic and deterministic harvesting models. J. Environ. Econ. Manage. 6, 350–363. Ren, B., Polasky, S., 2014. The optimal management of renewable resources under the risk of potential regime shift. J. Econ. Dyn. Control 40, 195–212. Roughgarden, J., Smith, F., 1996. Why fisheries collapse and what to do about it? Proc. Natl. Acad. Sci. U. S. A. 93, 5078–5083. Sethi, G., Costello, C., Fisher, C., Hanemann, M., Karp, L., 2005. Fishery management under multiple uncertainty. J. Environ. Econ. Manage. 50, 300–318. Sakamoto, H., 2014. Dynamic resource management under the risk of regime shifts. J. Environ. Econ. Manage. 68, 1–19. Springborn, M., Sanchirico, J.N., 2013. A density projection approach for non-trivial information dynamics: adaptive management of stochastic natural resources. J. Environ. Econ. Manage. 66, 609–624.

Please cite this article in press as: Baggio, M., Fackler, P.L., Optimal management with reversible regime shifts. J. Econ. Behav. Organ. (2016), http://dx.doi.org/10.1016/j.jebo.2016.04.016