A structured war-gaming framework for managing extreme risks

Ecological Economics 116 (2015) 369–377

Contents lists available at ScienceDirect

Ecological Economics journal homepage: www.elsevier.com/locate/ecolecon

Methodological and Ideological Options

A structured war-gaming framework for managing extreme risks Shuang Liu a,b,⁎, Jean-Philippe Aurambout c, Oscar Villalta b,d, Jacqueline Edwards b,d, Paul De Barro a, Darren J. Kriticos a, David C. Cook b,e,f a

CSIRO Land and Water Flagship, GPO Box 1700, Canberra, ACT 2601, Australia The Plant Biosecurity Cooperative Research Centre, Australia European Commission, Joint Research Centre, Institute for Environment and Sustainability, Via Enrico Fermi 2749, I-21027 Ispra, Italy d Victoria Department of Environment and Primary Industries, Agribio, 5 Ring Road, La Trobe University, Bundoora, Vic 3083, Australia e Department of Agriculture and Food Western Australia, Locked Bag 4, Bentley Delivery Centre, WA 6983, Australia f School of Agricultural and Resource Economics, The University of Western Australia, 35 Stirling Highway, Crawley, Perth, WA 6009, Australia b c

a r t i c l e

i n f o

Article history: Received 23 June 2014 Received in revised form 29 April 2015 Accepted 9 May 2015 Available online xxxx Keywords: Risk planning systems Multi-criteria decision analysis Public participation Scenario analysis Plant biosecurity Game with a purpose

a b s t r a c t Extreme risks are challenging to learn from, prepare for and protect against, and they invite the development of new approaches to complement existing methods of risk management. We describe a systematic ex ante approach to support the strategic preparedness of risk management and apply it to a biosecurity case study. Our framework integrates a war-game model and a structured decision making approach. The model provides interactive maps that help stakeholders in visualizing the economic impacts of the extreme risk under different management scenarios, and it facilitates adaptive management by translating science-based results into stakeholder perspectives. The structured decision making approach not only offers an analytical structure to organize the multiple objectives of risk management, but also functions as a platform for group deliberation among alternative courses of management action with uncertain consequences. We found that this integration helped stakeholders develop a better understanding of the complexities and interconnectedness of the extreme risk management and reached a consensus regarding the most preferred management option. Crown Copyright © 2015 Published by Elsevier B.V. All rights reserved.

1. Introduction The risks posed by epidemics, stock market crashes, and massive avalanches are characterized by losses of huge magnitude but infrequent occurrence (Cox, 2012). Analyzing and managing such extreme risks are inherently difficult. The limited data we collect on these rare events is unlikely to be representative (Franklin et al., 2008). This lack of data often results in a tendency for policymakers to under-invest in protecting against these risks. When these disastrous events finally eventuate, people are likely to over-invest in response due to their lack of experience and cognitive errors (Noll, 1996). Extreme risks pose challenges for conventional models of risk analysis and risk management, and they invite development of new approaches to complement existing methods (Buchholz and Schymura, 2012). Historically, these risks have usually been managed on a piecemeal and ad hoc basis (Scott, 1996). But researchers are now beginning to develop systematic and ex ante approaches to improve the efficiency and effectiveness of managing extreme risks (Ermoliev et al., 2000). Strategic preparedness is an example of such an approach. ⁎ Corresponding author at: CSIRO Land and Water Flagship, GPO Box 1700, Canberra, ACT 2601, Australia. E-mail address: [email protected] (S. Liu).

http://dx.doi.org/10.1016/j.ecolecon.2015.05.004 0921-8009/Crown Copyright © 2015 Published by Elsevier B.V. All rights reserved.

Strategic preparedness is a decision-making process aimed at reducing consequences and controlling their likelihood to a level considered acceptable though decision makers' implicit and explicit acceptance of various risks and tradeoffs (Crowther et al., 2007). Having a multifaceted and multi-objective nature, strategic preparedness not only has to address the inherent interconnectedness and interdependencies among the sub-systems of an affected system, but also government agencies, the private sector, and communities must negotiate a host of conflicting and competing goals and objectives (Haimes, 2012). The success of this negotiation, however, might be hampered because multiple participants are likely to have disagreements over extreme risks due to knowledge gaps in understanding the affected system and in quantifying consequences (Bristow et al., 2012). To address this problem, the state of the art in decision science calls for a process of group deliberation that is important for building resilient communities (Cox, 2012). For extreme risks that have not yet eventuated, reasoned imagination is also critical, because it not only implies anticipating by systematic analysis scenarios but also recognizing and communicating potential impacts in a graphical format (Pate-Cornell, 2012). To our knowledge, there is no existing framework featuring a process of integrated group deliberation and graphical scenario planning in the arena of environmental risk management. In this paper, we fill this gap by developing such an integrated framework to support

370

S. Liu et al. / Ecological Economics 116 (2015) 369–377

Indifference & inaction before it happens

Low probability

Extreme risk

Strategic preparedness

Learning by ‘experiencing’

Interactive visualization & system dynamics

Graphical scenario planning

Structured war-gaming approach

Disorientation & overreaction when it happens

High impact

Making transparent & informed decisions

Structured decision-making

Group deliberation & tradeoff analysis

Fig. 1. Using the structured war-gaming approach to tackle the challenges of extreme risks.

strategic preparedness for managing extreme risks, and we apply it to a biosecurity case study. Our framework integrates a war-game model and a structured decision making approach (Fig. 1). The war-game model demonstrates the interconnectedness and interdependencies of system dynamics and enables users to visualize the outcomes of the simulated management scenario in an environment that is easy to explore and understand. Using an interactive map-based interface, the model helps decision makers in “learning by experiencing” the extreme risk before it happens. The structured decision making provides an analytical structure to assess conflicting objectives with the benefits of stakeholder participation and group deliberation (Proctor and Drechsler, 2006). Based on explicit tradeoff analysis, this approach is effective in facilitating groups to make transparent and informed decision (Hajkowicz, 2009) and helping them avoid disorientation and over-reaction when extreme risk happens (Liu et al., 2012). Even though only documenting a biosecurity case in this study, we believe this integrated framework can be applied to manage other type of extreme risks. 2. Background Biological invasions and natural disasters are similar phenomena in that their occurrences are too rare to be predictable and they can generate considerable damage (Ricciardi et al., 2011). These characteristics of extreme risks present challenges for using standard risk assessments that usually require prior estimates of model parameter. For example, there are often no or very limited existing data for parameterization, and when asked for subjective estimates of prior distributions, even experts tend to be overconfident (Burgman, 2005) and disagree with each other (Humair et al., 2014). As a result, these risk assessments are likely to be too inaccurate to be useful (Hulme, 2012). Our case study concerns fire blight, a disease caused by the bacterium Erwinia amylovora, which principally affects plants of the Rosaceae family (CABI and EPPO, 1997). It can cause considerable damage to both apples (Malus domestica) and pears (Pyrus communis), and since it is not currently found in Australia, the disease is considered a high priority threat to the Australian apple and pear industries (Biosecurity Australia, 2004). We applied the integrated framework to analyze the risk posed by fire blight to Victoria's Goulburn Valley (Fig. A.1 in Appendix A), one of Australia's major apple and pear production areas where approximately

60% of the nation's apple and 80% of the pear production takes place (HAL, 2004). The aim was to better prepare both industries for an incursion of fire blight by using a mock incursion scenario. Our decision question was “In the event of a fire blight incursion in the Goulburn Valley, is “Eradication”, “Containment” or “Live with it” the preferable management option, given the size of the incursion upon detection?” Table 1 below summarizes the differences and similarities of the three policy options. We elicited the information from a group of experts on fire blight or incursion management during a workshop and following faceto-face interviews with those who could not make it to the workshop. These characteristics of the three options function as the assumptions for both the war-game model and the structured decision making approach. In the Live with it option, major investment was made in activities to mitigate the effects of the fire blight. Under this alternative, we assumed that the disease could not be eradicated. Therefore, any attempts to locate and destroy it were minimized. For the Containment option, we also assumed that it was impossible to eradicate the fire blight. Most efforts focused on reducing the rate of its expansion from the infected orchards to surrounding areas with intensive surveillance and movement control (e.g. by stopping bees, the main vector of the fire blight, from flying out of a quarantine zone). In the Eradication option, management activities focused on searching for, destroying, and preventing the expansion of the disease.

Table 1 The differences and similarities among the three policy options. Eradication Containment Live with it Organized response program in place? Orchardists apply antibiotic treatment? Infested fruit trees being destroyed? A quarantine zone being established to stop beesa from flying out of it? a

Yes No Yes Yes

Bees are the main vector for the spread of fire blight.

Yes No No Yes

No Yes No No

Fig. 2. The value tree developed for managing a fire blight incursion (Program$: saving taxpayers' money; Industry$: decreasing cost to the apple and pear industries; Outrage: minimizing public outrage; Market Access: enhancing the industries' competitiveness; Diagnostic science: advancing science; Social disruption: minimizing social disruption; Environmental effect: minimizing side effects of control treatments).

S. Liu et al. / Ecological Economics 116 (2015) 369–377

371

Table 2 The consequence table of the structured decision making. Objective

Indicator

Unit

Eradication

Containment

Live with it

Decrease cost to the industries Save taxpayer's money Minimize public outrage Enhance the industries' competitiveness Minimize social disruption Minimize side-effects of treatment Advance science

Industry$ Program$ Outrage Market access Social disruption Environmental effects Diagnostic science

$ $ NA NA NA NA NA

8000 2,314,000 1 1 2 0 0

123,000 1,910,000 5 0 3 0 1

38,000 0 6 0 4 1 1

3. Methods 3.1. The war game model Invasive species modeling is a very active area of research, and there are three main areas where models are used extensively in the study of plant spread and its management: 1) identifying the key drivers of spread to better target management, 2) determining the role the spatial structure of landscapes plays in plant invasions, and 3) integrating management components to guide the implementation of control measures (Caplat et al., 2012). Among the existing studies, two lines of research are of particular relevance to the model we developed. The first group addresses the decision-support needs of policy makers by integrating established ecological models with economic management frameworks (Barbier, 2001; Cacho et al., 2008; Carrasco et al., 2010; Ceddia et al., 2009; Crepin et al., 2011; Epanchin-Niell et al., 2014; Hyder et al., 2008; Sharov and Liebhold, 1998). The second line of research has developed spatially explicit approaches using stochastic simulations that combine environmental variables and dissemination behaviors in order to characterize uncertainty in spread patterns over time (Carrasco et al., 2012; Epanchin-Niell and Hastings, 2010; Fernandes et al., 2014; Hastings et al., 2005; Meier et al., 2014; Rafoss, 2003; Touza et al., 2010; Yemshanov et al., 2009). Building on these modeling approaches, the interactive simulation model presented here not only takes into account system dynamics and conveys the natural variation of the system, it also provides a more open and transparent means of summarizing complex interactions between invasion processes and management efforts over time for a policy audience. This ‘war game’ model, was designed as an interactive and enjoyable decision support tool to facilitate strategic preparedness.1 Originating thousands of years ago in ancient times, war-gaming is a creative tool emerging out of military practices for the purposes of strategy testing, crisis planning and management, and training and education (Schwarz, 2013). Modern war-gaming embraces the fact that decision makers are almost always forced to make choices that are based on incomplete information. A single decision maker, no matter how visionary, has limited imagination; the function of the war-game is then to allow a group of decision makers to 'experience' the future in a risk-free environment and find answers to questions that they had not been aware of before they began the war game (Herman et al., 2008). This is exactly what our war-game model was designed for. Using a map-based interface, our aim was to allow stakeholders to control what level of management would be applied where and when to control simulated incursions. The tool enables users to visualize the outcomes of the simulated management scenario in an environment that is easy to explore and understand. Inspired by GWAP or games with a purpose technology (von Ahn, 2006; von Ahn and Dabbish, 2008), we attempted to engage stakeholders by leveraging their willingness and desire to play games. Fig. A.2 (Appendix A) presents the interface of the war game model that is composed of a map interface, a parameter section and a reporting interface. The map interface allows users to visualize the selected area of interest by displaying a land-use map where orchards are highlighted.

Users can directly interact with the spatial interface to initiate fire blight infections at specific locations, view detected infections or specify quarantine areas and destruction zones. The parameter interface allows users to specify and change model parameters at any time during the course of a simulation and to pause or start model runs. The users can also switch between different policy options and management strategies by changing parameter values. For example, one can switch from the policy option of Eradication to the option of Containment by setting the parameter of the range to destroy infested fruit trees to zero. Last, the reporting interface allows users to visualize temporal changes in variables of interest via graphic and numeric reporters. The main outputs of the model include the various economic costs (e.g. cost to the industry and management cost) and the infested production area. The Goulburn Valley contains approximately 6000 ha of apple and pear orchards. We applied the fire blight management decision support system to a subset of this region, located in the vicinity of the town of Shepparton covering an area of 2300 ha of pear and 1200 ha of apple orchards (Fig. A.1 in Appendix A). The land-use data input for the model was derived from the Victorian Land Use Information System (VLUIS) (Morse-McNabb, 2011) on which a more detailed apple and pear orchard layer was overlaid (obtained from Victorian Department of Environment and Primary Industries). The obtained vector dataset was converted to raster format using ESRI ArcGIS 10 and exported into NetLogo (Wilensky, 1999), a freely available multi-agent programmable modeling environment that was used to design the war game model.

3.2. The structured decision making approach Structured decision making is the collaborative and facilitated application of multiple objective decision making and group deliberation methods (Gregory et al., 2012). The iterative process of structured decision making breaks a complex process into 5 key steps: (1) identification of the overall goal (i.e., the decision to be made), (2) formulation of objectives (or criteria), (3) development of management options (or alternatives), (4) estimation of consequences associated with each alternative, and (5) evaluation of trade-offs and selection of preferred alternatives (Hammond et al., 1999; Hurley et al., 2009; Liu et al., 2010, 2012; Proctor and Drechsler, 2006; Walshe and Burgman, 2010). 100 80 60 Weight 40 Mean

20

Median

0

Standard deviation

Objective 1

This tool will be the subject of a separate publication and is only described briefly here.

Fig. 3. Weights elicited from the stakeholders using the method of swing weighting.

372

100 90 80 70 60 Preference 50 score 40 30 20 10 0

S. Liu et al. / Ecological Economics 116 (2015) 369–377

Market access Social disruption Outrage Environmental effects Industry$ Program$ Diagnostic science

Fig. 4. The preference ranking of the three policy options using the median group weight.

Three participatory workshops were held in November 2011, December 2011 and April 2012. The first or scoping workshop identified objectives for response programs of a potential fire blight incursion, and the eight participants of this workshop were mainly key industry stakeholders of the apple and pear industry. The second or expert workshop discussed candidate options of the response program and provided feedback on a draft war game model the project team had developed. The seven participants that attended this workshop were experts on fire blight or incursion management, and they examined the logic, core assumptions, and parameters of the war game model. After these workshops, individual interviews with industry stakeholders, managers, and experts were conducted in order to invite inputs from those who could not make it to the workshop and to marshal the best available information to populate a consequence table describing the performance of each option against each objective. Over 40 invitations to the final workshop were dispatched, and the invitees included representatives from the apple and pear industries based both in and outside Victoria, officers from industry liaison bodies, biosecurity managers from both Commonwealth and State/Territory governments, and scientists with expertise in fire blight. This group of participants was selected because they either would have a major influence on the response or would be heavily influenced by the decisions made, and Fig. A.3 in Appendix A presents a stakeholder map of the workshop participants. A total of 23 industry stakeholders, biosecurity managers, and scientists attended the final workshop in April 2012. Over the course of two days, participants were invited to review the formulation and characterization of the decision problem, to make value judgments on the importance of objectives, and to work towards a consensus on those judgments and the overall merit of candidate strategies via deliberation. At the end of the workshop, we conducted a sensitivity analysis to demonstrate how the preferred alternatives might change with changes in key model assumptions and performance scores.

infection was located in Shepparton East, chosen due to its high density of orchards, nurseries, and highways, (2) the infection first occurred in an apple orchard during the flowering season and then spread to other apple and pear orchards in the study area over time, and (3) the weather was not conducive for the disease to spread rapidly. After the presentation and subsequent discussion, every participant was asked to name their most preferred option from the three choices, Eradication, Containment or Live with it. The aim of this question was to investigate choice based only on their intuition and feelings, often referred to as System 1 thinking (Kahneman, 2003). We used Turning Point, an audience response system to collect and collate the immediate responses (Keepad Interactive, 2012). The participants were then assembled into small groups of five to six people to interact with the model using 52 in. NEC touch screen monitors in order to understand the activities and impacts associated with each policy option. The project team assigned the groups so that people with different backgrounds and expertise could mingle and learn from each other. In the last step on Day 1, all participants validated the structured decision making model that the project team developed based on the inputs of the previous workshops. Collectively, the stakeholders reviewed the objectives elicited from the scoping workshop and validated the consequence table. On Day 2 of the workshop, the project team elicited weights for the objectives using the method of swing weighting (Von Vinterfeldt and Edwards, 1986), a simple and intuitive approach with weights sensitive to the range of performance values. The use of these weights and the values in the consequence table enabled the project team to estimate the overall preference score for each policy option. The overall score for each option is the weighted average of its scores on all the objectives, according to a linear additive model (Catalyze, 2008; UK Department for Communities and Local Government, 2009). Letting the performance score for option i on objective j be presented by Sij and the weight for each objective by Wj, then n objective the overall score for each option, Si, is given by Eq. (1): Si ¼ W1Si1 þ W2Si2 þ … þ WnSin ¼

Xn j¼1

WiSij

ð1Þ

where Si is the overall preference score of each policy option. Through the use of the linear additive model, we transform diverse criteria, whether measured in a qualitative or quantitative manner into one common dimensionless scale of value, as expressed by Si. The rest of Day 2 was used to explore the robustness of the decision. The stakeholders were given more time to interact with the war game model with the aim of testing different incursion and management scenarios. This process was essentially an interactive sensitivity analysis to detect the influence of changing parameters and assumptions on the final ranking result. We used a multi-criteria decision analysis software called Hiview to run the structured decision making model and conduct the sensitivity analysis (Catalyze, 2008). Using each participant's own set of weights and the collectively validated in the consequence table, we calculated each individual's preference ranking for the three policy options. Each participant's most favorable option based on the structured decision making approach (System 2 thinking) was then compared with their earlier choice based on System 1 thinking.

3.3. The integrative modeling and structured decision making approach 4. Results Integration between the war game model and the structured decision making approach was carried out throughout the project, and this integration strengthened the effectiveness of the joint product of the structured war-gaming framework. The first two workshops (scoping and expert workshops) enabled the project team to elicit key inputs for the model and receive feedback about its effectiveness as a communication tool. During the final workshop, the war game model allowed stakeholders to explore various strategies of biosecurity management and visualize the impacts of the strategies using scenario analysis. On Day 1 of the final workshop, the project team first presented a scenario of the outbreak, which included the following features: (1) the initial

4.1. The multiple objectives in extreme risk management The participants of the scoping workshop considered the question of “What matters?” in the event of a fire blight incursion in the Goulburn Valley and identified 46 objectives. The objectives were summarized with eight fundamental or ends objectives, which are the basic things that matter or the outcomes the participants really care about. By comparison, the majority of the 46 objectives are means objectives that are related to specific methods of meeting the fundamental ones. Fig. B.1 in Appendix B presents a means–ends diagram of the objectives. The project team

S. Liu et al. / Ecological Economics 116 (2015) 369–377

refined the list to eight fundamental objectives by going through each of the 46 objectives and asking the question “Why is it important?” During the final workshop the fundamental objective of “Minimizing impacts to native host species” was eliminated from the final analysis because experts pointed out that the impact of fire blight on native plants is unlikely to be discernible (Biosecurity Australia, 2006). The fundamental objectives included four short-term (5 years) and three long-term (10 years) objectives. Fig. 2 below presents a value tree that depicts the decisions (which decision path: Eradication, Containment or Live with it) and the grouped objectives. The short-term objectives for fire blight management were (1) saving taxpayers' money (Program$), (2) decreasing cost to the apple and pear industries (Industry $), (3) minimizing public outrage (Outrage), and (4) Enhancing the industries' competitiveness (Market access). In the long-term, what mattered for the management program included (1) advancing science (Diagnostic science), (2) minimizing social disruption (Social disruption), and (3) minimizing side effects of control treatments (Environmental effect). Table B.1 in Appendix B lists the seven fundamental objectives and their corresponding indicators that were selected to evaluate the objectives. Of the seven fundamental objectives, only two were quantitative; the estimates of cost to industry and cost of response programs were derived from the war-game model. To account for the stochasticity inherent in the model, we ran 60 iterations for each of the three policy options to estimate the average cost figures. For the five qualitative objectives, we had to develop a scale ranging from two (yes vs. no) to seven tiers. Table 2 summarizes the consequences of each policy option that was evaluated (or scored) against the seven objectives. Using the technique of swing weighting (Von Vinterfeldt and Edwards, 1986), we elicited the weights for each of the seven objectives (an integer between 1 and 100) from each workshop participant. The majority of the workshop participants rated the objective of enhancing the industries' competitiveness as most important (Fig. 3). The second most important objective was to minimize economic costs to the industry, followed by the objective of minimizing social disruption. Median weights indicated that the next two objectives, saving taxpayers' money and minimizing public outrage, were ranked as equivalent importance. However, the standard deviation of the weights revealed that more people assigned a larger weight to the minimizing public outrage objective.

4.2. The effect of the structured war-gaming approach on extreme risk management Using the median of the objective weights, Fig. 4 shows the ranking of the three policy options as shown in Hiview. Eradication was the most preferred option for the participants, followed by Live with it, while Containment was the least preferred option (although there is no large difference between the scores of Containment and Live with it). Fig. 4 also explains the contribution of each objective's cumulative weight (WnSi in Eq. (1)) to the overall preference score of that option. The group's preference for Eradication is primarily due to the high cumulative weight of the market share objective (Fig. 3). The majority of the participants assigned 100 points to this objective, and Eradication is the only policy option that makes regaining market share possible, according to the consequence table (Table 2). The ranking remained the same as we varied the consequence scores during the interactive sensitivity analysis. This process demonstrated the robustness of the decision and showed that Eradication was the most preferred policy option, given the uncertainty associated with the scores. Using each participant's own set of weights and the collectively agreed values in the consequence table, we also calculated each individual's preference ranking for the three policy options. Eradication was the most preferred decision path for all 23 participants. By comparison, about 76% of the group nominated Eradication and the rest of the group selected Containment at the beginning of the workshop.

373

5. Discussion 5.1. Two systems of thinking and preparing for extreme risks Daniel Kahneman (2003) distinguished two general systems of thinking, which he called System 1 and System 2. The operations of System 1 are fast, automatic, effortless, associative, and difficult to control or modify. The operations of System 2 are slower, serial, effortful, and deliberately controlled. One implication of this theory is that people are not always the rational utility maximizers portrayed by expected utility theory (Kahneman and Tversky, 1979). Instead, they routinely make use of simplified rules of thumb or “heuristics” and this is particularly true when people have to cope with the complex cognitive demands associated with responding to uncertain situations (Tversky and Kahneman, 1974). Sometimes these rules of thumb help us by offering a degree of efficiency that allows us to make the many small decisions we need to make every day. However, jumping to a conclusion by only resorting to System 1 thinking fails miserably when the situation is unfamiliar, the stakes are high and there is no time to collect more information (Michel-Kerjan and Slovic, 2011). These are indeed the features of the extreme risks. Characterized by low probability and high impacts, extreme risks place high cognitive demands on people and present major risk management challenges (Cox, 2012). Before a disaster happens, there is indifference and inaction, and the excuse is that these events are so rare as to be unimaginable (Pate-Cornell, 2012). When the disaster strikes at high speed and with large-scale destruction, the suddenness can paralyze and its large-scale can disorient (Michel-Kerjan, 2008). These are the circumstances in which intuitive errors are probable, but may be prevented by a deliberate intervention of System 2 thinking. In our case study, we elicited people's immediate response by asking them to name their most preferred policy option after the introduction of a mock fire blight incursion scenario. With the help of the audience response system (Keepad Interactive, 2012), we collected every participant's answer in less than 30 seconds. Due to the lack of time for analytical thinking and information gathering, the group had to use System 1 and relied on their gut feelings to provide an intuitive answer. Eradication was the most favored option for about 76% of the participants, and the rest of the group selected Containment. During the course of the next day, we applied the structured wargaming approach to help the group develop their System 2 answer. The structured decision-making approach assessed the impacts of each policy option and addressed the trade-offs among the multiple objectives that matter for selecting the decision path. The war game model demonstrated the inter-connectedness of the subsystems of the socio-ecological system of the fire blight incursion. The interactive spatial maps also provided a visual aid for the participants to understand the potential spread and impact of the fire blight incursion. Using everyone's own set of weights and the collectively agreed consequence values, we calculated each participant's preference ranking. Eradication became the most preferred policy option for all stakeholders when they used System 2 thinking to make their judgment, and a consensus was reached regarding the decision path for managing the risk posed by fire blight. 5.2. Contribution of the structured war-gaming approach to extreme risk management Our structured war-gaming approach proved to be effective in the sense that it changed approximately a quarter of the participants' most preferred policy option from Containment to Eradication. We believe two interlinked features of this approach were particularly important for achieving this effectiveness. The first is the provision of interactive visual aids for scenario planning, and the second is the provision of a platform for group deliberation and social learning. The visualization was designed not only for grabbing participants' attention but also for helping them ‘experience’ impacts of extreme risk. The latter is critical because a robust policy response must appreciate both the

374

S. Liu et al. / Ecological Economics 116 (2015) 369–377

external and internal dimensions of risks: the former is usually based on scientific risk analysis, performed by experts, of system characteristics of the socio-ecological world; and the latter acknowledges that to be real, danger has to be either experienced or perceived — it is the individual or collective experience or perception of insecurity that constitutes the risk (Dessai et al., 2004). The importance of internal risks underscores the need for scientists to transfer knowledge effectively. The war game model enables the participants to visualize the outcomes of the simulated management scenarios, and this visualization helps turn cold non-spatial numbers (Table 2) into vivid spatially-explicit pictures. The interactive scenario analysis encourages the participants to weight the possible benefits and costs of each policy option as outlined by the structured decision making approach. Extreme risk presents an unstructured challenge in the sense that both the science and the values surrounding the issue are likely to be contested (e.g. climate change). One important way to address an unstructured issue is to engage in dialog, and the purpose of such dialog is to engage in scientific and social learning, and to understand the problems from different perspectives in a non-political setting (Gupta and van Asselt, 2006). The structured decision making approach provides a platform for different stakeholders to interact in the process of assessing and communicating and, eventually, making risk management decisions. These two key features are interlinked in the sense that the role of the model is to serve as an aid to ‘learning-by-experiencing (Groot and Rossing, 2011)’ and structured decision making, not to give ‘the right answer’. The model represents the collective view of the group at any point during its generation and modification, and serves as a means to examine the impact of differences in perspective or vagueness in the data. Because the model is projected for all participants to see as it is created, it is less likely to be perceived by participants as a ‘black box’. This transparency helps to build confidence in model results (Phillips and Bana e Costa, 2007). In our case, the capacity to alter model settings (e.g. parameters, the initial location of the incursion and the number of starting locations) and visualize the outcomes accordingly allowed users to quickly gain an understanding of the model behavior and the implications of different management strategies. The use of the touch screen medium complementing the model graphic user interface was particularly valuable for encouraging engagement and promoting group investigations and discussions of fire blight management scenarios. 6. Conclusions and future directions Extreme risks are notoriously difficult to prepare for and protect against, because they can prompt too little concern before an event

happens and too much afterwards (Sunstein and Zeckhauser, 2011). We developed the structured war-gaming approach to support strategic preparedness for managing extreme risks and applied it in the case of biosecurity management. The methodology allows the participation of experts, biosecurity managers, and industry representatives to collectively refine system models. We argue that strategic preparedness for managing extreme risks should not only be grounded scientifically but be expanded to include methods that address value issues concerning how stakeholders perceive, understand, and make decisions on how to assess and manage risk. ‘Risks as experienced’ — warrants at least as much attention as ‘risk as defined’ (Dessai et al., 2004). The feedback we received from our participants showed that during the course of the workshop they were motivated to find a scientifically valid and socially viable policy option in the event of the mock fire blight incursion. They also found the process not only illuminating but also enjoyable. The workshop was refreshing compared to their past experience where the stakeholders had been informed that they had to accept a ‘correct’ decision that was prescribed by outsiders. In spite of the positive feedback, we wish to emphasize the necessity of refining and testing the structured war-gaming approach before it can be applied in managing extreme risks in the real world. In future research we have the vision of building a tool box for extreme risk management where different tools will be featured and fit for different purposes. For example, using an on-line survey to elicit weights for structured decision making requires less administrative time and canvases a broader sample of society (Liu et al., 2013). Recent development in using spatial agent-based modeling approach to investigate the emergence of cooperative behaviors in dynamic resource management is also a promising direction for our future work (Touza et al., 2013).

Acknowledgments The authors would like to acknowledge the support of the The Plant Biosecurity Cooperative Research Centre, established and supported under the Australian Government's Cooperative Research Centres Program. We would also like to thank the support from CSIRO and the Victorian Department of Environment and Primary Industries. We are deeply indebted to all those who participated in all of our workshops. We also thank David Stern and Stuart Whitten for their comments on an earlier version of the paper.

Appendix A

Fig. A.1. Focus area of the fire blight incursion case study.

S. Liu et al. / Ecological Economics 116 (2015) 369–377

375

Fig. A.2. Interface of the spatial interactive decision facilitation tool. The map interface allows users to visualize the selected area of interest by displaying a land-use map where orchards are highlighted over a white background. Apple orchards are displayed in green and pear orchards in yellow. Users can also directly interact with the spatial interface to initialise fire blight infections at specific locations, view detected infections (appearing in red) or specify quarantine areas (displayed in gray) and destruction zones (displayed in black). The parameter interface allows users to specify and change model parameters at any time during the course of a simulation, to define and apply management strategies and to pause or start model runs. The reporting interface allows user to visualize temporal changes in variables of interest via graphic and numeric reporters. Finally, a data export option allows weekly map outputs to be saved in GIS format for later analysis.

Influence

Australiangovernment’s Department of Agriculture, Fisheries and Forestry

Plant Health Australia

Victoria Dept. of primary industry

Horticulture Australia New South Wales Dept. of primary industry

Corporate Research Centre Plant Biosecurity Private consultants

Queensland Dept. of employment, economic development and innovation

Apple & Pear Australia Ltd.

Fruit growers Victoria Limited

SPC Ardomona Ltd.

Fruit growers Tasmania

Interest

Fig. A.3. The background, influence and interest of the participants of the final workshop.

376

S. Liu et al. / Ecological Economics 116 (2015) 369–377

Appendix B

Fig. B.1. The means–ends diagram for the decision on fire blight incursion management, showing 46 means objectives (rectangular shapes) elicited from the scoping workshop and the eight fundamental objectives (oval shapes). Table B.1 The objectives and their indicators used in the structured decision-making process. Objective

Indicator

Unit

Definition and scale of indicators

Decrease cost to the industries Save taxpayer's money Minimize public outrage

Industry$ Program$ Outrage

$ $ NA

Enhance the industries' competitiveness

Market access

NA

Minimize social disruption

Social disruption

NA

Minimize side-effects of treatment

Environmental effects

NA

Advance science

Diagnostic science

NA

Cumulative cost to the industry for 5 years due to loss of productivity Cumulative cost of a response program for 5 years, which is the sum of management cost and owner reimbursement cost 5 years after the decision point, the general public attitudes towards a response program (in the case of eradication and containment) or lack of such a program (in the case of LWI) 7: Negative media coverage in national media and public protests 6: Negative media coverage only in national media 5: Organized groups voice their concern 4: Individual submissions of concern to organizations 3: Letter to the editors in local newspaper 2: You hear gossips in passing 1: No problem 5 years after the decision point, whether the orchardists can regain market access for both domestic and international markets 1: Yes 0: No 10 years after the decision point, the potential negative impacts on the orchardists in the Goulburn Valley 5: Significant suffering (e.g. existing the business) for large number of people (over 50% of orchardists) 4: Minor suffering (e.g. losing experienced workers) for a large number of people (over 50% of orchardists) 3: Significant suffering for a small number of people (less than 50% of orchardists) 2: Minor suffering for a small number of people (less than 50% of orchardists) 1: No loss for anybody 10 years after the decision point, the cumulative side-effects of chemical and antibiotic treatments 1: Yes 0: No 10 years after the decision point, the progress will have been made in advancing diagnostic technology for the fire blight. 1: Progress made 0: No progress

S. Liu et al. / Ecological Economics 116 (2015) 369–377

References Australia, Biosecurity, 2006. Final Import Risk Analysis Report for Apples From New Zealand, Part C. Biosecurity Australia, Canberra. Barbier, E.B., 2001. A note on the economics of biological invasions. Ecol. Econ. 39, 197–202. Biosecurity Australia, 2004. Importation of Apples from New Zealand — Revised Draft IRA Report. In: Australia, B. (Ed.), Department of Agriculture, Fisheries and Forestry, p. 533. Bristow, M., Fang, L., Hipel, K.W., 2012. System of systems engineering and risk management of extreme events: concepts and case study. Risk Anal. 32, 1935–1955. Buchholz, W., Schymura, M., 2012. Expected utility theory and the tyranny of catastrophic risks. Ecol. Econ. 77, 234–239. Burgman, M.A., 2005. Risks and Decisions for Conservation and Environmental Management. Cambridge University Press. CABI, EPPO, 1997. Data Sheets on Quarantine Pests: Erwinia amylovora. In: Organization, E.a.M.P.P. (Ed.), (http://www.eppo.int/QUARANTINE/bacteria/Erwinia_amylovora/ ERWIAM_ds.pdf). Cacho, O.J., Wise, R.M., Hester, S.M., Sinden, J.A., 2008. Bioeconomic modeling for control of weeds in natural environments. Ecol. Econ. 65, 559–568. Caplat, P., Coutts, S., Buckley, Y.M., 2012. Modeling population dynamics, landscape structure, and management decisions for controlling the spread of invasive plants. Year Ecol. Conserv. Biol. 1249, 72–83. Carrasco, L.R., Baker, R., MacLeod, A., Knight, J.D., Mumford, J.D., 2010. Optimal and robust control of invasive alien species spreading in homogeneous landscapes. J. R. Soc. Interface 7, 529–540. Carrasco, L.R., Cook, D., Baker, R., MacLeod, A., Knight, J.D., Mumford, J.D., 2012. Towards the integration of spread and economic impacts of biological invasions in a landscape of learning and imitating agents. Ecol. Econ. 76, 95–103. Catalyze, 2008. Hiview 3.2.0.7. Ceddia, M.G., Heikkilä, J., Peltola, J., 2009. Managing invasive alien species with professional and hobby farmers: insights from ecological–economic modelling. Ecol. Econ. 68, 1366–1374. Cox, J.L.A., 2012. Community resilience and decision theory challenges for catastrophic events. Risk Anal. 32, 1919–1934. Crepin, A.-S., Norberg, J., Maler, K.-G., 2011. Coupled economic–ecological systems with slow and fast dynamics — modelling and analysis method. Ecol. Econ. 70, 1448–1458. Crowther, K.G., Haimes, Y.Y., Taub, G., 2007. Systemic valuation of strategic preparedness through application of the inoperability input–output model with lessons learned from Hurricane Katrina. Risk Anal. 27, 1345–1364. Dessai, S., Adger, W.N., Hulme, M., Turnpenny, J., Kohler, J., Warren, R., 2004. Defining and experiencing dangerous climate change — an editorial essay. Clim. Chang. 64, 11–25. Epanchin-Niell, R.S., Hastings, A., 2010. Controlling established invaders: integrating economics and spread dynamics to determine optimal management. Ecol. Lett. 13, 528–541. Epanchin-Niell, R.S., Brockerhoff, E.G., Kean, J.M., Turner, J.A., 2014. Designing costefficient surveillance for early detection and control of multiple biological invaders. Ecol. Appl. 24, 1258–1274. Ermoliev, Y.M., Ermolieva, T.Y., MacDonald, G.J., Norkin, V.I., Amendola, A., 2000. A system approach to management of catastrophic risks. Eur. J. Oper. Res. 122, 452–460. Fernandes, R.F., Vicente, J.R., Georges, D., Alves, P., Thuiller, W., Honrado, J.P., 2014. A novel downscaling approach to predict plant invasions and improve local conservation actions. Biol. Invasions 16, 2577–2590. Franklin, J., Sisson, S.A., Burgman, M.A., Martin, J.K., 2008. Evaluating extreme risks in invasion ecology: learning from banking compliance. Divers. Distrib. 14, 581–591. Gregory, R., Failing, L., Harstone, M., Long, G., McDaniels, T., Ohlson, D., 2012. Structured Decision Making: A Practical Guide to Environmental Management Choices. WileyBlackwell, West Sussex, UK. Groot, J.C.J., Rossing, W.A.H., 2011. Model-aided learning for adaptive management of natural resources: an evolutionary design perspective. Methods Ecol. Evol. 2, 643–650. Gupta, J., van Asselt, H., 2006. Helping operationalise Article 2: a transdisciplinary methodological tool for evaluating when climate change is dangerous. Glob. Environ. Chang. 16, 83–94. Haimes, Y.Y., 2012. Strategic preparedness for recovery from catastrophic risks to communities and infrastructure systems of systems. Risk Anal. 32, 1834–1845. Hajkowicz, S., 2009. Cutting the cake: supporting environmental fund allocation decisions. J. Environ. Manag. 90, 2737–2745. HAL, 2004. The Australian Horticulture Statistics Handbook. Horticulture Australia Limited, Sydney. Hammond, J.S., Keeney, R.L., Raiffa, H., 1999. Smart Choices: A Practical Guide to Making Better Decisions. Harvard Business School Press, Boston. Hastings, A., Cuddington, K., Davies, K.F., Dugaw, C.J., Elmendorf, S., Freestone, A., Harrison, S., Holland, M., Lambrinos, J., Malvadkar, U., Melbourne, B.A., Moore, K., Taylor, C., Thomson, D., 2005. The spatial spread of invasions: new developments in theory and evidence. Ecol. Lett. 8, 91–101. Herman, M., Frost, M., Kurz, R., 2008. Wargaming for Leaders: Strategic Decision Making From the Battlefield to the Boardroom. McGraw-Hill, USA.

377

Hulme, P.E., 2012. Weed risk assessment: a way forward or a waste of time? J. Appl. Ecol. 49, 10–19. Humair, F., Edwards, P.J., Siegrist, M., Kueffer, C., 2014. Understanding misunderstandings in invasion science: why experts don't agree on common concepts and risk assessments. NeoBiota 20, 1–30. Hurley, M., Lowell, K., Cook, D., Liu, S., Siddique, A.-B., Diggle, A., 2009. How Model Outputs can be Used to Improve Decision-Making: Using Bio-Economic Model Outputs in Deliberative Multi-Criteria Evaluation to Prioritize Invasive Pest Species. Hyder, A., Leung, B., Miao, Z.W., 2008. Integrating data, biology, and decision models for invasive species management: application to leafy spurge (Euphorbia esula). Ecol. Soc. 13, 1–12. Kahneman, D., 2003. A perspective on judgment and choice — mapping bounded rationality. Am. Psychol. 58, 697–720. Kahneman, D., Tversky, A., 1979. Prospect theory — analysis of decision under risk. Econometrica 47, 263–291. Keepad Interactive, 2012. Audience Response System. Liu, S., Proctor, W., Cook, D., 2010. Using an integrated fuzzy set and deliberative multicriteria evaluation approach to facilitate decision-making in invasive species management. Ecol. Econ. 69, 2374–2382. Liu, S., Walshe, T., Long, G., Cook, D., 2012. Evaluation of potential responses to invasive non-native species with structured decision making. Conserv. Biol. 26, 539–546. Liu, S., Crossman, N.D., Nolan, M., Ghirmay, H., 2013. Bringing ecosystem services into integrated water resources management. J. Environ. Manag. 129, 92–102. Meier, E.S., Dullinger, S., Zimmermann, N.E., Baumgartner, D., Gattringer, A., Huelber, K., 2014. Space matters when defining effective management for invasive plants. Divers. Distrib. 20, 1029–1043. Michel-Kerjan, E.O., 2008. Toward a new risk architecture: the question of catastrophe risk calculus. Soc. Res. 75, 819–854. Michel-Kerjan, E.O., Slovic, P., 2011. The Irrational Economist: Making Decisions in a Dangerous World. Public Affairs, New York. Morse-McNabb, E., 2011. The Victorian Land Use Information System (VLUIS): A New Method for Creating Land Use Data for Victoria, Australia, Surveying and Spatial Sciences Biennial Conference, Wellington, New Zealand. pp. 155–167. Noll, R., 1996. The complex politics of catastrophe economics. J. Risk Uncertain. 12, 141–146. Pate-Cornell, E., 2012. On “Black Swans” and “Perfect Storms”: risk analysis and management when statistics are not enough. Risk Anal. 32, 1823–1833. Phillips, L.D., Bana e Costa, C.A., 2007. Transparent prioritisation, budgeting and resource allocation with multi-criteria decision analysis and decision conferencing. Ann. Oper. Res. 154, 51–68. Proctor, W., Drechsler, M., 2006. Deliberative multicriteria evaluation. Environ. Plann. C Gov. Policy 24, 169–190. Rafoss, T., 2003. Spatial stochastic simulation offers potential as a quantitative method for pest risk analysis. Risk Anal. 23, 651–661. Ricciardi, A., Palmer, M.E., Yan, N.D., 2011. Should biological invasions be managed as natural disasters? Bioscience 61, 312–317. Schwarz, J.O., 2013. Business wargaming for teaching strategy making. Futures 51, 59–66. Scott, K.E., 1996. Social treatment of catastrophic risk — introduction. J. Risk Uncertain. 12, 99–100. Sharov, A.A., Liebhold, A.M., 1998. Bioeconomics of managing the spread of exotic pest species with barrier zones. Ecol. Appl. 8, 833–845. Sunstein, C.R., Zeckhauser, R., 2011. Overreaction to fearsome risks. Environ. Resour. Econ. 48, 435–449. Touza, J., Drechsler, M., Johst, K., Dehnen-Schmutz, K., 2010. The Role of Space in Invasive Species Management. Touza, J., Drechsler, M., Smart, J.C.R., Termansen, M., 2013. Emergence of cooperative behaviours in the management of mobile ecological resources. Environ. Model Softw. 45, 52–63. Tversky, A., Kahneman, D., 1974. Judgment under uncertainty — heuristics and biases. Science 185, 1124–1131. UK Department for Communities and Local Government, 2009. Multi-Criteria Analysis: A Manual, London. von Ahn, L., 2006. Games with a purpose. Computer 39, 92–94. von Ahn, L., Dabbish, L., 2008. Designing games with a purpose. Commun. ACM 51, 58–67. Von Vinterfeldt, D., Edwards, W., 1986. Decision Analysis and Behavioral Research. Cambridge University Press, Cambridge, UK. Walshe, T., Burgman, M., 2010. A framework for assessing and managing risks posed by emerging diseases. Risk Anal. 30, 236–249. Wilensky, U., 1999. NetLogo. Center for Connected Learning and Computer-Based Modeling. Northwestern University, Evanston, IL. Yemshanov, D., Koch, F.H., McKenney, D.W., Downing, M.C., Sapio, F., 2009. Mapping invasive species risks with stochastic models: a cross-border United States–Canada application for Sirex noctilio Fabricius. Risk Anal. 29, 868-884.