An integrated modelling toolbox for water resources assessment and management in highland catchments: Sensitivity analysis and testing

AGRICULTURAL SYSTEMS Agricultural Systems 89 (2006) 132–164 www.elsevier.com/locate/agsy

An integrated modelling toolbox for water resources assessment and management in highland catchments: Sensitivity analysis and testing R.A. Letcher

a,c,*

, B.F.W. Croke a,c, W.S. Merritt A.J. Jakeman a,b

a,d

,

a

Integrated Catchment Assessment and Management Centre, The Australian National University, Canberra, ACT 0200, Australia b Centre for Resource and Environmental Studies, The Australian National University, Canberra, ACT 0200, Australia c Department of Mathematics, The Australian National University, Canberra, ACT 0200, Australia d School of Resources, Environment and Society, Building 48a, The Australian National University, Canberra, ACT 0200, Australia Received 17 March 2005; received in revised form 21 June 2005; accepted 23 August 2005

Abstract An earlier companion paper developed a framework for river basin management to assess trade-offs among economic, social and environmental factors and their spatiotemporal variability (Letcher, R.A., Merritt, W.S., Jakeman, A.J., Croke, B.F., this issue. An integrated modelling toolbox for water resources assessment and management in Northern Thailand: model description, Agricultural Systems). That paper discussed an integrated modelling toolbox (IMT) that has been developed for highland catchments with specific application to the Mae Chaem catchment in Northern Thailand. The aim of the IMT is to provide information on these types of trade-offs. The toolbox uses a scenario modelling approach that translates policy and uncontrollable drivers into scenario inputs to the biophysical toolbox, a component of the integrated modelling toolbox. The biophysical toolbox (BPT) outputs indicators *

Corresponding author. Tel.: +61 2 6125 8132; fax: +61 2 6125 8395. E-mail address: [email protected] (R.A. Letcher).

0308-521X/$ - see front matter Ó 2005 Elsevier Ltd. All rights reserved. doi:10.1016/j.agsy.2005.08.007

R.A. Letcher et al. / Agricultural Systems 89 (2006) 132–164

133

from models of crop growth, erosion and rainfall-runoff, whereas socioeconomic decisionmaking and impact models are complementary components of the IMT. This paper presents results from a sensitivity analysis of the model to changes in various input assumptions. These results show that policy recommendations based on socioeconomic or flow impacts are not likely to be affected by small levels of uncertainty in prices or costs, or recent variability in climate. Erosion is the most sensitive output, and is strongly sensitive to a change in price. But the direction of change in erosion appears to be consistent across different price assumptions. Overall the model shows plausible levels and patterns of sensitivity to changes in input parameters. Ó 2005 Elsevier Ltd. All rights reserved. Keywords: Integrated model; IWRAM; Socioeconomic modelling; Trade-offs; Water resource assessment and management

1. Introduction Throughout the world, deforestation, agricultural intensification and the associated competition for water resources produce environmental, economic and social impacts. In parts of the developing world these problems are often more striking because of faster growing populations and the associated rapidity of the manifested problems. In Northern Thailand, agricultural expansion has produced competition for water at various scales and has resulted in erosion problems, downstream water quality deterioration, groundwater depletion, biodiversity loss, and shifts in the distribution of economic and social well-being. The monsoonal nature of rainfall also intensifies demand for water in the dry season and, with the accompanying seasonal shift in flow regimes from dam regulation, especially at larger scales where this influence is more considerable, exacerbates problems with instream biodiversity and habitat. In 1997, the Thai government adopted a ÔPolicy and Prospective Plan for Enhancement and Conservation of Natural Environmental Quality, 1997–2016Õ (UN, 2002). One major commitment relating to surface water resources in this Plan is to develop and conserve surface and groundwater sources at the basin level, taking into account socioeconomic and environmental impacts (OEPP, 2002). Further details on water management in Thailand can be found in Sethaputra et al. (2002). The Integrated Water Resource Assessment and Management (IWRAM) project (Jakeman et al., 1997) has been developing an integrative methodology and associated software toolbox, the integrated modelling toolbox (IMT) to assess these natural resource management issues by exploring the economic, environmental and sociocultural implications of different levels and patterns of cultivation and other water use in a representative catchment. The focus has been on working at the subcatchment scale (100 km2) in the Mae Chaem (Fig. 2 in Letcher et al., this issue) catchment (4000 km2), principally with the Royal Project Foundation and Land Development Department (LDD) in Thailand, to provide them with a toolbox to

134

R.A. Letcher et al. / Agricultural Systems 89 (2006) 132–164

assist with their land-use planning activities. With respect to issues, initial attention has been given to the spatiotemporal distribution of water supply, erosion, rice production and farm income throughout case study catchments in relation to input drivers such as climate, commodity prices, technological improvements, government regulations and investments. Necessarily the toolbox requires integration of various disciplinary contributions including agronomy, climate, economics, hydrology and soil science. The methodology has to be able to incorporate information of different kinds – spatial, time-series, financial, political and cultural. To focus this diverse information, the unifying factor is the scenario. A Ôdata ! scenario ! modelling ! indicatorsÕ framework was adopted. This framework allows scenarios to be generated as inputs to the toolbox and a range of biophysical and socioeconomic indicators to be produced as outputs. A scenario based approach was preferred to an overall optimisation approach (e.g., Schluter et al., 2005) for its ability to promote system learning more directly, and its flexibility with respect to management objectives. Rather than specify a subjective Ôbest outcomeÕ to be optimised, it was desired to learn about the system and model behaviour from scenario runs. A scenario based approach also allows the potential to be eclectic about management objectives such as seeking a solution based on minimising the risk of a bad decision (Pallottino et al., 2005). This paper details sensitivity analysis and testing of the integrated modelling toolbox. This toolbox comprises biophysical modelling tools (erosion, crops, hydrology), referred to as the biophysical toolbox, and socioeconomic decision and impact models. The results shown in this paper relate to the Mae Uam subcatchment of the Mae Chaem. For a detailed description of the Mae Uam catchment and water issues in the catchment, see Merritt et al. (2004). 2. Overview of the integrated modelling toolbox The IMT consists of a number of modelling components as outlined in Letcher et al. (this issue): socioeconomic decision making models; a decision disaggregation model; a biophysical toolbox (BPT); and, a socioeconomic impact simulation model. The BPT consists of a crop model, CATCHCROP (Perez et al., 2002), a hydrological modelling component (Croke et al., 2004) based on the IHACRES rainfall-runoff model (Jakeman and Hornberger, 1993), a water allocation model based on stakeholder recommendations for irrigation priority in the catchment, and an erosion model, based on a version of the Universal Soil Loss Equation (Wischmeier and Smith, 1978) modified for Thailand. A full description of the biophysical toolbox can be found in Merritt et al. (2004). The IMT and links to the biophysical toolbox are described in detail in Letcher et al. (this issue). This section provides a brief overview of the component models, their interactions and key parameters in the IMT. A summary of key parameters from each of the component models that are tested in the sensitivity analysis is given in Table 1. This table also summarises the scenario numbers used to describe the sensitivity tests. The component models and their links are shown in Fig. 1 and discussed below.

R.A. Letcher et al. / Agricultural Systems 89 (2006) 132–164

135

Table 1 Component model parameters tested Symbola

Name

Model component

Description

Scenario numbers

hm

Households

DDM, SISM

1–5

pm

People

SISM

p

RiceCost

SISM

pi

Price i

HDM

Number of households of RMU type m Number of people in each household of RMU type m Cost per unit of buying additional rice to meet the rice deficit Price received for each unit of crop i produced

ci

Cost i

HDM

Variable cost for each unit of area of crop i planted

w

Wage

HDM

H

Labour

HDM

X

Water

HDM

Au, Ap, A

Land

HDM, DDM

Yi

Expected yield for crop i

HDM

Wi

Expected water use for crop i

HDM

Wage rate both paid to onfarm employees sourced from outside the family, and to family members working off-farm Hours of family labour available in each season for work on or off-farm Expected water available for extraction in each year. This is user defined in the first year only, then is updated by the model Paddy and upland area available for agriculture to each household Expected yield for crop i, affects expected household income that the planting decision is based on (referred to as Y in the text) Expected water demand for crop i, affects the constraints on the planting decision (referred to as W) in the text

a

6–10 11–15

Paddy rice: 16–20 Upland rice: 21–25 Sorghum: 26–30 Groundnut: 31–35 Paddy rice: 36–40 Upland rice: 41–45 Sorghum: 46–50 Groundnut: 51–55 56–60

61–65

66–70

71–75

Y value (all scenarios run for each level of Y)

W value (all scenarios run for each level of W)

Symbols correspond to those used in Letcher et al. (this issue).

2.1. Household decision models The household decision models (HDM) simulate wet- and dry-season cropping decisions using a linear programming formulation to optimise expected annual

136

R.A. Letcher et al. / Agricultural Systems 89 (2006) 132–164

Fig. 1. Framework and interactions between component models in the integrated modelling toolbox. Models interacting with the BPT models are household decision models (HDM), the decision disaggregation model (DDM) and the socioeconomic impact simulation model (SISM).

household income. Calculations are performed on a Ôper household per RMU basisÕ. Expected household income depends on expected gross margins (prices, variable costs, expected yields), the cost of hiring additional labour on-farm and income earned off-farm. Decisions relate to the area to be planted to each crop (irrigated and/or rainfed) and the level of fertiliser to be applied, as well as the hours that will be worked off-farm and the hours of labour to be hired on-farm from non-family members. These decisions are constrained by the amount of land available (paddy and upland fields) to the household, the expected water available for extraction and the hours of labour available to the household from family members. The expected water available for extraction is updated each year based on the water actually available in the previous year. In the first year of the simulation the expected water available for extraction is user-defined. The sensitivity of the outputs to this initial value is tested in this paper. Households have been classified into several different types, called resource management units (RMU), based on their access to land and water. For the Mae Uam catchment there are three household types (corresponding to three household decision models): RMU2, with irrigated paddy land only; RMU3 with rainfed upland fields only; and, RMU8 with both irrigated paddy and rainfed upland fields. Crops able to be chosen differ by household:  RMU2 is able to choose irrigated paddy rice in the wet and dry seasons, and irrigated sorghum in the dry season;

R.A. Letcher et al. / Agricultural Systems 89 (2006) 132–164

137

 RMU3 is able to choose rainfed upland rice in the wet season and rainfed sorghum in the dry season;  RMU8 is able to choose irrigated paddy rice and rainfed sorghum, upland rice and groundnut in the wet season and irrigated and rainfed sorghum in the dry season.

2.2. Decision disaggregation model The biophysical toolbox considers biophysical processes on a land unit basis. Land units correspond to unique soil types and slope classes (for further details, see Merritt et al., 2004). Household decisions for each RMU type need to be disaggregated to individual land units then summed over the entire catchment in order to be passed to the biophysical toolbox. The decision disaggregation model (DDM) uses a procedure described in the accompanying paper (Letcher et al., this issue) to disaggregate crop decisions to each land unit, using the household decisions for each RMU and the number of households of each RMU type in the catchment. The DDM outputs the total area of crops in each season on each land unit as well as the total forest cover in the catchment. 2.3. Flow model The flow model, based on a modified version of the IHACRES model (Croke and Jakeman, 2004), is a lumped parameter conceptual rainfall-runoff model. Model parameters are modified by changes in forest cover in the catchment, so that a change in forest cover affects wet- and dry-season flows. Full details are given in Croke et al. (2004). The flow model produces daily streamflow given daily observations of rainfall and temperature, and forest cover. 2.4. Crop model The crop model, CATCHCROP, simulates crop growth on a 10-day time step, using rainfall and temperature data. It outputs crop yields and irrigation demands per unit area for each of the crops, considering soil characteristics and the level of fertiliser applied. Outputs are produced for both the wet and dry seasons. 2.5. Erosion model Annual erosion is modelled using a version of the Universal Soil Loss Equation (USLE), modified for Thailand. The model considers slope and soil characteristics, crop choice, rainfall for each year and management practices to produce erosion estimates for each wet and dry season and for each year of the simulation. Erosion outputs used in this paper refer to annual erosion.

138

R.A. Letcher et al. / Agricultural Systems 89 (2006) 132–164

2.6. Water allocation model The water allocation model allocates water to different crops based on a crop priority list. This list mimics irrigation allocation processes in the catchment. The model considers water available for extraction over each 10-day time period (from the flow model), wet- and dry-season crop decisions (from household decision models) and irrigation demands for each crop (determined by the crop model) and allocates water according to demands and availability. Crop failure or reduced yield is then determined for crops that do not receive their water demand. Post-extraction flows are also determined here. This model essentially integrates outputs of models in the biophysical toolbox. 2.7. Socioeconomic impact simulation model The socioeconomic impact simulation model (SISM) calculates the result of actual yields given water availability and household decisions, determined by the biophysical toolbox, on household income. A set of socioeconomic indicators of household well-being is produced, including the total cash available to the household each year, the cost of supplying the rice deficit and income earned off-farm. Each person is assumed to require 300 kg of rice each year. The cost of the rice deficit per person for each RMU is calculated from total household rice production, the number of people per household, the rice requirement of 300 kg per year per person and the cost of buying rice. Household cash consists of the total gross margin on agricultural production, minus the costs of hiring labour on-farm and the cost of supplying the rice deficit, plus the income earned off-farm.

3. Actual and potential use of the integrated modelling toolbox Initially, the main stakeholder focus for the IMT was the Land Development Department, which aimed to use the IMT to assist its land-use planning activities. However other Government Agencies and universities, such as the Royal Irrigation Department and the Royal Forestry Department, also became involved in the development of the IMT in order to develop the capacity to undertake their own integrated assessments of future development and policy scenarios. Adoption of the IMT (and other IWRAM project products) by Government Departments and universities was facilitated by training workshops on both the individual model components and the IMT itself. Overall the development of the IMT had three primary objectives:  to provide a common tool for the Government agencies concerned with water resource management;  to investigate the benefits and impacts of land-use change and land conversion that might occur in the catchment;

R.A. Letcher et al. / Agricultural Systems 89 (2006) 132–164

139

 to recommend alternative crops and management practices for sustainable land and water management, as well as income sustainability. In the words of the main Thai institution responsible for project coordination, the development of the IMT was to ‘‘support sustainable use of Thailand rural catchments, specifically in relation to their land and water management, while maintaining a robust local economy’’ (Royal Project Foundation., 2003). The development of the IMT has played a key role in the project in terms of developing the ideas and capacity of the project team, who largely come from fairly narrow disciplinary backgrounds, necessary to undertake an integrated assessment of the catchment issues. Consideration of environmental sustainability, poverty alleviation and income sustainability involves a very broad range of factors from different disciplines. If sustainability is to be achieved or even targeted, these factors cannot be considered in isolation from each other because of interdependencies. A change in an economic driver such as price can have a large influence on the success of strategies to reduce erosion or control surface flow extraction. Changes in biophysical conditions or policies attempting to control access to resources will also have substantial economic and social impacts. The IMT has been developed to allow users to do the following.  Quantify or qualify the links between biophysical and socioeconomic factors as well as the influence of these factors on system outcomes. This should allow for the magnitude and direction of influences and the resultant change in system behaviour from a change in a system driver, such as a policy, trading conditions or climate, to be estimated. This allows the IMT to play a role in the development of policy in highland catchments in northern Thailand, given its capacity to quantify and illustrate trade-offs relating to environmental sustainability, poverty alleviation and income sustainability resulting from potential policy changes.  Understand the links between drivers, processes and outcomes. Development of the integration framework of the IMT is a major output of the project. This framework is largely independent of the specific component models used to populate the system and summarises understanding developed by the project team of the links between system components. Development of this framework has been important to develop the capacity of the project team to understand the linkages in the system and test the importance of these linkages for system outcomes. As such the IMT plays an important role in training and capacity building within the collaborative partners and within other organisations who have an interest in adoption of IWRAM principles and products.  Ensure that the often conflicting policies developed by various government departments can be explored within a single system that illustrates potential conflicts between policies and outcomes and allows for a broader system perspective than would normally be taken in policy setting. The IMT thus plays a useful role in integrating models and data sets used by various departments to assess their policies and implementation plans.

140

R.A. Letcher et al. / Agricultural Systems 89 (2006) 132–164

In order to ensure that the IMT or later modifications of the model based on the same integrative framework and similar component models are able to achieve this potential, the model must be validated or evaluated sufficiently that users are able to assess the accuracy of the model for their intended use and any limitations in the recommendations they are able to make from it. This type of testing is also required to determine which parts of the model and potentially the Ôreal worldÕ system have the greatest influence on sustainability outcomes. Testing can be used to direct efforts in data collection for refining model performance where sensitivities do not reflect system understanding or cannot be explained as the result of logical extensions of understanding. It is also useful for determining the focus of policy intervention in the system where sensitivities are determined to be plausible. In the longer term as the model becomes more routinely and widely used, it would also be desirable if the ongoing model development and use followed Quality Assurance procedures, such as those specified in Refsgaard et al. (2005).

4. Model evaluation Traditionally validation of a model consists of evaluating the modelÕs forecasting ability on Ôdata other than that used in the identification and estimation studiesÕ (e.g., Young, 1977). Young also states that the Ôvalidation exercise is a continuing procedure since the model will need to be re-assessed in the light of future developments and additional dataÕ. This validation paradigm is useful and often sufficient when validating disciplinary models. For integrated models, however, comprehensive testing of the model against observed data not used in the modelÕs construction is generally impossible, both because such data are generally not available and because any available data have often been used for model construction. Testing such a complex model against a single time series or even multiple time series does not generally give a good appreciation of the validity of the model. The degrees of freedom in the model are too great for this test alone to suffice. In addition such testing does not consider the validity of the model for undertaking the types of simulations for which these types of models are generally developed. Policy or other drivers that cannot be replicated in historic time periods also commonly affect outputs. Therefore, most integrated models should not be developed as prediction or forecasting tools for a set of systems outputs, but to understand the relative magnitude and direction of change of system outcomes (trade-offs between social, economic and environmental system health) in response to changes in policy and other drivers (Jakeman and Letcher, 2003). The issue of model validity then becomes a question of whether or not the model is able to accurately differentiate between two policy options; whether the impacts demonstrated are a result of the policy instrument and are robust across cases presented by the model; or whether these changes are the result of initial inputs that are often uncertain or model assumptions that may not be realistic.

R.A. Letcher et al. / Agricultural Systems 89 (2006) 132–164

141

A more comprehensive process of evaluation that does not rely heavily on history matching of all model outputs is required for integrative models to allow for these differences. Several other authors have suggested approaches that are more appropriate to integrated models (see Risbey et al., 1996; Ravetz, 1997). These approaches rely on a more comprehensive view of model evaluation, rather than validation, for the purposes for which the model is to be used. Following this line of reasoning, the model described in this paper has been evaluated using a range of criteria. These criteria have been based both on evaluating the process by which the model has been developed including the data and assumptions that underlie the model, as well as testing the accuracy of model components using traditional validation approaches (i.e., fit against observed data) and the sensitivity and working of both component models and the integrated models (the integrated modelling toolbox and the biophysical toolbox). The components of this evaluation include:  Testing and peer review of the component models. The crop and hydrological models have been analysed and described in full in Perez et al. (2002) and Croke et al. (2004), respectively; the Household decision models, decision disaggregation model and the socioeconomic impact simulation model are described in detail in Letcher et al. (this issue).  Sensitivity analysis, testing and peer review of the biophysical toolbox (see Merritt et al., 2004; Merritt et al., 2005; Croke et al., 2004).  Testing, sensitivity analysis and description of underlying assumptions and inputs of the integrated modelling toolbox. The sensitivity analysis and testing is described in this paper. A full description of the model, its assumptions and source data for model construction are given in Letcher et al. (this issue).

5. Summary of previous findings on the biophysical toolbox Merritt et al. (2005) conducted a detailed sensitivity analysis of the biophysical toolbox. The Mae Uam catchment was used as a case study to explore the impact of varying the value of crop model parameters on the outputs from the biophysical toolbox. This sensitivity analysis found:  The linkage between the crop and hydrology models is the major point of interaction between the models in the biophysical toolbox.  Some interactions between the models – particularly between the crop and hydrologic component – show nonlinearity.  Only seven of the 19 parameters in the CATCHCROP model were found to influence estimates of deep drainage (DD) and surface runoff (RO).  Quick flow and slow flow proportions in the IHACRES model are highly sensitive to the three CATCHROP infiltration parameters.  The re-scaling of the IHACRES quick and slow flow parameters exhibits strong nonlinearity to the parameters of CATCHCROP describing infiltration, soil depth, and total available water.

142

R.A. Letcher et al. / Agricultural Systems 89 (2006) 132–164

 Perturbing the values of CATCHCROP parameters did not greatly affect estimates of seasonal or annual discharge, despite the potentially large impacts of CATCHCROP parameter values on the re-calculation of the hydrology parameters.  Errors or uncertainties in the infiltration parameters, as well as one parameter defining the available water for plant growth and two for crop yield, have the potential to obscure modelled differences between different land management combinations.

6. Analysis of sensitivity of the integrated modelling toolbox A detailed sensitivity analysis of the integrated modelling toolbox has been undertaken. This section describes the indicators and measures of sensitivity used, and the sensitivity tests that were performed. The next section presents results from the analysis. 6.1. Indicators and measure of sensitivity The sensitivity of the model was analysed in terms of the sensitivity of eight economic, social and biophysical indicators: household cash for all three RMUs; the cost of the rice deficit for all three RMUs; annual erosion; and total annual flow left after extraction. These indicators were evaluated for each year of the simulation and for both nodes (Node 1 and Node 2) on the stream network. For simplicity results presented in this paper are for Node 1 only. The final sensitivity of the model was then measured, where possible, in two ways: the actual model output as the relative change in the indicator divided by the relative change in the parameter value in that year (as reported in the tables). Relative sensitivity of a model output (Si,j) is given by: S i;j ¼

 i;t;W Þ P j ðOi;t;W  O  ;  ðP j  P j Þ Oi;t;W

ð1Þ

 i;t;W is the value of the ith output indicator for the base value ðP j Þ of jth where O parameter in time t at expected water use level W, and Oi,t,W is the value of the output indicator for the changed jth parameter value (Pj). A positive sensitivity value implies that the model output and parameter value move in the same direction (i.e., an increase in the parameter increases the model output). Where the magnitude of the sensitivity is less than 100%, a change in the parameter leads to a less than proportionate change in the output value. 6.2. Sensitivity tests The sensitivity analysis was conducted considering two broad groups of parameters. The first group of parameters were: the number of households of each RMU

R.A. Letcher et al. / Agricultural Systems 89 (2006) 132–164

143

type at each node (Households); the number of people per household of each type (People); the cost of buying rice to supply the household rice deficit (RiceCost); the price of each of the crops considered as an option in the catchment (price 1–4, where paddy rice = 1, upland rice = 2, sorghum = 3 and groundnut = 4); the variable cost of producing each crop (Cost 1–4); the cost of wages per hour (Wage); the amount of labour available to each household type (Labour); the expected volume of water available for each household type in the first year of the simulation (Water); and the amount of land available to each RMU type (Land). The second set of input assumptions considered to be essential drivers of the model results were the expected yields (Y) and water uses (W) of each crop. These assumptions are used by all household decision models to form a decision on the area of different crops to plant. Parameters in the first group were varied one at a time (±20%, 0, ±10%) and then the model was run for different expected yields (0.9Y, 1Y, 1.1Y where Y is the initial value) and water use assumptions (0.8W, 0.9W, 1W, 1.1W where W is the initial value). The model was run for each option over 5 years using climate data from 1989 to 1993.

7. Results Table 2 provides a summary of the relative sensitivity of the model to parameters across all values of expected yield and water use levels. A value of ‘‘No’’ means that the indicator had a relative sensitivity value of zero for all levels of expected yield and water use, for all years and all levels of change in the first group parameter. It was found that the level of sensitivity observed is not affected by changes in the expected yield. That is, the model is not sensitive to 10% changes in the expected yield but is sensitive to changes in the expected water use of crops. This means that the sensitivity values for each incremental change in the parameter for different yield values is the same as that for the base case (expected water use = W). Thus the shape of the sensitivity curve for yield options is not discussed explicitly but is captured by the base case situation. Only the rice deficit indicators are sensitive to the value of the RiceCost parameter. Costs and wages only affect the household cash indicators, not the level of the rice deficit, erosion or flow. The number of people in each household only affects the value of the rice deficit, not cash, flow or erosion. Parameters for which a value of ‘‘No’’ is not given are discussed in more detail in the following sections. In order to simplify the discussion, results are grouped into the effect on ‘‘Cash’’, ‘‘Rice’’, ‘‘Erosion’’ and ‘‘Flow’’ indicators. Table 2 summarises the sensitivity of each output indicator in response to changes in the first group of parameters and levels of expected water use. Table 3 provides the greatest magnitude, positive sensitivity values for each option, showing the year, change level in the first group parameters and expected water use level at which this extreme sensitivity is achieved. Table 4 shows the analogous results for greatest magnitude, negative sensitivity values. In the following subsections, the results are discussed for the Cash indicators (Section 7.1), Rice indicators (Section 7.2), erosion (Section 7.3) and flow (Section 7.4).

144

Table 2 Summary of modelÕs relative sensitivity across all expected water use values CashRMU3

CashRMU8

Erosion

Flow

RiceRMU2

RiceRMU3

RiceRMU8

Households

No

No

No

No

No

No

No

No

Nonlinear, increases as W decreases No

No

People

Mirror image, larger for larger change No

Linear

RiceCost Price 1

No Linear except 20% for all W No

No No

No Nonlinear, same pattern all W Nonlinear, same pattern all W Nonlinear, same pattern all W Nonlinear, same pattern all W Linear except 20%, increases as W decreases Linear except 20%, increases as W decreases Linear, increases as W decreases

No Nonlinear, largest for small increase Nonlinear, largest for small increase

No No

Linear No

For increase Nonlinear only, more years for larger increase No Linear No Nonlinear

No

No

No

Nonlinear

Very small (0.2%) for 20% only No

No

No

Nonlinear

No

No

Nonlinear

No

No

No

No

No

No

No

No

No

No

No

No

No

No

No

Price 2

Price 3

Price 4

Linear except 20% for all W No

Nonlinear, no sensitivity to W Nonlinear, no sensitivity to W No

Cost 1

Linear except 20% for all W

No

Cost 2

No

Linear except 20%, no sensitivity to W

Cost 3

Linear

Linear, no sensitivity to W

Nonlinear, largest for small increase Nonlinear, greatest for small increase

(continued on next page)

R.A. Letcher et al. / Agricultural Systems 89 (2006) 132–164

CashRMU2

Table 2 (continued) CashRMU3

CashRMU8

No

No

Wage

Linear, increases as W and t increase Nonlinear, largest at 20%, more linear as W decreases

Linear, no sensitivity to W

Linear except No 20%, increases as W decreases Linear No

Labour

Linear except 20%, no sensitivity to W

Erosion

Flow

RiceRMU2

RiceRMU3

RiceRMU8

No

No

No

No

No

No

No

No

No

At 10% only in Linear last year except 20% for extremes of W, nonlinear for 0.9W and W No Linear except 20% for extremes of W, nonlinear for 0.9W and W At 20% for 4 Linear for years, At 10% extremes of only in last year W, nonlinear for 0.9W and W

Nonlinear

Nonlinear, largest for 20%

Nonlinear, increases as W decreases

Water

No Nonlinear, largest at 20%, more linear as W decreases

Nonlinear

Nonlinear largest for 20%

At 20% Nonlinear, largest for 20%, only increases as W decreases

Land

Linear, no Nonlinear, largest at 20%, sensitivity to W more linear as W decreases

Nonlinear except 1.1W

Mirror, largest for bigger change

No Nonlinear, largest for 20%, increases as W decreases

R.A. Letcher et al. / Agricultural Systems 89 (2006) 132–164

CashRMU2 Cost 4

145

146

Table 3 Summary of greatest magnitude positive sensitivity by output indicator and first group parameter House- People Rice- Paddy Upland Sorghum Groundnut Paddy Upland Sorghum Groundnut Wages Labour Water Land holds Cost rice rice price price rice cost rice cost cost cost price price

Expected water use value where Cash RMU2 – – Cash RMU3 – – Cash RMU8 – – Erosion All – Flow 1.1 – Year in which Cash RMU2 Cash RMU3 Cash RMU8 Erosion Flow

45.4 39.5 85.4 0.1 –

– – 85.4 – –

– – – – –

– – – – –

– – – – –

– – – – –

92.4 49.7 60.1 – –

94.5 82.0 83.1 98.7 1

44.5 – 43.9 98.7 1

19.2 25.9 35 98.7 1

sensitivity is achieved (W) – 0.8 – 1.1 – – All All – 1,1.1 1,1.1 1,1.1 – All All All – – – –

– – 1,1.1 – –

– – – – –

– – – – –

– – – – –

– – – – –

1.1 All 1.1 – –

1.1 All 1.1 All 1.1

1.1 – 0.9 All 1.1

1 All 0.8 All 1.1 1 4 1 1,3, 4,5 5

sensitivity is achieved – – – – – – – – – All – – 5

–

–

1 – 1 2

– 2 1 2

1 5 1 2

– – 1 –

– – – –

– – – –

– – – –

– – – –

5 1 5 –

5 1 5 1,3,4,5

1 – 1 1,3,4,5

–

–

–

–

–

–

–

–

–

5

5

– – –

– – –

– – –

– – –

All All All

20 20 20

20 – 20

– –

– –

– –

– –

– –

20 20

20 20

First group parameter change where sensitivity is achieved (%) Cash RMU2 – – – 20 – ±10, +20 – Cash RMU3 – – – – 20 10 – Cash RMU8 – – – 10 10 10 10 Erosion Flow

±20 20

– –

– –

20 –

20 –

20 –

– –

20 All 20, 10 ±20 20

R.A. Letcher et al. / Agricultural Systems 89 (2006) 132–164

Greatest magnitude positive sensitivity value (%) Cash RMU2 – – – 56.7 – Cash RMU3 – – – – 49 Cash RMU8 – – – 85.4 85.4 Erosion 98.7 – – 0.2 103.3 Flow 0.6 – – – –

Table 4 Summary of greatest magnitude negative sensitivity by output indicator and first group parameter House- People Rice- Paddy holds Cost rice price

Upland Sorghum Groundnut Paddy Upland Sorghum Groundnut Wages Labour Water Land rice price price rice cost rice cost cost cost price – – – 206.5 –

13.7 – 12.1 – –

– 9.9 12.1 – –

9.5 5.4 12.1 – –

– – 12.1 – –

– – – – –

– – – – 3.7

– – – – 9.2

– – – – 3.7

– – – All –

0.8 – 0.8 – –

– All 0.8 – –

0.8 All 0.8 – –

– – 0.8 – –

– – – – –

– – – – 0.8

– – – – 0.8

– – – – 0.8

– – – 2 –

1 – 1 – –

– 1 1 – –

2 1 1 – –

– – 1 – –

– – – – –

– – – – 1

– – – – 1

– – – – 1

First group parameter change where sensitivity is achieved (%) Cash RMU2 – – – – – – Cash RMU3 – – – – – – Cash RMU8 – – – – 20 –

– – –

All All 20, ±10 – –

– – 20, ±10 – –

– – –

– – –

– – –

10 –

– 20 20, ±10 – –

– – –

Erosion Flow

20 – 20, ±10 – –

– –

– 10

– 20

– 10, 20

Expected water use value where Cash RMU2 – – Cash RMU3 – – Cash RMU8 – – Erosion – – Flow 0.8 – Year in which Cash RMU2 Cash RMU3 Cash RMU8 Erosion Flow

sensitivity is achieved (W) – – – – – – – – – – 1.1 – – All All All – 0.8 – 0.8,1,1.1

sensitivity is achieved – – – – – – – – – – – – 1 – –

– 10

– –

– – – 206.5 0.2

– –

– – – 2 1

10 20

– – 5 2 –

10 –

– – – 2 1,2

10 20

R.A. Letcher et al. / Agricultural Systems 89 (2006) 132–164

Greatest magnitude negative sensitivity value (%) Cash RMU2 – – – – – Cash RMU3 – – – – – Cash RMU8 – – – – 11.4 Erosion – – – 206.5 206.5 Flow 10.1 – – 1.6 –

147

148

R.A. Letcher et al. / Agricultural Systems 89 (2006) 132–164

7.1. Cash The output values for Cash at RMU2 and RMU8 for all change levels in the first group of parameters (scenarios 1–75) and all levels of expected water use are shown in Figs. 2 and 3, respectively. Scenario numbers shown correspond to those given in Table 1. For each group of five runs (e.g., scenarios 1–5, scenarios 21–25), the scenarios are ordered according to the percentage change in the first group parameter being considered by the scenario group (20%, 10%, 0%, 10%, 20%). Thus scenarios for the base case correspond to the sequence numbers 3, 8, 12, . . ., 73. Results for these in the figures always correspond to the most constant values. Cash at RMU3 is not sensitive to changes in expected water use. Cash at RMU3 for each scenario is shown in Fig. 4. 7.1.1. Prices (scenarios 16–35) Cash at RMU2 shows roughly the same pattern across all levels of expected water use, with the level of sensitivity increasing for all change levels except 20% as expected water use increases. At RMU2 the indicator has near to linear sensitivity to the price of paddy rice and sorghum. There is some nonlinearity when the change in price is as low as 20%. Sensitivities are all positive, and have a magnitude of approximately 50% for both prices and all change levels. Cash at RMU8 has a positive sensitivity to changes in all prices. The maximum sensitivity observed is 85%. The pattern of sensitivity is very similar across all levels of expected water use. Cash at RMU3 is sensitive to the price of upland rice in a nonlinear way. All sensitivities are positive and have a magnitude of less than 50%. This indicator is also positively sensitive to the price of sorghum. The sensitivity value is greater for an increase in price than a decrease, with a magnitude of less than 40% in all cases. 7.1.2. Costs (scenarios 36–55) Cash at RMU2 has a smaller magnitude sensitivity (<14%) to costs than prices, with the sensitivity negative in non-zero cases. It is more sensitive to the cost of paddy rice than the cost of sorghum. The same pattern is seen for all levels of expected water use. The sensitivity decreases over time. Cash at RMU3 shows a nearly linear sensitivity to the cost of upland rice, and linear sensitivity to the cost of sorghum. Where nonlinearities exist they occur when costs change by 20%. The maximum sensitivity observed is very small (approximately 5%). There is less difference between the indicator values for different years than is the case for Cash at RMU2 or RMU8. Cash at RMU8 has a nearly linear sensitivity to all costs, with nonlinearities in some cases at the 20% change level. The level of sensitivity increases as W decreases. The same sensitivity pattern is seen in every year, but the level of sensitivity decreases over time.

30,000 30,000

25,000

15,000

Year 1 Year 2 Year 3 Year 4 Year 5

10,000

5,000

Cash RMU2 (baht)

20,000

20,000

15,000

10,000

Year 1 Year 2 Year 3 Year 4 Year 5

5,000

0

0 1

6

11

16

21

26

31

36

41

46

51

56

61

66

1

71

6

11

16

21

26

41

46

51

56

61

66

71

20,000 Cash RMU2 (baht)

2 0 ,0 0 0 Cash RMU2 (baht)

36

0.9W

25,000

0.8W

2 5 ,0 0 0

31

Scenario

Scenario

1 5 ,0 0 0

1 0 ,0 0 0 Year Year Year Year Year

5 ,0 0 0

0 1

6

11

16

21

26

31

36

41

S c e n a rio

1W

46

51

56

61

66

71

1 2 3 4 5

15,000

10,000 Year 1 Year 2 Year 3 Year 4 Year 5

5,000

R.A. Letcher et al. / Agricultural Systems 89 (2006) 132–164

Cash RMU2 (baht)

25,000

0 1

6

11

16

21

26

31

36

41

46

51

56

61

66

71

Scenario

1.1W 149

Fig. 2. Sensitivity in each year for all scenarios and all levels of W for household cash (Cash) at RMU2. Discrete points are joined to illustrate trends.

25,000

20,000

15,000

Year 1 Year 2 Year 3 Year 4 Year 5

10,000

5,000

Cash RMU8 (baht)

25,000

20,000

15,000 Year 1 Year 2 Year 3 Year 4 Year 5

10,000

5,000

0

0 1

6

11

16

21

26

31

36

41

46

51

56

61

66

1

71

6

11

16

21

26

31

41

46

51

56

61

66

71

0.9W

0.8W 25,000

30,000

25,000

20,000

20,000

15,000

Year 1 Year 2 Year 3 Year 4 Year 5

10,000

5,000

Cash RMU8 (baht)

Cash RMU8 (baht)

36

Scenario

Scenario

15,000

10,000 Year 1 Year 2 Year 3 Year 4 Year 5

5,000

0

0 1

6

11

16

21

26

31

36

41

46

51

56

61

66

71

1

6

11

16

21

26

31

36

41

Scenario

Scenario

1W

1.1W

46

Fig. 3. Sensitivity in each year for all scenarios and all levels of W for household cash (Cash) at RMU8.

51

56

61

66

71

R.A. Letcher et al. / Agricultural Systems 89 (2006) 132–164

Cash RMU8 (baht)

30,000

150

30,000

R.A. Letcher et al. / Agricultural Systems 89 (2006) 132–164

151

18,000 16,000

Cash RMU3 (baht)

14,000 12,000 10,000 8,000

Year 1 Year 2 Year 3 Year 4 Year 5

6,000 4,000 2,000 0 1

6

11

16

21

26

31

36

41

46

51

56

61

66

71

Scenario Fig. 4. Sensitivity in each year for all scenarios and all levels of W for household cash (Cash) at RMU3. No sensitivity to expected water use (W).

7.1.3. Wages (scenarios 56–60) The Cash indicators at all RMU are linearly sensitive to a change in wages. The sensitivity level increases as the expected water use level increases at RMU2 and RMU8. Sensitivity also increases over time for these RMU. The sensitivity level is more constant over time at RMU3. All sensitivities are positive indicating that households are choosing to hire their own labour off-farm rather than hire in additional workers. The level of sensitivity to wages is greatest at RMU2 (92%), then at RMU8 (60%). 7.1.4. Labour (scenarios 61–65) Cash at RMU2 is nonlinearly sensitive to changes in labour availability. The greatest sensitivity is seen at the 20% change level. The sensitivity gets more linear as the expected water use level decreases, with the sensitivity at the 20% change level much greater for higher levels of expected water use. The same pattern of sensitivity is seen for all years and expected water use levels. The sensitivity is always positive. Cash at RMU3 shows the same pattern of sensitivity to labour availability as other households. The sensitivity is greatest at the 20% level of change. For other levels of change the sensitivity is more linear. The sensitivity is positive. Cash at RMU8 shows a nonlinear sensitivity pattern to labour availability. All sensitivities are positive (<84%). The greatest sensitivity level is seen for a 20% decrease in labour availability.

152

R.A. Letcher et al. / Agricultural Systems 89 (2006) 132–164

7.1.5. Water (scenarios 66–70) Cash at RMU2 shows a nonlinear sensitivity pattern to available water. The greatest level of sensitivity is at the 20% change level in water availability. The pattern of sensitivity becomes more linear as the expected water use decreases. The indicator also becomes less sensitive over time (ie. as move from year 1 to year 5 of the simulation). It is less sensitive to a change in water availability (<45%) than to a change in labour availability (94.5%). The sensitivity is always positive. Cash at RMU8 is nonlinearly sensitive to a change in water availability. The pattern of sensitivity becomes more linear as the expected water use level decreases. The sensitivity is always positive. The greatest level of sensitivity is seen for a 20% decrease in water available (44%). This indicator is also less sensitive to a change in water availability than labour availability. 7.1.6. Land (scenarios 71–75) Cash at RMU2 is less sensitive to land than water or labour. The sensitivity is greater for a decrease in land than an increase. Cash at RMU3 is linearly sensitive to changes in land. The greatest sensitivity occurs in year 4 (25.9%). All sensitivities for Cash at RMU3 are positive. The sensitivity pattern of Cash at RMU8 to Land is very similar to that for Water, with only a greater sensitivity to a 20% decrease in Land than the same change in Water. Cash at RMU8 is more sensitive to decreases in Land than increases. This is as would be expected when other production factors begin to limit production, rather than simply land availability. 7.2. Rice Figs. 5–7 show the values of the Rice indicators for all scenarios for RMU2, RMU8 and RMU3, respectively. The Rice indicators are relatively insensitive to changes in the input assumptions compared to the other indicators. This is because under the base case assumptions these indicators have a value of zero in many years. The Ôstep functionÕ nature of these indicators means that a reasonably large change may be required to shift the value of these indicators from zero. The zero value for the base case also means that the measure of the relative sensitivity defined in Eq. (1) is not generally relevant for these indicators. The discussion is based around input assumptions for which the values of the indicators are non-zero. Rice at RMU3 is only sensitive to extreme increases (20%) in the number of people per household, a change in labour in one year (year 5) and reductions in the land available. For RMU2 and RMU8 the rice deficit generally increases over time. For RMU2 the rice deficit increases as the expected water use increases. This relationship is not as clear at RMU8, where it decreases from 0.8W to 0.9W then increases as expected water use increases.

7

7

6 5

Year Year 2 Year 3 Year 4 Year 5

4 3 2

Rice RMU2 (baht per person)

8

6 5

Year 1 Year 2 Year 3 Year 4 Year 5

4 3 2 1

1

0

0 1

6

11

16

21

26

31

36 41 Scenario

46

51

56

61

66

1

71

6

11

16

21

26

36

41

46

51

56

61

66

71

Scenario

0.9W

0.8W 8

8

7

7

6 5 4

Year 1 Year 2 Year 3 Year 4 Year 5

3 2

Rice RMU2 (baht per person)

Rice RMU2 (baht per person)

31

6

Year 1 Year 2 Year 3 Year 4 Year 5

5 4 3 2

1

R.A. Letcher et al. / Agricultural Systems 89 (2006) 132–164

Rice RMU2 (baht per person)

8

1 0 1

6

11

16

21

26

31

36

41

Scenario

W

46

51

56

61

66

71

0 1

6

11

16

21

26

31

36

41

46

51

56

61

66

71

Scenario

1.1W

153

Fig. 5. Sensitivity in each year for all scenarios and all levels of W for the cost of rice deficit (Rice) at RMU2.

6

4

Rice RMU8 (baht per person)

5

Year 1 Year 2 Year 3 Year 4 Year 5

5

Year 1 Year 2 Year 3 Year 4 Year 5

3 2

4

3

R.A. Letcher et al. / Agricultural Systems 89 (2006) 132–164

Rice RMU8 (baht per person)

6

2

1

1

0

0 1

6

11

16

21

26

31

36

41

46

51

56

61

66

1

71

6

11

16

21

26

31

36

41

Scenario

Scenario

0.8W

0.9W

7

7

6

6 Rice RMU8 (baht per person)

Rice RMU8 (baht per person)

154

7

Year 1 Year 2 Year 3 Year 4 Year 5

5 4 3 2

46

51

56

61

66

71

Year 1 Year 2 Year 3 Year 4 Year 5

5 4 3 2 1

1

0

0 1

6

11

16

21

26

31

36

41

Scenario

W

46

51

56

61

66

71

1

6

11

16

21

26

31

36

41

46

51

Scenario

1.1W

Fig. 6. Sensitivity in each year for all scenarios and all levels of W for the cost of rice deficit (Rice) at RMU8.

56

61

66

71

R.A. Letcher et al. / Agricultural Systems 89 (2006) 132–164

155

1.2

Rice RMU3 (baht per person)

1.0

0.8

0.6

0.4 Year 1 Year 2 Year 3 Year 4 Year 5

0.2

0.0 1

6

11

16

21

26

31

36

41

46

51

56

61

66

71

Scenario

Fig. 7. Sensitivity in each year for all scenarios and all levels of W for the cost of rice deficit (Rice) at RMU3. No sensitivity to expected water use (W).

7.2.1. Number of people (scenarios 6–10) and the Rice Cost (scenarios 11–15) Where the indicator has non-zero value it is linearly sensitive to changes in the number of people per household and the cost of buying rice. These sensitivities are seen for both RMU2 and RMU8. 7.2.2. Prices (scenarios 16–35) The rice deficit is not generally sensitive to changes in prices at RMU2. However, the largest decrease in the price of sorghum (20%) does lead to a decrease in the rice deficit in some years and for some levels of expected water use. Rice at RMU8 shows a greater degree of sensitivity to prices. It is sensitive to increases in all prices, and to large decreases in the price of sorghum. 7.2.3. Labour, Water and Land (scenarios 61–75) Rice at RMU2 is only sensitive to a 20% decrease in Water, with an increase in the rice deficit seen for this change. At RMU8, Rice is sensitive to Labour, Water and Land. These sensitivities are linear for all changes except 20%, where the rice deficit falls rather than increases in some cases. 7.3. Erosion Erosion is not sensitive to changes in expected water use. Fig. 8 demonstrates the pattern of sensitivity of erosion to changes in all first group parameters in each year.

156

R.A. Letcher et al. / Agricultural Systems 89 (2006) 132–164

180 160 140

Erosion (T)

120 100 80

Year 1 Year 2 Year 3 Year 4 Year 5

60 40 20 0 1

6

11

16

21

26

31

36

41

46

51

56

61

66

71

Scenario Fig. 8. Sensitivity in each year for all scenarios and all levels of W for Erosion. No sensitivity to expected water use (W).

7.3.1. Number of households (scenarios 1–5) Erosion shows a nearly linear sensitivity pattern to changes in the number of households. The sensitivity across years and expected water use levels is the same. The level of sensitivity is less than 100% (ranging from 92% to 99%). Erosion is positively sensitive to changes in the number of households, as increasing households reduces forest cover and increases erosion. 7.3.2. Prices (scenarios 16–35) Erosion is sensitive to changes in all prices. The pattern of sensitivity is the same between years. For paddy rice, sorghum and groundnut, erosion is only sensitive to an increase in price. In this case a large negative sensitivity is observed (100% to 200%). This sensitivity corresponds to a one-off shift of erosion to a higher level for a 10% increase in price. From here erosion does not increase further for the 20% increase in price (thus the sensitivity value is halved). The change in the level of erosion is very similar for all prices. For upland price the same pattern of change is observed for increases in price. In this case, however, a 20% decrease in price leads to an increase in erosion. 7.3.3. Labour, Water and Land (scenarios 61–75) Erosion is linearly sensitive to changes in labour, water and land availability except at the 20% decrease level of change, where sensitivity is the greatest. Erosion is most sensitive to a change in available land as this impacts most directly on the drivers of

R.A. Letcher et al. / Agricultural Systems 89 (2006) 132–164

157

14,000

12,000

Flow (ML)

10,000

8,000

6,000

Year 1 Year 2 Year 3 Year 4 Year 5

4,000

2,000

0 1

6

11

16

21

26

31

36

41

46

51

56

61

66

71

Scenario Fig. 9. Sensitivity in each year for all scenarios and expected water use of 0.8W for Flow. Little sensitivity is shown to any change in parameters. Sensitivity patterns for other expected water use levels are very similar.

erosion (forest cover and crop mix). Erosion shows a similar pattern and level of sensitivity for labour and water availability. 7.4. Flow Flow shows very little sensitivity to any change in first group parameters. The sensitivity of flow to changes in first group parameters at a constant level of expected water use (0.8W) is shown in Fig. 9. Sensitivity patterns for other levels of expected water use are very similar. Flow shows the greatest sensitivity to a change in the number of households (approximately 10%). This sensitivity is nearly linear and its magnitude decreases over time. The sensitivity also decreases as the expected water use increases. Flow has a very small sensitivity to changes in price in some cases. This sensitivity has a very small magnitude (<1%). Flow is more sensitive to a change in the price of sorghum than other crops, but this sensitivity is very small. Flow has a similar and small level of sensitivity to changes in labour and land availability. It is more sensitive to changes in water availability (greatest is 9.2%). This sensitivity increases as expected water use decreases.

158

R.A. Letcher et al. / Agricultural Systems 89 (2006) 132–164

8. Discussion This paper presented the results of a detailed sensitivity analysis of an integrated modelling toolbox for considering economic, social and environmental trade-offs of land development and use in northern Thailand. The analysis complements earlier work by Merritt et al. (2005) where the interactions between the models in a component, the biophysical toolbox, were analysed in detail. The sensitivity of the integrated modelling toolbox was tested with respect to two sets of parameters: expected yields and water uses; and, other key economic and social parameters affecting both the household decisions and the economic and social well-being of households (measured through household cash and the ability of households to meet their subsistence production requirements). Overall the model shows plausible sensitivity to parameters. In many cases this sensitivity is linear, or is linear for all changes except extreme (20%) decreases in parameter values. This means that in the vicinity of the base case parameter values, the model is relatively stable and robust. Greater sensitivity is experienced at the extreme decrease level as thresholds in the model (for example in the crop choice from the linear programming model) are crossed. These thresholds relate to factors affecting the gross margins (prices, costs) as well as those affecting resource constraints (Water, Land, Labour). Increases in these parameters do not usually affect the model as greatly, as other factors then become constraining to the decision. The model showed no sensitivity to a 10% change in expected yield. This result indicates a certain level of model robustness. The model requires further testing, however, as the sensitivity analysis presented in this paper changed expected yields for all crops by the same proportion. Thus the relative expected yields were unchanged. Further sensitivity testing could consider changes in the relative expected yields of different crops. Expected water use does affect the sensitivity of most model indicators. Indicators for households with only rainfed upland fields (RMU3) are unaffected as this household type does not have access to water. Generally the highest sensitivities are experienced at the extremes of expected water use (0.8W and 1.1W where W is the base case level of expected water use). This indicates that the initial value of expected water use defined in the base case is a fairly robust choice and does not strongly affect model results. Persistence of this and other initial assumptions in later years could be a problem with respect to the value of outputs, but does not appear to affect the ranking or relative value of outcomes in different years. Overall model sensitivities are generally less than 100%, indicating that the model outputs are not extremely sensitive to changes in these parameters. This means that when impacts of policy drivers on the system are being assessed using the integrated modelling toolbox, large changes in model outputs are likely to occur as a result of changes in policy rather than uncertainty in the model. In particular, policy recommendations, especially those based on flow and socioeconomic impacts, are not likely to be due to small uncertainties in price or cost assumptions. Erosion is the most sensitive output, with sensitivities of magnitude greater than 100%. Erosion is most sensitive to a change in prices because changes in price can lead to a change

R.A. Letcher et al. / Agricultural Systems 89 (2006) 132–164

159

in crop choice, a fairly direct impact on erosion. Erosion is not affected by a change in expected water use, indicating that the direction of change in erosion to a policy affecting other inputs or parameters may not be impacted by this price sensitivity. Further investigation of this type of uncertainty may be warranted. Flow is the least sensitive of all outputs, with very small changes in the indicator seen for changes in the number of households, Land, Labour, Water and some prices. These sensitivities generally have a magnitude of less than 10%. The flow indicator used in this analysis was annual flow. It is possible that a measure of dry-season flow may be more relevant and more sensitive to these changes and should be included in future analysis. The insensitivity of flow corresponds to the previous finding by Merritt et al. (2004) that the crop model parameters do not greatly affect flow estimates. The greatest impact on flow occurs for a change in the number of households in the catchment. This sensitivity is negative as expected, indicating that an increase in the number of households decreased flow. This means that the increase in extraction by households more than compensates for increases in flow due to reduced forest cover. This effect is relatively very small.

9. Conclusions: Implications of the sensitivity results for model use and reapplication The sensitivity results described in this paper and for the model components (Merritt et al., 2005; Croke et al., 2004; Perez et al., 2002) demonstrate several important features that potential users of the IMT or those wishing to reapply the model or approach should consider. These conclusions relate to the limitations of the approach and the way in which the model results should be reported as well as to the types of data that should be collected for reapplication or improvement of the model. 9.1. Reporting results and estimating impacts The results demonstrate that the pattern of results is the same (or very similar) between years but the absolute value of indicators is not the same. This means that the direction and magnitude of change is robust to small changes in assumed parameter values but the absolute value of output indicators may not be. Thus results from the model should be reported as a percentage change or relative impact from a reasonable base case not as an absolute impact or indicator value. This is important for target setting, as it may indicate the potential for the model to assess the capacity of a land-use or management change to achieve a level of change, rather than a specific target value for an indicator. That is, the model is likely to be more robust for assessing the potential for a change to achieve say a 50% change in erosion, flow or economic outcome from the current condition, than achieving a set target for a specific indicator, such as household income exceeding a specified value or erosion meeting a specific target. In many cases it seems that model runs across multiple years are not necessary as the model simulates the same impact in different years (in terms of magnitude and

160

R.A. Letcher et al. / Agricultural Systems 89 (2006) 132–164

direction of change). This means that the information on impacts in the fifth year of simulation is similar to that in the first or second. Testing the influence of the sequence of climate years on this pattern of impacts might be useful to confirm that long simulation periods are not necessary to estimate impacts. This would test whether the model is showing bias in indicator values (rather than relative changes) to the assumption of naı¨ve expectations (i.e., the model output gets pushed down in each year, in which case initial years are a better indication of impact), or if the temporal trend in results reflects the pattern of years used to derive the response. The results also show that if users are interested in social impacts and subsistence production then the single most important parameter is the number of people in each household, as this has the greatest impact on the rice deficit indicators. Where users are not so interested in this impact or where these data are not trusted or known accurately, it may be better to consider the impact of drivers on total rice production rather than rice deficit indicators. This is because total rice production is not sensitive to the number of households. Additionally, it can be expected to change more linearly than the rice deficit (the equation defining the rice deficit does not have a continuous first derivative). Annual flows are shown to be sensitive to very large changes in forest cover only. This means that forest encroachment is not very likely to change annual flows unless the area cleared is very large. The focus in future assessments and model runs should be on assessing the impact of changes on ecologically and economically relevant flow types (such as baseflows in the dry-season), rather than considering impacts on annual flows. This would provide more useful information on the nature of impacts. Overall the sensitivity results demonstrate that less confidence should be placed on small changes in indicator values than for larger changes. For example, if a policy has a relative impact of less than 100% then this is comparable to the order of magnitude of the modelÕs sensitivity to parameter values. The certainty users place on outputs should be relative to the levels of sensitivity of impact demonstrated in the sensitivity analysis. Consider, for example, a change in policy that has a relative impact on household cash in RMU2 of less than 95%. Then this is an impact that is within the uncertainty created by a 20% change in wage levels. If this is a reasonable change in wage levels then the policy impact is relatively uncertain, given uncertainties in input assumptions. If the uncertainty in wage levels is known to be much less than 20%, at which point the model sensitivity to wages falls to a much lower level (e.g., below 50%) then this policy impact is relatively certain. The sensitivity analysis therefore provides an indication of confidence that can be placed on the policy recommendations arising from the model. A comparison of the sensitivity results with the scenario results given in the companion paper shows that the model is less sensitive to changes in parameter assumptions than it is to key policy scenarios. A 10% change in the land available to each household had a 100% and greater impact on household cash and erosion. This means that this impact is likely to be very robust and not the result of parameter uncertainty or assumptions. Thus the model is capable of providing a good indication of the impacts of these types of large scale policy or management changes. Care should be taken in interpreting results from the model where impacts are of a lesser order of magnitude than the model sensitivity.

R.A. Letcher et al. / Agricultural Systems 89 (2006) 132–164

161

9.2. Reapplying the model and data collection This paper demonstrated that the model outputs were not sensitive to a change in the relative value of expected yield. That is, so long as yields for each crop are changed by the same proportion, no change is seen in any of the indicator values considered. Therefore, for reapplication of the model the most important consideration when setting expected yield values is to ensure that the relative expected yields of crops are accurate. This is a substantial benefit for reapplication of the approach because the relative yields of different crops are usually fairly well known compared with the actual values, and are not so affected by local conditions. In addition, peopleÕs expectations of yield are usually able to be grouped together for different types of crops, simplifying the data that needs to be collected and the complexity of crop types that need to be represented in the model e.g., irrigated versus rainfed, subsistence versus cash crops. In order to improve the current model or for reimplementation in another area it is best to focus survey data collection on understanding peopleÕs expectations of the relative yields of different crops, rather than exhaustively documenting household types and resources. Similarly, the greatest sensitivities in the model are seen for extremes of expected water available. Data collection and surveys should also focus on understanding peopleÕs expectations of water and how these change given actual resource availability in previous years. So long as expectations or water availability are close to correct then this assumption will have relatively little effect on recommendations arising from the model. Overall socioeconomic data collection for model improvement or reapplication should focus on a smaller survey designed to develop understanding of how each group of farmers/households forms expectations, what these expectations are based on and what level of yield and water available is expected. This would be more useful than a larger survey focused on typing households and classifying the crops they grow, as many of these features can be adequately captured or understood from secondary data (e.g., land-use maps or surveys conducted by the government departments, crop production data, etc.). 9.3. Future developments and limitations of the current model Overall the sensitivity analysis of the model demonstrates that it is capable of demonstrating some of the trade-offs associated with highland development and changes in resource use and management. This analysis does however demonstrate some limitations in the current model structure and components. Firstly, the current model is not capable of considering production decisions that are non-seasonal. This means that decisions to plant fruit trees, or products such as coffee or tea, which have returns over many periods are not currently incorporated. These types of decisions could be incorporated in future versions of the model with relatively little difficulty. Other major limitations in the model structure and components include key hydrological issues. The lack of impact on water quality, as opposed to erosion, and the exclusion of groundwater systems, extraction from them and their links with

162

R.A. Letcher et al. / Agricultural Systems 89 (2006) 132–164

surface water availability significantly limit the usefulness of the biophysical toolbox in assessing the downstream impacts of changes in highland development and management in new situations. For the Mae Chaem catchments these issues were considered to be of less concern, however for many highly developed catchments in northern Thailand and elsewhere these issues are paramount. Future application and development of the IWRAM approach will require these issues to be considered. Finally, the impact of forest encroachment on the forest ecosystem is currently limited to a very crude estimate of area lost. This does not consider the impact on forest biodiversity of the links between ecosystem functions, human developments and the reliance of human populations on forest resources. In many cases village populations rely heavily on forest products for medicines, housing materials and subsistence requirements. At present these issues are not considered by the model and require further consideration in future developments of the model. 9.4. Value of the approach and its potential for reapplication The model described in this paper and its companion (Letcher et al., this issue) has been shown to be capable of demonstrating trade-offs in achieving environmental, social and economic sustainability and of producing plausible results. For the researchers and agency staff involved in the development of this tool, however, the development of an Ôintegrated understandingÕ of the way in which the catchment system operates has been the major benefit. Overall experiences in the development of the IMT show that focus should be placed on developing a conceptual framework for integration of system components and populating this with relatively simple models before emphasis is placed on developing and incorporating more complex representations of component processes. This approach allows the relative value of these complexities to be continually assessed throughout the project, allows the project team to develop their own language and concepts to communicate across disciplinary perspectives before they focus on adding complexity to disciplinary models, and provides a framework for model development that is issue and needs-focused. The integrated model described in this paper can thus represent a good starting point from which other researchers can further develop component models or concepts in future applications. The conceptual framework, model descriptions and sensitivity results are useful in their own right for considering policies in the catchments to which they have already been applied, as well as having potential value for informing future applications, either in their current form or using the concepts underlying the IMT to inform development of similar approaches in other catchment situations. The project has also been influential at the national level. The IWRAM approach has been adopted as the framework for a major initiative of the National Research Council, which will see Thailand work with neighbouring countries in the Greater Mekong sub-region to implement IWRAM. Through the project, the Royal Project Foundation (RPF) has played a key facilitation role in focussing government agency support in the northern catchments. There has been a significant investment by the RPF and government agencies in understanding the environmental, social and economic impact of changes in water use and management practices in the catchment,

R.A. Letcher et al. / Agricultural Systems 89 (2006) 132–164

163

with the key word being ÔsustainableÕ. This involves a strong emphasis on ensuring economic returns to the local farmers in exchange for modifying agricultural practices.

Acknowledgements The IWRAM project was supported by the Australian Centre for International Agricultural Research and involved many contributors to the key research components as detailed in the earlier companion paper.

References Croke, B.F.W., Jakeman, A.J., 2004. A Catchment moisture deficit module for the IHACRES rainfallrunoff model. Environmental Modelling and Software 19, 1–5. Croke, B.F.W., Merritt, W.S., Jakeman, A.J., 2004. A dynamic model for predicting hydrologic response to land cover changes in gauged and ungauged catchments. Journal of Hydrology 291, 115–131. Jakeman, A.J., Hornberger, G.M., 1993. How much complexity is warranted in a rainfall-runoff model?. Water Resources Research 29 (8) 2637–2649. Jakeman, A.J., Letcher, R.A., 2003. Integrated assessment and modelling: features, principles and examples for catchment management. Environmental Modelling and Software 18, 491–501. Jakeman, A.J., Saowapon, C., Jintrawet, A., Trisophon, K., Evans, J., Wong, F., 1997. Biophysical component of an integrated water resources assessment project in the Upper Chao Phraya Headwater, Northern Thailand. In: McDonald, A.D. et al. (Eds.), Proceedings International Congress on Modelling and Simulation, MODSIM97, vol. 2. University of Tasmania, pp. 687–691. Letcher, R.A., Merritt, W.S., Jakeman, A.J., Croke, B.F.W., this issue. An integrated modelling toolbox for water resources assessment and management in Northern Thailand: model description. Agricultural Systems, doi:10.1016/j.agsy.2005.08.006. Merritt, W.S., Croke, B.F.W., Jakeman, A.J., 2005. Sensitivity testing of a model for exploring water resources utilisation and management options. Environmental Modelling and Software 20, 1013–1030. Merritt, W.S., Croke, B.F.W., Jakeman, A.J., Letcher, R.A., Perez, P., 2004. A biophysical toolbox for assessment and management of land and water resources in rural catchments in Northern Thailand. Ecological Modelling 171, 279–300. OEPP, 2002. Policy and prospective plan for enhancement and conservation of national environmental quality, 1997–2016. Office of Environmental Policy and Planning. (accessed August 2002). Pallottino, G.S., Sechi, G.M., Zuddas, P., 2005. A DSS for water resources management under uncertainty by scenario analysis. Environmental Modelling and Software 20, 1031–1042. Perez, P., Ardlie, N., Kuneepong, P., Dietrich, C., Merritt, W.S., 2002. CATCHCROP: modelling crop yield and water demand for an Integrated Catchment Assessment in Northern Thailand. Environmental Modelling and Software 17, 251–259. Ravetz, J.R., 1997. Integrated Environmental Assessment Forum: developing guidelines for ‘‘good practice’’. ULYSSES WP-97-1, ULYSSES, Darmstadt University of Technology. Refsgaard, J.C., Henriksen, H.J., Harrar, W.G., Scholten, H., Kassahun, A., 2005. Quality assurance in model based water management – review of existing practice and outline of new approaches. Environmental Modelling and Software 20, 1201–1215. Risbey, J., Kandlikar, M., Patwardhan, A., 1996. Assessing integrated assessments. Climatic Change 34, 369–395. Royal Project Foundation, 2003. Integrated Water Resource Assessment and Management Project. Royal Project Foundation, Chiang Mai, Thailand. (accessed 30.11.04).

164

R.A. Letcher et al. / Agricultural Systems 89 (2006) 132–164

Schluter, M., Savitzky, A.G., McKinney, D.C., Leith, H., 2005. Optimizing long-term water allocation in the Amudarya River delta: a water management model for ecological impact assessment. Environmental Modelling and Software 20, 529–545. Sethaputra, S., Thanopanuwat, S., Kumpa, L., Pattanee, S., 2002. ThailandÕs water vision: a case study. Food and Agriculture Organisation (FAO) of the United Nations. (accessed August 2002). UN, 2002. Natural resource aspects of sustainable development in Thailand. United Nations SystemsWide Web Site on National Implementation of the Rio Commitments. (accessed August 2002). Wischmeier, W.H., Smith, D.D., 1978. Predicting rainfall erosion losses. USDA Agricultural Research Services Handbook, 537. Young, P., 1977. A general theory of modeling for badly defined systemsAS/R9, Centre for Resource and Environmental Studies. The Australian National University, Canberra.