A General Probabilistic Framework for uncertainty and global sensitivity analysis of deterministic models: A hydrological case study

Environmental Modelling & Software 51 (2014) 26e34

Contents lists available at ScienceDirect

Environmental Modelling & Software journal homepage: www.elsevier.com/locate/envsoft

A General Probabilistic Framework for uncertainty and global sensitivity analysis of deterministic models: A hydrological case study G. Baroni a, *, S. Tarantola b a b

Institute of Earth and Environmental Sciences, University of Potsdam, Karl-Liebknecht-Str. 24-25, 14476 Potsdam, Germany Econometrics and Applied Statistics Unit, Joint Research Centre, Ispra, Italy

a r t i c l e i n f o

a b s t r a c t

Article history: Received 22 August 2012 Received in revised form 25 September 2013 Accepted 27 September 2013 Available online 17 October 2013

The present study proposes a General Probabilistic Framework (GPF) for uncertainty and global sensitivity analysis of deterministic models in which, in addition to scalar inputs, non-scalar and correlated inputs can be considered as well. The analysis is conducted with the variance-based approach of Sobol/ Saltelli where first and total sensitivity indices are estimated. The results of the framework can be used in a loop for model improvement, parameter estimation or model simplification. The framework is applied to SWAP, a 1D hydrological model for the transport of water, solutes and heat in unsaturated and saturated soils. The sources of uncertainty are grouped in five main classes: model structure (soil discretization), input (weather data), time-varying (crop) parameters, scalar parameters (soil properties) and observations (measured soil moisture). For each source of uncertainty, different realizations are created based on direct monitoring activities. Uncertainty of evapotranspiration, soil moisture in the root zone and bottom fluxes below the root zone are considered in the analysis. The results show that the sources of uncertainty are different for each output considered and it is necessary to consider multiple output variables for a proper assessment of the model. Improvements on the performance of the model can be achieved reducing the uncertainty in the observations, in the soil parameters and in the weather data. Overall, the study shows the capability of the GPF to quantify the relative contribution of the different sources of uncertainty and to identify the priorities required to improve the performance of the model. The proposed framework can be extended to a wide variety of modelling applications, also when direct measurements of model output are not available. Ó 2013 Elsevier Ltd. All rights reserved.

Keywords: Global sensitivity analysis Non-scalar input factors Hydrological model Multi-variables

1. Introduction The advances in computer science have taken to a growing number of modelling studies explicitly considering uncertainty analysis (UA) and sensitivity analysis (SA) of model output (e.g., Matott et al., 2009; Wagener and Montanari, 2011). Depending on the fields of application, different definitions can be encountered. UA is frequently referred to as error propagation, uncertainty propagation or functional evaluation (e.g., Heuvelink, 1998; Arbia et al., 1998; Vereecken et al., 1992). For each specific context, different methodologies are proposed from the use of relatively small number of simulation runs (Christiaens and Feyen, 2001; Baroni et al., 2010; Hengl et al., 2010; Chen et al., 2011) to Monte Carlo approaches where a high number of simulations are required and the comparison with direct measurements are considered (Beven and Freer, 2001; Vrugt et al., 2008a). In general,

* Corresponding author. Tel.: þ49 3319772805; fax: þ49 3319772092. E-mail address: [email protected] (G. Baroni). 1364-8152/$ e see front matter Ó 2013 Elsevier Ltd. All rights reserved. http://dx.doi.org/10.1016/j.envsoft.2013.09.022

all the studies refer to the evaluation of the effect of the error in the model results explicitly considering the uncertainty in data, parameters, model structure or output observation or a combination of them (Kavetski et al., 2006; Ajami et al., 2007; Renard et al., 2011; Reichert and Mieleitner, 2009). In Saltelli et al. (2006), SA is defined as the study of how the uncertainty in the model output can be apportioned to the different sources of input. SA methods can be classified as either local or global. In local sensitivity analysis each factor is perturbed in turn while keeping all the others fixed at their nominal value. Local sensitivity measures can easily be computed numerically by performing multiple simulations varying the inputs around a nominal value and computing partial derivatives (Saltelli et al., 2006; Spruill et al., 2000; Holvoet et al., 2005; Saltelli and Annoni, 2010). On the other hand, Global Sensitivity Analysis (GSA) studies the effects of input variations on the outputs in the entire allowable ranges of the input space and they account for the effects of interactions between different inputs. GSA methods range from qualitative screening (Morris, 1991; Campolongo et al., 2011; Saltelli et al., 2010) to quantitative techniques based on variance decomposition in which

G. Baroni, S. Tarantola / Environmental Modelling & Software 51 (2014) 26e34

2. The General Probabilistic Framework (GPF) The General Probabilistic Framework (GPF) proposed for the uncertainty and global sensitivity analysis is shown in Fig. 1. Its steps are described as follows. The first step identifies the experimental site and the goal of the study, which includes the setting of a scalar objective function (called variable of interest in de Rocquigny et al., 2008). The objective function can be defined without any limitation but it has to be appropriate for the specific case study. In

2. Collect the data and define the model(s) 3. Define the sources of uncertainty Ui 4. Create ni realisations for each Ui

5. Associate a UD Fi [ 1…ni ] for each Ui

6. sampling Fi 7. Run the model(s)

10. Refine the model set up

A - Definition

1. Define the area and the goal

B - Analysis

the Fourier amplitude sensitivity test (FAST) (Cukier et al., 1978) and Sobol’ methods (Sobol’, 1993) are the most widely investigated (Cibin et al., 2010; Xu and Gertner, 2007; Refsgaard et al., 2007; Tang et al., 2007a,b; Yang et al., 2008; Yang, 2011; Nossent et al., 2011; Zhan et al., 2013). GSA is recommended in environmental model applications because it does not assume any specific relationship between input and model predictions and so they can be applied to any kind of model (e.g., Rosolem et al., 2012; Makler-Pick et al., 2011). However, despite the variety of GSA tools available, in most of the cases these methods are generally applied on scalar and independent parameters as they cannot easily be adopted to consider other sources of uncertainty. Scalar factors are in fact quantities that can be fully defined by a single probabilistic distribution. This distribution can be considered in sampling designs and different methods can be applied (Saltelli et al., 2000). On the other hand, environmental models are also affected by uncertainty in non-scalar factors, as it is the case for example of boundary conditions (e.g., Oudin et al., 2010), distributed and correlated parameters (Legleiter et al., 2011; Moreau et al., 2013) or model structures (e.g., Clark et al., 2008). These types of input factors increase the complexity of the analysis of the model and they limit the possibility to apply sensitivity methods. Some studies tried to overcome these limitations. In the application of distributed models some authors for example consider in the analysis the same perturbation for each cell (van Griensven et al., 2006) or they execute the analysis for each model cell independently (Tang et al., 2007a). In some cases the analysis of random errors in rainfall measurement is also conducted with the use of multipliers for each storm event (Renard et al., 2011). However, these different strategies cannot be extended to all the possible sources of uncertainty. Moreover, they often rely on strong simplifications of the description of the uncertainty (e.g., lumped parameters for distributed models). So, in many cases, the assessment of non-scalar and correlated sources of uncertainty is still made by One-at-A-Time approaches, in which the analyst quantifies the effect, on the model prediction, of a new source of uncertainty that is added to the existing ones (e.g., Mazzilli et al., 2012). In this sense, the limited body of literature that considers uncertainty and sensitivity analysis with all and different sources of uncertainty highlights the importance and significant challenges posed by this topic (Vázquez et al., 2008; Vrugt et al., 2009; Renard et al., 2010). In this context, this paper proposes a General Probabilistic Framework (GPF) based on Monte Carlo simulation for the uncertainty and global sensitivity analysis of deterministic models. In this framework non-scalar sources of uncertainty can be explicitly considered without any constraint (e.g., spatiallydistributed parameters, data, observations and alternative models). The framework could also include correlated uncertainty sources, if present. The framework is applied to the SoileWatere AtmosphereePlant model SWAP (Kroes and van Dam, 2003), a 1D hydrological model for the transport of water, solutes and heat in unsaturated and saturated soils. The proposed framework can be extended to a wide variety of modelling applications, also when direct measurements of model output are not available.

27

8. UA good?

NO

9b. SA

YES

9a. Use the model Fig. 1. Flowchart of the General Probabilistic Framework (GPF).

case measurements of the model output are not available, the variability of the simulated output can be directly considered in the analysis. The second step consists in the collection of all necessary data and the selection of the simulation model for the specific application. In the third step all the sources of uncertainty Ui are defined (i.e. input data, distributed parameters, alternative model structures and, if available, observations). This is an utterly important step of the analysis for which several aspects have to be considered (e.g., Beven et al., 2011; Beven and Westerberg, 2011). In particular, the uncertainty of each source should be characterized using all available information: measurements, estimations, physical bounds considerations and expert opinion. In case of complex model applications, the distinction between aleatory and epistemic sources of uncertainty (e.g., lack of knowledge) is also considered a good practice for a realistic evaluation of the model (e.g., Beven and Young, 2013). If the analysis is exploratory, then also rather crude assumptions may be adequate (see Helton, 1993 for more details). In the fourth step, for each specific source of uncertainty, n independent realizations are generated without any constrain (e.g., different set of correlated parameters or different input time series). The number of realizations has to be large enough so that the generated sample is representative of the uncertainty associated to that source. An increasing number of realizations could be also considered to verify the stability of the results and the assumption introduced to treat the specific source of uncertainty (e.g., Beven and Young, 2013). In step 5, each realization is associated to an integer number in the range (1, n). If model uncertainty is present, e.g., m alternative model structures are available, each model structure is associated to an integer number in the range (1, m).

28

G. Baroni, S. Tarantola / Environmental Modelling & Software 51 (2014) 26e34

Therefore, we can associate a discrete uniform distribution Fi [1.n] to each input and, in the presence of model uncertainty with m available alternative models, we can associate a discrete uniform distribution Fi [1.m]. The sampling is carried out for the discrete factors Fi (step 6). In step 7, the model (or the different models if several structures are considered) is run considering the combinations created by sampling Fi. Uncertainty analysis (UA) of the variable of interest is carried out in step 8. If its uncertainty is deemed acceptable, as defined for the specific application, the model results can be used for decision-making (step 9a). Otherwise (step 9b) sensitivity analysis is necessary to quantify the relative contribution of the different sources of uncertainty (i.e. uncertainty decomposition). The identified most influencing factors can be object of additional investigation (step 10) to improve their knowledge base (going back to step 2) with the final objective of reducing the uncertainty of the variable of interest below acceptable limits. In the step 10, several methodologies can be applied, from automated calibration algorithms of different complexity (e.g., Vrugt et al., 2008b) to direct measurements conducted in the field. For an extensive review of methods we refer to Matott et al. (2009). The use, at step 5 of the framework, of a discrete scalar factor of the size of the realizations generated, enables us to extend the GSA also to non-scalar sources of uncertainty. This approach was introduced by Crosetto and Tarantola (2001), who proposed the use of a sensitivity analysis of a binary input to ‘switch’ the uncertainties of a rainfall intensity map on and off at the same rate (i.e. for N/2 runs, the switch is set to off and for the remaining N/2 runs it is set to on), allowing their relative importance to be determined. The same approach was then improved by Lilburne et al. (2003) and Lilburne and Tarantola (2009) who explicitly introduced the discrete uniform distribution associated to the different realizations of each specific source of uncertainty as considered in this framework. 3. The global sensitivity analysis The sensitivity analysis considered in the framework is a variance-based method. As discussed before, this method is a global approach, in which all sources of uncertainties are varied simultaneously, and model-free, as they have the capability to compute sensitivity indices regardless to the linearity or monotonicity, or other generic assumptions on the underlying model. In this approach, first order (Si) and total order sensitivity indices (ST) are estimated as follows:

Si ¼

V½EðYjXi Þ VðYÞ

(1)

ST ¼

E½VðYjXwi Þ VðYÞ

(2)

where Xwi indicates the array of all inputs except Xi, V and E denote variance and expectation operators. High values of S mean high sensitivity. While first-order effects quantify the importance of a single input by itself, total order indices measure the overall importance of a given input, taking into account its interactions with all other possible inputs. Sensitivity information can be useful for model improvement, parameter estimation, or model simplification. In particular, two settings for sensitivity analysis can be applied using the first and total order sensitivity indices calculated: factor prioritization and factor fixing (Saltelli et al., 2006). Factor prioritization identifies, through first order indices, the factors Fi that allow for achieving the

major reduction in the uncertainty in the output. Factor prioritization helps improving the allocation of resources in defining priorities in monitoring activities and their design. In the factor fixing setting, small values of the total order sensitivity indices allow for identifying the sources of uncertainty that are not relevant for the model output. In factor fixing, the sources of uncertainty can be fixed to their nominal value leaving the variance of the model output unchanged (Tang et al., 2007a; Nossent et al., 2011). Factor fixing has a useful role in the simplification of simulation models. The settings can be used in a loop for a revision and improvement of the model according to the proposed framework. The calculation of the sensitivity indices requires a high number of model runs that is often not feasible with complex models. For this, several methods are proposed for the estimation considering different sampling designs with a relative smaller number of samples (Saltelli et al., 2000). In the specific framework the estimation of the sensitivity indices is done based on the variance decomposition proposed by Sobol’ (2001) and further developed by Saltelli (2002) and Saltelli et al. (2010). As shown from several authors (e.g., Tang et al., 2007b; Yang, 2011; Glen and Isaacs, 2012), this method yields more accurate sensitivity rankings than other approaches. Moreover, as discussed in Lilburne and Tarantola (2009), with this approach it is not necessary to sort the sequence of realizations according to the objective function because the analysis is independent of the order of the input considered. In this way, the analysis can be directly performed on the scalar input factor used to associate the different realizations without any other transformation necessary in other approaches (Lilburne et al., 2003). In the case of independent input factors, the estimation of the sensitivity indices is conducted with a quasi-random sampling and a total number of model runs Nr ¼ N (k þ 2), where N is the number of samples and k the number of input factors considered. The analysis can also be extended to correlated input factors with the method recently introduced by Kucherenko et al. (2012). In this case, the correlation matrix between the input factors has to be defined and the computational cost increases to a total number of model runs Nr ¼ N (2k þ 2). Readers interested in the details of the Saltelli/Sobol technique and in its implementation to correlated and spatially-dependent inputs can make reference to the specific studies (Saltelli et al., 2010; Lilburne and Tarantola., 2009; Kucherenko et al., 2012). 4. Case study 4.1. Goal, experimental site and model The General Probabilistic Framework is applied to SWAP model, a widely used 1D hydrological model for the transport of water, solutes and heat in unsaturated and saturated soils based on Penman-Monteith and Richards’ equations (Kroes and van Dam, 2003). The model demonstrated good capability to reproduce the main component of the water balance in several and different applications (van Dam et al., 2008). The framework is applied (i) to define the uncertainty in the model output and quantify the main sources of uncertainty, (ii) to optimize the monitoring activities and increase the model performance. The model application is conducted for a cropped flat field of 30 ha located in Bornim (Brandenburg, Germany), where surface run off can be neglected and the 1D vertical fluxes play the most important role. The area situated 40 m a.s.l. is characterized by mean annual precipitation of 595 mm and minimum and maximum daily values of 15  C (February) and 30  C (July), respectively (Meteorological Station Potsdam Telegrafenberg e Germany). Soil texture of the site was reported to be dominated up

G. Baroni, S. Tarantola / Environmental Modelling & Software 51 (2014) 26e34

to 1 m by 75% sand content, 17.2% silt content and 7.8% clay content (Gebbers et al., 2009) referring to a loamy-sand soil classification (USDA). The groundwater level is w5 m below the surface as suggested by information from the State Environmental Agency based on a groundwater well nearby. For more details of the experimental site and monitoring activities see Rivera Villarreyes et al. (2011). The study was conducted for MayeSeptember 2011, and focuses on the temporal variability of evapotranspiration, soil moisture in the root zone and bottom fluxes below the root zone simulated by the model. The model performance is evaluated considering the variability of the cumulative model outputs in case of evapotranspiration and bottom fluxes. The performance of the soil moisture dynamic simulated is done comparing the soil moisture measured by Theta Probes (Delta-T Devices, Cambridge, UK) installed in the field at three depths (0, 20 and 40 cm). In this case the RMSE (m3 m3) between simulated and measured mean soil moisture in the root zone (50 cm) is calculated for the period considered. The RMSE is commonly used for the assessment of this type of model and output and it is also used here for comparisons with other studies (e.g., Islam et al., 2006; Guber et al., 2009). Statistical measures other than RMSE can be appropriate for other case studies and should be considered in the framework. 4.2. Materials and methods 4.2.1. Definition of the sources of uncertainty The characterization of the sources of uncertainty is an important step of the analysis and of the framework considered (e.g., Beven et al., 2011; Beven and Westerberg, 2011). In this case study, during the season, direct monitoring activities were conducted to collect the input and parameters to set up the model and to define the uncertainty in the data available. In particular, meteorological data (i.e. temperature, air humidity, solar radiation, wind velocity and precipitation) were available from Meteorological Station Potsdam Telegrafenberg. However, the station is located approximately 6 km east of the experimental site. So, some measurements in the field were collected during the season and compare to the reference one to define the range of uncertainty for each variable. No systematic error was found in the time series but a daily random error was detected due to local events that occur shifted in time or possible error of measurements. In according to the comparison between data from the weather station and the direct measurements in the field, a random error was introduced to the daily nominal values of each variable (Table 1). Each variable was physicallyconstrained to avoid unrealistic values (e.g., negative value for rainfall). As it is possible to see, the relative error in the rain time series was relatively low. On the other hand particular differences were achieved when considering air humidity and wind velocity. These can have a big effect on the uncertainty in the estimation of the evepotranspiration rate. In 2011 the field was cropped with sunflowers. Crop parameters to set up the model were based on Allen et al. (1998). Field measurements of crop height Hc (cm) were conducted in the field biweekly to define the uncertainty on the parameter presented in literature. The same relative level of uncertainty detected for the crop height was considered for the uncertainty of the other crop parameters necessary to set up the model, i.e., Leaf Area Index LAI () and root depth Rd (cm) (Table 2). During the season 2011 direct soil samples were also collected in the field at different depths, for analysis of the soil texture and bulk density. Then Pedotransfer Functions (PTFs) were used for the estimation of the soil hydraulic parameters used by the model (i.e. parameters of the van Genucthen and Mualem equations). In particular, considering the ranges of the soil texture in the field, a homogeneous soil profile was considered and PTFs of Zacharias and Wessolek (2007) and PTFs of Rawls and Brakensiek (1989) were applied for the estimation of the parameters of the soil retention curve and for the estimation of the saturated hydraulic conductivity Ksat (cm d1), respectively. The uncertainty of each parameter was then fixed considering ranges in the parameters as proposed in literature (e.g., Ungaro et al., 2005; Merdun et al., 2006; Baroni et al., 2010) (Table 2). The definition of the soil parameters variability could be also improved taken into account more then one PTF, see for instance Guber et al. (2009). However, in this study the high percentage of sand in the experimental site limited the availability of PTFs applicable. The interaction between root zone (0e50 cm) and groundwater can be neglected in the specific case studies (groundwater level below 5 m). Then, free drainage was set as bottom boundary condition and no other options were considered to account for this possible source of uncertainty. Finally a warm-up period was used to eliminate the sensitivity to the initial conditions. Uncertainty in model input and parameters may justify the use of simplified model, as long as the resulting structural error is smaller than the model input error.

29

Table 1 Ranges of uncertainty defined for the weather data: ranges of the daily random error introduced in the time series. Variable

Range

Air temperature ( C) Air humidiy (hPa) Wind (m s1) Global radiation (W m2) Rain (mm d1)

1.8 0.27 0.9 21 1.4

To account for this, the number of vertical nodes considered in the model for the soil discretization of the root zone was gradually decreased from 50 to 5, corresponding to the extremes of 1 and 10 cm increments, respectively. A similar approach to account for uncertainty in model structure in unsaturated hydrological models was introduced by Schoups and Hopmans (2006). Other parameters are present in SWAP model and we refer to the specific manual for an exhaustive description (Kroes and van Dam, 2003). The values of these parameters are considered well known for the case study presented. So, they are not treated as being affected by uncertainty and they are not included in the framework. However, it is important to note that this assumption should not be valid in all the model applications and, in case, other sources of uncertainty should be considered in the framework. Finally, field measurements of the output of the model are not free of error and they can limit the capability of the assessment of the model (e.g., Liu et al., 2009). In this particular case study, initially, no specific calibration of the soil moisture probes are considered and, according to the manual, a random error is introduced in the soil moisture measured (i.e., RMSE w 0.04 m3 m3).

4.2.2. Set up of the General Probabilistic Framework (GPF) Considering the monitoring activities and the discussion reported above, the sources of uncertainty were grouped in five main classes: model structure (soil discretization M), input (weather data e W), time-varying parameters (crop parameters e C), scalar parameters (soil properties e S) and observations (soil moisture measured e obs). For the soil discretization, eight model structures were created with vertical nodes decreased from 50 to 5, corresponding to the extremes of 1 and 10 cm increments, respectively. For the other sources of uncertainty, 128 realizations were considered to be good to describe in an efficient way the space of the variability. Further analysis can consider an increasing number of realizations. This was not considered in the present study. As an example, the realizations of retention curve, unsaturated hydraulic conductivity curve, height crop Hc (cm) and root depth Rd (cm) are plotted in Fig. 2. The realizations of each source of uncertainty were then associated, as proposed in the GPF, to a scalar input factor M ¼ [1.8], W ¼ [1.128], C ¼ [1.128], S ¼ [1.128], obs ¼ [1.128], respectively. According to the method presented in Saltelli et al. (2010), a quasi-random sampling design is used to sample the five discrete uniform distributions. The five groups of input factors are assumed to be independent and no correlations are considered in the sampling design. This assumption is commonly considered in the literature given that no information regarding possible correlation is usually available (e.g., Ciriello et al., 2013; Zhan et al., 2013). However, further research should be focused on the evaluation of the effect of this assumption on the experimental results. This was beyond the scope of the present study. The simulations were run using a number of sampling points N ¼ 1024 corresponding to a total number of runs NR ¼ N (k þ 2) ¼ 7168. MatLAB codes developed at the Join Research Centre (JRC, 2010) were used to run the sensitivity analysis. The sensitivity indices are calculated using a sequential approach to monitor convergence of the estimation process. Finally, bootstrap is used to estimate confidence intervals for the sensitivity indices. In the bootstrap, the samples generated by the Sobol’s sequence are resampled Nb times when calculating

Table 2 Levels of uncertainty defined for the crop parameters: mean and random error introduced at maximum stage; Soil: mean and random error of parameters of Van Genuchten eq. (qr and L were fixed to 0.05 (m3 m3) and 0.5 () respectively). All the distributions are normal except Ksat that was defined as log normal.

Crop parameters (C)

Soil parameters (S)

Parameter

Mean

CV

Hc max (cm) Rd max (cm) LAI max () qs (m3 m3) n () a (cm1) Ksat (cm d1)

130 40 2.5 0.38 1.26 0.08 200

8% 8% 8% 5% 1% 12% 40%

30

G. Baroni, S. Tarantola / Environmental Modelling & Software 51 (2014) 26e34

Fig. 2. Realizations considered in the analysis: soil properties a) retention curve and b) unsaturated hydraulic conductivity curve where h refer to hydraulic head (cm); crop parameters c) height crop Hc (cm) and d) root depth Rd (cm).

the sensitivity indices for each factor Fi, resulting in a distribution of the sensitivity indices. The resample dimension Nb was set to 1000 based on prior literature discussions. Readers interested in a detailed description of the bootstrap implementation can see Archer et al. (1997), Efron and Tibshirani (1993) and Saltelli et al. (2010).

4.3. Results 4.3.1. Uncertainty and sensitivity analysis Fig. 3 shows the probability distribution of (a) the RMSE (m3 m3) calculated between the simulated mean soil moisture of the root zone and the measurements collected in the field, (b) the cumulative evapotranspiration (ETa) and (c) the bottom fluxes (Qbot) below the root zone (50 cm). It has to be noted that run off simulated by the model was always zero for all the simulations and the three output considered fully describe the water balance of the soileplanteatmosphere system. The RMSE results show a general good performance of the model even without a specific model

calibration, underlining the good capability of the model to simulate the process. The uncertainty in the estimation of the evapotranspiration is also relatively low with mean value and range of w280  45 mm respectively. On the other hand, the bottom fluxes are quite limited but the relative error is quite important, with range of w55  45 mm. For both processes the intervals of uncertainty are comparable. The results of the sensitivity analysis are shown in Fig. 4. For each output considered, first (Si) and total (ST) sensitivity indices were estimated. The differences between the two indices (STeSi) are calculated to show the interactions between the factors. The Sobol/Saltelli method shows a clear ranking of the input factors in order of importance, even with a relatively low number of simulations. It can be noted that the total sensitivity estimates are more stable than the first order ones. This is confirmed by the width of the error bars for total indices, estimated with the bootstrap method at the highest sample size, which are much smaller than those of the first order indices.

Fig. 3. Probability distribution function (%) of the cumulative values of the RMSE of the soil moisture qv, evapotranspiration (ETa) and bottom flux below the root zone (Qbot).

G. Baroni, S. Tarantola / Environmental Modelling & Software 51 (2014) 26e34

31

Fig. 4. First (Si), total (ST) sensitivity indices of the different factors considered [model structure (M), weather data (W), crop parameters (C), soil properties (S) and observation (obs)] plotted for the three output variables considered [RMSE qv, evapotranspiration ETa and bottom fluxes Qbot]. Bar error represents the confidence estimation with bootstrap method. In the lower three graphs, the differences between Si and ST, estimated with the maximum sample size, are calculated.

The results of the sensitivity analysis show that the uncertainty in the model structure (M, number of nodes used for the soil discretisation) has no effect on the simulation results in all the model output considered (low sensitivity indices). The same consideration arises for the crop parameters (C). Just when bottom fluxes are considered (Qbot), the sensitivity indices of this factor slightly increase to a value of 0.25 (). As we expected, the uncertainty in the measured soil moisture (obs) and in the soil properties (S) control the uncertainty in the soil moisture simulated by the model (high value of sensitivity indices). Moreover, it is interesting to note that these factors mainly contribute via interactions as the difference between ST and Si is high. Consequently, the improvement in the knowledge (i.e., reduce uncertainty) of just one of these factors will not improve the performance of the model. Finally, a small interaction is also detected with the weather data and it will be further discussed in the next section. When the fluxes are considered as variables of interest, we find that soil properties (S) are not important sources of uncertainty (the first order sensitivity indices are almost zero). This means that even more precise uncertainty distributions of the soil properties will not improve the performance of flux simulation. Sensitivity analysis shows that, to improve performance in flux simulation, we need to improve our knowledge (i.e., reduce uncertainty) on weather condition (W), by installing a new meteorological station close to the experimental site or improving the extrapolation of the existing weather data. Finally, no particular differences are detected between total and first sensitivity indices,

underlying the general additivity of the model and the absence of interactions between input factors when these outputs are considered. This is consistent with the type of model considered and the specific case study. The length of the crop period is in fact considered constant in the simulations without any feedback between weather conditions and crop development. Moreover, the actual evapotranspiration rate is almost close to the potential, meaning that the variability is mainly due to the weather conditions and the soil properties play a negligible role. However, further research could extend the analysis to detect possible interactions of the variables within each group. 4.3.2. Towards a reduction of uncertainty in model output The result of the sensitivity analysis can be used to refine the model as reported in step 10 of Fig. 1. For this case study, based on the result of the previous sensitivity analysis, four hypothetical scenarios are explored decreasing the level of uncertainty of each source considered. In particular: WS. The sensitivity analysis showed that the error in the measured soil moisture (obs) is relevant for the assessment of the simulated soil moisture. For this reason, we assume that a better calibration of the soil moisture probe has been conducted eliminating the uncertainty in the observations (e.g., by collecting gravimetric soil samples in the field). Therefore, the nominal values of the measured soil moisture are considered for the model comparison. The sensitivity analysis also showed that the model structure and the crop parameters have small values

32

G. Baroni, S. Tarantola / Environmental Modelling & Software 51 (2014) 26e34

Fig. 5. Probability distribution of the output simulated (RMSE of soil moisture, cumulative evapotranspration ETa and bottom fluxes Qbot) in the four hypothetical scenarios (WS, W, S and Ref). Definitions of the scenarios are reported in the text.

of the total order sensitivity indices. For this reason, these input factors are fixed to their nominal values as they do not affect the model simulations. In this scenario the model outputs are just affected by the uncertainty in weather data (W) and soil properties (S). Therefore, we refer to this scenario as WS. W. The sensitivity analysis showed that soil properties are mostly responsible for uncertainty in the simulated soil moisture (high values of Si). For this reason, soil moisture measurements are used for the calibration of the soil parameters and subsequently considered for the reduction of the uncertainty in the model output. In particular, based on the results of scenario WS, we select the simulations that minimize the RMSE between measured and simulated soil moisture as proposed in similar studies (e.g., Islam et al., 2006; Guber et al., 2009). In this way, the model is just affected by uncertainty in weather data (W) and we refer to this scenario as W; S. The sensitivity analysis showed that weather data are relevant for simulated fluxes (high values of Si). For these model output we do not have direct measurements to compare with the simulations as it was conducted for the soil moisture. In this case, based on scenario WS and assuming, as an example, that the weather station is closer to the experimental field, weather data are fixed to their nominal values without any uncertainty. In this way, the model is just affected by uncertainty in soil properties (S) and we refer to this scenario as S; Ref. Finally, based on scenario S (the model is just affected by uncertainty in soil properties), the soil properties are calibrated minimizing the RMSE between measured and simulated soil moisture. We consider this as reference scenario (Ref) without any uncertainty.

The results are presented in Fig. 5. In the case WS (i.e., with uncertainty in model structure, crop parameters and observations fixed to their nominal values), the model reproduces the same uncertainty quantified in the case where all the sources of uncertainty were considered (Fig. 3). These results confirm the sensitivity analysis. In particular it is interesting to note that the uncertainty in the model output is not reduced eliminating the uncertainty in the observations because this factor shows strong interactions with the other factors (Fig. 4, height value of STeSi). In the case W, the calibration conducted for the soil parameters achieve a reasonable result for this type of model (RMSE between 0.0086 and 0.0139 m3 m3). Similar results were obtained in other case studies (e.g., Islam et al., 2006; Guber et al., 2009). On the other hand, the calibration of the soil properties does not have a big effect on the reduction of the uncertainty in the predicted fluxes (ETa and Qbot) as just w20% of the uncertainty is reduced. These also confirm the results obtained by the sensitivity analysis. In the case S (weather data are fixed to the nominal value and soil properties are the only source of uncertainty), the uncertainty in the predicted soil moisture is not particularly affected by fixing weather data. On the other hand, a considerable amount of the uncertainty in estimated fluxes, especially ETa, is eliminated, i.e. the range is reduced to w65% of the initial range. These findings confirm the results obtained by the sensitivity analysis that weather factors are important for predicted fluxes. Finally, the reference simulation obtained with no sources of uncertainty and calibrating the soil properties is plotted in all the graphs of Fig. 5 (vertical black dashed line). It is interesting to note that the RMSE obtained with this simulation (RMSE ¼ 0.0107 m3 m3) is higher then the best RMSE of the case W (RMSE ¼ 0.0086 m3 m3).

G. Baroni, S. Tarantola / Environmental Modelling & Software 51 (2014) 26e34

This result indicates that the weather data might be unrealistic for the experimental site considered. It has to be noted that this data are collected from a weather station 6 km far from the site. In this sense, a new weather station closer to the experimental site or a methodology to extrapolate the data for the specific field should be considered. This result was also suggested in the sensitivity analysis by the interactions detected in this input factor (in Fig. 4, when the RMSE is considered, the difference between ST and Si are small but not negligible for the factor W). 5. Summary and conclusions In this study a General Probabilistic Framework (GPF) is proposed for uncertainty and global sensitivity analysis which is also valid for non-scalar and correlated inputs. The framework is based on the Sobol/Saltelli sensitivity analysis and on the use of discrete uniform factors to select realizations of non-scalar sources of uncertainty (as proposed by Lilburne and Tarantola, 2009). The framework could also include correlated uncertainty sources, thanks to the approach proposed by Kucherenko et al. (2012). The framework is conceptually simple and relatively easy to implement, it requires no modifications to existing source codes of simulation models and it can be applied also when direct observations of the model output are not available. The framework was used for the assessment of the Richardsbased hydrological model SWAP (Kroes and van Dam, 2003). The aim was (i) to explore the effect on the hydrological balance terms of the uncertainty of five different sources of input factors: model structure (soil discretsation), input (weather data e W), timevarying parameters (crop parameters e C), scalar parameters (soil properties e S) and observations (soil moisture measurements e obs) and (ii) to identify the major sources of uncertainty to be optimized. In the framework, errors were introduced based on direct monitoring activities. The main conclusions are summarized as follows.  The Sobol/Saltelli sensitivity analysis shows a clear rank of the inputs considered even with a limited number of model evaluations but care has to be taken in the estimation of the indices. The use of bootstrap is suggested to monitor the error of the estimates.  The pattern of soil moisture is quite well simulated. Further improvement in the simulation of the soil moisture could be obtained by improving the accuracy of the field measurements (obs) and by estimation of soil properties (S). These factors have in fact the highest sensitivity indices (see Fig. 4).  The major source of uncertainty related to the prediction of evapotranspiration and of bottom fluxes at 50 cm is the weather data (W). In this way, the calibration of soil properties will not reduce the uncertainty in these model predictions. To reduce prediction uncertainty it is necessary to reduce uncertainty on W e.g., by installing a new meteorological station close to the experimental site or improving the extrapolation of the existing weather data.  Overall, these results confirm the relative importance of the uncertainty in input data and the need to consider multiple output variables for a proper assessment of the model, as it has been shown in other hydrological studies (e.g., Yatheendradas et al., 2008; Van Werkhoven et al., 2009; Vázquez et al., 2008). Moreover, the results suggest the different strategies that should be applied depending on the output considered and the availability of measurements to constrain the model. In particular, the results of the sensitivity analysis can identify which (and in case where and when) model output to consider if direct measurements are

33

not available and the analysis is applied directly to the variance of the model output. In conclusion, the study shows the capability of the General Probabilistic Framework (GPF) to decompose the total uncertainty of the model according to the different sources (i.e., input, parameters, model structure and observations). These sources of uncertainty are analysed simultaneously and the relative contribution is quantify directly (first order sensitivity index) or via interactions with other factors (total order sensitivity index). This information is important for a global assessment of the model and to identify the priorities required to improve the performance of the model. Parameter estimation approaches of different complexity, or direct measurements in the field, can be considered for this purpose. It has to be noted that the choice of the methodologies to refine the models strongly depends on the case study, the sources of uncertainty, the model selected and the availability of measurements of the model output. It is beyond the scope of the general framework to focus on one specific methodology. The framework is rather developed to be able to consider all the sources of uncertainty without any limitation and to help the choice of the methodology to improve the performance of the model. In this way, the GPF can be used to improve the model application in a goal-oriented and continuous learning process. Acknowledgements We would like to thank the three anonymous reviewers. Critics and suggestions have been very useful to improve the quality of the manuscript. References Ajami, N.K., Duan, Q., Sorooshian, S., 2007. An integrated hydrologic Bayesian multimodel combination framework: confronting input, parameter, and model structural uncertainty in hydrologic prediction. Water Resour. Res. 43. Allen, R., Pereira, L.S., Raes, D., Smith, M., 1998. FAO, Irrigation and Drainage Paper 56, Crop Evapotranspiration, Guidelines for Computing Crop Water Requirements. Arbia, G., Griffith, D., Haining, R., 1998. Error propagation modelling in raster GIS: overlay operations. Int. J. Geogr. Inf. Sci. 12, 145e167. Archer, G.E.B., Saltelli, A., Sobol, I.M., 1997. Sensitivity measures, anova-like techniques and the use of bootstrap. J. Stat. Comput. Simul. 58, 99e120. Baroni, G., Facchi, A., Gandolfi, C., Ortuani, B., Horeschi, D., van Dam, J.C., 2010. Uncertainty in the determination of soil hydraulic parameters and its influence on the performance of two hydrological models of different complexity. Hydrol. Earth Syst. Sci. 14, 251e270. Beven, K., Freer, J., 2001. Equifinality, data assimilation, and uncertainty estimation in mechanistic modelling of complex environmental systems using the GLUE methodology. J. Hydrol. 249, 11e29. Beven, K., Westerberg, I., 2011. On red herrings and real herrings: disinformation and information in hydrological inference. Hydrol. Process. 25, 1676e1680. Beven, K., Young, P., 2013. A guide to good practice in modeling semantics for authors and referees. Water Resour. Res. 49, 1e7. Beven, K., Smith, P.J., Wood, A., 2011. On the colour and spin of epistemic error (and what we might do about it). Hydrol. Earth Syst. Sci. 15, 3123e3133. Campolongo, F., Saltelli, A., Cariboni, J., 2011. From screening to quantitative sensitivity analysis. A unified approach. Comput. Phys. Commun. 182, 978e988. Chen, J., Brissette, F.P., Poulin, A., Leconte, R., 2011. Overall uncertainty study of the hydrological impacts of climate change for a Canadian watershed. Water Resour. Res. 47. Christiaens, K., Feyen, J., 2001. Analysis of uncertainties associated with different methods to determine soil hydraulic properties and their propagation in the distributed hydrological MIKE SHE model. J. Hydrol. 246, 63e81. Cibin, R., Sudheer, K.P., Chaubey, I., 2010. Sensitivity and identifiability of stream flow generation parameters of the SWAT model. Hydrol. Process. 24, 1133e1148. Ciriello, V., Guadagnini, A., Di Federico, V., Edery, Y., Berkowitz, B., 2013. Comparative analysis of formulations for conservative transport in porous media through sensitivity-based parameter calibration. Water Resour. Res. 49, 1e15. Clark, M.P., Slater, A.G., Rupp, D.E., Woods, R.A., Vrugt, J.A., Gupta, H.V., Wagener, T., Hay, L.E., 2008. Framework for Understanding Structural Errors (FUSE): a modular framework to diagnose differences between hydrological models. Water Resour. Res. 44.

34

G. Baroni, S. Tarantola / Environmental Modelling & Software 51 (2014) 26e34

Crosetto, M., Tarantola, S., 2001. Uncertainty and sensitivity analysis: tools for GISbased model implementation. Int. J. Geogr. Info. Sci. 15, 415e437. Cukier, R.., Levine, H.., Shuler, K., 1978. Nonlinear sensitivity analysis of multiparameter model systems. J. Comput. Phys. 26, 1e42. de Rocquigny, E., de Victor, N., Tarantola, S. (Eds.), 2008. Uncertainty in Industrial Practice. Wiley. Efron, B., Tibshirani, R., 1993. An Introduction to the Bootstrap. Chapman Hall, NewYork, USA. Gebbers, R., Lück, E., Dabas, M., Domsch, H., 2009. Comparison of instruments for geoelectrical soil mapping at the field scale. Near Surf. Geophys. 7, 179e190. Glen, G., Isaacs, K., 2012. Estimating Sobol sensitivity indices using correlations. Environ. Model. Softw. 37, 157e166. Guber, A.K., Pachepsky, Y.A., van Genuchten, M.T., Simunek, J., Jacques, D., Nemes, A., Nicholson, T.J., Cady, R.E., 2009. Multimodel simulation of water flow in a field soil using pedotransfer functions. Vadose Zone J. 8, 1. Helton, J.C., 1993. Uncertainty and sensitivity analysis techniques for use in performance assessment for radioactive waste disposal. Reliab. Eng. Syst. Safe. 42, 327e367. Hengl, T., Heuvelink, G.B.M., van Loon, E.E., 2010. On the uncertainty of stream networks derived from elevation data: the error propagation approach. Hydrol. Earth Syst. Sci. 14, 1153e1165. Heuvelink, G.B.M., 1998. Error Propagation in Environmental Modelling with GIS. Taylor & Francis, London. Holvoet, K., van Griensven, A., Seuntjens, P., Vanrolleghem, P.A., 2005. Sensitivity analysis for hydrology and pesticide supply towards the river in SWAT. Phys. Chem. Earth, Parts A/B/C 30, 518e526. Islam, N., Wallender, W.W., Mitchell, J.P., Wicks, S., Howitt, R.E., 2006. Performance evaluation of methods for the estimation of soil hydraulic parameters and their suitability in a hydrologic model. Geoderma 134, 135e151. JRC, 2010. Matlab Codes for the Sensitivity Analysis. Joint Research Centre of the European Commission. Downloadable at: http://ipsc.jrc.ec.europa.eu/?id¼756. Kavetski, D., Kuczera, G., Franks, S.W., 2006. Bayesian analysis of input uncertainty in hydrological modeling: 1. Theory. Water Resour. Res. 42. Kroes, J.G., van Dam, J.C., 2003. Reference Manual SWAP Version 3.0.3. Alterrarapport 773, ISSN 1566e7197. Kucherenko, S., Tarantola, S., Annoni, P., 2012. Estimation of global sensitivity indices for models with dependent variables. Comput. Phys. Commun. 183, 937e946. Legleiter, C.J., Kyriakidis, P.C., McDonald, R.R., Nelson, J.M., 2011. Effects of uncertain topographic input data on two-dimensional flow modeling in a gravel-bed river. Water Resour. Res 47. Lilburne, L., Tarantola, S., 2009. Sensitivity analysis of spatial models. Int. J. Geogr. Inf. Sci. 23, 151e168. Lilburne, L.R., Webb, T.H., Francis, G.S., 2003. Relative effect of climate, soil, and management on risk of nitrateleaching under wheat production in Canterbury, New Zealand. Soil Res. 41, 699e709. Liu, Y., Freer, J., Beven, K., Matgen, P., 2009. Towards a limits of acceptability approach to the calibration of hydrological models: extending observation error. J. Hydrol. 367, 93e103. Makler-Pick, V., Gal, G., Gorfine, M., Hipsey, M.R., Carmel, Y., 2011. Sensitivity analysis for complex ecological models e a new approach. Environ. Model. Softw. 26, 124e134. Matott, L.S., Babendreier, J.E., Purucker, S.T., 2009. Evaluating uncertainty in integrated environmental models: a review of concepts and tools. Water Resour. Res. 45. Mazzilli, N., Guinot, V., Jourde, H., 2012. Sensitivity analysis of conceptual model calibration to initialisation bias. Application to karst spring discharge models. Adv. Water Resour. 42, 1e16. Merdun, H., Çınar, Ö., Meral, R., Apan, M., 2006. Comparison of artificial neural network and regression pedotransfer functions for prediction of soil water retention and saturated hydraulic conductivity. Soil Tillage Res. 90, 108e116. Moreau, P., Viaud, V., Parnaudeau, V., Salmon-Monviola, J., Durand, P., 2013. An approach for global sensitivity analysis of a complex environmental model to spatial inputs and parameters: a case study of an agro-hydrological model. Environ. Model. Softw. 47, 74e87. Morris, M.D., 1991. Factorial sampling plans for preliminary computational experiments. Technometrics 33, 161. Nossent, J., Elsen, P., Bauwens, W., 2011. Sobol’ sensitivity analysis of a complex environmental model. Environ. Model. Softw. 26, 1515e1525. Oudin, L., Moulin, L., Bendjoudi, H., Ribstein, P., 2010. Estimating potential evapotranspiration without continuous daily data: possible errors and impact on water balance simulations. Hydrol. Sci. J. 55, 209e222. Rawls, W.J., Brakensiek, D.L., 1989. Estimation of soil water retention and hydraulic properties. In: Morel-Seytoux, H.J. (Ed.), Unsaturated Flow in Hydrologic Modelling, Theory and Practice. Kluwer Academic Publishers, pp. 275e300. Refsgaard, J.C., van der Sluijs, J.P., Højberg, A.L., Vanrolleghem, P.A., 2007. Uncertainty in the environmental modelling process e a framework and guidance. Environ. Model. Softw. 22, 1543e1556. Reichert, P., Mieleitner, J., 2009. Analyzing input and structural uncertainty of nonlinear dynamic models with stochastic, time-dependent parameters. Water Resour. Res 45. Renard, B., Kavetski, D., Kuczera, G., Thyer, M., Franks, S.W., 2010. Understanding predictive uncertainty in hydrologic modeling: the challenge of identifying input and structural errors. Water Resour. Res. 46.

Renard, B., Kavetski, D., Leblois, E., Thyer, M., Kuczera, G., Franks, S.W., 2011. Toward a reliable decomposition of predictive uncertainty in hydrological modeling: characterizing rainfall errors using conditional simulation. Water Resour. Res. 47. Rivera Villarreyes, C.A., Baroni, G., Oswald, S.E., 2011. Integral quantification of seasonal soil moisture changes in farmland by cosmic-ray neutrons. Hydrol. Earth Syst. Sci. 15, 3843e3859. Rosolem, R., Gupta, H.V., Shuttleworth, W.J., Zeng, X., de Gonçalves, L.G.G., 2012. A fully multiple-criteria implementation of the Sobol’ method for parameter sensitivity analysis. J. Geophys. Res. 117. Saltelli, A., 2002. Making best use of model evaluations to compute sensitivity indices. Comput. Phys. Commun. 145, 280e297. Saltelli, A., Annoni, P., 2010. How to avoid a perfunctory sensitivity analysis. Environ. Model. Softw. 25, 1508e1517. Saltelli, A., Chan, K., Scott, E.M., 2000. Sensitivity Analysis, first ed. Wiley. Saltelli, A., Ratto, M., Tarantola, S., Campolongo, F., 2006. Sensitivity analysis practices: strategies for model-based inference. Reliab. Eng. Syst. Safe. 91, 1109e 1125. Saltelli, A., Annoni, P., Azzini, I., Campolongo, F., Ratto, M., Tarantola, S., 2010. Variance based sensitivity analysis of model output. Design and estimator for the total sensitivity index. Comput. Phys. Commun. 181, 259e270. Schoups, G., Hopmans, J.W., 2006. Evaluation of model complexity and input uncertainty of field-scale water flow and salt transport. Vadose Zone J. 5, 951. Sobol’, I.M., 1993. Sensitivity estimates for nonlinear mathematical models. Math. Model. Comput. Exp. 1, 407e414. Sobol’, I.M., 2001. Global sensitivity indices for nonlinear mathematical models and their Monte Carlo estimates. Math. Comput. Simulat. 55, 271e280. Spruill, C.A., Workman, S.R., Taraba, J.L., 2000. Simulation of daily and monthly stream discharge from small watersheds using the SWAT model. Trans. ASAE 43 (6), 1431e1439. Tang, Y., Reed, P., van Werkhoven, K., Wagener, T., 2007a. Advancing the identification and evaluation of distributed rainfall-runoff models using global sensitivity analysis. Water Resour. Res. 43. Tang, Y., Reed, P., Wagener, T., van Werkhoven, K., 2007b. Comparing sensitivity analysis methods to advance lumped watershed model identification and evaluation. Hydrol. Earth Syst. Sci. 11, 793e817. Ungaro, F., Calzolari, C., Busoni, E., 2005. Development of pedotransfer functions using a group method of data handling for the soil of the Pianura Padanoe Veneta region of North Italy: water retention properties. Geoderma 124, 293e 317. van Dam, J.C., Groenendijk, P., Hendriks, R.F.A., Kroes, J.G., 2008. Advances of modeling water flow in variably saturated soils with SWAP. Vadose Zone J. 7, 640. van Griensven, A., Meixner, T., Grunwald, S., Bishop, T., Diluzio, M., Srinivasan, R., 2006. A global sensitivity analysis tool for the parameters of multi-variable catchment models. J. Hydrol. 324, 10e23. Van Werkhoven, K., Wagener, T., Reed, P., Tang, Y., 2009. Sensitivity-guided reduction of parametric dimensionality for multi-objective calibration of watershed models. Adv. Water Res. 32, 1154e1169. Vázquez, R.F., Willems, P., Feyen, J., 2008. Improving the predictions of a MIKE SHE catchment-scale application by using a multi-criteria approach. Hydrol. Process. 22, 2159e2179. Vereecken, H., Diels, J., van Orshoven, J., Feyen, J., Bouma, J., 1992. Functional evaluation of pedotransfer functions for the estimation of soil hydraulic properties. Soil. Sci. Soc. Am. J. 56, 1371e1378. Vrugt, J.A., ter Braak, C.J.F., Clark, M.P., Hyman, J.M., Robinson, B.A., 2008a. Treatment of input uncertainty in hydrologic modeling: doing hydrology backward with Markov chain Monte Carlo simulation. Water Resour. Res. 44. Vrugt, J.A., ter Braak, C.J.F., Gupta, H.V., Robinson, B.A., 2008b. Equifinality of formal (DREAM) and informal (GLUE) Bayesian approaches in hydrologic modeling? Stoch. Environ. Res. Risk Assess. 23, 1011e1026. Vrugt, J.A., ter Braak, C.J.F., Diks, C.G.H., Robinson, B.A., Hyman, J.M., Higdon, D., 2009. Accelerating Markov chain Monte Carlo simulation by differential evolution with self-adaptive randomized subspace sampling. Int. J. Nonlinear Sci. Numer. Simul. 10, 273e290. Wagener, T., Montanari, A., 2011. Convergence of approaches toward reducing uncertainty in predictions in ungauged basins. Water Resour. Res. 47. Xu, C., Gertner, G., 2007. Extending a global sensitivity analysis technique to models with correlated parameters. Comput. Stat. Data Anal. 51, 5579e 5590. Yang, J., 2011. Convergence and uncertainty analyses in Monte-Carlo based sensitivity analysis. Environ. Model. Softw. 26, 444e457. Yang, J., Reichert, P., Abbaspour, K.C., Xia, J., Yang, H., 2008. Comparing uncertainty analysis techniques for a SWAT application to the Chaohe Basin in China. J. Hydrol. 358, 1e23. Yatheendradas, S., Wagener, T., Gupta, H., Unkrich, C., Goodrich, D., Schaffner, M., Stewart, A., 2008. Understanding uncertainty in distributed flash flood forecasting for semiarid regions. Water Resour. Res. 44, W05S19. Zacharias, S., Wessolek, G., 2007. Excluding organic matter content from pedotransfer predictors of soil water retention. Soil Sci. Soc. Am. J. 71, 43. Zhan, C., Song, X., Xia, J., Tong, C., 2013. An efficient integrated approach for global sensitivity analysis of hydrological model parameters. Environ. Model. Softw. 41, 39e52.