Testing winter wheat simulation models' predictions against observed UK grain yields

AGRICULTURAL AND FOREST METEOROLOGY

ELSEVIER

Agricultural and Forest Meteorology 89 (1998) 85-99

Testing winter wheat simulation models' predictions against observed UK grain yields S. Landau

a,

R.A.C. Mitchell b V. Barnett c,, j.j. Coils K.L. Moore d, R.W. Payne e

a,

j. Craigon

a

a University of Nottingham, Department of Physiology and Environmental Science, Sutton Bonington, Loughborough, Leicestershire LE12 5RD, UK b IACR-Rothamsted, Biochemistry and Physiology Department, Harpenden, HertfordshireAL5 2JQ, UK c University of Nottingham, Department of Mathematics, Nottingham NG7 2RD, UK d Unilever Research, Colworth Laboratory, Sharnbrook, Bedford, Bedfordshire MK44 IQL UK e IACR-Rothamsted, Statistics Department, Harpenden, Her(fordshireAL5 2JQ, UK Received 13 September 1996; received in revised form 15 August 1997; accepted 15 August 1997

Abstract Wheat models such as CERES-wheat, AFRCWHEAT2 and SIRIUS predict grain yield and have been widely used, in particular to assess possible effects of climate change. Here, observed yields from well-managed and documented UK agricultural experiments were used for a large-scale study of these models' grain yield predictions. None of the models accurately predicted historical grain yields between 1976 and 1993. Substantial disagreement was found between the models' predictions of both yield and yield loss due to water limitation. A regression of observed yields on monthly climatic variables indicated that indirect climatic effects play a considerable role in UK well-managed yields. The study shows that more work is needed before such yield predictions can be used with confidence in decision support or climate change assessment in the UK. © 1998 Elsevier Science B.V. Keywords: Winter wheat; Grain yields; Yield prediction; Crop simulation model

1. Introduction Mechanistic simulation models use daily weather data, latitude, sowing date, nitrogen applications and soil characteristics to predict development and growth of wheat crops, leading to a prediction of final grain

* Corresponding author.

yield. These complex simulation models employ a large set of input variables and many internal parameters. They are often regarded as embodying the current 'state of the art' on crop environmental response and have been used in development and research work (Weir et al., 1984). Recently, crop models have been employed in decision support (Thomton et al., 1991; Tsuji et al., 1994) and to construct production scenarios under a changed climate. In particular, there has been a trend to use the

0168-1923/98/$19.00 © 1998 Elsevier Science B.V. All rights reserved. PII S 0 1 6 8 - 1 9 2 3 ( 9 7 ) 0 0 0 6 9 - 5

86

S. Landau et aL / Agricultural and Forest Meteorology 89 (1998) 85-99

final crop model yield predictions to estimate both current production potentials as an aid in strategic decision making such as land use planning and food supply considerations (van Keulen et al., 1987; Aggarwal, 1988; van Diepen et al., 1990; Aggarwal, 1993), and future production potentials if climate change resulted in a shift in average climatic conditions (Adams et al., 1990; Semenov et al., 1993; Wolf, 1993; Rosenzweig and Parry, 1994; Harrison et al., 1995; Rosenzweig and Tubiello, 1996) or a changed climate variability (Mearns et al., 1992; Aggarwal and Sinha, 1993; Semenov and Porter, 1995; Wolf et al., 1995). Some empirical validation studies have been published (Otter-Nacke et al., 1986; Fei and Ripley, 1988; Jamieson et al., 1991; Mearns et al., 1992; Moulin and Beckie, 1993; Porter et al., 1993; Tour6 et al., 1994; Wolf et al., 1995; Rosenzweig and Tubiello, 1996). The largest was for the US model CERES-wheat (Ritchie and Otter, 1985) where 283 globally-observed yields were compared with model predictions. This study (Otter-Nacke et al., 1986) showed that CERES-wheat was able to explain about 60% of the variation in grain yield. Further CERESwheat validation studies involved a small number of sites, mainly in Canada (Fei and Ripley, 1988; Moulin and Beckie, 1993; Tour6 et al., 1994) or the US (Mearns et al., 1992; Rosenzweig and Tubiello, 1996). Other models which have been used to make predictions for crops grown under UK climatic conditions, for example AFRCWHEAT2 (Weir et al., 1984; Porter, 1984, 1993) or SIRIUS (Jamieson et al., 1997), have been tested for few locations. These studies showed that the models predicted a response of yield to extremely different water supply conditions after calibration for each site (Wolf et al., 1995) but did not show a positive correlation between observed and predicted yields when three New Zealand sites were tested (Jamieson et al., 1991; Porter et al., 1993). In the present study, a large yield data set from agricultural experiments on winter wheat in the UK is used to test the accuracy of grain yield predictions under UK climatic conditions. These experiments contain well-managed wheat crops grown with sufficient nutrient supply. The objective of this work, therefore, was to test whether major crop models (CERES-wheat, modified version 3.0, AFR-

CWHEAT2 and SIRIUS) at their current stage of development can explain the variation in wellmanaged UK winter wheat grain yields.

2. Data sources and m e t h o d s

2.1. Winter wheat trials database and weather data

A database was established consisting of wheat trials undertaken during the period September 1975 to August 1993. The database contains grain yields together with additional information such as treatments, experimental design, grid reference, altitude, sowing date, cultivar and type of trial (e.g., variety trial, nitrogen response trial). Trials from most UK agricultural institutes were included. The data were restricted to autumn-sown, fungicide-treated trials of cultivars of bread-making varieties that had been on the UK recommended list. For each variety the treatment combination that produced the highest average yield was taken to reflect the trial's best managed yield. This procedure implicitly assumed that an optimal treatment had been applied and that the overall management factors such as cultivation, spraying regime and nutrient supply were also optimal. The locations of the 341 field trials from which 815 yield observations were employed in this study are shown in Fig. 1. The crop models require daily climatic input-minimum and maximum temperature, sunshine duration, radiation and rainfall. Daily weather data for the period of interest were available from the Meteorological Office database at IACR-Rothamsted, comprising 212 meteorological stations within the UK. Because meteorological stations do not necessarily coincide with the crop sites, methods to interpolate yearly series of daily weather data for the crop sites were developed (Landau and Barnett, 1996). The selected interpolation method took account of the spatial and temporal variation in the weather data and explained variation in observed weather well for a set of dates at randomly selected sites (94%, 97%, 84% and 74% of the variation in minimum and maximum temperature, sun hours and rainfall were explained respectively). Because radiation records were sparse (only 19 meteorological stations recorded

87

S. Landau et al. / Agricultural and Forest Meteorology 89 (1998) 85-99

method was able to explain 95% of the variation in daily radiation measurements during two years at the available recording stations. oo~

2.2. Running the crop models

0

o

o o oO

o o

Q

~o o

o

.

o

P *'0oo

o

°~

o.

$*

Fig. 1. Locations of agricultural trials considered in the study. radiation throughout the period of interest) radiation was estimated from sunshine duration by a standard linear relationship (Rietfield, 1978). The latter

As is often done in climate change studies (Adams et al., 1990; Semenov et al., 1993; Semenov and Porter, 1995) or in land suitability mapping (van Keulen et al., 1987; Aggarwal, 1988; van Diepen et al., 1990; Aggarwal, 1993), the crop models were run as supplied. The trials were conducted at high fertiliser nitrogen input or contained a high nitrogen treatment, so nitrogen limiting factors were switched off in the models. Trial sites are generally selected to be good wheat-growing locations, thus the model parameters were set for a soil with good water-holding capacity (Rothamsted soil, Salter and Williams, 1969). In particular, soil water-holding parameters were set to values equivalent to those supplied by SIRIUS for a Rothamsted soil (Table 1). All models were run in the mode requiring only the five climatic variables listed above. Non-climatic model input was supplied from the crop data base where available, and estimated otherwise. Unknown sowing dates (20% of yield values) were estimated by the average date in the crop database (7 October). The sowing densities and the row spacings were hardly ever recorded and were estimated as 300 seeds m - ] and 12.5 cm respectively. Yield is insensitive to these variables in well-managed trials within the bounds of recommended practice. Cultivar-specific parameters, such as those for phenological development, were

Table 1 Soil water-holding properties for a Rothamsted and Woburn soil. The Rothamsted soil parameters were taken from SIR/US and the Woburn parameters were derived from Salter and Williams (1969) Soil parameter

Rothamsted soil

Wobum soil

Saturation soil moisture (volumetric fraction) Soil capacity for available water in layers 0-0.25 m and and 0.25-0.5 m depth (volumetric fraction) Soil capacity for availahle water in layers 0.5-0.75 m, 0.75-1 m, 1-1.25 m and 1.25-1.5 m depth (volumetric fraction) Soil capacity for unavailable water in layer 0-0.25 m depth (volumetric fraction) Soil capacity for unavailable water in layers 0.25-0.5 m, 0.5-0.75 m, 0.75-1 m, 1-1.25 m and 1.25-1.50 m depth (volumetric fraction)

0.44 0.16

0.44 0.12

0.16

0.06

0.08

0.06

0.06

0.02

88

S. Landau et al. / Agricultural and Forest Meteorology 89 (1998) 85-99

used when supplied with the models; otherwise settings for cultivar Avalon were used to ensure consistency with AFRCWHEAT2, which had been optimised for Avalon. 19% of the 815 observed yield values were for Avalon.

2.3. Comparison of observed and predicted yields Observed yields were averaged within 1-km squares within each year to match the precision of the interpolated weather variables, resulting in 303 distinct data values. Squares of this size represent the accuracy of the crop trial grid references. As a result, trials located within the same square had the same weather pattern assigned to them. The root mean square error (RMSE) of differences between observed and predicted yields, and correlations, were used to quantify and compare the models' prediction errors. The mean square error (MSE) was also decomposed into components to show the different error sources (Allen and Rektoe, 1981). The prediction accuracy was further analysed with respect to potentially important factors. Accuracy measures were calculated within years, trial type (variety or non-variety trial) and region (North, Southeast and Southwest region of the UK). Year was chosen as a factor reflecting the overall yearly weather pattern, region to reflect locational characteristics (e.g., soil type) and locational differences in weather pattern, and finally, the factor trial type was included to assess the impact of different management regimes on predictive accuracy. For example, in variety trials fungicide-treated and untreated plots are jointly managed. Therefore, nitrogen applications in variety trials may be restricted by the need to avoid lodging in untreated plots. Furthermore, complete disease control is intended in fungicide-treated variety trials whereas the Agricultural Development Advisory Service (ADAS) aims at economic disease control in its trials.

2.4. Sensitivity analysis A sensitivity study was conducted to test whether estimating some of the crop models' input variables could have worsened the models' yield predictions. Two analyses were performed using restricted data

sets to check whether the predictive accuracy was improved. In the first, trials were restricted to the ones for which cultivar-dependent parameters were known. In the second, they were restricted to trials which were relatively insensitive to water supply. A statistical simulation study was also performed to assess the sensitivity of the crop model predictions to the interpolation or approximation errors in the climate variables. The distributional properties of the prediction errors in the climate inputs had been determined previously (Landau and Barnett, 1996). Also, the available sowing dates in the database were used to estimate the distribution of the sowing dates. This knowledge was applied to mimic the joint distribution of the errors in daily minimum and maximum temperatures, sunshine hours, radiation and rainfall, and of the error in sowing date. The 95%-tolerance intervals for the models' yield predictions were generated by repeatedly (200 runs) sampling errors from these distributions for a small set of trials (due to the computational expense of the approach, simulations were performed for five sites only). The widths of the tolerance intervals were used as a measure of the extreme deviation from predicted values based on accurate climate inputs and sowing dates. The sensitivity of the yield predictions to water supply and soil water-holding capacity was assessed by comparing yield predictions between three water availability regimes: regime (i) predicted potential yields by turning off the model's water routine (CERES-wheat and SIRIUS) or by supplying sufficiently high rainfall values (AFRCWHEAT2 had 16 mm added to the interpolated rainfall every day, which resulted in no yield loss); regime (ii) used the interpolated rainfall data to predict yield assuming a soil type with good water-holding capacity (Rothamsted soil); finally, regime (iii) used interpolated rainfall and a soil type with poor water-holding capacity (Woburn soil, Salter and Williams, 1969; for parameter settings see Table 1). The difference in yield predictions between regimes (i) and (ii) indicates the yield loss that the models assign to insufficient water supply under good water-holding soil conditions. The loss between regimes (ii) and (iii) measures the predicted effects of reduced soil water-holding capacity under current rainfall conditions. The sum of the two differences indicates the

S. Landau et al. /Agricultural and Forest Meteorology 89 (1998) 85-99

size of the yield losses due to combined water supply and water-storage effects.

2.5. Regression analysis The simulation models assume optimal management (sufficient nutrient supply, diseases, pests and weeds controlled artd optimal cultivation practices). The observed yields employed in this study reflect currently recommended agricultural practice. Any departure from optimality might have lead to lower and more variable yields due to sub-optimal nutrient supply, disease, pest or weed infestation or sub-optimal cultivation. Also, differences in soil water-holding capacities or differences in properties of breadmaking wheat cultivars may have introduced further variations into the observed yields. To investigate the question of whether such non-climatic variations could prevent the detection of climatic effects, regression modelling of yields from monthly climate variables was performed. A multiple regression model was developed from the original set of temporally and spatially distributed yield observations (815 observations). A simple linear model of the type: k

E/3 x j

j=l

where I1/ denotes the ith yield observation and Xi/ the respective measurement of the jth monthly climatic explanatory variable, was fitted. Monthly averages of daily minimum and maximum temperatures and daily radiation and rainfall measurements (on the log scale) were considered as potential explanatory variables. Climate data from all months of the crop year except September were considered. In general, winter wheat crops are sown in October and harvested in August the following year. It was suspected that consistently low yearly observed yields prior to 1981 were the result of changes in technology during the early 1980s (introduction of new harvester technology, new higher yielding varieties). Therefore, harvest years prior to 1981 were excluded from the analysis (reducing the data set to 785 observations). As is common in multiple regression type studies (e.g., Stooksbury and Michaels, 1994), an automatic

89

variable selection procedure based on forward selection was employed to choose an empirical yieldclimate model. In the automatic forward procedure, monthly climate variables were added step-wise to the minimal model when their variance ratios were statistically significant at the 5%-level. The forward variable selection procedure was also employed to decompose the part of the yield variance explained by the fitted model into components relating to each selected climatic variable. The fitted empirical model indicated a decrease in yields with increased rainfall levels. In contrast, the crop models only allow for a decrease in yields with decreased rainfall levels due to water stress. Therefore, the observed yield data was grouped according to whether or not the empirical model predicted relatively little yield loss due to high rainfall levels. The predictive performance of the crop models was then re-assessed for these two subsets to see whether the models' predictive power could be improved by only considering trials which were relatively little affected by unexpected climate effects. This was done by splitting the yield sample into two groups of 378 relatively-little-affected and 407 relativelystrongly-affected yield observations. The latter group was defined to contain all yield observations for which the negative rainfall effect predicted by the empirical model exceeded the average effect. Since the crop models' water stress routines may be inadequate in the presence of negative rainfall effects, the accuracy of crop model predictions allowing for sub-optimal water supply (supplied rainfall) as well as of potential yields (water routine turned off) was assessed. The assumption that drought has little effect on UK winter wheat yields is, in any case, justified by experimentation (Cannell et at., 1984).

2.6. Other variables predicted by the simulation models In order to investigate the behaviour of the crop models, further predictions of intermediate growth variables for the field trials were obtained. Model predictions of the developmental variables 'date of anthesis', 'duration of grain fill' and 'date of maturity' were generated together with predictions of variables related to crop growth rate ('maximum leaf

90

S. Landau et al. /Agricultural and Forest Meteorology 89 (1998) 85-99

area index', 'water-use efficiency', 'radiation-use efficiency', 'final biomass') and yield components ('number of grains'). Water-use efficiency was determined as the ratio of final biomass to cumulative transpiration. Radiation-use efficiency was defined as the ratio of final biomass to cumulative intercepted radiation. Histograms were employed to contrast the models' prediction ranges with each other for each of these variables. Finally, inter-correlations between model predictions were used to measure the agreement between models for each considered variable.

o)

16 14 .

12

5;,÷ .

.

,

•

8

4

~31 ,- .."

S o*

i

4

I

I

I

6

8

10

3. R e s u l t s

Fig. 2 a - c contrast the models' yield predictions with the 303 aggregated observed yields. All yields refer to grain at 85% dry matter. The yields cover the crop years 1976-1993, with a majority of trials (85%) from 1982 onwards. None of the models predicted the observed yields satisfactorily. Their overall prediction errors measured by the RMSE (Table 2a) lie between 2.2 and 3 t / h a , a large inaccuracy given the range of 6 to 12 t/ha. Most of the prediction error of AFRCWHEAT2 was caused by failure to follow the pattern in observed yields, whereas that of CERES-wheat was due mainly to an overall bias (Table 2a). CERES-wheat generally predicted low yields (overall bias = - 2.6 t / h a ) whereas AFRCWHEAT2 (bias = 1 t / h a ) and SIRIUS predicted high values (bias = 2.1 t/ha). None of the correlations between observed and predicted yields was significantly different from zero at the 5% level (Table 2b). Low inter-correlations between model predictions (0.23 for CERES-wheat and AFRCWHEAT2, 0.35 for CERES-wheat and SIRIUS, 0.24 for AFRCWHEAT2 and SIRIUS) showed that the models also disagreed among themselves. No consistent differences were found in prediction accuracies in the North, Southeast and Southwest UK regions. However, there was a consistent difference in RMSE between variety trials and nonvariety trials. Table 3 shows that AFRCWHEAT2 and SIRIUS predicted variety trials with 0.29 t / h a and 0.4 t / h a lower RMSE than other trials (but only 0.09 t / h a and 0.13 t / h a lower than the weighted RMSE over all trials). The reduction of the predic-

I

12

14

16

b) .,~

..'"""

14

loL

•

..

g:, 4

6

8

10

12

14

16

c)

16

7" ,,"

14 ,4

12 *÷ , ,

10 8 6

,,

+

+ % ÷ ,

,j"

•

÷

÷

4 4

I

I

I

I

I

I

6

8

10

12

14

16

Predicfed yield (f/ha) Fig. 2. Observed grain yields plotted against yields predicted by (a) CERES-wheat; (b) AFRCWHEAT2 and; (c) SIRIUS. The 1:1 line is shown, representing perfect agreement.

91

S. Landau et al. / Agricultural and Forest Meteorology 89 (1998) 85-99

Table 2 Accuracy measures for crop models: (a) RMSE of grain yield predictions and decomposition of MSE according to error source; (b) correlations between ob.,;erved and predicted grain yields; (c) RMSE of grain yield predictions for which the cultivar dependent model parameters were known; (d) RMSE of grain yield predictions of the 10 trials which were relatively insensitive to water supply and soil water-holding capacity (see text); and (e) mean predicted yield loss due to the use of interpolated rainfall or soil water-storage capacity (a) RMSE (t/ha) Bias (%) Systematic and random error (%) (b) Correlation (c) RMSE (t/ha) (d) RMSE (t/ha) (e) Yield loss for interpolated rainfall + high water-holding capacity soil relative to no water limitation (t/ha) Yield loss for interpolated rainfall + low water-holding capacity soil relative to high water-holding capacity soil (t/ha)

tion errors was caused by decreases in both components of the RMSE, error variation and positive bias. CERES-wheat also showed a decrease in error variation for variety trials but, because the negative bias increased, there was no improvement in RMSE. All models showed differences in R M S E between years and did not follow the observed yield pattern over the years (Fig. 3). CERES-wheat showed the least variation in bias, reflecting a general under-prediction. Predictive performance was not generally improved by considering only cultivars for which each model was specifically developed (at least 147 Avalon yield values). However, the R M S E for CERES-wheat was reduced by 0.28 t / h a relative to the accuracy of the unrestricted data set (Table 2c).

CERES- w h e a t

AFRCWHEAT2

SIRIUS

2.96 77.4 22.6

2.25 20.5 79.6

2.74 61.1 38.9

0.02

0.00

0.04

2.68

2.33

2.77

3.24

3.80

2.33

0.29

1.69

0.85

2.10

0.95

3.82

Restricting attention to the few trials (10 data points spread over the years 1985, 1987, 1991 and 1993) which were relatively insensitive to water supply and soil water-holding capacity (where all models predicted less than 1 t / h a yield loss when comparing regimes (i) and (iii)) gave higher R M S E values for CERES-wheat and A F R C W H E A T 2 and a lower value for SIRIUS (Table 2d). Table 4 presents the results of the statistical simulation study. Widths of 95%-tolerance intervals averaged 24% (CERES-wheat), 27% ( A F R C W H E A T 2 ) and 22% (SIRIUS) of predicted yields for the five sites investigated. Therefore, it was possible for errors in the predicted climate series or in the estimated sowing date to affect the models' yield prediction considerably. However, considering that devia-

Table 3 RMSE of model grain yield predictions and decomposition of MSE into bias and variance of differences between observed and predicted yields within the groups variety (V) and non-variety trials (nV) CERES-wheat V RMSE (t/ha) Bias (t/ha) Variance (t2/ha 2)

2.97 - 2.64 1.87

AFRCWHEAT2 nV 2.92 - 2.52 2.22

SIRIUS

V

nV

V

nV

2.16 0.96 3.77

2.45 1.21 4.55

2.61 2.03 2.69

3.01 2.39 3.35

92

S. Landau et al. / Agricultural and Forest Meteorology 89 (1998) 85-99

12 11

.-;.-'-...'..-.:...

.:

"''.

10

-. ......

-

..-

o" ..

"".

"

......................

A -

t--

9 v

~___

8

>.-

7

I

I

I

83

84

I

I

I

I

E

,

.

85

86

87

88

I

I

I

I

I

J

,

i

i

89

90

91

92

93

3 r'-

2 v

-

1

o CO

0 i

82

/

,

.

/

Year Fig. 3. Mean yearly observed (O) and predicted grain yields and standard errors (s.e.) of differences between predicted and observed yields within years for CERES-wheat (C), AFRCWHEAT2 (A) and SIRIUS (S). The data set was restricted to variety trials carried out during the period 1982-1993.

tions of the size of the tolerance intervals presented extreme events, and that smaller yield deviations were more likely (if the generated yield predictions were normally distributed, the lengths of the inter-

vals obtained would be equivalent to a RMSE of differences between input error affected and 'true' model predictions of c. 0.65 t / h a for AFRCWHEAT2, c. 0.35 t / h a for CERES-wheat and c.

Table 4 Widths of 95%-tolerance intervals for grain yield predictions for selected trials in absolute terms (t/ha) and relative to predicted yield (%) Trial

Old Leake 1983/1984 Woburn 1984/1985 Bridgets 1985/1986 Crichton 1988/1989 Cambridge 1992/1993

CERES-wheat

AFRCWHEAT2

SIRIUS

Absolute

Relative

Absolute

Relative

Absolute

Relative

1.9

28.2

2.5

28.6

4.2

46

0.8

11,7

2.2

18.2

1.4

11.6

0.8

13.0

1.9

18.2

2.1

18.8

2.6

49.8

4.0

38.3

2.9

26.0

0.8

14.8

3.5

33.8

1.2

9.4

S. Landau et al. / Agricultural and Forest Meteorology 89 (1998) 85-99

93

Table 5 Regression model fitted to observed yields after selection of monthly climatic explanatory variables by forward selection (see text)

/30 Xt X2 X3 X4 X5 X6 X7 Xs X9 Xlo XH

XI2 Xi3

X14 X15 Xt6 Xl7

Climate variable

Daily values averaged in

Coefficient estimate

s.c.

Importance (%)

Constant (t/ha) Rainfall in m m (log transformed) Rainfall in m m (log transformed) Rainfall in nun (log transformed) Rainfall in m m (log transformed) Rainfall in m m (log transformed) Radiation in M J / m 2 Radiation in M J / m 2 Radiation in M J / m 2 Min. temperature in °C Min. temperature in °C Min. temperature in °C Min. temperature in °C Min. temperature in °C Max. temperature in °C Max. temperature in °C Max. temperature in °C Max. temperature in °C

October March April May July March June July October January May June July November January February July

8.55 -0.23 - 0.56 0.25 -0.53 - 0.49 -0.14 0.15 -0.15 0.20 - 0.21 -0.22 0.22 -0.39 0.39 0.43 -0.14 -0.15

0.04 0.12 0.11 0.09 0.12 0.11 0.05 0.03 0.04 0.05 0.08 0.08 0.07 0.09 0.07 0.08 0.03 0.05

1.9 5.8 2.1 3.8 11.9 1.7 9.8 4.6 3.5 3.1 1.3 1.0 10.4 6.7 28.4 2.3 1.7

All explanatory variables were centered by their respective average value over all trials. The percentage of variance accounted for by the model was 32%. The importance of each wtriable was calculated by decomposing the part of the yield variation explained by the model according to climate effects on the basis of forward variable selection.

0.6 t / h a for SIRIUS) the sizes of the tolerance intervals suggested that errors in the weather or sowing date inputs could be responsible for only a small part of the predictive inaccuracy (see Table 2a). Finally, the unexplained variation in average yields between years (Fig. 3) cannot be attributed to

uncertainty in the climate input since there was no evidence to suggest that input errors would affect yield predictions systematically from year to year. Table 2e shows how the models attributed yield loss to water supply and water holding capacity. Whereas CERES-wheat and SIRIUS predicted aver-

Table 6 RMSE of model grain yield predictions, correlations between observed and predicted yields and decomposition of MSE into bias and variance of differences between observed and predicted yields for observed yields within the groups: relatively high (D) and relatively little (nD) yield loss due to high rainfall levels (see text) Model prediction

CERES-wheat nD

Water limiting

Water optimal

RMSE (t/ha) Bias (t/ha) Variance (t2/ha 2) Correlation I~,ISE (t/ha) Bias (t/ha) Variance (t 2 / h a 2) Correlation

3.43 - 3.16 1.78 0.1 3.05 - 2.68 2.17 0.1

AFRCWHEAT2 D 2.51 - 2.15 1.66 0.00 2.44 - 2.02 1.88 - 0.01

nD 1.84 0.17 3.37 0.19 2.7 1.98 3.35 0.06

D 2.6 1.79 3.55 - 0.05 3.64 3.25 2.7 0.05

SIRIUS nD 2.4 1.6 3.22 - 0.06 3.45 2.98 3.03 0.17

D 3.05 2.68 2.14 0.13 3.43 3.0 2.79 0.03

Model yield predictions were considered for the case when water was allowed to be a limiting factor (calculation with supplied rainfall) and when water supply was assumed to be optimal. The yield data set was restricted to the period 1981-1993.

94

S. Landau et al. / Agricultural and Forest Meteorology 89 (1998) 85-99

age yield losses of 5% and 7% for the effect of using interpolated rainfall with Rothamsted soil, AFRCWHEAT2 suggested a yield loss of 15%. In contrast, when comparing the effects of soil water-holding capacity AFRCWHEAT2 predicted an average yield loss of 8% whereas the other models suggested yield losses of 34% (CERES-wheat) and 33% (SIRIUS). Also, SIRIUS yield losses totalled 4.7 t / h a whereas the other two models suggested totals of 2-3 t/ha. The models did not agree on the sensitivity of yield to water-limitation, nor about

which trials were more prone to yield loss. The highest correlation between predicted yield losses for soil water-holding capacity effects was for CERESwheat and SIRIUS (0.56), followed by CERES-wheat and AFRCWHEAT (0.13), and SIRIUS and AFRCWHEAT2 (0.02). Table 5 presents the results from the regression analysis. The simple empirical yield-climate model was able to explain 32% of the variation in observed yields. Evaluation of the importance of each fitted climate variable showed that average daily maximum

150-

1 2OO

125~ e-

100

~

150 1oo

I,

Ll_

25°° 144

5Oo 156

168

180

a,. -

25 30 35 40 45 50 55 60 65

Date of anthesis (day of year)

Duration of grain fill (days) /

I00-

300

i

8O

200]

GO b_

40

,o

0

1 204 ' 216 ' 2i8 ' 210

Date of m0turity (day of year)

tO0

!

0

''1-'

M0×imum ~1

Fig. 4. Histograms of predicted dates of anthesis, predicted durations of grain fill, predicted dates of maturity and predicted maximum leaf area indexes (LAD. CERES-wheat predictions are presented by light-shaded bars, AFRCWHEAT2 predictions by solid-shaded bars and SIRIUS predictions by medium shaded bars. In SIRIUS, LAI is pre-set to a maximum value of 8.5 which was almost always achieved in practice.

95

S. landau et al. /Agricultural and Forest Meteorology 89 (1998) 85-99

temperatures in January, average rainfall levels in July, average daily minimum temperatures in July and average radiation levels in June were able to account for 61% of the explained variation. Maximum temperature in .lanuary was estimated to have a positive effect on grain yield. The empirical model also included a negative effect of minimum temperature in January. But because the coefficient estimate of the latter ( - 0 . 2 1 t h a - 1 / ° C ) was small in absolute value relative to the coefficient estimate of the former (0.43 t h a - t / ° C ) the model predicted an overall positive effect of temperatures in January.

Increased rainfall levels in July were estimated to decrease yields. The empirical model also assigned negative rainfall effects to March, May and October rainfall. Furthermore, decreases in minimum temperature (and maximum temperature) in July and increases in radiation levels in June were predicted to increase yields (Table 5). The observed yields were grouped according to whether or not the empirical model predicted a large reduction in yields due to high rainfall levels in October, March, May and July. Models were run in two modes, where water could be limiting ('water

200-

/ / / /

150 150100-

2

/ /

L.t..

5o]

50

5i2

6

' 6.8

' 7.6

RUE

/

50"

/ / / /

q 100

/

1.12 '

1.2"8 ' 1.44 '

(g biomass/MJ total radn.)

150

/ / / / / /

~' I00"

H"' 1 I

0 ,',, ~, 0.96 0.8

8.4

WUE (g biomass/kg H20) 150-

/ / /

l,

50-

z, 150

o-, ~. . . . . . , 200

250

300

Number of grains (million/ha)

I0

15

20

I 25

!

30

Final biomass (t/ha)

Fig. 5. Histograms of predicted water-use efficiencies (WUE), predicted radiation-use efficiencies (RUE), predicted number of grains and predicted final biomass. CERES-wheatpredictions are presented by light-shaded bars, AFRCWHEAT2predictions by solid-shaded bars and SIR/US predictions by medium-shaded bars. SIRIUS assigns a maximum value of 1.2 g biomass per MJ total radiation to RUE and the model does not predict the number of grains.

96

S. Landau et al. /Agricultural and Forest Meteorology 89 (1998) 85-99

limiting') and where water limitation was switched off, so that rainfall would not have a positive effect on predicted yield ('water optimal'). Comparing the low-yield-loss group ('nD') to the high-yield-loss group ('D'), bias errors were worsened for CERESwheat and improved for AFRCWHEAT and SIRIUS, and these differences were reflected in the RMSE values (Table 6). There appears to be a slight improvement in correlation using the nD set for CERES-wheat, for AFRCWHEAT2 for water limiting case and SIRIUS for the water optimal case, but the correlation coefficients were not statistically significant at the 1%-level. The histograms displayed in Figs. 4 and 5 showed that not only the ranges of the yield predictions differed between the models but also the ranges of some further considered intermediate growth variables. SIRIUS generally predicted later dates for maturity (average date 18 August) than the other two models (average dates were 31 July for CERES-wheat and 2 August for AFRCWHEAT2). CERES-wheat predictions of maximum leaf area index (average index 3.9) were generally smaller than predictions by AFRCWHEAT2 (average value 7.1) which included the pre-set value for the index in SIRIUS. CERESwheat also predicted overall smaller values for the other two variables, water-use efficiency and radiation-use efficiency, which affected the predicted growth rate and predictions of the yield components were smaller. In particular, the number of grains predicted by CERES-wheat (in average 119 million/ha) differed from the AFRCWHEAT2 prediction (218 million/ha). Inter-correlations between model predictions were generated to try and identify the reasons for the disagreement between models in yield predictions. The largest inter-correlation coefficients were detected for the developmental variables. For anthesis dates, durations of grain-fill and maturity dates correlations between CERES-wheat and AFRCWHEAT2 were in the range 0.86 to 0.92 followed by correlations between CERES-wheat and SIRIUS (range 0.7 to 0.71) and between AFRCWHEAT2 and SIRIUS (range 0.62 to 0.73). Lower correlation coefficients (range 0.19 to 0.71) were found between model predictions of radiation-use efficiencies, grain numbers and final biomass and the least agreement between model predictions (correlation coefficients below 0.32) was detected for maxi-

mum leaf area index and water-use efficiency. In particular, negative correlation coefficients between CERES-wheat leaf area index predictions and predictions by AFRCWHEAT2 (correlation - 0 . 3 5 ) or SIRIUS (correlation - 0 . 1 6 ) and between AFRCWHEAT2 water-use efficiency predictions and predictions by CERES-wheat (correlation - 0.12) or SIRIUS (correlation - 0 . 2 9 ) were detected.

4. Discussion and conclusions

In summary, for our data set of experimental yields, none of the three models considered-CERES-wheat, AFRCWHEAT2 and SIRIUS--was able to predict grain yield (Table 2a,b, Fig. 2). The average annual yields were also not predicted (Fig. 3). Furthermore, the models did not agree with each other on grain yield predictions, other growth variable predictions, or on predictions of yield loss due to water limitations (Table 2e). There are three possible explanations for the poor performance of the models: (1) faulty input data, (2) factors other than weather influencing yield (sub-optimal management) or (3) model faults. These prospects are examined. (1) There is no indication that predictive power has been masked by errors in weather variables or sowing date (Table 4) or by considering cultivars for which the models were not specifically developed (Table 2c). The results from the regression analysis also suggest that the climate data is of sufficient accuracy to model grain yields. The empirical model was able to explain 32% of the variance in observed yields and the fitted climate effects, which contributed to the majority of the variance explained, could be explained from a physiological background (Table 5). The four major climate effects detected were a positive effect of maximum daily temperature in January, a negative effect of rainfall in July, a negative effect of minimum daily temperature in July and a positive effect of radiation in June. These may be due to winter kill of plants, damage due to disease infection, shortening of grain filling and radiation intercepted during anthesis, respectively. The findings were consistent with those of Chmielewski and Potts (1995) for grain yields from a long-term exper-

S. Landau et al. / Agricultural and Forest Meteorology 89 (1998) 85-99

iment at Rothamsted. The effect of using estimated soil water-holding capacity is less clear. It was not possible to supply accurate soil water-holding characteristics and the models have shown a surprising sensitivity to soil water conditions. However, there is little agreement between models about the trials' sensitivity to water and soil conditions (Table 2e). (2) It was assumed that agricultural stations in the UK would have tight controls of diseases and pests, would ensure sufficient nutrient supply and would apply optimal cultivation procedures. The negative rainfall effects del~cted by the regression model (during October, March, May and July) suggest that diseases and pests encouraged by certain weather patterns, water logging (Cannell et al., 1984) and/or nitrogen leaching due to excess rainfall play a role even in these well-managed trials. The assumption that such effects were fully controlled may have caused some of the inaccuracies, for example the within-year biases (Fig. 3). It may also explain the differences between variety and non-variety trials: e.g., variety trials may have better disease control, thus reducing the non-weather variation in observed yields (Table 3). The yield data were then restricted to observations for which the regression model predicted relatively little yield loss due to increased rainfall levels, in ant attempt to exclude observations that were affected by sub-optimal management (e.g., disease). While there was a slight improvement in correlation coefficie, nts in most cases, the values of these were still not significantly different from zero at the l%-level (Table 6). Sub-optimal management will reduce yields and introduce variation into the yield data which cannot be accounted for by the models, and will also change the response of yield to weather. Attempts to improve the models' yield predictions must aim to account for these variations, since complete elimination of diseases is never economically optimal. (3) Inaccurate simulation model predictions may have been caused by mis-specified parameters or relationships within any of the many modelling steps employed to predict grain yields. Also, the models' highly complex structure may have caused grain yield predictions to respond to climatic conditions which have little effect on well-managed wheat crops in the UK. However, it is possible that simulation models operate better within more marginal environ-

97

ments, where yield variation is more dominated by environment than for wheat in the UK. The situations for which simulation models' ability to predict grain yields have been demonstrated have mostly been where water limitation (Fei and Ripley, 1988; Otter-Nacke et al., 1986; Mearns et al., 1992; Tour6 et al., 1994; Rosenzweig and Tubiello, 1996) or growing season (Aggarwal et al., 1994) dominate yield variation. The disagreement between models' yield predictions shows that there are substantially different underlying hypotheses in the models, leading to different dependencies of predicted yield on weather. The examination of further growth variable predictions has shown that these differences in yield predictions are associated with differences in both maximum leaf area index and radiation-use efficiency, rather than differences in phenology. The low absolute values of yields predicted by CERESwheat are due to low maximum leaf area index (unrealistically low for the UK), and, to a lesser extent, low radiation-use efficiency (Figs. 4 and 5). The study highlights the dangers of basing policy (e.g., on the impact of climate change) or decision making on wheat yield predictions generated by current models. There remains a need to establish and quantify the dominant relationships between wheat yield variation and environment.

Acknowledgements This work was funded by the Biotechnological and Biological Sciences Research Council. Funding for R.A.C. Mitchell from the UK Ministry of Agriculture, Fisheries and Food is acknowledged. We thank the Agricultural Development Advisory Service, the National Institute of Agricultural Botany, Biomathematics and Statistics Scotland, the Scottish Agricultural College, the Department of Agriculture for Northern Ireland, the Morley Research Centre and the Harper Adams College for providing data. We are also grateful to J.T. Ritchie, J.R. Porter and M.A. Semenov for supplying the crop models.

References Adams, R.M., Rosenzweig, C., Peart, R.M., Ritchie, J.T., McCarl, B.A., Glyer, J.D., Curry, R.B., Jones, J.W., Boote, K.J., Allen,

98

S. Landau et al. / Agricultural and Forest Meteorology 89 (1998) 85-99

L.H. Jr., 1990. Global climate change and US agriculture. Nature 345, 219-224. Aggarwal, P.K., 1988. Wheat production in Southeast A s i a - potential and constraints. Outlook Agric. 17, 49-53. Aggarwal, P.K., 1993. Agro-ecological zoning using crop simulation models: characterisation of wheat environments in India. In: Penning de Vries, F.W.T., Teng, P.S. (Eds.), Systems Approach to Agricultural Development. Kluwer, Dordrecht, The Netherlands, 97-109. Aggarwal, P.K., Kalra, N., Singh, A.K., Sinha, S.K., 1994. Analyzing the limitations set by climatic factors, genotype, water and nitrogen availability on productivity of wheat: I. The model description, parametrization and validation. Field Crops Res. 38, 73-91. Aggarwal, P.K., Sinha, S.K., 1993. Effect of probable increase in carbon dioxide and temperature on productivity of wheat in India. J. Agric. Meteorol. 48, 811-814. Allen, O.B., Rektoe, B.L., 1981. Accuracy analysis with special reference to the prediction of grassland yield. Biom. J. 23, 371-388. Cannell, R.Q., Belford, R.K., Gales, K., Thomson, R.J., Webster, C.P., 1984. Effects of water-logging and drought on winter wheat and winter barley grown on a clay soil and a sandy loam soil. Plant and Soil 80, 53-66. Chmielewski, F.-M., Potts, J.M., 1995. The relationship between crop yields from an experiment in southern England and long-term climate variations. Agric. For. Meteorol. 73, 43-66. Fei, Q.P., Ripley, E.A., 1988. Simulation of spring wheat yields in the Saaskatoon crop district 1960 to 1984 using the CERESwheat model. SRC Publication no. E-906-46-B-85, Saskatschewan Research Council, Saskatoon, SK. Harrison, P.A., Butterfield, R.E., Gawith, M.J., 1995. Modelling the effects of climate change on crops on the regional scale: effects on winter wheat, sunflower, onion and grassland in Europe. In: Harrison, P.A., Butterfield, R.E., Downing, T.E. (Eds.), Climate Change and Agriculture in Europe. Research Report Environmental Change Unit 9, Oxford, UK, 330-379. Jamieson, P.D., Porter, J.R., Wilson, D.R., 1991. A test of the computer simulation model ARCWHEAT1 on wheat crops grown in New Zealand. Fields Crops Res. 27, 337-350. Jamieson, P.D., Semenov, M.A., Brooking, I.R., Francis, G.S., 1997. SIRIUS: a mechanistic model of wheat response to environmental variation. Field Crops Res., in press. Landau, S., Barnett, V., 1996. A comparison of methods for climate data interpolation, in the context of yield predictions from winter wheat simulation models. In: White, E.M., Benjamin, L.R., Brain, P., Hamer, P.J.C., Mugglestone, M.A., Russell, G., Topp, C.F. (Eds.), Aspects of Applied Biology, Modelling in Applied Biology: Spatial Aspects Association of Applied Biologists. Wellesbourne, UK, 46, 13-22. Mearns, L.O., Rosenzweig, C., Goldberg, R., 1992. The effect of changes in interannual climatic variability on CERES-wheat yields: sensitivity and 2 × CO 2 studies. Agric. For. Meteorol. 62, 159-189. Moulin, A.P., Beckie, H.J., 1993. Evaluation of the CERES and EPIC models for predicting spring wheat grain yield. Can. J. Plant Sci. 73, 713-719.

Otter-Nacke, S., Godwin, D.C., Ritchie, J.T., 1986. Testing and validating the CERES-wheat model in diverse environments. AgRISTARS Technical Report, YM-15-00407, JSC-20244, Johnson Space Center Houston, TX, USA. Porter, J.R., 1984. A model of canopy development in winter wheat. J. Agric. Sci., Camb. 102, 383-392. Porter, J.R., 1993. AFRCWHEAT2: a model of the growth and development of wheat incorporating responses to water and nitrogen. Eur. J. Agron. 2, 69-82. Porter, J.R., Jamieson, P.D., Wilson, D.R., 1993. Comparison of the wheat simulation models AFRCWHEAT2, CERES-wheat and SWHEAT for non-limiting conditions of crop growth. Field Crops Res. 33, 131-157. Rietfield, M.R., 1978. A new method for estimating the regression coefficients in the formula relating solar radiation to sunshine. Agric. Met. 19, 243-252. Ritchie, J.T., Otter, S., 1985. Description and performance of CERES-wheat: a user-orientated wheat yield model. Report US Department of Agriculture, ARS, US 38, 159-175. Rosenzweig, C., Parry, M.L., 1994. Potential impact of climate change on world food supply. Nature 367, 133-137. Rosenzweig, C., Tubiello, F.N., 1996. Effects of changes in minimum and maximum temperature on wheat yields in the central U S - - a simulation study. Agric. For. Meteorol. 80, 215-230. Salter, P.J., Williams, J.B., 1969. The moisture characteristics of some Rothamsted, Woburn and Saxmundham soils. J. Agric. Sci., Camb. 73, 155-158. Semenov, M.A., Porter, J.A., 1995. Climatic variability and the modelling of crop yields. Agric. For. Meteorol. 73, 265-283. Semenov, M.A., Porter, J.A., Del~colle, R., 1993. Climate change and the growth and development of wheat in the UK and France. Eur. J. Agron. 2, 293-304. Stooksbury, D.E., Michaels, P.J., 1994. Climate change and large-area corn yield in the Southeastern United States. Agron. J. 86, 564-569. Thornton, P.K., Dent, J.B., Basci, Z., 1991. A framework for crop growth simulation model applications. Agric. Sys. 37, 327340. Tour6, A., Major, D.J., Lindwall, C.W., 1994. Comparison of five wheat simulation models in southern Alberta. Can. J. Plant Sci. 75, 61-68. Tsuji, G.Y., Uehara, G., Balas, S (Eds.), 1994. DSSAT version 3, IBSNAT, University of Hawaii, Honolulu, HI, USA. van Diepen, C.A., de Koning, G.H.J., Reinds, G.J., Bulens, J.D., van Lanen, H.A.J., 1990. Regional analysis of physical potential of crop production in the European Community. In: Goudriaan, J., van Keulen, H., van Laar, H.H. (Eds.), The Greenhouse Effect and Primary Productivity in European Agro-ecosystems. Pudoc, Wageningen, The Netherlands, pp. 74-79. van Keulen, H., Berkhout, J.A.A, van Diepen, C.A., van Heemst, H.D.J., Janssen, B.H., Rappoldt, C., Wolf, J., 1987. Quantitative land evaluation for agro-ecological characterization. In: Bunting, A.H. (Ed.), Agricultural Environments. Characterization, Classification and Mapping, CAB International, 185-197. Weir, A.H., Bragg, P. L, Porter, J.R., Rayner, J.H., 1984. A

S. Landau et al. /Agricultural and Forest Meteorology 89 (1998) 85-99 winter wheat crop sirtmlation model without water or nutrient limitations. J. Agric. Sci., Camb. 102, 371-382. Wolf, J., 1993. Effects tff climate change on wheat production potential in the European Community. Eur. J. Agron. 2 (4), 281-292. Wolf, J., Semenov, M.A., Eckerstein, H., Evans, L.G., Iglesias,

99

A., Porter, J.R., 1995. Effects on winter wheat: a comparison of five models. In: Harrison, P.A., Butterfield, R.E., Downing, T.E. (Eds.), Climate Change and Agriculture in Europe, Research Report Environmental Change Unit 9, Oxford, UK, pp. 231-280.