- Email: [email protected]
Statistical experiments with simulation models—A dairy genetics example
,.-
,
AgriculturalSystems45 (1994) 203-216
j
© 1994 Elsevier Science Limited Printed in Great Britain. All rights reserved 0308-521X/94/$7.00 ELSEVIER
Statistical Experiments with Simulation Models A Dairy Genetics Example D. G. Mayer, ~ M. L. Tierney b & P. N. T h u rb o n b a Biometry and b Dairy and Animal Breeding Branches, Queensland Department of Primary Industries, GPO Box 46, Brisbane 4001, Australia (Received 9 December 1991; accepted 9 June 1993)
ABSTRA CT Simulation models have been developed for many agricultural applications. Model experimentation, using analysis of variance of a large-scale factorial, is a valuable exploratory exercise to investigate the interacting effects of managerial options and external factors. To achieve this analysis, consideration must first be given to a number of potential problems, including definition of treatments, experimental design, underlying statistical assumptions, replication, appropriateness of the error structure, and interpretation of results. Such analyses are useful in identifying the key main effects and interactions in the system, which can then be recommended to industry or investigated further. The practicalities and advantages of this approach for multi-optional models are illustrated using a stochastic dairy genetics model.
INTRODUCTION
In most fields of agricultural research, increasing numbers of decision-aid packages or simulation models are becoming available. These are usually developed for a defined agricultural system, or, if generic in nature, specifically tuned to a target system. Following validation against realworld data (Dent & Blackie, 1979; Kleijnen, 1987), models can be a valuable aid to understanding or optimising the target system. If a key economic or production variable can be nominated, optimisation is recommended, with a range of methods available (Mayer et al., 1991). 203
204
D. G. Mayer, M. L. Tierney, P. N. Thurbon
In many modelling situations, 'what-if' investigations and experimentation (Naylor et al., 1967) are more applicable or important than optimisation. Typically, the managers of the agricultural system wish to investigate the likely effects of the various options available, often in conjunction with expected external (uncontrollable) effects such as climate or market prices. This simulation approach was used by Dommerholt and Wilmink (1986) with genetic parameters in the optimisation of dairy farm returns in the Netherlands. In Australian dairying, Beard (1987) evaluated index selection to maximise herd genetic improvement, and Maher (1989) outlines 'Hybreed', a simulation model for extension and educational purposes which evaluates the potential gain from including heifers in the breeding plan. Notter and Johnson (1987, 1988) also used genetic models to investigate the potential genetic improvement in beef herds. In most situations, modelling studies such as these are conducted on a 'one-at-a-time' basis, whereby the model is configured to some average or optimal scenario and individual management options are altered and investigated in turn. This approach considers only main effects. However, in many complex agriculture systems, it can be the higher-order interactions between both management options and external influences that are of prime importance. These interactions can be used to understand the system and to identify recommended practices. For example, the most profitable level of one management factor (e.g. applied fertiliser) usually depends on a whole range of other factors (fertiliser and commodity prices, weather, soil type and fertility, applied levels of other fertilisers and irrigation, crop and variety, etc). Even if high-order interactions do not eventuate, it is necessary to confirm that the main effects of the treatments are consistent across all the other options tested. These hypotheses are best investigated with large-scale factorial experimentation, as recommended by Naylor et al. (1967), Shannon (1975) and Poole and Szymankiewicz (1977). Under this approach, each independent run of the simulation model is considered an experimental unit for the analysis of variance (ANOVA).
STATISTICAL CONSIDERATIONS A number of studies of model validation techniques have recommended the use of formal statistical tests, providing the underlying assumptions are not violated (Harrison, 1990; Loague & Green, 1991; Whitmore, 1991). The key statistical assumptions of the ANOVA are that observations are independent, variances are homogeneous, and responses are normally
Statistical experiments with simulation models
205
distributed about their expected values (Snedecor & Cochran, 1980). Considering these in turn, the assumption of independence often causes concern in modelling experiments, for two reasons. Firstly, many studies generate time-series, which are auto-correlated. This problem can be overcome by either reducing these series to a single value or index, or by identifying appropriately long time periods which ensure independence and condensing data within each period (Wright, 1971). The second reason the problem of dependence may occur is if modellers use the same random number sequence in successive runs. Whilst this practice has variance reduction advantages, it introduces correlations into output results. If different (independent) random number sequences are used, this statistical assumption is upheld (Phillips, 1971). The remaining assumptions, concerning variance (second moment of the distribution) and normality (higher moments), are related. The ANOVA technique has been shown to be quite robust to departures from these assumptions (Boneau, 1960; Lunney, 1970; Nishisato, 1980). As with any field experiment, mild departures from these assumptions may be expected, and as suitable statistical transformations can be applied to correct for any moderate or serious departures (Mead & Curnow, 1983), these assumptions should cause few concerns. As with any scientific trial, an appropriate statistical design is important. With the availability of general model packages (e.g. GLIM, GENSTAT), both factors (of fixed discrete levels) and variates (of a continuous nature, allowing response surface analysis (Cochran & Cox, 1957)) can be utilised. To fit the latter, the shape of the surface must be assumed a priori and specific in the analytical model, with quadratics or higher-degree polynomials often used. These however, may be inappropriate when considering agricultural systems and models of these, as their smooth nature cannot, for example, mimic the 'crashes' observed when a system is over-utilised (e.g. over-stocked). A safer approach, and one which statistically fits a similar number of constants, is to use fixed levels of each continuous variate, and fit curves to the estimated responses at these levels if appropriate. Whilst modelling experiments can be easier to set up and cheaper to run than traditional field trials, resources are still not infinite. For example, a complete factorial of ten factors at five levels would require approximately ten million model runs, which may well be excessive in terms of run-time and data storage. Use of fractional replication designs, which can reduce the size by a factor of two or four at the expense of having effects confounded, is one possible solution (Kleijnen, 1987). More sophisticated analytical packages (such as GENSTAT) are required to analyse these designs. Usually complete factorials are preferred
206
D. G. Mayer, M. L. Tierney, P. N. Thurbon
(Naylor et al., 1967; Shannon, 1975), as interpretation of results is unambiguous. In practical terms, the capacity of available computer or analytical packages often determines the upper size limit of the simulation experiment. Factors and their levels are then chosen to comply with this limit, with the important agronomic factors having more levels (usually four to six) and others having fewer levels (often only two) or being omitted altogether. The statistical soundness of the estimated error structure depends on the type of simulation model used to generate the results, with three possibilities. Firstly, the model may be stochastic with realistic random variation if the major processes contributing variability have been built into the model. Here, each model run becomes an experimental unit, so (with moderate or high numbers of factors and levels) two runs per treatment combination would supply ample degrees of freedom for the error. Given that the error in this situation is random, the underlying statistical assumptions hold. Secondly, if the model is deterministic (or if stochastic variation is only relatively minor), then the necessary error structure can be generated by blocking. Here, the chosen treatments are repeated across a number of blocks (soil type, breeds, varieties, climatic conditions, farm sizes or structures, etc.). These block are analysed as a main effect (i.e. allowed to be different), but all treatment interactions with blocks are used for the error. If the blocks are randomly taken from the available populations, the ANOVA is valid. The third situation, to be avoided if possible, occurs when the model is deterministic and realistic blocks cannot be identified. Here, no valid measure of experimental error exists. Higher-order interactions may be pooled to estimate this; however, the resultant probability levels cannot be used. When interpreting ANOVA results from data generated by models, the relative sizes of the treatment mean squares (as also reflected in the variance ratios or F-values) are often more important than the calculated probability levels. These probability levels, usually taken as significant if p < 0.01 or p < 0.05, depend heavily on the generated error (which is specific to the model under study), and on the number of observations, which can fairly easily be increased by running a larger experiment. In theory, any treatment contrast or interaction (no matter how minor) can be shown significant with enough replication, so actual probability levels should assume less importance. Also, with the large number of contrasts likely to occur with analysis of modelling experiments, some significant results will occur by random chance (e.g. 1 in 20 if using p -- 0.05), so these probability levels form a guide only. The mean squares and F-values, on the other hand, reflect the relative magnitude of the effects, and are more important when interpreting
Statistical experiments with simulation models
207
results and making recommendations. For example, it would be unwise to quote a four-way interaction with an F-value of 1.8 (p < 0-01) if none of the three-way interactions were significant and there existed a dominant two-way interaction with an F-value of 45 (p also < 0.01). Here, the two-way interaction is 25 times the magnitude of the four-way, and is a far more parsimonious result to quote. On a similar basis, all significant higher-order interactions should be thoroughly screened to see if the effects cannot be found and reported in simpler terms.
D A I R Y GENETICS MODEL The wider availability of artificial breeding in the Australian dairy industry has allowed producers far greater control over the genetic improvement of their herds. The Australian Dairy Herd Improvement Scheme estimates Australian Breeding Values (ABVs) for several production and conformation traits as outlined in Jones (1985), and producers can select sires according to the needs of their herd. Whilst economic indices may be used to maximise genetic gain (Beard, 1987), most producers tend to select directly on either milk or fat ABVs. These ABVs, estimated using the methods of Henderson (1984), are widely accepted and used within the industry. The additive deterministic effects are easily calculated as the average of the sire and dam ABVs. For any given management option, the direct effects on individual heifers used for herd replacements can be estimated. However, the dynamic effect on the whole herd of a range of management strategies over time is not as easily perceived, especially when regarding complex attributes such as overall herd fat percentage. Combinations of options may interact with each other or time, especially in the presence of random processes. These processes include death, disease, reproductive culling, matings from on-farm bulls and half of the resultant calves being male. Allen and McGlade (1987) have shown that it is often individual events and chance fluctuations which shape the evolutionary nature of a system, as opposed to average population trends. The most effective method of testing all these effects in combination is with a stochastic simulation model. This study outlines a genetic improvement model which allows all the commonly used herd improvement strategies available to dairy farmers. An extensive factorial experiment is run to evaluate their effects and interactions, in the presence of various random events as described previously. The results are interpreted in terms of the likely production gains to be achieved from each combination of management options.
208
D. G. Mayer, M. L. Tierney, P. N. Thurbon
These must then be balanced against probable risk and the expected capital and labour costs of each. Methods Model
Each year, the milking herd loses cows through death, disease, reproductive problems, and culling on yield and age. Their replacements are selected from the maiden heifer group, which have been produced onfarm from either artificial breeding or farm bulls. A range of management options are available in the model. Sires can be chosen from artificial breeding (proven or unproven sires) or on-farm bulls, or any combination of these three. Depending on farm needs, these specified sires can be ranked and used according to milk, fat or fat % ABVs. Of the resulting heifers, varying numbers can be kept for herd replacements. The calves produced from the early breeding of heifers, a contentious point in the industry due to calving difficulties at this early age, may or may not be included in this group. Each year, varying numbers of cows in the milking herd can be culled, according to age or low-yield basis, or a combination of both. Their replacements may be chosen either according to ABV ranking (if ABV records are kept), or according to other factors such as growth, conformation, temperament, etc. (i.e. effectively at random with respect to production ABVs). Whilst the client is free to choose any combination of these options, a number of safeguards are built into the model. The first concerns fat percentage of the herd, with the user specifying an acceptable lower limit. If this limit is crossed in the simulations, the nominated sire and herd replacement criteria (usually milk or fat ABV ranking) is automatically replaced by selection according to fat percentage ABV ranking. This maximises the rate of increase of herd fat percentage, and should rapidly correct the problem. The second safeguard is the specification of an upper allowable limit of inbreeding. The pedigree of each dam and sire used in the model is traced by identifying their paternal sire, two grandsires and four great-grand-sires. For each dam, according to selection and mating criteria, the list of available sires is searched (best to worst) until the inbreeding limit is not exceeded. If it is exceeded with all sires, a warning is printed, indicating insufficient genetic diversity in the available sires. M o d e l assumptions
1.
Deaths and reproductive/disease culling occur at random across the herd.
Statistical experiments with simulation models
2. 3. 4. 5. 6.
209
Approximately half the calves produced (allocated at random) are male, and thus not available as herd replacements. The milking herd is stable, i.e. numbers are unchanged over years. Heifers are available for mating at two years of age. If mixed siring methods are used, cows are allocated to sires at random. Milk and fat yields of all animals are determined by ABVs.
Validation Statistical validation of simulated herd milk and fat production is not possible, as the influences of annual environmental variation and changing management strategies have at least as much influence on production as does genetic gain. Two identical years will never be observed, hence it is impossible to estimate production increase due to genetic improvement alone. Validation of this model lies in the acceptance of the additivity of ABVs, and the model has been subjected to extensive testing and verification.
Experimental design A number of smaller-scale analyses were initially conducted as a preliminary test of the effects and interactions of various suggested strategies, of the correctness of the error terms in the ANOVA, and of the appropriateness of the time span for the experiment. For the main analyses, exogenous variables were set at 2% per annum deaths, 5°,/oper annum reproductive/disease culling, a minimum herd fat of 3.6% and a maximum allowable inbreeding of 12.5%. The factorial treatments tested were: • 5 'sires selection' (0%, 25%, 50%, 75% or 100% proven sires; remainder unproven) by • 4 'culling levels' (10%, 15%, 20% or 25% of the herd per year) by • 2 'selection criteria' (selected on milk or fat ABVs) by • 2 'use of heifers' calves' (yes or no) by • 2 'herd replacement polices' (replacements selected on ABVs or random) by • 2 'culling methods' (age or yield culling) by • 2 'times' (year 1 or year 3). Available farms within the herd recording scheme were stratified according to average ABVs, with one taken from each of the low, average and high strata. These represented a range of different initial herd conditions (ABVs, ages and genetic backgrounds) typical of recorded herds in Queensland. Farm types were considered as a blocking
210
D. G. Mayer, M. L. Tierney, P. N. Thurbon
term, as any treatments to be recommended should perform well over all farms, not specifically over one or two. The results of overall herd increases in milk (litres) and fat (kg) production (per cow.year) and herd fat percentage (per year), were subjected to split-plot analysis of variance (for times), with farm types as blocks providing the experimental error. Results
Preliminary analyses Results from the initial ANOVAs were used to identify and then exclude 'unrealistic' treatment combinations, i.e. those parts of the factorial which are unlikely to be observed in practice and which contribute strongly to the observed interactions. Also, a double split-plot ANOVA was conducted comparing the block error (from farm types; split over times) with the within error (the second split, from stochastic replications). Across all variates analysed, the within errors were less than the block errors (coefficients of variation were 6.9% and 12.0% for milk ABVs respectively, and 5.3% and 7-4% for fat ABVs). This indicates that the random processes within farms were relatively less important than the differential responses from farm types, and that the latter term should be used as the appropriate error line in analyses. The actual blocks term (representing farm types) was always significant (p < 0.01), with the 'poorer' farms (as indicated by average ABVs) demonstrating the greatest gains. There were no failures due to inbreeding, indicating sufficient genetic diversity in the current sires list. In only 2% of runs was selection criteria changed to fat percentage due to herd fat percentage falling below the specified minimum; as expected, this occurred only with combinations of high culling rates, selection on milk ABVs and higher ratios of proven sires. These failures only occurred in the fourth or fifth years of the same policy, so in practice the producer would have ample warning to change policy.
Main analyses With over 100 individual tests of main effects and interactions in each analysis of variance, interpretation was not expected to be simple. Fortunately, only one three-way interaction, namely 'sires selection' by 'selection criteria' by 'culling level', was both dominant and significant (p < 0.01) for each of the three main variates. Results of this interaction are shown in Fig. 1. Whilst complex, the patterns observed in Fig. 1(a) are logical in terms
Statistical experiments with simulation models
211
of dairying. If selecting on fat, the increases in milk ABVs are largely independent of the ratio between unproven and proven sires. There is approximately a constant response within each culling level, with culling levels being significantly different. If selecting on milk, however, proven sires are clearly superior. Culling levels, again clearly different on a 'main effect' level, also show larger differences with proven than with unproven sires. The patterns of increase in fat ABVs (Fig. l(b)), are similar but converse to those observed for milk. Obviously, selection on fat gives higher gains than selection on milk, but in this case proven bulls are superior on both counts, as indicated by the positive slopes of all lines. Again culling level is dominant as a main effect, and with fat selection only interacts with sires selection in that higher responses are observed as the percentage of proven sires increases. The changes in herd fat percentage (Fig. l(c)) are particularly important for producers to note. If choosing unproven bulls, the herd fat percentage will rise, with the extent positively correlated with culling level. If selecting on fat ABVs, increasing the proportion of proven sires increases the rate of change in fat percentage, with higher culling rates acting synergistically. If selecting on milk ABVs, with more than 25% proven sires, herd fat percentage will fall, with the rate depending on the percentage of proven sires. At mid-levels (around 50%), culling level has little effect, with a decline in herd fat percentage of around 0-015 per year. As proven sires tend towards 100%, culling level becomes more important, with declines in fat percentage of 0.027 per year at 10%, to 0.045 per year at 25% culling. The second dominant effect across each variate was the 'culling method' by 'time' interaction, as presented in Table 1. These data show large differences initially between age and yield culling, with approximately equal effects in the third year. This is due to the presence of TABLE 1 Increase in Australian Breeding Values (cow.year) -l and Change in Herd Fat Percentage per Year for Culling Method and Time Interaction Milk ABVs (litres)
Age culling Yield culling Average 1.s.d. a
Fat ABVs (kg)
Year 1
Year 3
Year 1
116-9 138-8
109.1 111.8
4.58 5.89
2.2
a l.s.d. = least significant difference.
Year 3 4.83 4.83
0-06
Change in herd f a t percentage Year 1
Year 3
0-002 0.015 0.013 0.012 0.002
~
22O m
o
> I
25~
~
20~
o u \ J v > 8
< x X
-- . . . . . . . . .
~
- 20~
. . . . . . .
60
0
-
2t5
715
5tO
~ercent
10~
I O0
proven
bulls
(a)
f
0 U \
~
~6
/-
~
25~
I I
<
It
4 ~
~0
215
510 Percent
715 proven
I0~
I
1O0 bulls
(b) Fig. 1.
Effect of sire selection policy (X-ordinate), selection criteria ( - - ,
selected on
milk ABVs; . . . . . . , selected on fat ABVs), and culling policy (annual percentage is annotated) on: (a) average increase in herd milk ABVs (cow.year)-l; (b) average increase in herd fat ABVs (cow.year) l; (c) change in herd fat percentage year -l. Vertical bars indicate l.s.d. (p = 0-01).
Statistical experiments with simulation models
213
O. 075:
0 L
0.
05C
0.
000
~ "~
I
~
e r
g
-o.
o251-
~
I
~
~
-~....~~-------
I
1 o~
~ ~ - " - -
I
I
Bx
2o,, 25~
--0.06 0
25
75
50
Percent
100
proven
~ulls
(c) Fig. 1.---contd.
genetically inferior cows of various ages in each of the herds tested. Initially, direct yield culling eliminates these from the herd, giving larger gains. However, with improvement policies, the herd soon stabilises to a situation where age and ABVs are highly correlated, and culling according to either age or yield have equal effects. The remaining factors, namely 'herd replacement policy' and 'use of heifers' calves', appeared (except for one instance) as significant (p < 0.01) and dominant main effects, as shown in Table 2. These results indicate an obvious advantage in using each of these policies, provided the gains warrant the additional costs and efforts involved. TABLE 2 Increase in Australian Breeding Values (cow.year) -1 and Change in Herd Fat Percentage per Year for Herd Replacement Management Options (Main Effects) Option
Selection o f herd replacements Yes
Milk ABVs (litres) Fat ABVs (kg)
125.2 5.26 Change in herd fat percentage 0.010
No 113.1 4-80 0.011
Ls.d. (p = 0.01) 1-4 0.04 (n.s.)
Use of heifers' calves Yes
No
120.7 117-5 5.14 4.93 0.012 0.010
l.s.d. (p = 0.01) 1.4 0.04 0.001
214
D. G. Mayer, M. L. Tierney, P. N. Thurbon
Discussion
Dairy producers can use these results to balance the genetic needs of their herds. The relative gains from each option can be estimated, ranked and evaluated. The results indicate that maximal increases in ABVs will be achieved with high culling on yield, using proven sires, selection according to desired trait, and selecting herd replacements on estimated ABVs. The likely overall genetic gain from the early breeding of heifers is only relatively minor, and probably not worth the trouble. Regarding potential milk production, substantial gains are possible, of the order of 240 litres (cow.year)-~. However, gains from this 'maximal' policy are offset by herd fat percentage, which is expected to fall by 0.042 per year. This problem can easily be solved by substituting fat for milk selection, giving expected ABV increases of 9.0 kg (cow.year) ~ for fat and 143 litres (cow.year)-1 for milk. More importantly, herd fat percentage would increase by 0.076 per year, roughly twice the rate lost by maximal milk selection. In practice, alternating between total milk or fat selection is unlikely, with a balanced intermediate level probable. If fat percentage is becoming a problem, sires with 'good' milk but 'better' fat ABVs would tend to be used, with the selection depending on the current herd fat percentage, the minimum allowable, and the producer's individual perceptions. It is difficult to simulate such an 'artistic' approach, as different people tend to interpret this 'rule-of-thumb' differently. In these situations the model can be interactively run on an individual property level, and appropriate recommendations made. This overall study has been useful in determining which managerial options (and combinations thereof) are likely to give the largest consistent genetic gains across the herd, and to estimate the relative magnitude of the gains.
CONCLUSIONS With the increasing power and capacity of modern computers, and the availability of appropriate statistical packages, there appears no valid reason not to conduct a thorough investigative analysis of a completed simulation model. Results can aid in the understanding of the system, and the process of interpreting and explaining the interactions forms an extensive verification of the model. Agronomic recommendations should only be made after all options have been evaluated, and often results indicate uncertain areas of the system which require further research. If model recommendations are at odds with current agronomic practices
Statistical experiments with simulation models
215
the model becomes a hypothesis-generating tool: the recommended method of system management should then be tested in field trials against the current system. The form of experimentation presented here has shown that large-scale statistical analyses are both feasible and meaningful. Dairy producers can consider the range of herd improvement options available, and maximise likely production gains whilst guarding against detrimental effects such as inbreeding and declining herd fat percentage. Results from the artificial breeding scheme and model are currently being extended to the industry, but this first required an analysis and understanding of just which effects were most important.
REFERENCES Allen, P. M. & McGlade, J. M. (1987). Evolutionary drive: The effect of microscopic diversity, error making, and noise. Foundations of Physics, 17, 723-38. Beard, K. T. (1987). Efficiency of index selection for dairy cattle using economic weights for major milk constituents. Australian J. Agricultural Research, 38, 273-84. Boneau, C. A. (1960). The effects of violations of assumptions underlying the t test. Psychological Bulletin, 57, 49-64. Cochran, W. G. & Cox, G. M. (1957). Experimental Designs, 2nd edn. Wiley, New York. Dent, J. B. & Blackie, M. J. (1979). Systems Simulation in Agriculture. Applied Science Publishers, London. Dommerholt, J. & Wilmink, J. B. M. (1986). Optimal selection responses under varying milk prices and margins for milk production. Livestock Production Science, 14, 109-21. Harrison, S. R. (1990). Regression of a model on real-system output: An invalid test of model validity. Agricultural Systems, 34, 183-90. Henderson, C. R. (1984). Estimation of variances and covariances under multiple trait models. J. Dairy Science, 67, 1581-9. Jones, L. P. (1985). Australian breeding values for production characteristics. Paper presented at 5th Conference of the Australian Association of Animal Breeding and Genetics, University of New South Wales, Sydney, Australia, 26-28 August. Kleijnen, J. P. C. (1987). Statistical Tools for Simulation Practitioners. Marcel Dekker, New York. Loague, K. & Green, R. E. (1991). Statistical and graphical methods for evaluating solute transport models: Overview and application. J. Contaminant Hydrology, 7, 51-73. Lunney, G. H. (1970). Using analysis of variance with a dichotomous dependent variable: An empirical study. J. Educational Measurement, 7, 263-9. Maher, J. (1989). HYBREED--an opportunity to increase dairy farm profitability. Paper presented at 4th National Computers in Agriculture Conference, Qld Department of Primary Industries, Maroochydore, 8-12 May.
216
D. G. Mayer, M. L. Tierney, P. N. Thurbon
Mayer, D. G., Schoorl, D., Butler, D. G. & Kelly, A. M. (1991). Efficiency and fractal behaviour of optimisation methods on multiple-optima surfaces. Agricultural Systems, 36, 315-28. Mead, R. & Curnow, R. N. (1983). Statistical Methods in Agriculture and Experimental Biology. Chapman and Hall, London. Naylor, T. H., Burdick, D. S. & Sasser, W. E. (1967). Computer simulation experiments with economic systems: The problem of experimental design. Z the American Statistical Association, 62, 1315-37. Nishisato, S. (1980). Analysis of Categorical Data: Duel Scaling and Its Applications. University of Toronto Press, Toronto, Canada. Notter, D. R. & Johnson, M. H. (1987). Simulation of genetic control of reproduction in beef cows. III. Within-herd breeding value estimation with known breeding dates. J. Animal Science, 65, 88-98. Notter, D. R. & Johnson, M. H. (1988). Simulation of genetic control of reproduction in beef cows. IV. Within-herd breeding value estimation with pasture mating. J. Animal Science, 66, 280-6. Phillips, J. B. (1971). Statistical methods in systems analysis. In Systems Analysis in Agricultural Management, ed. J. B. Dent & J. R. Anderson. Wiley, Sydney, Australia, pp. 34-52. Poole, T. G. & Szymankiewicz, J. Z. (1977). Using Simulation to Solve Problems. McGraw-Hill, London. Shannon, R. E. (1975). Systems Simulation. Prentice-Hall, N J, USA. Snedecor, G. W. & Cochran, W. G. (1980). Statistical Methods, 7th edn. Iowa State University Press, Ames, IA, USA. Whitmore, A. P. (1991). A method for assessing the goodness of computer simulation of soil processes. J. Soil Science, 42, 289-99. Wright, A. (1971). Farming systems, models and simulation. In Systems Analysis in Agricultural Management, ed. J. B. Dent & J. R. Anderson. Wiley, Sydney, Australia, pp. 17-33.
,
AgriculturalSystems45 (1994) 203-216
j
© 1994 Elsevier Science Limited Printed in Great Britain. All rights reserved 0308-521X/94/$7.00 ELSEVIER
Statistical Experiments with Simulation Models A Dairy Genetics Example D. G. Mayer, ~ M. L. Tierney b & P. N. T h u rb o n b a Biometry and b Dairy and Animal Breeding Branches, Queensland Department of Primary Industries, GPO Box 46, Brisbane 4001, Australia (Received 9 December 1991; accepted 9 June 1993)
ABSTRA CT Simulation models have been developed for many agricultural applications. Model experimentation, using analysis of variance of a large-scale factorial, is a valuable exploratory exercise to investigate the interacting effects of managerial options and external factors. To achieve this analysis, consideration must first be given to a number of potential problems, including definition of treatments, experimental design, underlying statistical assumptions, replication, appropriateness of the error structure, and interpretation of results. Such analyses are useful in identifying the key main effects and interactions in the system, which can then be recommended to industry or investigated further. The practicalities and advantages of this approach for multi-optional models are illustrated using a stochastic dairy genetics model.
INTRODUCTION
In most fields of agricultural research, increasing numbers of decision-aid packages or simulation models are becoming available. These are usually developed for a defined agricultural system, or, if generic in nature, specifically tuned to a target system. Following validation against realworld data (Dent & Blackie, 1979; Kleijnen, 1987), models can be a valuable aid to understanding or optimising the target system. If a key economic or production variable can be nominated, optimisation is recommended, with a range of methods available (Mayer et al., 1991). 203
204
D. G. Mayer, M. L. Tierney, P. N. Thurbon
In many modelling situations, 'what-if' investigations and experimentation (Naylor et al., 1967) are more applicable or important than optimisation. Typically, the managers of the agricultural system wish to investigate the likely effects of the various options available, often in conjunction with expected external (uncontrollable) effects such as climate or market prices. This simulation approach was used by Dommerholt and Wilmink (1986) with genetic parameters in the optimisation of dairy farm returns in the Netherlands. In Australian dairying, Beard (1987) evaluated index selection to maximise herd genetic improvement, and Maher (1989) outlines 'Hybreed', a simulation model for extension and educational purposes which evaluates the potential gain from including heifers in the breeding plan. Notter and Johnson (1987, 1988) also used genetic models to investigate the potential genetic improvement in beef herds. In most situations, modelling studies such as these are conducted on a 'one-at-a-time' basis, whereby the model is configured to some average or optimal scenario and individual management options are altered and investigated in turn. This approach considers only main effects. However, in many complex agriculture systems, it can be the higher-order interactions between both management options and external influences that are of prime importance. These interactions can be used to understand the system and to identify recommended practices. For example, the most profitable level of one management factor (e.g. applied fertiliser) usually depends on a whole range of other factors (fertiliser and commodity prices, weather, soil type and fertility, applied levels of other fertilisers and irrigation, crop and variety, etc). Even if high-order interactions do not eventuate, it is necessary to confirm that the main effects of the treatments are consistent across all the other options tested. These hypotheses are best investigated with large-scale factorial experimentation, as recommended by Naylor et al. (1967), Shannon (1975) and Poole and Szymankiewicz (1977). Under this approach, each independent run of the simulation model is considered an experimental unit for the analysis of variance (ANOVA).
STATISTICAL CONSIDERATIONS A number of studies of model validation techniques have recommended the use of formal statistical tests, providing the underlying assumptions are not violated (Harrison, 1990; Loague & Green, 1991; Whitmore, 1991). The key statistical assumptions of the ANOVA are that observations are independent, variances are homogeneous, and responses are normally
Statistical experiments with simulation models
205
distributed about their expected values (Snedecor & Cochran, 1980). Considering these in turn, the assumption of independence often causes concern in modelling experiments, for two reasons. Firstly, many studies generate time-series, which are auto-correlated. This problem can be overcome by either reducing these series to a single value or index, or by identifying appropriately long time periods which ensure independence and condensing data within each period (Wright, 1971). The second reason the problem of dependence may occur is if modellers use the same random number sequence in successive runs. Whilst this practice has variance reduction advantages, it introduces correlations into output results. If different (independent) random number sequences are used, this statistical assumption is upheld (Phillips, 1971). The remaining assumptions, concerning variance (second moment of the distribution) and normality (higher moments), are related. The ANOVA technique has been shown to be quite robust to departures from these assumptions (Boneau, 1960; Lunney, 1970; Nishisato, 1980). As with any field experiment, mild departures from these assumptions may be expected, and as suitable statistical transformations can be applied to correct for any moderate or serious departures (Mead & Curnow, 1983), these assumptions should cause few concerns. As with any scientific trial, an appropriate statistical design is important. With the availability of general model packages (e.g. GLIM, GENSTAT), both factors (of fixed discrete levels) and variates (of a continuous nature, allowing response surface analysis (Cochran & Cox, 1957)) can be utilised. To fit the latter, the shape of the surface must be assumed a priori and specific in the analytical model, with quadratics or higher-degree polynomials often used. These however, may be inappropriate when considering agricultural systems and models of these, as their smooth nature cannot, for example, mimic the 'crashes' observed when a system is over-utilised (e.g. over-stocked). A safer approach, and one which statistically fits a similar number of constants, is to use fixed levels of each continuous variate, and fit curves to the estimated responses at these levels if appropriate. Whilst modelling experiments can be easier to set up and cheaper to run than traditional field trials, resources are still not infinite. For example, a complete factorial of ten factors at five levels would require approximately ten million model runs, which may well be excessive in terms of run-time and data storage. Use of fractional replication designs, which can reduce the size by a factor of two or four at the expense of having effects confounded, is one possible solution (Kleijnen, 1987). More sophisticated analytical packages (such as GENSTAT) are required to analyse these designs. Usually complete factorials are preferred
206
D. G. Mayer, M. L. Tierney, P. N. Thurbon
(Naylor et al., 1967; Shannon, 1975), as interpretation of results is unambiguous. In practical terms, the capacity of available computer or analytical packages often determines the upper size limit of the simulation experiment. Factors and their levels are then chosen to comply with this limit, with the important agronomic factors having more levels (usually four to six) and others having fewer levels (often only two) or being omitted altogether. The statistical soundness of the estimated error structure depends on the type of simulation model used to generate the results, with three possibilities. Firstly, the model may be stochastic with realistic random variation if the major processes contributing variability have been built into the model. Here, each model run becomes an experimental unit, so (with moderate or high numbers of factors and levels) two runs per treatment combination would supply ample degrees of freedom for the error. Given that the error in this situation is random, the underlying statistical assumptions hold. Secondly, if the model is deterministic (or if stochastic variation is only relatively minor), then the necessary error structure can be generated by blocking. Here, the chosen treatments are repeated across a number of blocks (soil type, breeds, varieties, climatic conditions, farm sizes or structures, etc.). These block are analysed as a main effect (i.e. allowed to be different), but all treatment interactions with blocks are used for the error. If the blocks are randomly taken from the available populations, the ANOVA is valid. The third situation, to be avoided if possible, occurs when the model is deterministic and realistic blocks cannot be identified. Here, no valid measure of experimental error exists. Higher-order interactions may be pooled to estimate this; however, the resultant probability levels cannot be used. When interpreting ANOVA results from data generated by models, the relative sizes of the treatment mean squares (as also reflected in the variance ratios or F-values) are often more important than the calculated probability levels. These probability levels, usually taken as significant if p < 0.01 or p < 0.05, depend heavily on the generated error (which is specific to the model under study), and on the number of observations, which can fairly easily be increased by running a larger experiment. In theory, any treatment contrast or interaction (no matter how minor) can be shown significant with enough replication, so actual probability levels should assume less importance. Also, with the large number of contrasts likely to occur with analysis of modelling experiments, some significant results will occur by random chance (e.g. 1 in 20 if using p -- 0.05), so these probability levels form a guide only. The mean squares and F-values, on the other hand, reflect the relative magnitude of the effects, and are more important when interpreting
Statistical experiments with simulation models
207
results and making recommendations. For example, it would be unwise to quote a four-way interaction with an F-value of 1.8 (p < 0-01) if none of the three-way interactions were significant and there existed a dominant two-way interaction with an F-value of 45 (p also < 0.01). Here, the two-way interaction is 25 times the magnitude of the four-way, and is a far more parsimonious result to quote. On a similar basis, all significant higher-order interactions should be thoroughly screened to see if the effects cannot be found and reported in simpler terms.
D A I R Y GENETICS MODEL The wider availability of artificial breeding in the Australian dairy industry has allowed producers far greater control over the genetic improvement of their herds. The Australian Dairy Herd Improvement Scheme estimates Australian Breeding Values (ABVs) for several production and conformation traits as outlined in Jones (1985), and producers can select sires according to the needs of their herd. Whilst economic indices may be used to maximise genetic gain (Beard, 1987), most producers tend to select directly on either milk or fat ABVs. These ABVs, estimated using the methods of Henderson (1984), are widely accepted and used within the industry. The additive deterministic effects are easily calculated as the average of the sire and dam ABVs. For any given management option, the direct effects on individual heifers used for herd replacements can be estimated. However, the dynamic effect on the whole herd of a range of management strategies over time is not as easily perceived, especially when regarding complex attributes such as overall herd fat percentage. Combinations of options may interact with each other or time, especially in the presence of random processes. These processes include death, disease, reproductive culling, matings from on-farm bulls and half of the resultant calves being male. Allen and McGlade (1987) have shown that it is often individual events and chance fluctuations which shape the evolutionary nature of a system, as opposed to average population trends. The most effective method of testing all these effects in combination is with a stochastic simulation model. This study outlines a genetic improvement model which allows all the commonly used herd improvement strategies available to dairy farmers. An extensive factorial experiment is run to evaluate their effects and interactions, in the presence of various random events as described previously. The results are interpreted in terms of the likely production gains to be achieved from each combination of management options.
208
D. G. Mayer, M. L. Tierney, P. N. Thurbon
These must then be balanced against probable risk and the expected capital and labour costs of each. Methods Model
Each year, the milking herd loses cows through death, disease, reproductive problems, and culling on yield and age. Their replacements are selected from the maiden heifer group, which have been produced onfarm from either artificial breeding or farm bulls. A range of management options are available in the model. Sires can be chosen from artificial breeding (proven or unproven sires) or on-farm bulls, or any combination of these three. Depending on farm needs, these specified sires can be ranked and used according to milk, fat or fat % ABVs. Of the resulting heifers, varying numbers can be kept for herd replacements. The calves produced from the early breeding of heifers, a contentious point in the industry due to calving difficulties at this early age, may or may not be included in this group. Each year, varying numbers of cows in the milking herd can be culled, according to age or low-yield basis, or a combination of both. Their replacements may be chosen either according to ABV ranking (if ABV records are kept), or according to other factors such as growth, conformation, temperament, etc. (i.e. effectively at random with respect to production ABVs). Whilst the client is free to choose any combination of these options, a number of safeguards are built into the model. The first concerns fat percentage of the herd, with the user specifying an acceptable lower limit. If this limit is crossed in the simulations, the nominated sire and herd replacement criteria (usually milk or fat ABV ranking) is automatically replaced by selection according to fat percentage ABV ranking. This maximises the rate of increase of herd fat percentage, and should rapidly correct the problem. The second safeguard is the specification of an upper allowable limit of inbreeding. The pedigree of each dam and sire used in the model is traced by identifying their paternal sire, two grandsires and four great-grand-sires. For each dam, according to selection and mating criteria, the list of available sires is searched (best to worst) until the inbreeding limit is not exceeded. If it is exceeded with all sires, a warning is printed, indicating insufficient genetic diversity in the available sires. M o d e l assumptions
1.
Deaths and reproductive/disease culling occur at random across the herd.
Statistical experiments with simulation models
2. 3. 4. 5. 6.
209
Approximately half the calves produced (allocated at random) are male, and thus not available as herd replacements. The milking herd is stable, i.e. numbers are unchanged over years. Heifers are available for mating at two years of age. If mixed siring methods are used, cows are allocated to sires at random. Milk and fat yields of all animals are determined by ABVs.
Validation Statistical validation of simulated herd milk and fat production is not possible, as the influences of annual environmental variation and changing management strategies have at least as much influence on production as does genetic gain. Two identical years will never be observed, hence it is impossible to estimate production increase due to genetic improvement alone. Validation of this model lies in the acceptance of the additivity of ABVs, and the model has been subjected to extensive testing and verification.
Experimental design A number of smaller-scale analyses were initially conducted as a preliminary test of the effects and interactions of various suggested strategies, of the correctness of the error terms in the ANOVA, and of the appropriateness of the time span for the experiment. For the main analyses, exogenous variables were set at 2% per annum deaths, 5°,/oper annum reproductive/disease culling, a minimum herd fat of 3.6% and a maximum allowable inbreeding of 12.5%. The factorial treatments tested were: • 5 'sires selection' (0%, 25%, 50%, 75% or 100% proven sires; remainder unproven) by • 4 'culling levels' (10%, 15%, 20% or 25% of the herd per year) by • 2 'selection criteria' (selected on milk or fat ABVs) by • 2 'use of heifers' calves' (yes or no) by • 2 'herd replacement polices' (replacements selected on ABVs or random) by • 2 'culling methods' (age or yield culling) by • 2 'times' (year 1 or year 3). Available farms within the herd recording scheme were stratified according to average ABVs, with one taken from each of the low, average and high strata. These represented a range of different initial herd conditions (ABVs, ages and genetic backgrounds) typical of recorded herds in Queensland. Farm types were considered as a blocking
210
D. G. Mayer, M. L. Tierney, P. N. Thurbon
term, as any treatments to be recommended should perform well over all farms, not specifically over one or two. The results of overall herd increases in milk (litres) and fat (kg) production (per cow.year) and herd fat percentage (per year), were subjected to split-plot analysis of variance (for times), with farm types as blocks providing the experimental error. Results
Preliminary analyses Results from the initial ANOVAs were used to identify and then exclude 'unrealistic' treatment combinations, i.e. those parts of the factorial which are unlikely to be observed in practice and which contribute strongly to the observed interactions. Also, a double split-plot ANOVA was conducted comparing the block error (from farm types; split over times) with the within error (the second split, from stochastic replications). Across all variates analysed, the within errors were less than the block errors (coefficients of variation were 6.9% and 12.0% for milk ABVs respectively, and 5.3% and 7-4% for fat ABVs). This indicates that the random processes within farms were relatively less important than the differential responses from farm types, and that the latter term should be used as the appropriate error line in analyses. The actual blocks term (representing farm types) was always significant (p < 0.01), with the 'poorer' farms (as indicated by average ABVs) demonstrating the greatest gains. There were no failures due to inbreeding, indicating sufficient genetic diversity in the current sires list. In only 2% of runs was selection criteria changed to fat percentage due to herd fat percentage falling below the specified minimum; as expected, this occurred only with combinations of high culling rates, selection on milk ABVs and higher ratios of proven sires. These failures only occurred in the fourth or fifth years of the same policy, so in practice the producer would have ample warning to change policy.
Main analyses With over 100 individual tests of main effects and interactions in each analysis of variance, interpretation was not expected to be simple. Fortunately, only one three-way interaction, namely 'sires selection' by 'selection criteria' by 'culling level', was both dominant and significant (p < 0.01) for each of the three main variates. Results of this interaction are shown in Fig. 1. Whilst complex, the patterns observed in Fig. 1(a) are logical in terms
Statistical experiments with simulation models
211
of dairying. If selecting on fat, the increases in milk ABVs are largely independent of the ratio between unproven and proven sires. There is approximately a constant response within each culling level, with culling levels being significantly different. If selecting on milk, however, proven sires are clearly superior. Culling levels, again clearly different on a 'main effect' level, also show larger differences with proven than with unproven sires. The patterns of increase in fat ABVs (Fig. l(b)), are similar but converse to those observed for milk. Obviously, selection on fat gives higher gains than selection on milk, but in this case proven bulls are superior on both counts, as indicated by the positive slopes of all lines. Again culling level is dominant as a main effect, and with fat selection only interacts with sires selection in that higher responses are observed as the percentage of proven sires increases. The changes in herd fat percentage (Fig. l(c)) are particularly important for producers to note. If choosing unproven bulls, the herd fat percentage will rise, with the extent positively correlated with culling level. If selecting on fat ABVs, increasing the proportion of proven sires increases the rate of change in fat percentage, with higher culling rates acting synergistically. If selecting on milk ABVs, with more than 25% proven sires, herd fat percentage will fall, with the rate depending on the percentage of proven sires. At mid-levels (around 50%), culling level has little effect, with a decline in herd fat percentage of around 0-015 per year. As proven sires tend towards 100%, culling level becomes more important, with declines in fat percentage of 0.027 per year at 10%, to 0.045 per year at 25% culling. The second dominant effect across each variate was the 'culling method' by 'time' interaction, as presented in Table 1. These data show large differences initially between age and yield culling, with approximately equal effects in the third year. This is due to the presence of TABLE 1 Increase in Australian Breeding Values (cow.year) -l and Change in Herd Fat Percentage per Year for Culling Method and Time Interaction Milk ABVs (litres)
Age culling Yield culling Average 1.s.d. a
Fat ABVs (kg)
Year 1
Year 3
Year 1
116-9 138-8
109.1 111.8
4.58 5.89
2.2
a l.s.d. = least significant difference.
Year 3 4.83 4.83
0-06
Change in herd f a t percentage Year 1
Year 3
0-002 0.015 0.013 0.012 0.002
~
22O m
o
> I
25~
~
20~
o u \ J v > 8
< x X
-- . . . . . . . . .
~
- 20~
. . . . . . .
60
0
-
2t5
715
5tO
~ercent
10~
I O0
proven
bulls
(a)
f
0 U \
~
~6
/-
~
25~
I I
<
It
4 ~
~0
215
510 Percent
715 proven
I0~
I
1O0 bulls
(b) Fig. 1.
Effect of sire selection policy (X-ordinate), selection criteria ( - - ,
selected on
milk ABVs; . . . . . . , selected on fat ABVs), and culling policy (annual percentage is annotated) on: (a) average increase in herd milk ABVs (cow.year)-l; (b) average increase in herd fat ABVs (cow.year) l; (c) change in herd fat percentage year -l. Vertical bars indicate l.s.d. (p = 0-01).
Statistical experiments with simulation models
213
O. 075:
0 L
0.
05C
0.
000
~ "~
I
~
e r
g
-o.
o251-
~
I
~
~
-~....~~-------
I
1 o~
~ ~ - " - -
I
I
Bx
2o,, 25~
--0.06 0
25
75
50
Percent
100
proven
~ulls
(c) Fig. 1.---contd.
genetically inferior cows of various ages in each of the herds tested. Initially, direct yield culling eliminates these from the herd, giving larger gains. However, with improvement policies, the herd soon stabilises to a situation where age and ABVs are highly correlated, and culling according to either age or yield have equal effects. The remaining factors, namely 'herd replacement policy' and 'use of heifers' calves', appeared (except for one instance) as significant (p < 0.01) and dominant main effects, as shown in Table 2. These results indicate an obvious advantage in using each of these policies, provided the gains warrant the additional costs and efforts involved. TABLE 2 Increase in Australian Breeding Values (cow.year) -1 and Change in Herd Fat Percentage per Year for Herd Replacement Management Options (Main Effects) Option
Selection o f herd replacements Yes
Milk ABVs (litres) Fat ABVs (kg)
125.2 5.26 Change in herd fat percentage 0.010
No 113.1 4-80 0.011
Ls.d. (p = 0.01) 1-4 0.04 (n.s.)
Use of heifers' calves Yes
No
120.7 117-5 5.14 4.93 0.012 0.010
l.s.d. (p = 0.01) 1.4 0.04 0.001
214
D. G. Mayer, M. L. Tierney, P. N. Thurbon
Discussion
Dairy producers can use these results to balance the genetic needs of their herds. The relative gains from each option can be estimated, ranked and evaluated. The results indicate that maximal increases in ABVs will be achieved with high culling on yield, using proven sires, selection according to desired trait, and selecting herd replacements on estimated ABVs. The likely overall genetic gain from the early breeding of heifers is only relatively minor, and probably not worth the trouble. Regarding potential milk production, substantial gains are possible, of the order of 240 litres (cow.year)-~. However, gains from this 'maximal' policy are offset by herd fat percentage, which is expected to fall by 0.042 per year. This problem can easily be solved by substituting fat for milk selection, giving expected ABV increases of 9.0 kg (cow.year) ~ for fat and 143 litres (cow.year)-1 for milk. More importantly, herd fat percentage would increase by 0.076 per year, roughly twice the rate lost by maximal milk selection. In practice, alternating between total milk or fat selection is unlikely, with a balanced intermediate level probable. If fat percentage is becoming a problem, sires with 'good' milk but 'better' fat ABVs would tend to be used, with the selection depending on the current herd fat percentage, the minimum allowable, and the producer's individual perceptions. It is difficult to simulate such an 'artistic' approach, as different people tend to interpret this 'rule-of-thumb' differently. In these situations the model can be interactively run on an individual property level, and appropriate recommendations made. This overall study has been useful in determining which managerial options (and combinations thereof) are likely to give the largest consistent genetic gains across the herd, and to estimate the relative magnitude of the gains.
CONCLUSIONS With the increasing power and capacity of modern computers, and the availability of appropriate statistical packages, there appears no valid reason not to conduct a thorough investigative analysis of a completed simulation model. Results can aid in the understanding of the system, and the process of interpreting and explaining the interactions forms an extensive verification of the model. Agronomic recommendations should only be made after all options have been evaluated, and often results indicate uncertain areas of the system which require further research. If model recommendations are at odds with current agronomic practices
Statistical experiments with simulation models
215
the model becomes a hypothesis-generating tool: the recommended method of system management should then be tested in field trials against the current system. The form of experimentation presented here has shown that large-scale statistical analyses are both feasible and meaningful. Dairy producers can consider the range of herd improvement options available, and maximise likely production gains whilst guarding against detrimental effects such as inbreeding and declining herd fat percentage. Results from the artificial breeding scheme and model are currently being extended to the industry, but this first required an analysis and understanding of just which effects were most important.
REFERENCES Allen, P. M. & McGlade, J. M. (1987). Evolutionary drive: The effect of microscopic diversity, error making, and noise. Foundations of Physics, 17, 723-38. Beard, K. T. (1987). Efficiency of index selection for dairy cattle using economic weights for major milk constituents. Australian J. Agricultural Research, 38, 273-84. Boneau, C. A. (1960). The effects of violations of assumptions underlying the t test. Psychological Bulletin, 57, 49-64. Cochran, W. G. & Cox, G. M. (1957). Experimental Designs, 2nd edn. Wiley, New York. Dent, J. B. & Blackie, M. J. (1979). Systems Simulation in Agriculture. Applied Science Publishers, London. Dommerholt, J. & Wilmink, J. B. M. (1986). Optimal selection responses under varying milk prices and margins for milk production. Livestock Production Science, 14, 109-21. Harrison, S. R. (1990). Regression of a model on real-system output: An invalid test of model validity. Agricultural Systems, 34, 183-90. Henderson, C. R. (1984). Estimation of variances and covariances under multiple trait models. J. Dairy Science, 67, 1581-9. Jones, L. P. (1985). Australian breeding values for production characteristics. Paper presented at 5th Conference of the Australian Association of Animal Breeding and Genetics, University of New South Wales, Sydney, Australia, 26-28 August. Kleijnen, J. P. C. (1987). Statistical Tools for Simulation Practitioners. Marcel Dekker, New York. Loague, K. & Green, R. E. (1991). Statistical and graphical methods for evaluating solute transport models: Overview and application. J. Contaminant Hydrology, 7, 51-73. Lunney, G. H. (1970). Using analysis of variance with a dichotomous dependent variable: An empirical study. J. Educational Measurement, 7, 263-9. Maher, J. (1989). HYBREED--an opportunity to increase dairy farm profitability. Paper presented at 4th National Computers in Agriculture Conference, Qld Department of Primary Industries, Maroochydore, 8-12 May.
216
D. G. Mayer, M. L. Tierney, P. N. Thurbon
Mayer, D. G., Schoorl, D., Butler, D. G. & Kelly, A. M. (1991). Efficiency and fractal behaviour of optimisation methods on multiple-optima surfaces. Agricultural Systems, 36, 315-28. Mead, R. & Curnow, R. N. (1983). Statistical Methods in Agriculture and Experimental Biology. Chapman and Hall, London. Naylor, T. H., Burdick, D. S. & Sasser, W. E. (1967). Computer simulation experiments with economic systems: The problem of experimental design. Z the American Statistical Association, 62, 1315-37. Nishisato, S. (1980). Analysis of Categorical Data: Duel Scaling and Its Applications. University of Toronto Press, Toronto, Canada. Notter, D. R. & Johnson, M. H. (1987). Simulation of genetic control of reproduction in beef cows. III. Within-herd breeding value estimation with known breeding dates. J. Animal Science, 65, 88-98. Notter, D. R. & Johnson, M. H. (1988). Simulation of genetic control of reproduction in beef cows. IV. Within-herd breeding value estimation with pasture mating. J. Animal Science, 66, 280-6. Phillips, J. B. (1971). Statistical methods in systems analysis. In Systems Analysis in Agricultural Management, ed. J. B. Dent & J. R. Anderson. Wiley, Sydney, Australia, pp. 34-52. Poole, T. G. & Szymankiewicz, J. Z. (1977). Using Simulation to Solve Problems. McGraw-Hill, London. Shannon, R. E. (1975). Systems Simulation. Prentice-Hall, N J, USA. Snedecor, G. W. & Cochran, W. G. (1980). Statistical Methods, 7th edn. Iowa State University Press, Ames, IA, USA. Whitmore, A. P. (1991). A method for assessing the goodness of computer simulation of soil processes. J. Soil Science, 42, 289-99. Wright, A. (1971). Farming systems, models and simulation. In Systems Analysis in Agricultural Management, ed. J. B. Dent & J. R. Anderson. Wiley, Sydney, Australia, pp. 17-33.