- Email: [email protected]
Forecasting the consequences of accidental releases of radionuclides in the atmosphere from ensemble dispersion modelling
Journal of Environmental Radioactivity 57 (2001) 203–219
Forecasting the consequences of accidental releases of radionuclides in the atmosphere from ensemble dispersion modelling S. Galmarinia,*, R. Bianconib, R. Bellasiob, G. Graziania a
b
Joint Research Center, Environment Institute, TP 321, 21020 Ispra (VA), Italy ENVIROWARE srl, Centro Direzionale Colleoni, Palazzo Andromeda 1, I-20041 Agrate Brianza (MI), Italy Received 15 June 2000; received in revised form 15 December 2000; accepted 5 January 2001
Abstract The RTMOD system is presented as a tool for the intercomparison of long-range dispersion models as well as a system for support of decision making. RTMOD is an internet-based procedure that collects the results of more than 20 models used around the world to predict the transport and deposition of radioactive releases in the atmosphere. It allows the real-time acquisition of model results and their intercomparison. Taking advantage of the availability of several model results, the system can also be used as a tool to support decision making in case of emergency. The new concept of ensemble dispersion modelling is introduced which is the basis for the decision-making application of RTMOD. New statistical parameters are presented that allow gathering the results of several models to produce a single dispersion forecast. The devised parameters are presented and tested on the results of RTMOD exercises. # 2001 Elsevier Science Ltd. All rights reserved. Keywords: Long-range transport and dispersion; Model intercomparison; Ensemble dispersion forecasting; Support to decision making
1. Introduction In the event of the accidental release of a harmful material to the atmosphere and its transport over large distances, decision-making relies on the support of long-range *Corresponding author. E-mail address: [email protected] (S. Galmarini). 0265-931X/01/$ - see front matter # 2001 Elsevier Science Ltd. All rights reserved. PII: S 0 2 6 5 - 9 3 1 X ( 0 1 ) 0 0 0 1 7 - 0
204
S. Galmarini et al. / J. Environ. Radioactivity 57 (2001) 203–219
transport and dispersion models to forecast the locations potentially affected by the passage of the pollutant. The Chernobyl accident definitely outlined our ill preparedness with respect to the above-mentioned prediction capacity and the compelling necessity for an improvement of the modelling tools. Since then, several institutes around the world have invested effort in the development and implementation of long-range atmospheric transport models for real-time forecasting of atmospheric dispersion. The approach to long-range dispersion modelling is not unique and in most cases has to be based on the delicate balance that exists between having a reliable model and a tool that can be used in real time. The various approaches that have been developed and which are currently operational in meteorological services or national protection agencies are the source of an intrinsic diversity in model results. An important component that is responsible for the differences in the model performance is, for example, the quality and the completeness of the meteorological data (mainly atmospheric circulation, precipitation and stability). The meteorological input data are the results of routine operational weather forecasting and thus the result of numerical weather predictions (NWP). Each of the institutes using a long-range transport model has access to different NWP-data produced by different weather services. An intrinsic difference is reflected in the results given the different origins of the data (e.g. Klug, Graziani, Grippa, Pierce, & Tassone, 1992). One should therefore start with the consideration that different models can inevitably produce different results due to the simple fact that some of them (or the data that they use) may give more importance and attention to different aspects of the physical processes. This also implies that none of them is in principle wrong and, most of all, that all of them should be considered for decision-making support. In order to estimate the effect of these intrinsic differences, ad-hoc tracer experiments were in the past designed and carried out. The most recent of these was the European Tracer Experiment (ETEX) that took place in 1994 and allowed the comparison of several types of models with experimental evidence (Girardi et al., 1998). One of the conclusions of the experiment was that, although modelling has greatly advanced in the last few decades, several aspects still need to be improved in order to bring the various approaches to a state of reliable maturity. Moreover it was pointed out that model intercomparison and the evaluation of the results should be performed on a regular basis in order to promote constant improvement of the quality of the tools and advancements in its development. An activity that is presently ongoing and aims at improvement of dispersion model performance for nuclear emergency application is RTMOD (Real Time Model Evaluation (Bellasio, Bianconi, Graziani, & Mosca, 1999). Given the difficulty in performing tracer experiments and their cost (especially over the long range), one way to test modelling approaches to atmospheric dispersion is in fact model intercomparison. RTMOD is a system that gathers and compares pairs of results of models operated in different national institutions and applied to a predefined case. The modelers have therefore the possibility to check for the presence of systematic errors or biases in the prediction over a range of meteorological situations.
S. Galmarini et al. / J. Environ. Radioactivity 57 (2001) 203–219
205
RTMOD, other than being instrumental to the above-mentioned scope, is also useful in decision-making. Being an internet-based system that works in real time, it provides the possibility to obtain, in the event of an emergency, the results of a number of models that are homogeneous as far as the spatial and temporal representation of the forecast are concerned. A decision-maker has therefore the possibility to access the system and consult several model results, thus avoiding the problems connected with relying on a single or few modelling approaches and to homogenize results presented in different forms. One of the aspects to be taken care of at this point is the possibility to gather all the information provided by an ensemble of models in a quantitative manner so that few statistical parameters can be looked at to decide what action to take. A sort of ensemble dispersion forecasting can be conducted in a fashion similar to meteorological ensemble forecasting. The difference lies in the fact that, rather than applying one model to a series of slightly different initial conditions, in this case different models are applied to the same case, and the results presented with few and appropriate statistical parameters. This paper presents some of these parameters and an application of this new technique.
2. The RTMOD system for model intercomparison Briefly, RTMOD is an internet-based system that allows real time model intercomparison of long-range transport and dispersion models. The RTMOD web site is located on a server at the European Commission-Joint Research Center at Ispra (Italy) and can be accessed upon authentication by a community of modelers distributed all around the world. Upon notification of a release (fictitious or real), the modelers can find on the web site information on the release characteristics and can run their dispersion models. All models are run according to their usual setup but each of them has to produce a dispersion forecast over a prescribed grid and according to a pre-defined format protocol. After the completion of the simulation, the modeler uploads the results to the RTMOD web site and an automated statistical analysis compares them with the results of all the other models already available. A detailed description of the system (http://rtmod.ei.jrc.it/rtmod) is given in Bellasio et al. (1999). Numerical and graphical results of the intercomparison are published as web pages and are readily available to the modelers. Users have the opportunity to see the comparison of their model with the others, identifying discrepancies and similarities with one or more of the other models. The real advantage of the system is that each modeler can access the results of the intercomparison within hours of the simulation and uploading procedures. This aspect constitutes a great advantage for dispersion modelling since the modeler immediately has feedback on the quality of his simulation with respect to the others. Fig. 1 gives a representation of RTMOD in the context of the notification of the national services, the collection of the model results and the model intercomparison. At present, 22 models (Table 1), operated by
206
S. Galmarini et al. / J. Environ. Radioactivity 57 (2001) 203–219
Fig. 1. Sketch of the RTMOD system as a tool for model evaluation.
meteorological services and environmental protection agencies, are involved in the RTMOD activity. The statistical procedure used to compare the model results follows three main guidelines: *
*
*
Spatial analysis: the spatial distribution of the two model results at a fixed time and surface concentration level. The quantitative evaluation is based on parameters such as the figure of merit in space (FMS), i.e. the space distribution of a specific level of concentration that is shared by two models at a given time. Temporal analysis: two model results at a fixed location are compared for all available times. Global analysis: all predictions from the two models are compared regardless of their time or space distribution. Standard quantitative statistical methods such as scatter diagrams and box-plots are used for the representation.
Most of the statistical analysis used in RTMOD is that adopted in the ATMES (Klug et al., 1992), ETEX (Girardi et al., 1998; Graziani, Klug, & Mosca, 1998a; Graziani, Galmarini, Grippa, & Klug, 1998b) and ATMES-2 (Mosca, Bianconi, Bellasio, Graziani, & Klug, 1998a) projects. A detailed presentation can be found in Mosca, Graziani, Klug, Bellasio, and Bianconi (1998b). In the present paper only results from the spatial and temporal analyses will be presented. Four RTMOD simulation exercises were performed between 1998 and 1999 (Table 2). In the first two experiments, only surface concentration values were calculated and compared, while, in the third exercise, dry and wet deposition predictions were also provided and evaluated. The so-called Algeciras experiment represents a variation with respect to the usual application of the procedure since it
207
S. Galmarini et al. / J. Environ. Radioactivity 57 (2001) 203–219 Table 1 Participants in the RTMOD experiments Institute
Country
IMP } Institute of Meteorology and Physics, University of Wien CMG KMI } Royal Institute of Meteorology of Belgium NIMH } National Institute for Meteorology and Hydrology CMC } Canadian Meteorological Centre CHI } Czeck Hydrological Institute DMI } Danish Meteorological Institute NERI } National Environment Research Institute/ Risoe National Laboratories/University of Cologne FMI } Finnish Meteorological Centre IPSN } French Institute for Nuclear Protection and Safety EDF } French Electricity Meteo France DWD } German Weather Service IIBR ANPA } National Agency for Environmental Protection CNR } National Research Council JAERI } Japan Atomic Energy Research Institute MRI } Meteorological Research Institute JMA } Japan Meteorological Agency RIVM } Government Institute for the Environment KNMI } Royal Dutch Meteorological Institute DNMI } Norwegian Meteorological Institute NIMH } National Institute of Meteorology and Hydrology MSC } Meteorological Synthesizing Centre – East SPA Typhoon SHMU } Slovak Hydrological and Meteorological Institute FOA } Defence Research Establishment SMHI } Swedish Meteorological and Hydrological Institute IMS } Swiss Meteorological Institute UKMO } Meteorological Office NOAA } National Oceanic and Atmospheric Administration ARAP } Group of Titan Research and Technology LLNL } Lawrence Livermore National Laboratories SAIC } Science Applications International Corporation SRS } Westinghouse Savannah River Laboratory
Austria Austria Belgium Bulgaria Canada Czeck Rep. Denmark Denmark Denmark Finland France France France Germany Israel Italy Italy Japan Japan Japan The Netherlands The Netherlands Norway Romania Russia Russia Slovak Rep. Sweden Sweden Switzerland UK USA USA USA USA USA
refers to the simulation of a real accidental release that occurred at an industrial installation in the south of Spain in 1998. The models participating in RTMOD were applied to the case several months after the occurrence of the release and in this case real measurements were also available for the comparison. Fig. 2 gives an example of a model intercomparison in time for two models participating in the Algeciras exercise. The concentration time series measured at a location in the south of France are compared with the model predictions.
208
S. Galmarini et al. / J. Environ. Radioactivity 57 (2001) 203–219
Table 2 The RTMOD exercises
Date: Release loc.: Lon: Lat: Rate: Start (UTC): Duration:
Experiment 01
Experiment 02
Experiment 03
Experiment 04
28 April 1998 Chernobyl (Ukraine) 30.1 E 51.4 N 10 g/s 0900 6h
9 July 1998 North of London (UK) 0 53 N 10 g/s 0900 6h
30 May 1999 Algeciras (Spain) 5.27 E 36.08 N 5.14E+08 Bq 0000
10 June 1999 Edinburgh (UK) 3.283 E 56 N 80 g/s 1200 3h
3. RTMOD as a system for ensemble dispersion forecast As described in the previous section, RTMOD provides in real time the results of a number of models and an estimate of the one-to-one model comparison. As more model results become available, the procedure compares them in pairs producing NðN21Þ=2 comparisons. From the point of view of a modeler, the direct comparison of his model results with other models on an individual basis is quite a useful exercise. In this case, knowing the characteristics of each model in terms of the approach to the representation of a physical process and to numerical simulation and in terms of the meteorological data used, one can try to explain the reasons for the differences between model couples. On the other hand, if all this information has to be used for emergency response applications, the consultation by a decision-maker of 21 22 couples of results becomes quite complicated. One should bear in mind that, given the present state of atmospheric modelling, agreement between models does not necessarily coincide with a correct forecast. Therefore the direct comparison of models in couples is complicated from a practical point of view and it does not guarantee that the agreement between two models corresponds to a correct representation of reality. The one-to-one comparison does not allow one to put together a complete picture of the dispersion forecast and to estimate the model degree of agreement. Therefore, determination of the areas of the domain that might be influenced by the contaminant dispersion becomes a difficult task. In order to overcome this problem and to exploit fully the potential provided by the use in real time of the results of up to 22 models, new parameters have to be developed. These parameters differ from those presented in the previous section in the sense that they treat the model ensemble rather than the single model or couples of models. The concept behind ensemble dispersion is that a large number of model results should provide a wider spectrum of possible scenarios including an estimate of the dispersion-forecast uncertainty.
S. Galmarini et al. / J. Environ. Radioactivity 57 (2001) 203–219
209
Fig. 2. Temporal evolution of the concentration of 137Cs calculated by two models participating in the RTMOD Algesiras exercise. The time evolution relates to two locations in the south of France that measured the concentration of the nuclide after the release. The measurements are indicated by the horizontal lines and refer to the diurnal or two-day average.
Having defined ck ðx; y; tÞ as the surface concentration value predicted by model k at location ðx; yÞ; at time t and M as the total number of model results available, the following ensemble dispersion parameters can be introduced.1 3.1. ENV-models envelope This can be defined as ENVðx; y; tÞ ¼ fðx; yÞ 2 D j 9kck ðx; y; tÞ > cT g: It represents the spatial distributions of all models at a given time above a threshold value cT within calculation domain D: The envelope gives a conservative indication of the area covered by the forecasted clouds and where at least one model predicts a concentration value above a given threshold. 1
It is clear that the ensemble dispersion parameters can be used for any predicted time and spacedependent scalar field other than concentration, such as for example dry or wet deposition.
210
S. Galmarini et al. / J. Environ. Radioactivity 57 (2001) 203–219
3.2. CCL } confidence in contamination level The confidence in contamination level (CCL) is a two-dimensional surface giving the normalized number of models predicting a value above a given threshold, cT ; at a given time: ( M 1 if ck ðx; y; tÞ > cT ; 100 X CCLðx; y; tÞ ¼ dk where dk ¼ M k¼1 0 otherwise: This allows the decision-maker to identify the models’ density distribution in space for a given concentration value, in other words the number of models that predict values above a threshold in a specific region of the domain. The CCL can be based on the assumption of equal importance of predictions or it could be weighted and biased towards some specific models. It can be seen that ENV represents a form of CCL in the sense that the latter equals the CCL whenever it defines the part of the domain covered by at least one model. An application of the ENV and CCL parameters can be made using the results of the first RTMOD experiment relating to the simulation of a fictitious release from Chernobyl as from Table 2. The models participating in this case are 20 from those presented in Table 1. Fig. 3 shows a comparison between the model envelope and the CCL calculated for 20 models and a fixed concentration level for the Chernobyl release. The three pictures relate to three time intervals of the evolution (T0 þ 12 h, T0 þ 24 h, T0 þ 36 h where T0 is the release start time). The model envelope (left side panels) indicates the area obtained as the union of the concentration given by all the models above the threshold. As explained, the corresponding figure on the right side represents the percentage of models giving the same spatial distribution for a specific concentration level. The comparison of the model envelope and the CCL provides us with a straightforward and readily available representation of the model uncertainty. The complementary character of the information provided by the model envelope and the CCL is immediately clear. From this result one can deduce that a decision based on the model envelope can be on the prudent side since it assumes that all model predictions have the same probability of occurrence. On the other hand, the CCL gives the important information relating to the quantitative estimate of the model agreement at different locations otherwise not present in the ENV distribution. The result shown in the figures relate to the treatment of all the models defining the ensemble. These two parameters can also be used in the case of sub-groups of the ensemble. In fact, once the model agreement has been quantified, one can wonder whether sub-groups would show a better agreement than the whole ensemble and therefore different groups of scenarios with a lower uncertainty might be identified. Clusters of models can therefore be obtained by grouping those showing similar patterns and significant agreement in terms of statistical indexes. A simple approach to the model grouping is the evaluation of the FMS among couples of models at different times as explained in the appendix. Within the set of models used, seven
S. Galmarini et al. / J. Environ. Radioactivity 57 (2001) 203–219
211
Fig. 3. Model envelope (left panels) and CCL (right panels) for the RTMOD Chernobyl exercise. The spatial distribution refers to prediction at T0 þ 12 h, T0 þ 24 h, T0 þ 36 h, where T0 is the release time.
were selected based on the FMS and put together. Another group was formed with the remaining ones. The new CCL plots for each of the two groups are compared with the original CCL in Figs. 4 and 5. As expected, while the CCL for the first group improves,
212
S. Galmarini et al. / J. Environ. Radioactivity 57 (2001) 203–219
Fig. 4. Left panels: CCL of the 20-model ensemble for the RTMOD Chernobyl exercise (same as Fig. 3). Right panels: CCL obtained with the results of 7 models out of the 20.
deterioration is evident for the second one. A subset of concise information can therefore be passed to the decision-maker for a more accurate determination of the countermeasures. It should be clear at this stage that the main aspect that we want to emphasize is the fact that the same parameters can be used in a revealing way also for subsets of models rather than promoting the grouping method as a way to proceed. More quantitative methods exist rather than that proposed for a better clustering of the results and these will be considered in future analysis.
S. Galmarini et al. / J. Environ. Radioactivity 57 (2001) 203–219
213
Fig. 5. Left panels: CCL of the 20-model ensemble for the RTMOD Chernobyl exercise (same as Fig. 3). Right panels: CCL obtained with the results of the remaining 13 models of the original 20.
3.3. CTA } confidence in time of arrival The CTA gives the fraction of models that, at a given time, have already transited over the domain. It is defined as ( M 1 if 9t4tjck ðx; y; tÞ > 0; 100 X CTAðx; y; tÞ ¼ dk where dk ¼ M k¼1 0 otherwise:
214
S. Galmarini et al. / J. Environ. Radioactivity 57 (2001) 203–219
Fig. 6 gives an example of the CTA plots at different times for the Chernobyl release. The darkest area is the one where more than 80% of the models predict that the cloud has already transited. As in the case of the CCL, some models can be clustered and common behavior (and thus higher agreement than the whole set of models) can be obtained. 3.4. MCT } maximum concentration trend The maximum concentration trend (MCT) at a fixed location is defined as MCTðx; y; tÞ ¼ Max ck ðx; y; tÞ: 8k
It is a time series at a fixed location of the maximum concentration values calculated by any of the models of the ensemble. The MCT thus gives a conservative evaluation since it represents the worst scenario within the model ensemble for a given time interval and location. Fig. 7 gives an example of the MCT plot that the RTMOD system produces.
4. RTMOD as a system for decision support When a nuclear accident occurs, the decision-maker is interested to know where countermeasures should be taken, when they must be taken first and how serious the situation is expected to be in those locations. An alternative way to satisfy these points would be to use a single model and to run it over a variety of scenarios. By means of a probabilistic approach (e.g. Monte Carlo techniques), some of the input variables are randomly sampled from the distributions they belong to, a single model would be repeatedly run using these input values, and a distribution of the results would be obtained. However, this method has several drawbacks: * *
*
it requires long simulation times; it could be carried out with a poorly performing model and the results for all the scenarios will be biased towards the model considered; the number of parameters to vary is very large.
The RTMOD approach, on the contrary, exploits the intrinsic differences that exist among different models. Obviously, since several model results are considered, the prediction could be biased towards poorly performing ones. It is therefore important that the modelling systems admitted to the ensemble have been extensively tested against measurements. The parameters presented in this paper provide a clear answer to the decisionmaker’s requests on where, when and how. Namely: *
The envelope describes the area where all available models predict the presence of the contaminant, regardless of the agreement among the models. At the same time, the CCL associates with the space distribution of the model agreement.
S. Galmarini et al. / J. Environ. Radioactivity 57 (2001) 203–219
215
Fig. 6. CTA calculated for the 20-model ensemble (left panels) and the 7 model subgroup (right panels).
216
S. Galmarini et al. / J. Environ. Radioactivity 57 (2001) 203–219
Fig. 7. Time series of MCT for the city of Edinburgh obtained during the RTMOD Edinburgh release. Each bar in the histogram represents the maximum concentration level among all model results for that specific location and time interval.
*
*
The CTA indicates when actions must be taken first. If the decision-maker has resources available, then actions can be taken wherever CTA is greater than a threshold value which can be set to be at a reasonably low level. Otherwise the decision-maker can establish a threshold to determine the time at which to start the interventions. In this way resources can be saved: interventions can be planned in well-defined areas and at appropriate times. The seriousness of the situation (how) can be estimated through the MCT. Although it is a conservative indicator, it provides the decision-maker with an estimate of the worst predictable values at different times and specific locations. As for the CCL, it would be possible to take advantage of the ensemble available and to calculate a histogram for the Confidence in Concentration Trend (CCT), i.e. the cumulative distribution of predicted values against the number of models available.
The role of the decision-maker within RTMOD is depicted in Fig. 8. The picture summarizes how the RTMOD system acts as the center point for the community of modelers as far as model intercomparison and improvement are concerned but also decides who can access the system in the case of real emergencies. It is important to note that RTMOD takes advantage of forecasts that will be produced in any case at national level by specific institutions. The results of the ensemble dispersion forecasting are therefore extra information that can be used at national level as well
S. Galmarini et al. / J. Environ. Radioactivity 57 (2001) 203–219
217
Fig. 8. Sketch of the RTMOD system as a tool for decision support.
as super-national level for decision-making applications. The system can of course be conceived also as a training facility where by decision-makers and modelers can be trained on the use of information.
5. Conclusions The RTMOD system and its application to fictitious radioactive releases have been presented. The system, operational at JRC-Ispra, has been used in the last two years as a tool for model intercomparison, proving to be useful to the long-range atmospheric dispersion community around the world. The fact that the system allows the collection through the world wide web of a large number of model results in real time and their comparison gives the possibility to devise a set of parameters to produce the so called ensemble dispersion forecast. This technique aims at producing estimates of the uncertainty on the forecast and to identify, in a brief and concise form, the areas of the domain that are more likely to be influenced by the dispersion and the degree of reliability of this information. An example of a set of such parameters has been given and applied to the dispersion cases of the RTMOD project. The scope of these parameters is to present concise information, with confidence evaluation, to be used for decision support in
218
S. Galmarini et al. / J. Environ. Radioactivity 57 (2001) 203–219
case of nuclear emergency (for example). In this respect, the implementation of these parameters in the system can transform RTMOD into a tool for decision support. In the event of a nuclear accident or in general the dispersion in the atmosphere of radioactive material, the results of the models and the treatment of the ensemble can be produced as part of the RTMOD system and accessed by decision-makers. RTMOD can be considered as a prototype for a decision support system for nuclear emergencies. Also, confidence in the choices comes from the ensemble of data used, compared to just one single prediction. Apart from the purposes of operational support to decision-making, the RTMOD system has several other advantages. For the modelers, it is a place where their own mathematical tools can be checked for systematic errors (e.g. scale factors), or to identify general tendencies to under- or over-estimation with respect to other models. The system has also didactic purposes: staff can be trained to be ready and prompt in responding to emergencies. Also, the international dimension of the system, together with the participation of many institutions, is a motivation and stimulation for producing precise information in a short time.
Acknowledgements The authors would like to thank the modelling groups of the RTMOD community (Table 1) for their active and important contributions to the successful completion of the project.
Appendix There are several criteria according to which the model results of the RTMOD ensemble could be grouped that can vary considerably in sophistication. The easiest and most straightforward way is by analyzing the matrix of FMS that gives, for specific time intervals of the cloud evolution and concentration level, the normalized overlapping between pairs of models (1=total overlapping, 0=no intersection between the two concentration patterns). By looking at the matrix of FMS, one can deduce the relative agreement of models. Large values of the FMS give a quantitative indication of the relative agreement of model couples. By imposing a minimum level of overlapping, it is possible to identify groups of model with similar behaviors. The method does not give information that allows one to conclude that some model groups behave better than others, but at least the model grouping allows one to identify distinct scenarios. As mentioned in the text, this analysis has allowed the identification of a group of 7 models with a consistent agreement in FMS and a second group comprising the remaining 13 models.
S. Galmarini et al. / J. Environ. Radioactivity 57 (2001) 203–219
219
References Bellasio, R., Bianconi, R., Graziani, G., & Mosca, S. (1999). RTMOD: An Internet based system to analyse the predictions of long-range atmospheric dispersion models. Computers and Geosciences, 25(7), 819–833. Girardi, F., Graziani, G., van Veltzen, D., Galmarini, S., Mosca, S., Bianconi, R., Bellasio, R., Klug, W., & Fraser, G. (Eds.). (1998). ETEX } The European Tracer Experiment (106pp.). Office for Official Publications of the European Communities, Luxembourg. ISBN 92-828-5007-2. Graziani, G., Klug, W., & Mosca, S. (1998a). Real-time long-range dispersion model evaluation of the ETEX first release. EUR 17754 EN, Office for Official Publications of the European Communities, Luxembourg (216pp.). ISBN 92-828-3657-6. Graziani, G., Galmarini, S., Grippa, G., & Klug, W. (1998b). Real-time long-range dispersion model evaluation of the ETEX second release. EUR 17755 EN, Office for Official Publications of the European Communities, Luxembourg (252pp.). ISBN 92-828-3656-8. Klug, W., Graziani, G., Grippa, G., Pierce, D., & Tassone, C. (1992). Evaluation of long-range atmospheric models using environmental radioactivity data from the Chernobyl accident: ATMES Report (366pp.). Amsterdam: Elsevier, ISBN 1-85166-766-0. Mosca, S., Bianconi, R., Bellasio, R., Graziani, G., & W. Klug (1998a). ATMES II } Evaluation of longrange dispersion models using data of the 1st ETEX release. EUR 17756 EN, Office for Official Publications of the European Communities, Luxembourg, ISBN 92-828-3655-X (458pp.). Mosca, S., Graziani, G., Klug, W., Bellasio, R., & Bianconi, R. (1998b). A statistical methodology for the evaluation of long-range dispersion models: an application to the ETEX exercise. Atmospheric Environment, 32(24), 4307–4324.
Forecasting the consequences of accidental releases of radionuclides in the atmosphere from ensemble dispersion modelling S. Galmarinia,*, R. Bianconib, R. Bellasiob, G. Graziania a
b
Joint Research Center, Environment Institute, TP 321, 21020 Ispra (VA), Italy ENVIROWARE srl, Centro Direzionale Colleoni, Palazzo Andromeda 1, I-20041 Agrate Brianza (MI), Italy Received 15 June 2000; received in revised form 15 December 2000; accepted 5 January 2001
Abstract The RTMOD system is presented as a tool for the intercomparison of long-range dispersion models as well as a system for support of decision making. RTMOD is an internet-based procedure that collects the results of more than 20 models used around the world to predict the transport and deposition of radioactive releases in the atmosphere. It allows the real-time acquisition of model results and their intercomparison. Taking advantage of the availability of several model results, the system can also be used as a tool to support decision making in case of emergency. The new concept of ensemble dispersion modelling is introduced which is the basis for the decision-making application of RTMOD. New statistical parameters are presented that allow gathering the results of several models to produce a single dispersion forecast. The devised parameters are presented and tested on the results of RTMOD exercises. # 2001 Elsevier Science Ltd. All rights reserved. Keywords: Long-range transport and dispersion; Model intercomparison; Ensemble dispersion forecasting; Support to decision making
1. Introduction In the event of the accidental release of a harmful material to the atmosphere and its transport over large distances, decision-making relies on the support of long-range *Corresponding author. E-mail address: [email protected] (S. Galmarini). 0265-931X/01/$ - see front matter # 2001 Elsevier Science Ltd. All rights reserved. PII: S 0 2 6 5 - 9 3 1 X ( 0 1 ) 0 0 0 1 7 - 0
204
S. Galmarini et al. / J. Environ. Radioactivity 57 (2001) 203–219
transport and dispersion models to forecast the locations potentially affected by the passage of the pollutant. The Chernobyl accident definitely outlined our ill preparedness with respect to the above-mentioned prediction capacity and the compelling necessity for an improvement of the modelling tools. Since then, several institutes around the world have invested effort in the development and implementation of long-range atmospheric transport models for real-time forecasting of atmospheric dispersion. The approach to long-range dispersion modelling is not unique and in most cases has to be based on the delicate balance that exists between having a reliable model and a tool that can be used in real time. The various approaches that have been developed and which are currently operational in meteorological services or national protection agencies are the source of an intrinsic diversity in model results. An important component that is responsible for the differences in the model performance is, for example, the quality and the completeness of the meteorological data (mainly atmospheric circulation, precipitation and stability). The meteorological input data are the results of routine operational weather forecasting and thus the result of numerical weather predictions (NWP). Each of the institutes using a long-range transport model has access to different NWP-data produced by different weather services. An intrinsic difference is reflected in the results given the different origins of the data (e.g. Klug, Graziani, Grippa, Pierce, & Tassone, 1992). One should therefore start with the consideration that different models can inevitably produce different results due to the simple fact that some of them (or the data that they use) may give more importance and attention to different aspects of the physical processes. This also implies that none of them is in principle wrong and, most of all, that all of them should be considered for decision-making support. In order to estimate the effect of these intrinsic differences, ad-hoc tracer experiments were in the past designed and carried out. The most recent of these was the European Tracer Experiment (ETEX) that took place in 1994 and allowed the comparison of several types of models with experimental evidence (Girardi et al., 1998). One of the conclusions of the experiment was that, although modelling has greatly advanced in the last few decades, several aspects still need to be improved in order to bring the various approaches to a state of reliable maturity. Moreover it was pointed out that model intercomparison and the evaluation of the results should be performed on a regular basis in order to promote constant improvement of the quality of the tools and advancements in its development. An activity that is presently ongoing and aims at improvement of dispersion model performance for nuclear emergency application is RTMOD (Real Time Model Evaluation (Bellasio, Bianconi, Graziani, & Mosca, 1999). Given the difficulty in performing tracer experiments and their cost (especially over the long range), one way to test modelling approaches to atmospheric dispersion is in fact model intercomparison. RTMOD is a system that gathers and compares pairs of results of models operated in different national institutions and applied to a predefined case. The modelers have therefore the possibility to check for the presence of systematic errors or biases in the prediction over a range of meteorological situations.
S. Galmarini et al. / J. Environ. Radioactivity 57 (2001) 203–219
205
RTMOD, other than being instrumental to the above-mentioned scope, is also useful in decision-making. Being an internet-based system that works in real time, it provides the possibility to obtain, in the event of an emergency, the results of a number of models that are homogeneous as far as the spatial and temporal representation of the forecast are concerned. A decision-maker has therefore the possibility to access the system and consult several model results, thus avoiding the problems connected with relying on a single or few modelling approaches and to homogenize results presented in different forms. One of the aspects to be taken care of at this point is the possibility to gather all the information provided by an ensemble of models in a quantitative manner so that few statistical parameters can be looked at to decide what action to take. A sort of ensemble dispersion forecasting can be conducted in a fashion similar to meteorological ensemble forecasting. The difference lies in the fact that, rather than applying one model to a series of slightly different initial conditions, in this case different models are applied to the same case, and the results presented with few and appropriate statistical parameters. This paper presents some of these parameters and an application of this new technique.
2. The RTMOD system for model intercomparison Briefly, RTMOD is an internet-based system that allows real time model intercomparison of long-range transport and dispersion models. The RTMOD web site is located on a server at the European Commission-Joint Research Center at Ispra (Italy) and can be accessed upon authentication by a community of modelers distributed all around the world. Upon notification of a release (fictitious or real), the modelers can find on the web site information on the release characteristics and can run their dispersion models. All models are run according to their usual setup but each of them has to produce a dispersion forecast over a prescribed grid and according to a pre-defined format protocol. After the completion of the simulation, the modeler uploads the results to the RTMOD web site and an automated statistical analysis compares them with the results of all the other models already available. A detailed description of the system (http://rtmod.ei.jrc.it/rtmod) is given in Bellasio et al. (1999). Numerical and graphical results of the intercomparison are published as web pages and are readily available to the modelers. Users have the opportunity to see the comparison of their model with the others, identifying discrepancies and similarities with one or more of the other models. The real advantage of the system is that each modeler can access the results of the intercomparison within hours of the simulation and uploading procedures. This aspect constitutes a great advantage for dispersion modelling since the modeler immediately has feedback on the quality of his simulation with respect to the others. Fig. 1 gives a representation of RTMOD in the context of the notification of the national services, the collection of the model results and the model intercomparison. At present, 22 models (Table 1), operated by
206
S. Galmarini et al. / J. Environ. Radioactivity 57 (2001) 203–219
Fig. 1. Sketch of the RTMOD system as a tool for model evaluation.
meteorological services and environmental protection agencies, are involved in the RTMOD activity. The statistical procedure used to compare the model results follows three main guidelines: *
*
*
Spatial analysis: the spatial distribution of the two model results at a fixed time and surface concentration level. The quantitative evaluation is based on parameters such as the figure of merit in space (FMS), i.e. the space distribution of a specific level of concentration that is shared by two models at a given time. Temporal analysis: two model results at a fixed location are compared for all available times. Global analysis: all predictions from the two models are compared regardless of their time or space distribution. Standard quantitative statistical methods such as scatter diagrams and box-plots are used for the representation.
Most of the statistical analysis used in RTMOD is that adopted in the ATMES (Klug et al., 1992), ETEX (Girardi et al., 1998; Graziani, Klug, & Mosca, 1998a; Graziani, Galmarini, Grippa, & Klug, 1998b) and ATMES-2 (Mosca, Bianconi, Bellasio, Graziani, & Klug, 1998a) projects. A detailed presentation can be found in Mosca, Graziani, Klug, Bellasio, and Bianconi (1998b). In the present paper only results from the spatial and temporal analyses will be presented. Four RTMOD simulation exercises were performed between 1998 and 1999 (Table 2). In the first two experiments, only surface concentration values were calculated and compared, while, in the third exercise, dry and wet deposition predictions were also provided and evaluated. The so-called Algeciras experiment represents a variation with respect to the usual application of the procedure since it
207
S. Galmarini et al. / J. Environ. Radioactivity 57 (2001) 203–219 Table 1 Participants in the RTMOD experiments Institute
Country
IMP } Institute of Meteorology and Physics, University of Wien CMG KMI } Royal Institute of Meteorology of Belgium NIMH } National Institute for Meteorology and Hydrology CMC } Canadian Meteorological Centre CHI } Czeck Hydrological Institute DMI } Danish Meteorological Institute NERI } National Environment Research Institute/ Risoe National Laboratories/University of Cologne FMI } Finnish Meteorological Centre IPSN } French Institute for Nuclear Protection and Safety EDF } French Electricity Meteo France DWD } German Weather Service IIBR ANPA } National Agency for Environmental Protection CNR } National Research Council JAERI } Japan Atomic Energy Research Institute MRI } Meteorological Research Institute JMA } Japan Meteorological Agency RIVM } Government Institute for the Environment KNMI } Royal Dutch Meteorological Institute DNMI } Norwegian Meteorological Institute NIMH } National Institute of Meteorology and Hydrology MSC } Meteorological Synthesizing Centre – East SPA Typhoon SHMU } Slovak Hydrological and Meteorological Institute FOA } Defence Research Establishment SMHI } Swedish Meteorological and Hydrological Institute IMS } Swiss Meteorological Institute UKMO } Meteorological Office NOAA } National Oceanic and Atmospheric Administration ARAP } Group of Titan Research and Technology LLNL } Lawrence Livermore National Laboratories SAIC } Science Applications International Corporation SRS } Westinghouse Savannah River Laboratory
Austria Austria Belgium Bulgaria Canada Czeck Rep. Denmark Denmark Denmark Finland France France France Germany Israel Italy Italy Japan Japan Japan The Netherlands The Netherlands Norway Romania Russia Russia Slovak Rep. Sweden Sweden Switzerland UK USA USA USA USA USA
refers to the simulation of a real accidental release that occurred at an industrial installation in the south of Spain in 1998. The models participating in RTMOD were applied to the case several months after the occurrence of the release and in this case real measurements were also available for the comparison. Fig. 2 gives an example of a model intercomparison in time for two models participating in the Algeciras exercise. The concentration time series measured at a location in the south of France are compared with the model predictions.
208
S. Galmarini et al. / J. Environ. Radioactivity 57 (2001) 203–219
Table 2 The RTMOD exercises
Date: Release loc.: Lon: Lat: Rate: Start (UTC): Duration:
Experiment 01
Experiment 02
Experiment 03
Experiment 04
28 April 1998 Chernobyl (Ukraine) 30.1 E 51.4 N 10 g/s 0900 6h
9 July 1998 North of London (UK) 0 53 N 10 g/s 0900 6h
30 May 1999 Algeciras (Spain) 5.27 E 36.08 N 5.14E+08 Bq 0000
10 June 1999 Edinburgh (UK) 3.283 E 56 N 80 g/s 1200 3h
3. RTMOD as a system for ensemble dispersion forecast As described in the previous section, RTMOD provides in real time the results of a number of models and an estimate of the one-to-one model comparison. As more model results become available, the procedure compares them in pairs producing NðN21Þ=2 comparisons. From the point of view of a modeler, the direct comparison of his model results with other models on an individual basis is quite a useful exercise. In this case, knowing the characteristics of each model in terms of the approach to the representation of a physical process and to numerical simulation and in terms of the meteorological data used, one can try to explain the reasons for the differences between model couples. On the other hand, if all this information has to be used for emergency response applications, the consultation by a decision-maker of 21 22 couples of results becomes quite complicated. One should bear in mind that, given the present state of atmospheric modelling, agreement between models does not necessarily coincide with a correct forecast. Therefore the direct comparison of models in couples is complicated from a practical point of view and it does not guarantee that the agreement between two models corresponds to a correct representation of reality. The one-to-one comparison does not allow one to put together a complete picture of the dispersion forecast and to estimate the model degree of agreement. Therefore, determination of the areas of the domain that might be influenced by the contaminant dispersion becomes a difficult task. In order to overcome this problem and to exploit fully the potential provided by the use in real time of the results of up to 22 models, new parameters have to be developed. These parameters differ from those presented in the previous section in the sense that they treat the model ensemble rather than the single model or couples of models. The concept behind ensemble dispersion is that a large number of model results should provide a wider spectrum of possible scenarios including an estimate of the dispersion-forecast uncertainty.
S. Galmarini et al. / J. Environ. Radioactivity 57 (2001) 203–219
209
Fig. 2. Temporal evolution of the concentration of 137Cs calculated by two models participating in the RTMOD Algesiras exercise. The time evolution relates to two locations in the south of France that measured the concentration of the nuclide after the release. The measurements are indicated by the horizontal lines and refer to the diurnal or two-day average.
Having defined ck ðx; y; tÞ as the surface concentration value predicted by model k at location ðx; yÞ; at time t and M as the total number of model results available, the following ensemble dispersion parameters can be introduced.1 3.1. ENV-models envelope This can be defined as ENVðx; y; tÞ ¼ fðx; yÞ 2 D j 9kck ðx; y; tÞ > cT g: It represents the spatial distributions of all models at a given time above a threshold value cT within calculation domain D: The envelope gives a conservative indication of the area covered by the forecasted clouds and where at least one model predicts a concentration value above a given threshold. 1
It is clear that the ensemble dispersion parameters can be used for any predicted time and spacedependent scalar field other than concentration, such as for example dry or wet deposition.
210
S. Galmarini et al. / J. Environ. Radioactivity 57 (2001) 203–219
3.2. CCL } confidence in contamination level The confidence in contamination level (CCL) is a two-dimensional surface giving the normalized number of models predicting a value above a given threshold, cT ; at a given time: ( M 1 if ck ðx; y; tÞ > cT ; 100 X CCLðx; y; tÞ ¼ dk where dk ¼ M k¼1 0 otherwise: This allows the decision-maker to identify the models’ density distribution in space for a given concentration value, in other words the number of models that predict values above a threshold in a specific region of the domain. The CCL can be based on the assumption of equal importance of predictions or it could be weighted and biased towards some specific models. It can be seen that ENV represents a form of CCL in the sense that the latter equals the CCL whenever it defines the part of the domain covered by at least one model. An application of the ENV and CCL parameters can be made using the results of the first RTMOD experiment relating to the simulation of a fictitious release from Chernobyl as from Table 2. The models participating in this case are 20 from those presented in Table 1. Fig. 3 shows a comparison between the model envelope and the CCL calculated for 20 models and a fixed concentration level for the Chernobyl release. The three pictures relate to three time intervals of the evolution (T0 þ 12 h, T0 þ 24 h, T0 þ 36 h where T0 is the release start time). The model envelope (left side panels) indicates the area obtained as the union of the concentration given by all the models above the threshold. As explained, the corresponding figure on the right side represents the percentage of models giving the same spatial distribution for a specific concentration level. The comparison of the model envelope and the CCL provides us with a straightforward and readily available representation of the model uncertainty. The complementary character of the information provided by the model envelope and the CCL is immediately clear. From this result one can deduce that a decision based on the model envelope can be on the prudent side since it assumes that all model predictions have the same probability of occurrence. On the other hand, the CCL gives the important information relating to the quantitative estimate of the model agreement at different locations otherwise not present in the ENV distribution. The result shown in the figures relate to the treatment of all the models defining the ensemble. These two parameters can also be used in the case of sub-groups of the ensemble. In fact, once the model agreement has been quantified, one can wonder whether sub-groups would show a better agreement than the whole ensemble and therefore different groups of scenarios with a lower uncertainty might be identified. Clusters of models can therefore be obtained by grouping those showing similar patterns and significant agreement in terms of statistical indexes. A simple approach to the model grouping is the evaluation of the FMS among couples of models at different times as explained in the appendix. Within the set of models used, seven
S. Galmarini et al. / J. Environ. Radioactivity 57 (2001) 203–219
211
Fig. 3. Model envelope (left panels) and CCL (right panels) for the RTMOD Chernobyl exercise. The spatial distribution refers to prediction at T0 þ 12 h, T0 þ 24 h, T0 þ 36 h, where T0 is the release time.
were selected based on the FMS and put together. Another group was formed with the remaining ones. The new CCL plots for each of the two groups are compared with the original CCL in Figs. 4 and 5. As expected, while the CCL for the first group improves,
212
S. Galmarini et al. / J. Environ. Radioactivity 57 (2001) 203–219
Fig. 4. Left panels: CCL of the 20-model ensemble for the RTMOD Chernobyl exercise (same as Fig. 3). Right panels: CCL obtained with the results of 7 models out of the 20.
deterioration is evident for the second one. A subset of concise information can therefore be passed to the decision-maker for a more accurate determination of the countermeasures. It should be clear at this stage that the main aspect that we want to emphasize is the fact that the same parameters can be used in a revealing way also for subsets of models rather than promoting the grouping method as a way to proceed. More quantitative methods exist rather than that proposed for a better clustering of the results and these will be considered in future analysis.
S. Galmarini et al. / J. Environ. Radioactivity 57 (2001) 203–219
213
Fig. 5. Left panels: CCL of the 20-model ensemble for the RTMOD Chernobyl exercise (same as Fig. 3). Right panels: CCL obtained with the results of the remaining 13 models of the original 20.
3.3. CTA } confidence in time of arrival The CTA gives the fraction of models that, at a given time, have already transited over the domain. It is defined as ( M 1 if 9t4tjck ðx; y; tÞ > 0; 100 X CTAðx; y; tÞ ¼ dk where dk ¼ M k¼1 0 otherwise:
214
S. Galmarini et al. / J. Environ. Radioactivity 57 (2001) 203–219
Fig. 6 gives an example of the CTA plots at different times for the Chernobyl release. The darkest area is the one where more than 80% of the models predict that the cloud has already transited. As in the case of the CCL, some models can be clustered and common behavior (and thus higher agreement than the whole set of models) can be obtained. 3.4. MCT } maximum concentration trend The maximum concentration trend (MCT) at a fixed location is defined as MCTðx; y; tÞ ¼ Max ck ðx; y; tÞ: 8k
It is a time series at a fixed location of the maximum concentration values calculated by any of the models of the ensemble. The MCT thus gives a conservative evaluation since it represents the worst scenario within the model ensemble for a given time interval and location. Fig. 7 gives an example of the MCT plot that the RTMOD system produces.
4. RTMOD as a system for decision support When a nuclear accident occurs, the decision-maker is interested to know where countermeasures should be taken, when they must be taken first and how serious the situation is expected to be in those locations. An alternative way to satisfy these points would be to use a single model and to run it over a variety of scenarios. By means of a probabilistic approach (e.g. Monte Carlo techniques), some of the input variables are randomly sampled from the distributions they belong to, a single model would be repeatedly run using these input values, and a distribution of the results would be obtained. However, this method has several drawbacks: * *
*
it requires long simulation times; it could be carried out with a poorly performing model and the results for all the scenarios will be biased towards the model considered; the number of parameters to vary is very large.
The RTMOD approach, on the contrary, exploits the intrinsic differences that exist among different models. Obviously, since several model results are considered, the prediction could be biased towards poorly performing ones. It is therefore important that the modelling systems admitted to the ensemble have been extensively tested against measurements. The parameters presented in this paper provide a clear answer to the decisionmaker’s requests on where, when and how. Namely: *
The envelope describes the area where all available models predict the presence of the contaminant, regardless of the agreement among the models. At the same time, the CCL associates with the space distribution of the model agreement.
S. Galmarini et al. / J. Environ. Radioactivity 57 (2001) 203–219
215
Fig. 6. CTA calculated for the 20-model ensemble (left panels) and the 7 model subgroup (right panels).
216
S. Galmarini et al. / J. Environ. Radioactivity 57 (2001) 203–219
Fig. 7. Time series of MCT for the city of Edinburgh obtained during the RTMOD Edinburgh release. Each bar in the histogram represents the maximum concentration level among all model results for that specific location and time interval.
*
*
The CTA indicates when actions must be taken first. If the decision-maker has resources available, then actions can be taken wherever CTA is greater than a threshold value which can be set to be at a reasonably low level. Otherwise the decision-maker can establish a threshold to determine the time at which to start the interventions. In this way resources can be saved: interventions can be planned in well-defined areas and at appropriate times. The seriousness of the situation (how) can be estimated through the MCT. Although it is a conservative indicator, it provides the decision-maker with an estimate of the worst predictable values at different times and specific locations. As for the CCL, it would be possible to take advantage of the ensemble available and to calculate a histogram for the Confidence in Concentration Trend (CCT), i.e. the cumulative distribution of predicted values against the number of models available.
The role of the decision-maker within RTMOD is depicted in Fig. 8. The picture summarizes how the RTMOD system acts as the center point for the community of modelers as far as model intercomparison and improvement are concerned but also decides who can access the system in the case of real emergencies. It is important to note that RTMOD takes advantage of forecasts that will be produced in any case at national level by specific institutions. The results of the ensemble dispersion forecasting are therefore extra information that can be used at national level as well
S. Galmarini et al. / J. Environ. Radioactivity 57 (2001) 203–219
217
Fig. 8. Sketch of the RTMOD system as a tool for decision support.
as super-national level for decision-making applications. The system can of course be conceived also as a training facility where by decision-makers and modelers can be trained on the use of information.
5. Conclusions The RTMOD system and its application to fictitious radioactive releases have been presented. The system, operational at JRC-Ispra, has been used in the last two years as a tool for model intercomparison, proving to be useful to the long-range atmospheric dispersion community around the world. The fact that the system allows the collection through the world wide web of a large number of model results in real time and their comparison gives the possibility to devise a set of parameters to produce the so called ensemble dispersion forecast. This technique aims at producing estimates of the uncertainty on the forecast and to identify, in a brief and concise form, the areas of the domain that are more likely to be influenced by the dispersion and the degree of reliability of this information. An example of a set of such parameters has been given and applied to the dispersion cases of the RTMOD project. The scope of these parameters is to present concise information, with confidence evaluation, to be used for decision support in
218
S. Galmarini et al. / J. Environ. Radioactivity 57 (2001) 203–219
case of nuclear emergency (for example). In this respect, the implementation of these parameters in the system can transform RTMOD into a tool for decision support. In the event of a nuclear accident or in general the dispersion in the atmosphere of radioactive material, the results of the models and the treatment of the ensemble can be produced as part of the RTMOD system and accessed by decision-makers. RTMOD can be considered as a prototype for a decision support system for nuclear emergencies. Also, confidence in the choices comes from the ensemble of data used, compared to just one single prediction. Apart from the purposes of operational support to decision-making, the RTMOD system has several other advantages. For the modelers, it is a place where their own mathematical tools can be checked for systematic errors (e.g. scale factors), or to identify general tendencies to under- or over-estimation with respect to other models. The system has also didactic purposes: staff can be trained to be ready and prompt in responding to emergencies. Also, the international dimension of the system, together with the participation of many institutions, is a motivation and stimulation for producing precise information in a short time.
Acknowledgements The authors would like to thank the modelling groups of the RTMOD community (Table 1) for their active and important contributions to the successful completion of the project.
Appendix There are several criteria according to which the model results of the RTMOD ensemble could be grouped that can vary considerably in sophistication. The easiest and most straightforward way is by analyzing the matrix of FMS that gives, for specific time intervals of the cloud evolution and concentration level, the normalized overlapping between pairs of models (1=total overlapping, 0=no intersection between the two concentration patterns). By looking at the matrix of FMS, one can deduce the relative agreement of models. Large values of the FMS give a quantitative indication of the relative agreement of model couples. By imposing a minimum level of overlapping, it is possible to identify groups of model with similar behaviors. The method does not give information that allows one to conclude that some model groups behave better than others, but at least the model grouping allows one to identify distinct scenarios. As mentioned in the text, this analysis has allowed the identification of a group of 7 models with a consistent agreement in FMS and a second group comprising the remaining 13 models.
S. Galmarini et al. / J. Environ. Radioactivity 57 (2001) 203–219
219
References Bellasio, R., Bianconi, R., Graziani, G., & Mosca, S. (1999). RTMOD: An Internet based system to analyse the predictions of long-range atmospheric dispersion models. Computers and Geosciences, 25(7), 819–833. Girardi, F., Graziani, G., van Veltzen, D., Galmarini, S., Mosca, S., Bianconi, R., Bellasio, R., Klug, W., & Fraser, G. (Eds.). (1998). ETEX } The European Tracer Experiment (106pp.). Office for Official Publications of the European Communities, Luxembourg. ISBN 92-828-5007-2. Graziani, G., Klug, W., & Mosca, S. (1998a). Real-time long-range dispersion model evaluation of the ETEX first release. EUR 17754 EN, Office for Official Publications of the European Communities, Luxembourg (216pp.). ISBN 92-828-3657-6. Graziani, G., Galmarini, S., Grippa, G., & Klug, W. (1998b). Real-time long-range dispersion model evaluation of the ETEX second release. EUR 17755 EN, Office for Official Publications of the European Communities, Luxembourg (252pp.). ISBN 92-828-3656-8. Klug, W., Graziani, G., Grippa, G., Pierce, D., & Tassone, C. (1992). Evaluation of long-range atmospheric models using environmental radioactivity data from the Chernobyl accident: ATMES Report (366pp.). Amsterdam: Elsevier, ISBN 1-85166-766-0. Mosca, S., Bianconi, R., Bellasio, R., Graziani, G., & W. Klug (1998a). ATMES II } Evaluation of longrange dispersion models using data of the 1st ETEX release. EUR 17756 EN, Office for Official Publications of the European Communities, Luxembourg, ISBN 92-828-3655-X (458pp.). Mosca, S., Graziani, G., Klug, W., Bellasio, R., & Bianconi, R. (1998b). A statistical methodology for the evaluation of long-range dispersion models: an application to the ETEX exercise. Atmospheric Environment, 32(24), 4307–4324.