Hourly ozone prediction for a 24-h horizon using neural networks

Environmental Modelling & Software 23 (2008) 1407–1421

Contents lists available at ScienceDirect

Environmental Modelling & Software journal homepage: www.elsevier.com/locate/envsoft

Hourly ozone prediction for a 24-h horizon using neural networks Adriana Coman*, Anda Ionescu, Yves Candau CERTES, University of Paris XII, 61 avenue du Ge´ne´ral de Gaulle, F-94 010 Cre´teil Cedex, France

a r t i c l e i n f o

a b s t r a c t

Article history: Received 5 March 2008 Accepted 7 April 2008 Available online 20 May 2008

This study is an attempt to verify the presence of non-linear dynamics in the ozone time series by testing a ‘‘dynamic’’ model, evaluated versus a ‘‘static’’ one, in the context of predicting hourly ozone concentrations, one-day ahead. The ‘‘dynamic’’ model uses a recursive structure involving a cascade of 24 multilayer perceptrons (MLP) arranged so that each MLP feeds the next one. The ‘‘static’’ model is a classical single MLP with 24 outputs. For both models, the inputs consist of ozone and of exogenous variables: past 24-h values of meteorological parameters and of NO2; concerning the ozone inputs, the ‘‘static’’ model uses only the 24 past measurements, while the ‘‘dynamic’’ one uses, also, the previously forecast ozone concentrations, as soon as they are predicted by the model. The outputs are, for both configurations, ozone concentrations for a 24-h horizon. The performance of the two models was evaluated for an urban and a rural site, in the greater Paris. Globally, the results indicate a rather good applicability of these models for a short-term prediction of ozone. We notice that the results of the recursive model were comparable with those obtained via the ‘‘static’’ one; thus, we can conclude that there is no evidence of non-linear dynamics in the ozone time series under study. Ó 2008 Elsevier Ltd. All rights reserved.

Keywords: Ozone Short-term prediction Non-linear dynamics Neural networks Multilayer perceptron Greater Paris

1. Introduction Predicting atmospheric pollutant concentrations in the greater Paris is a true challenge but, at the same time, a necessary action allowing local authorities to take preventive measures, such as traffic limitations or public information. The strong relationship between atmospheric pollution and human health was emphasised by many studies conducted for our area of interest or for others, e.g. the European project APHEIS,1 which emphasised the pollution effects on human health in 26 cities from 12 European countries. Another good example is the regional project ERPURS,2 conducted in the greater Paris, with two declared goals: firstly to characterise the short-term relationship between air pollution and the population health; secondly, to determine the feasibility of obtaining, from multiple sources and on a regular basis, the data needed to create a surveillance system able to monitor this relationship (Medina et al., 1997). According to Grimfeld,3 ‘‘there are enough studies revealing the relationship between air pollution and health. It is known nowadays that air pollution increases asthma [.]. One must not focus only on the

* Corresponding author. Tel.: þ33 1 45 17 18 29; fax: þ33 1 45 17 65 51. E-mail address: [email protected] (A. Coman). 1 APHEIS: Air Pollution and Health: an European Information System, www. apheis.net. 2 ERPURS: Monitoring of Short-Term Health Effects of Urban Air Pollution. 3 www.debatdeplacements.paris.fr/arrondissement/telechargement/Compte_ rendu_Sante.pdf. 1364-8152/$ – see front matter Ó 2008 Elsevier Ltd. All rights reserved. doi:10.1016/j.envsoft.2008.04.004

pollution exceedances, it is important to be concerned also about the medium-level pollution’’. Modelling the ozone’s fluctuations and providing a good prediction are two of the most important tasks for the researchers. Two types of models can be used: deterministic or statistical (black boxes). The use of partial differential equations to construct a deterministic model to forecast the ozone concentrations in a limited domain is a rather complex process, which has to take into account a large number of physical and chemical interactions between predictor variables, and requires numerous accurate input data (e.g. emissions, meteorology and land cover). These are the main reasons why the deterministic models are very expensive to develop and maintain. Vautard et al. (2001) developed a hybrid statisticaldeterministic chemistry-transport model for the ozone prediction in the Paris area, during Summer 1999. The model uses real-time weather forecasts, performing ozone prediction up to three days ahead. The chemistry-transport model (CHIMERE) is forced at the boundaries by a statistical back trajectory-based model predicting the background concentrations of a few species on a grid of 6  6 km cells. In the urban area, ozone levels are fairly well forecast, with correlation coefficients between forecast and observations about 0.7–0.8, and RMSE (cf. Section 3.4) in the range 15–20 mg m3 at short-lead times, and 20–30 mg m3 at long-lead times. In contrast with the deterministic models, the statistical ones are easier to implement. There are many different approaches trying to establish a mathematical relationship between the predictors (input variables) and predictands (output variables).

1408

A. Coman et al. / Environmental Modelling & Software 23 (2008) 1407–1421

Generally, the statistical models are based on the detection of some patterns, which are used later to forecast the desired pollutant concentrations. Among the statistical approaches, the artificial neural networks (ANN), and in particular multilayer perceptrons (MLP), were largely applied in the last decade for short-term prediction of gas and particulate matter pollution. A complete review on applications of multilayer neural networks in the atmospheric sciences was written by Gardner and Dorling (1998). They show that the MLP can be trained to approximate any smooth measurable function and it doesn’t make any assumptions concerning the data distribution. Maier and Dandy (2000) tried to establish the general context and to develop some guidelines for the application of these models into various research areas, like water resources. They analysed 43 papers dealing with neural networks and remarked ‘‘the tendency among researchers to apply ANNs to problems for which other methods have been unsuccessful’’. In the last 10 years, many researchers in the environmental field have tried to improve the quality of ozone prediction for a horizon varying from 1 to 24 h. Viotti et al. (2002) applied ANN to forecast several pollutants: particulate matter, sulphur dioxide, nitrogen oxides, ozone, carbon monoxide and benzene (short- and middle–long-term prediction) using a general form of the logistic activation function with three adjustable parameters and the back-propagation training algorithm. The selected inputs were related to the meteorological conditions and to the traffic levels. Abdul-Wahab and Al-Alawi (2002) focused on the identification of the factors that regulate the ozone levels during daylight and examined the relative contribution of meteorology on the ozone concentration, a contribution found to vary within the range 33– 41%. They also found an important contribution of the temperature, NO, SO2, relative humidity and NO2, but lower effect than expected for the solar radiation. Balaguer Ballester et al. (2002) present a comparison of several predictive models used to forecast 24 h in advance the hourly ozone concentration: (i) autoregressive-moving average with exogenous inputs (ARMAX); (ii) multilayer perceptrons (MLP) and (iii) finite impulse response (FIR) neural network. They focused on the ozone summer peaks recorded by three Spanish monitoring stations situated in urban or rural environment, from 1996 to 1999. The five performance criteria calculated yield reasonably good results and indicated that the MLP neural networks were more effective than the linear ARMAX models, which performed better than the dynamic FIR neural networks. Within an exhaustive study, Schlink et al. (2003) tested 15 different statistical techniques (the persistence model, multivariate linear regression, autoregression, neural networks and generalised additive models) for ozone forecasting, applied to 10 data sets representing different meteorological and emission conditions throughout Europe, and compared their performance with a deterministic chemical trajectory model. According to their results, they reject the hypothesis of non-linear ozone dynamics and reveal the existence of static non-linearities between ozone, meteorological parameters and concentrations of other pollutants. For the comparison of different models using different data sets they recommended the index of agreement (d2) and the success index SI (cf. Section 3.4). They obtained limited success indices for linear forecasting techniques and a better prediction performance for both neural networks and generalised additive models, which are able to handle static non-linearities. Kukkonen et al. (2003) undertook a model inter-comparison using five neural network models, a linear statistical model and a deterministic one for the prediction of urban NO2 and PM10 concentrations at two stations in the central Helsinki. Their models used traffic flow and meteorological variables as input data to forecast the two pollutants on a 24-h horizon. To avoid overfitting,

the authors used a Bayesian regularisation scheme discussed in Foxall et al. (2002). The results showed that the non-linear ANN models performed slightly better than the deterministic model and the statistical linear one. Agirre-Basurko et al. (2006) present a real-time prediction approach permitting to forecast ozone and nitrogen dioxide levels from 1 to 8 h in advance using traffic variables, meteorological parameters, O3 and NO2. They tested three different models: MLP, linear regression and persistence using the criteria given by the Model Validation Kit4: the correlation coefficient (R), the Normalised Mean Square Error (NMSE), the factor FA2, the Fractional Bias (FB) and the Fractional Variance (FV) (cf. Section 3.4). The results indicated that the best model for prediction was the MLP, which proved its ability to predict O3(t þ k) and NO2(t þ k) concentrations (k ¼ 1, 2, ., 8) except in the cases of 2 and 3 h ahead. Sousa et al. (2007) applied feedforward ANN using principal components as inputs to obtain next-day hourly ozone concentrations. They used as predictor variables hourly concentrations of O3, NO, NO2, temperature, wind velocity and relative humidity. The results obtained using original variables were compared with those using principal components (PC). The application of the PC in ANN was considered to lead to better results than the use of original raw data, because it reduces the number of inputs, therefore decreasing the model complexity, but in practice, the performance indices were rather similar for the two approaches. The best indices of performance (using PC) show a correlation coefficient for the validation set of 0.73 and a RMSE of 21.78 mg m3 (cf. Section 3.4). Finally, Brunelli et al. (2007) present a recurrent Elman neural network for the prediction, two-days ahead, of daily maximum concentrations of SO2, O3, PM10, NO2, CO in the Palermo city, using as meteorological predictors: wind direction and speed, barometric pressure and ambient temperature. They obtained a correlation coefficient ranging from 0.72 to 0.97, for the various forecast pollutants. This brief state of the art shows that there is consensus for using non-linear models versus linear ones. Meanwhile, there is a topic which seems to be ignored: some authors used static models, some other dynamic ones, but the profitability of dynamic (more complex) architectures is not demonstrated. There are some researchers who tried, by various methods, to verify the presence of non-linearity in the dynamics of ozone time series. Among them, Palusˇ et al. (2001) used a technique of uni- and multivariate surrogate data, as well as an information-theoretic functionals named redundancies. They concluded that there is no evidence of the non-linear dynamics in the ozone time series recorded at the Czech Station Tusˇimice; the time series was found related to the meteorological variables by the slowly decreasing long-term linear dependence, in some cases enhanced by a shortlag non-linearity. Another remark made in the same study is that the neural networks can improve only the short-term (several hours) ozone concentration forecast. The studies conducted by Haase and Schlink (2001) and by Schlink et al. (2003) led to the same conclusion concerning the existence of non-linear dynamics in the ozone time series under their study. Schlink et al. (2001) detected a very weak dynamic non-linearity in the ozone time series recorded in the urban area of Berlin, Germany. In conclusion, the previously cited studies sustain that there is no evidence of the non-linear dynamics in the ozone time series, or it is very weak. In this study, we focus on the local ozone prediction using ANNs. This simpler tool can provide complementary information for the chemistry-transport models performing on large grid cells. The selection of ANNs among the statistical models was based on the experience achieved in the previously cited studies, which showed

4

www.harmo.org/kit/default.asp.

A. Coman et al. / Environmental Modelling & Software 23 (2008) 1407–1421

a better performance of the latter especially with respect to the linear ones. In the light of the previous discussion concerning the dynamic non-linearities in the ozone time series, we propose here a comparison between two neural network architectures: a ‘‘dynamic’’ and a ‘‘static’’ one. The ‘‘dynamic’’ architecture is represented by a cascade of 24 MLP with one output each, arranged so that the previous output of an MLP becomes an input for the next one. The ‘‘static’’ architecture consists of a single MLP with 24 outputs. We selected a database covering a whole year and not only the summer peaks, since medium-level pollution affects also the population health. For the same reason, we do not focus only on the daily maximum, we try to forecast the whole 24-h horizon. 2. Data and site description Hourly ozone and nitrogen dioxide concentrations used in this study were measured by AIRPARIF,5 the monitoring network responsible with the air quality in the greater Paris. Two different sites were selected in this work to study ozone prediction. The first site is an urban one, Aubervilliers, (2.3855 N, 48.9039 E), where the traffic influences are very important, and where AIRPARIF measures O3 and NO2; the second one, Prunay, is located in a rural area (1.6749 N, 48.8580 E) and only O3 measurements are available. Meteorological variables were measured by the Meteo France6 station located in Paris, in the Montsouris Park. The meteorological data at our disposal were temperature (T), relative humidity (RH), sunshine duration (SD), global solar radiation (SR), wind speed (WS) and direction (WD), all of them averaged over 60 min. Most of the year, the area of interest benefits of a wet, oceanic climate that cleans the atmosphere, but sometimes the anticyclones and the lack of wind favour the blockage of pollutants in the atmosphere around the city, resulting in significant pollution episodes. The data used in this study cover a period of one year, from August 2000 to July 2001. Both time series have gaps. When these gaps were not larger than 4 h, the missing values were replaced by the linear interpolation of the previous and subsequent values, otherwise the entire day was eliminated. The latter solution was applied for some days only at the Aubervilliers station. Another important feature of our database is the very high number of missing values of NO2, a fact which determined the splitting of Aubervilliers data in five blocks treated separately and then concatenated. Table 1 presents the general statistics calculated for the two pollutants used in this study: ozone and nitrogen dioxide. The mean of ozone measurements at the rural site, Prunay, is higher than the mean at the urban one, Aubervilliers; this remark remains true also for the maximum value and for the standard deviation. A larger database, containing ozone measurements at Prunay during two additional years, has been employed for comparative tests of the different architectures. For this three-year ozone database, no meteorological data were available. In order to construct a model that is able to estimate one day in advance the concentration of ozone for the two locations, a natural idea was to use the last 24 hourly ozone measurements or the values previously forecast by the model itself. The autocorrelation function for the two considered time series of ozone confirms a maximum lag for 24 h (Fig. 1). The role of the ozone precursors (e.g. oxides of nitrogen, carbon monoxide and VOCs) and the influence of the meteorological parameters (e.g. solar radiation and temperature) are undoubted

5 6

www.airparif.asso.fr. www.meteofrance.com.

1409

Table 1 General statistics for pollutant concentrations of O3 and NO2 Station

Pollutant

Missing values (h)

Mean (mg m3)

Max (mg m3)

Median (mg m3)

STD (mg m3)

Prunay

O3

102

53.47

189.00

53.00

28.65

Aubervilliers

O3 NO2

354 728

41.69 34.88

153.00 195.00

42.00 32.50

25.44 24.66

(EPA, 1999). When this data was available, we used it in the model’s input. Within the aim of constructing a pure predictive model, we selected a 24-h lag between measurements of the meteorological parameters and the ozone prediction. We tried different configurations for our inputs (with and without meteorological parameters), in order to test their usefulness: do they allow a better prediction of ozone concentrations? 3. The prediction models 3.1. Network architectures and data pre-processing MLP is the most commonly used neural network in the field of air quality prediction. We present here the results obtained using, on the one hand, a classical architecture for a neural network and, on the other hand, a special network architecture described later. The simpler architecture adapted for our purpose consists of a single MLP (to be called 1 MLP in this paper) with one hidden layer of 20 neurons, and an output layer with 24 neurons, for the 24 h of the prediction horizon of ozone. The more complex architecture is a cascade of 24 MLP (to be called 24 MLP), each of them with one hidden layer and one output, arranged so that each MLP feeds the next one (Fig. 2). Each input block is composed of the past 24 h of ozone measurements and meteorological data or eventually, of the previously forecast values of ozone. The first input layer is composed exclusively of measurements, while the next ones contain the measurements and the previously forecast values. Due to the high computational cost and the architecture complexity, we decided to use a single hidden layer with two, three or five neurons for each MLP in the cascade network. In addition to the known influent meteorological parameters, Gardner and Dorling (1998) propose to use also as an input, periodical variables representing the cycle of passing hours: sin(2ph/24) and cos(2ph/24), where h is the hour of prediction. As Maier and Dandy (2000) note, the relationship between the number of training samples and the number of adjustable connection weights is crucial. The number of weights should not exceed the number of training samples. Thus, in our case, with

Fig. 1. Autocorrelation function for O3. Maximum lag at 24 h.

1410

A. Coman et al. / Environmental Modelling & Software 23 (2008) 1407–1421 Xˆ t 1 INPUT X(t), X(t-1),…,X(t-23) +meteo(t-23)

st

1 layer

Xˆ t 2

nd

2 layer rd

3 layer

th

4 layer

Xˆ t 3 INPUT X(t), X(t-1),…,X(t-22) +meteo(t-22)

th

6 layer

th

5 layer

…. Xˆ t 23

Xˆ t 1 Xˆ t 22

INPUT X(t), X(t-1),…,X(t-21) +meteo(t-21)

. .

th

48 layer th

47 layer

. . .

Xˆ t 2

Xˆ t 1 INPUT X(t) +meteo(t)

Xˆ t 3

Xˆ t 24

Fig. 2. Network’s architecture.

a complex architecture and a big number of connection weights, we decided to overlap the data; starting prediction at different hours of the day and not only from a fixed hour, the working database is larger. For each input layer we shifted the data such that on each displacement the earliest value of ozone concentration is eliminated and the next one is appended. For example, if the first training sample begins at 1 a.m. at day one and finishes at midnight, the next generated sample will start at 2 a.m. day one and will finish at 1 a.m. day two, and so on. We could thus generate approximately 8000 training cases, even though, these cases differ slightly. Thus, a particularity of our network is its ability to make prediction whatever the initial hour of prediction is. Concerning the data pre-processing, we decided to standardise the input variables, which were multiplied by a factor of 0.7 and then randomised (Ionescu and Candau, 2007). This treatment is justified by two reasons presented in Section 3.2. We decided to compare the results obtained with these two architectures, with those obtained using the persistence model, which consists in assigning hourly predicted ozone concentrations equal to those measured one-day before, respectively, at the same hours. 3.2. Activation functions and training algorithms The problem of training or learning in neural networks can be formulated in terms of optimisation: the minimisation of an error function. The error function used in this study is the mean square error and it depends on the adaptive network’s parameters (weights and biases). The derivatives of the error function with respect to the network parameters can be obtained using the backpropagation technique described by Rumelhart et al. (1986). For training purposes, we selected two second-order local optimisation methods: the scaled conjugate gradient (SCG) (Moller, 1993) and the limited-memory BFGS quasi-Newton method

(Shanno, 1978; Gill et al., 1981; Bishop, 1995); in both cases, iterative techniques are used to change the weights in order to obtain the desired minimisation. For these algorithms the error function is guaranteed not to increase as a result of the weights update. A potential disadvantage is that if they reach a local minimum, they remain forever, as there is no mechanism for them to escape (Bishop, 1995). The selection of the previously cited optimisation methods is motivated by the memory-reduced requirements and not for their convergence rate. Nevertheless, SCG offers sometimes a significant improvement in speed compared to conventional conjugate gradient algorithms (Bishop, 1995). As these methods begin with a steepest descent step, they are scale sensitive; this was the first reason why we standardised our data. The activation functions selected in this study were the hyperbolic tangent for the neurons in the hidden layers and the linear identity for the neuron in each output layer. The hyperbolic tangent ranges from 1 to 1; this was the second reason to standardise our data (Fausett, 1994). 3.3. Generalisation ability For the purpose of forecasting, the most important property of an algorithm is its ability to generalise and filter out the noise. Generalisation ability is defined as a model’s ability to perform well on data which were not used to calibrate it. Sometimes, overtraining occurs when the model learns ‘‘by heart’’ the training examples and it is not able to generalise in the face of new situations (Schlink et al., 2003). In order to avoid overtraining, some regularisation techniques can be employed, the most frequently used being the so-called early stopping. To apply it, data must be divided at the beginning in three subsets: training, validation and test (Bishop, 1995; Ripley, 1996). The first subset is used for computing the gradient and updating the network parameters (weights and biases). The error

A. Coman et al. / Environmental Modelling & Software 23 (2008) 1407–1421

calculated on the second one is monitored during the training process; when the network begins to overfit the data, the error on the validation set begins to rise; this is the moment when training is stopped and the network parameters at the minimum of the validation error are returned. Finally, the third set (the test set) is used to evaluate the model performance (Nunnari et al., 2004). In our study, the training set was fixed at 60% of all available data, while the validation and test sets, at 20% each one. This splitting was performed after the randomisation.

NMSE ¼

1 n

Pn

i ¼ 1 ðOi

1411

 Pi Þ 2

(6)

OP

the factor FA2, which gives the percentage of forecast cases in which the values of the ratio O/P are in the range [0.5, 2], the Fractional Bias (FB): FB ¼ 2

OP

(7)

OþP

and the Fractional Variance (FV): 3.4. Performance indices FV ¼ 2 For a good evaluation of the results obtained with different architectures, we calculated different statistical parameters for each hour of the prediction horizon, but also some global indices for the entire test set. Among the classical statistical criteria, we evaluated the determination coefficient R2, the root mean square error RMSE, the mean absolute error MAE, and the mean bias error MBE; all these indices measure residual errors and give a global idea of the difference between observed and modelled values. Their formulas are reminded in Eqs. (1)–(4): vffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi uPn P u ðO  OÞ2  ni¼ 1 ðOi  Pi Þ2 R ¼ t i ¼ 1 iPn (1) 2 i ¼ 1 ðOi  OÞ vffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi u n u1 X ðO  Pi Þ2 RMSE ¼ t n i¼1 i

(2)

s2O  s2P s2O þ s2P

(8)

with the same meaning for O and P. For the prediction exceedances, Schlink et al. (2006) propose three specific indices. The first one, the true positive rate (TPR), corresponds to the fraction of correctly predicted exceedances and it can be defined as TPR ¼ A=M

(9)

where A represents the correctly predicted exceedances and M all observed exceedances, while the second one, the false positive rate (FPR), can be achieved according to the formula: FPR ¼ ðF  AÞ=ðN  MÞ

(10)

where N is the total number of days considered and F all the predicted exceedances. Finally, the success index SI is obtained by combining the previous rates: SI ¼ TPR  FPR

MAE ¼

n 1X jP  Oi j n i¼1 i

(3)

MBE ¼

n 1X ðP  Oi Þ n i¼1 i

(4)

(11)

4. Results and discussion 4.1. Results at Aubervilliers

where Pi and Oi are the predicted and observed concentrations, and  represents the observation mean. O In addition, we determined the index of agreement d2 (Willmott, 1982), which is a relative measure expressing the degree to which predictions are error-free (Gardner and Dorling, 2000), and which allows the comparison of different models using different data sets (Schlink et al., 2003): Pn 2 i ¼ 1 jPi  Oi j d2 ¼ 1  Pn (5) 2 i ¼ 1 ðjPi  Oj þ jOi  OjÞ Other indices mentioned in this study are the Normalised Mean Square Error (NMSE):

For the site of Aubervilliers we present eight ranges of statistics, for each simulation using the 24 MLP model, in terms of global indices calculated on the test set (Table 2). At this site we had a major problem with gaps in the NO2 time series. Although O3 gaps were not so big, we decided to eliminate many days in order to obtain in this way a common base for the O3 measurements, NO2 concentrations and meteorological variables. We performed eight simulations using two training algorithms (BFGS and SCG), and different combinations of meteorological parameters together with the pollutant concentrations as inputs. The first two simulations were performed using only O3 measurements or forecast values, but no meteorological predictors

Table 2 Global indices calculated on the entire horizon (cf. Section 3.4) on the test set for the simulations performed at Aubervilliers using the 24 MLP model Inputs (number): description

Training algorithm

Mean (mg m3)

STD (mg m3)

RMSE (mg m3)

MAE (mg m3)

d2

R2

(24): O3

BFGS SCG

32.05 31.41

28.85 27.80

19.55 19.01

14.87 14.51

0.46 0.17

0.83 0.83

0.54 0.53

(26): O3, T, NO2

BFGS SCG

29.85 30.84

27.43 27.71

18.48 18.49

14.07 14.14

0.05 0.42

0.84 0.84

0.55 0.55

(28): O3, T, RH, SR, NO2

BFGS SCG

30.13 30.81

27.24 27.84

17.87 18.32

13.66 13.90

0.08 0.25

0.85 0.85

0.57 0.57

(30): O3, T, RH, SR, SD, WS, NO2

BFGS SCG

29.83 30.50

26.74 26.90

17.62 18.00

13.54 13.87

0.01 0.03

0.85 0.85

0.57 0.55

MBE (mg m3)

Abbreviations: T – temperature, RH – relative humidity, SR – solar radiation, SD – sunshine duration, WS – wind speed, NO2 – nitrogen dioxide, all of them at 24 h before the prediction time; O3 – ozone concentrations during all the 24 h before the prediction time; t is a function of the prediction hour (h) (t ¼ 2ph/24); BFGS – quasi-Newton (Broyden, Fletcher, Goldfarb and Shanno algorithm); SCG – scaled conjugate gradient.

1412

A. Coman et al. / Environmental Modelling & Software 23 (2008) 1407–1421

(24 inputs). For the next ones, we added at these 24 inputs, 1 and then 3 meteorological parameters: temperature T, relative humidity RH, global solar radiation SR plus the NO2 measurements. Occasionally used predictors were sunshine duration SD and wind speed WS (30 inputs). A good compromise between the network size and the selected predictors in the forecasting process shows that the best results in terms of index of agreement and determination coefficient are obtained using the BFGS training algorithm and the following inputs: 24 ozone concentrations, the 3 meteorological parameters earlier mentioned and the NO2 measurements (cf. Table 2). On the prediction horizon, the best results varied from R2 ¼ 0.89, RMSE ¼ 9.03 mg m3, d2 ¼ 0.97, MAE ¼ 6.51 mg m3 for 1 h ahead, up to R2 ¼ 0.54, RMSE ¼ 18.95 mg m3, d2 ¼ 0.84, MAE ¼ 14.88 mg m3 for 24 h ahead. These variations are illustrated for the RMSE and R2 statistics, on Fig. 3. Nevertheless, we can notice that the applied models exhibit similar levels of performance. This is obvious if we analyse the Fig. 3, where we represented two performance indices (R2 and RMSE) for the worst (24 inputs) and, respectively, best (28 inputs)

simulations performed at this station. We observed a normal evolution for R2 and RMSE, the two indices presented in Fig. 3: the first one is decreasing and the second one is increasing gradually with the prediction horizon. The exogenous variables, i.e. meteorological parameters and NO2 concentration, do not increase sensibly the model performance. It is probably that their influence is not crucial for ozone prediction when there is no high variation from one day to another; such strong variations appear usually in the case of important pollution peaks, but there are very few such examples within our database. Another plausible explanation resides in the fact that the meteorological parameters are monitored by a station located in the centre of Paris, while the Aubervilliers station is located in Paris outskirts, rather far from it; thus the meteorological parameters used are not so representative as if they were measured at the same location or in its vicinity. We compared the best results obtained using the complex 24 MLP model with those given by the simpler architecture, 1 MLP, with the same input variables, and to the persistence (cf. Table 3).

Fig. 3. Performance indices (cf. Section 3.4) on the test set for the worst (RMSE (a), R2 (a)) and, respectively, the best (RMSE (b), R2 (b)) simulations performed at Aubervilliers for the entire prediction horizon using the 24 MLP model.

A. Coman et al. / Environmental Modelling & Software 23 (2008) 1407–1421 Table 3 Global indices calculated on the entire horizon (cf. Section 3.4) on the test set for the three models applied at Aubervilliers (24 MLP, 1 MLP and persistence) Method

Mean (mg m3)

STD (mg m3)

RMSE (mg m3)

MAE (mg m3)

MBE (mg m3)

d2

R2

24 MLP 1 MLP Persistence

30.13 30.51 30.13

27.24 27.71 27.24

17.87 17.70 22.31

13.66 13.36 16.37

0.08 0.34 0.21

0.85 0.86 0.82

0.57 0.59 0.44

One issue raised by this inter-comparison is the fact that the ‘‘static’’ architecture seems to perform a little bit better than the ‘‘dynamic’’ one. Indeed, if we look at the same statistics but for the entire prediction horizon (Fig. 4) we may remark that, for the first 8 h, the 1 MLP model slightly outperforms the other ones. This result, rather surprising, can argue for the absence of the dynamic non-linearities in the ozone time series and/or in the relationship between ozone and the meteorological variables. It is important to remind the studies conducted by Palusˇ et al. (2001), Haase and Schlink (2001) and Schlink et al. (2003) which led to similar results and conclusions via different methods than ours.

1413

4.2. Results at Prunay In contrast with the Aubervilliers database, at Prunay there were no NO2 measurements available and we had no gaps in the time series except for the end of July 2001. We performed eight simulations using at the beginning only the O3 measurements or forecast values (24 inputs), then we added either two periodical variables, sin(2ph/24) and cos(2ph/24), where h is the prediction hour (26 inputs), or four meteorological predictors (28 inputs), and finally we used all the available data (30 inputs). Each architecture was trained using again the two BFGS and SCG algorithms. The global indices obtained for all these cases are presented in Table 4. We can notice again a slight difference among the results, like for Aubervilliers. Compared to the latter, the use of all available parameters (30 inputs) was more effective for this site: from R2 ¼ 0.92, RMSE ¼ 8.2 mg m3, d2 ¼ 0.98, MAE ¼ 6.13 mg m3 for 1 h ahead, up to R2 ¼ 0.56, RMSE ¼ 19.55 mg m3, d2 ¼ 0.84, MAE ¼ 15.56 mg m3 for 24 h ahead (cf. Fig. 5). Globally, the scores obtained for Prunay were better than for Aubervilliers; the principal reason is, arguably, the larger database,

Fig. 4. Performance indices (cf. Section 3.4) on the test set for the three models (24 MLP, 1 MLP and persistence) tested at Aubervilliers for the entire prediction horizon.

1414

A. Coman et al. / Environmental Modelling & Software 23 (2008) 1407–1421

Table 4 Global indices calculated on the entire horizon (cf. Section 3.4) on the test set for the simulations performed at Prunay using the 24 MLP model d2

R2

SI

0.45 0.14

0.86 0.85

0.58 0.57

0.52 0.58

13.76 13.75

0.04 0.31

0.87 0.86

0.61 0.60

0.59 0.74

17.91 17.88

13.71 13.79

0.15 0.28

0.87 0.87

0.62 0.63

0.71 0.81

17.61 17.57

13.63 13.49

0.38 0.04

0.88 0.88

0.63 0.63

0.66 0.64

Inputs (number): description

Training algorithm

Mean (mg m3)

STD (mg m3)

RMSE (mg m3)

MAE (mg m3)

(24): O3

BFGS SCG

53.86 53.70

28.00 28.36

18.09 18.62

13.89 14.15

(26): O3, sin(t), cos(t)

BFGS SCG

53.69 54.16

28.66 28.38

17.99 17.91

(28): O3, T, RH, SR, SD

BFGS SCG

53.84 53.25

29.11 29.20

(30): O3, T, RH, SR, SD, sin(t), cos(t)

BFGS SCG

52.96 53.56

28.88 28.90

MBE (mg m3)

Abbreviations: T – temperature, RH – relative humidity, SR – solar radiation, SD – sunshine duration, all of them at 24 h before the prediction time; O3 – ozone concentrations during all the 24 h before the prediction time; t is a function of the prediction hour (h) (t ¼ 2ph/24); BFGS – quasi-Newton (Broyden, Fletcher, Goldfarb and Shanno algorithm); SCG – scaled conjugate gradient.

but also a decreased complexity of the rural environment with respect to the urban one. Examples of two performance indices for the worst and, respectively, the best simulations are given in Fig. 5. The curves behaviour remains the same with slight differences between them.

A first comparative evaluation of the forecasting performance for the two neural models and persistence for this site is presented in Table 5. Again, we notice a slight improvement when using the 1 MLP model, compared to the 24 MLP one, and a more significant one, when comparing to the persistence. These results are detailed

Fig. 5. Performance indices (cf. Section 3.4) on the test set for the worst (RMSE (a), R2 (a)) and, respectively, the best (RMSE (b), R2 (b)) simulations performed at Prunay for the entire prediction horizon using the 24 MLP model.

A. Coman et al. / Environmental Modelling & Software 23 (2008) 1407–1421

1415

Table 5 Global indices (cf. Section 3.4) on the test set for the three models applied at Prunay (24 MLP, 1 MLP and persistence) Model

Mean (mg m3)

STD (mg m3)

RMSE (mg m3)

MAE (mg m3)

MBE (mg m3)

d2

R2

SI

24 MLP 1 MLP Persistence

52.96 53.17 53.56

28.88 29.41 28.90

17.61 17.61 23.16

13.63 13.38 17.72

0.38 0.22 0.58

0.88 0.88 0.82

0.63 0.64 0.46

0.66 0.70 0.51

for the entire prediction horizon on Fig. 6. We also notice that the differences between the results using the two architectures are very low, especially for the RMSE. At Prunay, the number of exceedances (concentrations exceeding a threshold level) was large enough to permit us to calculate the success index (Section 3.4). This was not the case for the Aubervilliers station, where the number of exceedances being negligible, the success index is not representative; that is why it was not presented in the previous section. The reference level for human health protection is 120 mg m3 averaged over 8 h according to the World Health Organisation (WHO, 2001). Considering a model uncertainty of 20 mg m3, which

corresponds approximately to the RMSE, the success indices (SI) obtained for all the simulations using the 24 MLP model were presented in Table 4, last column. For a second performance comparison between the 1 MLP model and the cascade, we slightly modified the latter’s architecture such as each input block (cf. Fig. 2) contains the same past 24 ozone concentrations, but no meteorological data (to be called 24 MLPb). These changes were made with the purpose to have, for the two models, exactly the same input information used for each neuron of the prediction horizon. The cascade becomes a repetitive structure of the 1 MLP model without meteorological data (1 MLPb), with the difference that the former uses, in addition, the

Fig. 6. Performance indices (cf. Section 3.4) on the test set for the three models (24 MLP, 1 MLP and persistence) tested at Prunay for the entire prediction horizon.

1416

A. Coman et al. / Environmental Modelling & Software 23 (2008) 1407–1421

previously predicted values of ozone. The comparative evaluation of the forecasting performance for the two neural models is detailed, for the entire prediction horizon, on Fig. 7. We notice a very similar behaviour for the 1 MLPb model and the cascade (24 MLPb). In the 24 MLPb structure, meteorological data was ignored because leading to a too complex network; by decreasing the number of neurons in the hidden layers from three to one, it became possible to use also the meteorological data. This new network was then compared with the original 1 MLP model; again, the results were very similar.

4.3. Results obtained when using contemporaneous meteorological measurements Additional simulations were performed at Prunay in order to test the improvement when using meteorological data contemporaneous with the predicted ozone. In practice, this meteorological data should consist of a forecast provided by a meteorological service. Since there was no meteorological forecast available in our

study, we decided to use instead of it, the real meteorological measurements. In this case, the prediction is more accurate than in the previous cases, when we were using only the past information. The performance indices presented for the entire prediction horizon for the two architectures using the measured meteorological variables as forecast (Fig. 8), show higher values than those in Fig. 6 for the same station, but using past meteorological values. The total gain obtained in this case, compared to the results presented in Fig. 6, is approximately 2 mg m3 for the RMSE and MAE, and 0.05 for d2 and R2. When using contemporaneous values, the differences between the two models are more obvious: the ‘‘static’’ architecture outperforms the ‘‘dynamic’’ one, but, as before, the differences are not very important. Obviously, in practice, the results would be less accurate, because we used the real monitored meteorological values instead of the real forecast. These hypothetical simulations were performed in order to evaluate a maximum improvement, of course, not attained, because forecast is never perfect. In practice, it is meaningful to use only the true forecast value provided by a meteorological forecast service.

Fig. 7. Performance indices (cf. Section 3.4) on the test set applying two models 24 MLPb and 1 MLPb (described in Section 4.2) tested at Prunay.

A. Coman et al. / Environmental Modelling & Software 23 (2008) 1407–1421

1417

Fig. 8. Performance indices (cf. Section 3.4) on the test set applying two models 1 MLP and 24 MLP using the measured meteorological predictors as forecast (at Prunay).

4.4. Results on a larger database To avoid the apparent irregular dynamics due to the effect of external covariates, i.e. meteorological parameters and other pollutants, we decided to study also the univariate time series of ozone for the prediction of the next-day hourly concentration. We selected a larger database of hourly ozone concentrations covering three years, from 2000 to 2002, at the Prunay station. The second and main reason for a larger base selection is to increase the ratio between the number of training samples and network parameters. The test set was created using measurements from the third year only; this test set was the same for all the performed simulations. The models tested here are those previously presented in Section 4.2, 1 MLPb and 24 MLPb, and, in addition, a modified architecture derived from the 24 MLPb, where all the feedback connections were cut. The simple comparison (Fig. 9) showed, once again, a similar performance for the three architectures evaluated on the same test set for the entire prediction horizon. The comparison of the different network architectures, ‘‘static’’ and ‘‘dynamic’’, applied on different databases shows that there is

no evidence for the ‘‘most appropriate’’ neural architecture. In addition, there was no benefit when using a more complex ‘‘dynamic’’ architecture. 4.5. Sensitivity analysis As all the presented architectures are very complex, containing a high number of parameters, there is a danger of overfitting. In most applications, the main objective is to maximise the network’s generalisation ability. The network’s ability is related to its sensitivity. In order to test the generalisation ability of the two architectures, the sensitivity to changes in input data was numerically evaluated. Fixing the model parameters obtained by training, we perturbed the input values of the test set from 10 to 40% and we regarded the responses to the different changes. Fig. 10 shows that small changes in input induce small changes in output; a progressive degradation of the response, when increasing the perturbation amount, is obvious. This remark is true for all the architectures presented in Fig. 10.

1418

A. Coman et al. / Environmental Modelling & Software 23 (2008) 1407–1421

Fig. 9. Performance indices (cf. Section 3.4) on the test set applying three models 1 MLP, 24 MLP and 24 MLP without feedback connections on a larger database (at Prunay).

In addition, a perturbation on the input data has a variable effect on the prediction horizon. For example, a perturbation of 10% on all input data induced a variation in the network’s output (reported to the output obtained without input perturbation) which rises up to 30% (for a mean RMSE value of 7%) for the cascade and up to 40% (for a mean RMSE value of 5%) for the single MLP, for the case of the larger database (Fig. 10b and d). One can notice that the amplitude of the RMSE variation is higher for the first hours of the prediction horizon, while for the second half, the amplitude of variation becomes negligible. Fig. 10 shows the same behaviour when using the smaller one-year database, either at Prunay, or at Aubervilliers (Fig. 10a and c). The second test focuses on the influence of the perturbation performed on the input values inside the training set. The model obtained for each perturbed training set is then evaluated on the same original test set. The results (Fig. 11) show small changes in responses provided by the two architectures obtained from an amount of perturbation rising up to 50%. Globally, these tests show that small changes in input data lead to rather small changes in outputs and increasing perturbation induces important output changes. This can be considered

characteristic to a rather good generalisation ability. Nevertheless, a particularity is the case of the ‘‘static’’ 1 MLP architecture, when the response is rather constant for the second half of the prediction horizon (Fig. 10d). In general, the results and the generalisation ability are better for the first half of the prediction horizon, more precisely, for the first 8 h. In addition, we performed several (five) runs (starting from different initial random weights) and the results were quasiidentical, sustaining the network’s stability.

5. Summary and conclusions The aim of this study was to develop a statistical tool for ozone prediction, based on data provided by an air quality monitoring network station. Complementary to the forecast obtained via a deterministic chemistry-transport model, performing on largedimension cell grids (6  6 km), our statistical prediction permits to obtain more accurate results for the prediction location. Our study is conducted in the greater Paris, using one-year data monitored by an urban and by a rural station.

A. Coman et al. / Environmental Modelling & Software 23 (2008) 1407–1421

1419

Fig. 10. Performance indices (cf. Section 3.4) using a fixed model on the perturbed input values (form 10 to 40%) of the test set – at Aubervilliers and Prunay on the initial bases using the 24 MLP model (a and c) and at Prunay on the larger database using the 24 MLP model and 1 MLP model (b and d).

As Paris is rather a polluted city, in contrast with most of the previous studies of ozone prediction, we did not focus only on the prediction of the daily maximum, but on a 24-h horizon, and not only on the summer peaks, but on a whole year. This strategy permitted to study the exceedances of a reference health level, corresponding to an 8-h mean value higher than 120 mg m3, potentially rising all over the year. It appears from the previous studies that the ANN models show a better performance with respect to the linear ones, but the question is: which is the most appropriate neural architecture for the forecasting purpose? Is it worth having a ‘‘dynamic’’ model, or a ‘‘static’’ one is as good as the former for the one-day ahead ozone prediction? Many model comparisons have been performed, revealing the superiority of non-linear models versus the linear ones; some other comparisons have been done among different static models, and finally among dynamic ones. Very few studies focused on the comparison ‘‘static’’ versus ‘‘dynamic’’ and, as far as we know, this is the first study testing the same kind of model, the MLP, using two versions: one ‘‘static’’ versus another, ‘‘dynamic’’. The ‘‘dynamic’’ neural network is represented by a cascade of 24 MLP with one output each, arranged so that the previous output of an MLP becomes an input for the next one, while the ‘‘static’’ one consists of a single MLP with 24 outputs. Both models were applied for prediction of ozone concentrations on the 24-h horizon, although neglecting the hour of the day at which we begin the prediction. The skills of both architectures were evaluated via five performance indices. Our networks perform well, with a fair accuracy for medium and high levels, using ozone concentrations and, eventually, meteorological parameters. The performance of both architectures slightly

increases when using meteorological predictors and decreases gradually with the prediction lag. Even if the persistence in our databases is not as high as in other study cases, our results are comparable with those obtained by the other authors. Thus, the success indices (corresponding to exceedances with regard to the health level) obtained at Prunay (the rural station) vary between 0.52 and 0.81 for the ‘‘dynamic’’ architecture, while for the ‘‘static’’ one, the value obtained is 0.70. If we regard the comparison between the two neural models we remark that, for the urban site, the difference is more important than for the rural site, and this difference is in favour of the ‘‘static’’ architecture. So, we can conclude that this model is slightly more effective than the ‘‘dynamic’’ one for the urban site, at least for the first 8 h of the prediction horizon. As for the rural site, we cannot conclude in favour of one of these models. Anyway, both of the neural models outperform the linear one: the persistence. In conclusion, on our database, the simpler ‘‘static’’ architecture and the ‘‘dynamic’’ one lead to rather equivalent results, so we can argue for the absence of the dynamic non-linearities in the ozone time series and/or in the relationship between ozone and the meteorological variables. Some other authors sustain that there is no evidence of dynamic non-linearities in ozone time series and we arrived at similar results and conclusions via different methods than theirs. Obviously, the conclusions are site-dependent. We remind that these results were obtained for a 24-h horizon, and thus, this conclusion must not be extrapolated to other time scales. Thus, the success of one architecture or another is verified only for the site or database under study. For generalisation purposes, it should be interesting to extend this comparison to other sites. Our results pointing out the absence of dynamic non-linearities at a daily time

1420

A. Coman et al. / Environmental Modelling & Software 23 (2008) 1407–1421

Fig. 11. Performance indices (cf. Section 3.4) on the same original test set after perturbing the training set with an amount varying from 10 to 100% – at Aubervilliers and Prunay on the initial bases using the 24 MLP model (a and c) and at Prunay on the larger database using the 24 MLP model and 1 MLP model (b and d).

scale open the door to a discussion focusing on their physicochemical interpretation. As all the presented architectures are very complex, containing a high number of parameters, there is a danger of overfitting. In order to test the generalisation ability of the two architectures, the sensitivity to changes in input data was numerically evaluated. These tests showed that small changes in input data lead to rather small changes in outputs and increasing perturbation induces important output changes. This can be considered characteristic to a rather good generalisation ability. Nevertheless, a particularity is the case of the ‘‘static’’ 1 MLP architecture, when the response is rather constant for the second half of the prediction horizon. In general, the results and the generalisation ability are better for the first half of the prediction horizon, more precisely, for the first 8 h. References Abdul-Wahab, S.A., Al-Alawi, S.M., 2002. Assessment and prediction of tropospheric ozone concentration levels using artificial neural networks. Environmental Modelling & Software 17, 219–228. Agirre-Basurko, E., Ibarra-Berastegi, G., Madariaga, I., 2006. Regression and multilayer perceptron-based models to forecast hourly O3 and NO2 levels in the Bilbao area. Environmental Modelling & Software 21, 430–446. Balaguer Ballester, E., Camps i Valls, G., Carrasco-Rodriguez, J.L., Soria Olivas, E., del Valle-Tascon, S., 2002. Effective 1-day ahead prediction of hourly surface ozone concentrations in eastern Spain using linear models and neural networks. Ecological Modelling 156, 27–41. Bishop, C.M., 1995. Neural Networks for Pattern Recognition. Clarendon Press, Oxford. Brunelli, U., Piazza, V., Pignato, L., Sorbello, F., Vitabile, S., 2007. Two-days ahead prediction of daily maximum concentrations of SO2, O3, PM10, NO2, CO in the urban area of Palermo, Italy. Atmospheric Environment 41, 2967–2995. EPA, 1999. Guideline for Developing an Ozone Forecasting Program. Environmental Protection Agency, Office of Air Quality Planning and Standards, USA. EPA-454/ R-99-009.

Fausett, L., 1994. Fundamentals of Neural Networks. Architectures, Algorithms and Applications. Prentice Hall, Englewood Cliffs, NJ 07632. Foxall, R.J., Cawley, G.C., Dorling, S.R., Mandic, D.P., August 2002. Error functions for prediction of episodes of poor air quality. In: Proceedings of the International Conference on Artificial Neural Networks (ICANN-2002). Lecture Notes on Computer Science, vol. 2415. Springer, Madrid, Spain, pp. 1031–1036. Gardner, M.W., Dorling, S.R., 1998. Artificial neural network (the multilayer perceptron) – a review of applications in the atmospheric sciences. Atmospheric Environment 32, 2627–2636. Gardner, M.W., Dorling, S.R., 2000. Statistical surface ozone models: an improved methodology to account for non-linear behaviour. Atmospheric Environment 34, 21–34. Gill, P., Murray, W., Wright, M., 1981. Practical Optimisation. Academic Press, New York. Haase, P., Schlink, U., 2001. Non-parametric short-term prediction of ozone concentration in Berlin. In: Proceedings Air Pollution Modelling and Simulation Conference, Paris 2001. Ionescu, A., Candau, Y., 2007. Air pollutant emissions prediction by process modelling – application in the iron and steel industry in the case of a re-heating furnace. Environmental Modelling & Software 22, 1362–1371. Kukkonen, J., Partanen, L., Karppinen, A., Ruuskanen, J., Junninen, H., Kolehmainen, M., Niska, H., Dorling, S., Chatterton, T., Foxall, R., Cawley, G., 2003. Extensive evaluation of neural network models for the prediction of NO2 and PM10 concentrations, compared with a deterministic modeling system and measurements in central Helsinki. Atmospheric Environment 37, 4539–4550. Maier, H.R., Dandy, G.C., 2000. Neural networks for the prediction and forecasting of water resources variables: a review of modelling issues and applications. Environmental Modelling & Software 15, 101–124. Medina, S., Le Tertre, A., Que´nel, P., Le Moullec, Y., Lameloise, P., Guzzo, J.C., Festy, B., Ferry, R., Dab, W., 1997. Air pollution and doctor’s house calls: results from the ERPURS system for monitoring the effects of air pollution on public health in greater Paris, France, 1991–1995. Environmental Research 75, 73–84. Moller, M.F., 1993. A scaled conjugate gradient algorithm for fast supervised learning. Neural Networks 6, 525–534. Nunnari, G., Dorling, S., Schlink, U., Cawley, G., Foxall, R., Chatterton, T., 2004. Modelling SO2 concentration at a point with statistical approaches. Environmental Modelling & Software 19, 887–905. Palusˇ, M., Pelika´n, E., Eben, K., Krejcˇı´rˇ, P., Jurusˇ, P., 2001. Nonlinearity and prediction of air pollution. In: Artificial Neural Nets and Genetic Algorithms. Proceedings of the International Conference. Springer, Wien, pp. 473–476.

A. Coman et al. / Environmental Modelling & Software 23 (2008) 1407–1421 Ripley, B.D., 1996. Pattern Recognition and Neural Networks. Cambridge University Press. Rumelhart, D.E., Hinton, G.E., Williams, R.J., 1986. Learning internal representations by error propagation. In: Parallel Distributed Processing: Explorations in the Microstructure of Cognition, vol. 1. MIT Press, Cambridge, MA, pp. 318–362. Schlink, U., John, S., Herbarth, O., 2001. Transfer-Function Models Predicting Ozone in Urban Air, Contribution to the SATURN Project, Annual Report 2001. Schlink, U., Dorling, S., Pelikan, E., Nunnari, G., Cawley, G., Junninen, H., Greig, A., Foxall, R., Eben, K., Chatterton, T., Vondracek, J., Richter, M., Dostal, M., Bertucco, L., Kolehmainen, M., Doyle, M., 2003. A rigorous inter-comparison of ground-level ozone predictions. Atmospheric Environment 37, 3237–3253. Schlink, U., Herbarth, O., Richter, M., Dorling, S., Nunnari, G., Cawley, G., Pelikan, E., 2006. Statistical models to assess the health effects and to forecast ground-level ozone. Environmental Modelling & Software 21, 547–558.

1421

Shanno, D.F., 1978. Conjugate gradient methods with inexact line searches. Mathematics of Operations Research 3, 244–256. Sousa, S.I.V., Martins, F.G., Alvim-Ferraz, M.C.M., Pereira, M.C., 2007. Multiple linear regression and artificial neural networks based on principal components to predict ozone concentrations. Environmental Modelling & Software 22, 97–103. Vautard, R., Beekmann, M., Roux, J., Gombert, D., 2001. Validation of a hybrid forecasting system for the ozone concentrations over the Paris area. Atmospheric Environment 35, 2449–2461. Viotti, P., Liuti, G., Di Genova, P., 2002. Atmospheric urban pollution: applications of an artificial neural network (ANN) to the city of Perugia. Ecological Modelling 148, 27–46. WHO, 2001. Air Quality Guidelines for Europe, second ed. Willmott, C.J., 1982. Some comments on the evaluation of model performance. Bulletin of the American Meteorological Society 63 (11), 1309–1313.