A comparative performance evaluation of the AURAMS and CMAQ air-quality modelling systems

Atmospheric Environment 43 (2009) 1059–1070

Contents lists available at ScienceDirect

Atmospheric Environment journal homepage: www.elsevier.com/locate/atmosenv

A comparative performance evaluation of the AURAMS and CMAQ air-quality modelling systems Steven C. Smyth a, *, Weimin Jiang a, Helmut Roth a, Michael D. Moran b, Paul A. Makar b, Fuquan Yang a, Ve´ronique S. Bouchet c, Hugo Landry c a b c

Institute for Chemical Process and Environmental Technology, National Research Council of Canada, Canada Air Quality Research Division, Environment Canada, Canada Meteorological Service of Canada, Environment Canada, Canada

a r t i c l e i n f o

a b s t r a c t

Article history: Received 13 May 2008 Received in revised form 12 November 2008 Accepted 13 November 2008

A harmonized comparative performance evaluation of A Unified Regional Air-quality Modelling System (AURAMS) v1.3.1b and Community Multiscale Air Quality (CMAQ) v4.6 air-quality modelling systems was conducted on the same North American grid for July 2002 using the same emission inventories, emissions processor, and input meteorology. Comparison of AURAMS- and CMAQ-predicted O3 concentrations against hourly surface measurement data showed a lower normalized mean bias (NMB) of 20.7% for AURAMS versus 46.4% for CMAQ. However, AURAMS and CMAQ had more similar normalized mean errors (NMEs) of 46.9% and 54.2%, respectively. Both models did similarly well in predicting daily 1-h O3 maximums; however, AURAMS performed better in calculating daily minimums. CMAQ’s poorer performance for O3 is partly due to its inability to correctly predict nighttime lows. Total PM2.5 hourly surface concentration was under-predicted by both AURAMS and CMAQ with NMBs of 10.4% and 65.2%, respectively. However, as with O3, both models had similar NMEs of 68.0% and 70.6%, respectively. In general, AURAMS performance was better than CMAQ for all major PM2.5 species except nitrate and elemental carbon. Both models significantly under-predicted total organic aerosols (TOAs), although the mean AURAMS concentration was over four times larger than CMAQ’s. The underprediction of TOA was partly due to the exclusion of forest-fire emissions. Sea-salt aerosol made up approximately 50.2% of the AURAMS total PM2.5 surface concentration versus only 6.2% in CMAQ when averaged over all grid cells. When averaged over land cells only, sea-salt still contributed 13.9% to the total PM2.5 mass in AURAMS versus 2.0% in CMAQ. Crown Copyright Ó 2008 Published by Elsevier Ltd. All rights reserved.

Keywords: Air quality Particulate matter Ozone AURAMS CMAQ

1. Introduction The AURAMS (A Unified Regional Air-quality Modelling System) and CMAQ (Community Multiscale Air Quality) air quality (AQ) modelling systems, developed by Environment Canada (EnvCan) and the U.S. Environmental Protection Agency (EPA), respectively, are two major AQ modelling systems used in North America. AURAMS and CMAQ forecasts of O3 and PM2.5 for the 2004 International Consortium for Atmospheric Research on Transport and Transformation/New England Air Quality Study (ICARTT-2K4/ NEAQS) summer period were used by McKeen et al. (2005, 2007) as part of a comparison of real-time O3 and PM2.5 forecasts from seven

* Corresponding author. Current address: Environment Canada, 200 Sacre´-Coeur Blvd. Gatineau, Quebec, Canada K1A 0H3. Tel.: þ1 819 953 6079; fax: þ1 819 953 3006. E-mail address: [email protected] (S.C. Smyth).

regional AQ models. In that study, AURAMS and CMAQ were run over the same time period but on different computers by different groups using different domains, map projections, grids, and emission and meteorological inputs. The results of this comparison thus reflect the combined differences caused by the model grids and inputs as well as by the formulations of the models themselves. Such a comparison is sometimes referred to as a ‘‘native-state’’ intercomparison. Other AQ model intercomparisons have tried to ‘‘harmonize’’ one or more aspects of the modelling systems. The most common approach has been to use the same anthropogenic emission inventories (e.g., van Loon et al., 2007; Vautard et al., 2007; Stern et al., 2008). Another study also used the same meteorological driver as well as the same emissions inventory (Kang et al., 2005). The performance evaluation comparison presented here is unique in that many aspects of the AURAMS and CMAQ simulations were ‘‘aligned’’ to reduce some of the common sources of differences. The two AQ models were run for the same period over

1352-2310/$ – see front matter Crown Copyright Ó 2008 Published by Elsevier Ltd. All rights reserved. doi:10.1016/j.atmosenv.2008.11.027

1060

S.C. Smyth et al. / Atmospheric Environment 43 (2009) 1059–1070

a common North American domain on a common map projection with same horizontal grid spacing on the same computer platform. Emission files were generated from the same anthropogenic emission inventories and the same biogenic emissions model. The raw emissions were processed using the same emissions processing system SMOKE (Sparse Matrix Operator Kernal Emissions) (CEP, 2005). The same meteorological fields generated by the EnvCan GEM (Global Environmental Multiscale) meteorological model were used as input to both SMOKE and the AURAMS and CMAQ chemical transport models (CTMs). Such harmonization provides confidence that the differences in model predictions result mainly from differences in the models themselves, as opposed to meteorological and/or emissions inputs or horizontal grid spacing and location. Despite the above harmonization, there are numerous differences in the detailed science processes, algorithms, and numerical methods in the models. In addition, there still exist differences in vertical coordinate systems, meteorological pre-processors, and emissions processing methods. All of these differences affect each model’s behaviour and performance. This paper presents the results of an operational intercomparison of AURAMS and CMAQ predictions of surface concentrations of O3, PM2.5 mass, and PM2.5 major species for July 2002. Both modelmeasurement and model–model comparisons are considered. The level of modelling harmonization achieved permits a focus on the differences in model performance and the related uncertainty arising mainly from the use of different sets of science algorithms. It was not possible within the limits of this paper to investigate all of the differences in model performance, but in some cases possible reasons are given. This paper will thus serve as the departure point for subsequent process-level-based diagnostic evaluations. 2. Modelling systems, model set-up, and evaluation metrics This section provides a brief overview of the AURAMS and CMAQ modelling systems, the model set-ups and methodology, and the evaluation metrics. More details can be found in Smyth et al. (2007). 2.1. AURAMS modelling system AURAMS is a regional air-quality modelling system with sizeand composition-resolved particulate-matter (PM) representation. The modelling system consists of three major components: a CTM, a meteorological model and pre-processor, and an emissions processor (Gong et al., 2006).

2.1.1. AURAMS CTM The AURAMS CTM simulates gaseous and PM species formation and evolution, as well as their interactions through gaseous, aqueous, and heterogeneous reactions and physical processes. Table 1 provides a summary of some process representations used in the AURAMS CTM. AURAMS uses a sectional approach to represent the atmospheric PM size distribution: 12 size bins, ranging from 0.01 to 40.96 mm in diameter, are considered, with bins 1–8 representing PM2.5, bins 9 and 10 coarse PM (i.e. PM10–PM2.5), and bins 11 and 12 representing aerosols with a size greater than 10 mm. PM composition is represented by nine major species: sulphate (SU), nitrate (NI), ammonium (AM), black carbon (EC), primary organic aerosol (PC), secondary organic aerosol (OC), crustal material (CM), sea-salt (SE), and particle-bound water (WA). The PM components are assumed to be internally mixed in each size bin. AURAMS v1.3.1b uses zero-gradient chemical lateral boundary conditions. Chemical initial conditions (ICs) were set to background concentration profiles that decreased exponentially with height, with the exception of O3, which increased with height. For this study, AURAMS v1.3.1b was run for the 696-h simulation period starting at 0100 UTC 1 July 2002 and ending at 0000 UTC 30 July 2002 using 12-h restarts (due to a 2-GB size limitation on the binary files created by the AURAMS pre-processor). July 2002 was chosen because (a) O3 and PM2.5 episodes are likely to occur in July and (b) a number of AURAMS 2002 1-year simulations are available and have been evaluated (e.g., Moran et al., 2008). 2.1.2. AURAMS meteorology The GEM model (Coˆte´ et al., 1998a,b) is an integrated forecasting and data assimilation system designed to meet Canada’s short- and medium-range forecasting needs (Environment Canada, 2005). For this study, GEM version 3.2.0 with physics version 4.2 was run using a variable-resolution global grid with finer 24-km uniform resolution over North America and a 30-h analysis cycle with the first 6-h of each cycle used as spin-up and discarded. GEM output was processed by the AURAMS meteorological preprocessor (MPP). The AURAMS MPP (AMPP) (Cousineau, 2003), developed at EnvCan, is an interface program that interpolates the GEM rotated latitude–longitude grid and hybrid vertical coordinate system to the AURAMS coordinate system. 2.1.3. AURAMS emissions 2.1.3.1. Emission inventories and emissions processing. The 2000 Canadian emission inventories for point, area, non-road mobile, and on-road mobile sources released by EnvCan in January 2005

Table 1 Summary of AURAMS and CMAQ process representations and algorithms selected for this study. Process

AURAMS v1.3.1b

CMAQ v4.6

Gas-phase chemistry SOA formation Aqueous-phase chemistry Inorganic heterogeneous chemistry Vertical diffusion

ADOM-II (Stockwell and Lurmann, 1989; Lurmann et al., 1986) Instantaneous Aerosol Yield (IAY) (Jiang, 2003, 2004) Gong (2002)

SAPRC-99 (Carter, 2000a,b) Gas/particle partitioning (Schell et al., 2001) Gery et al. (1989)

HETV (based on ISORROPIA) (Makar et al., 2003)

ISORROPIA (Nenes et al., 1998)

Turbulence kinetic energy closure; solved using a Laasonen implicit differencing scheme CHRONOS non-oscillatory, semi-Lagrangian, semi-implicit scheme (Pudykiewicz et al., 1997; Sirois et al., 1999) Gases: based on Wesley’s resistance parameterization (Zhang et al., 2002); particles: Zhang et al. (2001) Gong et al. (2006) Gong et al. (2006) Mass-consistency and mass-conservation corrections applied in tracer advection (Gong et al., 2004) Modified version of predictor–corrector method of Young and Boris (1977)

K-theory eddy diffusivity

Advection Dry deposition Wet deposition Cloud processes Mass conservation Chemical solver

Yamo global mass-conserving scheme (Yamartino, 1993) M3DRY module; modified RADM scheme with Pleim-Xiu land surface model (Xiu and Pleim, 2000) Binkowski and Shankar (1995) Met. model resolved clouds; sub-grid clouds using ACM Yamo global mass-conserving scheme Euler backward iterative (EBI) scheme modified for SAPRC-99

S.C. Smyth et al. / Atmospheric Environment 43 (2009) 1059–1070

were used. U.S. emissions for point, area, non-road mobile, and onroad mobile sources were taken from the U.S. EPA 2001 Clean Air Interstate Rule (CAIR) emission inventories released in July 2004 and available from the U.S. EPA emissions modelling clearinghouse (http://www.epa.gov/ttn/chief/emch/index.html). Mexican emissions were taken from the 1999 raw emissions inventories for point, area and mobile sources released with the 2001 EPA CAIR data. Biogenic emissions for Canada, the U.S., and Mexico were generated using BEISv3.09 algorithms and the Biogenic Emissions Landcover Database, Version 3 (BELD3) land-use data. AURAMS requires emissions to be in mass units of g s1 and contained in five separate emissions input files for major-point, minor-point, on-road mobile, non-mobile, and biogenic sources, respectively. The minor-point sources are those point sources that emit into the first model layer only. The major-point file contains speciated emissions and stack parameters which the AURAMS CTM uses to compute plume rise. All emissions for AURAMS were processed using SMOKE v2.2 (CEP, 2005) to generate hourly, speciated, and gridded emission fields. The selection of minor- and major-point sources is done within SMOKE using an elevated source configuration file that selects major sources based on certain selection criteria. For this study, the elevated sources for each country were selected as the top 800 SO2 emitters, top 700 NOx emitters, top 50 VOC emitters, top 50 PM2.5 emitters, or those stacks with a height greater than 70 m. Depending on the emissions sources, bulk VOC emissions are speciated into 11 ADOM-II lumped VOC species, and NOx emissions are speciated into nitric oxide (NO), nitrogen dioxide (NO2), and/or nitrous acid (HONO). However, most NOx emissions are speciated using the default speciation profile where 90% of NOx becomes NO and 10% NO2. SO2 emissions are speciated into SO2 or H2SO4 based on the emission source profile. CO and NH3 emissions are used directly. PM2.5 and PM10 emissions are speciated by assigning all PM2.5 emissions to the AURAMS model species AFF (aerosol fine fraction) and calculating AURAMS model species ACF (aerosol coarse fraction) by subtracting PM2.5 from PM10. Then the AURAMS CTM speciates AFF and ACF into the appropriate PM components and distributes to the appropriate size bins based on emission source as shown in Table 2. The assumed average PM chemical compositions

1061

are derived from an emission-weighted set of PM speciation profiles from the U.S. EPA clearinghouse. The assumed average PM size distributions are based on weighted averages of representative PM size distributions for different source types presented in Eldering and Cass (1996). 2.1.3.2. Modifications to AURAMS emissions processing. In order to further align the AURAMS and CMAQ simulations, some modifications were made to the ‘‘standard’’ methods used in AURAMS emissions processing. First, AURAMS requires a ‘‘representative’’ week of emissions for each simulated month whereas CMAQ requires daily emissions files. Therefore, to eliminate any possible differences between the daily and ‘‘representative’’ week styles of emissions processing, daily emissions files were used in AURAMS. Second, AURAMS generates biogenic emissions ‘‘on-line’’ in the AURAMS pre-processor, whereas biogenic emissions for CMAQ are generated ‘‘off-line’’ within SMOKE. In order to eliminate any potential differences, biogenic emissions for both models were generated using SMOKE. 2.2. CMAQ modelling system Similar to AURAMS, CMAQ (Byun and Ching, 1999; Byun and Schere, 2006) consists of three major components: a CTM, a meteorological model and pre-processor, and an emissions processing system. CMAQ also contains three other pre-processors for initial conditions, boundary conditions, and photolysis rates. In CMAQ, particle size distributions are represented as a superposition of three log-normal sub-distributions or modes: Aitken or i-mode; accumulation or j-mode; and coarse or k-mode (Binkowski and Roselle, 2003). The PM concentrations generated by CMAQ cover the complete log-normal distributions and are not directly comparable with size-resolved measurement data. Therefore, the PMx post-processor developed at the National Research Council (NRC) (Jiang and Yin, 2001; Jiang et al., 2006) is used to calculate particle size distribution parameters and PM concentrations within required particle size ranges. 2.2.1. CMAQ CTM Table 1 provides a summary of some process representations used by CMAQ. In addition to the gaseous species, CMAQ considers 13 components to contribute to PM compositions of different

Table 2 Fractional assignment of AURAMS aerosol fine fraction (AFF) and aerosol coarse fraction (ACF) to AURAMS PM major species and size bins (mm). AURAMS species

Emission categories Mobile

SU NI AM PC OC EC CM SE Size bin 0.01–0.02 0.02–0.04 0.04–0.08 0.08–0.16 0.16–0.32 0.32–0.64 0.64–1.28 1.28–2.56 2.56–5.12 5.12–10.2

Non-mobile

Minor-point

Major-point

AFF

ACF

AFF

ACF

AFF

ACF

AFF

ACF

0.014 0.002 0 0.342 0 0.556 0.086 0

0.014 0.002 0 0.342 0 0.556 0.086 0

0.027 0.002 0 0.256 0 0.056 0.658 0

0.027 0.002 0 0.256 0 0.056 0.658 0

0.171 0.004 0 0.162 0 0.027 0.636 0

0.171 0.004 0 0.162 0 0.027 0.636 0

0.171 0.004 0 0.162 0 0.027 0.636 0

0.171 0.004 0 0.162 0 0.027 0.636 0

0 0.02 0.10 0.35 0.35 0.14 0.04 0

0 0 0.01 0.05 0.05 0.02 0.01 0.07 0 0

0.01 0.15 0.44 0.25 0.10 0.04 0.01 0 0.29 0.50

0.01 0.15 0.44 0.25 0.10 0.04 0.01 0 0 0

0 0

1062

S.C. Smyth et al. / Atmospheric Environment 43 (2009) 1059–1070

particle size modes: sulphate (ASO4), nitrate (ANO3), ammonium (ANH4), primary and secondary anthropogenic organics (AORGPA and AORGA), secondary biogenic organics (AORGB), elemental carbon (AEC), other fine PM mass (A25), aerosol water (AH2O), soilderived dust (ASOIL), other coarse mass (ACORS), and sea-salt, made up from chlorine (ACL), sodium (ANA), and coarse mode sulphate (ASO4K). CMAQ uses time-independent chemical lateral boundary conditions for most species. For this modelling study, the default boundary conditions were used for all species. Default values were also used for the chemical ICs in the CMAQ simulation. In addition to modelling sea-salt emissions, as is also done in AURAMS, fine-mode sea-salt species are treated in CMAQ by the ISORROPIA (Nenes et al., 1998) inorganic chemistry and equilibrium module, whereas sea-salt species in the coarse mode are treated as inert tracers. AURAMS does not consider sea-salt chemistry. CMAQ v4.6 was run for the complete simulation period using 24-h restarts for the convenience of dealing with daily files. 2.2.2. CMAQ meteorology The same GEM-generated meteorological fields used by the AURAMS MPP were also used for CMAQ. The GEM meteorological fields were first processed through GEM-MCIP (Yin, 2004; Smyth et al., 2005, 2006), an MPP developed at NRC based on the U.S. EPA’s MCIP program. An extension was added to MCIP v3.1 in order to enable the use of GEM-modelled meteorological fields as input to SMOKE or CMAQ. 2.2.3. CMAQ emissions In this study, CMAQ uses the same raw emissions input files as AURAMS, and the same emissions processor was used to process the emissions. CMAQ requires emissions in molar units of mol s1 for gaseous species and in mass units of g s1 for PM for each hour of the simulation period and all emission sources to be in one integrated emissions file. SMOKE is used to calculate the vertical distribution of major point source emissions in CMAQ. For maximum compatibility with the method used in AURAMS, CMAQ point sources were selected in the same way and then processed using hourly meteorological data and stack parameters to calculate plume rise for all point sources based on the Briggs algorithm (Briggs, 1975, 1984). Speciation of inventory emissions into the appropriate SAPRC99 modelled species occurs within SMOKE where 30 modelled VOC species are used. Just as in AURAMS, CO and NH3 emissions are used directly. Differing somewhat from NOx speciation in AURAMS, 95% of NOx emissions are assigned to NO with the remaining 5% to NO2. SO2 emissions are speciated into SO2 or H2SO4 based on emission source profiles. In CMAQ, the speciation of PM emissions in SMOKE is more rigorous than in AURAMS. Based on the numerous detailed emission source profiles contained within SMOKE, PM2.5 emissions in CMAQ are speciated into fine particles (PMFINE), primary organics (POA), sulphate (PSO4), nitrate (PNO3), and/or elemental carbon (PEC). PMC emissions are calculated by subtracting PM2.5 emissions from PM10. Since CMAQ represents particle size distributions using three log-normal size modes, the CMAQ CTM further assigns PM emissions to the appropriate mode. PSO4, PNO3, and PMFINE are assigned to the j-mode of ASO4, ANO3, and A25, respectively. For POA and PEC emissions, 99.9% are assigned to the jmode of AORGPA and AEC, respectively, with the remaining 0.1% assigned to the i-mode. Of PMC emissions, 90% are assigned to ASOIL and the remaining 10% are assigned to ACORS, where both ASOIL and ACORS exist only in the k-mode. In CMAQ, emitted particles are assumed to have fixed geometric mean

diameters of 0.03, 0.30, and 6.00 mm and geometric standard deviations of 1.7, 2.0, and 2.2 for the three log-normal volume/ mass distributions, respectively. 2.3. AURAMS and CMAQ modelling domain 2.3.1. Common horizontal domain and map projection Both the AURAMS and CMAQ model runs were set-up on a North American continental domain extending from northern Mexico to northern Canada and including the contiguous United States. In order to align the simulations in this study, CMAQ was configured to use the same polar stereographic (PS) map projection that was used in AURAMS. The EnvCan ‘‘cont42’’ horizontal PS domain was adopted, consisting of 150 columns and 106 rows, with 42-km grid resolution. The PS projection is defined with a central meridian of 100 W and a latitude of true scale at 60 N. The center of the Cartesian coordinate system is at 90 N and 100 W (i.e., North Pole) and the origin of the grid is located at (2776.2 km, 7392 km). Fig. 1 shows the modelling domain. 2.3.2. AURAMS vertical layers AURAMS uses the Gal-Chen vertical coordinate system (GalChen and Somerville, 1975) to define the vertical layers used in the model:

 z ¼ Z þ H0

HZ H

 (1)

Fig. 1. AURAMS and CMAQ modelling domain with locations of (a) O3 measurement sites; (b) PM2.5 measurement sites with usable July 2002 data indicated (black circles denote total PM2.5 sites and white squares denote speciated PM2.5 sites).

S.C. Smyth et al. / Atmospheric Environment 43 (2009) 1059–1070

where z is the height above sea-level, Z the Gal-Chen level in meters, H is 30,000 m (model top in GEM), and H0 is the height of the topography in meters. AURAMS was run with 30 vertical levels and a model top of approximately 25 km, but only the first 20 levels of data were output corresponding to a top output height of approximately 5000 m. 2.3.3. CMAQ vertical layers CMAQ uses sigma (s) levels based on a reference hydrostatic pressure to define the vertical layers, as

s¼

p0  ptop ps0  ptop

!

Table 4 Comparison statistics of surface meteorological fields generated by AURAMS meteorological pre-processor and GEM-MCIP (n ¼ 11 066 400). Meteorological field Pressure (hPa)

Model

AURAMS MPP GEM-MCIP Temperature AURAMS (K) MPP GEM-MCIP Specific humidity AURAMS (kg kg1) MPP GEM-MCIP

MD

NMD (%)

967.5

0.34

0.04 1.3

0.13

0.999

967.9 294.0

0.07

0.03 1.0

0.35

0.961

294.1 0.0112 1.5  104

1.4

MAD

NMAD r2 (%)

Mean

3.5  104 3.2

0.988

0.0110

(2)

where p0 is the hydrostatic pressure of the reference atmosphere, ptop the model top pressure, and ps0 a reference surface pressure that is constant in time, but varies with terrain height. CMAQ uses 16 vertical levels (15 layers) with a model top at 100 hPa or approximately 15 km. All analyses in this paper were done for the first layer, which has an average height of 14.8 m for AURAMS and 37.7 m for CMAQ. Heights of other layers are omitted here for brevity. 2.4. Statistical metrics The statistical metrics presented in Table 3 were used to evaluate the performance of AURAMS and CMAQ results against AQ measurement data. They were also used for intercomparison of meteorological data used by the models. 3. Comparison of meteorology and emissions 3.1. Meteorology In order to understand the differences in the meteorological input data caused by the different MPPs, surface-level pressure, temperature, and specific humidity fields generated by the AMPP and GEM-MCIP were compared. Table 4 presents comparison statistics of the meteorological fields generated by AMPP and GEM-MCIP for all hours and all surface grid cells. The results reveal that there are slight differences in the MPPs resulting in normalized mean absolute differences (NMADs) of 0.13%, 0.35%, and 3.2% for surface pressure, temperature, and specific humidity, respectively. 3.2. Emissions As a result of the modifications to AURAMS, biogenic emissions for the two AQ models were identical before speciation. Gaseous anthropogenic emissions were almost identical except for differences arising from SOx, VOC, and NOx speciation. Bulk PM

emissions were the same between AURAMS and CMAQ, but the methods used to speciate the PM emissions into the various size and chemical sub-components were different. Although the bulk emissions from elevated point sources were the same, the distribution of the emissions into the model vertical layers was different along with the structure of the vertical layers. 4. Comparative performance of AURAMS and CMAQ against measurements Results from both AURAMS and CMAQ for the July 2002 simulation period were evaluated against surface-level measurement data for O3, total PM2.5 mass, and PM2.5 sulphate (SO4), nitrate (NO3), ammonium (NH4), elemental carbon (EC), and total organic aerosol (TOA). When comparing the measured total PM2.5 mass with model results, modelled dry PM2.5 mass was used. For the TOA performance evaluation, the sum of PM2.5 OC and PC was used for AURAMS and the sum of PM2.5 AORGPA, AORGA, and AORGB was used for CMAQ. 4.1. Air-quality measurement data Air-quality measurement data for the performance evaluation were obtained from several Canadian and U.S. AQ measurement networks as summarized in Table 5. Fig. 1a shows the locations of O3 measurement stations while Fig. 1b shows the locations of the total and speciated PM2.5 stations. The performance of AURAMS and CMAQ were evaluated in several ways. The overall performance statistics were calculated for all measurement sites and evaluated species. In addition to the statistics over all hours for all species, 1-h daily maximum and minimum concentrations for O3 were also evaluated. Domain-wide O3 and PM2.5 diurnal patterns were also compared with measurement data. This latter analysis was not done for the PM2.5 major species, as only daily averaged measurements for these species were available. 4.2. Ground-level ozone (O3)

Table 3 Statistical metrics used for performance evaluation. Mean bias (MB); mean difference (MD) Normalized mean bias (NMB); normalized mean difference (NMD) Mean error (ME); mean absolute difference (MAD) Normalized mean error (NME); normalized mean absolute difference (NMAD) Coefficient of determination (r2)

1063

(1/n)Si(mi  oi)a [Si(mi  oi)/Sioi]  100% (1/n)Sijmi  oij [Sijmi  oij/Sioi]  100%

0

r2 ¼ 1  SSE/SST; SSE ¼ Si(mi  mi)2; 0 SST ¼ Sim2i  (1/n) (Simi)2; mi ¼ a þ boi

a n is the total number of data pairs and i ¼ 1, 2, ., n; for performance evaluations, o is the measured concentration and m is the modelled concentration; for meteorological pre-processor intercomparison, o is GEM-MCIP and m is AURAMS MPP.

4.2.1. Overall performance Table 6 shows the performance statistics for AURAMS- and CMAQ-modelled O3 for all hours except the first 48, which were used for model spin-up. Both models over-predicted 1-h O3 concentrations on average. AURAMS over-predicted to a lesser extent, with an MB of 7.4 ppbV and an NMB of 20.7% versus 16.5 ppbV and 46.4% for CMAQ. However, the errors for the two models were closer, with AURAMS and CMAQ having NMEs of 46.9% and 54.2%, respectively. AURAMS’ smaller bias is associated with more cancellation of negative and positive biases. Coefficient of determination (r2) values were similar: 0.395 for AURAMS versus 0.433 for CMAQ. A recent study by Yu et al. (2007) reported comparable

1064

S.C. Smyth et al. / Atmospheric Environment 43 (2009) 1059–1070

Table 5 Environment Canada and U.S. EPA measurement networks used for performance evaluation. network

Administrator

Pollutants

Sampling frequency

Sampling period

Units

# of sites

NAPS

Environment Canada

Hourly Hourly 1 day-in-6 OR 1 day-in-3

1-h 1-h 24-h

ppbV mg m3 mg m3

160 92 17

AQS

U.S. EPA

STN

U.S. EPA

O3 Total PM2.5 PM2.5 SO4 PM2.5 NO3 PM2.5 NH4 O3 Total PM2.5 Total PM10 PM2.5 SO4 PM2.5 NO3 PM2.5 NH4 PM2.5 EC PM2.5 OC

Hourly Hourly Hourly 1 day-in-3 OR 1 day-in-6

1-h 1-h 1-h 24-h

ppbV mg m3 mg m3 mg m3

1087 262 152 204 187 204 205 205

values for the 12-km Eta-CMAQ modelling system over central and eastern North America for a different summer period (July and early August 2004): NMB of 40.9%, NME of 54.8%, and r2 of 0.41. 4.2.2. Diurnal patterns and daily maximum/minimum concentration performance A time-series plot of observed and AURAMS- and CMAQ-predicted 1-h O3 concentrations averaged over all measurement sites and the corresponding grid cells, respectively, is shown in Fig. 2a. Both AURAMS and CMAQ performed well in predicting the diurnal pattern of O3 concentration. Both models over-predicted the daily maximums, with CMAQ daily maximums generally higher than AURAMS’. However, AURAMS did very well on average in predicting the nighttime minimum concentrations, whereas CMAQ over-predicted the nighttime minimums by approximately 15–20 ppbV. Daily maximum 1-h O3 performance statistics were calculated and are presented in Table 6. The statistics show that AURAMS had a slightly lower NMB of 12.6% in predicting 1-h daily maximum concentrations versus 17.8% for CMAQ. However, in terms of error, CMAQ had a slightly lower NME of 25.2% versus 28.1% for AURAMS, and for r2 the CMAQ value of 0.488 was higher than the AURAMS value of 0.342. Note that for the July and early August 2004 period, Yu et al. (2007) reported NMB, NME, and r2 values of 16.4%, 25.3%, and 0.37, respectively, for daily maximum 1-h O3. Performance statistics for daily minimum 1-h O3 concentrations are also displayed in Table 6. AURAMS did better in predicting daily minimums with an NMB of 40.4% and NME of 94.4% versus 178% and 186% for CMAQ. These results suggest that CMAQ’s poorer performance in predicting overall hourly O3 is mainly due to its inability to correctly predict nighttime concentrations.

4.3. Total PM2.5 mass concentration 4.3.1. Overall performance Performance statistics for surface-level PM2.5 concentrations are presented in Table 7. The results reveal that both models underpredicted total PM2.5 mass, although AURAMS’ under-prediction

Fig. 2. Time-series comparison of observed and AURAMS- and CMAQ-modelled surface-level (a) O3; (b) total PM2.5 hourly concentrations averaged over all measurement sites or grid cells containing the sites.

was less than CMAQ’s, with an NMB of 10.4% for AURAMS versus 65.2% for CMAQ. However, just as with O3, the models’ errors were closer, with NMEs of 68.0% and 70.6% for AURAMS and CMAQ, respectively. For both models, r2 values were low: 0.073 for AURAMS and 0.152 for CMAQ. Note that previous studies have also found that CMAQ underpredicts PM2.5 concentrations during the summer (McKeen et al., 2007; Mathur et al., 2008). Fig. 2b shows a spatially averaged time-series plot of observed and AURAMS- and CMAQ-predicted PM2.5 concentrations for July 2002. The AURAMS time-series is generally closer to the measured concentrations. The under-prediction of PM2.5 can be partly attributed to the exclusion of aerosol water from the modelled PM2.5 mass. For both models the modelled dry PM2.5 is used since the amount of aerosol water in the reported PM2.5 measurement data is not available. The modelled dry PM2.5 represents the lower bound values of PM2.5 mass concentrations. In addition, the under-prediction of both models may be related to station representativeness, as approximately 85% of all PM2.5 measurement sites were located in urban

Table 6 AURAMS and CMAQ performance statistics for surface-level 1-h O3, daily maximum 1-h O3, and daily minimum 1-h O3. Quantity

Model

No. sites

n

Meas. mean (ppb)

Mod. mean (ppb)

MB (ppb)

NMB (%)

ME (ppb)

NME (%)

r2

O3

AURAMS CMAQ AURAMS CMAQ AURAMS CMAQ

1245

774 946

35.6

1245

20 599

60.8

1245

20 599

11.7

43.0 52.2 68.4 71.6 16.5 32.6

7.4 16.5 7.6 10.8 4.7 20.9

20.7 46.4 12.6 17.8 40.4 178

16.7 19.3 17.1 15.3 11.1 21.8

46.9 54.2 28.1 25.2 94.4 186

0.395 0.433 0.342 0.488 0.104 0.086

Daily max. O3 Daily min. O3

S.C. Smyth et al. / Atmospheric Environment 43 (2009) 1059–1070

1065

Table 7 AURAMS and CMAQ performance statistics for surface-level 24-h total PM2.5, PM2.5 sulphate, nitrate, ammonium, elemental carbon, total organic aerosols, and total PM2.5 minus PM2.5 sea-salt. PM2.5 species

Model

No. sites

n

Meas. mean (mg m3)

Mod. mean (mg m3)

MB (mg m3)

NMB (%)

ME (mg m3)

NME (%)

r2

Total PM2.5

AURAMS CMAQ AURAMS CMAQ AURAMS CMAQ AURAMS CMAQ AURAMS CMAQ AURAMS CMAQ AURAMS CMAQ

350

211 141

14.4

221

1229

5.0

204

1112

0.90

221

1230

1.6

205

1180

0.52

205

1180

10.6

350

211 141

14.4

12.9 5.0 5.5 2.3 1.9 0.23 1.6 0.80 0.28 0.32 4.8 0.99 12.0 5.0

1.5 9.4 0.45 2.6 1.0 0.68 0.0 0.83 0.24 0.20 5.8 9.6 2.4 9.5

10.4 65.2 8.9 52.6 113 74.9 0.2 51.0 46.5 39.0 54.8 90.6 16.5 65.5

9.8 10.2 3.1 2.9 1.5 0.73 0.88 0.94 0.30 0.31 6.4 9.6 9.8 10.2

68.0 70.6 62.2 57.2 169 81.4 53.8 57.5 57.5 59.5 60.8 90.6 67.9 70.7

0.073 0.152 0.355 0.522 0.400 0.325 0.432 0.452 0.170 0.158 1  105 0.007 0.080 0.154

SO4 NO3 NH4 EC TOA Total PM2.5 no sea-salt

areas, including in the vicinity of major highways. The average pollutant concentrations within the relatively large 42-km model grid cells are likely to be lower than the high, localised PM concentrations expected around urban centres, contributing to the under-predictions.

4.4. Mass concentrations of major PM2.5 species 4.4.1. Sulphate (SO4), nitrate (NO3), and ammonium (NH4) Table 7 also shows the performance statistics for surface-level PM2.5 SO4, NO3, and NH4. AURAMS over-predicted the concentrations of all three species whereas CMAQ under-predicted. AURAMS bias was very low for both SO4 and NH4 with NMBs of 8.9% and 0.2%, respectively, whereas CMAQ had an NMB of 52.6% for SO4 and 51.0% for NH4. However, as previously seen for O3, even though the AURAMS bias was better, in terms of error, the two models performed comparably with NMEs of 62.2% and 57.2% for AURAMS and CMAQ SO4, respectively, and NMEs of 53.8% and 57.5% for AURAMS and CMAQ NH4. For NO3, AURAMS over-predicted with an NMB of 113% whereas CMAQ under-predicted with an NMB of 74.9%. In terms of error, the CMAQ NME value of 81.4% was much lower than the AURAMS value of 169% but the AURAMS r2 value of 0.400 was higher than the CMAQ value of 0.325. 4.4.2. Elemental carbon (EC) and total organic aerosol (TOA) Relative to the evaluation of other PM species, there is an additional uncertainty associated with the evaluation of TOA concentrations. To compare measured total organic carbon (TOC) concentrations with modelled TOA concentrations, the measured concentrations need to be scaled by a carbon multiplication factor (CMF) to account for the non-carbon elements not included in the measurement concentrations (Turpin and Lim, 2001). A CMF represents the average molecular weight per carbon weight. As suggested by Turpin and Lim (2001) a CMF of 1.6 was used for urban sites and a CMF of 2.1 was used for non-urban sites. Table 7 presents the performance statistics for ground-level PM2.5 EC and TOA. In contrast to the non-carbonaceous PM2.5 species previously discussed, both models under-predict EC and TOA. For EC, CMAQ’s NMB of 39.0% is better than the NMB of 46.5% for AURAMS. However, in terms of error, the models’ performances are similar with NMEs of 57.5% and 59.5% for AURAMS and CMAQ, respectively. Since EC is a primary pollutant and PM emissions were the same for the two modelling systems, one would expect the concentrations of EC to be similar if the science in the two models is comparable. However, as previously pointed out, the bulk PM2.5

emissions were the same, but not the speciation of those emissions into the various PM components, which could result in different EC emissions. By applying the source-based speciation method used in the AURAMS CTM to the bulk PM2.5 emissions, it was calculated that approximately 22 kt of EC was emitted in AURAMS. In contrast, approximately 32 kt was emitted in CMAQ. This partly explains the higher EC concentrations present in the CMAQ results. Differences in advection, deposition, and mass-conservation algorithms could also contribute to the concentration differences. Both models significantly under-predicted TOA concentrations. However, the AURAMS modelled station-mean was over four times larger than the CMAQ value, and AURAMS performed better both in terms of bias and error with an NMB of 54.8% and NME of 60.8% versus 90.6% and 90.6% for CMAQ. This difference in TOA performance is likely due at least in part to differences in the secondary organic aerosol (SOA) schemes (cf. Table 1). Both models also showed very poor r2 values for TOA whereas the values for PM2.5 EC, SO4, NO3, and NH4 were somewhat better (cf. Table 7). Note that both models’ inability to accurately predict the organic component concentrations likely contributed to the poor correlation coefficients for both models for total PM2.5 (cf. Table 7), since organic aerosols make up a large portion of the measured PM2.5 mass. The large under-prediction of PM2.5 TOA could be attributed in part to the uncertainty in the CMFs used to modify the OC measurements to account for non-carbon mass. In addition, forestfire emissions were not included in the emissions data sets due to Canadian forest-fire data not being readily available at the time of the study. This omission could affect not only modelled TOA concentrations, but total PM mass as well. During the modelling period there were many active forest fires throughout North America, including many in northern Que´bec during the July 3–11 period (e.g., Sapkota et al., 2005). The resulting elevated TOA levels in eastern Canada and the northeastern U.S. contributed to the observed peak in total PM2.5 mass during this period as shown in Fig. 2b. If forest-fire emissions were included for the entire domain, it is likely that the performance of total PM mass and PM species would improve for both models.

5. Model-to-model comparison In this section, spatial and temporal plots are presented for surface-level O3 and total PM2.5, while only temporal plots are presented for PM2.5 SO4, NO3, NH4, EC, primary organic aerosol (POA), SOA, other aerosols (e.g., crustal material), and sea-salt. Spatial plots of PM2.5 species are presented in the Appendix. In

1066

S.C. Smyth et al. / Atmospheric Environment 43 (2009) 1059–1070

constructing the spatial plots, the first 48-h of data was discarded as spin-up.

5.1. Ground-level ozone (O3) CMAQ and AURAMS ground-level O3 concentrations were averaged over the modelling domains for each hour and plotted in the time-series plot shown in Fig. 3a. The plot shows that the two models start with slightly different O3 ICs, with the AURAMS average IC at approximately 45 ppbV and CMAQ’s at 35 ppbV. The model results then diverge, with AURAMS average O3 concentrations decreasing to between 20 and 30 ppbV while CMAQ results increase to between 40 and 50 ppbV. Both models exhibit similar diurnal patterns, but the range between peaks and lows is larger for AURAMS.

Temporally averaged spatial plots of AURAMS and CMAQ O3 concentrations are shown in Fig. 4a. The plots show that averaged CMAQ O3 concentrations are generally higher across the domain. However, the two models exhibit very similar spatial distribution patterns, with higher concentrations over the U.S. northeast, Ohio River valley, and southern California. The lowest average concentrations in each model occur in northern Canada and at the boundaries over the Atlantic and Pacific Oceans. Overall, the two models produce similar O3 diurnal and spatial patterns. However, the CMAQ-predicted concentrations are generally higher than the AURAMS concentrations. One possible reason for these differences is the different chemical lateral boundary conditions used by the two models. In the CMAQ simulation, the default O3 time-invariant boundary condition profile was used (concentrations of 35 ppbV on the north and west boundaries and 30 ppbV on the south and east boundaries). Results from

Fig. 3. Time-series comparison of AURAMS- and CMAQ-modelled (a) O3; (b) total PM2.5; (c) PM2.5 SO4; (d) PM2.5 NO3; (e) PM2.5 NH4; (f) PM2.5 EC; (g) PM2.5 POA; (h) PM2.5 SOA; (i) other PM2.5 aerosols; (j) PM2.5 sea-salt hourly concentrations averaged over all surface-level grid cells.

S.C. Smyth et al. / Atmospheric Environment 43 (2009) 1059–1070

1067

Fig. 4. Spatial distributions of time-averaged concentrations of AURAMS- and CMAQ-modelled (a) O3 and (b) PM2.5.

Tarasick et al. (2007) suggest that these boundary conditions likely contributed to the higher O3 concentrations in the CMAQ results compared to the zero-gradient boundary condition used in this version of AURAMS. 5.2. Total PM2.5 mass concentrations Fig. 3b presents the time-series plots of surface-level PM2.5 concentrations. The AURAMS concentrations are much higher than those exhibited in the CMAQ results, due in large part to much higher AURAMS PM2.5 sea-salt concentrations (see Section 5.3.3). In addition, due to the large influence of PM2.5 sea-salt on AURAMS total PM2.5, the AURAMS total PM2.5 concentrations tend to build up over several days to reach a peak, and then decline over several days to reach a valley. This pattern repeats several times during the simulation period. In contrast, the CMAQ results tend to show well-defined daily maximums and minimums, with less build up in concentration. Fig. 4b shows spatial distributions of AURAMS- and CMAQmodelled PM2.5 mass concentrations. AURAMS has much higher average PM2.5 concentrations across the domain, with both models exhibiting areas of higher PM2.5 concentration in southern California and the eastern U.S. AURAMS PM2.5 concentrations are very high over large areas of the Atlantic and Pacific oceans due to much higher sea-salt aerosol concentrations. This is not the case in the CMAQ spatial plot. 5.3. Mass concentrations of major PM2.5 species 5.3.1. Sulphate (SO4), nitrate (NO3), and ammonium (NH4) Temporal plots are presented in Fig. 3c–e for AURAMS- and CMAQ-modelled PM2.5 SO4, NO3, and NH4. In Fig 3c, it can be seen that AURAMS and CMAQ average SO4 concentrations have a similar low-frequency pattern over the simulation period, with both

increasing from the start of the simulation to a peak around 9 July, followed by a decrease in concentration over several days and then another increase in concentration over several days. CMAQ also displays a more pronounced diurnal variation. AURAMS average NO3 concentrations are generally much higher than CMAQ values as shown in Fig. 3d. Both models show steep increases and decreases in NO3 concentration and similar diurnal profiles. AURAMS NH4 concentrations are generally higher than CMAQ, with both models exhibiting strong diurnal patterns. The higher AURAMS NH4 concentrations are a consequence of the higher AURAMS concentrations of NO3 and SO4. The differences in PM2.5 SO4, NO3, and NH4 concentrations may arise due to differences in the models’ treatments of gas-phase, aqueous-phase, and heterogeneous chemistry and dry and wet deposition. 5.3.2. Elemental carbon (EC), primary organic aerosol (POA), secondary organic aerosol (SOA) Fig. 3f shows that the temporal variations in PM2.5 EC concentrations are quite similar between AURAMS and CMAQ, with the AURAMS results showing a more consistent diurnal cycle than the CMAQ results. The temporal plot for spatially averaged POA (Fig. 3g) reveals that AURAMS concentrations are higher and vary more than the CMAQ concentrations. AURAMS results also show a more defined diurnal pattern than the CMAQ results. Fig. 3h shows that AURAMS SOA concentrations are much higher than CMAQ’s. The CMAQ results show well-defined peaks and lows throughout the simulation, whereas the AURAMS results show double peaks on some days. The different SOA schemes implemented in AURAMS and CMAQ are likely contributors to the major SOA concentration differences

1068

S.C. Smyth et al. / Atmospheric Environment 43 (2009) 1059–1070

between the models. For each modelling time step, AURAMS calculates instantaneous aerosol yields (IAYs) using the equations in Jiang (2003, 2004) and then estimates new SOA formation based on the IAY values and the amount of gas-phase semi-volatile organic compounds (SVOC) formed during the time step. In CMAQ, the approach of Schell et al. (2001) is used to re-partition all accumulated and newly formed SVOCs between the gas and particle phases. Since Jiang’s IAY equations and Schell’s approach are both based on the same gas/particle absorptive partitioning model, they should generate similar results when a system meets the gas/ particle partitioning equilibrium condition. When a system is far from the equilibrium condition, though, Schell’s approach possesses a theoretical advantage due to its capability to restore the partitioning equilibrium, which may lower SOA concentrations if SVOCs in the aerosol phase re-evaporate into the gas phase. However, a correct implementation of Schell’s approach in an AQ model requires detailed modelling and tracking of all gaseous and particle SVOC species throughout all of the atmospheric processes. In CMAQ v4.6, individual gas-phase SVOCs are not tracked outside the SOA module. This could introduce more errors than the simplified IAY parameterization used in AURAMS. As shown in Table 7, TOA is underestimated more by CMAQ than by AURAMS, possibly due to the combination of SVOC evaporation from the particle phase and the lack of tracking of all SVOC species throughout other atmospheric processes in CMAQ. In addition to the above differences, CMAQ used six pseudo species to track the reacted amount of gaseous SOA precursors, which include high-molecular-weight alkanes, olefins, cresol, highyield aromatics, low-yield aromatics, and monoterpenes. AURAMS SOA precursors included isoprene as well as alkanes, toluene, higher aromatics, anthropogenic alkenes, and monoterpenes. Neither model considered sesquiterpene contributions or NOxconcentration influences. 5.3.3. Other PM2.5 and sea-salt aerosols In the AURAMS results, ‘‘other aerosols’’ consist of PM2.5 crustal material (CM25), while in the CMAQ results it consists of the summation of the fractions of A25, ACORS, and ASOIL that are smaller than the 2.5 mm size cut-off. Fig. 3i shows the temporal plots of AURAMS and CMAQ ‘‘other PM2.5’’ concentrations averaged over all surface-level grid cells. The concentrations are very similar, with slightly higher peaks and a greater range between peaks and lows in AURAMS. Both models seem to show a double peak during most days of the simulation with the CMAQ double peaks more clearly defined. Enhanced road-dust emissions during morning and evening rush hours likely contribute to these double peaks. The temporal variation of PM2.5 sea-salt is displayed in Fig. 3j, which shows that the AURAMS PM2.5 sea-salt concentrations are much higher than CMAQ’s. The AURAMS concentrations tend to build up over several days and then decrease, whereas the CMAQ concentrations tend to be more constant and much lower throughout the simulation. Comparison of Fig. 3b and j shows the high influence that AURAMS PM2.5 sea-salt has on total PM2.5 both in terms of concentration level and temporal patterns. The differences in sea-salt concentrations are related to the different sea-salt emission flux functions used by the models. AURAMS uses the function from Monahan et al. (1986), while the function used in CMAQ is taken from Smith and Harrison (1998). 5.3.4. PM2.5 composition The domain-average composition of total PM2.5 mass for each model was determined by calculating the average concentration of each PM2.5 major species over all surface-level grid cells and over the last 648-h of the simulation period and then determining the percentage contribution of each species to the total mass. Fig. 5a

Fig. 5. Average PM2.5 composition over last 648-h of simulation period and all grid cells (a) including sea-salt; (b) excluding sea-salt.

shows the average percentage PM2.5 composition for AURAMS and CMAQ, respectively. Sea-salt contributes 50.2% to the average AURAMS PM2.5 mass of 8.24 mg m3, while the largest contribution to the CMAQ results is SO4, which contributes 44.7% to the total mass of 1.77 mg m3. In addition, sea-salt aerosols only make up 6.2% of the PM2.5 mass in the CMAQ results. The relative contributions of the major species to total PM2.5 mass are very different in each model. If sea-salt is excluded from the analysis, as shown in Fig. 5b, the largest contributor to AURAMS is PM2.5 SOA, followed in descending order by SO4, other PM2.5 aerosols, NH4, NO3, POA, and EC. For CMAQ, the largest contributor to total PM2.5 is SO4, followed by other PM2.5, NH4, SOA, POA, EC, and NO3. Note that this comparison is for all grid cells, which reflects the large impact on the overall PM2.5 composition of the very high seasalt concentrations present over the oceans in AURAMS. When averaged over land grid cells only (composition diagram not shown), sea-salt aerosols make up 13.9% of the average PM2.5 mass of 6.40 mg m3 in AURAMS and only 2.0% of the average PM2.5 mass of 2.23 mg m3 in CMAQ. This comparison shows the large influence that sea-salt aerosols have even over land in AURAMS. Average CMAQ PM2.5 concentrations over land grid cells are higher than the concentrations over all gird cells. The opposite is true for AURAMS, which generated higher PM2.5 concentrations over the ocean due to the very high sea-salt concentrations there. Since sea-salt aerosols contribute a relatively high percentage to PM2.5 mass, it was hypothesized that the high sea-salt concentrations in the AURAMS results were contributing to the better overall performance for total PM2.5. To test this, total PM2.5 excluding the sea-salt aerosol component was evaluated against measurement data for both models. The statistics shown in Table 7 reveal that the AURAMS station mean concentration decreased by 0.9 mg m3, whereas the CMAQ station mean concentration decreased by only 0.05 mg m3. These results show that AURAMS sea-salt aerosols do contribute more to the overall PM2.5 performance than CMAQ, but not so much as to change the overall PM2.5 performance of AURAMS relative to CMAQ.

S.C. Smyth et al. / Atmospheric Environment 43 (2009) 1059–1070

6. Summary and conclusions In this study AURAMS and CMAQ were comparatively evaluated by aligning input meteorological fields and emissions for the July 2002 period as well as horizontal modelling domains. The alignment helped to reveal some similarities in the spatial and temporal patterns of some pollutants, and to identify some differences between AURAMS and CMAQ model results caused by the differences in their respective process parameterizations and numerical algorithms. In general, the two modelling systems showed similar levels of error for 1-h O3 concentration, with AURAMS having a lower bias. CMAQ’s higher bias was mainly due to its inability to correctly predict nighttime lows with differences in the treatments of O3 lateral boundary conditions also affecting the results. Both models performed similarly in predicting daily maximum O3 concentrations. AURAMS and CMAQ also showed similar levels of error for total PM2.5 mass and for most PM2.5 major species. However, AURAMS had lower bias for all investigated species, except for PM2.5 NO3 and EC. This enhanced bias performance was due in part to more cancellation of positive and negative biases as indicated by the similar levels of error. A model-vs.-model comparison of PM2.5 species predictions provided valuable information about model behaviour that was complementary to the model-vs.-measurement evaluation. This study reflects the best effort up to now to closely align many operational aspects of the two modelling systems. However, due to the complexity of the model structures and the numerous interconnected science processes, it is difficult to systematically assess the contributions of individual science processes to the differences in model performance. Improved modularity at the process level would make such scientific process assessments more feasible. Acknowledgements The authors acknowledge Radenko Pavlovic of EnvCan for his efforts in transferring various data sets and AURAMS code to NRC. We also acknowledge Wanmin Gong and Sylvain Me´nard of EnvCan for their contributions to this project and the insightful comments of two anonymous reviewers. Several organizations provided information and/or funding support to this project and their contributions are appreciated. The Pollution Data Division of EnvCan supplied the Canadian raw emissions inventories. The U.S. EPA provided U.S. raw emissions data and CEP distributed CMAQ, MCIP, and SMOKE. Colorado State University developed the VIEWS database and EnvCan operates the NAtChem database, both of which were used to obtain measurement data. Funding for this project was provided by EnvCan and by the Program of Energy Research and Development (PERD) administered by Natural Resources Canada. Appendix. Supplementary material Supplementary data associated with this article can be found, in the online version, at doi:10.1016/j.atmosenv.2008.11.027. References Binkowski, F.S., Shankar, U., 1995. The regional particulate matter model: Part 1. Model description and preliminary results. J. Geophys. Res. 100, 26191–26209. Binkowski, F.S., Roselle, S.J., 2003. Models-3 community multiscale air quality (CMAQ) model aerosol component 1. Model description. J. Geophys. Res. 108 (D6), 4183. Briggs, G.A., 1975. Plume rise predictions. In: Lectures on Air Pollution and Environmental Impact Analyses, Workshop, Proceedings, Boston, MA, 1975, pp. 59–111.

1069

Briggs, G.A., 1984. Plume rise and buoyancy effects. In: Anderson, D.R. (Ed.), Atmospheric Science and Power Production. Technical Information Center, U.S. DOE, Oak Ridge, TN, DOE/TIC-27601 (DE84005177) 850 pp. Byun, D.W., Ching, J.K.S., 1999. Science Algorithms of the EPA Models-3 Community Multiscale Air Quality (CMAQ) Modelling System. U.S. EPA Report EPA/600/R99/030. Byun, D., Schere, K.L., 2006. Review of the governing equations, computational algorithms, and other components of the Models-3 Community Multiscale Air Quality (CMAQ) modeling system. Appl. Mech. Rev. 59, 51–77. Carter, W.P.L., 2000a. Documentation of the SAPRC-99 Chemical Mechanism for VOC Reactivity Assessment. Final Report to California Air Resources Board Contract 92-329 and Contract 95-308. Available from: http://pah.cert.ucr.edu/~carter/ reactdat.htm. Carter, W.P.L., 2000b. Implementation of the SAPRC-99 Chemical Mechanism into the Models-3 Framework. Report to the United States Environmental Protection Agency Available from: http://pah.cert.ucr.edu/~carter/reactdat.htm. CEP, 2005. SMOKE User Manual Version 2.2. Carolina Environmental Program, University of North Carolina, Chapel Hill, North Carolina. Available from: http:// www.smoke-model.org/version2.2/manual.pdf. Coˆte´, J., Desmarais, J.-G., Gravel, S., Me´thot, A., Patoine, A., Roch, M., Staniforth, A., 1998a. The operational CMC/MRB Global Environmental Multiscale (GEM) model. Part 1: design considerations and formulation. Mon. Weather Rev. 126, 1373–1395. Coˆte´, J., Desmarais, J.-G., Gravel, S., Me´thot, A., Patoine, A., Roch, M., Staniforth, A., 1998b. The operational CMC-MRB Global Environment Multiscale (GEM) model. Part II: results. Mon. Weather Rev. 126, 1397–1418. Cousineau, S., 2003. All You Need to Know About The AURAMS Meteorological Post Processor (AMPP) Version 2.1 – Purpose, Installation, Description. Air Quality Modelling Applications Group Operations Branch – Emergency Response Division, Canadian Meteorological Centre. Available from: Environment Canada, National Prediction Operations, 2121 Trans Canada Highway, Dorval, QC, Canada H9P 1J3. Eldering, A., Cass, G.R., 1996. Source-oriented model for air pollutant effects on visibility. J. Geophys. Res. 101, 19343–19369. Environment Canada, 2005. Recherche en Prevision Numerique – Introduction. http://collaboration.cmc.ec.gc.ca/science/rpn/gef_html_public/INTRODUCTION/ gem_intro.html (accessed 26.02.08). Gal-Chen, T., Somerville, R.C., 1975. On the use of a coordinate transformation for the solution of Navier–Stokes equations. J. Comput. Phys. 17, 209–228. Gery, M.W., Whitten, G.Z., Killus, J.P., Dodge, M.C., 1989. A photochemical kinetics mechanism for urban and regional scale computer modeling. J. Geophys. Res. 94, 12925–12956. Gong, W., 2002. Modelling cloud chemistry in a regional aerosol model: bulk vs. size resolved representation. In: Borrego, C., Schayes, G. (Eds.), Air Pollution Modelling and Its Application XV. Kluwer Academic/Plenium Publishers, New York, pp. 285–293. Gong, W., Makar, P.A., Moran, M.D., 2004. Mass-conservation issues in modelling regional aerosols. In: Borrego, C., Incecik, S. (Eds.), Air Pollution Modeling and Its Application: XVI. Kluwer Academic/Plenum Publishers, New York, pp. 383– 391. Gong, W., Dastoor, A.P., Bouchet, V.S., Gong, S., Makar, P.A., Moran, M.D., Pabla, B., Me´nard, S., Crevier, L.-P., Cousineau, S., Venkatesh, S., 2006. Cloud processing of gases and aerosols in a regional air quality model (AURAMS). Atmos. Res. 82, 248–275. Jiang, W., Yin, D., 2001. Development and Application of the PMx Software Package for Converting CMAQ Modal Particulate Matter Results into Size-resolved Quantities. ICPET Tech. Rep. PET-1497-01S. National Research Council, 1200 Montreal Road, Ottawa, ON, Canada K1A 0R6, 44 pp. Jiang, W., 2003. Instantaneous secondary organic aerosol yields and their comparison with overall aerosol yields for aromatic and biogenic hydrocarbons. Atmos. Environ. 37, 5439–5444. Jiang, W., 2004. Reply to the ‘‘Comment on ‘Instantaneous secondary organic aerosol yields and their comparison with overall aerosol yields for aromatic and biogenic hydrocarbons’’’, by Knipping et al. (2004). Atmos. Environ. 38, 2763– 2767. Jiang, W., Smyth, S., Giroux, E´., Roth, H., Yin, D., 2006. Differences between CMAQ fine mode particle and PM2.5 concentrations and their impact on model performance in the Lower Fraser Valley. Atmos. Environ. 40, 4973–4985. Kang, D., Eder, B.K., Stein, A.F., Grell, G.A., Peckham, S.E., McHenry, J., 2005. The New England Air Quality Forecasting Pilot Program: development of an evaluation protocol and performance benchmark. J. Air Waste Manag. Assoc. 55, 1782– 1796. Lurmann, F.W., Lloyd, A.C., Atkinson, R., 1986. A chemical mechanism for use in long-range transport/acid deposition computer modeling. J. Geophys. Res. 91, 10905–10936. Makar, P.A., Bouchet, V.S., Nenes, A., 2003. Inorganic chemistry calculations using  þ HETV – a vectorized solver for the SO2 4 –NO3 –NH4 system based on ISORROPIA algorithms. Atmos. Environ 37, 2279–2294. Mathur, R., Yu, S., Kang, D., Schere, K.L., 2008. Assessment of the wintertime performance of developmental particulate matter forecasts with the EtaCommunity Multiscale Air Quality modeling system. J. Geophys. Res. 113, D02303. doi:10.1029/2007JD008580. McKeen, S., Wilczak, J., Grell, G., Djalalova, I., Peckham, S., Hsie, E.-Y., Gong, W., Bouchet, V., Menard, S., Moffet, R., McHenry, J., McQueen, J., Tang, Y., Carmichael, G.R., Pagowski, M., Chan, A., Dye, T., Frost, G., Lee, P., Mathur, R., 2005. Assessment of an ensemble of seven real-time ozone forecasts over

1070

S.C. Smyth et al. / Atmospheric Environment 43 (2009) 1059–1070

eastern North America during the summer of 2004. J. Geophys. Res. 110, D21307. doi:10.1029/2005JD005858. McKeen, S., Chung, S.H., Wilczak, J., Grell, G., Djalalova, I., Peckham, S., Gong, W., Bouchet, V., Moffet, R., McHenry, J., Tang, Y., Carmichael, G.R., Mathur, R., Yu, S., 2007. Evaluation of several PM2.5 forecasts using data collected during the ICARTT/NEAQS 2004 field study. J. Geophys. Res. 112, D10S20. doi:10.1029/ 2006JD007608. Monahan, E.C., Spiel, D.E., Davidson, K.L., 1986. A model of marine aerosol generation via whitecaps and wave disruption. In: Monahan, E.C., Niocaill, G.M. (Eds.), Oceanic Whitecaps. D. Redeil Publishing, Dordrecht, Holland, pp. 167–174. Moran, M.D., Zheng, Q., Pavlovic, R., Cousineau, S., Bouchet, V.S., Sassi, M., Makar, P.A., Gong, W., Stroud, C., 2008. Predicted acid deposition critical-load exceedances across Canada from a one-year simulation with a regional particulate-matter model. In: Proceedings of the 15th Joint AMS/A&WMA Conference on Applications of Air Pollution Meteorology, January 21–24, New Orleans. American Meteorological Society, Boston. Available from: http://ams. confex.com/ams/pdfpapers/132916.pdf, 20 pp. Nenes, A., Pilinis, C., Pandis, S.N., 1998. ISORROPIA: a new thermodynamic equilibrium model for multiphase multicomponent inorganic aerosols. Aquat. Geochem. 4, 123–152. Pudykiewicz, J.A., Kallaur, A., Smolarkiewicz, P.K., 1997. Semi-Lagrangian modelling of tropospheric ozone. Tellus 49B, 231–248. Sapkota, A., Symons, J.M., Kleissl, J., Wang, L., Parlange, M.B., Ondov, J., Breysse, P.N., Diette, G.B., Eggleston, P.A., Buckley, T.J., 2005. Impact of the 2002 Canadian forest fires on particulate matter air quality in Baltimore City. Environ. Sci. Technol. 39, 24–32. Schell, B., Ackermann, I.J., Hass, H., Binkowski, F.S., Ebel, A., 2001. Modeling the formation of secondary organic aerosol within a comprehensive air quality modeling system. J. Geophys. Res. 106 (D22), 28275–28293. Sirois, A., Pudykiewicz, J.A., Kallaur, A., 1999. A comparison between simulated and observed ozone mixing ratios in eastern North America. J. Geophys. Res. 104, 21397–21423. Smith, M.H., Harrison, N.M., 1998. The sea spray generation function. J. Aerosol Sci. 29, 5189–5190. Smyth, S., Yin, D., Roth, H., Jiang, W., 2005. A Study of the Impact of GEM and MM5 Meteorology on CMAQ Modelling Results in Eastern Canada and the Northeastern United States. ICPET Tech. Rep. PET-1561-04S. National Research Council, 1200 Montreal Road, Ottawa, ON, Canada K1A 0R6, 77 pp. Smyth, S.C., Yin, D., Roth, H., Jiang, W., Moran, M.D., Crevier, L.-P., 2006. The impact of GEM and MM5 meteorology on CMAQ air quality modeling results in eastern Canada and the northeastern United States. J. Appl. Meteor. Climatol. 45, 1525– 1541. Smyth, S.C., Jiang, W., Roth, H., Yang, F., 2007. A Comparative Performance Evaluation of the AURAMS and CMAQ Air Quality Modelling Systems – Revised. National Research Council, 1200 Montreal Road, Ottawa, ON, Canada K1A 0R6, ICPET Tech. Rep. PET-1577-07S114 pp. Stern, R., Builtjes, P., Schaap, M., Timmermans, R., Vautard, R., Hodzic, A., Memmesheimer, M., Feldmann, H., Renner, E., Wolke, R., Kershbaumer, A., 2008.

A model inter-comparison study focussing on episodes with elevated PM10 concentrations. Atmos. Environ. 42, 4567–4588. Stockwell, W.R., Lurmann, F.W., 1989. Intercomparison of the ADOM and RADM Gas-Phase Chemical Mechanisms. Electrical Power Research Institute Topical Report. EPRI, 3412 Hillview Avenue, Palo Alto, CA, 254 pp. Tarasick, D.W., Moran, M.D., Thompson, A.M., Carey-Smith, T., Rochon, Y., Bouchet, V.S., Gong, G., Makar, P.A., Stroud, C., Me´nard, S., Crevier, L.-P., Cousineau, S., Pudykiewicz, J.A., Kallaur, A., Moffet, R., Me´nard, R., Robichaud, A., Cooper, O.R., Oltmans, S.J., Witte, J.C., Forbes, G., Johnson, B.J., Merrill, J., Moody, J.L., Morris, G., Newchurch, M.J., Schmidlin, F.J., Joseph, E., 2007. Comparison of Canadian air quality forecast models with tropospheric ozone profile measurements above mid-latitude North America during the IONS/ ICARTT campaign: evidence for stratospheric input. J. Geophys. Res. 112, D12S22. doi:10.1029/2006JD007782. Turpin, B.J., Lim, H.J., 2001. Species contributions to PM2.5 mass concentrations: revisiting common assumptions for estimating organic mass. Aerosol Sci. Tech. 35, 602–610. van Loon, M., Vautard, R., Schaap, M., Bergstro¨m, R., Bessagnet, B., Brandt, J., Builtjes, P.J.H., Christensen, J.H., Cuvelier, C., Graff, A., Jonson, J.E., Krol, M., Langner, J., Roberts, P., Rouil, L., Stern, R., Tarraso´n, L., Thunis, P., Vignati, E., White, L., Wind, P., 2007. Evaluation of long-term ozone simulations from seven regional air quality models and their ensemble. Atmos. Environ. 41, 2083–2097. Vautard, R., Builtjes, P.J.H., Thunis, P., Cuvelier, C., Bedogni, M., Bessagnet, B., Honore´, C., Moussiopoulos, N., Pirovano, G., Schaap, M., Stern, R., Tarraso´n, L., Wind, P., 2007. Evaluation and intercomparison of ozone and PM10 simulations by several chemistry transport models over four European cities within the CityDelta project. Atmos. Environ. 41, 173–188. Xiu, A., Pleim, J.E., 2000. Development of a land surface model. Part II: application in a mesoscale meteorology model. J. Appl. Meteorol. 40, 192. Yamartino, R.J., 1993. Nonnegative, conserved scalar transport using grid-cellcentered, spectrally constrained Blackman cubics for applications on a variablethickness mesh. Mon. Weather Rev. 121, 753–763. Yin, D., 2004. A Description of the Extension for Using Canadian GEM Data in MCIP and a Brief User’s Guide for GEM-MCIP. Institute for Chemical Processing and Environmental Technology, Available from: National Research Council, 1200 Montreal Road, Ottawa, ON, Canada K1A 0R6, 44 pp. Young, T.R., Boris, J.P., 1977. A numerical technique for solving stiff ordinary differential equations associated with the chemical kinetics of reactive flow problems. J. Phys. Chem. 81, 2424–2427. Yu, S., Mathur, R., Schere, K., Kang, D., Pleim, J., Otte, T.L., 2007. A detailed evaluation of the eta-CMAQ forecast model performance for O3, its related precursors, and meteorological parameters during the 2004 ICARTT study. J. Geophys. Res. 112, D12S14. doi:10.1029/2006JD007715. Zhang, L., Gong, S.-L., Padro, J., Barrie, L., 2001. A size-segregated particle dry deposition scheme for an atmospheric aerosol module. Atmos. Environ. 35, 549–560. Zhang, L., Moran, M.D., Makar, P.A., Brook, J.R., Gong, S.-L., 2002. Modelling gaseous dry deposition in AURAMS – a unified regional air-quality modelling system. Atmos. Environ. 36, 537–560.