Copyright © IFAC 12th Triennial World Congress, Sydney, Australia, 1993
PARALLEL SIMULATION OF AIR POLLUTION TRANSPORT FOR URBAN DECISION MAKING A. Sydow, M. Schmidt, S. Unger, T, Lux, p, Mieth and R.-P. Schlifer GMD . Research Institute for Computer Architecture and Software Technology, Rudower Chaussee 5 (13.7), D·1199 Berlin , Germany
Abstract. A concept for an air pollution simulation envirorunent to predict and manage smog situations is presented consisting of the necessary data bases, a mesoscale meteorological model, an air chemistry model, and decision support tools including result visualization. Simulation runs and scenario analyses of such numerically complex models take hours of computing time, even on today 's supercomputers. Therefore a strategy for model decomposition and implementation on massively parallel computers is described . Simulation results are shown for an ozone smog situation in Berlin. Key Words. Modeling; computer simulation; air pollution; parallel processing; urban systems; environmental engineering
1. INTRODUCTION
is a set of conservation principles (Pielke, 1984): - conservation of mass, - conservation of heat, - conservation of motion, - conservation of water, and - conservation of gaseous and aerosol materials.
Although in recent years a variety of numerical models for meteorology, air pollutant transport, and air chemistry has been developed, there is a lack of simulation environments for local authorities to support their decision making activities. In order to close this gap a combination of multidisciplinary subjects such as atmospheric physics, meteorology, air chemistry, ground-based sensing, mathematical modeling and computer science is required.
These principles form a coupled set of relations that must be satisfied simultaneously and that includes sources and sinks in the individual expressions. The sources and sinks for the gaseous and aerosol materials are calculated by an air chemistry model which itself consists of a coupled set of reaction equations.
The aim of this paper is to describe the basic modules of a simulation environment for air pollution analysis over conurbations and to point out the special importance of parallel computation for such simulations. At the end of the development a simulation system should exist which is easy to handle for an user in an environmental agency, which has an acceptable run-time, and which supports efficiently the forecasting of smog situations, the operational management, and the urban planning.
According to the complexity of the numerical models derived from these principles only massively parallel computers seems to be suited as the implementation platform for such highly time-consuming calculations.
2. SIMULATION CONCEPT
2.1. Data Assimilation
Atmospheric processes over conurbation areas are described with mesoscale models. The mesoscale (commonly known as the horiwntal region between 20 and 200 km) includes all meteorological structures (e.g. thunderstorms, cloud clusters, low-level jets) between the microscale (0 ...20 km) of local turbulence, and the macroscale (200... 10000 km) of high and low pressure regions. The foundation for any mesoscale model
To run simulations with meteorological and air chemistry models the following input data are necessary:
Topographic daw. The topographic data describe the surface properties of the model domain, namely, the height over N.N. (orography), and the land utilization of every surface grid point. The 391
The model REWIMET is driven by the suprascale stratification, the suprascale horizontal pressure gradient (geostrophic wind), and the surface temperature. The input of the geostrophic wind and surface temperature can be time-dependent. REWIMET considers the orography and the land utilization in the model domain. The transport of several air pollutants can be calculated simultaneously.
land utilization is represented by different classes (e.g. town centers, suburbs, forests, lakes, etc.). Meteorological data. These input data depend upon the model used. For mesoscale models the data needed are commonly the geostrophic wind, and a vertical temperature and pressure profile. Emission data. Topographic and meteorological data are usually given by local authorities and meteorological services. They do not need much preparatory effort to their use as input. In contrast to that the assimilation of emission data is difficult and very time-consuming. Especially, the exact description of the dynamics of such processes must be done with high solicitude. On dependence of polluters it is necessary to find a suitable compromise between accuracy and technical expenditure. One way to reduce the seizing expenditure exists in a separated calculation of important polluters as point or line sources. Smaller polluters are put together to one source per grid point.
REWIMET uses an explicit numerical solver with forward time steps and central space differences with the exception of the advection terms where upstream differences are applied (method by Smolarkiewicz). The Couranl stability criterion is proven at each iteration, and defines the new time step. The gradient of all variables at the horizontal boundaries is set equal to zero.
2.3. Air Chemistry Model
Ozone, the major component of so-called summer smog, already has a quite high natural background level. Additional concentrations of this substance are produced from a mixture of hydrocarbons and nitrogen oxides as a result of photochemical reactions where there is high insolation.
All emission information is stored in a relational database system separated into classes of polluters and different air pollutants. This guaranties the storing of all data without redundances, and a simple access over an SQL interface. According to the spatial and temporal requirements of the meteorological or air chemistry model the data can be prepared by means of special reports.
The scheme of chemical reactions leads to a system of ordinary nonlinear differential equations. Because of the huge differences in reaction kinetics such systems are extremely stiff. A special algorithm has to be selected to ensure stability and acceptable run-time. For every grid point the chemical equation system has to be solved. So, even the fastest sequential computers need e.g. for a one day simulation in threedimensional space for 200 x 200 x 3 grid points some hundred hours.
2.2. Meteorological Model
The mesoscale meteorological model used in the first version of the simulation environment is the model REWIMET (Heimann, 1985). REWIMET is distinguished by a simple model geometry and comparatively little computing effort. This results from the approximations applied during the derivation of the equation set from the basic principles mentioned above. The model is based on a hydrostatic, divergenceless and dry atmosphere.
To overcome these difficulties a lot of compressed chemical reaction schemes have been developed. Several methods have been used to simplify the reaction system. Some authors use special organic species to represent others (so-called surrogate species). Other authors try to lump the organics into classes. For example these classes can be specified by functional groups etc. (lumped structure) or by carbon bonds (lumped molecule).
In contrast to fully three-dimensional models calculating the variables at the nodes of a locally fixed spatial grid REWIMET uses the fixed grid structure only horizontally. Vertically, the model is subdivided into 3 layers lying on top of each other. A part of the model variables, namely, the horizontal wind components, the potential temperature, and the air pollutant concentrations, is calculated for each horizontal grid point as box average in all 3 layers. The vertical wind component, the pressure, and the turbulent flux of impulse, heat and air pollutants are determined at the boundaries between the layers.
In the developed simulation system the model CBM4 is used (Carbon Bond Mechanism, Gery et. al., 1988). CBM-4 is a condensed version of the original CBM. Carbon atoms with similar bonding are treated similarly. There is no need for the definition of an average molar weight so that this mechanism is mass balanced. Some species are handled explicit because of their special character in the chemical system (for example 392
3. PARALLEL MODEL IMPLEMENTATION
isoprene which is the most emitted biogenic species). The mechanism involves 34 species and 82 reactions, and contains 9 primary organic compounds. To profit from the features of the CBM-4 detailed information of the hydrocarbon mixture is necessary.
Section 2 shows that numerical models for air pollution transport and chemistry have a huge demand for computing time. Even on today 's sequential supercomputers the CPU time for scenario analysis and case studies is considerably too long.
The CBM-4 has been extensively tested against over 170 smog chamber experiments with good results. It is proposed from the U.S. Environmental Protection Agency for the usage in air pollution models.
To overcome these problems the models have been implemented on a parallel computer of MIMD architecture with distributed memory (SUPRENUM, compare Hoffmann and Maretis, 1990). A rectangular decomposition of the model space (geometrical decomposition) was carried out because of the small data exchange rate between the grid boxes in this case. According to the data flow structure a grid-type processor topology was used shown in Fig. I, whereby an additional root processor controls input/output and the stepwise integration. In Fig. 1 the connection between the root and the worker grid represents the bidirectional communication links between the root and each of the worker processors. The actual communication structure of the workers among themselves is shown in the lower part of Fig. 1.
In the transport equation (1) of the meteorological model the chemical rates (concentration per time unit) are represented by R j :
.i at C . = I
- V
vC. + aXj ~ (KJ. ~ C.) + Q . + R. (1) dXj I
I
I
I
with C j - concentration of the i-th species v - wind velocity vector K j - turbulent dispersion coefficient at point j Qj - emission of the i-th species.
2.4. Decision Support Strategies A decision maker in an environmental agency responsible for air pollution management has commonly to solve 3 kinds of problems: - approval of industrial facilities, - environment compatibility tests, and - precautionary measures to prevent smog situations.
grid of worker processors
Today he has little support for this tasks. The calculations for the approval of industrial facilities, for example, are carried out with Gaussian models representing the state of the art of the I 960s. Up to now measures against smog can only be taken when ground-sensing networks indicate the exceeding of concentration limits. The described simulation environment in its final version should support the user, mainly, in two ways : - by the graphic visualization of simulation results for air pollution transport processes with complex atmospheric models, and - by recommending measures derived with inference algorithms from a knowledge base of an expert system or from the simulation of a neural network.
'.
. ........................................................................................... Fig. 1. Processor topology At the beginning the root process carries out the read in and the initializing of all data arrays. The following time cycle for the integration of the model equations can principally run in parallel for every subregion of the model domain. In fact, one process integrates the boundary grid points because they need a special treatment. The remaining grid points are equally divided among processes on the other available processors. Each process calculates in parallel for its subregion the
Besides of the graphical output the simulation results are used to build up a knowledge base or to train a neural network. The computing effort for the simulation runs necessary for that again illustrates the need for massively parallel computers. 393
representing an area of 26 x 20 km 2 ) in the lowest model layer with ozone concentrations higher than 150 llg/m3 for a simulation period from 6 a.m. to 9 p.m. In the case of 50 % reduction there are only three. On the other hand the influence of reduction measures on the mean value of the ozone concentration is low. This results from the little contribution (10 to 15 %) of the ozone caused by man-made emissions in the model domain to the total ozone concentration for this special weather conditions and chosen initial and background concentrations of all chemical compounds.
meteorological quantItIes and the air pollutant concentrations. After every time step only two boundary column of each subregion has to be exchanged with the neighboring processes. This considerably decreases the effort of communication. The root process gets back all values necessary for control from the worker processes. If a data output is programmed at this time step the communication between root and workers includes the sending of all output arrays to the root. After testing the cycle condition and determining the current integration step width all parallel integration processes are restarted simultaneously.
6
Secondly, a reduction of the emissions caused by traffic to 50 % was simulated. This also reduces the number of grid boxes with ozone concentrations higher than 150 llg/m3, but to a lower extent than expected. The influence of the traffic-emission reduction on the mean ozone concentration can be neglected.
speed up
4
Thirdly, a reduction of the emISSIOns of hydrocarbons caused by private households to 50 % was carried out. The simulations showed that this measure has a negligible influence on the ozone concentrations for the special situation.
2 efficiency
10
20
30
40
50
number of processors
Finally, it can be stated that the ozone chemistry is a very complicated process, and that future research work is necessary to obtain a better insight into the reactions. For this purpose an extension of the model domain is inevitable to solve the problem of the determination of initial and background concentrations more successful, and to study the consequences of emissions from the city of Berlin on the ozone concentrations in the surrounding countryside.
Fig. 2. Performance for a REWlMET simulation Fig. 2 shows speed up and efficiency of a parallel simulation with REWlMET (no air chemistry) implemented on the SUPRENUM for a 34x26x3 model grid. It turns out that a good parallel performance can only be achieved when some hundred grid boxes per processor remain. In the other case the communication effort is too high. For a simulation with REWIMET and the chemical model CBM-4 the parallel performance is better because of the very little demand on communication needed for air chemistry.
5. REFERENCES Gery M.W., G.Z. WhiUen and J.P. Killus (1988). Development and testing of the CBM-4 for urban and regional modeling. U.S. Environmental Protection Agency, Report No. EPA-600/3-88-012, USA. Heimann D. (1985). Ein Dreischichten-Modell zur Berechnung mesoskaliger Wind- und Immissionsfelder tiber komplcxem Gelande. Dissertation, University Munich. Hoffmann G.-R. and D.K. Maretis (1990). The dawn of massively parallel processing in meteorology. Springer, Berlin. Mieth, P. and S. Ungcr (1992). Zum EinfluB der Reduktion von Emissionen verschiedener Emittenten auf die Ozonbildung. Editor: Senate of Berlin, Department for Urban Development and Environmental Protection. Pielke. R.A. (1984). Mesoscale meteorological mode ling . Academic Press, Orlando.
4. SIMULATION RESULTS Starting from an ozone simulation over Berlin on June 27, 1990, with the described simulation system the results of emission reduction measures were investigated (for detailed information see Mieth and Unger, 1992). It should be pointed out that the obtained results are only valid for this special emission and meteorological situation and cannot be generalized. Firstly, a reduction of the total amount of emissions caused by man to 50 % was simulated. This strong and practically not realizable measure leads to a considerable decrease of the ozone maxima in the lee-side of the town. In the case of no reduction there are 44 grid boxes (from 520 394