- Email: [email protected]
Distributed run of a one-dimensional model in a regional application using SOAP-based web services
ARTICLE IN PRESS
Atmospheric Environment 39 (2005) 1961–1966 www.elsevier.com/locate/atmosenv
Technical note
Distributed run of a one-dimensional model in a regional application using SOAP-based web services Gerhard Smiatek Institute for Meteorology and Climate Research (IMK-IFU), Forschungszentrum Karlsruhe, 82467 Garmisch-Partenkirchen, Germany Received 14 July 2004; received in revised form 10 December 2004; accepted 11 January 2005
Abstract This article describes the setup of a distributed computing system in Perl. It facilitates the parallel run of a onedimensional environmental model on a number of simple network PC hosts. The system uses Simple Object Access Protocol (SOAP) driven web services offering the model run on remote hosts and a multi-thread environment distributing the work and accessing the web services. Its application is demonstrated in a regional run of a processoriented biogenic emission model for the area of Germany. Within a network consisting of up to seven web services implemented on Linux and MS-Windows hosts, a performance increase of approximately 400% has been reached compared to a model run on the fastest single host. Published by Elsevier Ltd. Keywords: Distributed computing; Web services; SOAP; Perl; BVOC
1. Introduction Study, assessment and management of many environmental problems involves use of numerical models. Their application is often very complex. Results including different scenarios must be provided within a very short time frame. Therefore substantial computational power is needed. Supercomputers and cluster computers are used to speed up the operation of environmental models by running them on faster hardware or in massive parallel mode. Here, however, special soft- and hardware are required. On the other hand, there are often large numbers of simple PCs and workstations available within a corporate network that are not used during Tel.: +49 8821 183 282; fax: +49 8821 183 243.
E-mail address: [email protected]. 1352-2310/$ - see front matter Published by Elsevier Ltd. doi:10.1016/j.atmosenv.2005.01.017
large portion of the day and night. By joining their resources within a distributed computing system a powerful computational environment can be created for a fraction of costs of a cluster computer or purchase of faster hardware. The article describes the setup of such a distributed system in Perl and its application in a regional approach for the area of Germany in 10 km 10 km grid on the example of the Seasonal Isoprene Synthase Model–Biochemical Isoprenoid Biosynthesis Model (SIM–BIM) (Lehning et al., 2001). The model is used to forecast emissions of volatile organic compound (VOC) from vegetation. By exposing the SIM–BIM model on many simple single PCs and workstations as a Simple Object Access Protocol (SOAP)-based web service, regional applications can be operated in parallel within an existing heterogeneous computer network. The service uses the Perl SOAP::Lite library (Kulchenko, 2001) to
ARTICLE IN PRESS G. Smiatek / Atmospheric Environment 39 (2005) 1961–1966
1962
create the necessary communication environment and a Perl multi-thread environment to distribute the work to the participating hosts.
2. Problem description Empirical models are able to predict the mean volatile organic compounds emissions from biogenic sources (BVOC) (Guenther, 1997). However, they are not sensitive enough to changes in air temperature and radiation (Lehning et al., 2001). Recently, processoriented models have been developed. They take into account the physiological state of the leaves and the biochemical processes on the molecular level. SIM–BIM is such a process-oriented model. On the other hand, improved modeling approaches require much more computational resources, especially if applied regionally. A single 24 h forecast with SIM–BIM, run on the fastest available host (see Table 1) requires approximately 5 s of computation time. The forecast for the area of Germany at a grid cell size of 10 km 10 km with 66 times 90 cells and two layers for the sun light and shadow fraction of the canopy will require more than 16 h of computation. It is obvious that under these conditions even a shortterm run for an episode of a few days or weeks are very time consuming. The operation of full 3D air flow and chemistry transport models requires often much more computational resources. Pais et al. (2004) report of 8.5 days of computation on a midrange workstation in run of an ammonia exchange model.
3. Distributed computing environment To improve the performance of a given model code there are three options available: faster hardware (e.g. run on a supercomputer), run on a cluster computer, and a distributed computing approach. While the use of a supercomputer is often not possible, cluster computers
and distributed computing offer commonly available options. A cluster computer is a system of distributed memory multi-processor computers located close to each other and interconnected by a high-speed network. It provides a very powerful environment for massive parallel applications. In order to take of the cluster the application software has to be ported to the specific cluster system. The major problem here is the parallelization. Danelutto (2003) discusses several existing parallelization approaches including message passing interface (MPI), open multi processing (OpenMP) and high performance fortran (HPF). Distributed computing is a process of running a single computational task on more than one computer remote to each other. This differs from cluster computing in the way that the participating computers are not exclusively running a distributed application. Typically, the client/ server communications model is used. A central management server divides the complex task into smaller work packages and distributes them to the participating clients for processing. One of the most popular application of that type is SETI@Home, the search for intelligent life in the universe. Here, the application is run as a screen saver.
4. Regional model run 4.1. Approach For the regional application of the model, the area of interest is subdivided into a number of regions, e.g. grid cells. The work is distributed over a number of available hosts. Each host always calculates the emissions of only one grid cell for all required time steps. Any host with any operating system (OS) can be used, provided that the model code can be compiled for that OS. At the current stage of SIM–BIM development the participating hosts do not need to communicate with each other
Table 1 Host specification in distributed run of the SIM–BIM model for the area of Germany with 5940 grid cells Server
CPU (MHz)
No.
Code
1 2 3 4 5 6 7
A1 A2 B C D1 D2a E a
1.0 1.0 2.2 0.5 1.8 1.8 1.0
Memory (MB)
256 256 512 256 256 512 256
In addition host runs the administrative script.
OS
SuSE Linux SuSE Linux SuSE Linux SuSE Linux MS 2000 SuSE Linux MS NT4
Run of a single cell (s)
8.0 8.0 8.3 7.3 8.0
4.8 4.8 8.7 12 13.0 13.0
Distributed run Calc. cells
Share (%)
1388 1385 877 631 612 576 471
23.4 23.3 14.8 10.6 10.3 9.7 7.9
ARTICLE IN PRESS G. Smiatek / Atmospheric Environment 39 (2005) 1961–1966
(one-dimensional approach). If provided with the required input data, they will run the model for one grid cell and report the results to the management host, receive the new input data and calculate the next cell.
Read Input
Create Thread
Create Thread
Get cell ID
Get cell ID
Run Model (client)
Run Model (client)
Intranet (SOAP)
1963
The SIM–BIM model run itself is implemented as a web service. Fig. 1 shows the activity diagram of the system. It consists of four parts: (1) hosts offering the run of the SIM–BIM model as a web service, (2) communication protocol for web services, (3) clients assessing the web service, (4) multi-thread administration system allowing the parallel run of several clients each of them using one web service. The system consists of two classes Runbim and Runbimdistributed as well as an administrative script (see Fig. 2) written in object-oriented Perl. Perl has the advantage of running on all major operating systems. 4.2. System administration
Host 1
Host 2
Server
Server
Run Model
Run Model
Write output
Fig. 1. Activity diagram of the distributed system. For simplicity only two threads are depicted.
Runbimdistributed +id: integer +serverName : character +cellId: integer +new() +run() +runModelRemote() +nextId() +nextIdIfError() Admin +servers: character +CELL_ID: integer +JUL_TEMPERATURE: real +JUL_PAR: real +FOR_TEMPERATURE: real +FOR_PAR: real +bim_readInput() +bim_init() +bim_finish()
In order to run the SIM–BIM model in a regional application on a number of hosts, several clients accessing the web services must be run simultaneously. This is done with a Perl script in a multi-thread approach in a master–worker principle. Threads are sequential flows of control within the same program. The script runs on the master host (see Fig. 1). It reads all required input data for each grid cell and stores them in global associative array variables (hashes) in which the (hash) key is the grid identification number (id). These input data are: daily mean temperature and photosynthetically active radiation (PAR) for each
Runbim +julianDay: integer +bimExe: character +configFileName: character +new() +runBim() +makeTempForecastFile() +makeLightForecastFile() +makeTempFile() +makeLightFile() +makeConfigFile() +getResult()
Fig. 2. Class diagram of the distributed computing system. The Runbim class is required to run the model on a single host. Runbimdistributed class and the system administration (Admin) script create a distributed computing environment.
ARTICLE IN PRESS 1964
G. Smiatek / Atmospheric Environment 39 (2005) 1961–1966
Julian day from the beginning of the year and hourly temperature and PAR values for the forecast period. Then, for each participating remote host providing the web service, a thread is created using the threads4new(Runbimdistributed::run,$id, $serverName) command. In each thread the operations run independently, but at the same time. The threads are running on the master hosts. Here, the run method provided by the Runbimdistributed class is executed (see Fig. 2). The method creates a new Runbimdistributed object with the attributes server id and server name of the remote server which will be accessed by this thread to run SIM–BIM. Then it starts an endless loop. Within the loop the next cell id to be calculated is requested first (method nextId). Then, the method runModelRemote that installs a client accessing the web service is executed. A client always communicates with the same web service server. It reads the content of the global variables storing the input data for the next grid cell. Over SOAP connection the input data is passed to the remote web service host that runs the SIM–BIM model. After a successful run of the model, the client receives and returns the results. The results are stored in a global associative array and the loop continues with the next cell. Once in nextId the maximum cell id to be calculated is reached, the threads are closed using the join command and the results are stored in a file. 4.3. Communication The communication environment between clients assessing a web service and web service servers is established with the Perl SOAP::Lite library. SOAP is a project similar to CORBA (Common Object Request Broker Architecture) or DCOM, the distributed extension to COM (Component Object Model) (Box, 1999). Its specification was published in the year 2000 (Box et al., 2000). SOAP is a protocol specification for invoking objects and methods on remote servers and services using Extensible Markup Language (XML) and Hyper Text Transport Protocol (HTTP). Within the distributed computing system, the SOAP::Lite Perl library handles both client and server functions for SOAP communications. 4.4. Web service Corner stones of the presented SOAP-based approach of using a heterogeneous computer infrastructure to run a numerical model are web services exposing the SIM–BIM model to any allowed user within the network. A web service is installed on each participating host. The service implementation consist of three parts: SIM–BIM model, SIM–BIM Perl class (Runbim) and the SOAP server in Perl. Fig. 3 shows how easy it is to implement a SOAPbased web service server in Perl. First, we introduce the SOAP::Lite library and the Runbim class using the use
1 #!/usr/bin/perl -w 2 3 use SOAP::Transport::HTTP; 4 use Runbim; 5 6# do not die on Ctrl-C 7 $SIG{PIPE} = $SIG{INT} = ’IGNORE’; 8 9 $daemon = SOAP::Transport::HTTP::Daemon 10 -> new (LocalPort => 8083) 11 -> dispatch_to(’Runbim’) 12 ; 13 $daemon->handle Fig. 3. Web service server definition in Perl. It uses the HTTP protocol, listens at the specified port and dispatches all requests to the Runbim class.
operator. Then a SOAP daemon is defined. It will use HTTP as the transport protocol. The server listens at the port 8083 for connections that are dispatched to the Runbim class. Finally, $daemon-4handle starts the service. Once the service is started, any application at any host within the network requiring a Runbim object or a Runbim–method can invoke or execute it at the remote host exposing the web service. There is no need to describe the Runbim class in detail. It implements methods that are required to operate the model in a traditional way on a single host. In the case of SIM–BIM, the Runbim class contains methods to create a Runbim object (new) and to provide the required input data for temperature (makeTempForecastFile), PAR (makeLightForecastFile), temperature and PAR history from the beginning of the year as daily mean values (makeTempFile and makeLightFile). An additional method writes a configuration file (makeConfigFile). Finally, the SIM–BIM run is implemented on the operating system level (runBim) and the result is returned (getResult). 4.5. Client assessing the web service As simple as the server, the client requesting the web service can be defined. Fig. 4 shows a Perl method (runModelRemote) implementing the client. It is a part of the Runbimdistributed class. The SOAP_PROXY variable contains the information on which server and port web service will be used. In line 6 the client is defined. Over SOAP the Runbim class can be accessed. Then a Runbim object is created on the remote server (line 7). Finally, the method runBimDistributed on a Runbim object is executed. It performs the SIM–BIM model run (line 10) and receives the results (line 11). It is emphasised that the method is called from the client host but executed via SOAP on the foreign server host offering the web service. The class Runbimdistributed provides methods to create a new object (new) and
ARTICLE IN PRESS G. Smiatek / Atmospheric Environment 39 (2005) 1961–1966
1 sub runModelRemote { 2 my $self = shift; 3 my $serverName = $self >serverName(); 4 $SOAP_PROXY = join (’’,’http://’, $serverName,’.dom.name.de:8083’); 5 my $client = SOAP::Lite->uri(’Runbim’); 6 $client->proxy($SOAP_PROXY); 7 my $bim = $client 8 ->call(new =>) 9 ->result; 10 my $resp = $client-> runBim($bim); 11 my $bvoc = $resp->result; 12 Fig. 4. Definition of a client accessing a remote web service in Perl.
defines the attributes id (id) and name (serverName) of the host serving a SIM–BIM run, and the id of the cell (or area) for that calculation is performed. The methods run and nextId have already been described in the system administration section.
4.6. Error handling Due to its simplicity and reliability of its components the distributed system runs smoothly even over a long time period, however, it is incomplete. Some exception handling is required to prevent failures of the entire system caused by run-time errors in one of its parts. Errors can occur in the model run on a host exposing the web service, on the web service server, within SOAP communication including network errors, and in creation and operation of threads. They can be handled by including the appropriate method calls into a eval block. Eval is a function provided by Perl. I case of error, Perl skips the rest of the block, sets the variable $@ to the corresponding error text and continues with the program. Thus, run-time errors can be catched and handled. Errors that occur on a web services server are communicated via SOAP. SOAP::Lite provides a full range of error and exception handling methods. In running a distributed system in a heterogeneous environment a special problem must be considered. The calculations are performed on hosts with different resources. While a slow host still processes cell n, a faster hosts might already process cell m, where m4n: If an error occurs at that slower host, cell n remains unprocessed. Any additional call to the web service running at this host might add additional unprocessed cells. Therefore, in case, there is a failure of a thread, the entire thread is closed. In addition, the method nextIdifError is called. It sets the grid id to the cell id where the error occurred and the remaining threads continue with this id. In case, a required minimum number of threads cannot be run, the administrative script will stop the system and indicate an incomplete run.
1965
5. Application example The presented system has been applied to regional runs of the SIM–BIM model for the area of Germany. Seven PCs and workstations with different resources and operating systems including Linux, Windows2000 and Windows NT provide their CPU time to run the model. In a test case, a forecast of the isoprene emissions from Oaks for the Julian day 161 of the year 2002 and 1500 grid cells has been run with different numbers of hosts. Fig. 5 shows the decrease of required computation time. The host specifications are listed in Table 1. A SIM–BIM run for 1500 grid cells on the single host A1 requires 134 min of computation. By accessing a web service provided by the host A2, this time is already reduced to 69 min. Use of all available host reduces the computation time to only 33 min. Due to the different resources of the involved workstations, slower hosts contribute a smaller portion of calculated grid cells. Available hosts with any hardware contribute to the performance gain without slowing down faster hosts. It can be seen that the CPU speed and available memory are not the only determinants of the hosts performance. The outline of the entire workstation including network card, bios settings and network connection also play a significant role. Due to increasing network traffic and increasing requirements for administration of the distributed system, the performance gain decreases with the number of hosts. As a model run for a single cell does not require information from its neighboring grid cells, load balancing efforts are not needed. A 24-hour prediction for all 5940 grid cells needed for the total area of Germany required 130 min of computation time with all seven available hosts. Thus, using the idle time of the workstations during 14 h of night time the prediction range can be extended to 1 week. A key question of the presented approach is its stability. It has been tested with 100 runs using different numbers of
Fig. 5. Processing time versus number of used host in distributed run of the SIM–BIM model.
ARTICLE IN PRESS 1966
G. Smiatek / Atmospheric Environment 39 (2005) 1961–1966
hosts and calculated grid cells. All 100 runs have successfully been completed. In four cases, failures of a single host due to network problems have been experienced, but the runs continued with the remaining hosts.
German Program on Atmospheric Research (AFO2000) subproject BEWA2000.
6. Conclusions
References
The use of SOAP-based web services provides a very effective way of creating stable distributed applications. By exposing a single model run on a host offering CPU time as a web service a powerful computational environment is created at minimal costs. Any available workstation within a network can be used independent of the operating system, CPU type and speed. Although SOAP will principally allow for communication between the hosts, the system is specially suitable for onedimensional environmental models that run on single points or cells. The SOAP::Lite library simplifies the creation of a distributed computing environment to a minimum.
Box, D., 1999. House of COM. Microsoft Systems Journal 14 (9). Box, D., Ehnebuske, D., Kakivaya, G., Layman, A., Mendelsohn, N., Nielsen, H., Thatte, S., Winer, D., 2000. Simple Object Access Protocol (SOAP) 1.1., www.w3.org/TR/ SOAP/. Danelutto, M., 2003. HPC the easy way: new technologies for high performance application development and deployment. Journal of Systems Architecture 49 (10–11), 399–419. Guenther, A., 1997. Seasonal and spatial variations in the natural volatile organic compound emissions. Ecological Applications 7 (1), 34–45. Kulchenko, P., 2001. SOAP::Lite for Perl, www.soaplite.com. Lehning, A., Zimmer, W., Zimmer, I., Schnitzler, J.-P., 2001. Modeling of annual variations of oak (Quercus robur L.) isoprene synthase activity to oredict isoprene emission rates. Journal of Geophysical Research 106 (D3), 3157–3166. Pais, W.A., Fournier, N., Suttonc, M.A., Westonb, K.J., Dragositsc, U., 2004. Using high performance fortran to parallelise a multi-layer atmospheric transport model. Parallel Computing 30 (1), 21–33.
Acknowledgements The work presented was supported by the German Federal Ministry of Education and Research within the
Atmospheric Environment 39 (2005) 1961–1966 www.elsevier.com/locate/atmosenv
Technical note
Distributed run of a one-dimensional model in a regional application using SOAP-based web services Gerhard Smiatek Institute for Meteorology and Climate Research (IMK-IFU), Forschungszentrum Karlsruhe, 82467 Garmisch-Partenkirchen, Germany Received 14 July 2004; received in revised form 10 December 2004; accepted 11 January 2005
Abstract This article describes the setup of a distributed computing system in Perl. It facilitates the parallel run of a onedimensional environmental model on a number of simple network PC hosts. The system uses Simple Object Access Protocol (SOAP) driven web services offering the model run on remote hosts and a multi-thread environment distributing the work and accessing the web services. Its application is demonstrated in a regional run of a processoriented biogenic emission model for the area of Germany. Within a network consisting of up to seven web services implemented on Linux and MS-Windows hosts, a performance increase of approximately 400% has been reached compared to a model run on the fastest single host. Published by Elsevier Ltd. Keywords: Distributed computing; Web services; SOAP; Perl; BVOC
1. Introduction Study, assessment and management of many environmental problems involves use of numerical models. Their application is often very complex. Results including different scenarios must be provided within a very short time frame. Therefore substantial computational power is needed. Supercomputers and cluster computers are used to speed up the operation of environmental models by running them on faster hardware or in massive parallel mode. Here, however, special soft- and hardware are required. On the other hand, there are often large numbers of simple PCs and workstations available within a corporate network that are not used during Tel.: +49 8821 183 282; fax: +49 8821 183 243.
E-mail address: [email protected]. 1352-2310/$ - see front matter Published by Elsevier Ltd. doi:10.1016/j.atmosenv.2005.01.017
large portion of the day and night. By joining their resources within a distributed computing system a powerful computational environment can be created for a fraction of costs of a cluster computer or purchase of faster hardware. The article describes the setup of such a distributed system in Perl and its application in a regional approach for the area of Germany in 10 km 10 km grid on the example of the Seasonal Isoprene Synthase Model–Biochemical Isoprenoid Biosynthesis Model (SIM–BIM) (Lehning et al., 2001). The model is used to forecast emissions of volatile organic compound (VOC) from vegetation. By exposing the SIM–BIM model on many simple single PCs and workstations as a Simple Object Access Protocol (SOAP)-based web service, regional applications can be operated in parallel within an existing heterogeneous computer network. The service uses the Perl SOAP::Lite library (Kulchenko, 2001) to
ARTICLE IN PRESS G. Smiatek / Atmospheric Environment 39 (2005) 1961–1966
1962
create the necessary communication environment and a Perl multi-thread environment to distribute the work to the participating hosts.
2. Problem description Empirical models are able to predict the mean volatile organic compounds emissions from biogenic sources (BVOC) (Guenther, 1997). However, they are not sensitive enough to changes in air temperature and radiation (Lehning et al., 2001). Recently, processoriented models have been developed. They take into account the physiological state of the leaves and the biochemical processes on the molecular level. SIM–BIM is such a process-oriented model. On the other hand, improved modeling approaches require much more computational resources, especially if applied regionally. A single 24 h forecast with SIM–BIM, run on the fastest available host (see Table 1) requires approximately 5 s of computation time. The forecast for the area of Germany at a grid cell size of 10 km 10 km with 66 times 90 cells and two layers for the sun light and shadow fraction of the canopy will require more than 16 h of computation. It is obvious that under these conditions even a shortterm run for an episode of a few days or weeks are very time consuming. The operation of full 3D air flow and chemistry transport models requires often much more computational resources. Pais et al. (2004) report of 8.5 days of computation on a midrange workstation in run of an ammonia exchange model.
3. Distributed computing environment To improve the performance of a given model code there are three options available: faster hardware (e.g. run on a supercomputer), run on a cluster computer, and a distributed computing approach. While the use of a supercomputer is often not possible, cluster computers
and distributed computing offer commonly available options. A cluster computer is a system of distributed memory multi-processor computers located close to each other and interconnected by a high-speed network. It provides a very powerful environment for massive parallel applications. In order to take of the cluster the application software has to be ported to the specific cluster system. The major problem here is the parallelization. Danelutto (2003) discusses several existing parallelization approaches including message passing interface (MPI), open multi processing (OpenMP) and high performance fortran (HPF). Distributed computing is a process of running a single computational task on more than one computer remote to each other. This differs from cluster computing in the way that the participating computers are not exclusively running a distributed application. Typically, the client/ server communications model is used. A central management server divides the complex task into smaller work packages and distributes them to the participating clients for processing. One of the most popular application of that type is SETI@Home, the search for intelligent life in the universe. Here, the application is run as a screen saver.
4. Regional model run 4.1. Approach For the regional application of the model, the area of interest is subdivided into a number of regions, e.g. grid cells. The work is distributed over a number of available hosts. Each host always calculates the emissions of only one grid cell for all required time steps. Any host with any operating system (OS) can be used, provided that the model code can be compiled for that OS. At the current stage of SIM–BIM development the participating hosts do not need to communicate with each other
Table 1 Host specification in distributed run of the SIM–BIM model for the area of Germany with 5940 grid cells Server
CPU (MHz)
No.
Code
1 2 3 4 5 6 7
A1 A2 B C D1 D2a E a
1.0 1.0 2.2 0.5 1.8 1.8 1.0
Memory (MB)
256 256 512 256 256 512 256
In addition host runs the administrative script.
OS
SuSE Linux SuSE Linux SuSE Linux SuSE Linux MS 2000 SuSE Linux MS NT4
Run of a single cell (s)
8.0 8.0 8.3 7.3 8.0
4.8 4.8 8.7 12 13.0 13.0
Distributed run Calc. cells
Share (%)
1388 1385 877 631 612 576 471
23.4 23.3 14.8 10.6 10.3 9.7 7.9
ARTICLE IN PRESS G. Smiatek / Atmospheric Environment 39 (2005) 1961–1966
(one-dimensional approach). If provided with the required input data, they will run the model for one grid cell and report the results to the management host, receive the new input data and calculate the next cell.
Read Input
Create Thread
Create Thread
Get cell ID
Get cell ID
Run Model (client)
Run Model (client)
Intranet (SOAP)
1963
The SIM–BIM model run itself is implemented as a web service. Fig. 1 shows the activity diagram of the system. It consists of four parts: (1) hosts offering the run of the SIM–BIM model as a web service, (2) communication protocol for web services, (3) clients assessing the web service, (4) multi-thread administration system allowing the parallel run of several clients each of them using one web service. The system consists of two classes Runbim and Runbimdistributed as well as an administrative script (see Fig. 2) written in object-oriented Perl. Perl has the advantage of running on all major operating systems. 4.2. System administration
Host 1
Host 2
Server
Server
Run Model
Run Model
Write output
Fig. 1. Activity diagram of the distributed system. For simplicity only two threads are depicted.
Runbimdistributed +id: integer +serverName : character +cellId: integer +new() +run() +runModelRemote() +nextId() +nextIdIfError() Admin +servers: character +CELL_ID: integer +JUL_TEMPERATURE: real +JUL_PAR: real +FOR_TEMPERATURE: real +FOR_PAR: real +bim_readInput() +bim_init() +bim_finish()
In order to run the SIM–BIM model in a regional application on a number of hosts, several clients accessing the web services must be run simultaneously. This is done with a Perl script in a multi-thread approach in a master–worker principle. Threads are sequential flows of control within the same program. The script runs on the master host (see Fig. 1). It reads all required input data for each grid cell and stores them in global associative array variables (hashes) in which the (hash) key is the grid identification number (id). These input data are: daily mean temperature and photosynthetically active radiation (PAR) for each
Runbim +julianDay: integer +bimExe: character +configFileName: character +new() +runBim() +makeTempForecastFile() +makeLightForecastFile() +makeTempFile() +makeLightFile() +makeConfigFile() +getResult()
Fig. 2. Class diagram of the distributed computing system. The Runbim class is required to run the model on a single host. Runbimdistributed class and the system administration (Admin) script create a distributed computing environment.
ARTICLE IN PRESS 1964
G. Smiatek / Atmospheric Environment 39 (2005) 1961–1966
Julian day from the beginning of the year and hourly temperature and PAR values for the forecast period. Then, for each participating remote host providing the web service, a thread is created using the threads4new(Runbimdistributed::run,$id, $serverName) command. In each thread the operations run independently, but at the same time. The threads are running on the master hosts. Here, the run method provided by the Runbimdistributed class is executed (see Fig. 2). The method creates a new Runbimdistributed object with the attributes server id and server name of the remote server which will be accessed by this thread to run SIM–BIM. Then it starts an endless loop. Within the loop the next cell id to be calculated is requested first (method nextId). Then, the method runModelRemote that installs a client accessing the web service is executed. A client always communicates with the same web service server. It reads the content of the global variables storing the input data for the next grid cell. Over SOAP connection the input data is passed to the remote web service host that runs the SIM–BIM model. After a successful run of the model, the client receives and returns the results. The results are stored in a global associative array and the loop continues with the next cell. Once in nextId the maximum cell id to be calculated is reached, the threads are closed using the join command and the results are stored in a file. 4.3. Communication The communication environment between clients assessing a web service and web service servers is established with the Perl SOAP::Lite library. SOAP is a project similar to CORBA (Common Object Request Broker Architecture) or DCOM, the distributed extension to COM (Component Object Model) (Box, 1999). Its specification was published in the year 2000 (Box et al., 2000). SOAP is a protocol specification for invoking objects and methods on remote servers and services using Extensible Markup Language (XML) and Hyper Text Transport Protocol (HTTP). Within the distributed computing system, the SOAP::Lite Perl library handles both client and server functions for SOAP communications. 4.4. Web service Corner stones of the presented SOAP-based approach of using a heterogeneous computer infrastructure to run a numerical model are web services exposing the SIM–BIM model to any allowed user within the network. A web service is installed on each participating host. The service implementation consist of three parts: SIM–BIM model, SIM–BIM Perl class (Runbim) and the SOAP server in Perl. Fig. 3 shows how easy it is to implement a SOAPbased web service server in Perl. First, we introduce the SOAP::Lite library and the Runbim class using the use
1 #!/usr/bin/perl -w 2 3 use SOAP::Transport::HTTP; 4 use Runbim; 5 6# do not die on Ctrl-C 7 $SIG{PIPE} = $SIG{INT} = ’IGNORE’; 8 9 $daemon = SOAP::Transport::HTTP::Daemon 10 -> new (LocalPort => 8083) 11 -> dispatch_to(’Runbim’) 12 ; 13 $daemon->handle Fig. 3. Web service server definition in Perl. It uses the HTTP protocol, listens at the specified port and dispatches all requests to the Runbim class.
operator. Then a SOAP daemon is defined. It will use HTTP as the transport protocol. The server listens at the port 8083 for connections that are dispatched to the Runbim class. Finally, $daemon-4handle starts the service. Once the service is started, any application at any host within the network requiring a Runbim object or a Runbim–method can invoke or execute it at the remote host exposing the web service. There is no need to describe the Runbim class in detail. It implements methods that are required to operate the model in a traditional way on a single host. In the case of SIM–BIM, the Runbim class contains methods to create a Runbim object (new) and to provide the required input data for temperature (makeTempForecastFile), PAR (makeLightForecastFile), temperature and PAR history from the beginning of the year as daily mean values (makeTempFile and makeLightFile). An additional method writes a configuration file (makeConfigFile). Finally, the SIM–BIM run is implemented on the operating system level (runBim) and the result is returned (getResult). 4.5. Client assessing the web service As simple as the server, the client requesting the web service can be defined. Fig. 4 shows a Perl method (runModelRemote) implementing the client. It is a part of the Runbimdistributed class. The SOAP_PROXY variable contains the information on which server and port web service will be used. In line 6 the client is defined. Over SOAP the Runbim class can be accessed. Then a Runbim object is created on the remote server (line 7). Finally, the method runBimDistributed on a Runbim object is executed. It performs the SIM–BIM model run (line 10) and receives the results (line 11). It is emphasised that the method is called from the client host but executed via SOAP on the foreign server host offering the web service. The class Runbimdistributed provides methods to create a new object (new) and
ARTICLE IN PRESS G. Smiatek / Atmospheric Environment 39 (2005) 1961–1966
1 sub runModelRemote { 2 my $self = shift; 3 my $serverName = $self >serverName(); 4 $SOAP_PROXY = join (’’,’http://’, $serverName,’.dom.name.de:8083’); 5 my $client = SOAP::Lite->uri(’Runbim’); 6 $client->proxy($SOAP_PROXY); 7 my $bim = $client 8 ->call(new =>) 9 ->result; 10 my $resp = $client-> runBim($bim); 11 my $bvoc = $resp->result; 12 Fig. 4. Definition of a client accessing a remote web service in Perl.
defines the attributes id (id) and name (serverName) of the host serving a SIM–BIM run, and the id of the cell (or area) for that calculation is performed. The methods run and nextId have already been described in the system administration section.
4.6. Error handling Due to its simplicity and reliability of its components the distributed system runs smoothly even over a long time period, however, it is incomplete. Some exception handling is required to prevent failures of the entire system caused by run-time errors in one of its parts. Errors can occur in the model run on a host exposing the web service, on the web service server, within SOAP communication including network errors, and in creation and operation of threads. They can be handled by including the appropriate method calls into a eval block. Eval is a function provided by Perl. I case of error, Perl skips the rest of the block, sets the variable $@ to the corresponding error text and continues with the program. Thus, run-time errors can be catched and handled. Errors that occur on a web services server are communicated via SOAP. SOAP::Lite provides a full range of error and exception handling methods. In running a distributed system in a heterogeneous environment a special problem must be considered. The calculations are performed on hosts with different resources. While a slow host still processes cell n, a faster hosts might already process cell m, where m4n: If an error occurs at that slower host, cell n remains unprocessed. Any additional call to the web service running at this host might add additional unprocessed cells. Therefore, in case, there is a failure of a thread, the entire thread is closed. In addition, the method nextIdifError is called. It sets the grid id to the cell id where the error occurred and the remaining threads continue with this id. In case, a required minimum number of threads cannot be run, the administrative script will stop the system and indicate an incomplete run.
1965
5. Application example The presented system has been applied to regional runs of the SIM–BIM model for the area of Germany. Seven PCs and workstations with different resources and operating systems including Linux, Windows2000 and Windows NT provide their CPU time to run the model. In a test case, a forecast of the isoprene emissions from Oaks for the Julian day 161 of the year 2002 and 1500 grid cells has been run with different numbers of hosts. Fig. 5 shows the decrease of required computation time. The host specifications are listed in Table 1. A SIM–BIM run for 1500 grid cells on the single host A1 requires 134 min of computation. By accessing a web service provided by the host A2, this time is already reduced to 69 min. Use of all available host reduces the computation time to only 33 min. Due to the different resources of the involved workstations, slower hosts contribute a smaller portion of calculated grid cells. Available hosts with any hardware contribute to the performance gain without slowing down faster hosts. It can be seen that the CPU speed and available memory are not the only determinants of the hosts performance. The outline of the entire workstation including network card, bios settings and network connection also play a significant role. Due to increasing network traffic and increasing requirements for administration of the distributed system, the performance gain decreases with the number of hosts. As a model run for a single cell does not require information from its neighboring grid cells, load balancing efforts are not needed. A 24-hour prediction for all 5940 grid cells needed for the total area of Germany required 130 min of computation time with all seven available hosts. Thus, using the idle time of the workstations during 14 h of night time the prediction range can be extended to 1 week. A key question of the presented approach is its stability. It has been tested with 100 runs using different numbers of
Fig. 5. Processing time versus number of used host in distributed run of the SIM–BIM model.
ARTICLE IN PRESS 1966
G. Smiatek / Atmospheric Environment 39 (2005) 1961–1966
hosts and calculated grid cells. All 100 runs have successfully been completed. In four cases, failures of a single host due to network problems have been experienced, but the runs continued with the remaining hosts.
German Program on Atmospheric Research (AFO2000) subproject BEWA2000.
6. Conclusions
References
The use of SOAP-based web services provides a very effective way of creating stable distributed applications. By exposing a single model run on a host offering CPU time as a web service a powerful computational environment is created at minimal costs. Any available workstation within a network can be used independent of the operating system, CPU type and speed. Although SOAP will principally allow for communication between the hosts, the system is specially suitable for onedimensional environmental models that run on single points or cells. The SOAP::Lite library simplifies the creation of a distributed computing environment to a minimum.
Box, D., 1999. House of COM. Microsoft Systems Journal 14 (9). Box, D., Ehnebuske, D., Kakivaya, G., Layman, A., Mendelsohn, N., Nielsen, H., Thatte, S., Winer, D., 2000. Simple Object Access Protocol (SOAP) 1.1., www.w3.org/TR/ SOAP/. Danelutto, M., 2003. HPC the easy way: new technologies for high performance application development and deployment. Journal of Systems Architecture 49 (10–11), 399–419. Guenther, A., 1997. Seasonal and spatial variations in the natural volatile organic compound emissions. Ecological Applications 7 (1), 34–45. Kulchenko, P., 2001. SOAP::Lite for Perl, www.soaplite.com. Lehning, A., Zimmer, W., Zimmer, I., Schnitzler, J.-P., 2001. Modeling of annual variations of oak (Quercus robur L.) isoprene synthase activity to oredict isoprene emission rates. Journal of Geophysical Research 106 (D3), 3157–3166. Pais, W.A., Fournier, N., Suttonc, M.A., Westonb, K.J., Dragositsc, U., 2004. Using high performance fortran to parallelise a multi-layer atmospheric transport model. Parallel Computing 30 (1), 21–33.
Acknowledgements The work presented was supported by the German Federal Ministry of Education and Research within the