- Email: [email protected]
Distributed computing for FTU data handling
Fusion Engineering and Design 60 (2002) 325– 331 www.elsevier.com/locate/fusengdes
Distributed computing for FTU data handling A. Bertocchi *, G. Bracco, G. Buceti, C. Centioli, E. Giovannozzi, F. Iannone, M. Panella, V. Vitale Associazione Euratom/ENEA sulla Fusione, Centro Ricerche Frascati, CP 65, 00044 Frascati, Rome, Italy
Abstract The growth of data warehouse in tokamak experiment is leading fusion laboratories to provide new IT solutions in data handling. In the last three years, the Frascati Tokamak Upgrade (FTU) experimental database was migrated from IBM-mainframe to Unix distributed computing environment. The migration efforts have taken into account the following items: (1) a new data storage solution based on storage area network over fibre channel; (2) andrew file system (AFS) for wide area network file sharing; (3) ‘one measure/one file’ philosophy replacing ‘one shot/one file’ to provide a faster read/write data access; (4) more powerful services, such as AFS, CORBA and MDSplus to allow users to access FTU database from different clients, regardless their O.S.; (5) large availability of data analysis tools, from the locally developed utility SHOW to the multi-platform Matlab, interactive data language and jScope (all these tools are now able to access also the Joint European Torus data, in the framework of the remote data access activity); (6) a batch-computing cluster of Alpha/CompaqTru64 CPU based on CODINE/GRD to optimize utilization of software and hardware resources. © 2002 Elsevier Science B.V. All rights reserved. Keywords: Andrew file system;
CORBA;
Storage area network
1. Introduction During last year FTU, Frascati Tokamak Upgrade [1], a compact high magnetic field tokamak, has reached the complete working configuration with three auxiliary heating systems to investigate radio frequency power deposition. It operates producing 20–25 shots per day, each shot being 1.5 s long and producing presently about 30 – 40MB of data for 1400 channels. This paper is organized in order to give account on FTU data * Corresponding author. Tel.: +39-6-9400-5339; fax: + 396-9400-5524. E-mail address: [email protected] (A. Bertocchi).
architecture, access methods, analysis tools and storage solutions. 2. Data architecture and access Thanks to the experience gained in the previous FT experiment, FTU started in 1990 operating from the beginning with all the raw and elaborated data archived using two standard header channel descriptions. So the FTU users have been always able to access data coming from all diagnostics ignoring any details about the hardware setting; a unique library call with the two keys, shot number and channel name, let the user get all the different data.
0920-3796/02/$ - see front matter © 2002 Elsevier Science B.V. All rights reserved. PII: S 0 9 2 0 - 3 7 9 6 ( 0 2 ) 0 0 0 2 8 - 5
326
A. Bertocchi et al. / Fusion Engineering and Design 60 (2002) 325–331
Until 1997, all channels were collected to form a logical (under VMS also a physical) single pulse file to be stored on a mass storage under IBM mainframe. Following the advent of the various Unix flavors, we left the IBM main frame (the last one being IBM 9672-R21 running MVS/ESA) on which the hierarchical storage management (HSM) storage system was in use to store the files. At the same time we adopted a unique header channel description (for raw and elaborated data) and we moved also from the architecture ‘one shot/one file’ toward the ‘one channel/one file’ philosophy (Fig. 1). This change was partially forced by the higher dimension in the amount of data per shot but, more importantly, to get the full parallelism in accessing data, both in write and read modes. Of course, an effect of this architecture is a big proliferation of files because one pulse file generates more than 1000 channelfiles. In terms of occupancy, we may consider as an example the FTU shot number 17 826. In this case, to store 16 061 030 bytes of acquired data for 1093 channels, i.e. 1093 files, the system takes 16 792 000 bytes i.e. about 4% more than the original amount. This modification has greatly enhanced the performance both in acquisition phase and in retrieval of data. Now just after the end of the shot, the main channels (like the plasma current) are
available for graphic display and data analysis because there is no need to wait for a ‘pulse file’ closing. While an remote procedure call server has been installed to access the archive in writing mode by the data acquisition system, the data analysis tools may access data through several ways: AFS [2], MDSplus [3], CORBA [4].
2.1. Andrew file system The continuous growth of data, their spread over wide area network, the increasing user demands, the multi-vendor environment with several operating systems, have led towards the adoption of a robust shared file system. AFS is a client/server architecture that allows to access files and directories residing on geographically distributed machines like a unique virtual machine, under the /afs filesystem. Fig. 2 shows the AFS fusione.it cell structure. In particular all the FTU experimental data are under /afs and a single channel/file has name:/afs/fus/ project/ftudati/das/hundred/number/family name/ channel name. For example, the CO2 diagnostic is inside the following path:/afs/fus/project/ftudati/ddb / – 100000/ – 100902 – /co2/ – co2dens.
Fig. 1. One channel/one file architecture.
A. Bertocchi et al. / Fusion Engineering and Design 60 (2002) 325–331
327
ing VMS which is excluded from the AFS architecture), now it’s possible to access data from a PC using a local tool like interactive data language (IDL) [7] or Matlab [8] by integrating the routine calling the CORBA method to access data. Because of the very short but successful time spent in developing this first CORBA implementation, we are convinced to further pursue with CORBA in distributed computing direction [9].
3. Analysis tools
Fig. 2. AFS fusione.it cell structure.
The only drawback from the AFS usage is the implementation of a third level redundancy we cannot implement as typically occurs in storage area network (SAN) architecture [5] because the AFS file server mounts only disk partitions not shared with other file servers.
2.2. MDSplus MDSplus is a complex set of applications for use in data acquisition and analysis standard ‘defacto’ in the fusion community. An MDSplus server has been installed at FTU for remote data access. Taking advantage of the uniform description of the FTU data, the installation of MDSplus has requested a couple of days, the interface layer being just the library call with the two keys, shot number and channel name.
2.3. CORBA If AFS is a stable and reliable solution for a distributed data file system, CORBA represents the standard for distributed computing. The first implementation of a CORBA bus in the FTU environment has been a CORBA server for the data archive. Using the free package OMNIORB [6], available for most popular platforms (includ-
A unified high level interface, namely one routine, has been developed to access data for all channels (Fig. 3). This routine includes several packages developed under the responsibility of each plasma diagnostic group. The drawback of this approach is most of these packages are platform dependent forcing the data servers to run on the Alpha/CompaqTru64 machines.
3.1. SHOX The traditional utility for graphical analysis between the FTU users is SHOX [10], written in Fortran 90. This program has been developed over the years the first version, SHOW, was running under IBM mainframe for the FT experiment. Because of the flexible structure of the code, it has been possible move to SHOX across various flavors of Unix relieving the users from the burden of the operating system change. In the framework of the new operation scheme of the Joint European Torus (JET) [11] facility started in 2000 SHOX has been ported also to JET Linux Cluster and interfaced with JET data, so that FTU scientists can use the same utility both on FTU and on JET data.
3.2. jScope jScope is a Java tool for the display of waveforms. Its concepts derive from the MDSPlus Scope, the MDSplus tool for data visualization. After the MDSplus server installation, on FTU
328
A. Bertocchi et al. / Fusion Engineering and Design 60 (2002) 325–331
has been possible to take advantage of the data visualization tools developed over the MDSplus layer. Particularly, jScope [12] has been installed and give the users the chance to display data directly on their own desktop, PC/WNT, MacOS X and Linux.
3.3. Commercial tools IDL and Matlab are also normally used. For commercial license reasons, most users login directly into the Compaq UNIX nodes to start the Matlab or IDL session but in principle, through the CORBA layer or AFS client, it is also possible to perform the same analysis on the desktop. The large availability of data analysis tools can largely cope with the user needs. In practice, bottlenecks come from: 1. The software with no client– server architecture which force users to login on the Alpha Compaq nodes.
2. The location of the commercial software license, like Matlab. A 100MB fiber distributed data interface (FDDI) loop links the entire sub local area network and the available bandwidth is rarely filled. Nevertheless, a new layout of the network will be implemented to substitute the FDDI connection with the new standard GigaEthernet and install a local router for the FTU nodes in order to filter any trouble coming from the lab router.
3.4. User experience Users show the attitude in using always the same visualization tool so the traditional SHOX still continue to be popular in spite of the fact that jScope can run directly on desktop. Some preliminary test has shown that the standard CORBA access has time access to data comparable with the MDSplus server. So the user
Fig. 3. FTU data access and analysis tool.
A. Bertocchi et al. / Fusion Engineering and Design 60 (2002) 325–331
329
aiming to take advantage of general tools like Matlab are presently using the CORBA layer. Finally, AFS offers the advantage to access data at file system level allowing the users to develop any peculiar elaboration.
4. FTU data storage Several storage systems have been evaluated before moving away from the HSM IBM mainframe. Experiments have been done with the Digital RAID Array 450 and a SCSI JBOD system. Finally, the SAN over fiber channel (FC) technology was adopted as the storage architecture. SAN over FC represent a major departure, a true paradigm shift, from the traditional distributed computing networks where storage is directly attached to their dedicated servers using SCSI’s parallel bus. The SAN, a high-speed, high-performance network, permits multi-platform servers with heterogeneous operating systems to equally access multi-vendor storage devices. With a SAN, a data-centric model replaces the server-centric model of the traditional IT infrastructure. Sometimes referred to as the ‘network behind the servers’, SAN allows anyto-any connectivity between servers and storage, using storage resource management software and interconnect components such as host bus adapters, bridges, hubs and switches. Multiple servers can access multiple storage systems through redundant paths, resulting in much higher data availability and speeds. All data were moved to a Dot– Hill’s SANnet 4200, an external hardware RAID with two FC host channels (ranging from 100MB/s in single host server to 200MB/s for clustered host configuration) and five 80MB/s Ultra SCSI (LVD) disk drive channels. Two Raids SANnet 4200 have been installed in single controller configuration, each with ten 73GB UW SCSI LVD. Seagate disks (10 000 RPM) for a total capacity of 1.46TB. The first level of redundancy, between Global Spare Disk and RAID 5 level, reduces the native storage capacity of 730GB of each RAID to 489GB with a capacity loss of 33%.
Fig. 4. FTU data storage system.
Each RAID controller has 256MB of cache memory. A ‘point-to-point otherwise loop’ topology, i.e. FC-Arbitred Loop between two points, has been adopted with four SUN ULTRA 10 servers equipped with FC Host Bus Adapter Emulex LP 8000 in DE9 layout. The SANnet systems have been configured in RAID 5 level with one Global Spare Disk for each RAID. A full throughoutput of 200MB/s for each RAID controller can be achieved. An automatic tape library (ATL) 15 AIT Cartridge Treefrog of Spectralogic with differential SCSI adapter is used for data back up. As disaster recovery strategy, a further copy of the whole archive is weekly done on a different ATL system located in a different building of our lab. Presently about 200GB of data are available on the FTU data storage system, 140GB being experimental data (raw and elaborated) and about 60GB users data. Fig. 4 shows a schematic block diagram of FTU storage systems.
5. Batch computing, from LSF to CODINE/GRD FTU data processing and analysis is performed on a cluster of five Alpha/Compaq True64 under AFS. Homemade software are planned to be migrated to Linux operating system now that the open source project of compilers and numerical libraries allows solving the bugs quickly. Presently FTU Alpha Cluster uses
A. Bertocchi et al. / Fusion Engineering and Design 60 (2002) 325–331
330 Table 1 FTU cluster hosts Host
Type
Model
cpuf
S05 S08 S09 S10 S11 S12
ALPHA ALPHA ALPHA ALPHA ALPHA ALPHA
ALPHASERVER1200 ALPHA500AU ALPHA233 ALPHASERVER4100 ALPHA500AU ALPHA500AU
2 3 1 4 3 3
(400 (500 (233 (600 (500 (500
load sharing facility (LSF) [13], a software package for distributed load sharing and batch queuing in heterogeneous Unix environments. LSF manages jobs processing by providing a transparent, single view of all hardware and software resources and optimizing their use. Table 1 shows the static information of the cluster’s hosts. Under ftu queue post pulse processing is performed by batch jobs running with high priority and saving the post elaborated data in the AFS file system. AFS file sharing shows the same home directory of the users on all hosts in the cluster so that jobs can run everywhere no matter how they access files. Codine/GRD [14], a further cluster software management, is presently under test; Codine is a software package still targeted for heterogeneous networked environments, in particular large workstation clusters with integrated computing servers, like vector and parallel computers. Codine is a free distribution for SUN Solaris and LINUX. Codine/GRD and LSF are functionally very similar. Both packages support batch, interactive and parallel jobs with sharing login and provide dynamic and static load balancing. Codine/GRD supports check pointing while in LSF security is a major issue. An additional product based on Codine is called GRD. This global resource director provides optimal utilization of computing resources. GRD allows policyoriented modification of priorities during execution and guarantees optimum turnout times for batch users. GRD is a GUI-based, commercial administration tool that provides enterprises with powerful and cost-effective workload management.
MHz) MHz) MHz) MHz) MHz) MHz)
ncpus
maxmem
maxswp
Server
2 1 1 2 1 1
495 118 149 997 118 118
1001 292 403 2001 491 491
yes no yes yes yes yes
6. Further developments At FTU, all data are on line, accessible from every platform, over a network with a large bandwidth (E 100MB) and a large suite of analysis tools are available to perform any kind of elaboration. A further step in the evolution of data handling may be taken in the following directions: 1. Create customized data-mining and knowledge extraction tools. Object oriented database [15] can help in this direction but the creation of some metadata is not strictly related to any particular technical solution 2. Plan and implement scalable architectures for data storage in order to prevent the advent of sophisticated diagnostics acquiring large amounts of data 3. Take a special care in remote data access tools while large international fusion device is becoming the perspective of tokamak research. References [1] R. Andreani, et al., Fusion technology 1990, in: Proceedings of the 16th Symposium On Fusion Technology (SOFT) London 1990, vol. 1, North Holland, Amsterdam (1991) 218. [2] Available from http://www.transarc.ibm.com/. [3] Available from http://www.mdsplus.org/. [4] Available from http://www.corba.org/. [5] Available from http://www.dothill.com/. [6] Available from http://www.uk.research.att.com/ omniORB/. [7] Available from http://www.rsinc.com/. [8] Available from http://www.mathworks.com/products/. [9] A. Bertocchi, G. Buceti, C. Centioli, D. Di Muzio, F. Iannone, M. Panella, E. Vitale, Corba technology in Reengineering the FTU Data Acquisition System, this conference.
A. Bertocchi et al. / Fusion Engineering and Design 60 (2002) 325–331 [10] G. Bracco, G. Buceti, A. Imparato, M. Panella, S. Podda, G.B. Righetti, O. Tudisco, V. Zanza, Data analysis software on the FTU experiment and its recent developments, Fusion Eng. Des. 43 (1999) 425 – 432 First Workshop of the IAEA Technical Committee Meeting on Data Acquisition and Management for Fusion Research, IPP Garching, Germany, July 22 – 24, 1997. [11] Available from http://www.jet.efda.org/pages/content/ jia.html. [12] G. Manduchi, C. Taliercio, A. Lucchetta, The Java inter-
331
face of MDSplus: towards a unified approach for local and remote data access, Fusion Eng. Des. 48 (1,2) (2000) 2000. [13] Available from http://www.platform.com/. [14] Available from http://www.sun.com/grid/. [15] A. Bertocchi, G. Bracco, G. Buceti, C. Centioli, F. Iannone, G. Manduchi, U. Nanni, M. Panella, C. Stracuzzi, V. Vitale, Recent developments and object-oriented approach in FTU database, Fusion Eng. Des. 56-57 (2001) 981 – 986.
Distributed computing for FTU data handling A. Bertocchi *, G. Bracco, G. Buceti, C. Centioli, E. Giovannozzi, F. Iannone, M. Panella, V. Vitale Associazione Euratom/ENEA sulla Fusione, Centro Ricerche Frascati, CP 65, 00044 Frascati, Rome, Italy
Abstract The growth of data warehouse in tokamak experiment is leading fusion laboratories to provide new IT solutions in data handling. In the last three years, the Frascati Tokamak Upgrade (FTU) experimental database was migrated from IBM-mainframe to Unix distributed computing environment. The migration efforts have taken into account the following items: (1) a new data storage solution based on storage area network over fibre channel; (2) andrew file system (AFS) for wide area network file sharing; (3) ‘one measure/one file’ philosophy replacing ‘one shot/one file’ to provide a faster read/write data access; (4) more powerful services, such as AFS, CORBA and MDSplus to allow users to access FTU database from different clients, regardless their O.S.; (5) large availability of data analysis tools, from the locally developed utility SHOW to the multi-platform Matlab, interactive data language and jScope (all these tools are now able to access also the Joint European Torus data, in the framework of the remote data access activity); (6) a batch-computing cluster of Alpha/CompaqTru64 CPU based on CODINE/GRD to optimize utilization of software and hardware resources. © 2002 Elsevier Science B.V. All rights reserved. Keywords: Andrew file system;
CORBA;
Storage area network
1. Introduction During last year FTU, Frascati Tokamak Upgrade [1], a compact high magnetic field tokamak, has reached the complete working configuration with three auxiliary heating systems to investigate radio frequency power deposition. It operates producing 20–25 shots per day, each shot being 1.5 s long and producing presently about 30 – 40MB of data for 1400 channels. This paper is organized in order to give account on FTU data * Corresponding author. Tel.: +39-6-9400-5339; fax: + 396-9400-5524. E-mail address: [email protected] (A. Bertocchi).
architecture, access methods, analysis tools and storage solutions. 2. Data architecture and access Thanks to the experience gained in the previous FT experiment, FTU started in 1990 operating from the beginning with all the raw and elaborated data archived using two standard header channel descriptions. So the FTU users have been always able to access data coming from all diagnostics ignoring any details about the hardware setting; a unique library call with the two keys, shot number and channel name, let the user get all the different data.
0920-3796/02/$ - see front matter © 2002 Elsevier Science B.V. All rights reserved. PII: S 0 9 2 0 - 3 7 9 6 ( 0 2 ) 0 0 0 2 8 - 5
326
A. Bertocchi et al. / Fusion Engineering and Design 60 (2002) 325–331
Until 1997, all channels were collected to form a logical (under VMS also a physical) single pulse file to be stored on a mass storage under IBM mainframe. Following the advent of the various Unix flavors, we left the IBM main frame (the last one being IBM 9672-R21 running MVS/ESA) on which the hierarchical storage management (HSM) storage system was in use to store the files. At the same time we adopted a unique header channel description (for raw and elaborated data) and we moved also from the architecture ‘one shot/one file’ toward the ‘one channel/one file’ philosophy (Fig. 1). This change was partially forced by the higher dimension in the amount of data per shot but, more importantly, to get the full parallelism in accessing data, both in write and read modes. Of course, an effect of this architecture is a big proliferation of files because one pulse file generates more than 1000 channelfiles. In terms of occupancy, we may consider as an example the FTU shot number 17 826. In this case, to store 16 061 030 bytes of acquired data for 1093 channels, i.e. 1093 files, the system takes 16 792 000 bytes i.e. about 4% more than the original amount. This modification has greatly enhanced the performance both in acquisition phase and in retrieval of data. Now just after the end of the shot, the main channels (like the plasma current) are
available for graphic display and data analysis because there is no need to wait for a ‘pulse file’ closing. While an remote procedure call server has been installed to access the archive in writing mode by the data acquisition system, the data analysis tools may access data through several ways: AFS [2], MDSplus [3], CORBA [4].
2.1. Andrew file system The continuous growth of data, their spread over wide area network, the increasing user demands, the multi-vendor environment with several operating systems, have led towards the adoption of a robust shared file system. AFS is a client/server architecture that allows to access files and directories residing on geographically distributed machines like a unique virtual machine, under the /afs filesystem. Fig. 2 shows the AFS fusione.it cell structure. In particular all the FTU experimental data are under /afs and a single channel/file has name:/afs/fus/ project/ftudati/das/hundred/number/family name/ channel name. For example, the CO2 diagnostic is inside the following path:/afs/fus/project/ftudati/ddb / – 100000/ – 100902 – /co2/ – co2dens.
Fig. 1. One channel/one file architecture.
A. Bertocchi et al. / Fusion Engineering and Design 60 (2002) 325–331
327
ing VMS which is excluded from the AFS architecture), now it’s possible to access data from a PC using a local tool like interactive data language (IDL) [7] or Matlab [8] by integrating the routine calling the CORBA method to access data. Because of the very short but successful time spent in developing this first CORBA implementation, we are convinced to further pursue with CORBA in distributed computing direction [9].
3. Analysis tools
Fig. 2. AFS fusione.it cell structure.
The only drawback from the AFS usage is the implementation of a third level redundancy we cannot implement as typically occurs in storage area network (SAN) architecture [5] because the AFS file server mounts only disk partitions not shared with other file servers.
2.2. MDSplus MDSplus is a complex set of applications for use in data acquisition and analysis standard ‘defacto’ in the fusion community. An MDSplus server has been installed at FTU for remote data access. Taking advantage of the uniform description of the FTU data, the installation of MDSplus has requested a couple of days, the interface layer being just the library call with the two keys, shot number and channel name.
2.3. CORBA If AFS is a stable and reliable solution for a distributed data file system, CORBA represents the standard for distributed computing. The first implementation of a CORBA bus in the FTU environment has been a CORBA server for the data archive. Using the free package OMNIORB [6], available for most popular platforms (includ-
A unified high level interface, namely one routine, has been developed to access data for all channels (Fig. 3). This routine includes several packages developed under the responsibility of each plasma diagnostic group. The drawback of this approach is most of these packages are platform dependent forcing the data servers to run on the Alpha/CompaqTru64 machines.
3.1. SHOX The traditional utility for graphical analysis between the FTU users is SHOX [10], written in Fortran 90. This program has been developed over the years the first version, SHOW, was running under IBM mainframe for the FT experiment. Because of the flexible structure of the code, it has been possible move to SHOX across various flavors of Unix relieving the users from the burden of the operating system change. In the framework of the new operation scheme of the Joint European Torus (JET) [11] facility started in 2000 SHOX has been ported also to JET Linux Cluster and interfaced with JET data, so that FTU scientists can use the same utility both on FTU and on JET data.
3.2. jScope jScope is a Java tool for the display of waveforms. Its concepts derive from the MDSPlus Scope, the MDSplus tool for data visualization. After the MDSplus server installation, on FTU
328
A. Bertocchi et al. / Fusion Engineering and Design 60 (2002) 325–331
has been possible to take advantage of the data visualization tools developed over the MDSplus layer. Particularly, jScope [12] has been installed and give the users the chance to display data directly on their own desktop, PC/WNT, MacOS X and Linux.
3.3. Commercial tools IDL and Matlab are also normally used. For commercial license reasons, most users login directly into the Compaq UNIX nodes to start the Matlab or IDL session but in principle, through the CORBA layer or AFS client, it is also possible to perform the same analysis on the desktop. The large availability of data analysis tools can largely cope with the user needs. In practice, bottlenecks come from: 1. The software with no client– server architecture which force users to login on the Alpha Compaq nodes.
2. The location of the commercial software license, like Matlab. A 100MB fiber distributed data interface (FDDI) loop links the entire sub local area network and the available bandwidth is rarely filled. Nevertheless, a new layout of the network will be implemented to substitute the FDDI connection with the new standard GigaEthernet and install a local router for the FTU nodes in order to filter any trouble coming from the lab router.
3.4. User experience Users show the attitude in using always the same visualization tool so the traditional SHOX still continue to be popular in spite of the fact that jScope can run directly on desktop. Some preliminary test has shown that the standard CORBA access has time access to data comparable with the MDSplus server. So the user
Fig. 3. FTU data access and analysis tool.
A. Bertocchi et al. / Fusion Engineering and Design 60 (2002) 325–331
329
aiming to take advantage of general tools like Matlab are presently using the CORBA layer. Finally, AFS offers the advantage to access data at file system level allowing the users to develop any peculiar elaboration.
4. FTU data storage Several storage systems have been evaluated before moving away from the HSM IBM mainframe. Experiments have been done with the Digital RAID Array 450 and a SCSI JBOD system. Finally, the SAN over fiber channel (FC) technology was adopted as the storage architecture. SAN over FC represent a major departure, a true paradigm shift, from the traditional distributed computing networks where storage is directly attached to their dedicated servers using SCSI’s parallel bus. The SAN, a high-speed, high-performance network, permits multi-platform servers with heterogeneous operating systems to equally access multi-vendor storage devices. With a SAN, a data-centric model replaces the server-centric model of the traditional IT infrastructure. Sometimes referred to as the ‘network behind the servers’, SAN allows anyto-any connectivity between servers and storage, using storage resource management software and interconnect components such as host bus adapters, bridges, hubs and switches. Multiple servers can access multiple storage systems through redundant paths, resulting in much higher data availability and speeds. All data were moved to a Dot– Hill’s SANnet 4200, an external hardware RAID with two FC host channels (ranging from 100MB/s in single host server to 200MB/s for clustered host configuration) and five 80MB/s Ultra SCSI (LVD) disk drive channels. Two Raids SANnet 4200 have been installed in single controller configuration, each with ten 73GB UW SCSI LVD. Seagate disks (10 000 RPM) for a total capacity of 1.46TB. The first level of redundancy, between Global Spare Disk and RAID 5 level, reduces the native storage capacity of 730GB of each RAID to 489GB with a capacity loss of 33%.
Fig. 4. FTU data storage system.
Each RAID controller has 256MB of cache memory. A ‘point-to-point otherwise loop’ topology, i.e. FC-Arbitred Loop between two points, has been adopted with four SUN ULTRA 10 servers equipped with FC Host Bus Adapter Emulex LP 8000 in DE9 layout. The SANnet systems have been configured in RAID 5 level with one Global Spare Disk for each RAID. A full throughoutput of 200MB/s for each RAID controller can be achieved. An automatic tape library (ATL) 15 AIT Cartridge Treefrog of Spectralogic with differential SCSI adapter is used for data back up. As disaster recovery strategy, a further copy of the whole archive is weekly done on a different ATL system located in a different building of our lab. Presently about 200GB of data are available on the FTU data storage system, 140GB being experimental data (raw and elaborated) and about 60GB users data. Fig. 4 shows a schematic block diagram of FTU storage systems.
5. Batch computing, from LSF to CODINE/GRD FTU data processing and analysis is performed on a cluster of five Alpha/Compaq True64 under AFS. Homemade software are planned to be migrated to Linux operating system now that the open source project of compilers and numerical libraries allows solving the bugs quickly. Presently FTU Alpha Cluster uses
A. Bertocchi et al. / Fusion Engineering and Design 60 (2002) 325–331
330 Table 1 FTU cluster hosts Host
Type
Model
cpuf
S05 S08 S09 S10 S11 S12
ALPHA ALPHA ALPHA ALPHA ALPHA ALPHA
ALPHASERVER1200 ALPHA500AU ALPHA233 ALPHASERVER4100 ALPHA500AU ALPHA500AU
2 3 1 4 3 3
(400 (500 (233 (600 (500 (500
load sharing facility (LSF) [13], a software package for distributed load sharing and batch queuing in heterogeneous Unix environments. LSF manages jobs processing by providing a transparent, single view of all hardware and software resources and optimizing their use. Table 1 shows the static information of the cluster’s hosts. Under ftu queue post pulse processing is performed by batch jobs running with high priority and saving the post elaborated data in the AFS file system. AFS file sharing shows the same home directory of the users on all hosts in the cluster so that jobs can run everywhere no matter how they access files. Codine/GRD [14], a further cluster software management, is presently under test; Codine is a software package still targeted for heterogeneous networked environments, in particular large workstation clusters with integrated computing servers, like vector and parallel computers. Codine is a free distribution for SUN Solaris and LINUX. Codine/GRD and LSF are functionally very similar. Both packages support batch, interactive and parallel jobs with sharing login and provide dynamic and static load balancing. Codine/GRD supports check pointing while in LSF security is a major issue. An additional product based on Codine is called GRD. This global resource director provides optimal utilization of computing resources. GRD allows policyoriented modification of priorities during execution and guarantees optimum turnout times for batch users. GRD is a GUI-based, commercial administration tool that provides enterprises with powerful and cost-effective workload management.
MHz) MHz) MHz) MHz) MHz) MHz)
ncpus
maxmem
maxswp
Server
2 1 1 2 1 1
495 118 149 997 118 118
1001 292 403 2001 491 491
yes no yes yes yes yes
6. Further developments At FTU, all data are on line, accessible from every platform, over a network with a large bandwidth (E 100MB) and a large suite of analysis tools are available to perform any kind of elaboration. A further step in the evolution of data handling may be taken in the following directions: 1. Create customized data-mining and knowledge extraction tools. Object oriented database [15] can help in this direction but the creation of some metadata is not strictly related to any particular technical solution 2. Plan and implement scalable architectures for data storage in order to prevent the advent of sophisticated diagnostics acquiring large amounts of data 3. Take a special care in remote data access tools while large international fusion device is becoming the perspective of tokamak research. References [1] R. Andreani, et al., Fusion technology 1990, in: Proceedings of the 16th Symposium On Fusion Technology (SOFT) London 1990, vol. 1, North Holland, Amsterdam (1991) 218. [2] Available from http://www.transarc.ibm.com/. [3] Available from http://www.mdsplus.org/. [4] Available from http://www.corba.org/. [5] Available from http://www.dothill.com/. [6] Available from http://www.uk.research.att.com/ omniORB/. [7] Available from http://www.rsinc.com/. [8] Available from http://www.mathworks.com/products/. [9] A. Bertocchi, G. Buceti, C. Centioli, D. Di Muzio, F. Iannone, M. Panella, E. Vitale, Corba technology in Reengineering the FTU Data Acquisition System, this conference.
A. Bertocchi et al. / Fusion Engineering and Design 60 (2002) 325–331 [10] G. Bracco, G. Buceti, A. Imparato, M. Panella, S. Podda, G.B. Righetti, O. Tudisco, V. Zanza, Data analysis software on the FTU experiment and its recent developments, Fusion Eng. Des. 43 (1999) 425 – 432 First Workshop of the IAEA Technical Committee Meeting on Data Acquisition and Management for Fusion Research, IPP Garching, Germany, July 22 – 24, 1997. [11] Available from http://www.jet.efda.org/pages/content/ jia.html. [12] G. Manduchi, C. Taliercio, A. Lucchetta, The Java inter-
331
face of MDSplus: towards a unified approach for local and remote data access, Fusion Eng. Des. 48 (1,2) (2000) 2000. [13] Available from http://www.platform.com/. [14] Available from http://www.sun.com/grid/. [15] A. Bertocchi, G. Bracco, G. Buceti, C. Centioli, F. Iannone, G. Manduchi, U. Nanni, M. Panella, C. Stracuzzi, V. Vitale, Recent developments and object-oriented approach in FTU database, Fusion Eng. Des. 56-57 (2001) 981 – 986.