- Email: [email protected]
Status of JT-60 data processing system
Fusion Engineering and Design 48 (2000) 99 – 104 www.elsevier.com/locate/fusengdes
Status of JT-60 data processing system T. Matsuda a,*, T. Tsugita a, T. Oshima a, S. Sakata a, M. Sato a, M. Koiwa a, T. Aoyagi b a
Naka Fusion Research Establishment, Japan Atomic Energy Research Institute, 801 -1 Mukouyama Naka-machi, Naka-gun, Ibaraki 311 -0193, Japan b Research Organization for Information Science and Technology, Tokyo 105 -0013, Japan Received 1 July 1999; received in revised form 17 December 1999; accepted 29 March 2000
Abstract The JT-60 data processing system is a large computer complex and gradually modernized by utilizing progressing computer and network technology. There are two major changes in our system. A main computer of FACOM M-780 has been replaced with compatible GS8300 using state-of-art CMOS technology, which results in lower power and space usage with nearly the same performance. Now it can handle 500 MB of data per discharge. A gigabit ethernet switch with FDDI ports has been introduced to cope with the increase of handling data. The switch will connect a tera-byte (TB) data server at the bandwidth of a gigabit per second with the main computer and many data acquisition workstations. Other developments in our system are the realization of three workstation-based plans, the TB data server, the VME-based fast data acquisition system and a CICU. The TB data server is basically a UNIX workstation with 100 GB RAID disks and 900 GB MO auto-exchangers. The VME-based fast data acquisition system has been developed to enlarge the present TMDS. The CICU, which has a function of interfacing the main computer with the CAMAC system, has been replaced with the workstation-based system after the fine tuning. © 2000 Elsevier Science S.A. All rights reserved. Keywords: JT-60; Data processing; Workstation; CMOS; Gigabit ethernet; FDDI; VMEbus; RAID disk; MO auto-exchanger
1. Introduction JT-60 data processing system has been the main data acquisition and processing computer system in the JT-60 diagnostic system since the start of the operation in 1985. It also controls each diag* Corresponding author. Tel.: +81-29-2707662; fax: + 8129-2707419. E-mail address: [email protected] (T. Matsuda).
nostics by using a CAMAC system. The original system was a large computer complex of a mainframe computer, mini-computers and microcomputers and gradually modernized by utilizing progressing computer and network technology [1,2]. As shown in Table 1, many workstations connected by fast networks are now applied to subsystems of the data processing system. Details of the status and recent developments of the JT-60 data processing system are described.
0920-3796/00/$ - see front matter © 2000 Elsevier Science S.A. All rights reserved. PII: S 0 9 2 0 - 3 7 9 6 ( 0 0 ) 0 0 1 4 0 - X
T. Matsuda et al. / Fusion Engineering and Design 48 (2000) 99–104
100
2. System configuration The configuration of the JT-60 data processing system is shown schematically in Fig. 1. It communicates with the JT-60 control system to synchronize the discharge sequence and exchange various data. Acquired raw data from each diagnostics are stored in the system and processed data are transferred to the JT-60 database server for the data analysis. The mainframe computer, inter-shot processor (ISP), supervises other computers in the system. CAMAC interface control unit (CICU) interfacing ISP with the CAMAC system communicates with ACM-A (auxiliary crate controller with micro-processor, type (A) in local CAMAC crates and collects CAMAC data to send them to ISP via data acquisition network, DAQnet. DAQnet also connects fast VME data acquisition systems (FDS), a tera-byte (TB) data server for the fast data acquisition. A communication computer and a real time processor (RTP) are communicate with ISP via a diagnostics LAN, connecting many diagnostic workstations. Now the system can handle 500 MB of raw data during a discharge; 50 MB of CAMAC data, 100 MB of FDS data and 350 MB of transient mass data storage system (TMDS) data.
3. Recent developments There were two major changes in our system recently. The main computer, ISP, of FACOM M-780/10 has been replaced with a compatible Table 1 Composition of JT-60 data processing 1. 2. 3. 4. 5. 6.
Inter-Shot Processor (ISP): Fujitsu GS8300/20 CAMAC System: FORCE CPU 5VTa (CICU) Timing System Real Time Processor (RTP): Concurrent System 9100 Communication Computer: Sun SPARCstation IPX Transient Mass Data Storage System (TMDS): FACOM A-400 7. Fast VME Data Acquisition System (FDS): Sun SPARCstation 20 8. TB Data Server: Sun Ultra Enterprise 3000 a
Compatible to Sun SPARCstation 5.
mainframe computer GS8300/20 using state-of-art CMOS technology. This replacement results in lower power and space requirements with nearly the same performance, shown in Table 2. GS8300 with 2 CPU’s seems to be effective in reducing elapsed time of I/O dominant jobs, though each CPU is less powerful. All peripherals and software were easily transferred to the new computer and the replacement was realized smoothly after careful benchmark tests using various data processing programs and data. A gigabit ethernet switch, Cabletron SmartSwitch 9000, with FDDI ports has been introduced in DAQnet to cope with the increase of handling data, shown in Fig. 2. DAQnet is an isolated fast network for the fast data transfer and is used for two purposes; the data acquisition in the JT-60 data processing system and the data transfer to the JT-60 database server. To separate these two kinds of traffics, the fast switch was added to the network consisting of an FDDI concentrator. The switch will connect the TB data server at the bandwidth of a gigabit per second with ISP and many data acquisition workstations, such as CICU, FDS and new TMDS workstations, described in Section 4. Now, the function of the FDDI switch is already used for a running test. Other developments accomplished in many subsystems of JT-60 data processing system are as follows: 1. The RTP system has been upgraded to a RISC-based workstation to meet the requests of increasing signals and processing [3]. 2. The TB data server, basically a UNIX workstation with 100 GB RAID disks and 900 GB MO (magneto-optical disk) auto-exchangers, has been developed. Now it can receive data from the FDS directly and will be equipped with a gigabit ethernet port for the mass data transfer. 3. The FDS has been developed to enlarge the present TMDS. It can acquire the data of 6 MB per a channel with 1 or 5 ms sampling. Now there are 2 FDS’s with 24 channels. 4. The new CICU, which is composed of a workstation and a VMEbus byte serial highway driver, has been developed [4] and tested care-
T. Matsuda et al. / Fusion Engineering and Design 48 (2000) 99–104
101
Fig. 1. Configuration of the JT-60 data processing system.
fully. After the fine tuning and careful tests, the new CICU has been finally in practical operation without any major problems.
5. Several CAMAC modules were redesigned and a new timing module for VMEbus was developed with the help of our electronics division.
T. Matsuda et al. / Fusion Engineering and Design 48 (2000) 99–104
102
Table 2 Comparison between FACOM M780/10 and GS8300/20
No. of CPU Type of CPU Cooling method Power consumption Area of the base Elapsed timea a
M780/10
GS8300/20
1 ECL Water 34.0 kVA 47.2 m2 1.0
2 CMOS Air 1.2 kVA 1.75 m2 1.08
Relative values estimated from benchmark tests.
Fig. 2. DAQnet and the hardware interconnection.
CAMAC modules are related to the timing system and the optical communication, which are required for the addition of new diagnostics. Software has been also developed to support network environments of data acquisition and processing, shown in Fig. 3. Data acquired by FDS and other diagnostic workstations are transferred to ISP by using NEDB (network experimental database manager). NEDB for the TB data server is also developed for the direct data transfer from FDS and other diagnostic workstations. Processed data on ISP are transferred to the diagnostic database in JT-60 database server by using a data transfer program IFDB. In similar way, FAME (fast analyzer for MHD equilibrium) transfers equilibrium data [5] and the data acquisition system for the negative NBI transfers processed data to the diagnostic database in the database server. A subset of the database is also stored in the database server for the remote participation in JT-60 experiments [6].
Fig. 3. Data flow in the data acquisition and the diagnostic database utilizing network-based software NEDB, IFDB and NDBS.
T. Matsuda et al. / Fusion Engineering and Design 48 (2000) 99–104
103
Fig. 4. Transient mass data storage system TMDS (top), fast VME data acquisition system FDS (middle) and new TMDS (bottom).
4. Future plan TMDS, which is composed of a mini-computer and 61 channels of 4/6 MB memory modules with 5 ms sampling, will be replaced with a new system based on the technology of FDS, shown in Fig. 4. Digital input signals from various diagnostics are connected with electrical inputs of the VMEbus memory modules used in FDS. To cope with mass data transfer to the TB data server, they will be connected with the gigabit ethernet. One of outdated CAMAC ACM-A modules with Intel 8086 CPU will be replaced with a workstation by applying the hardware configuration used in new CICU, shown in Fig. 5. ACMA controls CAMAC modules in the crate on the serial highway, acquires data from them and .
transfers data to the main computer via CICU. New ACM will combine the function of CICU and original ACM-A and transfer CAMAC data to ISP by using NEDB. CTL (cartridge tape library) is a powerful data storage system on ISP, but the amount of data in a cartridge 250 MB is too small. New tape library system is now under investigation using a high capacity tape and SAN (storage area network) technology.
5. Summary Recent computer and network technology is utilized to renew and reinforce the JT-60 data processing system. The main computer has been replaced with the latest compatible model using
104
T. Matsuda et al. / Fusion Engineering and Design 48 (2000) 99–104
Fig. 5. CAMAC system configuration with CICU and new ACM.
CMOS technology. Workstations using VMEbus have been applied to replacements of many subsystems using mini-computers and microcomputers. The gigabit ethernet switch with FDDI ports, which seems the most promising network technology for the data acquisition and processing system, has been introduced for the fast data acquisition of several hundred MB per discharge.
Acknowledgements The authors would like to thank the members of JT-60 team for their helpful discussions and supports. They also thanks to Drs T. Ozeki, H. Ninomiya, A. Funahashi for their continuous supports.
.
References [1] T. Aoyagi, Examples of data processing systems: data processing system for JT-60, J. Plasma Fusion Res. (in Japanese) 72 (1996) 1370 – 1375. [2] T. Matsuda, T. Aoyagi, N. Saitoh, et al., Recent developments in JT-60 data processing system, Fusion Eng. Des. 43 (1999) 285 – 291. [3] S. Sakata, M. Koiwa, T. Aoyagi, et al., Real Time Processor in JT-60 Data Processing System, Fusion Eng. Des. JAERI technical report. [4] T. Aoyagi, M. Sato, S. Sakata, et al., Development of New CICU, JAERI-Tech 97-073, February 1998 (in Japanese). [5] Y. Hasegawa, Y. Nakamura, H. Shirai, et al., Development and performance of high speed processing system of magnetohydrodynamic equilibria for discharge analyses on the JT-60 tokamak, J. At. Energy Soc. Japan, 41 (1999) 48–56 (in Japanese). [6] T. Matsuda, T. Tsugita, T. Oshima, et al., Systems for remote participation in JT-60 experiments, Rev. Sci. Instrum. 70 (1999) 502 – 504.
Status of JT-60 data processing system T. Matsuda a,*, T. Tsugita a, T. Oshima a, S. Sakata a, M. Sato a, M. Koiwa a, T. Aoyagi b a
Naka Fusion Research Establishment, Japan Atomic Energy Research Institute, 801 -1 Mukouyama Naka-machi, Naka-gun, Ibaraki 311 -0193, Japan b Research Organization for Information Science and Technology, Tokyo 105 -0013, Japan Received 1 July 1999; received in revised form 17 December 1999; accepted 29 March 2000
Abstract The JT-60 data processing system is a large computer complex and gradually modernized by utilizing progressing computer and network technology. There are two major changes in our system. A main computer of FACOM M-780 has been replaced with compatible GS8300 using state-of-art CMOS technology, which results in lower power and space usage with nearly the same performance. Now it can handle 500 MB of data per discharge. A gigabit ethernet switch with FDDI ports has been introduced to cope with the increase of handling data. The switch will connect a tera-byte (TB) data server at the bandwidth of a gigabit per second with the main computer and many data acquisition workstations. Other developments in our system are the realization of three workstation-based plans, the TB data server, the VME-based fast data acquisition system and a CICU. The TB data server is basically a UNIX workstation with 100 GB RAID disks and 900 GB MO auto-exchangers. The VME-based fast data acquisition system has been developed to enlarge the present TMDS. The CICU, which has a function of interfacing the main computer with the CAMAC system, has been replaced with the workstation-based system after the fine tuning. © 2000 Elsevier Science S.A. All rights reserved. Keywords: JT-60; Data processing; Workstation; CMOS; Gigabit ethernet; FDDI; VMEbus; RAID disk; MO auto-exchanger
1. Introduction JT-60 data processing system has been the main data acquisition and processing computer system in the JT-60 diagnostic system since the start of the operation in 1985. It also controls each diag* Corresponding author. Tel.: +81-29-2707662; fax: + 8129-2707419. E-mail address: [email protected] (T. Matsuda).
nostics by using a CAMAC system. The original system was a large computer complex of a mainframe computer, mini-computers and microcomputers and gradually modernized by utilizing progressing computer and network technology [1,2]. As shown in Table 1, many workstations connected by fast networks are now applied to subsystems of the data processing system. Details of the status and recent developments of the JT-60 data processing system are described.
0920-3796/00/$ - see front matter © 2000 Elsevier Science S.A. All rights reserved. PII: S 0 9 2 0 - 3 7 9 6 ( 0 0 ) 0 0 1 4 0 - X
T. Matsuda et al. / Fusion Engineering and Design 48 (2000) 99–104
100
2. System configuration The configuration of the JT-60 data processing system is shown schematically in Fig. 1. It communicates with the JT-60 control system to synchronize the discharge sequence and exchange various data. Acquired raw data from each diagnostics are stored in the system and processed data are transferred to the JT-60 database server for the data analysis. The mainframe computer, inter-shot processor (ISP), supervises other computers in the system. CAMAC interface control unit (CICU) interfacing ISP with the CAMAC system communicates with ACM-A (auxiliary crate controller with micro-processor, type (A) in local CAMAC crates and collects CAMAC data to send them to ISP via data acquisition network, DAQnet. DAQnet also connects fast VME data acquisition systems (FDS), a tera-byte (TB) data server for the fast data acquisition. A communication computer and a real time processor (RTP) are communicate with ISP via a diagnostics LAN, connecting many diagnostic workstations. Now the system can handle 500 MB of raw data during a discharge; 50 MB of CAMAC data, 100 MB of FDS data and 350 MB of transient mass data storage system (TMDS) data.
3. Recent developments There were two major changes in our system recently. The main computer, ISP, of FACOM M-780/10 has been replaced with a compatible Table 1 Composition of JT-60 data processing 1. 2. 3. 4. 5. 6.
Inter-Shot Processor (ISP): Fujitsu GS8300/20 CAMAC System: FORCE CPU 5VTa (CICU) Timing System Real Time Processor (RTP): Concurrent System 9100 Communication Computer: Sun SPARCstation IPX Transient Mass Data Storage System (TMDS): FACOM A-400 7. Fast VME Data Acquisition System (FDS): Sun SPARCstation 20 8. TB Data Server: Sun Ultra Enterprise 3000 a
Compatible to Sun SPARCstation 5.
mainframe computer GS8300/20 using state-of-art CMOS technology. This replacement results in lower power and space requirements with nearly the same performance, shown in Table 2. GS8300 with 2 CPU’s seems to be effective in reducing elapsed time of I/O dominant jobs, though each CPU is less powerful. All peripherals and software were easily transferred to the new computer and the replacement was realized smoothly after careful benchmark tests using various data processing programs and data. A gigabit ethernet switch, Cabletron SmartSwitch 9000, with FDDI ports has been introduced in DAQnet to cope with the increase of handling data, shown in Fig. 2. DAQnet is an isolated fast network for the fast data transfer and is used for two purposes; the data acquisition in the JT-60 data processing system and the data transfer to the JT-60 database server. To separate these two kinds of traffics, the fast switch was added to the network consisting of an FDDI concentrator. The switch will connect the TB data server at the bandwidth of a gigabit per second with ISP and many data acquisition workstations, such as CICU, FDS and new TMDS workstations, described in Section 4. Now, the function of the FDDI switch is already used for a running test. Other developments accomplished in many subsystems of JT-60 data processing system are as follows: 1. The RTP system has been upgraded to a RISC-based workstation to meet the requests of increasing signals and processing [3]. 2. The TB data server, basically a UNIX workstation with 100 GB RAID disks and 900 GB MO (magneto-optical disk) auto-exchangers, has been developed. Now it can receive data from the FDS directly and will be equipped with a gigabit ethernet port for the mass data transfer. 3. The FDS has been developed to enlarge the present TMDS. It can acquire the data of 6 MB per a channel with 1 or 5 ms sampling. Now there are 2 FDS’s with 24 channels. 4. The new CICU, which is composed of a workstation and a VMEbus byte serial highway driver, has been developed [4] and tested care-
T. Matsuda et al. / Fusion Engineering and Design 48 (2000) 99–104
101
Fig. 1. Configuration of the JT-60 data processing system.
fully. After the fine tuning and careful tests, the new CICU has been finally in practical operation without any major problems.
5. Several CAMAC modules were redesigned and a new timing module for VMEbus was developed with the help of our electronics division.
T. Matsuda et al. / Fusion Engineering and Design 48 (2000) 99–104
102
Table 2 Comparison between FACOM M780/10 and GS8300/20
No. of CPU Type of CPU Cooling method Power consumption Area of the base Elapsed timea a
M780/10
GS8300/20
1 ECL Water 34.0 kVA 47.2 m2 1.0
2 CMOS Air 1.2 kVA 1.75 m2 1.08
Relative values estimated from benchmark tests.
Fig. 2. DAQnet and the hardware interconnection.
CAMAC modules are related to the timing system and the optical communication, which are required for the addition of new diagnostics. Software has been also developed to support network environments of data acquisition and processing, shown in Fig. 3. Data acquired by FDS and other diagnostic workstations are transferred to ISP by using NEDB (network experimental database manager). NEDB for the TB data server is also developed for the direct data transfer from FDS and other diagnostic workstations. Processed data on ISP are transferred to the diagnostic database in JT-60 database server by using a data transfer program IFDB. In similar way, FAME (fast analyzer for MHD equilibrium) transfers equilibrium data [5] and the data acquisition system for the negative NBI transfers processed data to the diagnostic database in the database server. A subset of the database is also stored in the database server for the remote participation in JT-60 experiments [6].
Fig. 3. Data flow in the data acquisition and the diagnostic database utilizing network-based software NEDB, IFDB and NDBS.
T. Matsuda et al. / Fusion Engineering and Design 48 (2000) 99–104
103
Fig. 4. Transient mass data storage system TMDS (top), fast VME data acquisition system FDS (middle) and new TMDS (bottom).
4. Future plan TMDS, which is composed of a mini-computer and 61 channels of 4/6 MB memory modules with 5 ms sampling, will be replaced with a new system based on the technology of FDS, shown in Fig. 4. Digital input signals from various diagnostics are connected with electrical inputs of the VMEbus memory modules used in FDS. To cope with mass data transfer to the TB data server, they will be connected with the gigabit ethernet. One of outdated CAMAC ACM-A modules with Intel 8086 CPU will be replaced with a workstation by applying the hardware configuration used in new CICU, shown in Fig. 5. ACMA controls CAMAC modules in the crate on the serial highway, acquires data from them and .
transfers data to the main computer via CICU. New ACM will combine the function of CICU and original ACM-A and transfer CAMAC data to ISP by using NEDB. CTL (cartridge tape library) is a powerful data storage system on ISP, but the amount of data in a cartridge 250 MB is too small. New tape library system is now under investigation using a high capacity tape and SAN (storage area network) technology.
5. Summary Recent computer and network technology is utilized to renew and reinforce the JT-60 data processing system. The main computer has been replaced with the latest compatible model using
104
T. Matsuda et al. / Fusion Engineering and Design 48 (2000) 99–104
Fig. 5. CAMAC system configuration with CICU and new ACM.
CMOS technology. Workstations using VMEbus have been applied to replacements of many subsystems using mini-computers and microcomputers. The gigabit ethernet switch with FDDI ports, which seems the most promising network technology for the data acquisition and processing system, has been introduced for the fast data acquisition of several hundred MB per discharge.
Acknowledgements The authors would like to thank the members of JT-60 team for their helpful discussions and supports. They also thanks to Drs T. Ozeki, H. Ninomiya, A. Funahashi for their continuous supports.
.
References [1] T. Aoyagi, Examples of data processing systems: data processing system for JT-60, J. Plasma Fusion Res. (in Japanese) 72 (1996) 1370 – 1375. [2] T. Matsuda, T. Aoyagi, N. Saitoh, et al., Recent developments in JT-60 data processing system, Fusion Eng. Des. 43 (1999) 285 – 291. [3] S. Sakata, M. Koiwa, T. Aoyagi, et al., Real Time Processor in JT-60 Data Processing System, Fusion Eng. Des. JAERI technical report. [4] T. Aoyagi, M. Sato, S. Sakata, et al., Development of New CICU, JAERI-Tech 97-073, February 1998 (in Japanese). [5] Y. Hasegawa, Y. Nakamura, H. Shirai, et al., Development and performance of high speed processing system of magnetohydrodynamic equilibria for discharge analyses on the JT-60 tokamak, J. At. Energy Soc. Japan, 41 (1999) 48–56 (in Japanese). [6] T. Matsuda, T. Tsugita, T. Oshima, et al., Systems for remote participation in JT-60 experiments, Rev. Sci. Instrum. 70 (1999) 502 – 504.