- Email: [email protected]
Real-Time data acquisition Prototype proposal of the ITER radial neutron camera and gamma-ray spectrometer
G Model
ARTICLE IN PRESS
FUSION-9276; No. of Pages 5
Fusion Engineering and Design xxx (2017) xxx–xxx
Contents lists available at ScienceDirect
Fusion Engineering and Design journal homepage: www.elsevier.com/locate/fusengdes
Real-Time data acquisition Prototype proposal of the ITER radial neutron camera and gamma-ray spectrometer R.C. Pereira a,∗ , N. Cruz a , A. Fernandes a , J. Sousa a , C.M.B.A. Correia b , M. Riva c , C. Centioli c , D. Marocco c , M. Tardocchi d , M. Nocente e , B. Gonc¸alves a , B. Esposito c a
Instituto de Plasmas e Fusão Nuclear, Instituto Superior Técnico, Universidade de Lisboa, 1049-001 Lisboa, Portugal LIBPhys-UC, Dept. de Física, Universidade de Coimbra, 3004-516 Coimbra, Portugal c ENEA C.R. Frascati, Dipartimento FSN, Via E. Fermi, 45-Frascati, 00044 Rome, Italy d Istituto di Fisica del Plasma “P. Caldirola,” Consiglio Nazionale delle Ricerche, Milano, Italy e Dipartimento di Fisica “G. Occhialini,” Università degli Studi di Milano-Bicocca, Milano, Italy b
h i g h l i g h t s • • • • •
A high-frequency DAQ system based on PCI Express and FPGA. Prototype modularity and scalability allows a high no of channels architecture. Real-Time pulse processing algorithms: PSD, PHA. Up to 8 GB/s data throughput from digitizer to host. Real-Time lossless compress method to compress raw data.
a r t i c l e
i n f o
Article history: Received 3 October 2016 Received in revised form 16 March 2017 Accepted 17 March 2017 Available online xxx Keywords: RNC & RGRS diagnostics FMC High-Rate Data Acquisition PCI Express FPGA Real-Time
a b s t r a c t The Radial Neutron Camera (RNC) and the Radial Gamma-Ray Spectrometer (RGRS) are two ITER diagnostics, devoted, respectively, to the Real-Time (RT) measurement of the neutron emissivity profile and to the measurement of the confined alpha profile and runaway electrons. The two systems are closely related as they share the same equatorial port plug and part of the lines-of-sight and both require the acquisition of event-based signals from radiation detectors. The RNC Data Acquisition and Processing (DAQP) system should be capable of handling peak count rates of the order of 106 counts per second for a time duration up to 500 s. Moreover, for a continuous data throughput, the DAQP system of both diagnostics shall provide two separate DMA channels, capable to transfer simultaneously event-based data and RT processed data from the digitizers to the host. A DAQP prototype will be developed to identify and study critical issues. The present paper will: i) present the RNC DAQP prototype showing its compliancy with the RNC plant system Fast Controller; ii) show the scalability of the actual RNC DAQP from the prototype concept; iii) identify the differences between the RNC and RGRS DAQP needs; iv) describe the RGRS DAQP system and its interface to the ITER Control Data Access and Communication. © 2017 Elsevier B.V. All rights reserved.
1. Introduction The Radial Neutron Camera (RNC) and the Radial Gamma-Ray Spectrometer (RGRS) are two ITER diagnostics located in equatorial port #1. Their main roles are, respectively, the Real-Time (RT) measurement of the neutron emissivity profile (for plasma control
∗ Corresponding author. E-mail address: [email protected] (R.C. Pereira).
purposes) and the measurement of the confined alpha profile and runaway electrons. RGRS is one of the RNC enabled diagnostics, for late implementation, meaning that only interfaces must be prepared for the time being. The two systems are closely related as they share the same equatorial port plug and part of the Lines-OfSight (LOS) and both require the acquisition of event-based signals from radiation detectors. The framework Partnership agreement for RNC and RGRS diagnostics development and design (F4E-FPA-327) established the design activities for each diagnostic, to be performed under the
http://dx.doi.org/10.1016/j.fusengdes.2017.03.096 0920-3796/© 2017 Elsevier B.V. All rights reserved.
Please cite this article in press as: R.C. Pereira, et al., Real-Time data acquisition Prototype proposal of the ITER radial neutron camera and gamma-ray spectrometer, Fusion Eng. Des. (2017), http://dx.doi.org/10.1016/j.fusengdes.2017.03.096
G Model FUSION-9276; No. of Pages 5
ARTICLE IN PRESS R.C. Pereira et al. / Fusion Engineering and Design xxx (2017) xxx–xxx
2
contract. The RNC design activities are: i) System Level Design (SLD); ii) prototyping and testing; iii) engineering design. In particular, the functional specifications of RNC Data AcQuisition and Processing (DAQP) system must be detailed to the level where subsequent design and procurement activities can be undertaken without any additional input from the beneficiary. Regarding RGRS, design activities are limited to the SLD detailing the interfaces with the RNC, to be frozen during the RGRS Conceptual Design Review [1,2]. Furthermore the RNC diagnostic benefits from the knowledge obtained with the JET Neutron Profile Monitor diagnostic [3]. Section 2 will describe the two plant systems architecture in terms of plasma coverage which will define the number of acquisition channels to be considered on the diagnostic Instrumentation & Control (I&C) design. Section 3 will present: i) the ITER I&C rules and guidelines; ii) the reason for prototyping the RNC DAQP system, as well as the prototype itself; iii) the differences between RNC and RGRS features, crucial to the DAQP system design; and iv) the RGRS DAQP system interface to the ITER Control Data Access and Communication (CODAC). Finally some considerations will be made. 2. RNC & RGRS diagnostics To design a plant system I&C architecture it is a good practice to understand the nature of the diagnostic measurements as well as the type and number of sensors to read and/or actuators to control. 2.1. RNC measurements The RNC diagnostic is a collimated multichannel neutron detection system intended to characterize ITER fusion plasma neutron sources by retrieving both the neutron spectrum and the neutron emissivity granting the primary role in the ITER program for advanced control measurements and physics exploitation. The first design driver for this diagnostic is to measure the neutron flux in the conditions appropriate to infer the neutron emissivity profile. The target is to get the neutron and alpha source profile within the range 1014 –6 × 1018 n.s−1 m−3 , 10% of accuracy, 10 ms of time resolution and a/10 of spatial resolution [4]. 2.2. RNC lines-of-Sight RNC diagnostic is composed of two fan-shaped collimating structures: In-Port and Ex-port. RNC LOS are uniformly distributed in order to cover the plasma up to r/a ∼0.9. Regarding the latest development on the RNC diagnostic, it was defined, as design baseline, six architectural options covering three distinct architectural elements: i) measurement of neutron emissivity providing the best achievable performance; ii) costs and iii) technological problems/risk. These different architectures spans from a minimum possible number of LOS, 11, to a maximum of 27 LOS, and each architectural performance is going to be evaluated and ranked in order to choose the final architecture [5]. Concerning the RNC I&C system design it is worth to consider the architecture which presents the maximum number of LOS, 27, and that up to 270 channels are foreseen for the RNC I&C system. 2.3. RGRS measurements The RGRS diagnostic is a collimated multichannel gamma detection intended to provide time and spatial-resolved measurements of gamma-ray and Hard-X-Ray (HXR) emissivity supplying information on fast ions behaviour in burning plasmas, especially the alpha-particles, and runaway electrons distribution in ITER.
The RGRS primary role measurements are dedicated to physics exploitation only. No basic or advanced control are foreseen [6]. 2.4. RGRS lines-of-Sight The RGRS [7] will be installed behind the RNC detectors sharing 10 LOS. The detectors are displayed in linear geometry layout. The latest development envisages up to 2 detectors per each LOS. The central LOS will have 2 different detectors, a LaBr3 scintillator [8] and a High Purity Germanium [9]. Up to now, each detector features one channel only and no more than 20 channels are foreseen for the RGRS I&C system. 3. ITER plant system I&C architecture At ITER, all plant systems I&C must comply with the rules and guidelines presented at the ITER Plant Control Design Handbook. Therefore, an ITER plant system I&C can feature one or more plant system controller(s) interfacing to actuators and sensors via signal interface(s). The system Controllers can be divided into Fast Controllers (FC) for fast signals (data acquisition faster than 100 Hz) and Slow Controllers for measuring and activating slow Input/Output (I/O) channels (eg. detector temperatures, magnetic fields). The FCs which includes the specific diagnostic DAQP system, have some key criteria and requirements to apply, such as:i) PCI express (PCIe) and Ethernet will be used as I/O intercommunication and ii) no local storage[10,11]; Like other plant systems, both RNC and RGRS will interface with ITER central I&C systems through the ITER CODAC Plant Operation Network (PON) and High Performance Network (HPN). HPN comprises the Time Communication Network (TCN), Synchronous Databus Network (SDN), Data Archiving Network (DAN) and Audio Video Network (AVN). ITER plant systems will always interface PON and TCN, while depending on the diagnostic features, the SDN (RNC only), DAN (RNC and RGRS) and AVN (not used) may or may not be interfaced [10]. 3.1. RNC & RGRS DAQP systems ITER diagnostics will experiment variable and generally high count rates and long pulse discharges which will produce a huge amount of data. Therefore their DAQP systems must have the capability of RT pulse processing and RT data throughput, to overcome problems on the availability of the event base data, acquired from sensors, to ITER central I&C Systems. 3.1.1. RNC DAQP prototyping reasons The main purpose to have the RNC prototype is to: i) show that it is feasible to implement all the needed RT algorithms in the envisaged DAQP system with the expected performance. Although there is expertise in the implementation of RT algorithms, the highrate/high-bandwidth signals demand further R&D on new methods and testing; ii) measure the peak performance of the system (maximum count rate supported) to dimension the final system with optimal cost/performance relation; iii) Evaluate the need of RT data compression aiming at reducing the event based data throughput, and iv) evaluate if the spatial inversion algorithms can be employed in RT (R&D) [12]. The prototype does not define a complete system with a platform standard proposed by ITER (eg. xTCA, PxIe [13]), focusing only on the two identified drawbacks: i) data loss due to the DAQ architectural problem; ii) no advance control due to processing power problems. The prototype will answer the following architectural questions:
Please cite this article in press as: R.C. Pereira, et al., Real-Time data acquisition Prototype proposal of the ITER radial neutron camera and gamma-ray spectrometer, Fusion Eng. Des. (2017), http://dx.doi.org/10.1016/j.fusengdes.2017.03.096
G Model FUSION-9276; No. of Pages 5
ARTICLE IN PRESS R.C. Pereira et al. / Fusion Engineering and Design xxx (2017) xxx–xxx
Fig. 1. RT emissivity reconstruction flow chart based on the Tikhonov regularization, where bk is the line integrated signals from LOS k (neutron counts), and lkj the length of intersection of LOS k with magnetic surfaces j and j–1.
1. How many channels can be accomodated by the host, to avoid data loss, in an event base, from digitizer to host? 2. How many input channels can the Field Programmable Gate Array (FPGA) accomodate, to process, in RT, all detector signals? 3. Evaluation of the Gigabit Ethernet link (10 GbE or 40 GbE) in order to avoid data loss, in event base operation mode, from the host to the ITER databases. 4. Can the state-of-the-art FPGAs implement all RT algorithms at processing rates up to 200 MHz or higher? 5. Can the host perform the emissivity reconstruction in less than 10 ms? The third question will not be completely assessed by this prototype. It will be determined the throughput needed by the host, but the real performance of data througput from the host to databases will not be studied during the prototype phase. Finally, the emissivity reconstruction is divided into three distinct tasks, presented at Fig. 1 [12]. Only the processing code related to task 2 and 3 will be evaluated, by running it at the host with simulated data (neutron counts and equilibrium data). Task1, will only be assessed for 4 channels. Preliminary tests showed that for the maximum predicted RNC number of LOS, 27, and up to 20 magnetic surfaces (10 for RNC) the master host can perform the neutron profile within 10 ms. 3.1.2. RNC DAQP prototype To specify a DAQP system dedicated to a diagnostic with spectroscopic capabilities two types of features must be addressed: i) architectural aspects such as the number of channels and the maximum expected event rate; and ii) the characterization of the digitizer itself: Analog-to-Digital Converter (ADC) specifications and processing power capabilities. Therefore the knowledge of the detectors minimum pulse width (Full Width Half Maximum
3
(FWHM) – RNC detector signals FWHM spans from 10 to 200 ns), as well as the detectors capability of providing Pulse Height Spectra (PHS) and neutron/gamma PSD will define the ADC frequency (≥400 MHz) and its acceptable resolution (≥12 bit) [14]. The event rate as well as the detectors maximum pulse width define the data throughput. The aim of this prototype is to sustain a 2 MHz peak event. If the event is covered by 128 samples then one channel will produce ∼0.5 GB/s of raw data (event base). Four channels data throughput will be of ∼2 GB/s. The foreseen DAQP system physical throughput limitations are the PCIe interface between the digitizers and the link from host to ITER databases, DAN network (out of the scope of the DAQP prototype tests); To comply with the ITER rules and guidelines, previously stated, the best compromise between cost and performance of PCIe based solution was an Evaluation (EV) board from Xilinx, EK-K7-KC705G (KC705), with a high-performance Kintex-7 FPGA and a × 8 PCIe 2.0 interface. The board was also chosen due to the full access to the FPGA in order to test all the needed RT algorithms. The KC705 allows data throughput of 4 GB/s and features an I/O expansion High Pin Connector (HPC) to accommodate FPGA Mezzanine Cards (FMC). Depending on the choice of the FMC, the KC705 bandwidth can be divided into several channels. Presently, in the market, the analog-to-digital FMCs compatible with KC705 and complying with the diagnostic needs (sampling rate and resolution), feature only ac-coupling with an input bandwidth leading to distortions caused by the ac inherent high-pass filter. The best solution was a twochannel 12-bit 1.6 GHz FMC custom module featuring both ac/dccoupling solutions [15]. Therefore a KC705 will divide its 4 GB/s bandwidth with two channels. The aim of this prototype is to have two architectures: i) one host PC with two KC705 modules and four channels, Fig. 2a). This configuration evaluates the exact number of channels allowed by an host; ii) Two hosts with two channels each. In this case, one host will send processed data, more specifically the detectors brightness information, to the other host (master), through a 1 GbE link. This will test the performance of the master host to calculate the emissivity profile each 10 ms, Fig. 2b). 3.1.3. RNC real-time algorithms The RNC DAQP system implements RT fast data processing algorithms (≥200 MHz), both in the FPGA and host side environment. The FPGA processes the digitized data to give the necessary physical measurement, either n/␥ counts, energy values or spectra and Pile-Up (PU) occurrences. The RT pulse processing algorithms to implement will minimize data storage and transfer issues from digitizers to host, and are as follow: i) pulse detection featuring smart triggering system with PU management; ii) Pulse integration; iii) PSD; iv) PHS; and v) gain instability corrections; Concerning the host environment three distinct RT processing algorithms are envisaged. Two for data reduction (several orders of magnitude) of the event based data retrieved from the digitizers: i) pulse processing algorithms, the same defined for the FPGA environment; and ii) lossless data compression algorithms. The third RT algorithm calculates the neutron emissivity based on the retrieved FPGA processed data [12]. For continuous data throughput, the DAQP system firmware architecture shall provide two separate Direct Memory Access (DMA) channels with enough bandwidth, capable to transfer event based data packets and RT processed data packets from the digitizer to the host. For event based data packets, pulse windows, are stored in FPGA internal buffers. Pulse windows size depends on detection of PU. If no PU is assigned then the pulse window is of the size of the event, Pwidth (user defined, eg. 128 samples). Otherwise it can be incremented n times the Pwidth (number of PU assignments). The prototype will test a set of RT FPGA algorithms able to perform PU discrimination. Every time a PU occurs a pulse window is
Please cite this article in press as: R.C. Pereira, et al., Real-Time data acquisition Prototype proposal of the ITER radial neutron camera and gamma-ray spectrometer, Fusion Eng. Des. (2017), http://dx.doi.org/10.1016/j.fusengdes.2017.03.096
G Model
ARTICLE IN PRESS
FUSION-9276; No. of Pages 5
R.C. Pereira et al. / Fusion Engineering and Design xxx (2017) xxx–xxx
4
Fig. 2. Prototype architecture diagram block. a) one host PC with two digitizer modules; b) two PCs with one digitizer module each with an 1 GbE connection between PCs.
Table 1 Differences between RNC and RGRS for each of the DAQ system key features. Key features
LOS N◦ of channels Event rate FWHM Key features
Diagnostic Physical Features RGRS
RNC
6 20 (max) >1Mevents/s 150 ns
Up to 27 270 (max) 2 Mevent/s 10–200 ns
RT Processing Capabilities RGRS
Pile-Up discrimination/resolving n/␥ Pulse Discrimination PHS Data Compression Emissivity Reconstruction
FPGA √ – √ – –
RNC Host √ – √ √ –
FPGA √ √ √ – –
Host √ √ √ √ √
tagged as PU window to be processed, also in RT, in the host side with smarter algorithms, which due to its complexity cannot be implemented in FPGA. The host side will then attempt to resolve PU. 3.1.4. Differences between RNC and RGRS DAQP needs Assuming that the RGRS FC is to be based on the RNC DAQP prototype, differences between both systems are identified and presented at Table 1. The knowledge of the two diagnostics key features differences assures that the RNC DAQP system suits RGRS. RGRS presents much lower number of channels and less stringent needs for the digitizer characterization. From the point of view of processing capabilities, this diagnostic is less demanding as the PSD algorithm is not needed and RGRS will not provide RT feedback control. 3.1.5. RGRS DAQP system According to the functional analysis and to the processing needs described in the main documents, and to the functional requirements of the I&C system of the RGRS [16], more specifically on DAQP system, part of the RGRS FC shall: i) perform RT digital signal processing to transmit event based and processed packets to
CODAC for long term storage; ii) measure gamma flux and emissivity profile: the DAQP system must provide spectra for count rates higher than 1 Mevent/s and provide position and shape within a/10 spatial resolution and 10 ms/100 ms of temporal resolution for HXR/gamma-ray source; iii) calculate the fast ions and electron distributions; iv) interface with CODAC, physical location of network connections within the plant system cubicle (PON;TCN;DAN). Concerning the interface with CODAC, the PON interface will be responsible for: i) data acquisition (measurements); ii) alarm monitoring; iii) control and configuration: commands and configuration data from CODAC to the RGRS diagnostic; iv) collaborative data, i.e, non-RT data needed for offline analysis. The TCN interface will be responsible for timing information (absolute time and global trigger). Finally the DAN will be responsible for all archive data. The SDN Network provides transport for RT feedback control. No feedback control is expected for this diagnostic, therefore this connection is not foreseen. Fig. 3 presents a block diagram of RGRS Plant control system including the interfaces with CODAC networks.
4. Considerations Continuous operation is the ultimate goal for next generation of nuclear fusion devices, thus in order to be able to calculate the neutron and alpha source profile every 10 ms, a new developed DAQP system allows up to 8 GB throughput to host with RT pulse processing capabilities. Besides the RT pulse processing, other RT lossless compress methods are envisaged for the host side to compress the event base data, estimated to produce ∼0.5 GB/s per channel during 2 MHz of peak events. The modularity and scalability of the described prototype allow a short implementing time of the FC for the overall diagnostic channels. Although a platform standard is not yet settled the proposed architecture is easily available in the standards proposed by ITER. Work is ongoing to check whether more than 4 channels/host can be supported in order to decrease substantially the space allocation requirement in the cubicles and the cost of DAQP system. The limit of the scalability will be the space allocated for RNC at the ITER diagnostic cubicles which was based on the worst case of 8 channels per host.
Please cite this article in press as: R.C. Pereira, et al., Real-Time data acquisition Prototype proposal of the ITER radial neutron camera and gamma-ray spectrometer, Fusion Eng. Des. (2017), http://dx.doi.org/10.1016/j.fusengdes.2017.03.096
G Model FUSION-9276; No. of Pages 5
ARTICLE IN PRESS R.C. Pereira et al. / Fusion Engineering and Design xxx (2017) xxx–xxx
5
Fig. 3. RGRS Plant control system. I&C interfaces with CODAC networks.
Acknowledgments The work leading to this publication has been funded partially by Fusion for Energy under the Specific Grants Agreement F4E-FPA-327 SG03 & SG04. IST activities also received financial support from “Fundac¸ão para a Ciência e Tecnologia” through project UID/FIS/50010/2013. This publication reflects the views only of the author, and Fusion for Energy cannot be held responsible for any use which may be made of the information contained therein. References [1] Y. Kaschuck, et al., ITER Reference Document: SDDD 55. B1 Radial Neutron Camera, ITER D 7ABP3F, 1.2, 2012. [2] D. Marocco, B. Esposito, F. Moro, Neutron measurements in ITER using radial neutron camera, J. Instrum. 7 (2012) (JINST 7 C03033, March 2012). [3] M. Riva, et al., The new digital electronics for the JET Neutron Profile Monitor: performances and first experimental results, Fusion Eng. Des. 86 (6–8) (2011) 1191–1195 (October 2011). [4] L. Bertalot, Technical Specifications for R&D and Preparatory Design for Diagnostic Radial Neutron Camera 55. B1 & Radial Gamma Ray Spectrometer 55. B7, ITER D 9TJ92H, 2.1, 2012. [5] D. Marocco, et al., Sstem level design and performances of the ITER radial neutron camera, in: 26th IAEA Fusion Energy Conference, Kyoto, Japan, 2016. [6] A. Costley, et al., RD-55 (Diagnostics) from DOORS, ITER D 28B39L, 4.0, 2009. [7] M. Nocente, et al., Conceptual design of the Radial Gamma Ray Spectrometers system for alpha particle and runaway electron measurements at ITER, in: 26th IAEA Fusion Energy Conference Kyoto, Japan 17–22 October 2016, 2016.
[8] M. Nocente, et al., High resolution gamma ray spectroscopy at MHz counting rates with LaBr3 scintillators for fusion plasma applications, IEEE Trans. Nucl. Sci. 60 (2013) 1408–1415. [9] M. Tardocchi, et al., Spectral broadening of characteristic gamma-ray emission peaks from C-12(He-3, p gamma)N-14 reactions in fusion plasmas, Phys. Rev. Lett. 107 (2011) 205002. [10] J.Y. Journeaux, Plant Control Design Handbook 27LH2 V, 7.0, 2013. [11] P. Makijarvi, Guideline for Fast Controllers, I/O Bus Systems and Communication Methods Between Interconnected Systems ITER D 333K4C, 2.0, 2013. [12] N. Cruz, et al., Real-time software tools for the performance analysis of the ITER radial neutron camera, in: 29th Symposium on Fusion Technology, Prague, Czech Republic, September, 2016. [13] P. Makijarvi, ITER Catalog of I&C Products – Fast Controller, ITER D 345 × 28, 4, 2013. [14] R.C. Pereira, et al., Proceedings of the First EPS Conference on Plasma Diagnostics, Villa Mondragone, Rome, Italy, 2015. [15] R.C. Pereira, A. Combo, M. Correia, A.P. Rodrigues, A. Fernandes, J. Sousa, C.M.B.A. Correia, B. Gonc¸alves, C.A.F. Varandas, Ultra high-frequency data acquisition AMC module for high performance applications, Fusion Eng. Des. 88 (6–8) (2013) 1409–1413 (October 2013). [16] G. Smith, F. Sagot, Functional Analysis of the Radial Gamma, ITER D 6XR3BX, 1.1, 2012.
Please cite this article in press as: R.C. Pereira, et al., Real-Time data acquisition Prototype proposal of the ITER radial neutron camera and gamma-ray spectrometer, Fusion Eng. Des. (2017), http://dx.doi.org/10.1016/j.fusengdes.2017.03.096
ARTICLE IN PRESS
FUSION-9276; No. of Pages 5
Fusion Engineering and Design xxx (2017) xxx–xxx
Contents lists available at ScienceDirect
Fusion Engineering and Design journal homepage: www.elsevier.com/locate/fusengdes
Real-Time data acquisition Prototype proposal of the ITER radial neutron camera and gamma-ray spectrometer R.C. Pereira a,∗ , N. Cruz a , A. Fernandes a , J. Sousa a , C.M.B.A. Correia b , M. Riva c , C. Centioli c , D. Marocco c , M. Tardocchi d , M. Nocente e , B. Gonc¸alves a , B. Esposito c a
Instituto de Plasmas e Fusão Nuclear, Instituto Superior Técnico, Universidade de Lisboa, 1049-001 Lisboa, Portugal LIBPhys-UC, Dept. de Física, Universidade de Coimbra, 3004-516 Coimbra, Portugal c ENEA C.R. Frascati, Dipartimento FSN, Via E. Fermi, 45-Frascati, 00044 Rome, Italy d Istituto di Fisica del Plasma “P. Caldirola,” Consiglio Nazionale delle Ricerche, Milano, Italy e Dipartimento di Fisica “G. Occhialini,” Università degli Studi di Milano-Bicocca, Milano, Italy b
h i g h l i g h t s • • • • •
A high-frequency DAQ system based on PCI Express and FPGA. Prototype modularity and scalability allows a high no of channels architecture. Real-Time pulse processing algorithms: PSD, PHA. Up to 8 GB/s data throughput from digitizer to host. Real-Time lossless compress method to compress raw data.
a r t i c l e
i n f o
Article history: Received 3 October 2016 Received in revised form 16 March 2017 Accepted 17 March 2017 Available online xxx Keywords: RNC & RGRS diagnostics FMC High-Rate Data Acquisition PCI Express FPGA Real-Time
a b s t r a c t The Radial Neutron Camera (RNC) and the Radial Gamma-Ray Spectrometer (RGRS) are two ITER diagnostics, devoted, respectively, to the Real-Time (RT) measurement of the neutron emissivity profile and to the measurement of the confined alpha profile and runaway electrons. The two systems are closely related as they share the same equatorial port plug and part of the lines-of-sight and both require the acquisition of event-based signals from radiation detectors. The RNC Data Acquisition and Processing (DAQP) system should be capable of handling peak count rates of the order of 106 counts per second for a time duration up to 500 s. Moreover, for a continuous data throughput, the DAQP system of both diagnostics shall provide two separate DMA channels, capable to transfer simultaneously event-based data and RT processed data from the digitizers to the host. A DAQP prototype will be developed to identify and study critical issues. The present paper will: i) present the RNC DAQP prototype showing its compliancy with the RNC plant system Fast Controller; ii) show the scalability of the actual RNC DAQP from the prototype concept; iii) identify the differences between the RNC and RGRS DAQP needs; iv) describe the RGRS DAQP system and its interface to the ITER Control Data Access and Communication. © 2017 Elsevier B.V. All rights reserved.
1. Introduction The Radial Neutron Camera (RNC) and the Radial Gamma-Ray Spectrometer (RGRS) are two ITER diagnostics located in equatorial port #1. Their main roles are, respectively, the Real-Time (RT) measurement of the neutron emissivity profile (for plasma control
∗ Corresponding author. E-mail address: [email protected] (R.C. Pereira).
purposes) and the measurement of the confined alpha profile and runaway electrons. RGRS is one of the RNC enabled diagnostics, for late implementation, meaning that only interfaces must be prepared for the time being. The two systems are closely related as they share the same equatorial port plug and part of the Lines-OfSight (LOS) and both require the acquisition of event-based signals from radiation detectors. The framework Partnership agreement for RNC and RGRS diagnostics development and design (F4E-FPA-327) established the design activities for each diagnostic, to be performed under the
http://dx.doi.org/10.1016/j.fusengdes.2017.03.096 0920-3796/© 2017 Elsevier B.V. All rights reserved.
Please cite this article in press as: R.C. Pereira, et al., Real-Time data acquisition Prototype proposal of the ITER radial neutron camera and gamma-ray spectrometer, Fusion Eng. Des. (2017), http://dx.doi.org/10.1016/j.fusengdes.2017.03.096
G Model FUSION-9276; No. of Pages 5
ARTICLE IN PRESS R.C. Pereira et al. / Fusion Engineering and Design xxx (2017) xxx–xxx
2
contract. The RNC design activities are: i) System Level Design (SLD); ii) prototyping and testing; iii) engineering design. In particular, the functional specifications of RNC Data AcQuisition and Processing (DAQP) system must be detailed to the level where subsequent design and procurement activities can be undertaken without any additional input from the beneficiary. Regarding RGRS, design activities are limited to the SLD detailing the interfaces with the RNC, to be frozen during the RGRS Conceptual Design Review [1,2]. Furthermore the RNC diagnostic benefits from the knowledge obtained with the JET Neutron Profile Monitor diagnostic [3]. Section 2 will describe the two plant systems architecture in terms of plasma coverage which will define the number of acquisition channels to be considered on the diagnostic Instrumentation & Control (I&C) design. Section 3 will present: i) the ITER I&C rules and guidelines; ii) the reason for prototyping the RNC DAQP system, as well as the prototype itself; iii) the differences between RNC and RGRS features, crucial to the DAQP system design; and iv) the RGRS DAQP system interface to the ITER Control Data Access and Communication (CODAC). Finally some considerations will be made. 2. RNC & RGRS diagnostics To design a plant system I&C architecture it is a good practice to understand the nature of the diagnostic measurements as well as the type and number of sensors to read and/or actuators to control. 2.1. RNC measurements The RNC diagnostic is a collimated multichannel neutron detection system intended to characterize ITER fusion plasma neutron sources by retrieving both the neutron spectrum and the neutron emissivity granting the primary role in the ITER program for advanced control measurements and physics exploitation. The first design driver for this diagnostic is to measure the neutron flux in the conditions appropriate to infer the neutron emissivity profile. The target is to get the neutron and alpha source profile within the range 1014 –6 × 1018 n.s−1 m−3 , 10% of accuracy, 10 ms of time resolution and a/10 of spatial resolution [4]. 2.2. RNC lines-of-Sight RNC diagnostic is composed of two fan-shaped collimating structures: In-Port and Ex-port. RNC LOS are uniformly distributed in order to cover the plasma up to r/a ∼0.9. Regarding the latest development on the RNC diagnostic, it was defined, as design baseline, six architectural options covering three distinct architectural elements: i) measurement of neutron emissivity providing the best achievable performance; ii) costs and iii) technological problems/risk. These different architectures spans from a minimum possible number of LOS, 11, to a maximum of 27 LOS, and each architectural performance is going to be evaluated and ranked in order to choose the final architecture [5]. Concerning the RNC I&C system design it is worth to consider the architecture which presents the maximum number of LOS, 27, and that up to 270 channels are foreseen for the RNC I&C system. 2.3. RGRS measurements The RGRS diagnostic is a collimated multichannel gamma detection intended to provide time and spatial-resolved measurements of gamma-ray and Hard-X-Ray (HXR) emissivity supplying information on fast ions behaviour in burning plasmas, especially the alpha-particles, and runaway electrons distribution in ITER.
The RGRS primary role measurements are dedicated to physics exploitation only. No basic or advanced control are foreseen [6]. 2.4. RGRS lines-of-Sight The RGRS [7] will be installed behind the RNC detectors sharing 10 LOS. The detectors are displayed in linear geometry layout. The latest development envisages up to 2 detectors per each LOS. The central LOS will have 2 different detectors, a LaBr3 scintillator [8] and a High Purity Germanium [9]. Up to now, each detector features one channel only and no more than 20 channels are foreseen for the RGRS I&C system. 3. ITER plant system I&C architecture At ITER, all plant systems I&C must comply with the rules and guidelines presented at the ITER Plant Control Design Handbook. Therefore, an ITER plant system I&C can feature one or more plant system controller(s) interfacing to actuators and sensors via signal interface(s). The system Controllers can be divided into Fast Controllers (FC) for fast signals (data acquisition faster than 100 Hz) and Slow Controllers for measuring and activating slow Input/Output (I/O) channels (eg. detector temperatures, magnetic fields). The FCs which includes the specific diagnostic DAQP system, have some key criteria and requirements to apply, such as:i) PCI express (PCIe) and Ethernet will be used as I/O intercommunication and ii) no local storage[10,11]; Like other plant systems, both RNC and RGRS will interface with ITER central I&C systems through the ITER CODAC Plant Operation Network (PON) and High Performance Network (HPN). HPN comprises the Time Communication Network (TCN), Synchronous Databus Network (SDN), Data Archiving Network (DAN) and Audio Video Network (AVN). ITER plant systems will always interface PON and TCN, while depending on the diagnostic features, the SDN (RNC only), DAN (RNC and RGRS) and AVN (not used) may or may not be interfaced [10]. 3.1. RNC & RGRS DAQP systems ITER diagnostics will experiment variable and generally high count rates and long pulse discharges which will produce a huge amount of data. Therefore their DAQP systems must have the capability of RT pulse processing and RT data throughput, to overcome problems on the availability of the event base data, acquired from sensors, to ITER central I&C Systems. 3.1.1. RNC DAQP prototyping reasons The main purpose to have the RNC prototype is to: i) show that it is feasible to implement all the needed RT algorithms in the envisaged DAQP system with the expected performance. Although there is expertise in the implementation of RT algorithms, the highrate/high-bandwidth signals demand further R&D on new methods and testing; ii) measure the peak performance of the system (maximum count rate supported) to dimension the final system with optimal cost/performance relation; iii) Evaluate the need of RT data compression aiming at reducing the event based data throughput, and iv) evaluate if the spatial inversion algorithms can be employed in RT (R&D) [12]. The prototype does not define a complete system with a platform standard proposed by ITER (eg. xTCA, PxIe [13]), focusing only on the two identified drawbacks: i) data loss due to the DAQ architectural problem; ii) no advance control due to processing power problems. The prototype will answer the following architectural questions:
Please cite this article in press as: R.C. Pereira, et al., Real-Time data acquisition Prototype proposal of the ITER radial neutron camera and gamma-ray spectrometer, Fusion Eng. Des. (2017), http://dx.doi.org/10.1016/j.fusengdes.2017.03.096
G Model FUSION-9276; No. of Pages 5
ARTICLE IN PRESS R.C. Pereira et al. / Fusion Engineering and Design xxx (2017) xxx–xxx
Fig. 1. RT emissivity reconstruction flow chart based on the Tikhonov regularization, where bk is the line integrated signals from LOS k (neutron counts), and lkj the length of intersection of LOS k with magnetic surfaces j and j–1.
1. How many channels can be accomodated by the host, to avoid data loss, in an event base, from digitizer to host? 2. How many input channels can the Field Programmable Gate Array (FPGA) accomodate, to process, in RT, all detector signals? 3. Evaluation of the Gigabit Ethernet link (10 GbE or 40 GbE) in order to avoid data loss, in event base operation mode, from the host to the ITER databases. 4. Can the state-of-the-art FPGAs implement all RT algorithms at processing rates up to 200 MHz or higher? 5. Can the host perform the emissivity reconstruction in less than 10 ms? The third question will not be completely assessed by this prototype. It will be determined the throughput needed by the host, but the real performance of data througput from the host to databases will not be studied during the prototype phase. Finally, the emissivity reconstruction is divided into three distinct tasks, presented at Fig. 1 [12]. Only the processing code related to task 2 and 3 will be evaluated, by running it at the host with simulated data (neutron counts and equilibrium data). Task1, will only be assessed for 4 channels. Preliminary tests showed that for the maximum predicted RNC number of LOS, 27, and up to 20 magnetic surfaces (10 for RNC) the master host can perform the neutron profile within 10 ms. 3.1.2. RNC DAQP prototype To specify a DAQP system dedicated to a diagnostic with spectroscopic capabilities two types of features must be addressed: i) architectural aspects such as the number of channels and the maximum expected event rate; and ii) the characterization of the digitizer itself: Analog-to-Digital Converter (ADC) specifications and processing power capabilities. Therefore the knowledge of the detectors minimum pulse width (Full Width Half Maximum
3
(FWHM) – RNC detector signals FWHM spans from 10 to 200 ns), as well as the detectors capability of providing Pulse Height Spectra (PHS) and neutron/gamma PSD will define the ADC frequency (≥400 MHz) and its acceptable resolution (≥12 bit) [14]. The event rate as well as the detectors maximum pulse width define the data throughput. The aim of this prototype is to sustain a 2 MHz peak event. If the event is covered by 128 samples then one channel will produce ∼0.5 GB/s of raw data (event base). Four channels data throughput will be of ∼2 GB/s. The foreseen DAQP system physical throughput limitations are the PCIe interface between the digitizers and the link from host to ITER databases, DAN network (out of the scope of the DAQP prototype tests); To comply with the ITER rules and guidelines, previously stated, the best compromise between cost and performance of PCIe based solution was an Evaluation (EV) board from Xilinx, EK-K7-KC705G (KC705), with a high-performance Kintex-7 FPGA and a × 8 PCIe 2.0 interface. The board was also chosen due to the full access to the FPGA in order to test all the needed RT algorithms. The KC705 allows data throughput of 4 GB/s and features an I/O expansion High Pin Connector (HPC) to accommodate FPGA Mezzanine Cards (FMC). Depending on the choice of the FMC, the KC705 bandwidth can be divided into several channels. Presently, in the market, the analog-to-digital FMCs compatible with KC705 and complying with the diagnostic needs (sampling rate and resolution), feature only ac-coupling with an input bandwidth leading to distortions caused by the ac inherent high-pass filter. The best solution was a twochannel 12-bit 1.6 GHz FMC custom module featuring both ac/dccoupling solutions [15]. Therefore a KC705 will divide its 4 GB/s bandwidth with two channels. The aim of this prototype is to have two architectures: i) one host PC with two KC705 modules and four channels, Fig. 2a). This configuration evaluates the exact number of channels allowed by an host; ii) Two hosts with two channels each. In this case, one host will send processed data, more specifically the detectors brightness information, to the other host (master), through a 1 GbE link. This will test the performance of the master host to calculate the emissivity profile each 10 ms, Fig. 2b). 3.1.3. RNC real-time algorithms The RNC DAQP system implements RT fast data processing algorithms (≥200 MHz), both in the FPGA and host side environment. The FPGA processes the digitized data to give the necessary physical measurement, either n/␥ counts, energy values or spectra and Pile-Up (PU) occurrences. The RT pulse processing algorithms to implement will minimize data storage and transfer issues from digitizers to host, and are as follow: i) pulse detection featuring smart triggering system with PU management; ii) Pulse integration; iii) PSD; iv) PHS; and v) gain instability corrections; Concerning the host environment three distinct RT processing algorithms are envisaged. Two for data reduction (several orders of magnitude) of the event based data retrieved from the digitizers: i) pulse processing algorithms, the same defined for the FPGA environment; and ii) lossless data compression algorithms. The third RT algorithm calculates the neutron emissivity based on the retrieved FPGA processed data [12]. For continuous data throughput, the DAQP system firmware architecture shall provide two separate Direct Memory Access (DMA) channels with enough bandwidth, capable to transfer event based data packets and RT processed data packets from the digitizer to the host. For event based data packets, pulse windows, are stored in FPGA internal buffers. Pulse windows size depends on detection of PU. If no PU is assigned then the pulse window is of the size of the event, Pwidth (user defined, eg. 128 samples). Otherwise it can be incremented n times the Pwidth (number of PU assignments). The prototype will test a set of RT FPGA algorithms able to perform PU discrimination. Every time a PU occurs a pulse window is
Please cite this article in press as: R.C. Pereira, et al., Real-Time data acquisition Prototype proposal of the ITER radial neutron camera and gamma-ray spectrometer, Fusion Eng. Des. (2017), http://dx.doi.org/10.1016/j.fusengdes.2017.03.096
G Model
ARTICLE IN PRESS
FUSION-9276; No. of Pages 5
R.C. Pereira et al. / Fusion Engineering and Design xxx (2017) xxx–xxx
4
Fig. 2. Prototype architecture diagram block. a) one host PC with two digitizer modules; b) two PCs with one digitizer module each with an 1 GbE connection between PCs.
Table 1 Differences between RNC and RGRS for each of the DAQ system key features. Key features
LOS N◦ of channels Event rate FWHM Key features
Diagnostic Physical Features RGRS
RNC
6 20 (max) >1Mevents/s 150 ns
Up to 27 270 (max) 2 Mevent/s 10–200 ns
RT Processing Capabilities RGRS
Pile-Up discrimination/resolving n/␥ Pulse Discrimination PHS Data Compression Emissivity Reconstruction
FPGA √ – √ – –
RNC Host √ – √ √ –
FPGA √ √ √ – –
Host √ √ √ √ √
tagged as PU window to be processed, also in RT, in the host side with smarter algorithms, which due to its complexity cannot be implemented in FPGA. The host side will then attempt to resolve PU. 3.1.4. Differences between RNC and RGRS DAQP needs Assuming that the RGRS FC is to be based on the RNC DAQP prototype, differences between both systems are identified and presented at Table 1. The knowledge of the two diagnostics key features differences assures that the RNC DAQP system suits RGRS. RGRS presents much lower number of channels and less stringent needs for the digitizer characterization. From the point of view of processing capabilities, this diagnostic is less demanding as the PSD algorithm is not needed and RGRS will not provide RT feedback control. 3.1.5. RGRS DAQP system According to the functional analysis and to the processing needs described in the main documents, and to the functional requirements of the I&C system of the RGRS [16], more specifically on DAQP system, part of the RGRS FC shall: i) perform RT digital signal processing to transmit event based and processed packets to
CODAC for long term storage; ii) measure gamma flux and emissivity profile: the DAQP system must provide spectra for count rates higher than 1 Mevent/s and provide position and shape within a/10 spatial resolution and 10 ms/100 ms of temporal resolution for HXR/gamma-ray source; iii) calculate the fast ions and electron distributions; iv) interface with CODAC, physical location of network connections within the plant system cubicle (PON;TCN;DAN). Concerning the interface with CODAC, the PON interface will be responsible for: i) data acquisition (measurements); ii) alarm monitoring; iii) control and configuration: commands and configuration data from CODAC to the RGRS diagnostic; iv) collaborative data, i.e, non-RT data needed for offline analysis. The TCN interface will be responsible for timing information (absolute time and global trigger). Finally the DAN will be responsible for all archive data. The SDN Network provides transport for RT feedback control. No feedback control is expected for this diagnostic, therefore this connection is not foreseen. Fig. 3 presents a block diagram of RGRS Plant control system including the interfaces with CODAC networks.
4. Considerations Continuous operation is the ultimate goal for next generation of nuclear fusion devices, thus in order to be able to calculate the neutron and alpha source profile every 10 ms, a new developed DAQP system allows up to 8 GB throughput to host with RT pulse processing capabilities. Besides the RT pulse processing, other RT lossless compress methods are envisaged for the host side to compress the event base data, estimated to produce ∼0.5 GB/s per channel during 2 MHz of peak events. The modularity and scalability of the described prototype allow a short implementing time of the FC for the overall diagnostic channels. Although a platform standard is not yet settled the proposed architecture is easily available in the standards proposed by ITER. Work is ongoing to check whether more than 4 channels/host can be supported in order to decrease substantially the space allocation requirement in the cubicles and the cost of DAQP system. The limit of the scalability will be the space allocated for RNC at the ITER diagnostic cubicles which was based on the worst case of 8 channels per host.
Please cite this article in press as: R.C. Pereira, et al., Real-Time data acquisition Prototype proposal of the ITER radial neutron camera and gamma-ray spectrometer, Fusion Eng. Des. (2017), http://dx.doi.org/10.1016/j.fusengdes.2017.03.096
G Model FUSION-9276; No. of Pages 5
ARTICLE IN PRESS R.C. Pereira et al. / Fusion Engineering and Design xxx (2017) xxx–xxx
5
Fig. 3. RGRS Plant control system. I&C interfaces with CODAC networks.
Acknowledgments The work leading to this publication has been funded partially by Fusion for Energy under the Specific Grants Agreement F4E-FPA-327 SG03 & SG04. IST activities also received financial support from “Fundac¸ão para a Ciência e Tecnologia” through project UID/FIS/50010/2013. This publication reflects the views only of the author, and Fusion for Energy cannot be held responsible for any use which may be made of the information contained therein. References [1] Y. Kaschuck, et al., ITER Reference Document: SDDD 55. B1 Radial Neutron Camera, ITER D 7ABP3F, 1.2, 2012. [2] D. Marocco, B. Esposito, F. Moro, Neutron measurements in ITER using radial neutron camera, J. Instrum. 7 (2012) (JINST 7 C03033, March 2012). [3] M. Riva, et al., The new digital electronics for the JET Neutron Profile Monitor: performances and first experimental results, Fusion Eng. Des. 86 (6–8) (2011) 1191–1195 (October 2011). [4] L. Bertalot, Technical Specifications for R&D and Preparatory Design for Diagnostic Radial Neutron Camera 55. B1 & Radial Gamma Ray Spectrometer 55. B7, ITER D 9TJ92H, 2.1, 2012. [5] D. Marocco, et al., Sstem level design and performances of the ITER radial neutron camera, in: 26th IAEA Fusion Energy Conference, Kyoto, Japan, 2016. [6] A. Costley, et al., RD-55 (Diagnostics) from DOORS, ITER D 28B39L, 4.0, 2009. [7] M. Nocente, et al., Conceptual design of the Radial Gamma Ray Spectrometers system for alpha particle and runaway electron measurements at ITER, in: 26th IAEA Fusion Energy Conference Kyoto, Japan 17–22 October 2016, 2016.
[8] M. Nocente, et al., High resolution gamma ray spectroscopy at MHz counting rates with LaBr3 scintillators for fusion plasma applications, IEEE Trans. Nucl. Sci. 60 (2013) 1408–1415. [9] M. Tardocchi, et al., Spectral broadening of characteristic gamma-ray emission peaks from C-12(He-3, p gamma)N-14 reactions in fusion plasmas, Phys. Rev. Lett. 107 (2011) 205002. [10] J.Y. Journeaux, Plant Control Design Handbook 27LH2 V, 7.0, 2013. [11] P. Makijarvi, Guideline for Fast Controllers, I/O Bus Systems and Communication Methods Between Interconnected Systems ITER D 333K4C, 2.0, 2013. [12] N. Cruz, et al., Real-time software tools for the performance analysis of the ITER radial neutron camera, in: 29th Symposium on Fusion Technology, Prague, Czech Republic, September, 2016. [13] P. Makijarvi, ITER Catalog of I&C Products – Fast Controller, ITER D 345 × 28, 4, 2013. [14] R.C. Pereira, et al., Proceedings of the First EPS Conference on Plasma Diagnostics, Villa Mondragone, Rome, Italy, 2015. [15] R.C. Pereira, A. Combo, M. Correia, A.P. Rodrigues, A. Fernandes, J. Sousa, C.M.B.A. Correia, B. Gonc¸alves, C.A.F. Varandas, Ultra high-frequency data acquisition AMC module for high performance applications, Fusion Eng. Des. 88 (6–8) (2013) 1409–1413 (October 2013). [16] G. Smith, F. Sagot, Functional Analysis of the Radial Gamma, ITER D 6XR3BX, 1.1, 2012.
Please cite this article in press as: R.C. Pereira, et al., Real-Time data acquisition Prototype proposal of the ITER radial neutron camera and gamma-ray spectrometer, Fusion Eng. Des. (2017), http://dx.doi.org/10.1016/j.fusengdes.2017.03.096