- Email: [email protected]
Suitability study of a histogramming software architecture in the data acquisition system of position sensitive X-ray detectors
ARTICLE IN PRESS
Nuclear Instruments and Methods in Physics Research A 513 (2003) 211–214
Suitability study of a histogramming software architecture in the data acquisition system of position sensitive X-ray detectors I. Magrans*, J.A. Perlas, D. Beltra! n, I. Ramos-Lerate, J.C. Mart!ınez, J. Bordas ! Edifici Ci"encies, Campus UAB, 08193-Bellaterra, Barcelona, Spain Laboratori de Llum Sincrotro,
Abstract We describe here an approach, based on software algorithms and techniques for the histogramming of raw data, to the position encoding in X-ray detectors. This runs on a standard platform and uses a commercial data acquisition board. We also describe the hardware and software set-up, and the results of the tests carried out in order to show its suitability. The results show that: it is possible to achieve a throughput, for a sustained mean event rate, in excess of 3 MHz; the Linux operating system, with an appropriate driver development, is a suitable option, and; it is possible to reach acceptable event throughput, whilst increasing flexibility and reducing the overall cost of the data acquisition system. r 2003 Elsevier B.V. All rights reserved. PACS: 07.05.Hd; 07.07.D Keywords: DAQ; Linux; MWPC; Software histogramming; TDC
1. Introduction The performance of CPUs, busses and system memories has evolved a great deal in recent years, so much so that functions that until recently had to be implemented using specifically designed hardware, can now be software implemented in a general-purpose platform. The traditional way of implementing a position sensitive X-ray detectors data acquisition system is by using a VME-based architecture [1]. Both the data collection (from the time-to-digital converter *Corresponding author. Tel.: +34-93-581-2698; fax: +3493-581-3213. E-mail address: [email protected] (I. Magrans).
(TDC) sub-system) and the histogramming function are done with specialized VME modules built for this purpose. In this paper, we present a feasibility study for an alternative approach to the architecture of the data acquisition system [2]. Thus, we attempt to overcome the classic architecture by adopting a new industry standard bus (having the same or higher performance) and replacing the histogrammer hardware module by a software solution in a suitable operating system. This new approach has to allow easy additions or change to functionality by just replacing a software module. In addition, the overall cost is reduced. This article presents new results that prove the suitability of this new architecture.
0168-9002/$ - see front matter r 2003 Elsevier B.V. All rights reserved. doi:10.1016/j.nima.2003.08.034
ARTICLE IN PRESS 212
I. Magrans et al. / Nuclear Instruments and Methods in Physics Research A 513 (2003) 211–214
2. MWPC data acquisition system When a photon enters the detector, the charge induced by its ionization is collected and travels to the ends of the delay line. Two high-speed, lownoise pre-amplifiers (AMP) [3], at each end of the delay line, together with their corresponding constant fraction discriminators (CFD), are used to measure the difference between the time of arrival of the two end pulses (using a TDC). This time difference is proportional to the position where the photon impinged and thus one obtains the x-coordinate (the same process is applied to the orthogonal cathode which represents the ycoordinate). From the position of the photons, the diffraction pattern (image) is obtained. Since the detector is primarily meant for time resolving experiments, a user configurable time frame generator (TFG) is also necessary. The data acquisition system (DAQ) [4] has to acquire the images generated by the 2D detector and store them onto disk for later analysis. The DAQ system specifications are shown in Table 1. A core component of the readout system is the histogramming sub-system (which we will call HCS, Histogramming Control System). This acquires the data generated by the TDC and, according to the TFG pattern, stores either 1D or 2D histograms in memory. Usually, both the TDC sub-system and the HCS sub-system (including memory modules, TFG and
Histogramming module) are housed in a VMEbased architecture.
3. Software histogramming A convenient architecture for our alternative software histogramming approach is shown in Fig. 1. The data words coming from the TDC and TFG are acquired in a high-speed commercial (e.g. Ref. [5]) digital Input/Output module (DIO in Fig. 2). From there, the data is stored in two buffers that are alternatively switched using the Reader task; this scheme substantially increases the data throughput.
Fig. 1. Software architecture adopted for the HCS module.
Table 1 DAQ general specification summary Characteristic
Required specification
Total count rate (mean) Data throughput Photon position resolver
B1 MHz >4 Mb/s Space–time–space-based method One-dimensional or twodimensional 1024 1024 pixels
Histogramming modes Maximum 2D image resolution Online data monitoring Time resolving engine Working environment
Yes Programmable time frame generator Standard industrial specs
Fig. 2. TDC and HCS processing event rates as a function of the data acquisition input event rate.
ARTICLE IN PRESS I. Magrans et al. / Nuclear Instruments and Methods in Physics Research A 513 (2003) 211–214
The Histogrammer task has to build the output image using the pre-programmed framing configuration. An optional Monitoring task can be used to interrogate the data online to ensure that the experiment is proceeding correctly. The CPU, DIO, TFG and memory modules are all housed in the 3U CompactPCI crate whereas the TDC can also be included (preferred solution), or be implemented as a NIM module or standalone instrument. Regarding the operating system, a real-time OS would be the ideal choice (OS-9 or VxWorks) but an attractive solution like Linux seems also a good candidate (RT-Linux is also being considered). A serious drawback of the Linux option is, however, the lack of software drivers currently available although this can be overcome by additional inhouse development.
4. Experimental set-up In order to test the suitability of the ideas presented above, i.e. to verify whether a CPU is able to carry out the histogramming job at the required mean event rate (>1 MHz), a test bench was set up. Firstly, one should note that we have chosen PCI rather than VME for the bus architecture at the moment, which avoids the high cost of VME systems. However, in a future design we foresee the use of CompactPCI due to its superior noise immunity and availability. The main components of this test bench are: 1. Desktop PC (Intel VC820 chipset) with a Pentium III processor at 800 MHz with RAMbus memory (128 Mb) technology, running Windows 2000 or Linux Red Hat 7.1. 2. PCI-DIO-32HS digital I/O data acquisition PCI card from National Instruments. 3. Five CFD channels module, TDC module, Pulse counter module, and fan out module. All of them in NIM standard. 4. NIM Pulse generator with variable frequency. The PC was chosen because of its fast memory access, which derives from a good chipset and the RAMbus technology. The PCI-DIO/Win2000
213
combination provides a convenient and fast prototyping environment with the help of offthe-shelf board drivers and a visual C/C++ IDE. On the other hand, National Instruments does not offer the Linux drivers, and so they have been developed under the framework of the so-called Comedi project [6] running under Red Hat 7.1. Note that due to the poor public documentation available, the DMA access capability has not been implemented. To easily control the data acquisition input event rate, virtual events have been created connecting the pulse generator to the fan out module. From there, five copied signals are connected to the adequate CFD channel. Following this scheme we can generate virtual input events at a controlled rate. The pulse counter input, connected to the output of the TDC monitor, lets us know the TDC processing rate for every data acquisition input event rate. We have also connected the TDC data word output to the DIO input (see Fig. 1), in order to know the HCS processing event rate for every data acquisition input event rate.
5. Results The first test was carried out using the Windows drivers to take advantage of the DMA access. Varying the frequency of the pulse generator from 3 to 7 MHz in steps of 100 kHz, for every data acquisition input event rate we obtained two results: the first, from connecting the TDC monitor to the pulse counter for 32 s to have an estimate of the TDC processing rate; the second, from connecting the TDC data word output to the HCS, and acquiring and histogramming 32f words (f=frequency of the pulse generator in Hz) coming from the TDC. The results of this test are shown in Fig. 2. In both cases we obtain almost the same relationship between the processing event rate and the input event rate. From this, we can deduce that the HCS, implemented following the ideas presented above, is a suitable module. We can see also that the TDC module is now the bottleneck of the data acquisition system.
ARTICLE IN PRESS I. Magrans et al. / Nuclear Instruments and Methods in Physics Research A 513 (2003) 211–214
214
Using the results shown in Fig. 2, and under the hypothesis that the arrival time of the events, coming from a real detector working in real conditions, follows a Poisson distribution, we can also obtain an approximation of the data acquisition maximum mean processing event rate: Rm ¼
N X k¼0
KPk
fmk efm k!
where fm is the data acquisition input mean event rate, Rm is the data acquisition mean processing event rate, K is the data acquisition input event rate, and Pk is the probability that a data acquisition input event, with rate k; can be histogrammed. The parameter Pk has been approximated from the relationship between the HCS processing event rate and the data acquisition input event rate; that can be easily found from the results of Fig. 2. The results obtained after varying fm from 100 kHz to 7 MHz show that the maximum mean processing event rate is 3.3 Mevents/s. The second test was carried out using both the Windows driver with the DMA access disabled, and the home-made Linux driver. In both cases varying the frequency of the pulse generator from 10 to 400 kHz in steps of 10 kHz, for every input event rate we obtained both the results for the Windows and the Linux case. The results are shown in Fig. 3.
In both cases these results have been obtained by connecting the TDC data word output to the HCS and acquiring/histogramming 32f words (f=frequency of the pulse generator in Hz) coming from the TDC. Note that under Linux the saturation of the processing event rate is above 340 Kevents/s, whilst under Windows is around 75 Kevents/s, i.e. the fomer is 4.5 times faster than the latter.
6. Conclusion The results of the tests presented here show that our architecture for the histogramming module is a suitable solution for the data acquisition system of an X-ray MWPC position sensitive detector. This consists on software tasks implemented in the Linux operating system running on an industrial hardware platform. Although the final system based on CompactPCI has not been yet tested, the test set-up described herein validates the presented ideas for mean event rates higher than 3 MHz using the currently available prototype TDC. Since the HCS is not the limiting element in the DAQ chain, the throughput might be higher if/when faster versions of the TDC become available. This approach complies with the specifications required from the data acquisition system and, in addition, substantially reduces overall costs and maintenance requirements.
References
Fig. 3. Processing event rate of the data acquisition system (HCS processing event rate) as a function of the input event rate, where the HCS is implemented under Windows and under Linux.
[1] C. Herv!e, T. Le Ca.er, Rev. Sci. Instrum. 70 (1) (1998) 226. [2] J.A. Perlas, I. Magrans, Feasibility study of new ideas for the data acquisition system for MWPC X-ray detectrors, Proceedings of the 12th IEEE International Congress on Real Time for Nuclear and Plasma Sciences, Valencia, 2001, pp. 134–138. [3] J.A. Perlas, D. Beltr!an, Rev. Sci. Instrum. 73 (8) (2002) 3075. [4] J.A. Perlas, CRG DAQ/control system requirements, LLS Report LLS/03/00, 2000. [5] PCI-DIO-32HS, National Instruments, www.ni.com, April 2002. [6] http://stm.lbl.gov/comedi
Nuclear Instruments and Methods in Physics Research A 513 (2003) 211–214
Suitability study of a histogramming software architecture in the data acquisition system of position sensitive X-ray detectors I. Magrans*, J.A. Perlas, D. Beltra! n, I. Ramos-Lerate, J.C. Mart!ınez, J. Bordas ! Edifici Ci"encies, Campus UAB, 08193-Bellaterra, Barcelona, Spain Laboratori de Llum Sincrotro,
Abstract We describe here an approach, based on software algorithms and techniques for the histogramming of raw data, to the position encoding in X-ray detectors. This runs on a standard platform and uses a commercial data acquisition board. We also describe the hardware and software set-up, and the results of the tests carried out in order to show its suitability. The results show that: it is possible to achieve a throughput, for a sustained mean event rate, in excess of 3 MHz; the Linux operating system, with an appropriate driver development, is a suitable option, and; it is possible to reach acceptable event throughput, whilst increasing flexibility and reducing the overall cost of the data acquisition system. r 2003 Elsevier B.V. All rights reserved. PACS: 07.05.Hd; 07.07.D Keywords: DAQ; Linux; MWPC; Software histogramming; TDC
1. Introduction The performance of CPUs, busses and system memories has evolved a great deal in recent years, so much so that functions that until recently had to be implemented using specifically designed hardware, can now be software implemented in a general-purpose platform. The traditional way of implementing a position sensitive X-ray detectors data acquisition system is by using a VME-based architecture [1]. Both the data collection (from the time-to-digital converter *Corresponding author. Tel.: +34-93-581-2698; fax: +3493-581-3213. E-mail address: [email protected] (I. Magrans).
(TDC) sub-system) and the histogramming function are done with specialized VME modules built for this purpose. In this paper, we present a feasibility study for an alternative approach to the architecture of the data acquisition system [2]. Thus, we attempt to overcome the classic architecture by adopting a new industry standard bus (having the same or higher performance) and replacing the histogrammer hardware module by a software solution in a suitable operating system. This new approach has to allow easy additions or change to functionality by just replacing a software module. In addition, the overall cost is reduced. This article presents new results that prove the suitability of this new architecture.
0168-9002/$ - see front matter r 2003 Elsevier B.V. All rights reserved. doi:10.1016/j.nima.2003.08.034
ARTICLE IN PRESS 212
I. Magrans et al. / Nuclear Instruments and Methods in Physics Research A 513 (2003) 211–214
2. MWPC data acquisition system When a photon enters the detector, the charge induced by its ionization is collected and travels to the ends of the delay line. Two high-speed, lownoise pre-amplifiers (AMP) [3], at each end of the delay line, together with their corresponding constant fraction discriminators (CFD), are used to measure the difference between the time of arrival of the two end pulses (using a TDC). This time difference is proportional to the position where the photon impinged and thus one obtains the x-coordinate (the same process is applied to the orthogonal cathode which represents the ycoordinate). From the position of the photons, the diffraction pattern (image) is obtained. Since the detector is primarily meant for time resolving experiments, a user configurable time frame generator (TFG) is also necessary. The data acquisition system (DAQ) [4] has to acquire the images generated by the 2D detector and store them onto disk for later analysis. The DAQ system specifications are shown in Table 1. A core component of the readout system is the histogramming sub-system (which we will call HCS, Histogramming Control System). This acquires the data generated by the TDC and, according to the TFG pattern, stores either 1D or 2D histograms in memory. Usually, both the TDC sub-system and the HCS sub-system (including memory modules, TFG and
Histogramming module) are housed in a VMEbased architecture.
3. Software histogramming A convenient architecture for our alternative software histogramming approach is shown in Fig. 1. The data words coming from the TDC and TFG are acquired in a high-speed commercial (e.g. Ref. [5]) digital Input/Output module (DIO in Fig. 2). From there, the data is stored in two buffers that are alternatively switched using the Reader task; this scheme substantially increases the data throughput.
Fig. 1. Software architecture adopted for the HCS module.
Table 1 DAQ general specification summary Characteristic
Required specification
Total count rate (mean) Data throughput Photon position resolver
B1 MHz >4 Mb/s Space–time–space-based method One-dimensional or twodimensional 1024 1024 pixels
Histogramming modes Maximum 2D image resolution Online data monitoring Time resolving engine Working environment
Yes Programmable time frame generator Standard industrial specs
Fig. 2. TDC and HCS processing event rates as a function of the data acquisition input event rate.
ARTICLE IN PRESS I. Magrans et al. / Nuclear Instruments and Methods in Physics Research A 513 (2003) 211–214
The Histogrammer task has to build the output image using the pre-programmed framing configuration. An optional Monitoring task can be used to interrogate the data online to ensure that the experiment is proceeding correctly. The CPU, DIO, TFG and memory modules are all housed in the 3U CompactPCI crate whereas the TDC can also be included (preferred solution), or be implemented as a NIM module or standalone instrument. Regarding the operating system, a real-time OS would be the ideal choice (OS-9 or VxWorks) but an attractive solution like Linux seems also a good candidate (RT-Linux is also being considered). A serious drawback of the Linux option is, however, the lack of software drivers currently available although this can be overcome by additional inhouse development.
4. Experimental set-up In order to test the suitability of the ideas presented above, i.e. to verify whether a CPU is able to carry out the histogramming job at the required mean event rate (>1 MHz), a test bench was set up. Firstly, one should note that we have chosen PCI rather than VME for the bus architecture at the moment, which avoids the high cost of VME systems. However, in a future design we foresee the use of CompactPCI due to its superior noise immunity and availability. The main components of this test bench are: 1. Desktop PC (Intel VC820 chipset) with a Pentium III processor at 800 MHz with RAMbus memory (128 Mb) technology, running Windows 2000 or Linux Red Hat 7.1. 2. PCI-DIO-32HS digital I/O data acquisition PCI card from National Instruments. 3. Five CFD channels module, TDC module, Pulse counter module, and fan out module. All of them in NIM standard. 4. NIM Pulse generator with variable frequency. The PC was chosen because of its fast memory access, which derives from a good chipset and the RAMbus technology. The PCI-DIO/Win2000
213
combination provides a convenient and fast prototyping environment with the help of offthe-shelf board drivers and a visual C/C++ IDE. On the other hand, National Instruments does not offer the Linux drivers, and so they have been developed under the framework of the so-called Comedi project [6] running under Red Hat 7.1. Note that due to the poor public documentation available, the DMA access capability has not been implemented. To easily control the data acquisition input event rate, virtual events have been created connecting the pulse generator to the fan out module. From there, five copied signals are connected to the adequate CFD channel. Following this scheme we can generate virtual input events at a controlled rate. The pulse counter input, connected to the output of the TDC monitor, lets us know the TDC processing rate for every data acquisition input event rate. We have also connected the TDC data word output to the DIO input (see Fig. 1), in order to know the HCS processing event rate for every data acquisition input event rate.
5. Results The first test was carried out using the Windows drivers to take advantage of the DMA access. Varying the frequency of the pulse generator from 3 to 7 MHz in steps of 100 kHz, for every data acquisition input event rate we obtained two results: the first, from connecting the TDC monitor to the pulse counter for 32 s to have an estimate of the TDC processing rate; the second, from connecting the TDC data word output to the HCS, and acquiring and histogramming 32f words (f=frequency of the pulse generator in Hz) coming from the TDC. The results of this test are shown in Fig. 2. In both cases we obtain almost the same relationship between the processing event rate and the input event rate. From this, we can deduce that the HCS, implemented following the ideas presented above, is a suitable module. We can see also that the TDC module is now the bottleneck of the data acquisition system.
ARTICLE IN PRESS I. Magrans et al. / Nuclear Instruments and Methods in Physics Research A 513 (2003) 211–214
214
Using the results shown in Fig. 2, and under the hypothesis that the arrival time of the events, coming from a real detector working in real conditions, follows a Poisson distribution, we can also obtain an approximation of the data acquisition maximum mean processing event rate: Rm ¼
N X k¼0
KPk
fmk efm k!
where fm is the data acquisition input mean event rate, Rm is the data acquisition mean processing event rate, K is the data acquisition input event rate, and Pk is the probability that a data acquisition input event, with rate k; can be histogrammed. The parameter Pk has been approximated from the relationship between the HCS processing event rate and the data acquisition input event rate; that can be easily found from the results of Fig. 2. The results obtained after varying fm from 100 kHz to 7 MHz show that the maximum mean processing event rate is 3.3 Mevents/s. The second test was carried out using both the Windows driver with the DMA access disabled, and the home-made Linux driver. In both cases varying the frequency of the pulse generator from 10 to 400 kHz in steps of 10 kHz, for every input event rate we obtained both the results for the Windows and the Linux case. The results are shown in Fig. 3.
In both cases these results have been obtained by connecting the TDC data word output to the HCS and acquiring/histogramming 32f words (f=frequency of the pulse generator in Hz) coming from the TDC. Note that under Linux the saturation of the processing event rate is above 340 Kevents/s, whilst under Windows is around 75 Kevents/s, i.e. the fomer is 4.5 times faster than the latter.
6. Conclusion The results of the tests presented here show that our architecture for the histogramming module is a suitable solution for the data acquisition system of an X-ray MWPC position sensitive detector. This consists on software tasks implemented in the Linux operating system running on an industrial hardware platform. Although the final system based on CompactPCI has not been yet tested, the test set-up described herein validates the presented ideas for mean event rates higher than 3 MHz using the currently available prototype TDC. Since the HCS is not the limiting element in the DAQ chain, the throughput might be higher if/when faster versions of the TDC become available. This approach complies with the specifications required from the data acquisition system and, in addition, substantially reduces overall costs and maintenance requirements.
References
Fig. 3. Processing event rate of the data acquisition system (HCS processing event rate) as a function of the input event rate, where the HCS is implemented under Windows and under Linux.
[1] C. Herv!e, T. Le Ca.er, Rev. Sci. Instrum. 70 (1) (1998) 226. [2] J.A. Perlas, I. Magrans, Feasibility study of new ideas for the data acquisition system for MWPC X-ray detectrors, Proceedings of the 12th IEEE International Congress on Real Time for Nuclear and Plasma Sciences, Valencia, 2001, pp. 134–138. [3] J.A. Perlas, D. Beltr!an, Rev. Sci. Instrum. 73 (8) (2002) 3075. [4] J.A. Perlas, CRG DAQ/control system requirements, LLS Report LLS/03/00, 2000. [5] PCI-DIO-32HS, National Instruments, www.ni.com, April 2002. [6] http://stm.lbl.gov/comedi