- Email: [email protected]
The data acquisition system for the GRAAL experiment at the ESRF in Grenoble
Nuclear
Instrument
and Methods
in Physics
Research
A 388 (1997) 226-234
NUCLEAR fNSTRUMENtS 8 METNOOS IN PHYStCS RESEARCH SectIon A
ELSEVIER
The data acquisition system for the GRAAL experiment at the ESRF in Grenoble D. BarancourV’, G. Barbiera, J. Bouvief, B. MeillorP*, M. Tourniera, R. Douetb, H. Harrochb aInstitut des Sciences Nucliaires 53. Auenue des mar@rs. 38 026 Grenoble Cede.r, France b Institut de Phvsique Nuclkaire. Received
91 405 0rsa.v Cedex. France
23 July 1996; revised form received 23 December
1996
Abstract A specific data acquisition system System Acquisition Graal Asic, (SAGA) has been implemented for a multidetector experiment. Its main feature is an hardware event builder associated with compact and programmable electronics based on two kinds of ASIC circuits linked by a bus, located on the detectors. The transfer rate on this bus is about 23 Mbyte/s.
1. Introduction
2. Description of GRAAL experiment
As we had to develop a new data acquisition system for the Grenoble Anneau Accelerateur Laser (GRAAL) [l] experiment with multidetectors, we had to take into account several requirements for building up a specific system: _ First, the large number of channels (about 4000) to be read and the high acquisition rate obliged us to use a fast read out system in order to reduce the busy time to the minimum. - Secondly, since the distance between the detectors and the experiment room was about 50 m, there was the problem of transporting small signals over great distances (deterioration of signals, high cost of cabling, connection problems). ~ Thirdly, some of the detectors were already interfaced to CAMAC modules and their data read out was performed with a FERA bus. It was decided to keep this feature for the detectors concerned. _ Finally, it was important to read only the meaningful channels (about loo), in order to minimize the amount of data collected. Section 2 presents a short description of the GRAAL experiment followed by an overview of the proposed solution, and more details about the different electronic implementations. Last, the performance of the system is given.
This nuclear physics experiment is located in Grenoble, at the European Synchrotron Radiation Facility (ESRF). The project objective is to produce a beam of tagged photons and to first study the beam interactions with a liquid-hydrogen target. The beam production is quite original: the photons are produced by the Compton backscattering of a laser beam on the 6 GeV electrons produced by the synchrotron. The photons gain 1.5 GeV of energy and keep the polarization of the laser beam. The first topics to be studied are the strangeness photoproduction and q disintegration. For this purpose, a 472 detection system has been designed around the target (see Fig. 1). Its goal is: 1. to restore charged particles trajectories and identification by ~ 2 planes of multiwire proportional chambers MWPC, ~ 2 cylindrical multiwire proportional chambers MWPC, ~ 2 scintillating plastic detectors, a double wall and a barrel; 2. to measure the emission angle and the energy of neutral particles (photons) by ~ a BGO ball calorimeter detector [Z], ~ a wall made of plastic-lead sandwiches. A tagging detector (silicium n-strips) has been placed in the synchrotron tunnel, to detect the position of Compton electrons deflected by a magnetic dipole in order to determine the energy of the corresponding photons.
*Corresponding
author.
0168-9002/97/$17.00 Copyright PII SO168-9002(97)00312-4
&I 1997 Elsevier Science B.V. All rights reserved
r
D. Barancourf
ASK bus
227
et al. / Nucl. fnstr. and Meth. in Phys. Rex A 388 (1997) 226-234
VME
CRATE ASIC bus
I
spaghetti monitor cables
barrel FERA cables
Tagging detector Internal Cylindrkal
Chamber
External Cylindrical Chamber
UV multiwire proportional chamber
GFUAL Experiment Fig. 1. Overview of the GRAAL experiment and its specific data acquisition system
3.Overview of the solution The design criteria for the new data acquisition system depend on two main ideas: 1. To put linear, logical, digital and transfer electronic modules close to the detectors, so that only digital information could be transmitted to the experiment room. 2. To perform hardware event building in order to reduce the dead time. Different modules and ASIC (Application Specific Integrated Circuits) have been thought up for this data acquisition system, whose architecture is based on a 68 040 microprocessor in a VME crate: - A specific bus, named ASIC bus [3], links all the detectors together. This bus is a one-way communication path and it transfers 32 bit data, with command and synchronization signals. A bus master - an intelligent VME module - allows the bus control and the automatic hardware production of events collected on the bus. ASIC: The space occupied by the electronics had to be reduced to the minimum since there was not a lot of
free space near the detectors. This limitation brought us to develop ASIC circuits in order to link the linear electronics of the detectors and the bus. Two kinds of ASIC have been designed: _ ASIC16 [4] provides information coming from 16 channel analogical to digital converters. CPT32 [S] provides information coming from 32 detectors: A 32 channel counter (32 bits per channel) _ Two 16 bit patterns, one bit per detector ASIC are located on different kinds of electronic modules adapted to each detector (see Fig. 2). The distance between the electronic modules and the experiment room, the dispersion of all these modules, and the large number of necessary settings brought us to design a remote control of all these settings via the ASIC bus from a SUN computer located in the experiment room. This workstation communicates with the VME crate through ETHERNET networking, via TCP/IP protocol. For the 4 MWPC chambers, CPT32 can provide - patterns of all the current “fired” anode wire numbers at a given moment (two 16 bit patterns for 32 wires),
228
D. Barancourl et al. ,’Nucl. Imtr. and Me/h. UI PhJs. Res. A 3HX (1997) 226-234
T T L INTERNAL B US
EC
DATA
LINEAR ELECTRONIC CONTROL
BLOCK DIAGRAM OF AN ELECTRONIC
MODULE
Fig. 2. Block diagram or an electronic module located on the ASIC bus
_ the number of times each wire has “fired” since a specific moment (32 counters, one per wire). For the 2 cylindrical MWPC, additional ADC16 allow transfers of the measurement of the charge induced on the cathode strips. The wall electronics is based on ASIC16 in order to transfer digital data (time, amplitude), coming from ADC, and on CPT32 to memorize bar scalers. The tagging detector is also associated with CPT32 circuits in order to get the patterns of the “fired” strips and the contents of conditioned scalers, giving additional information (efficiency meas-ure. etc.).
4. Electronic implementations 4.1. ASK
bus
This bus is a parallel one-way synchronous token ring whose function is to transfer all the information (data and control signals) between the bus master and the ASKS. For the transfer, we chose to use a differential ECL logic. the only one capable of meeting the timing
specifications of the signals over large distances (see Fig. 3). Data transfers are performed in block mode: The different modules distributed along the bus can be reached one after the other in geographical order. A command. sent by the bus master and set on the proper lines, selects a specific action to be performed. All the data concerning this action will be read or written (it depends on the type of the command) when the selection token arrives and validates the concerned modules from the beginning to the end of the bus. The acquisition control and the setting values of electronic modules flow along the bus, driven by the bus master. The bus requires 32 bits for data and 16 bits for the control (commands and specific signals such as clock, token, data sampling, RESET line, etc.). The bus transfers the data according to the following mechanisms: Writing: The data to be written come from the bus master towards the ASICs. After a delay, the loading of the writing command code generates the token for the first ASIC on the bus. If this ASIC is involved by the command, it waits for the data. Otherwise, it gives the token to the next ASIC.
229
D. Barancourf et al. ! Nucl. Instr. and h4eth. in Ph_vs.Rrs. A 388
t
I
I
DETECTORS I
MASTER
32 bits DATA BUS
ASIC BUS OVERVIEW
Fig. 3. ASIC bus overview. The bus master performs the data read out. The ASIC bus transfers these data between the electronic modules and the dual-port memory bank.
- Reading: The data to be read come from the ASICs towards the bus master. After a delay, the loading of the reading command code generates the token for the first ASIC on the bus. When the ASIC receives this code command, it gives the token to the next ASIC if it has nothing to transfer (or if it is not concerned by this command), or it keeps the token and transfers its own valid data. The data are put on the bus and a STROBE signal is sent by the ASIC. The ASIC does the same for all its data and gives the token to the next ASIC. 4.2. Bus master The main part of the bus master occupies 2 VME boards located in a single 2 unit card. The ASIC bus begins in the front end of this module, with 3 HE10 34 pin connectors (2 for the data lines, and 1 for the control lines). Another single one unit board performs the adaptation function for the ASIC BUS to go back to the VME crate (see Fig. 4). 4.2.1. Storage memoty The use of a dual-port memory of 1 Mbyte is an important point of the bus master design: The ASIC bus and the VME bus can reach this storage memory, divided into 2 data blocks (BANKS). One BANK can be read while the other one is written, which means using a main flip-flop circuit for the timing and memory location.
From the VME angle, this memory is seen as a classical RAM and can be reached both by reading and writing operations on the VME bus. From the ASIC bus angle, the memory addressing is produced by an indirect access: It is the content of a specific location memory, loaded beforehand, whose own address is produced by the current command code flowing on the bus and by the flip-flop circuit. This method allows storing of different data types in specific memory areas. Two kinds of commands can be performed: _ the “variable length” commands (the amount of data depends on the amount of significant data at a specific time for a specific command); _ the “fixed length” commands (the amount of data depends on the type of command and on the current bus configuration). 4.2.2. Acquisition timing and processing The bus master manages the data acquisition process via hardware signals in the front-end of the board and via software writing and reading operations in specific registers. 4.2.2.1. The busy time. The system processing is mainly conditioned by the busy time, which prevents any interruptions from the trigger when it is set. A RESET pulse on the bus master sets the busy time in its active state. Then, if a specific bit is set by program
D. Barancowt et aC ; Nucl. Ins&. and M&h. in Phys. Rex A 388 (1997) 226-234
230
VME
ER : ::
: T A
z
BUS
MEMORY DUAL, PORT
4 V
s
M
i c 0
: R
E
:
::
I B
R
G
I
K
I
0
s
B
I
t
;: K
k
I
I
1
-E
RESERVED AREA
EB4---I
s
L
G I
5 D
: I
! TIMING
CONTROL AND TOKEN LOGIC
+ START
t
BUS MASTER
BLOCK
DIAGRAM
REJET
t
DEAD TIME
Fig. 4. Simplified scheme of the bus master, an intelligent VME module which allows the dialogue between the VME bus and the ASIC bus. Upon receiving the START signal of the experiment, the bus master generates a token on the ASIC bus. Each ASIC module puts its valid data on the ASIC bus when the token arrives
in the control register (to start acquisition, for example), the busy time is no longer active. In this acquisition phase, the clock signal is stopped to reduce the noise. This signal will be electronically reactived after a START signal arrives, when the next high to low clock transition occurs, which will produce the busy time, the conversion time and the transfer of an event. When a START signal arrives, there are 2 possibilities: ~ The busy time is not uctive. In this case the event is accepted. The busy time becomes active and a GATE is generated. At the end of the conversion time, the data are available in each ASIC. If the bus master is ready to read the data (free space left in the dual-port memory), the automatic read out of the data begins. If it is not ready, it waits until a BANK is free. After the read out is performed, the busy time is no longer active. During the conversion time, the current event can be canceled via a signal on the BAD input line; the event is lost, and the busy time and conversion time become inactive. ~ The busy time is active. In this case the event is rejected. The single action is a very short impulse
on the REJET events.
line in order
to count
the rejected
4.2.2.2. The readout mechanism. The main feature of the bus master is the automatic data read out and event building in 2 memory blocks. For each event coming from the general trigger of the experiment, the read out command collects its valid parameters. Each parameter is marked with a specific identifier so that it can be easily characterized during the analysis phase. For each event, the bus master provides a header word (each bit of this word has a specific meaning on the origin of the event, its length etc.) and a terminal word specifying a 28 bit event counter or a 10 bit event counter and a 20 bit time counter. The resulting event is stored in one of the two BANKS. When the current BANK is full (a threshold value has been given by a program), the bus master switches to the other bank when it is free and an interruption is produced on VME bus so that the CPU can process this buffer of events. The busy time is then set to its unactive state.
D. Barancourt et al. / Nucl. Instr. and Meth. in Ph_vs.Res. A 388 (1997) 226-234 4.3.
131
detectors or counting and transfer of 32 input channels (see details of these two functions below). This circuit can be used whenever counting scalers are necessary, thanks to its counting performances (32 bits, 100 MHz). The CPT32 is able to provide 2 different and independent read out data, depending on the type of the command sent through the bus: 1. Memorization of 32 binary information (2 patterns of 16 bits each) at any moment given by an external module (VALID): During the acquisition phase, the data read out cycle provides for all the CPT32 the 32 binary information of their respective hit detectors for the current event. A preceding 16 bit identifier word is associated with each pattern in order to facilitate the future analysis. If none of these 16 bits is set for a pattern, the corresponding pair identifier-pattern will not appear in the event. 2. Counting of the number of active edges of these binary data during a fixed time. The choice for the active edge is globally done for each CPT32 (electric setting of
ASK Developments
A lot of specific developments have been performed during the 70s~80s [6,7] to process signals issued from multiwire proportional chambers to offer compact electronic systems at low cost. The experimentator’s aim was to produce compact, high performance, reliable electronics and, finally, at a lower cost than the traditional one. For that purpose, we decided to design ASIC circuits (standard cells CMOS technology) distributed on boards close to the detectors. Two ASIC circuits allow the transfer of the numerical data on the ASK bus: 4.3.1. CPT32 The CPT32 circuit is placed on the detector module, immediately behind the memory output of an integrated circuit (bipolar technology) developed for the signal processing (amplification, amplitude discrimination, delay, coincidence, etc.) coming from the detectors [S] (see Fig. 5). This CPT32 can provide patterns of the “fired”
COIJNTER READING
READING BUS
m READING
COMMAND
32 COUNTER
32 BIT
GENERAL
II
LOGIC +
i
IDENTIFIER PRIORITY
I
01I
11
32 LOGIC lNPUTS
x C 1. 0 UR s I v rL
I VALID
P A T T E R N
WRl’l 1NG
WRITING
COMMAND
+ RkADING
L o G I c
1
I
-
1
+
j
READINGBUS
B I D I 8 E C T I 0 N N A L
DATA BUS 32 bit!
II
B II s
2 Bits external command
$
CPT32 BLOCK DIAGRAM
Fig. 5. Diagram of the CPT32, an ASIC circuit located on a board close to the detectors, capable to provide coming from 32 detectors and the counting of the number of active edges of these binary information.
32 binary
information
232
D. Barancourt rt al. ! Nucl. lnstr. and Meth. in Phw. Rex A 38X (1997) 226-234
the plug O/l). It is possible to write and read the internal registers of the CPT32 and to test the proper operation with a software program. These counters are read in block mode (32 values preceded or not by the identifier). Periodically, during the acquisition phase, a command is sent to read the current scaler contents in order to fill and display spectra. These scaler contents can be globally or individually erased at any moment by a specific command. 4.3.2. ASICl6 The ASIC16 circuit is placed on the detector module and has been designed to process analogical measurements coming from 18 bit audio Analog to Digital Converters (see Fig. 6). It allows to process 16 conversion channels (from 10 to 13 bits) Each channel is preceded by its own identifier (16 bit word previously loaded) so that only hit converters provide data. The ASICI6 is able to do the following: 1, Memorize 16 binary information (one per channel) at any moment given by an external module (VALID). Each set bit indicates a valid channel to be read. It is
possible to extend each bit as its left and right sides to read the bordering channels. 2. Generate a command (CONVERT) for the ADC to begin the conversion (CLOCK 4 MHz) and a serial pulse stream for the conversion. 3. Shift the 16 serial input coming from the audio analog to digital converter to transform them into 16 parallel data. 4. Suppress the transfer for channels whose content is negative. 5. Put the data in a valid format for each channel. 6. Transfer the valid identifier-value pairs on the ASIC bus (DATA BUS 32 bits). 7. Format its data on 10, 11, 12 or 13 bits. 8. Read and write internal registers. An important feature common to CPT32 and ASIC16 circuits is the possibility to address external modules on the board. The hardware timing for this is inside the circuits, and the choice can be done by software for the whole bus (specific command code reserved for external access). This specificity is used to set thresholds, delays etc. of modules located on the electronic board. 32 front panel input channels can be read by one validation pulse.
KIXDING
LOGICAL
COMMAND
DATA BUS 32 bits
INPUTS
SERIAL INPUTS
4 Bits external command
ASIC16 BLOCK DIAGRAM
Fig. 6. Diagram
of the ASIC16.
an ASIC circuit capable
to provide
16 binary
information
coming
from ADC.
D. Barancourt et al. / Nurl. Instr. and Meth. in Phss. Rex A 388 (1997) 226-234 5. Software 5.1. Programmable
electronics
The idea of electronics close to the detectors has been put in concrete form by the design of different modules, combining analogical and logical programmable electronics, interfacing electronics and ASIC bus via ASICl6 and CPT32 circuits. A software program has been written for the settings of electronic modules located on the different boards. This program has been developed on a SUN station in C language. A powerful graphic user interface, named SL-GMS [9], has been used to design graphical representations of the different kinds of boards. The first panel is the stylized scheme of the detection system. Each detector can be clicked and a graphical representation of the detector and all of its boards appears. Each board can be selected and a simplified drawing of the concerned board appears (see Fig. 7).
133
A set of sliders and switches allows the user to realize the different electronic settings (change of thresholds, delay. amplitude, width values, connection of inspection channels on an oscilloscope, etc). This program communicates with VME CPU thanks to a client-server link via ETHERNET. Upon receiving requests from the SUN, a server program running on the VME CPU is able to send any command to the ASIC16 and CPT32 connected to the ASIC bus via the bus master. 5.2. GRAAL data acquisition sqftware system The data acquisition system derives from OASIS [lo] and has been modified for ASIC specifications. The user interface has been written with SL-GMS. The VME CPU is a MVME167 and runs the VxWorks real-time system. The dialogue between SUN and VME CPU is done by RPC commands and sockets for the event buffer transfer.
Fig. 7. Graphical representation of a plane MWPC board: A specific program has be written which allows to perform the different kinds of boards from a SUN station located in the experiment room.
the settings of all
234
D. Baraneou~~ ef ai. i Nuci. Instr. and h4eth. in Phvs. Rex A 388 (IYY7I 226-234
The first task of the acquisition program is to read the bus configuration by sending specific commands to the ASICs via the bus. Proper identifiers are written for each channel of each ASIC according to the configuration currently found on the bus. An electronic data file (previously recorded during the electronic setting phase) is sent to the ASIC to perform all the adequate settings for the experiment. Finally. a control panel is used to start and stop the acquisition, to record data on DLT media, to create biparametric and linear conditioned spectra and to display spectra. Once the acquisition is running, the bus master fills one of these storage memories with the valid events coming from ASIC and FERA bus. Each event is preceded by a header word (16 bits are reserved for the event length) and ended by a specific “end of event” word. When this memory is full, it sends an interruption to the VME bus. Upon receiving the interruption, the CPU fills one of the 2 flip-flop event buffers which are sent to the SUN client program via sockets. Once transfered in the shared memory of the SUN station, these buffers are processed by the recording task, the spectra building task and the control task running together in the SUN station. 5.3. Pet@7nances The data acquisition following formula:
time for an event is given by the
T + t + (n*t,), where T is the largest conversion time (4 ps for audio converter), t depends on the length of the bus (5 nsjm). In this experiment, the length of the bus is about 150 m. and t1 = 125 ns (2 periods of the VME 16 MHz clock) and n is the number of 32 bit words flowing on the bus for a nuclear event. The data acquisition time for an average event (100 words of 32 bits) is about 17.5 pis. Therefore, the transfer rate is about 23 Mbyte/s. At this moment, the rate is limited by ETHERNET transfer (about 600 Kbyte/s). This limitation could be reduced later on with a lOOMbit FAST ETHERNET.
6. Conclusion Some other modules have been designed and inserted in order to extend the data acquisition system to external buses: ~ FASIC module allows to handle FERA parameters in the global acquisition. ~ PCASIC module allows to send commands to a GPIB bus in order to manage the remote control of the laser beam optical elements via a PC computer. All the boards close to the detectors have now been tested, thanks to the electronics setting program. The data acquisition system is now operational. The solution of electronic boards and customized ASIC circuits close to the detectors fits to the required specifications for the GRAAL experiment and could be used in any other experiments needing the same requirements.
References 111 T. Russew, Etude et simulation d’un dttecteur pour I’expirience GRAAL A I’ESRF-Application a la photoproduction d’ttrangett, Thesis. ISN Grenoble, 1995. [‘I P. Levi Sandri et al.. Nucl. Instr. and Meth. A 370 (1996) 396. 131 J. Bouvier and M. Tournier, Le bus ASIC. Internal report of ISN. 1996. c41 J. Bouvier and M. Tournier. ASIC16, Internal repel-t of ISN. 1994. I51 J. Bouvier and M. Tournier, CPT32, Internal report of ISN. 1993. 161 R. Foglio, C. Perrin, J. Pouxe, U. Bart and E. Schwarz. A fast readout system for multiwire proportional chambers. Proc. 2nd Ispra Nuclear Electronics Symp., 1975. 171 EFCIS. 8 channel on chamber MWPC readout module. Technical manual. [81 J. Bugnet, Conception et test d’un circuit inttgrt (ASIC): application aux chambres multifils et aux photomultipicateurs de I’expirience GRAAL, Thesis, ISN Grenoble, 1995. Sherrill-Lubinski Corporation, Graphic 191 SL-GMS, Modeling System. [lOI R Douet. N. Borome, H. Harroch, T. Tran-Khan, P. Didelon and J. Navarre, A simple data acquisition architecture for low cost to high data throughput: OASISOpen-Acquisition-System-I.P.N.-L.N. Saturne, Internal report of IPN. 1994.
Instrument
and Methods
in Physics
Research
A 388 (1997) 226-234
NUCLEAR fNSTRUMENtS 8 METNOOS IN PHYStCS RESEARCH SectIon A
ELSEVIER
The data acquisition system for the GRAAL experiment at the ESRF in Grenoble D. BarancourV’, G. Barbiera, J. Bouvief, B. MeillorP*, M. Tourniera, R. Douetb, H. Harrochb aInstitut des Sciences Nucliaires 53. Auenue des mar@rs. 38 026 Grenoble Cede.r, France b Institut de Phvsique Nuclkaire. Received
91 405 0rsa.v Cedex. France
23 July 1996; revised form received 23 December
1996
Abstract A specific data acquisition system System Acquisition Graal Asic, (SAGA) has been implemented for a multidetector experiment. Its main feature is an hardware event builder associated with compact and programmable electronics based on two kinds of ASIC circuits linked by a bus, located on the detectors. The transfer rate on this bus is about 23 Mbyte/s.
1. Introduction
2. Description of GRAAL experiment
As we had to develop a new data acquisition system for the Grenoble Anneau Accelerateur Laser (GRAAL) [l] experiment with multidetectors, we had to take into account several requirements for building up a specific system: _ First, the large number of channels (about 4000) to be read and the high acquisition rate obliged us to use a fast read out system in order to reduce the busy time to the minimum. - Secondly, since the distance between the detectors and the experiment room was about 50 m, there was the problem of transporting small signals over great distances (deterioration of signals, high cost of cabling, connection problems). ~ Thirdly, some of the detectors were already interfaced to CAMAC modules and their data read out was performed with a FERA bus. It was decided to keep this feature for the detectors concerned. _ Finally, it was important to read only the meaningful channels (about loo), in order to minimize the amount of data collected. Section 2 presents a short description of the GRAAL experiment followed by an overview of the proposed solution, and more details about the different electronic implementations. Last, the performance of the system is given.
This nuclear physics experiment is located in Grenoble, at the European Synchrotron Radiation Facility (ESRF). The project objective is to produce a beam of tagged photons and to first study the beam interactions with a liquid-hydrogen target. The beam production is quite original: the photons are produced by the Compton backscattering of a laser beam on the 6 GeV electrons produced by the synchrotron. The photons gain 1.5 GeV of energy and keep the polarization of the laser beam. The first topics to be studied are the strangeness photoproduction and q disintegration. For this purpose, a 472 detection system has been designed around the target (see Fig. 1). Its goal is: 1. to restore charged particles trajectories and identification by ~ 2 planes of multiwire proportional chambers MWPC, ~ 2 cylindrical multiwire proportional chambers MWPC, ~ 2 scintillating plastic detectors, a double wall and a barrel; 2. to measure the emission angle and the energy of neutral particles (photons) by ~ a BGO ball calorimeter detector [Z], ~ a wall made of plastic-lead sandwiches. A tagging detector (silicium n-strips) has been placed in the synchrotron tunnel, to detect the position of Compton electrons deflected by a magnetic dipole in order to determine the energy of the corresponding photons.
*Corresponding
author.
0168-9002/97/$17.00 Copyright PII SO168-9002(97)00312-4
&I 1997 Elsevier Science B.V. All rights reserved
r
D. Barancourf
ASK bus
227
et al. / Nucl. fnstr. and Meth. in Phys. Rex A 388 (1997) 226-234
VME
CRATE ASIC bus
I
spaghetti monitor cables
barrel FERA cables
Tagging detector Internal Cylindrkal
Chamber
External Cylindrical Chamber
UV multiwire proportional chamber
GFUAL Experiment Fig. 1. Overview of the GRAAL experiment and its specific data acquisition system
3.Overview of the solution The design criteria for the new data acquisition system depend on two main ideas: 1. To put linear, logical, digital and transfer electronic modules close to the detectors, so that only digital information could be transmitted to the experiment room. 2. To perform hardware event building in order to reduce the dead time. Different modules and ASIC (Application Specific Integrated Circuits) have been thought up for this data acquisition system, whose architecture is based on a 68 040 microprocessor in a VME crate: - A specific bus, named ASIC bus [3], links all the detectors together. This bus is a one-way communication path and it transfers 32 bit data, with command and synchronization signals. A bus master - an intelligent VME module - allows the bus control and the automatic hardware production of events collected on the bus. ASIC: The space occupied by the electronics had to be reduced to the minimum since there was not a lot of
free space near the detectors. This limitation brought us to develop ASIC circuits in order to link the linear electronics of the detectors and the bus. Two kinds of ASIC have been designed: _ ASIC16 [4] provides information coming from 16 channel analogical to digital converters. CPT32 [S] provides information coming from 32 detectors: A 32 channel counter (32 bits per channel) _ Two 16 bit patterns, one bit per detector ASIC are located on different kinds of electronic modules adapted to each detector (see Fig. 2). The distance between the electronic modules and the experiment room, the dispersion of all these modules, and the large number of necessary settings brought us to design a remote control of all these settings via the ASIC bus from a SUN computer located in the experiment room. This workstation communicates with the VME crate through ETHERNET networking, via TCP/IP protocol. For the 4 MWPC chambers, CPT32 can provide - patterns of all the current “fired” anode wire numbers at a given moment (two 16 bit patterns for 32 wires),
228
D. Barancourl et al. ,’Nucl. Imtr. and Me/h. UI PhJs. Res. A 3HX (1997) 226-234
T T L INTERNAL B US
EC
DATA
LINEAR ELECTRONIC CONTROL
BLOCK DIAGRAM OF AN ELECTRONIC
MODULE
Fig. 2. Block diagram or an electronic module located on the ASIC bus
_ the number of times each wire has “fired” since a specific moment (32 counters, one per wire). For the 2 cylindrical MWPC, additional ADC16 allow transfers of the measurement of the charge induced on the cathode strips. The wall electronics is based on ASIC16 in order to transfer digital data (time, amplitude), coming from ADC, and on CPT32 to memorize bar scalers. The tagging detector is also associated with CPT32 circuits in order to get the patterns of the “fired” strips and the contents of conditioned scalers, giving additional information (efficiency meas-ure. etc.).
4. Electronic implementations 4.1. ASK
bus
This bus is a parallel one-way synchronous token ring whose function is to transfer all the information (data and control signals) between the bus master and the ASKS. For the transfer, we chose to use a differential ECL logic. the only one capable of meeting the timing
specifications of the signals over large distances (see Fig. 3). Data transfers are performed in block mode: The different modules distributed along the bus can be reached one after the other in geographical order. A command. sent by the bus master and set on the proper lines, selects a specific action to be performed. All the data concerning this action will be read or written (it depends on the type of the command) when the selection token arrives and validates the concerned modules from the beginning to the end of the bus. The acquisition control and the setting values of electronic modules flow along the bus, driven by the bus master. The bus requires 32 bits for data and 16 bits for the control (commands and specific signals such as clock, token, data sampling, RESET line, etc.). The bus transfers the data according to the following mechanisms: Writing: The data to be written come from the bus master towards the ASICs. After a delay, the loading of the writing command code generates the token for the first ASIC on the bus. If this ASIC is involved by the command, it waits for the data. Otherwise, it gives the token to the next ASIC.
229
D. Barancourf et al. ! Nucl. Instr. and h4eth. in Ph_vs.Rrs. A 388
t
I
I
DETECTORS I
MASTER
32 bits DATA BUS
ASIC BUS OVERVIEW
Fig. 3. ASIC bus overview. The bus master performs the data read out. The ASIC bus transfers these data between the electronic modules and the dual-port memory bank.
- Reading: The data to be read come from the ASICs towards the bus master. After a delay, the loading of the reading command code generates the token for the first ASIC on the bus. When the ASIC receives this code command, it gives the token to the next ASIC if it has nothing to transfer (or if it is not concerned by this command), or it keeps the token and transfers its own valid data. The data are put on the bus and a STROBE signal is sent by the ASIC. The ASIC does the same for all its data and gives the token to the next ASIC. 4.2. Bus master The main part of the bus master occupies 2 VME boards located in a single 2 unit card. The ASIC bus begins in the front end of this module, with 3 HE10 34 pin connectors (2 for the data lines, and 1 for the control lines). Another single one unit board performs the adaptation function for the ASIC BUS to go back to the VME crate (see Fig. 4). 4.2.1. Storage memoty The use of a dual-port memory of 1 Mbyte is an important point of the bus master design: The ASIC bus and the VME bus can reach this storage memory, divided into 2 data blocks (BANKS). One BANK can be read while the other one is written, which means using a main flip-flop circuit for the timing and memory location.
From the VME angle, this memory is seen as a classical RAM and can be reached both by reading and writing operations on the VME bus. From the ASIC bus angle, the memory addressing is produced by an indirect access: It is the content of a specific location memory, loaded beforehand, whose own address is produced by the current command code flowing on the bus and by the flip-flop circuit. This method allows storing of different data types in specific memory areas. Two kinds of commands can be performed: _ the “variable length” commands (the amount of data depends on the amount of significant data at a specific time for a specific command); _ the “fixed length” commands (the amount of data depends on the type of command and on the current bus configuration). 4.2.2. Acquisition timing and processing The bus master manages the data acquisition process via hardware signals in the front-end of the board and via software writing and reading operations in specific registers. 4.2.2.1. The busy time. The system processing is mainly conditioned by the busy time, which prevents any interruptions from the trigger when it is set. A RESET pulse on the bus master sets the busy time in its active state. Then, if a specific bit is set by program
D. Barancowt et aC ; Nucl. Ins&. and M&h. in Phys. Rex A 388 (1997) 226-234
230
VME
ER : ::
: T A
z
BUS
MEMORY DUAL, PORT
4 V
s
M
i c 0
: R
E
:
::
I B
R
G
I
K
I
0
s
B
I
t
;: K
k
I
I
1
-E
RESERVED AREA
EB4---I
s
L
G I
5 D
: I
! TIMING
CONTROL AND TOKEN LOGIC
+ START
t
BUS MASTER
BLOCK
DIAGRAM
REJET
t
DEAD TIME
Fig. 4. Simplified scheme of the bus master, an intelligent VME module which allows the dialogue between the VME bus and the ASIC bus. Upon receiving the START signal of the experiment, the bus master generates a token on the ASIC bus. Each ASIC module puts its valid data on the ASIC bus when the token arrives
in the control register (to start acquisition, for example), the busy time is no longer active. In this acquisition phase, the clock signal is stopped to reduce the noise. This signal will be electronically reactived after a START signal arrives, when the next high to low clock transition occurs, which will produce the busy time, the conversion time and the transfer of an event. When a START signal arrives, there are 2 possibilities: ~ The busy time is not uctive. In this case the event is accepted. The busy time becomes active and a GATE is generated. At the end of the conversion time, the data are available in each ASIC. If the bus master is ready to read the data (free space left in the dual-port memory), the automatic read out of the data begins. If it is not ready, it waits until a BANK is free. After the read out is performed, the busy time is no longer active. During the conversion time, the current event can be canceled via a signal on the BAD input line; the event is lost, and the busy time and conversion time become inactive. ~ The busy time is active. In this case the event is rejected. The single action is a very short impulse
on the REJET events.
line in order
to count
the rejected
4.2.2.2. The readout mechanism. The main feature of the bus master is the automatic data read out and event building in 2 memory blocks. For each event coming from the general trigger of the experiment, the read out command collects its valid parameters. Each parameter is marked with a specific identifier so that it can be easily characterized during the analysis phase. For each event, the bus master provides a header word (each bit of this word has a specific meaning on the origin of the event, its length etc.) and a terminal word specifying a 28 bit event counter or a 10 bit event counter and a 20 bit time counter. The resulting event is stored in one of the two BANKS. When the current BANK is full (a threshold value has been given by a program), the bus master switches to the other bank when it is free and an interruption is produced on VME bus so that the CPU can process this buffer of events. The busy time is then set to its unactive state.
D. Barancourt et al. / Nucl. Instr. and Meth. in Ph_vs.Res. A 388 (1997) 226-234 4.3.
131
detectors or counting and transfer of 32 input channels (see details of these two functions below). This circuit can be used whenever counting scalers are necessary, thanks to its counting performances (32 bits, 100 MHz). The CPT32 is able to provide 2 different and independent read out data, depending on the type of the command sent through the bus: 1. Memorization of 32 binary information (2 patterns of 16 bits each) at any moment given by an external module (VALID): During the acquisition phase, the data read out cycle provides for all the CPT32 the 32 binary information of their respective hit detectors for the current event. A preceding 16 bit identifier word is associated with each pattern in order to facilitate the future analysis. If none of these 16 bits is set for a pattern, the corresponding pair identifier-pattern will not appear in the event. 2. Counting of the number of active edges of these binary data during a fixed time. The choice for the active edge is globally done for each CPT32 (electric setting of
ASK Developments
A lot of specific developments have been performed during the 70s~80s [6,7] to process signals issued from multiwire proportional chambers to offer compact electronic systems at low cost. The experimentator’s aim was to produce compact, high performance, reliable electronics and, finally, at a lower cost than the traditional one. For that purpose, we decided to design ASIC circuits (standard cells CMOS technology) distributed on boards close to the detectors. Two ASIC circuits allow the transfer of the numerical data on the ASK bus: 4.3.1. CPT32 The CPT32 circuit is placed on the detector module, immediately behind the memory output of an integrated circuit (bipolar technology) developed for the signal processing (amplification, amplitude discrimination, delay, coincidence, etc.) coming from the detectors [S] (see Fig. 5). This CPT32 can provide patterns of the “fired”
COIJNTER READING
READING BUS
m READING
COMMAND
32 COUNTER
32 BIT
GENERAL
II
LOGIC +
i
IDENTIFIER PRIORITY
I
01I
11
32 LOGIC lNPUTS
x C 1. 0 UR s I v rL
I VALID
P A T T E R N
WRl’l 1NG
WRITING
COMMAND
+ RkADING
L o G I c
1
I
-
1
+
j
READINGBUS
B I D I 8 E C T I 0 N N A L
DATA BUS 32 bit!
II
B II s
2 Bits external command
$
CPT32 BLOCK DIAGRAM
Fig. 5. Diagram of the CPT32, an ASIC circuit located on a board close to the detectors, capable to provide coming from 32 detectors and the counting of the number of active edges of these binary information.
32 binary
information
232
D. Barancourt rt al. ! Nucl. lnstr. and Meth. in Phw. Rex A 38X (1997) 226-234
the plug O/l). It is possible to write and read the internal registers of the CPT32 and to test the proper operation with a software program. These counters are read in block mode (32 values preceded or not by the identifier). Periodically, during the acquisition phase, a command is sent to read the current scaler contents in order to fill and display spectra. These scaler contents can be globally or individually erased at any moment by a specific command. 4.3.2. ASICl6 The ASIC16 circuit is placed on the detector module and has been designed to process analogical measurements coming from 18 bit audio Analog to Digital Converters (see Fig. 6). It allows to process 16 conversion channels (from 10 to 13 bits) Each channel is preceded by its own identifier (16 bit word previously loaded) so that only hit converters provide data. The ASICI6 is able to do the following: 1, Memorize 16 binary information (one per channel) at any moment given by an external module (VALID). Each set bit indicates a valid channel to be read. It is
possible to extend each bit as its left and right sides to read the bordering channels. 2. Generate a command (CONVERT) for the ADC to begin the conversion (CLOCK 4 MHz) and a serial pulse stream for the conversion. 3. Shift the 16 serial input coming from the audio analog to digital converter to transform them into 16 parallel data. 4. Suppress the transfer for channels whose content is negative. 5. Put the data in a valid format for each channel. 6. Transfer the valid identifier-value pairs on the ASIC bus (DATA BUS 32 bits). 7. Format its data on 10, 11, 12 or 13 bits. 8. Read and write internal registers. An important feature common to CPT32 and ASIC16 circuits is the possibility to address external modules on the board. The hardware timing for this is inside the circuits, and the choice can be done by software for the whole bus (specific command code reserved for external access). This specificity is used to set thresholds, delays etc. of modules located on the electronic board. 32 front panel input channels can be read by one validation pulse.
KIXDING
LOGICAL
COMMAND
DATA BUS 32 bits
INPUTS
SERIAL INPUTS
4 Bits external command
ASIC16 BLOCK DIAGRAM
Fig. 6. Diagram
of the ASIC16.
an ASIC circuit capable
to provide
16 binary
information
coming
from ADC.
D. Barancourt et al. / Nurl. Instr. and Meth. in Phss. Rex A 388 (1997) 226-234 5. Software 5.1. Programmable
electronics
The idea of electronics close to the detectors has been put in concrete form by the design of different modules, combining analogical and logical programmable electronics, interfacing electronics and ASIC bus via ASICl6 and CPT32 circuits. A software program has been written for the settings of electronic modules located on the different boards. This program has been developed on a SUN station in C language. A powerful graphic user interface, named SL-GMS [9], has been used to design graphical representations of the different kinds of boards. The first panel is the stylized scheme of the detection system. Each detector can be clicked and a graphical representation of the detector and all of its boards appears. Each board can be selected and a simplified drawing of the concerned board appears (see Fig. 7).
133
A set of sliders and switches allows the user to realize the different electronic settings (change of thresholds, delay. amplitude, width values, connection of inspection channels on an oscilloscope, etc). This program communicates with VME CPU thanks to a client-server link via ETHERNET. Upon receiving requests from the SUN, a server program running on the VME CPU is able to send any command to the ASIC16 and CPT32 connected to the ASIC bus via the bus master. 5.2. GRAAL data acquisition sqftware system The data acquisition system derives from OASIS [lo] and has been modified for ASIC specifications. The user interface has been written with SL-GMS. The VME CPU is a MVME167 and runs the VxWorks real-time system. The dialogue between SUN and VME CPU is done by RPC commands and sockets for the event buffer transfer.
Fig. 7. Graphical representation of a plane MWPC board: A specific program has be written which allows to perform the different kinds of boards from a SUN station located in the experiment room.
the settings of all
234
D. Baraneou~~ ef ai. i Nuci. Instr. and h4eth. in Phvs. Rex A 388 (IYY7I 226-234
The first task of the acquisition program is to read the bus configuration by sending specific commands to the ASICs via the bus. Proper identifiers are written for each channel of each ASIC according to the configuration currently found on the bus. An electronic data file (previously recorded during the electronic setting phase) is sent to the ASIC to perform all the adequate settings for the experiment. Finally. a control panel is used to start and stop the acquisition, to record data on DLT media, to create biparametric and linear conditioned spectra and to display spectra. Once the acquisition is running, the bus master fills one of these storage memories with the valid events coming from ASIC and FERA bus. Each event is preceded by a header word (16 bits are reserved for the event length) and ended by a specific “end of event” word. When this memory is full, it sends an interruption to the VME bus. Upon receiving the interruption, the CPU fills one of the 2 flip-flop event buffers which are sent to the SUN client program via sockets. Once transfered in the shared memory of the SUN station, these buffers are processed by the recording task, the spectra building task and the control task running together in the SUN station. 5.3. Pet@7nances The data acquisition following formula:
time for an event is given by the
T + t + (n*t,), where T is the largest conversion time (4 ps for audio converter), t depends on the length of the bus (5 nsjm). In this experiment, the length of the bus is about 150 m. and t1 = 125 ns (2 periods of the VME 16 MHz clock) and n is the number of 32 bit words flowing on the bus for a nuclear event. The data acquisition time for an average event (100 words of 32 bits) is about 17.5 pis. Therefore, the transfer rate is about 23 Mbyte/s. At this moment, the rate is limited by ETHERNET transfer (about 600 Kbyte/s). This limitation could be reduced later on with a lOOMbit FAST ETHERNET.
6. Conclusion Some other modules have been designed and inserted in order to extend the data acquisition system to external buses: ~ FASIC module allows to handle FERA parameters in the global acquisition. ~ PCASIC module allows to send commands to a GPIB bus in order to manage the remote control of the laser beam optical elements via a PC computer. All the boards close to the detectors have now been tested, thanks to the electronics setting program. The data acquisition system is now operational. The solution of electronic boards and customized ASIC circuits close to the detectors fits to the required specifications for the GRAAL experiment and could be used in any other experiments needing the same requirements.
References 111 T. Russew, Etude et simulation d’un dttecteur pour I’expirience GRAAL A I’ESRF-Application a la photoproduction d’ttrangett, Thesis. ISN Grenoble, 1995. [‘I P. Levi Sandri et al.. Nucl. Instr. and Meth. A 370 (1996) 396. 131 J. Bouvier and M. Tournier, Le bus ASIC. Internal report of ISN. 1996. c41 J. Bouvier and M. Tournier. ASIC16, Internal repel-t of ISN. 1994. I51 J. Bouvier and M. Tournier, CPT32, Internal report of ISN. 1993. 161 R. Foglio, C. Perrin, J. Pouxe, U. Bart and E. Schwarz. A fast readout system for multiwire proportional chambers. Proc. 2nd Ispra Nuclear Electronics Symp., 1975. 171 EFCIS. 8 channel on chamber MWPC readout module. Technical manual. [81 J. Bugnet, Conception et test d’un circuit inttgrt (ASIC): application aux chambres multifils et aux photomultipicateurs de I’expirience GRAAL, Thesis, ISN Grenoble, 1995. Sherrill-Lubinski Corporation, Graphic 191 SL-GMS, Modeling System. [lOI R Douet. N. Borome, H. Harroch, T. Tran-Khan, P. Didelon and J. Navarre, A simple data acquisition architecture for low cost to high data throughput: OASISOpen-Acquisition-System-I.P.N.-L.N. Saturne, Internal report of IPN. 1994.