- Email: [email protected]
Interfacing CAMAC to a UNIX system
Iuterfaces in Cornputing, 1 ( 1 9 8 3 ) 329 - 337
329
I N T E R F A C I N G CAMAC TO A UNIX SYSTEM G. LEE a n d D. W I E G A N D T
Organisation Europdenne p o u r la Recherche Nucldaire, CH- 121 l Geneva 23 (Switzerland) (Received July 7, 1 9 8 3 )
Summary In this paper t he interfacing of a CAMAC system to a PDP-11/45 running UNIX version 7 is described. The main emphasis o f the paper is to describe the CAMAC device driver for UNIX and to illustrate the advantages o f programming the driver in the high level language C as compared with programming in assembler. It is assumed that the readership is primarily f r o m a hardware background, and t h e r e f o r e we summarize some of the background and history o f t he UNIX system.
1. The UNIX system and phi l os ophy
1.1. Evolution o f UNIX The first version of the UNIX operating system was developed by T h o m p s o n o f Bell Laboratories for a PDP-7 c o m p u t e r . Later, in c o n j u n c t i o n with Ritchie, a second version was developed for t he PDP-11/20 c o m p u t e r [ 1 - 3 ]. During the 1970s, UNIX for the PDP-11 range evolved through several versions with version 7 (V7) being the most widely used t o d a y and indeed the version th at runs on the PDP-11/45 at the Organisation Europ~enne pour la Rech er ch e Nucl~aire (CERN). The latest version is system III which was released at the end o f 1981 and the n e x t version, system V, is due for release in mid-1983. In addition to the developments for the PDP-11 range, UNIX was implemented on m a n y ot her machines, notably the Digital E q u i p m e n t C o r p o r atio n (DEC) VAX range. 32V and system III are UNIX systems for VAX hardware and are distributed by Bell Laboratories. In a separate d e v e l o p m e n t at the C o m p u t e r Science Division of the University of California at Berkeley, a virtual m e m o r y version o f UNIX for t he VAX was produced. T o d a y , Berkeley UNIX 4.1 BSD is the most widely used UNIX system on VAX hardware. UNIX is now available f or a large range o f machines from m i c r o c o m p u t e r s to mainframe co m put er s and the use and popul ari t y o f UNIX have grown e n o rmo u s ly . F u r t h e r m o r e , with t he advent o f cheap 16-bit m i c r o c o m p u t e r s and work stations, UNIX is rapidly becoming the de facto standard operating system f o r such machines. 0252-7308/83/$3.00
C) Elsevier S e q u o i a / P r i n t e d in T h e N e t h e r l a n d s
330
1.2. S o m e UNIX terminology The kernel of the UNIX system is almost entirely written in the language C, a language developed from BCPL by Kernighan and Ritchie [4]. The kernel itself is relatively small in terms of code and, with simplicity an important design goal, it provides a limited well-defined set of system calls for process and file management. The term UNIX system is often used to describe not only the kernel but also the large number of utilities and application programs which make up the UNIX " t o o l k i t " of software modules. The use of this tool kit is an important aspect of software development under UNIX. Rather than the development of a large number of specialized software packages, the philosophy in UNIX is to take the basic software tool kit and to combine tools in order to perform more elaborate tasks. The UNIX "shell" and " p i p e " mechanisms are instrumental in this combination, and it is through this modular approach to software development that UNIX achieves a large degree of flexibility. 1.3. The U N I X shell The UNIX shell is a process which handles the user interface to the system. In addition to being the c o m m a n d language for issuing and executing commands, it provides a variety of control flow primitives for iteration and branching. As UNIX is an interactive system, the shell normally accepts its input from a terminal, but it can easily be redirected to take its input from a file. Files containing lists of shell commands are called shell scripts and, using the control structures mentioned above, complex programs can be developed. The shell language, although sometimes a little cryptic, is very powerful and is a flexible way of combining the software tools referred to in Section 1.2. 1.4. U N I X pipes The UNIX pipe mechanism allows the o u t p u t of one process to be used as input to another in a manner that is totally transparent to the user. Pipes make use of both shared m e m o r y and temporary files depending on the quantity of data being exchanged and all problems of data flow control in the pipe are handled by the system. Pipes provide the mechanism whereby users can easily chain together simple processes or "filters" to perform a more complex operation. It should be noted, however, that pipes are not a general-purpose interprocess communication facility since they only operate between " r e l a t e d " processes, i.e. processes which share a c o m m o n ancestor. 1.5. U N I X and real-time applications UNIX was developed primarily as a time-sharing system and is not aimed at real-time applications. Although several attempts have been made to build a "real-time UNIX" [5, 6], there are a number of deficiencies in standard UNIX which restrict its applicability in the area of real-time problems.
331
(1) No user-accessible mechanisms exist in standard UNIX to lock a process in m e m o r y so as to guarantee response within a given time limit to external stimuli. In keeping with its time-sharing philosophy, processes compete in an equal m a nne r for m e m o r y resources and are always candidates for swapping. (2) As y n ch r onous i n p u t - o u t p u t (I/O) requests have only recently been introduced in UNIX. In V7, once an l/O request has been issued, the issuing process will wait until the request has been completed. A c o m m o n requirem e n t in real-time applications is to be able to wait on several I/O requests or events and, with the above limitation, such a r e q u i r e m e n t cannot be met. (3) The e v e n t - w a i t mechanism in UNIX V7 does not associate m e m o r y (an event control block) with an event. Under these circumstances, the process waiting for an event t ha t has already happened will wait forever as there is no way to inform the process t hat the event has already occurred. The semaphores and shared m e m o r y features that will be available in system V will r e m e d y this situation. (4) UNIX V7 lacks a general-purpose interprocess c o m m u n i c a t i o n mechanism. Such facilities are particularly i m p o r t a n t for networking applications and with this in mind UNIX system V and Berkeley UNIX 4.2 BSD, b o t h due for release in the middle of 1983, have provisions for interprocess communication.
2. UNIX at CERN In 1981, CERN acquired a UNIX source licence and i nt roduced an experimental UNIX V7 service based on a PDP-11/45. The aim of the service was to evaluate the role that UNIX could play in the areas of software d ev elo p men t and d o c u m e n t preparation. In terms o f hardware the PDP-11/45 has the m a x i m u m 256 kbytes of m e m o r y and several small disc packs which, in total, provide about 30 Mbytes of disc space. Of the 30 Mbytes, about 18 Mbytes are available as user file space, the rest being used for the system and on-line d o c u m e n t a t i o n and manuals. A CAMAC crate is also c o n n e c t e d to the machine to provide the hardware c o n n e c t i o n to the main CERN network, CE RN E T [7]. The current service based on the PDP-11/45 is severely limited both by the 16-bit addressing of PDP-11 machines and by the physical m e m o r y limitation o f 256 kbytes on t he model 45. When, as a result o f the evaluation, it was decided to base future microprocessor support on UNIX systems, we agreed to set up a full-scale service based on a VAX 11/780 system running Berkeley UNIX.
2.1. Connection o f the UNIX PDP-11/45 to C E R N E T C E R N E T is a packet-switching n e t w o r k and is primarily used to link small or medium minicomputers with the central mainframe machines, IBM 3081, Siemens 7880 and CDC 7600. These mainframes offer large file
332 bases as well as access to specialized peripherals such as laser printers and a mass store. User services in C E R N E T are based on a n u m b e r of protocols t h a t are arranged in a series of layers. This " o n i o n skin" approach is similar to t hat proposed in t he open system model of the International Organization for Standardization but, in some cases, the exact functions p e r f o r m e d by the p r o t o c o l layers differ. (1) The line level protocol handles the correct transmission of packets between adjacent nodes in the net w or k. In CERNET these packets are at most 2046 bytes in length. (2) The transport service protocol layer is responsible for ghe establishm e n t and maintenance o f the virtual circuit which is used by any two processes th at wish to c o m m u n i c a t e over CERNET. The transport layer treats such problems as flow control, message d e c o m p o s i t i o n into packets and their eventual reassembly, packet routing etc. I m p l e m e n t a t i o n of the transport layer in an efficient m a nne r involves a considerable software effort and this has some bearing on the m a nne r in which the UNIX PDP-11/45 is c o n n e c t e d to CERNET. (3) The most i m p o r t a n t o f the application layer protocols in CE RN E T is t h e file access p r o t o c o l . This provides r e m o t e file access at the record level and is heavily used f or file transfer and archiving to the IBM central file base. The absence of general interprocess c o m m u n i c a t i o n facilities in UNIX implies th at a full C E R N E T transport service cannot easily be implemented in an efficient manner. Thus, it was decided to c o n n e c t the PDP-11/45 to C E R N E T using the simple file transfer (SFT) approach [8]. The SFT protocol was originally developed for bootstrapping small machines using software in the read-only m e m o r y and t h e r e f o r e it offers only a limited subset o f the C E R N E T file access facilities. However, it has t he great advantage o f not requiring a full transport service in the subscribing machine. Transport layer services are managed by software in the nearest C E R N E T node. SFT does n o t allow file access at t he record level but, instead, the com pl et e file must be transferred to the subscribing machine. In practice, this limitation has not been a serious restriction for the UNIX c o n n e c t i o n to CERNET.
3. I n p u t - o u t p u t under UNIX This section is a brief review o f the UNIX I/O system and its relation to the i m p l e m e n t a t i o n o f t he CAMAC driver. Much of what is m e n t i o n e d here is described in greater detail in two excellent articles in the UNIX Programmers ' Manual [9] and the Bell System Technical Journal [10]. UNIX distinguishes between two categories o f device: " b l o c k " devices and " c h a r a c t e r " devices. Block devices have a relatively com pl ex interface to t he system and are used f or highly structured devices such as discs and tapes, typically devices which are capable of supporting a file system. Access to a block device is always through a layer o f intermediate buffering software which operates using a cache of system buffers. This approach can
333 r e d u c e t h e a m o u n t o f p h y s i c a l I / O carried o u t b y t h e s y s t e m b u t t h e l a t e n c y in t h e final o u t p u t t o t h e device can give rise t o p r o b l e m s in t h e e v e n t o f a s y s t e m crash. C h a r a c t e r devices have a s i m p l e r s y s t e m i n t e r f a c e and are used f o r all o t h e r devices including o u r C A M A C s y s t e m . B o t h b l o c k and c h a r a c t e r devices have an associated " m a j o r " and " m i n o r " device n u m b e r . T h e m a j o r device n u m b e r identifies t h e t y p e o f device and h e n c e t h e device driver whereas t h e m i n o r device n u m b e r is used b y t h e driver to i d e n t i f y a subdevice. As an e x a m p l e , t h e m i n o r n u m b e r will be used to i d e n t i f y a p a r t i c u l a r drive f o r a disc c o n t r o l l e r w h e r e a s , f o r a c o m m u n i c a t i o n device, it is o f t e n used to identi:~y a p a r t i c u l a r virtual circuit.
3.1. U N I X device drivers Device drivers in U N I X f o r m p a r t o f t h e k e r n e l and are c o d e d in t h e language C. Access to t h e device f r o m a p r o c e s s is via o n e o f five i n t e r f a c e r o u t i n e s : O P E N , C L O S E , R E A D , W R I T E , I O C T L . T h e last r o u t i n e I O C T L is used to pass I / O c o n t r o l f u n c t i o n s to t h e driver. In a d d i t i o n to t h e five e n t r y p o i n t s , an i n t e r r u p t h a n d l e r m u s t also be specified. T h e link b e t w e e n t h e U N I X kernel and t h e i n t e r f a c e r o u t i n e s f o r a device driver is a file called t h e c o n f i g u r a t i o n file. This file is a " s w i t c h t a b l e " and consists o f a series o f entries, o r d e r e d b y t h e m a j o r device n u m b e r , which are t h e e n t r y p o i n t s o f t h e five driver r o u t i n e s m e n t i o n e d above.
4. C A M A C h a r d w a r e C A M A C h a r d w a r e o n the C E R N U N I X s y s t e m consists o f a Fisher c o n t r o l s s y s t e m c r a t e w i t h a U n i b u s i n t e r f a c e . Access to t h e c r a t e f r o m s o f t w a r e is via U n i b u s registers. In a d d i t i o n to t w o m o d u l e s f o r C E R N E T usage (the s e n d e r a n d receiver) t h e s y s t e m c r a t e c o n t a i n s an i n t e r r u p t v e c t o r g e n e r a t o r and a l o o k - a t - m e ( L A M ) grader. No d i r e c t m e m o r y access m o d u l e is i n c l u d e d in t h e h a r d w a r e and t h e r e f o r e all d a t a are t r a n s f e r r e d u n d e r program control. With t h e simplified p r o t o c o l used, o n l y C E R N E T receiver L A M s are e n a b l e d . T h e PDP-11 is i n t e r r u p t e d w h e n e v e r a c o n t r o l w o r d or a b l o c k o f i n p u t d a t a has arrived o v e r t h e p h y s i c a l d a t a link.
5. S o f t w a r e f o r C A M A C h a n d l i n g T h e device driver f o r C A M A C in U N I X is w r i t t e n e n t i r e l y in language C and i m p l e m e n t s C A M A C as a c h a r a c t e r device. As the driver p r o v i d e s no possibility f o r t h e q u e u i n g o f I/O r e q u e s t s f r o m p r o c e s s e s , t h e C A M A C can o n l y be used b y o n e p r o c e s s at a t i m e . T h e r e s e r v a t i o n and release o f t h e C A M A C device are assured b y a p r i m i t i v e l o c k i n g m e c h a n i s m , similar to t h a t used w i t h files. In a d d i t i o n , e a c h r e q u e s t f r o m t h e subscribing p r o c e s s is
334 t r e a t e d entirely sequentially. T h e s e limitations apply t o m o s t c h a r a c t e r devices in UNIX b u t b l o c k devices such as t h e disc driver are m o r e sophisticated and can service several processes c o n c u r r e n t l y . T h e i m p l e m e n t a t i o n o f t h e CAMAC s o f t w a r e was d o n e in t w o steps as described in Sections 5.1 and 5.2.
5.1. Implementation o f ESONE CAMAC calls First a minimal subset o f E u r o p e a n S t a n d a r d s o f Nuclear Electronics ( E S O N E ) CAMAC calls [11] was i m p l e m e n t e d . T h e O P E N s y s t e m call assures r e s e r v a t i o n o f t h e CAMAC device and the C L O S E s y s t e m call serves to release the device for use b y a n o t h e r process. In t h e event t h a t a process t e r m i n a t e s a b n o r m a l l y and is unable to issue the C L O S E call, the UNIX system assumes responsibility f o r device and file closure. T h e E S O N E calls are i m p l e m e n t e d using t h e I O C T L system call t o establish c o m m u n i c a t i o n b e t w e e n t h e user process and t h e driver. In t h e subset o f E S O N E calls i m p l e m e n t e d , no b l o c k t r a n s f e r f u n c t i o n s were p r o v i d e d and so it did n o t m a k e sense to use t h e R E A D and W R I T E e n t r y p o i n t s in this first i m p l e m e n t a t i o n o f t h e software. S o m e e x a m p l e s o f t h e c o d e for t h e E S O N E call i m p l e m e n t a t i o n in t h e driver are given in A p p e n d i x A.
5.2. Implementation of the file transfer protocol T h e s e c o n d stage was t h e i m p l e m e n t a t i o n o f the S F T p r o t o c o l . Here a p r i m a r y goal was to f o l l o w t h e UNIX p h i l o s o p h y and to m a k e the r e m o t e file access m e c h a n i s m s as t r a n s p a r e n t as possible f r o m the user's p o i n t o f view. Thus, r e m o t e file access and local file access should be highly similar o p e r a t i o n s in t h e sense t h a t OPEN, R E A D , W R I T E and C L O S E should w o r k for r e m o t e files as f o r local files. A s e c o n d driver was w r i t t e n to i m p l e m e n t these s t a n d a r d system calls. F o r t h e O P E N s y s t e m call it was n o t possible to m a p fully local and r e m o t e s y s t e m calls as O P E N requires an additional p a r a m e t e r f o r r e m o t e file access. In a d d i t i o n , for e f f i c i e n c y , the i m p l e m e n t a t i o n o f t h e R E A D and W R I T E r o u t i n e s in t h e driver was n o t d o n e using E S O N E calls b u t with a tight block transfer loop. If r e m o t e access to a single h o s t file base is restricted, in o u r case an IBM m a c h i n e in t h e C E R N c o m p u t e r c e n t r e , t h e n the m i n i m u m r e q u i r e m e n t in t h e O P E N s y s t e m call is to add a r e m o t e file n a m e in a d d i t i o n to t h e local device name. Thus, a r o u t i n e IBMOPEN was w r i t t e n t h a t accepts a r e m o t e file n a m e as its single i n p u t p a r a m e t e r . This r o u t i n e can t h e n be used to replace t h e standard O P E N system call in a n y utility p r o g r a m s w h e r e r e m o t e file access r a t h e r t h a n local file access is required. Using this idea, a series o f utilities was d e v e l o p e d in the f o r m o f UNIX c o m m a n d s to p e r f o r m the following f u n c t i o n s : (1) file t r a n s f e r b e t w e e n UNIX and t h e IBM file base; (2) t r a n s f e r o f UNIX p r i n t files to t h e IBM laser p r i n t e r ; (3) file archiving o n t h e IBM using t h e UNIX " t a r " (tape archive) c o m m a n d in c o n j u n c t i o n with file t r a n s f e r c o m m a n d s ; (4) i n c r e m e n t a l d u m p i n g o f t h e UNIX file base to t h e IBM; (5) IBM job c o n t r o l and e n q u i r y .
335
6. Experience and conclusions Several lessons were learnt f r o m o u r e x p e r i e n c e o f c o n n e c t i n g C A M A C to a U N I X system. (1) The a d d i t i o n o f a n e w device driver into U N I X is greatly facilitated b y the simple and well-defined interface to the U N I X kernel. (2) The use o f the language C for device drivers allows c o n t r o l structures to be c o d e d in an efficient and y e t readable m a n n e r . This is in c o n t r a s t with m a n y systems w h e r e device drivers m u s t be c o d e d in assembler. (3) The d e b u g c y c l e is q u i t e rapid since rebuilding the U N I X kernel to i n c o r p o r a t e a new version o f a driver takes o n l y a few m i n u t e s . (4) As regards the d e b u g tools themselves, t h e U N I X d e b u g g e r is terse and c r y p t i c b u t , nevertheless, q u i t e a p o w e r f u l tool. (5) The l i m i t a t i o n s o f UNIX in the area o f real-time a p p l i c a t i o n s have n o t been a serious p r o b l e m f o r o u r p a r t i c u l a r a p p l i c a t i o n o f c o n n e c t i n g to C E R N E T via C A M A C . Moreover, it should be n o t e d t h a t several o f t h e U N I X limitations listed in S e c t i o n 1.5 will disappear with release o f s y s t e m V and Berkeley UNIX 4.2 BSD.
Acknowledgments We w o u l d like to t h a n k Peter Villemoes and Kate S p e n c e for their invaluable help and t h e m a j o r c o n t r i b u t i o n t h a t t h e y m a d e in integrating the S F T facilities as c o m m a n d s in the U N I X s y s t e m .
References 1 D. M. Ritchie and K. Thompson, The UNIX time-sharing system, Commun. ACM, 17 (7) (1974) 365 - 375. 2 BellSyst. Tech. J., 57 (6) (1978) 1897 - 2312. 3 S. R. Bourne, The UNIX System, Addison-Wesley, Reading, MA, 1982. 4 B. Kernighan and D. M. Ritchie, The C Programming Language, Prentice-Hall, Englewood Cliffs, NJ, 1978. 5 H. Lycklama and D. L. Bayer, The MERT operating system, Bell Syst. Tech. J., 57 (6) (1978) 2049 - 2086. 6 D. M. Harland, Running a realtime process and a time-shared system (UNIX) concurrently, Software--Pract. Exper., 10 (1980)273- 281. 7 J. M. Gerard (ed.), CERNET -- a high-speed packet-switching network, CERN Yellow Rep. 81-12, 1981 (Organisation Europ6enne pour la Recherche Nucl6aire). 8 D. Brown and C. Piney, Bootstrapping and simplified file transfer, CERNET Project Note 88, 1981~ (Organisation Europ6enne pour la Recherche Nucl6aire). 9 D. Ritchie, The UNIX I/O system, UNIX Programmers' Manual, Vol. 2b, 7th edn., 1979, Section 32. 10 K. Thompson, UNIX implementation, BelISyst. Tech. J., 57 (6) (1978) 1931 - 1946. 11 Subroutines for CAMAC, ESONE Doc. SR/1, 1978 (European Standards of Nuclear Electronics).
336 Appendix A
A. 1. Examples o f C programming in the CAMAC driver The following essential data structures are used in the ESONE call implementation. struct cssab ( int ext; char top; unsigned q: 1 ; unsigned f: 7; int ints;
/*control block used for cssa and cfsa / * t h e coded b, c, n, a combination / * t h e top 8 bits of a 24-bit value /* a 1-bit q value /* a 7-bit function value / * t h e lower 16 bits of a 24-bit value
*/ */ */ */ */ */
/* control block used for cdreg /*result of cdreg: coded b, c, n, a /* branch number /* crate number /* station number /* register number
*/ */ */ */ */ */
/* control block desc Unibus registers
*/
}; struct cdregb { int ext; char b; char c; char n; char a;
}; struct device ( int cmcsr; char cmdbh; char cmnotusd; int cmihr; int cmggl;
}; Other important definitions are as follows. #define #define #define #define #define #define
CAMAC ((struct device*)0166000) FIELD0 0160000 CMBCMASK 077000 CMANMASK 0777 CMWBIT 020 CMTBIT 010
/* defines Unibus address * /
The following example o f ESONE CAMAC call implementations in the driver shows the essentials of routines cmcdreg and cmcssa. cmcdreg(cmp) register struct cdregb *cmp;
{
cmp->ext = ((int)(cmp->b)<<12)l((int)(cmp->c)<<9)]((int){cmp->a) < < 5)l(cmp->n); /* BBBBCCCAAAANNNNN */
337
Cmcdreg is called with the address of an appropriate cont rol block structure (as defined above) as an argument. It t hen picks out the b, c, n and a numbers, converts t h e m to integers and shifts t hem into place. These data are then ORed t oge t he r and the result is stored back in the control block. cmcssa(cmp) register struct cssab *cmp;
/* arg is addr(cssa cont rol block)
register struct device *camp; /* structure to address Unibus regs
*/ */
int *cur; int csr;
/* Unibus address used in read/write/test * / /* local variable to construct csr contents */
camp = CAMAC;
/* set local p o i n t e r to Unibus address(csr)* /
/*set csr f r o m (b, c, f) and set Unibus address from (a, n)
,/
csr = ( ( ( c m p - > e x t ) & C M B C M A S K ) > > l ) ] ( c m p - > f ) ; cur = ( ( ( c m p - > e x t ) & C M A N M A S K ) < < 1)IFIELD0; camp - >cmcs r = csr; if ((cmp->f)&CMTBIT) ( if (*cur); } else { if ((cmp->f)&CMWBIT) *cur = cmp-~ints; else cmp-~ints = *cur; cmp->q = camp->cmcsr<0?l:0;
/* set csr / *t est operat i on / * d o nothing (but set q)
*/ */ */
/* write o p e r a t i o n / * w r i t e 16-bit value /* read o p e r a t i o n /* read 16-bit value
*/ */ */ */
/* if csr< 0 t hen 1 else 0
*/
As an example o f the very concise coding t hat is possible in language C, let us look at the loop t ha t is used to o u t p u t a block o f data to a CAMAC register. Let us assume t hat the block start address is held in variable wd and the word c o u n t in variable w c ount . Also let us assume t hat the Unibus address to be used in the transfer is in variable cur and that variable camp holds a p o i n t e r to the Unibus address o f the CSR. The code to implement block data transfer in Q-stop m o d e t h e n reads as follows. camp - >cm c s r = 16;
do
/*set funct i on value in csr
*/
{ *cur = *wd++ ;
if ( cam p- > c m cs r > = 0) break; } while (--wcount);
/ * w r i t e 16 bit, incr. p o i n t e r */ / * s t o p if q = 0 */ / * d e c wcount, stop if zero */
As can be seen f r o m this example, an efficient C compiler can generate code th at is as efficient as programming in assembler.
329
I N T E R F A C I N G CAMAC TO A UNIX SYSTEM G. LEE a n d D. W I E G A N D T
Organisation Europdenne p o u r la Recherche Nucldaire, CH- 121 l Geneva 23 (Switzerland) (Received July 7, 1 9 8 3 )
Summary In this paper t he interfacing of a CAMAC system to a PDP-11/45 running UNIX version 7 is described. The main emphasis o f the paper is to describe the CAMAC device driver for UNIX and to illustrate the advantages o f programming the driver in the high level language C as compared with programming in assembler. It is assumed that the readership is primarily f r o m a hardware background, and t h e r e f o r e we summarize some of the background and history o f t he UNIX system.
1. The UNIX system and phi l os ophy
1.1. Evolution o f UNIX The first version of the UNIX operating system was developed by T h o m p s o n o f Bell Laboratories for a PDP-7 c o m p u t e r . Later, in c o n j u n c t i o n with Ritchie, a second version was developed for t he PDP-11/20 c o m p u t e r [ 1 - 3 ]. During the 1970s, UNIX for the PDP-11 range evolved through several versions with version 7 (V7) being the most widely used t o d a y and indeed the version th at runs on the PDP-11/45 at the Organisation Europ~enne pour la Rech er ch e Nucl~aire (CERN). The latest version is system III which was released at the end o f 1981 and the n e x t version, system V, is due for release in mid-1983. In addition to the developments for the PDP-11 range, UNIX was implemented on m a n y ot her machines, notably the Digital E q u i p m e n t C o r p o r atio n (DEC) VAX range. 32V and system III are UNIX systems for VAX hardware and are distributed by Bell Laboratories. In a separate d e v e l o p m e n t at the C o m p u t e r Science Division of the University of California at Berkeley, a virtual m e m o r y version o f UNIX for t he VAX was produced. T o d a y , Berkeley UNIX 4.1 BSD is the most widely used UNIX system on VAX hardware. UNIX is now available f or a large range o f machines from m i c r o c o m p u t e r s to mainframe co m put er s and the use and popul ari t y o f UNIX have grown e n o rmo u s ly . F u r t h e r m o r e , with t he advent o f cheap 16-bit m i c r o c o m p u t e r s and work stations, UNIX is rapidly becoming the de facto standard operating system f o r such machines. 0252-7308/83/$3.00
C) Elsevier S e q u o i a / P r i n t e d in T h e N e t h e r l a n d s
330
1.2. S o m e UNIX terminology The kernel of the UNIX system is almost entirely written in the language C, a language developed from BCPL by Kernighan and Ritchie [4]. The kernel itself is relatively small in terms of code and, with simplicity an important design goal, it provides a limited well-defined set of system calls for process and file management. The term UNIX system is often used to describe not only the kernel but also the large number of utilities and application programs which make up the UNIX " t o o l k i t " of software modules. The use of this tool kit is an important aspect of software development under UNIX. Rather than the development of a large number of specialized software packages, the philosophy in UNIX is to take the basic software tool kit and to combine tools in order to perform more elaborate tasks. The UNIX "shell" and " p i p e " mechanisms are instrumental in this combination, and it is through this modular approach to software development that UNIX achieves a large degree of flexibility. 1.3. The U N I X shell The UNIX shell is a process which handles the user interface to the system. In addition to being the c o m m a n d language for issuing and executing commands, it provides a variety of control flow primitives for iteration and branching. As UNIX is an interactive system, the shell normally accepts its input from a terminal, but it can easily be redirected to take its input from a file. Files containing lists of shell commands are called shell scripts and, using the control structures mentioned above, complex programs can be developed. The shell language, although sometimes a little cryptic, is very powerful and is a flexible way of combining the software tools referred to in Section 1.2. 1.4. U N I X pipes The UNIX pipe mechanism allows the o u t p u t of one process to be used as input to another in a manner that is totally transparent to the user. Pipes make use of both shared m e m o r y and temporary files depending on the quantity of data being exchanged and all problems of data flow control in the pipe are handled by the system. Pipes provide the mechanism whereby users can easily chain together simple processes or "filters" to perform a more complex operation. It should be noted, however, that pipes are not a general-purpose interprocess communication facility since they only operate between " r e l a t e d " processes, i.e. processes which share a c o m m o n ancestor. 1.5. U N I X and real-time applications UNIX was developed primarily as a time-sharing system and is not aimed at real-time applications. Although several attempts have been made to build a "real-time UNIX" [5, 6], there are a number of deficiencies in standard UNIX which restrict its applicability in the area of real-time problems.
331
(1) No user-accessible mechanisms exist in standard UNIX to lock a process in m e m o r y so as to guarantee response within a given time limit to external stimuli. In keeping with its time-sharing philosophy, processes compete in an equal m a nne r for m e m o r y resources and are always candidates for swapping. (2) As y n ch r onous i n p u t - o u t p u t (I/O) requests have only recently been introduced in UNIX. In V7, once an l/O request has been issued, the issuing process will wait until the request has been completed. A c o m m o n requirem e n t in real-time applications is to be able to wait on several I/O requests or events and, with the above limitation, such a r e q u i r e m e n t cannot be met. (3) The e v e n t - w a i t mechanism in UNIX V7 does not associate m e m o r y (an event control block) with an event. Under these circumstances, the process waiting for an event t ha t has already happened will wait forever as there is no way to inform the process t hat the event has already occurred. The semaphores and shared m e m o r y features that will be available in system V will r e m e d y this situation. (4) UNIX V7 lacks a general-purpose interprocess c o m m u n i c a t i o n mechanism. Such facilities are particularly i m p o r t a n t for networking applications and with this in mind UNIX system V and Berkeley UNIX 4.2 BSD, b o t h due for release in the middle of 1983, have provisions for interprocess communication.
2. UNIX at CERN In 1981, CERN acquired a UNIX source licence and i nt roduced an experimental UNIX V7 service based on a PDP-11/45. The aim of the service was to evaluate the role that UNIX could play in the areas of software d ev elo p men t and d o c u m e n t preparation. In terms o f hardware the PDP-11/45 has the m a x i m u m 256 kbytes of m e m o r y and several small disc packs which, in total, provide about 30 Mbytes of disc space. Of the 30 Mbytes, about 18 Mbytes are available as user file space, the rest being used for the system and on-line d o c u m e n t a t i o n and manuals. A CAMAC crate is also c o n n e c t e d to the machine to provide the hardware c o n n e c t i o n to the main CERN network, CE RN E T [7]. The current service based on the PDP-11/45 is severely limited both by the 16-bit addressing of PDP-11 machines and by the physical m e m o r y limitation o f 256 kbytes on t he model 45. When, as a result o f the evaluation, it was decided to base future microprocessor support on UNIX systems, we agreed to set up a full-scale service based on a VAX 11/780 system running Berkeley UNIX.
2.1. Connection o f the UNIX PDP-11/45 to C E R N E T C E R N E T is a packet-switching n e t w o r k and is primarily used to link small or medium minicomputers with the central mainframe machines, IBM 3081, Siemens 7880 and CDC 7600. These mainframes offer large file
332 bases as well as access to specialized peripherals such as laser printers and a mass store. User services in C E R N E T are based on a n u m b e r of protocols t h a t are arranged in a series of layers. This " o n i o n skin" approach is similar to t hat proposed in t he open system model of the International Organization for Standardization but, in some cases, the exact functions p e r f o r m e d by the p r o t o c o l layers differ. (1) The line level protocol handles the correct transmission of packets between adjacent nodes in the net w or k. In CERNET these packets are at most 2046 bytes in length. (2) The transport service protocol layer is responsible for ghe establishm e n t and maintenance o f the virtual circuit which is used by any two processes th at wish to c o m m u n i c a t e over CERNET. The transport layer treats such problems as flow control, message d e c o m p o s i t i o n into packets and their eventual reassembly, packet routing etc. I m p l e m e n t a t i o n of the transport layer in an efficient m a nne r involves a considerable software effort and this has some bearing on the m a nne r in which the UNIX PDP-11/45 is c o n n e c t e d to CERNET. (3) The most i m p o r t a n t o f the application layer protocols in CE RN E T is t h e file access p r o t o c o l . This provides r e m o t e file access at the record level and is heavily used f or file transfer and archiving to the IBM central file base. The absence of general interprocess c o m m u n i c a t i o n facilities in UNIX implies th at a full C E R N E T transport service cannot easily be implemented in an efficient manner. Thus, it was decided to c o n n e c t the PDP-11/45 to C E R N E T using the simple file transfer (SFT) approach [8]. The SFT protocol was originally developed for bootstrapping small machines using software in the read-only m e m o r y and t h e r e f o r e it offers only a limited subset o f the C E R N E T file access facilities. However, it has t he great advantage o f not requiring a full transport service in the subscribing machine. Transport layer services are managed by software in the nearest C E R N E T node. SFT does n o t allow file access at t he record level but, instead, the com pl et e file must be transferred to the subscribing machine. In practice, this limitation has not been a serious restriction for the UNIX c o n n e c t i o n to CERNET.
3. I n p u t - o u t p u t under UNIX This section is a brief review o f the UNIX I/O system and its relation to the i m p l e m e n t a t i o n o f t he CAMAC driver. Much of what is m e n t i o n e d here is described in greater detail in two excellent articles in the UNIX Programmers ' Manual [9] and the Bell System Technical Journal [10]. UNIX distinguishes between two categories o f device: " b l o c k " devices and " c h a r a c t e r " devices. Block devices have a relatively com pl ex interface to t he system and are used f or highly structured devices such as discs and tapes, typically devices which are capable of supporting a file system. Access to a block device is always through a layer o f intermediate buffering software which operates using a cache of system buffers. This approach can
333 r e d u c e t h e a m o u n t o f p h y s i c a l I / O carried o u t b y t h e s y s t e m b u t t h e l a t e n c y in t h e final o u t p u t t o t h e device can give rise t o p r o b l e m s in t h e e v e n t o f a s y s t e m crash. C h a r a c t e r devices have a s i m p l e r s y s t e m i n t e r f a c e and are used f o r all o t h e r devices including o u r C A M A C s y s t e m . B o t h b l o c k and c h a r a c t e r devices have an associated " m a j o r " and " m i n o r " device n u m b e r . T h e m a j o r device n u m b e r identifies t h e t y p e o f device and h e n c e t h e device driver whereas t h e m i n o r device n u m b e r is used b y t h e driver to i d e n t i f y a subdevice. As an e x a m p l e , t h e m i n o r n u m b e r will be used to i d e n t i f y a p a r t i c u l a r drive f o r a disc c o n t r o l l e r w h e r e a s , f o r a c o m m u n i c a t i o n device, it is o f t e n used to identi:~y a p a r t i c u l a r virtual circuit.
3.1. U N I X device drivers Device drivers in U N I X f o r m p a r t o f t h e k e r n e l and are c o d e d in t h e language C. Access to t h e device f r o m a p r o c e s s is via o n e o f five i n t e r f a c e r o u t i n e s : O P E N , C L O S E , R E A D , W R I T E , I O C T L . T h e last r o u t i n e I O C T L is used to pass I / O c o n t r o l f u n c t i o n s to t h e driver. In a d d i t i o n to t h e five e n t r y p o i n t s , an i n t e r r u p t h a n d l e r m u s t also be specified. T h e link b e t w e e n t h e U N I X kernel and t h e i n t e r f a c e r o u t i n e s f o r a device driver is a file called t h e c o n f i g u r a t i o n file. This file is a " s w i t c h t a b l e " and consists o f a series o f entries, o r d e r e d b y t h e m a j o r device n u m b e r , which are t h e e n t r y p o i n t s o f t h e five driver r o u t i n e s m e n t i o n e d above.
4. C A M A C h a r d w a r e C A M A C h a r d w a r e o n the C E R N U N I X s y s t e m consists o f a Fisher c o n t r o l s s y s t e m c r a t e w i t h a U n i b u s i n t e r f a c e . Access to t h e c r a t e f r o m s o f t w a r e is via U n i b u s registers. In a d d i t i o n to t w o m o d u l e s f o r C E R N E T usage (the s e n d e r a n d receiver) t h e s y s t e m c r a t e c o n t a i n s an i n t e r r u p t v e c t o r g e n e r a t o r and a l o o k - a t - m e ( L A M ) grader. No d i r e c t m e m o r y access m o d u l e is i n c l u d e d in t h e h a r d w a r e and t h e r e f o r e all d a t a are t r a n s f e r r e d u n d e r program control. With t h e simplified p r o t o c o l used, o n l y C E R N E T receiver L A M s are e n a b l e d . T h e PDP-11 is i n t e r r u p t e d w h e n e v e r a c o n t r o l w o r d or a b l o c k o f i n p u t d a t a has arrived o v e r t h e p h y s i c a l d a t a link.
5. S o f t w a r e f o r C A M A C h a n d l i n g T h e device driver f o r C A M A C in U N I X is w r i t t e n e n t i r e l y in language C and i m p l e m e n t s C A M A C as a c h a r a c t e r device. As the driver p r o v i d e s no possibility f o r t h e q u e u i n g o f I/O r e q u e s t s f r o m p r o c e s s e s , t h e C A M A C can o n l y be used b y o n e p r o c e s s at a t i m e . T h e r e s e r v a t i o n and release o f t h e C A M A C device are assured b y a p r i m i t i v e l o c k i n g m e c h a n i s m , similar to t h a t used w i t h files. In a d d i t i o n , e a c h r e q u e s t f r o m t h e subscribing p r o c e s s is
334 t r e a t e d entirely sequentially. T h e s e limitations apply t o m o s t c h a r a c t e r devices in UNIX b u t b l o c k devices such as t h e disc driver are m o r e sophisticated and can service several processes c o n c u r r e n t l y . T h e i m p l e m e n t a t i o n o f t h e CAMAC s o f t w a r e was d o n e in t w o steps as described in Sections 5.1 and 5.2.
5.1. Implementation o f ESONE CAMAC calls First a minimal subset o f E u r o p e a n S t a n d a r d s o f Nuclear Electronics ( E S O N E ) CAMAC calls [11] was i m p l e m e n t e d . T h e O P E N s y s t e m call assures r e s e r v a t i o n o f t h e CAMAC device and the C L O S E s y s t e m call serves to release the device for use b y a n o t h e r process. In t h e event t h a t a process t e r m i n a t e s a b n o r m a l l y and is unable to issue the C L O S E call, the UNIX system assumes responsibility f o r device and file closure. T h e E S O N E calls are i m p l e m e n t e d using t h e I O C T L system call t o establish c o m m u n i c a t i o n b e t w e e n t h e user process and t h e driver. In t h e subset o f E S O N E calls i m p l e m e n t e d , no b l o c k t r a n s f e r f u n c t i o n s were p r o v i d e d and so it did n o t m a k e sense to use t h e R E A D and W R I T E e n t r y p o i n t s in this first i m p l e m e n t a t i o n o f t h e software. S o m e e x a m p l e s o f t h e c o d e for t h e E S O N E call i m p l e m e n t a t i o n in t h e driver are given in A p p e n d i x A.
5.2. Implementation of the file transfer protocol T h e s e c o n d stage was t h e i m p l e m e n t a t i o n o f the S F T p r o t o c o l . Here a p r i m a r y goal was to f o l l o w t h e UNIX p h i l o s o p h y and to m a k e the r e m o t e file access m e c h a n i s m s as t r a n s p a r e n t as possible f r o m the user's p o i n t o f view. Thus, r e m o t e file access and local file access should be highly similar o p e r a t i o n s in t h e sense t h a t OPEN, R E A D , W R I T E and C L O S E should w o r k for r e m o t e files as f o r local files. A s e c o n d driver was w r i t t e n to i m p l e m e n t these s t a n d a r d system calls. F o r t h e O P E N s y s t e m call it was n o t possible to m a p fully local and r e m o t e s y s t e m calls as O P E N requires an additional p a r a m e t e r f o r r e m o t e file access. In a d d i t i o n , for e f f i c i e n c y , the i m p l e m e n t a t i o n o f t h e R E A D and W R I T E r o u t i n e s in t h e driver was n o t d o n e using E S O N E calls b u t with a tight block transfer loop. If r e m o t e access to a single h o s t file base is restricted, in o u r case an IBM m a c h i n e in t h e C E R N c o m p u t e r c e n t r e , t h e n the m i n i m u m r e q u i r e m e n t in t h e O P E N s y s t e m call is to add a r e m o t e file n a m e in a d d i t i o n to t h e local device name. Thus, a r o u t i n e IBMOPEN was w r i t t e n t h a t accepts a r e m o t e file n a m e as its single i n p u t p a r a m e t e r . This r o u t i n e can t h e n be used to replace t h e standard O P E N system call in a n y utility p r o g r a m s w h e r e r e m o t e file access r a t h e r t h a n local file access is required. Using this idea, a series o f utilities was d e v e l o p e d in the f o r m o f UNIX c o m m a n d s to p e r f o r m the following f u n c t i o n s : (1) file t r a n s f e r b e t w e e n UNIX and t h e IBM file base; (2) t r a n s f e r o f UNIX p r i n t files to t h e IBM laser p r i n t e r ; (3) file archiving o n t h e IBM using t h e UNIX " t a r " (tape archive) c o m m a n d in c o n j u n c t i o n with file t r a n s f e r c o m m a n d s ; (4) i n c r e m e n t a l d u m p i n g o f t h e UNIX file base to t h e IBM; (5) IBM job c o n t r o l and e n q u i r y .
335
6. Experience and conclusions Several lessons were learnt f r o m o u r e x p e r i e n c e o f c o n n e c t i n g C A M A C to a U N I X system. (1) The a d d i t i o n o f a n e w device driver into U N I X is greatly facilitated b y the simple and well-defined interface to the U N I X kernel. (2) The use o f the language C for device drivers allows c o n t r o l structures to be c o d e d in an efficient and y e t readable m a n n e r . This is in c o n t r a s t with m a n y systems w h e r e device drivers m u s t be c o d e d in assembler. (3) The d e b u g c y c l e is q u i t e rapid since rebuilding the U N I X kernel to i n c o r p o r a t e a new version o f a driver takes o n l y a few m i n u t e s . (4) As regards the d e b u g tools themselves, t h e U N I X d e b u g g e r is terse and c r y p t i c b u t , nevertheless, q u i t e a p o w e r f u l tool. (5) The l i m i t a t i o n s o f UNIX in the area o f real-time a p p l i c a t i o n s have n o t been a serious p r o b l e m f o r o u r p a r t i c u l a r a p p l i c a t i o n o f c o n n e c t i n g to C E R N E T via C A M A C . Moreover, it should be n o t e d t h a t several o f t h e U N I X limitations listed in S e c t i o n 1.5 will disappear with release o f s y s t e m V and Berkeley UNIX 4.2 BSD.
Acknowledgments We w o u l d like to t h a n k Peter Villemoes and Kate S p e n c e for their invaluable help and t h e m a j o r c o n t r i b u t i o n t h a t t h e y m a d e in integrating the S F T facilities as c o m m a n d s in the U N I X s y s t e m .
References 1 D. M. Ritchie and K. Thompson, The UNIX time-sharing system, Commun. ACM, 17 (7) (1974) 365 - 375. 2 BellSyst. Tech. J., 57 (6) (1978) 1897 - 2312. 3 S. R. Bourne, The UNIX System, Addison-Wesley, Reading, MA, 1982. 4 B. Kernighan and D. M. Ritchie, The C Programming Language, Prentice-Hall, Englewood Cliffs, NJ, 1978. 5 H. Lycklama and D. L. Bayer, The MERT operating system, Bell Syst. Tech. J., 57 (6) (1978) 2049 - 2086. 6 D. M. Harland, Running a realtime process and a time-shared system (UNIX) concurrently, Software--Pract. Exper., 10 (1980)273- 281. 7 J. M. Gerard (ed.), CERNET -- a high-speed packet-switching network, CERN Yellow Rep. 81-12, 1981 (Organisation Europ6enne pour la Recherche Nucl6aire). 8 D. Brown and C. Piney, Bootstrapping and simplified file transfer, CERNET Project Note 88, 1981~ (Organisation Europ6enne pour la Recherche Nucl6aire). 9 D. Ritchie, The UNIX I/O system, UNIX Programmers' Manual, Vol. 2b, 7th edn., 1979, Section 32. 10 K. Thompson, UNIX implementation, BelISyst. Tech. J., 57 (6) (1978) 1931 - 1946. 11 Subroutines for CAMAC, ESONE Doc. SR/1, 1978 (European Standards of Nuclear Electronics).
336 Appendix A
A. 1. Examples o f C programming in the CAMAC driver The following essential data structures are used in the ESONE call implementation. struct cssab ( int ext; char top; unsigned q: 1 ; unsigned f: 7; int ints;
/*control block used for cssa and cfsa / * t h e coded b, c, n, a combination / * t h e top 8 bits of a 24-bit value /* a 1-bit q value /* a 7-bit function value / * t h e lower 16 bits of a 24-bit value
*/ */ */ */ */ */
/* control block used for cdreg /*result of cdreg: coded b, c, n, a /* branch number /* crate number /* station number /* register number
*/ */ */ */ */ */
/* control block desc Unibus registers
*/
}; struct cdregb { int ext; char b; char c; char n; char a;
}; struct device ( int cmcsr; char cmdbh; char cmnotusd; int cmihr; int cmggl;
}; Other important definitions are as follows. #define #define #define #define #define #define
CAMAC ((struct device*)0166000) FIELD0 0160000 CMBCMASK 077000 CMANMASK 0777 CMWBIT 020 CMTBIT 010
/* defines Unibus address * /
The following example o f ESONE CAMAC call implementations in the driver shows the essentials of routines cmcdreg and cmcssa. cmcdreg(cmp) register struct cdregb *cmp;
{
cmp->ext = ((int)(cmp->b)<<12)l((int)(cmp->c)<<9)]((int){cmp->a) < < 5)l(cmp->n); /* BBBBCCCAAAANNNNN */
337
Cmcdreg is called with the address of an appropriate cont rol block structure (as defined above) as an argument. It t hen picks out the b, c, n and a numbers, converts t h e m to integers and shifts t hem into place. These data are then ORed t oge t he r and the result is stored back in the control block. cmcssa(cmp) register struct cssab *cmp;
/* arg is addr(cssa cont rol block)
register struct device *camp; /* structure to address Unibus regs
*/ */
int *cur; int csr;
/* Unibus address used in read/write/test * / /* local variable to construct csr contents */
camp = CAMAC;
/* set local p o i n t e r to Unibus address(csr)* /
/*set csr f r o m (b, c, f) and set Unibus address from (a, n)
,/
csr = ( ( ( c m p - > e x t ) & C M B C M A S K ) > > l ) ] ( c m p - > f ) ; cur = ( ( ( c m p - > e x t ) & C M A N M A S K ) < < 1)IFIELD0; camp - >cmcs r = csr; if ((cmp->f)&CMTBIT) ( if (*cur); } else { if ((cmp->f)&CMWBIT) *cur = cmp-~ints; else cmp-~ints = *cur; cmp->q = camp->cmcsr<0?l:0;
/* set csr / *t est operat i on / * d o nothing (but set q)
*/ */ */
/* write o p e r a t i o n / * w r i t e 16-bit value /* read o p e r a t i o n /* read 16-bit value
*/ */ */ */
/* if csr< 0 t hen 1 else 0
*/
As an example o f the very concise coding t hat is possible in language C, let us look at the loop t ha t is used to o u t p u t a block o f data to a CAMAC register. Let us assume t hat the block start address is held in variable wd and the word c o u n t in variable w c ount . Also let us assume t hat the Unibus address to be used in the transfer is in variable cur and that variable camp holds a p o i n t e r to the Unibus address o f the CSR. The code to implement block data transfer in Q-stop m o d e t h e n reads as follows. camp - >cm c s r = 16;
do
/*set funct i on value in csr
*/
{ *cur = *wd++ ;
if ( cam p- > c m cs r > = 0) break; } while (--wcount);
/ * w r i t e 16 bit, incr. p o i n t e r */ / * s t o p if q = 0 */ / * d e c wcount, stop if zero */
As can be seen f r o m this example, an efficient C compiler can generate code th at is as efficient as programming in assembler.