- Email: [email protected]
Low-cost distributed realtime multitasking system
Low-cost distributed realtime multitasking system Realtime distributed multitasking implies rigorous performance constraints within a system, but how can such a system be implemented cheaply? Kam Wing Ng describes a network of 68000-based nodes which will be useful in embedded realtime applications
The paper describes the design and implementation of DRM68K, a low-cost distributed realtime multitasking system. Each node in DRM68K consists of a minimally configured 68000 microprocessor and a network interface unit that connects the n o d e s together to form a token ring local area network. The system is built on top of a distributed multitasking kemel. DRM68K is useful in realtime e m b e d d e d computing applications such as robotics, process control and flight simulation. microsystems local area networks reallime systems
multitasking
68000
This paper describes the DRM68K project, the aim of which is the design and implementation of a low-cost distributed realtime multitasking system. Before tackling the hardware and software details of DRM68K, the term 'distributed realtime multitasking system' should first be defined in its proper context. Peterson and Silberschatz 1 define that: 'A realtime system is often used as a control device in a dedicated application. Sensors bring data to the computer. The computer must analyse the data and possibly adjust controls to modify the sensor inputs . . . (with) well defined constraints. Processing must be done within the defined constraints or the system will fail.' 'Multitasking' refers to the ability of an operating system to effectively manage several tasks, from one computer program, that are operating simultaneously2. Thus 'realtime' and 'multitasking' are attributes and performance constraints imposed on the hardware and software of a computer system. Realtime embedded computing applications such as robotics, process control, flight simulation etc. require this category of realtime multitasking systems. In applications of this type, the overall processing function is often distributed naturally over a localized area. In robotics, for example, the processing is distributed through equipment distributed around the plant or site. A logical evolution of Department of ComputerScience,ChineseUniversityof Hong Kong, Shatin, NT, Hong Kong Paperreceived: 10 July 1987. Revised:5 July 1988
this physical distribution of processing functions is to implement each function by means of a separate computing unit or node. The nodes which make up the system are then linked together via a local area network (LAN) 3 to perform the overall system processing function. DRM68K is a low-cost implementation of this system configu ration.
HARDWARE DETAILS The block diagram of a standard node in the DRM68K system is shown in Figure 1, which depicts the system configuration. The low cost requirement demands that each node must fit onto a single board, and a minimally configured CPU that satisfies most of the needs of the target applications must be designed. Therefore each node has the following specification • 8 kbyte of static RAM, expandable on board to 32 kbyte • 16 kbyte of EPROM, expandable on board to 64 kbyte • two RS232C ports • 16 parallel I/O lines • three 16-bit timers.
CPU configuration The 16/32-bit MC68000 microprocessor4 was chosen as the DRM68K CPU because of its high performance and the availability of inexpensive compatible 8-bit peripheral components. Each node is a complete microcomputer system (Figure 2) built around an 8 M H z processor designed with low cost and minimal parts count in mind. Examples of 68000-based system design are abundant in the literature (see, for example, References 5-8); therefore only the salient points are described here. Moreover, Figure 2 shows the logic functions required in each node in gate form which in practice can be implemented using programmable logic devices. The minimal configuration shown here can also be expanded quite easily to suit various application requirements.
0141-9331/88/09519-08 $03.00 © 1988 Butterworth & Co. (Publishers) Ltd Vol 12 No 9 November 1988
519
Computing station (node)
cP0
i RAM
I P.oM
I 'mer I
I
I
I
1
I
Parallel I/O [
I
[ eria, ,O I
1
I interface e'w°r unit
II... ]
Figure 1.
I
~ r J i g e n ! ~
DRM68K system configuration
Figure 3 shows the system memory map. Address decoding is performed by a three-to-eight-line decoder, IC1 (74LS138). The EPROM and RAM outputs are further gated with the upper and lower data strobe signals, UDS* and LDS*, by IC2 (74LS32) for defining upper- and lowerbyte PROMs and RAMs. The three enable inputs of IC1 also qualify the output strobes. Address strobe AS* allows an output to be enabled only when a valid address appears on the address bus. To ensure that the outputs are selected only when the data bus carries valid data, the two data strobes are combined in IC3. Since DRM68K uses synchronous peripheral devices of the M6800 family, no acknowledge signal is required. However, to satisfy the requirements of asynchronous bus transfers, an acknowledge signal (DTACK*) must be sent by the memory or peripheral back to the processor to inform it that the transfer is complete. If necessary, the processor inserts wait states into the cycle until it receives the acknowledgement. Peripheral devices in the M68000 family have DTACK open-drain outputs which are directly connected to the processor's DTACK input along with a pullup resistor. However, PROMs and RAMs do not have such an output so an equivalent signal must be created. On more expensive systems DTACK is normally generated by either a multitap delay line or an active delaying device such as a shift register driven by the processor clock to produce a delayed chip select signal for the DTACK* input. There are often different circuits for each type of device, and the delay is selectable so it may be set to the optimum required for each type of device in the system. To keep thngs simple in the DRM68K, the PROM and RAM select outputs of IC1 are combined in IC4 and then inverted by the open-collector inverter IC5, pulling DTACK* low if either select output goes low. Memory capacity consists of two PROM and two RAM sockets. Two of each byte-wide memory chip are required to give a 16-bit data width. Link la allows EPROMs with standard pin configurations from the 2732-256 family to be fitted, providing from 8 to 64 kbyte. RAM sockets currently accept either 2 or 8 kbyte static RAMs through link l b (i.e. either 6116 or 6264 devices) to give either 4 or 16 kbyte. The PCB for each node is laid out to accept 16 and 32 kbyte devices, which will give up to 64 kbyte of RAM. The peripheral devices used in the DRM68K are synchronous. When the AS* line is asserted (low), the synchronous peripherals must pull the VPA* processor pin low to generate the synchronous timing signals VMA* and
520
E. In the DRM68K, the peripheral devices that use synchronous transfers are the parallel adapter (PIA, IC6), the programmable timer module (PTM, IC7) and the asynchronous communications interface adaptors (ACIAs, IC8 and IC9). Address line A23 is tied to VPA* through IC10 and AS. When either of the devices is selected, A23 must be high, which pulls VPA* low and initiates the synchronous data transfer. After the transfer is complete, the processor automatically resumes asynchronous operation. This scheme limits addresses using A23 to synchronous devices only (Figure 3). The basis of the DRM68K serial communication facility is the MC6850 ACIA. Data rates are switch selectable to generate the clock rate required by the LAN. DRM68K uses the autovector mode for interrupts. When the CPU responds to an interrupt, all address bus lines except A1-A3 are high (because these address bits are set equal to the priority of interrupt being serviced). Since A23 is connected to VPA through IC10, the CPU is forced into the autovector mode for all interrupts. Interrupts are signalled by lines IPL0*-IPL2* via a priority encoder ICl 1. Inputs of IC11 are connected to interrupt signals through link 3, which is provided to vary manually the levels of interrupts required by the different devices used in various applications. An interrupt acknowledge signal lACK* is provided by CPU status lines FC0-FC2. When processing an interrupt the CPU places a unique code of all '1 's on status lines FC0-FC2; this code is used to generate lACK* via IC12. The lACK* signal disables memory access via IC1 during an interrupt.
N e t w o r k interface unit
Before describing the network interface unit (NIU), the LAN characteristics must be specified. In addition, ideal features of the LAN are low cost, high bandwidth, good reliability, low error rate, minimal dependence upon centralized subsystems for control, fair access by all devices, ease of configuration and maintenance. Basic techniques 3, 9 used with LANs are concerned with • network topology • transmission medium • network (medium) access control methods. A ring topology 3 is most natural for distributed realtime applications with a typical data transmission rate of 500 kbit s-1. Two types of mechanism to control access to the transmission medium have been adopted by the IEEE 802 Standards Committee: CSMA/CD (used solely with bus networks) and token passing access 1°. Performance evaluations of CSMA/CD 11' 12 have indicated that this system has a poor utilization characteristic under high load. Token ring systems are deterministic under all loads, and perform best under high load. Thus a token ring, message-passing network has been adopted for DRM68K. A shielded twisted pair was chosen as the transmission medium because it is inexpensive, and because it is straightforward to make a physical connection to the cable. A block diagram of the NIU is shown in Figure 4. The hardware may be broken up into two main areas, as follows. • Direct memory access. A 6800 family chip, the MC6844 DMA controller (DMAC), is used to perform DMA. Its
Microprocessors and Microsystems
0 0
~.
~.
1:,
~
+
+
~.
~-o
a
~
•
.
/T~iiir~
J
.._,..---,
il~li iI
79779777 I I I 11
ililil]l
i~ ~i~ ~~
~>
I I !
00o0
I 0
E
u~ cO %0
~a ~6
..Q
~D
E i-
i
.
!,b ~,I~ L.) =-_
:5
S$S
~
~
rq
E
Vol 12 No 9 November 7988
U~
521
Unused
FFFFFFH DOOOOOH
PTM
C0000FH C00000H
P[A
B00007H B00000H
ACIAs
A00003H A000OOH
Unused DIPSW DMAC AD LC
800000H 700000H 600000H 500000H
Unused 1FFFFFH
t 100FFFH-6116 103FFFH-6264 RAM Reservedfor monitor
10FFFFH-8832
100400H
100000H
001FFFH-2732 003FFF H-2764
PROM
007FFFH-27128 00FFFFH-27256
000000H
Figure 3.
System memory map
function is to transmit data (messages to be sent via the LAN) to the MC6854 ADLC (advanced data link controller) and to receive data (messages coming in from the LAN) from the ADLC without involving the CPU directly. To minimize the complexity of the hardware and reduce the required IC count for each node, all incoming data is stored in lower memory and all outgoing data is transmitted from lower memory. The DMAC must be programmed by the CPU before it can respond to a receive signal or transmit signal command. This is done using the asynchronous interface and associated decoding circuits. • Ring controller. This scheme also uses a 6800 family chip to interface to the LAN. The ADLC is addressed by the CPU via the address decoder, asynchronous interface and A1, A2 and Tristate R/W lines. The ADLC needs to have its registers programmed before it can be configured on the ring.
synchronizes 68000 accesses to the DMAC (and ADLC) with the E clock TM. Initially, flipflops ICl 5A and IC15B are cleared. This situation forces the IC15B DTACK* output high, puts latch 16 in the high impedance state and enables latch 17. At the start of a DMAC (or ADLC) write cycle latch 17 remains enabled, passing data from the 68000 system bus to the DMAC (or ADLC) as soon as LATCH__ENABLE* is asserted. During a read cycle, the signals Tristate R/W and START__TRANSFERare asserted high. Then NAND gate 18 forces latch 17 to the high impedance state and enables latch 16. When the signal LATCH__ENABLE* is asserted the data is latched over from the DMAC or ADLC bus. START TRANSFER is asserted when either the ADLC or DMAC are addressed, but the DMAC or ADLC are selected by the assertion of DMAS*. This signal is further decoded to chip select either the DMAC or ADLC depending on the outputs of the address decoder (IC1). Flipflop 15A is clocked high on the first falling edge of E after START TRANSFERand LDS* are asserted. The Q output of 15A is applied to NAND gate 19, asserting DMAS*. Chip selecting the DMAC and ADLC in this manner ensures adequate time for setting up a valid address, and is the principal reason for incorporating the asynchronous interface. On the falling edge of E, the Q* output of flipflop 15B is clocked low, asserting DTACK* and latching data into the enabled latch. The asserted DTACK* (open collector) is connected to Q* of flipflop 15B and sends the required DTACK* back to the CPU to satisfy CPU requirements. This output negates DMAS* so that it goes back to a high output. This results in deselecting the ADLC or DMAC. Once the CPU receives the DTACK* signal it terminates the access by negating LDS*, clearing flipflops 15A and 15B (by placing a low-level voltage on the CLR input). This initializes the interface for the next access. The DTACK* signal is sent through an open collector buffer so that other devices (such as PROM or RAM) may use DTACK* when the ADLC or DMAC are not being accessed. The above description is equally valid for a read or a write operation.
Transmitting onto the I_AN The DMAC has four DMA channels which can be configured independently by software using 15 addressable registers. The NIU only uses two of the channels; channel 1 and channel 0. Channel 0 is used for receiving data from the ring; channel 1 is used for transmitting data onto the ring. Using the control registers the ADLC may be ~RS0,
I CPUcircuit
RS1and/
R/w
g..... ,or
The following subsections describe the logic of the asynchronous interface and analyse the bus arbitration and decoding logic (RS0, RSI and Tristate R/W) for transmitting and receiving from the LAN. Further details can be found in the data sheets of the AD LC and DMAC 13.
-,JAsyoc. ......
'----'1 interface RAM and decodng Io9c l ~ ' /
"--'1
Bus a,bitrotio°
K--~I
II
I
Asynchronous
IllIll
ADLC
~--' ~
DMA
I"-----1
..... o,,er
Asynchronous interface The circuit for the asynchronous interface consists mainly of IC15, IC16, IC17 and IC18 (Figure 2). This circuit
522
Systembus
Figure 4.
} I
q
Network interface unit
MiCroprocessors and.Microsystems
programmed to operate in various modes. The ADLC can be programmed to on-loop mode or point-to-point mode (off-loop mode). When the NIU software programs the ADLC to on-loop mode, the NIU software is only listening to the messages that are being sent by the present master of the ring (the node which holds the token is considered the master). When the NIU software programs the ADLC to off-loop mode, the NIU software will send messages onto the loop and look for the message that was sent to return. When the ADLC is in on-loop mode it stores the incoming byte into its receiver FIFO register and adds a single-bit delay to the byte before retransmitting that data onto the next node. Bus arbitration When the ADLC asserts TXRQ1 it leads to DRQ2* being asserted by the DMAC. The assertion of TXRQ1 results in BR* being asserted. BR* is an open-collector output so that other devices may bid for the system bus. At this stage DCRNT, BGACK* and TxSTB* are all negated. The CPU will relinquish control of the system bus and assert BG*. All outputs of the 68000 will go to a high impedance state. The 68000 is designed to assume itself the lowest-priority IC in the hardware system. The 68000 will negate the signals BCACK*, AS* and DTACK*. DMA may now proceed.The above conditions lead to output Y3* of IC22 becoming asserted. Since the output of IC23A is asserted high (because DRQ2* is asserted), this results in NAND flipflop IC23 asserting DCRNT and BGACK*. BCACK is passed through an open-collector buffer to allow other system devices to assert this signal. Asserting BGACK* negates BG*, BR* and output Y3* of IC22. However, DGRNT will still be asserted. From the timing diagram (Figure 5) the DMAC does not negate DRQ2* until after the DMA transfer has started. Well before DRQ2* is negated, TxSTB* becomes asserted. DGRNT is negated when both DRQ2* and TxSTB* are negated. Since TxSTB* is only negated after a DMA transfer has occurred, DG RNT is negated after DMA is finished. Thus DGRNT may be used to set up conditions for the DMA transfer. TxSTB* is used to enable the ADLC during a DMA transfer so that TxTRQO and TxRQ1 do not return to the negated state until after the data is latched in the appropriate direction. The DMAC now has control over the system address bus and system R/W. The DMAC asserts the system R/W
///
\ \
\,
E clock TxRQ
\\
/-
/ \
DRQ2* \\ /
DGRANT TxSTB* MEM SELECT*
"4
R/W
(((
)))
p0-D7
~ A S *
~/~
/ ~
LDS*
/
ADLC CS* LATCH_ENABLE*
[This pulse latchesdata ' acrossto the ADLC Tx register
Figure 5. LAN
Timing diagram for DMA transmission onto the
Vol "12 No 9 November 1988
into a high state (R/W = 1). This is used to read data from the RAM. It is also used as an input to Tristate R/W along with DGRNT. Since DGRNT is asserted, Tristate R/W will be asserted low, to write data into the ADLC and allow data to be passed in the correct direction through the asynchronous interface. Under these conditions latch 17 is enabled. When TxSTB* is asserted the data is latched across to the ADLC/DMAC asynchronous data bus. It is important that the data is ready well before TxSTB* is asserted. For this reason the chip select logic for the RAM is designed to enable the RAM as soon as DGRNT is asserted. The signal DGRNT is also used to assert AS* and LDS*. The address decoder uses system address lines A21, A22 and A23 to select the particular ICs being accessed. The address for RAM is A21 = 1, A22 = O and A23 = 1. During DMA, RAM is addressed by using DGRNT to enable/ disable IC24. Decoding logic Figure 2 shows the decoding logic used to generate Tristate R/W. When no DMA is in progress the output of AN D gate 26A is asserted low and that of AN D gate 26B is the same as the system R/W. This allows the CPU to read and write to the ADLC normally. The primary task of this circuit is to allow the DMAC to write to the ADLC while reading data from RAM and vice versa. For this purpose, when DGRNT is asserted, the system IVW must be inverted by the logic shown in Figure 2. During a DMA transfer (either transmission or reception) DCRNT is asserted. This disables AND gate 26B and enables AND gate 26A. AND gate 26A passes an inverted system R/W. Then the output of OR gate 27A becomes R/W. If the DMAC is transmitting data then R/W is asserted high and Tristate R/W is asserted low. If the DMAC is receiving data then R/W is asserted low and the Tristate R/W is asserted high. This is the r(~quired output duringa DMA cycle and is the reason for the symbolic name Tristate R/W. The decoding logic used to generate the register select values RS0 and RS1 are designed so that RS0 = A1 and RS1 = A2 while DMA is not in progress. When DMA occurs the circuit will set RS0 = 0 (asserted low) and RS1 = 1 (asserted high). For transmission onto the loop, Tristate R/W will be asserted low. This will select the transmit FIFO frame continue register. When the last byte of data is to be transmitted by the DMAC the signal DEND* becomes asserted setting RS0 (asserted high) and RS1 (asserted high) to 1. The address control bit must be set to zero at initialization so that the transmit FIFO frame terminate register is selected when DEND* is asserted. The ADLC will then send the last byte, the frame check sequence (FCS) and the closing flag. The ADLC will not assert TDSR until it is ready to transmit again. When the CPU is reading RAM the RAML* decoding logic in Figure 2 is designed to pass the signal MEM~SELECT* through unchanged. When the DMAC is transmitting data onto the loop it must first read the data from RAM. To avoid the possibility of the asynchronous interface latching over data that is not correct (due to insufficient access time), the RAM is selected as soon as MEM~SELECT* and DGRNT* become asserted. In this way, when the signal TxSTB* becomes asserted and latches the data across to the ADLC, the RAM will have been enabled for more than its access time. Thus data on the system bus will be uncorrupted. The complexity of
523
the RAM select logic would not be necessary if the timing considerations for a read and write were the same.
E clock
///
} \\\\~k\
TxRQ0 DRQ2*
Receiving from the LAN _
When the ADLC receives data from the LAN it stores it in the receiver FIFO stack and asserts TxRQ0 to tell the DMAC that it requires service of its read FIFO register. The ADLC also asserts its interrupt telling the CPU that data or a message has been received by the LAN. The CPU may then accept the interrupt and reprogram the interrupt so that it indicates the end of the message. In this way the CPU may reprogram the ADLC and DMAC before another message is received. Figure 6 shows the timing diagram for DMA reception from the ring.
_
DGRANT TxSTB*
kx
R/W MEM_SELECT
/7/
\
DO-D7 (system) AS*
/'/,/
\
LDS* A D L C CS* LATCH_ENABLE*
Bus arbitration When the ADLC asserts TxRQ0 (the receiver channel) it will also lead to DRQ2* being asserted by the DMAC and the assertion of BR*. In all other respects the operation of bus arbitration is identical for reception and transmission.
This pulse latches data across to memory from the o A D L C Rx register
Figure 6. LAN
Timing diagram for DMA reception from the
Decoding logic The operation of Tristate R/W for reception is identical to that for transmission except that Tristate R/W is asserted high during reception. From Figure 6 it can be seen that when TxSTB* becomes asserted the RAM is still in the setup stage. It will not be ready to have data written to it for about 270 ns (access time) after the E clock has fallen. For this purpose the RAM is chip selected on the falling edge of the TxSTB* signal. It remains enabled until the rising edge of the TxSTB* signal. The signals DGRNT and system R/W ensure that RAM is accessed in this manner only during a receiver DMA cycle. The signals TxSTB* and E* latch the valid data onto the system data bus (SDO-SD7) as soon as they become asserted. Thus by the time the RAM is ready to be written to, the data is valid. In this way uncorrupted data should be stored in memory. The RAML* memory decode logic has been designed to satisfy the timing requirements of Figures 5 and 6.
SOFTWARE OVERVIEW The DRM68K software system is based on the process model of distributed operating system designis. Distributed realtime multitasking is supported by a kernel as described in Reference 16. The communications manager (CM) is a system task that is responsible for communications (message sending and receiving) between atask in one node and some other tasks in another node. Each task is uniquely identified by a process number and a node number which is set using a dual-inline switch DI PSW. A FORTH 17 interpreter system is also implemented in the DRM68K, and extended with two words for sending and receiving messages: 'send' and 'receive'. Therefore user processes are written in FORTHand they can send or receive messages in a transparent manner. 'Send' and 'receive' are translated into calls to kernel primitives, which in turn put messages into the corresponding mailboxes of the CM task (if the destination process is in another node). The CM task transmits messages onto the
524
[,AN only when it is in possession of the token. The messages may be control messages, data messages or acknowledgement messages.
I.AN protocol The communications protocol adopted in DRM68K is based on the AppleTalk protocol 18 but the various protocol layers (message, packet, data link and physical) are simplified, integrated and embodied inside the CM task in order to reduce the communications and storage overhead. This decision was taken because of the simplicity of the AppleTalk protocol and the desire to use the Apple Macintosh computer as a development tool for DRM68K and at the same time satisfy the design constraints. One of the DRM68K nodes will act as a bridge between the DRM68K LAN and the AppleTalk network, or even MAP and TOP 19 networks, if required.
Message structure Messages are broken down into packets which are then transmitted or received via the ADLC in the form of frames. Figure 7 shows the message format. Packets are transmitted that use a positive-acknowledgement, retransmission-on-timeout protocol which is the responsibility of the talker and listener procedures inside the CM Task. Some messages which are passed around the DRM68K I_AN are broadcast messages that have a destination node identification number of 255. Broadcast messages are intended for every node on the LAN The CM task will issue three types of broadcast messages, namely 'master of the ring', 'EOT' and 'token'. The 'master of the ring' message only occurs at initialization and is used to establish the existence of a single token in the ring. The EOT message precedes the sending of the token and tells all other nodes to go to off-loop (point-to-point) mode. The 'token' message is then passed onto the next node on the I_AN.
Microprocessors and Microsystems
Data link / frame format
Opening flag 0
Destination node [D
1
Length
2
Source node ID
Packet
3 Destination process ID 4 5
Message
Source process ID Message type
"•l
] Packet
I1
121
I 2
, I
Packet
Sequence number
Packet number
I
Total no. of packets
Data (0-1 17 byte)
[
\ Figure Z
Talker protocol
I
IJ[
(Sequence 1)
I K (SequenceN)
Frame check sequence (2 byte) Closing flag
Network message format
Initialization protocol On power up of the DRM68K I_AN, the NIU hardware on each node must be initialized by the initialization procedure of the CM task, which goes through the following protocol. (1) Set the ADLC to ON LOOP mode. (2) If a 'master of the ring' message is received, leave the ADLC in the on-loop mode. Control passes on to the listener procedure which now monitors the receiver. All further 'master of the ring' messages and any other messages that are not addressed to this particular node will be ignored. No transmission can be initiated until the token has been received. (3) If no 'master of the ring' message is received, then this node may bid for possession of the initial token. A 'master of the ring' message which contains the node identification number is sent onto the LAN. In this way the initialization procedure is able to distinguish between its 'master of the ring' message and the same message from another node. (4) Initialize a system timer to generate a timeout. Monitor the receiver for the return of its own 'master of the ring' message. If the message is not received when the timeout is generated, repeat steps 3 and 4. (5) If the initialization procedure receives its own 'master of the ring' message then it may now assume possession of the token. It is the only node allowed to transmit data or messages onto the I_AN. At this stage, control passes to the talker procedure. (6) If another node's 'master of the ring' message is received, then a token generation collision has occurred which must be resolved by an arbitration protocol. Arbitration protocol (1) The node identification number is used to generate a timeout delay during which period the initialization procedure will not bid for possession of the token. After the timeout period it will proceed to send out the 'master of the ring' message again. (2) If in the timeout period another node sends a'master of the ring' message (that was not sent during the initial collision) then the initialization procedure
Vol 12 No 9 November 1988
takes no further part in bidding for possession of the token. The main routine of the CM task takes over, recognizes that it does not possess the token, goes to the on-loop mode and activates the listener procedure. (3) If the initialization procedure receives back its 'master of the ring' message then it remains in the off-loop mode. It is in possession of the token and control passes over to the talker procedure. If the 'master of the ring' message of some other node is received then the above steps are repeated.
Once the token has been successfully received by a node, control passes to the talker procedure. (1) Send acknowledgements for all messages received. Acknowledgement messages have already been linked together in the acknowledgement block by the listener procedure. (2) Retransmit all unacknowledged messages. (3) Send new messages, if any. If the length of a message exceeds 117 byte, then the talker procedure will packet the message before sending it. Otherwise, the message is assigned the next sequence number and sent. Those messages that are broken into packets have a common sequence number, a unique packet number (which starts from 1) and a total packet count. (This byte tells the receiving node how many packets there are.) Single-packet messages have a null byte for the packet number and total packet count. For each packet sent, an entry will be created in the unacknowledged block. (4) If no more messages are to be sent the talker prepares to send the token. Before the token is sent, all other nodes on the LAN must be in point-to-point mode. To achieve this the talker broadcasts an EOT message. Once the EOT message returns, the 'token' message is sent. (5) The CM task timer is loaded and control passes to the listener procedure. Listener protocol After receiving an EOT broadcast and timing out, all CM tasks resident in different nodes will jump to the beginning of the listener procedure. (1) If no transmission occurs on the I.AN duringthe next 60 seconds, a timeout occurs. The node which last had possession of the token will then resend the EOT and 'token' messages. (2) Once transmission occurs on the LAN, the listener concentrates on the messages which the node with the token is transmitting. The listener operates two buffers. When a message is successfully received without error the listener procedure will swap to the other buffer, so that no messages Will be lost or written over. (3) If the message just received is a 'master of the ring' or a 'token' message, it is ignored. If it is an EOT message then the listener sets a timeout and programs the N IU hardware to point-to-point mode, expecting a token to be received. If a timeout occurs first then the listener returns to the beginning of the procedure. If a token is successfully received then the listener waits a short time before jumping to the talker procedure.
525
This delay allows all the other nodes time to return to on-loop mode. (4) If the message is a broadcast message but is not one of the control messages, then it is treated in the same way as a message addressed to this node. In this way it is possible fora process to send broadcast messages to other processes in all the other nodes. It is also possible for kernel initialization messages to be passed in this way. (5) If the message received is not a broadcast message then the destination node identification number is compared to the node number of the node in which the listener is resident. If the message is addressed to some other node it is ignored, but if the message is addressed to the relevant node the listener tests to see whether it is an acknowledgement message. If so the listener searches for the corresponding unacknowledged block and returns the associated message buffer and unacknowledged block buffer to the kernel. If no such message is found the acknowledgement message is ignored. (6) If the message received is not an acknowledgement message or a broadcast message then the listener forms the message slot required by the kernel for interprocess message passing16. The listener also updates its acknowledgement block linked list so that next time this node is executing the talker procedure it will send an acknowledgement message for the message which has just been received. The listener then calls the SEND_MESSAGE kernel primitive to pass the message onto the destination process within that node. The listener then repeats the procedure for the next message.
ACKNOWLEDGEMENTS The author would like to thank members of the DRM68K project group - - Peter Vial, Amir Sarlak, Hendra Gondokusumo and Michael C h u e - - f o r their contributions to the work described in this paper. The major part of the DRM68K project was carried out when the author was with the Department of Electrical and Computer Engineering at the University of Wollongong in Australia. The author is grateful to Professor Brian Smith and other members of the Department for their support in this project.
REFERENCES 1 Peterson, J L and Silberschatz, A OperatingSystem Concepts Addison-Wesley, Wokingham, Berks, UK (1983) pp 22-23 2 Wolochow, P I 'Microcomputer operating system trends' Digital Des. (December 1981) pp 7-11
526
Kam Wing Ng is a senior lecturer in the Department of Computer Science at the Chinese University of Hong Kong. He received a PhD in electronics engineering from Bradford University, UK, in 197Z He was a lecturer in the Department of Electrical and ~ ~Computer Engineering at the University of Wollongong in Australia during 1986. His research interests include distributed operating systems, logic programming and silicon compilers. He is a member of the British Computer Society. 3 Halsall, F Introduction to data communications and computer networks Addison-Wesley, Wokingham, Berks, UK (1985) pp 202-246 4 M68000 16~32-bit microprocessor, fourth edition Prentice-Hall, Englewood Cliffs, N J, USA (1984) 5 Clements, A 'The 68000 and its interface' Microprocessors MicrosysL Vol 8 No 7 (September 1984) pp 324-337 6 Carter, E M and Bonds, A B 'The VU68K single-board computer' Byte (January 1984) pp 403-416 7 Morales, A J 'Interface 6800-microprocessor peripherals to the 68000' EDN (4 March 1981) pp 159-161 8 William, S Programming the 68000 Sybex, USA (1985) 9 Forlier, P J Design and analysis of distributed real-time systems McGraw-Hill, New York, NY, USA (1985) 10 Myers, W 'Towards a local network standard' IEEE Micro (August 1982) p 28 11 Tobagi, L A 'Multiple protocols in packet communication systems' IEEE Trans. Commun. Vol 28 No 4 (April 1980) pp 468-488 12 Lissack, T, Maglaris, B and Frisch, I f 'Digital switching in local area networks' IEEECommun. Mag. Vol 21 No 3 (May 1983) 13 The complete Motorola microcomputer data library Motorola, Austin, TX, USA 14 Morales, A J 'Access memory directly in 16-bit luC systems' Electron. Des. (11 June 1981) pp 221-226 15 Fortier, P J Design of distributed operating systems McGraw-Hill, New York, NY, USA (1986) 16 Ng, K W 'Message-passing primitives for multimicroprocessor systems' Microprocessors Microsyst. Vol 10 No 3 (April 1986) pp 156-160 17 Brodie, L Starting FORTHForth Inc., Manhattan Beach, CA, USA (1981)
18 Sidhu, G S, Andrews, R G and Oppenheimer, A B Inside AppleTalk Apple Computer, Cupertino, CA, USA (1986) 19 Kaminski, M A Jr 'Protocols for communicating in the factory' IEEESpectrum (April 1986) pp 56-62
Microprocessors and Microsystems
The paper describes the design and implementation of DRM68K, a low-cost distributed realtime multitasking system. Each node in DRM68K consists of a minimally configured 68000 microprocessor and a network interface unit that connects the n o d e s together to form a token ring local area network. The system is built on top of a distributed multitasking kemel. DRM68K is useful in realtime e m b e d d e d computing applications such as robotics, process control and flight simulation. microsystems local area networks reallime systems
multitasking
68000
This paper describes the DRM68K project, the aim of which is the design and implementation of a low-cost distributed realtime multitasking system. Before tackling the hardware and software details of DRM68K, the term 'distributed realtime multitasking system' should first be defined in its proper context. Peterson and Silberschatz 1 define that: 'A realtime system is often used as a control device in a dedicated application. Sensors bring data to the computer. The computer must analyse the data and possibly adjust controls to modify the sensor inputs . . . (with) well defined constraints. Processing must be done within the defined constraints or the system will fail.' 'Multitasking' refers to the ability of an operating system to effectively manage several tasks, from one computer program, that are operating simultaneously2. Thus 'realtime' and 'multitasking' are attributes and performance constraints imposed on the hardware and software of a computer system. Realtime embedded computing applications such as robotics, process control, flight simulation etc. require this category of realtime multitasking systems. In applications of this type, the overall processing function is often distributed naturally over a localized area. In robotics, for example, the processing is distributed through equipment distributed around the plant or site. A logical evolution of Department of ComputerScience,ChineseUniversityof Hong Kong, Shatin, NT, Hong Kong Paperreceived: 10 July 1987. Revised:5 July 1988
this physical distribution of processing functions is to implement each function by means of a separate computing unit or node. The nodes which make up the system are then linked together via a local area network (LAN) 3 to perform the overall system processing function. DRM68K is a low-cost implementation of this system configu ration.
HARDWARE DETAILS The block diagram of a standard node in the DRM68K system is shown in Figure 1, which depicts the system configuration. The low cost requirement demands that each node must fit onto a single board, and a minimally configured CPU that satisfies most of the needs of the target applications must be designed. Therefore each node has the following specification • 8 kbyte of static RAM, expandable on board to 32 kbyte • 16 kbyte of EPROM, expandable on board to 64 kbyte • two RS232C ports • 16 parallel I/O lines • three 16-bit timers.
CPU configuration The 16/32-bit MC68000 microprocessor4 was chosen as the DRM68K CPU because of its high performance and the availability of inexpensive compatible 8-bit peripheral components. Each node is a complete microcomputer system (Figure 2) built around an 8 M H z processor designed with low cost and minimal parts count in mind. Examples of 68000-based system design are abundant in the literature (see, for example, References 5-8); therefore only the salient points are described here. Moreover, Figure 2 shows the logic functions required in each node in gate form which in practice can be implemented using programmable logic devices. The minimal configuration shown here can also be expanded quite easily to suit various application requirements.
0141-9331/88/09519-08 $03.00 © 1988 Butterworth & Co. (Publishers) Ltd Vol 12 No 9 November 1988
519
Computing station (node)
cP0
i RAM
I P.oM
I 'mer I
I
I
I
1
I
Parallel I/O [
I
[ eria, ,O I
1
I interface e'w°r unit
II... ]
Figure 1.
I
~ r J i g e n ! ~
DRM68K system configuration
Figure 3 shows the system memory map. Address decoding is performed by a three-to-eight-line decoder, IC1 (74LS138). The EPROM and RAM outputs are further gated with the upper and lower data strobe signals, UDS* and LDS*, by IC2 (74LS32) for defining upper- and lowerbyte PROMs and RAMs. The three enable inputs of IC1 also qualify the output strobes. Address strobe AS* allows an output to be enabled only when a valid address appears on the address bus. To ensure that the outputs are selected only when the data bus carries valid data, the two data strobes are combined in IC3. Since DRM68K uses synchronous peripheral devices of the M6800 family, no acknowledge signal is required. However, to satisfy the requirements of asynchronous bus transfers, an acknowledge signal (DTACK*) must be sent by the memory or peripheral back to the processor to inform it that the transfer is complete. If necessary, the processor inserts wait states into the cycle until it receives the acknowledgement. Peripheral devices in the M68000 family have DTACK open-drain outputs which are directly connected to the processor's DTACK input along with a pullup resistor. However, PROMs and RAMs do not have such an output so an equivalent signal must be created. On more expensive systems DTACK is normally generated by either a multitap delay line or an active delaying device such as a shift register driven by the processor clock to produce a delayed chip select signal for the DTACK* input. There are often different circuits for each type of device, and the delay is selectable so it may be set to the optimum required for each type of device in the system. To keep thngs simple in the DRM68K, the PROM and RAM select outputs of IC1 are combined in IC4 and then inverted by the open-collector inverter IC5, pulling DTACK* low if either select output goes low. Memory capacity consists of two PROM and two RAM sockets. Two of each byte-wide memory chip are required to give a 16-bit data width. Link la allows EPROMs with standard pin configurations from the 2732-256 family to be fitted, providing from 8 to 64 kbyte. RAM sockets currently accept either 2 or 8 kbyte static RAMs through link l b (i.e. either 6116 or 6264 devices) to give either 4 or 16 kbyte. The PCB for each node is laid out to accept 16 and 32 kbyte devices, which will give up to 64 kbyte of RAM. The peripheral devices used in the DRM68K are synchronous. When the AS* line is asserted (low), the synchronous peripherals must pull the VPA* processor pin low to generate the synchronous timing signals VMA* and
520
E. In the DRM68K, the peripheral devices that use synchronous transfers are the parallel adapter (PIA, IC6), the programmable timer module (PTM, IC7) and the asynchronous communications interface adaptors (ACIAs, IC8 and IC9). Address line A23 is tied to VPA* through IC10 and AS. When either of the devices is selected, A23 must be high, which pulls VPA* low and initiates the synchronous data transfer. After the transfer is complete, the processor automatically resumes asynchronous operation. This scheme limits addresses using A23 to synchronous devices only (Figure 3). The basis of the DRM68K serial communication facility is the MC6850 ACIA. Data rates are switch selectable to generate the clock rate required by the LAN. DRM68K uses the autovector mode for interrupts. When the CPU responds to an interrupt, all address bus lines except A1-A3 are high (because these address bits are set equal to the priority of interrupt being serviced). Since A23 is connected to VPA through IC10, the CPU is forced into the autovector mode for all interrupts. Interrupts are signalled by lines IPL0*-IPL2* via a priority encoder ICl 1. Inputs of IC11 are connected to interrupt signals through link 3, which is provided to vary manually the levels of interrupts required by the different devices used in various applications. An interrupt acknowledge signal lACK* is provided by CPU status lines FC0-FC2. When processing an interrupt the CPU places a unique code of all '1 's on status lines FC0-FC2; this code is used to generate lACK* via IC12. The lACK* signal disables memory access via IC1 during an interrupt.
N e t w o r k interface unit
Before describing the network interface unit (NIU), the LAN characteristics must be specified. In addition, ideal features of the LAN are low cost, high bandwidth, good reliability, low error rate, minimal dependence upon centralized subsystems for control, fair access by all devices, ease of configuration and maintenance. Basic techniques 3, 9 used with LANs are concerned with • network topology • transmission medium • network (medium) access control methods. A ring topology 3 is most natural for distributed realtime applications with a typical data transmission rate of 500 kbit s-1. Two types of mechanism to control access to the transmission medium have been adopted by the IEEE 802 Standards Committee: CSMA/CD (used solely with bus networks) and token passing access 1°. Performance evaluations of CSMA/CD 11' 12 have indicated that this system has a poor utilization characteristic under high load. Token ring systems are deterministic under all loads, and perform best under high load. Thus a token ring, message-passing network has been adopted for DRM68K. A shielded twisted pair was chosen as the transmission medium because it is inexpensive, and because it is straightforward to make a physical connection to the cable. A block diagram of the NIU is shown in Figure 4. The hardware may be broken up into two main areas, as follows. • Direct memory access. A 6800 family chip, the MC6844 DMA controller (DMAC), is used to perform DMA. Its
Microprocessors and Microsystems
0 0
~.
~.
1:,
~
+
+
~.
~-o
a
~
•
.
/T~iiir~
J
.._,..---,
il~li iI
79779777 I I I 11
ililil]l
i~ ~i~ ~~
~>
I I !
00o0
I 0
E
u~ cO %0
~a ~6
..Q
~D
E i-
i
.
!,b ~,I~ L.) =-_
:5
S$S
~
~
rq
E
Vol 12 No 9 November 7988
U~
521
Unused
FFFFFFH DOOOOOH
PTM
C0000FH C00000H
P[A
B00007H B00000H
ACIAs
A00003H A000OOH
Unused DIPSW DMAC AD LC
800000H 700000H 600000H 500000H
Unused 1FFFFFH
t 100FFFH-6116 103FFFH-6264 RAM Reservedfor monitor
10FFFFH-8832
100400H
100000H
001FFFH-2732 003FFF H-2764
PROM
007FFFH-27128 00FFFFH-27256
000000H
Figure 3.
System memory map
function is to transmit data (messages to be sent via the LAN) to the MC6854 ADLC (advanced data link controller) and to receive data (messages coming in from the LAN) from the ADLC without involving the CPU directly. To minimize the complexity of the hardware and reduce the required IC count for each node, all incoming data is stored in lower memory and all outgoing data is transmitted from lower memory. The DMAC must be programmed by the CPU before it can respond to a receive signal or transmit signal command. This is done using the asynchronous interface and associated decoding circuits. • Ring controller. This scheme also uses a 6800 family chip to interface to the LAN. The ADLC is addressed by the CPU via the address decoder, asynchronous interface and A1, A2 and Tristate R/W lines. The ADLC needs to have its registers programmed before it can be configured on the ring.
synchronizes 68000 accesses to the DMAC (and ADLC) with the E clock TM. Initially, flipflops ICl 5A and IC15B are cleared. This situation forces the IC15B DTACK* output high, puts latch 16 in the high impedance state and enables latch 17. At the start of a DMAC (or ADLC) write cycle latch 17 remains enabled, passing data from the 68000 system bus to the DMAC (or ADLC) as soon as LATCH__ENABLE* is asserted. During a read cycle, the signals Tristate R/W and START__TRANSFERare asserted high. Then NAND gate 18 forces latch 17 to the high impedance state and enables latch 16. When the signal LATCH__ENABLE* is asserted the data is latched over from the DMAC or ADLC bus. START TRANSFER is asserted when either the ADLC or DMAC are addressed, but the DMAC or ADLC are selected by the assertion of DMAS*. This signal is further decoded to chip select either the DMAC or ADLC depending on the outputs of the address decoder (IC1). Flipflop 15A is clocked high on the first falling edge of E after START TRANSFERand LDS* are asserted. The Q output of 15A is applied to NAND gate 19, asserting DMAS*. Chip selecting the DMAC and ADLC in this manner ensures adequate time for setting up a valid address, and is the principal reason for incorporating the asynchronous interface. On the falling edge of E, the Q* output of flipflop 15B is clocked low, asserting DTACK* and latching data into the enabled latch. The asserted DTACK* (open collector) is connected to Q* of flipflop 15B and sends the required DTACK* back to the CPU to satisfy CPU requirements. This output negates DMAS* so that it goes back to a high output. This results in deselecting the ADLC or DMAC. Once the CPU receives the DTACK* signal it terminates the access by negating LDS*, clearing flipflops 15A and 15B (by placing a low-level voltage on the CLR input). This initializes the interface for the next access. The DTACK* signal is sent through an open collector buffer so that other devices (such as PROM or RAM) may use DTACK* when the ADLC or DMAC are not being accessed. The above description is equally valid for a read or a write operation.
Transmitting onto the I_AN The DMAC has four DMA channels which can be configured independently by software using 15 addressable registers. The NIU only uses two of the channels; channel 1 and channel 0. Channel 0 is used for receiving data from the ring; channel 1 is used for transmitting data onto the ring. Using the control registers the ADLC may be ~RS0,
I CPUcircuit
RS1and/
R/w
g..... ,or
The following subsections describe the logic of the asynchronous interface and analyse the bus arbitration and decoding logic (RS0, RSI and Tristate R/W) for transmitting and receiving from the LAN. Further details can be found in the data sheets of the AD LC and DMAC 13.
-,JAsyoc. ......
'----'1 interface RAM and decodng Io9c l ~ ' /
"--'1
Bus a,bitrotio°
K--~I
II
I
Asynchronous
IllIll
ADLC
~--' ~
DMA
I"-----1
..... o,,er
Asynchronous interface The circuit for the asynchronous interface consists mainly of IC15, IC16, IC17 and IC18 (Figure 2). This circuit
522
Systembus
Figure 4.
} I
q
Network interface unit
MiCroprocessors and.Microsystems
programmed to operate in various modes. The ADLC can be programmed to on-loop mode or point-to-point mode (off-loop mode). When the NIU software programs the ADLC to on-loop mode, the NIU software is only listening to the messages that are being sent by the present master of the ring (the node which holds the token is considered the master). When the NIU software programs the ADLC to off-loop mode, the NIU software will send messages onto the loop and look for the message that was sent to return. When the ADLC is in on-loop mode it stores the incoming byte into its receiver FIFO register and adds a single-bit delay to the byte before retransmitting that data onto the next node. Bus arbitration When the ADLC asserts TXRQ1 it leads to DRQ2* being asserted by the DMAC. The assertion of TXRQ1 results in BR* being asserted. BR* is an open-collector output so that other devices may bid for the system bus. At this stage DCRNT, BGACK* and TxSTB* are all negated. The CPU will relinquish control of the system bus and assert BG*. All outputs of the 68000 will go to a high impedance state. The 68000 is designed to assume itself the lowest-priority IC in the hardware system. The 68000 will negate the signals BCACK*, AS* and DTACK*. DMA may now proceed.The above conditions lead to output Y3* of IC22 becoming asserted. Since the output of IC23A is asserted high (because DRQ2* is asserted), this results in NAND flipflop IC23 asserting DCRNT and BGACK*. BCACK is passed through an open-collector buffer to allow other system devices to assert this signal. Asserting BGACK* negates BG*, BR* and output Y3* of IC22. However, DGRNT will still be asserted. From the timing diagram (Figure 5) the DMAC does not negate DRQ2* until after the DMA transfer has started. Well before DRQ2* is negated, TxSTB* becomes asserted. DGRNT is negated when both DRQ2* and TxSTB* are negated. Since TxSTB* is only negated after a DMA transfer has occurred, DG RNT is negated after DMA is finished. Thus DGRNT may be used to set up conditions for the DMA transfer. TxSTB* is used to enable the ADLC during a DMA transfer so that TxTRQO and TxRQ1 do not return to the negated state until after the data is latched in the appropriate direction. The DMAC now has control over the system address bus and system R/W. The DMAC asserts the system R/W
///
\ \
\,
E clock TxRQ
\\
/-
/ \
DRQ2* \\ /
DGRANT TxSTB* MEM SELECT*
"4
R/W
(((
)))
p0-D7
~ A S *
~/~
/ ~
LDS*
/
ADLC CS* LATCH_ENABLE*
[This pulse latchesdata ' acrossto the ADLC Tx register
Figure 5. LAN
Timing diagram for DMA transmission onto the
Vol "12 No 9 November 1988
into a high state (R/W = 1). This is used to read data from the RAM. It is also used as an input to Tristate R/W along with DGRNT. Since DGRNT is asserted, Tristate R/W will be asserted low, to write data into the ADLC and allow data to be passed in the correct direction through the asynchronous interface. Under these conditions latch 17 is enabled. When TxSTB* is asserted the data is latched across to the ADLC/DMAC asynchronous data bus. It is important that the data is ready well before TxSTB* is asserted. For this reason the chip select logic for the RAM is designed to enable the RAM as soon as DGRNT is asserted. The signal DGRNT is also used to assert AS* and LDS*. The address decoder uses system address lines A21, A22 and A23 to select the particular ICs being accessed. The address for RAM is A21 = 1, A22 = O and A23 = 1. During DMA, RAM is addressed by using DGRNT to enable/ disable IC24. Decoding logic Figure 2 shows the decoding logic used to generate Tristate R/W. When no DMA is in progress the output of AN D gate 26A is asserted low and that of AN D gate 26B is the same as the system R/W. This allows the CPU to read and write to the ADLC normally. The primary task of this circuit is to allow the DMAC to write to the ADLC while reading data from RAM and vice versa. For this purpose, when DGRNT is asserted, the system IVW must be inverted by the logic shown in Figure 2. During a DMA transfer (either transmission or reception) DCRNT is asserted. This disables AND gate 26B and enables AND gate 26A. AND gate 26A passes an inverted system R/W. Then the output of OR gate 27A becomes R/W. If the DMAC is transmitting data then R/W is asserted high and Tristate R/W is asserted low. If the DMAC is receiving data then R/W is asserted low and the Tristate R/W is asserted high. This is the r(~quired output duringa DMA cycle and is the reason for the symbolic name Tristate R/W. The decoding logic used to generate the register select values RS0 and RS1 are designed so that RS0 = A1 and RS1 = A2 while DMA is not in progress. When DMA occurs the circuit will set RS0 = 0 (asserted low) and RS1 = 1 (asserted high). For transmission onto the loop, Tristate R/W will be asserted low. This will select the transmit FIFO frame continue register. When the last byte of data is to be transmitted by the DMAC the signal DEND* becomes asserted setting RS0 (asserted high) and RS1 (asserted high) to 1. The address control bit must be set to zero at initialization so that the transmit FIFO frame terminate register is selected when DEND* is asserted. The ADLC will then send the last byte, the frame check sequence (FCS) and the closing flag. The ADLC will not assert TDSR until it is ready to transmit again. When the CPU is reading RAM the RAML* decoding logic in Figure 2 is designed to pass the signal MEM~SELECT* through unchanged. When the DMAC is transmitting data onto the loop it must first read the data from RAM. To avoid the possibility of the asynchronous interface latching over data that is not correct (due to insufficient access time), the RAM is selected as soon as MEM~SELECT* and DGRNT* become asserted. In this way, when the signal TxSTB* becomes asserted and latches the data across to the ADLC, the RAM will have been enabled for more than its access time. Thus data on the system bus will be uncorrupted. The complexity of
523
the RAM select logic would not be necessary if the timing considerations for a read and write were the same.
E clock
///
} \\\\~k\
TxRQ0 DRQ2*
Receiving from the LAN _
When the ADLC receives data from the LAN it stores it in the receiver FIFO stack and asserts TxRQ0 to tell the DMAC that it requires service of its read FIFO register. The ADLC also asserts its interrupt telling the CPU that data or a message has been received by the LAN. The CPU may then accept the interrupt and reprogram the interrupt so that it indicates the end of the message. In this way the CPU may reprogram the ADLC and DMAC before another message is received. Figure 6 shows the timing diagram for DMA reception from the ring.
_
DGRANT TxSTB*
kx
R/W MEM_SELECT
/7/
\
DO-D7 (system) AS*
/'/,/
\
LDS* A D L C CS* LATCH_ENABLE*
Bus arbitration When the ADLC asserts TxRQ0 (the receiver channel) it will also lead to DRQ2* being asserted by the DMAC and the assertion of BR*. In all other respects the operation of bus arbitration is identical for reception and transmission.
This pulse latches data across to memory from the o A D L C Rx register
Figure 6. LAN
Timing diagram for DMA reception from the
Decoding logic The operation of Tristate R/W for reception is identical to that for transmission except that Tristate R/W is asserted high during reception. From Figure 6 it can be seen that when TxSTB* becomes asserted the RAM is still in the setup stage. It will not be ready to have data written to it for about 270 ns (access time) after the E clock has fallen. For this purpose the RAM is chip selected on the falling edge of the TxSTB* signal. It remains enabled until the rising edge of the TxSTB* signal. The signals DGRNT and system R/W ensure that RAM is accessed in this manner only during a receiver DMA cycle. The signals TxSTB* and E* latch the valid data onto the system data bus (SDO-SD7) as soon as they become asserted. Thus by the time the RAM is ready to be written to, the data is valid. In this way uncorrupted data should be stored in memory. The RAML* memory decode logic has been designed to satisfy the timing requirements of Figures 5 and 6.
SOFTWARE OVERVIEW The DRM68K software system is based on the process model of distributed operating system designis. Distributed realtime multitasking is supported by a kernel as described in Reference 16. The communications manager (CM) is a system task that is responsible for communications (message sending and receiving) between atask in one node and some other tasks in another node. Each task is uniquely identified by a process number and a node number which is set using a dual-inline switch DI PSW. A FORTH 17 interpreter system is also implemented in the DRM68K, and extended with two words for sending and receiving messages: 'send' and 'receive'. Therefore user processes are written in FORTHand they can send or receive messages in a transparent manner. 'Send' and 'receive' are translated into calls to kernel primitives, which in turn put messages into the corresponding mailboxes of the CM task (if the destination process is in another node). The CM task transmits messages onto the
524
[,AN only when it is in possession of the token. The messages may be control messages, data messages or acknowledgement messages.
I.AN protocol The communications protocol adopted in DRM68K is based on the AppleTalk protocol 18 but the various protocol layers (message, packet, data link and physical) are simplified, integrated and embodied inside the CM task in order to reduce the communications and storage overhead. This decision was taken because of the simplicity of the AppleTalk protocol and the desire to use the Apple Macintosh computer as a development tool for DRM68K and at the same time satisfy the design constraints. One of the DRM68K nodes will act as a bridge between the DRM68K LAN and the AppleTalk network, or even MAP and TOP 19 networks, if required.
Message structure Messages are broken down into packets which are then transmitted or received via the ADLC in the form of frames. Figure 7 shows the message format. Packets are transmitted that use a positive-acknowledgement, retransmission-on-timeout protocol which is the responsibility of the talker and listener procedures inside the CM Task. Some messages which are passed around the DRM68K I_AN are broadcast messages that have a destination node identification number of 255. Broadcast messages are intended for every node on the LAN The CM task will issue three types of broadcast messages, namely 'master of the ring', 'EOT' and 'token'. The 'master of the ring' message only occurs at initialization and is used to establish the existence of a single token in the ring. The EOT message precedes the sending of the token and tells all other nodes to go to off-loop (point-to-point) mode. The 'token' message is then passed onto the next node on the I_AN.
Microprocessors and Microsystems
Data link / frame format
Opening flag 0
Destination node [D
1
Length
2
Source node ID
Packet
3 Destination process ID 4 5
Message
Source process ID Message type
"•l
] Packet
I1
121
I 2
, I
Packet
Sequence number
Packet number
I
Total no. of packets
Data (0-1 17 byte)
[
\ Figure Z
Talker protocol
I
IJ[
(Sequence 1)
I K (SequenceN)
Frame check sequence (2 byte) Closing flag
Network message format
Initialization protocol On power up of the DRM68K I_AN, the NIU hardware on each node must be initialized by the initialization procedure of the CM task, which goes through the following protocol. (1) Set the ADLC to ON LOOP mode. (2) If a 'master of the ring' message is received, leave the ADLC in the on-loop mode. Control passes on to the listener procedure which now monitors the receiver. All further 'master of the ring' messages and any other messages that are not addressed to this particular node will be ignored. No transmission can be initiated until the token has been received. (3) If no 'master of the ring' message is received, then this node may bid for possession of the initial token. A 'master of the ring' message which contains the node identification number is sent onto the LAN. In this way the initialization procedure is able to distinguish between its 'master of the ring' message and the same message from another node. (4) Initialize a system timer to generate a timeout. Monitor the receiver for the return of its own 'master of the ring' message. If the message is not received when the timeout is generated, repeat steps 3 and 4. (5) If the initialization procedure receives its own 'master of the ring' message then it may now assume possession of the token. It is the only node allowed to transmit data or messages onto the I_AN. At this stage, control passes to the talker procedure. (6) If another node's 'master of the ring' message is received, then a token generation collision has occurred which must be resolved by an arbitration protocol. Arbitration protocol (1) The node identification number is used to generate a timeout delay during which period the initialization procedure will not bid for possession of the token. After the timeout period it will proceed to send out the 'master of the ring' message again. (2) If in the timeout period another node sends a'master of the ring' message (that was not sent during the initial collision) then the initialization procedure
Vol 12 No 9 November 1988
takes no further part in bidding for possession of the token. The main routine of the CM task takes over, recognizes that it does not possess the token, goes to the on-loop mode and activates the listener procedure. (3) If the initialization procedure receives back its 'master of the ring' message then it remains in the off-loop mode. It is in possession of the token and control passes over to the talker procedure. If the 'master of the ring' message of some other node is received then the above steps are repeated.
Once the token has been successfully received by a node, control passes to the talker procedure. (1) Send acknowledgements for all messages received. Acknowledgement messages have already been linked together in the acknowledgement block by the listener procedure. (2) Retransmit all unacknowledged messages. (3) Send new messages, if any. If the length of a message exceeds 117 byte, then the talker procedure will packet the message before sending it. Otherwise, the message is assigned the next sequence number and sent. Those messages that are broken into packets have a common sequence number, a unique packet number (which starts from 1) and a total packet count. (This byte tells the receiving node how many packets there are.) Single-packet messages have a null byte for the packet number and total packet count. For each packet sent, an entry will be created in the unacknowledged block. (4) If no more messages are to be sent the talker prepares to send the token. Before the token is sent, all other nodes on the LAN must be in point-to-point mode. To achieve this the talker broadcasts an EOT message. Once the EOT message returns, the 'token' message is sent. (5) The CM task timer is loaded and control passes to the listener procedure. Listener protocol After receiving an EOT broadcast and timing out, all CM tasks resident in different nodes will jump to the beginning of the listener procedure. (1) If no transmission occurs on the I.AN duringthe next 60 seconds, a timeout occurs. The node which last had possession of the token will then resend the EOT and 'token' messages. (2) Once transmission occurs on the LAN, the listener concentrates on the messages which the node with the token is transmitting. The listener operates two buffers. When a message is successfully received without error the listener procedure will swap to the other buffer, so that no messages Will be lost or written over. (3) If the message just received is a 'master of the ring' or a 'token' message, it is ignored. If it is an EOT message then the listener sets a timeout and programs the N IU hardware to point-to-point mode, expecting a token to be received. If a timeout occurs first then the listener returns to the beginning of the procedure. If a token is successfully received then the listener waits a short time before jumping to the talker procedure.
525
This delay allows all the other nodes time to return to on-loop mode. (4) If the message is a broadcast message but is not one of the control messages, then it is treated in the same way as a message addressed to this node. In this way it is possible fora process to send broadcast messages to other processes in all the other nodes. It is also possible for kernel initialization messages to be passed in this way. (5) If the message received is not a broadcast message then the destination node identification number is compared to the node number of the node in which the listener is resident. If the message is addressed to some other node it is ignored, but if the message is addressed to the relevant node the listener tests to see whether it is an acknowledgement message. If so the listener searches for the corresponding unacknowledged block and returns the associated message buffer and unacknowledged block buffer to the kernel. If no such message is found the acknowledgement message is ignored. (6) If the message received is not an acknowledgement message or a broadcast message then the listener forms the message slot required by the kernel for interprocess message passing16. The listener also updates its acknowledgement block linked list so that next time this node is executing the talker procedure it will send an acknowledgement message for the message which has just been received. The listener then calls the SEND_MESSAGE kernel primitive to pass the message onto the destination process within that node. The listener then repeats the procedure for the next message.
ACKNOWLEDGEMENTS The author would like to thank members of the DRM68K project group - - Peter Vial, Amir Sarlak, Hendra Gondokusumo and Michael C h u e - - f o r their contributions to the work described in this paper. The major part of the DRM68K project was carried out when the author was with the Department of Electrical and Computer Engineering at the University of Wollongong in Australia. The author is grateful to Professor Brian Smith and other members of the Department for their support in this project.
REFERENCES 1 Peterson, J L and Silberschatz, A OperatingSystem Concepts Addison-Wesley, Wokingham, Berks, UK (1983) pp 22-23 2 Wolochow, P I 'Microcomputer operating system trends' Digital Des. (December 1981) pp 7-11
526
Kam Wing Ng is a senior lecturer in the Department of Computer Science at the Chinese University of Hong Kong. He received a PhD in electronics engineering from Bradford University, UK, in 197Z He was a lecturer in the Department of Electrical and ~ ~Computer Engineering at the University of Wollongong in Australia during 1986. His research interests include distributed operating systems, logic programming and silicon compilers. He is a member of the British Computer Society. 3 Halsall, F Introduction to data communications and computer networks Addison-Wesley, Wokingham, Berks, UK (1985) pp 202-246 4 M68000 16~32-bit microprocessor, fourth edition Prentice-Hall, Englewood Cliffs, N J, USA (1984) 5 Clements, A 'The 68000 and its interface' Microprocessors MicrosysL Vol 8 No 7 (September 1984) pp 324-337 6 Carter, E M and Bonds, A B 'The VU68K single-board computer' Byte (January 1984) pp 403-416 7 Morales, A J 'Interface 6800-microprocessor peripherals to the 68000' EDN (4 March 1981) pp 159-161 8 William, S Programming the 68000 Sybex, USA (1985) 9 Forlier, P J Design and analysis of distributed real-time systems McGraw-Hill, New York, NY, USA (1985) 10 Myers, W 'Towards a local network standard' IEEE Micro (August 1982) p 28 11 Tobagi, L A 'Multiple protocols in packet communication systems' IEEE Trans. Commun. Vol 28 No 4 (April 1980) pp 468-488 12 Lissack, T, Maglaris, B and Frisch, I f 'Digital switching in local area networks' IEEECommun. Mag. Vol 21 No 3 (May 1983) 13 The complete Motorola microcomputer data library Motorola, Austin, TX, USA 14 Morales, A J 'Access memory directly in 16-bit luC systems' Electron. Des. (11 June 1981) pp 221-226 15 Fortier, P J Design of distributed operating systems McGraw-Hill, New York, NY, USA (1986) 16 Ng, K W 'Message-passing primitives for multimicroprocessor systems' Microprocessors Microsyst. Vol 10 No 3 (April 1986) pp 156-160 17 Brodie, L Starting FORTHForth Inc., Manhattan Beach, CA, USA (1981)
18 Sidhu, G S, Andrews, R G and Oppenheimer, A B Inside AppleTalk Apple Computer, Cupertino, CA, USA (1986) 19 Kaminski, M A Jr 'Protocols for communicating in the factory' IEEESpectrum (April 1986) pp 56-62
Microprocessors and Microsystems