- Email: [email protected]
Distributed processing: a strategy for health care computing
Distributed processing: a strategy for health care computing
Adrian V Stokes describes the evolution and present day status of a network used by a London Health Authority to link three hospitals
Hospitals deal with large amounts of data, which need to be accesed by different internal departments, each with its own particular needs. When information is passed within a District Health Authority from hospital to hospital the situation is more complex. A network in a major London hospital with external links is described with particular emphasis on its evolution and the need for versatility with cost effectiveness. Future plans are also discussed. Keywords: computer networks, distributed processin& District Health Authority
Introduction St Thomas' Hospital (London, UK) has an extensive history of computing. Itwas one of the first hospitals in the UK to make major use of computers in a health care context. Its early computing capability (from the beginning of the 1970s) was based on a mainframe using an interactive system, but, towards the end of the 1970s, as a prelude to implementing its second generation of systems, a strategic plan was drawn up involving distributing functions to a decentralized network of various sized processors and interconnecting these processors by means of a comunications network. This paper gives the background to the computer project at St Thomas' Hospital, and indicates some of the problems of the District Health Authority which make a distributed strategy even more appealing. It also discusses the concepts of the strategic plan, and then outlines its implementation, from the initial selection of equipment to the present situation. The paper also discusses future plans.
The DHSS experimental projects
West Lambeth Health Authority, St Thomas' Hospital, Lambeth PalaceRoad, London SE1, UK
Followingthe Flowers Report in 1966, a Committee was set up at St Thomas' Hospital to look at the possible use of computers in the hospital, and this Committee recommended the setting up of a new Department within the
hospital for this purpose. The Departmerit of Health and Social Security (DHSS) supported this idea, and, in 1969, the 'ADP Project Team' was formed to conduct a feasibility study. In 1971, St Thomas' became part of the 'Coordinated Hospitals Project' in which four hospitals (Addenbrookes' Hospital Cambridge, Charing Cross Hospital, University College Hospital and St Thomas') were provided with the same computer with the intention of being able to exchange large amounts of software. The computer chosen was the Xerox Data Systems' Sigma-6 and it was installed at St Thomas' in late 1973. The initial configuration had 112 kbyte of main (core) memory and 200 Mbyte of disc storage, by 1976, it had 20 online terminals. Over the next few years, a large number of applications were written for the computer. Two major areas were c o v e r e d - patient administration and pathology laboratories - - as well as a number of smaller ones, such as nursing sickness and absence recording. In the late 1970s, the Sigma-6 was coming to the end of its useful life and a study was commissioned from external consultants to recommend the future of computing at St Thomas'. They produced a Strategic Plan for Computing which recommended a distributed approach. There were various reasons for this, including the uncertain financial climate in the National Health Service (NHS).
0140-3664/85/060306-10 $03.00 © 1985 Butterworth & Co (Publishers) Ltd 306
computer communications
Planning in an age of uncertainty St Thomas' is the largest hospital in the West Lambeth Health Authority (in London), which was created by the 1981 NHS reorganization. West Lambeth is one of 14 Health Authorities in the South East Thames Region and is at the far north-west of the Region, adjacent to the River Thames and opposite the Palace of Westminster. It is an inner city area and for a number of years has had a declining population. This trend is expected to slow down and the population at the end of the decade is expected to be the same as at present, although the age distribution will change. Within the District, there are now only three hospitals, St Thomas', the South Westem (at Stockwell) and Tooting Bec with approximately 900, 300 and 800 beds respectively. Although the District serves the needs of the local population of about 180 000, it also deals with a much wider catchment area, not only because of the traffic patterns in South London (especially with Waterloo railway station a few minutes walk from St Thomas'), but also because St Thomas' is a major teaching hospital, and as such has patients referred from all over the country and the world. Table 1 shows the workload of St Thomas' Hospital. It should be noted that there are records for about 750 000 patients, of which over 650 000 are recorded on the computer-based Patient Master Index. The latter is growing at the rate of about 30 000 per annum and the rate of increase is growing. There is very considerable uncertainty about the future. Although this applies generally to the NHS (and many other organizations), it applies especially to West Lambeth Health Authority. The Govemment has declared a policy of redistributing msources within the NHS, and, as a result, the South-East Thames Region will lose resources. Within the Region, West Lambeth will lose considerably more than a v e r a g e - about 15% of its revenue over the next ten years (i.e. over £1 M per year cumulatively). Similar considerations apply to capital
vol 8 no 6 december 1985
Table 1. Annual statistics of West Lambeth Health Authority 650 000 Patient records on computer (30 000 new each year) 290 000 Outpatient attendances (64 000 new each year) 14 000 Outpatient clinic sessions 35 000 Inpatient attendances 2 900 Day cases 64 000 A & E attendances 600 000 Pathology tests funding. The position is further complicated by the way in which funding occurs. In common with similar organizations, funding occurs on a yearby-year basis and there is no 'canyover' from year to year. Thus, a flexible strategy was required which took all these factors into account as well as the more obvious technical ones such as changes in computer technology and the emergence of local area networks and microcomputers. Such a strategy was drawn up in the late 1970s by a team consisting of external consultants, together with staff from St Thomas'. This was known as the Strategic Plan for Computing.
The strategic plan Since the plan was originally drawn up, it has evolved significantly. This section discusses the basic concepts of the plan, together with details of its different aspects. Where it is important, differences between the original plan and its current state are pointed out.
Basic concepts A hospital may be looked upon as a number of semiautonomous units which have considerable internal needs for computing and communication but with relatively few links to the outside. For example, from the point of view of a user, a pathology laboratory will receive a specimen, then produce a report. In fact, this report is the result of very extensive work, much of it involving computers.
Thus a model of hospital has a number of important features. First, there is a large number of semiautonomous activities which are physically distributed and which intercommunicate by means of a complex network, using media such as printed reports and the telephone. Second, there is the financial uncertainty referred to previously. There are two ways of implementing the model that has been outlined on a computer. First (classically), each separate activity may be a separate process on a large computer and these may be interconnected by interprocess calls and shared files. Second, each separate activity may take place on a different processor with these processors linked by a communications network. Both of these have the same end result, which can be summarized as the single objective of improving the planning, monitoring and management of the Health District. Of the two approaches mentioned, it was the second that was adopted by the plan. There were various justifications for this approach. In particular, it allowed an incremental changover, followed by incremental growth. Once the transition to a distributed network had occurred, there would be no need to replace a mainframe once every seven to ten years, but the various computers on the network could be replaced gradually, thus spreading out the capital costs. Furthermore, with such an approach, it is probably easier to integrate other machines into the system. Frequently, users within the hospital are able to purchase computers with, for example, research funds and then wish these computers to be connected to other systems (in particular, to the Patient Master Index). There is often little, if any, control over the computers chosen and so a highly flexible approach is needed and this can be provided by adopting a distributed strategy. It was realized that the strategy suggested had some disadvantages. In particular, the technology was considerably newer and problems could be envisaged as a result. Second, it was likely that the overall costs
307
Applications S~llam$ Shared Services
J
Off-site communications
Figure 1. Schematic representation of the strategic plan would be higher. Nevertheless, the advantages seemed to outweigh the disadvantages. The plan thus proposed that, at least in theory, each application should reside on a separate processor and that these processors should be interconnected. In practice, of course, it was realized that a number of different (but usually related) applications were likely to share the same machine but this did not detract from the general principle. However, using this approach, a number of problems are evident and these arise as a result of the (semi-) autonomy of the 'owners' of these applications. First, the data needed generally within the hospital was distributed over a number of processors which were not centrally controlled. Thus, it would be possible for data to be unavailable at critical times. Also, the method of accessing this data would vary from machine to machine (or, from the user's point of view, from application to application). In order to overcome these problems, the plan proposed that there should be a particular system on the network which provided any services which needed to be shared amongst the users of the network and this was called the shared services system. A final component of the plan was that of links to external systems (for example, the Regional Computer Centre). Figure 1 is a schematic representation of the strategic plan.
Communications network Perhaps the major component of the
308
plan is the main communications medium on the St Thomas' site. Without this, the plan could not operate. Clearly this communications medium had to be a local area network, but this term could include not only conventional LANs but also devices such as PACXs. The plan did not lay down specific criteria for such a network but made it clear that a high degree of flexibility and functionality was required. The selection of the network is discussed below.
Shared services system The shared services system was proposed to provide services that needed to be shared between all users within the District and various such functions were identified. First (and, probably, most important), it was necessary to provide a general enquiry service for all the commonly used data. This was designated the 'bulletin board' and was intended to provide a uniform method of access to all data required for the day-to-day running of the hospital. Clearly, different users (or classes of users) had different requirements for access to data and the bulletin board therefore had to implement a high degree of security. It could be characterized by the feature that the information providers were not able, at the time of providing the information, to specify to whom it was addressed but that this was to be done as a management function at an earlier stage. Thus, for example, it may be decided that all information from the nursing system sent to the bulletin board can only be accessed by nurses but no mechanism is provided for restricting access to information to a particular nurse. The second shared service was intended to provide that facility. It was designated the 'message board' and messages had to be addressed to specific recipients (i.e. an 'electronic mail' system). The 'message' could, in fact, be the same data as was sent to the bulletin board. If it were required to send a message to a number of recipients, one copy had to be sent to each recipient. The third service was called the 'logan server' and was intended to
provide a mechanism for controlling access to the network and for making the appropriate connections. When a single computer system (which may, in fact, comprise a number of different processors) is used, it is usual to 'log in' once only, then to access the particular facilities required. The user is unaware, if it is a rnultiprocessor system, on which processor his process is runnin~ nor does it make any difference. Just because the network is more loosely connected with dissimilar processors, there seems to be no reason why the user interface should be complicated. It was therefore decided to implement the logan server so as to emulate the interface that would be seen on a single machine. That is, the user would log in to the network and not to individual machines within that network. To some extent, the logan server was intended to perform the functions of a network access machine 1, but at a considerably less sophisticated level. More to the point, it was intended to be able to provide access control and security. Various other functions have since been identified for the shared services system but these are not relevant to the present discussion.
External communications Since the district is geographically widespread, there is a need to communicate between the main site and, not only the other hospitals in the District, but also with the smaller health clinics, and general practitioners etc. In addition, access is required to external facilities such as the regional computer centre. There are also desirable links - - for example, it would be helpful to be able to access external databases from terminals within the district (although such usage has considerable implications from the charging and secudty aspects).
Implementation of the plan Implementation of the plan started in 1981 and is still proceeding.
computer communications
Indeed, as the plan is evolutionary, it will never have a definite date of completion. The details of the implementation are discussed in the following sections, with the emphasis being placed on the internal and extemal communications links, although, for completeness, some attention is paid to the other aspects of the strategic plan, such as the shared services. The current status is also defined.
Installation and assessment Selection of equipment Following the drawing up of the strategic plan and its acceptance by hospital, work proceeded on implementation. The first stage in this was the selection and procurement of equipment. Two major items were required - - the shared services system and the network itself. A second consultants' report was commisioned and a steering group set up, consisting of the consultants together with representatives from St Thomas' Hospital. This group made various recommendations which were approved by the District Management Team and work began on the procurement of the equipment. An operational requirement was written and a number of selection criteria defined as follows: • closeness to the operational requirement • understanding of operational requirement • estimated costs • reputation of supplier • level of support offered • expected delivery dates • range of equipment available • expansion capability • performance of prototype systems Two basic solutions emerged; the first was a PACX, the second a LAN (all the proposals of this type suggested an Ethemet). After a detailed evaluation, it was decided that the communications network would be an Ethernet 2.
vol 8 no 6 december 1985
There were many reasons for this choice, the most important of which were that Ethemet was a relatively well proven technology and had reached the stage of a marketable product that could be bought 'offthe-shelf'. The transmission medium is passive and is also well shielded, both attributes that were likely to be necessary in a hospital environment. It conformed to a formal standard 3 which had been proposed as a national standard 4 (and, more recently, as an International Standard5). Finally, there was the possibility of an 'upgrade path' from baseband transmission to broadband should this be required in the future. The DHSS was approached and asked to fund a Technical Study into the implementation of an Ethemet and this was approved in January 1982. An order was placed for a Net/One system from Ungermann-Bass and this was delivered in March 1982. The Net/One system Net/One presents the user with considerably more facilities than the basic Ethemet. At the lowest level, the system consists of a cable to the full Ethernet2 specification 3 allowing data transfer at 10 Mbit/s. A transceiver is required to transmit data onto the cable and this is clamped directly onto the cable (since the transmission medium is passive, transceivers may be added or removed while the network is in use). The transceiver is connected, in turn, to a network interface unit (N IU) which must be located within 50 m of the transceiver. The early NIUs were based on the Z-80 processor (although more recent versions have used 16-bit processors). The N l Us purchased for the Technical Study were NIU-2As, each of which contained up to four boards. Each board (known as an application processor board (APB)) had a Z-80A processor, 64 kbyte RAM and 4 kbyte ROM. The APB is interfaced to an I/O board which may handle, for example, four serial (RS-232) and two parallel connections. In fact, a variety of different I/O interfaces are available, including IEEE-488. For the technical study, five NIU-
2As were bought, mostly with a full complement of APBs. The I/O boards chosen were '4+2's (i.e. 4 serial and 2 parallel) or 'sixpac's (6 serial) and so each NIU could handle a maximum of 24 devices. A later version of the NIU based on the Fujitsu Ethemet chip set (N IU-1/50) became available in 1983 and was evaluated during Phase II. This NIU had six I/O ports. More recently, a number of other NIUs have become available (with the main processing being handled by an Intel 80186 processor rather than a Z-80A, allowing much greater throughput). The NlUs implement the basic Ethemet protocols but, in addition, provide the user with higher level facilities enabling for example, connection to a named resource elsewhere on the network. The NIUs also guarantee error free transmission through the network. Various types of service may be obtained through an NIU. The most common is access to the command interpreter which allows functions such as Connect to another port, Disconnect and Examine the status of a connection. A user who has access to the command interpreter has access to all the ports on the network but has to issue specific commands to make and break connections. There is a special class of user who has access to various additional functions, such as the ability to disconnect circuits between two other ports and to make 'third party connections'. Needless to say, access to these services is normally heavily restricted. A second service is that of the permanent virtual circuit (PVC). The network can be configured so that one port appears to be permanently connected to another. The user has no need to (and, indeed, cannot) issue network commands. A PVC reserves the network resources required and they cannot be accessed by another user. A further facility, introduced later, is known as a'demanded circuit' and this is discussed below. Initial tasting The technical study for the evaluation
309
of the network was intended to take 18 months to complete, but, for reasons discussed later, was extended by a further six months. Thus it terminated at the end of March 1984. It consisted of two phases. The first was intended to be a thorough test of the system (both hardware and software) in an experimental environment. When the system had been proved to be acceptable, it would be installed and then used in an operational environment. The equipment purchased for Phase I consisted of one Network Configuration Facility (NCF-1) using twin 8in floppy discs as the storage medium, five NIUs providing a total of 102 ports (most RS-232 although there were a few parallel ports) together with 500 m Ethemet cable with all the appropriate fittings. In fact, the equipment delivered did not quite conform to the order. In particular, the NCF-1 had been dropped during transit and an NCF-2 (with a hard disc instead of one of the floppy drives) was supplied on loan until a new NCF-1 could be shipped from the USA. The equipment was powered up and a fault discovered on one NIU. This was due to some chips on one board becoming loose during transit and so the fault was quickly rectified. The initial tests were extremely simple and consisted of connecting a terminal to each (serial) port on each NIU to ensure that the command interpreter could be accessed. The next set of tests involved two VDUs connected to different ports. Since testing was intended to be exhaustive, as many combinations of ports as possible were checked (although it was decided that testing the 5 000 or so possible combinations was too extreme) and as many facilities as possible were used. Thus, for each pair of ports, tests were conducted with both ports atthe same baud rate (over all possible baud rates), at different baud rates, at the same and different parities etc. In general, these tests were successful but a few problems did materialize (thus demonstrating the need for such exhaustive tests). In particular, a fault was discovered on the sixpac boards (i.e. those supporting six serial ports rather than the '4+2' - -
310
four serial and two parallel boards). This manifested itself by the echoing of a spurious character when local echo had been selected and this was found to occur at regular intervals, equal to the input buffer size. This was clearly due to a software error concerned with buffer pointers. Having made successful connections, disconnection tests were performed and these were successful, but a problem was discovered later. For each port, a sequence of characters is defined to be that which performs the disconnect function and this must clearly be chosen to be a sequence that would not normally occur. For most devices, the sequence '@END' Was chosen. While this was found to work when typed in via a terminal, it was later discovered that it did not work when sent by a machine, unless the machine paused between characters! The explanation seemed obvious and was presumably due to the speed of transmission. The third series of tests was concerned with flow control. Net/One allows ports to be configured with one of a variety of flow control mechanisms such as XON/XOFF or RS-232 line signalling. While the network appeared to perform correctly with the various types of flow control, a problem was discovered when connecting the network to the mainframe computer, the Sigma-6. This arose since the Sigma did not implement XON/XOFF flow control but, in addition, although the documentation suggested that it did implement line signalling, this was found notto bethe case. The problem was eventually solved by Honeywell (who maintained the Sigma), who supplied a 'patch' to the system software to implement RTS/CTS line signalling correctly. A further problem occured when a port was established without flow control and data was sent too rapidly to the NIU; instead of discarding characters, it wrote beyond the end of the buffer and 'hung'. This problem was solved in a later software release. Having established that the system appeared to perform satisfactorily (with a few problems) under normal circumstances, tests were conducted to simulate failure conditions. For example, while in the middle of
transmitting data, an N IU was powered off. In all cases, the appropriate recovery procedures were effected. In the test just mentioned, the NIU performed the normal power-up procedure (self diagnosis, followed by requesting the NCF to down-load its configuration) with all circuits prior to the 'crash' being terminated. Clearly, in an operational environment, this is not convenient, and the users would have to reconnect to the particular destination. The alternative of some third party (or the network itself) performing the reconnections would imply more centralized control than presented by Net/One. In any case, it is expected that such occurrences would be rare. During the initial tests, various problems were discovered, as outlined above. Some hardware faults were noted, and one in particular took a long time to clear since it seemed to be due to a combination of faults. The symptoms were that some boards of an NIU kept ceasing operation. A number of different solutions were proposed - - for example, the Ethemet cable is marked at 2.5 m intervals and NIUs should be connected to a mark, but, in our case, this had not been done. Rectification of this produced no permanent solution. Another proposal was that it was caused by the Ethemet cable being looped around on a drum; it was unwound without effect. Eventually, the problem was solved by replacing the transmitter board and a new version of the network software. Various faults were noticed in the software and, during Phase I, about eight were serious enough to be reported to the supplier. All were fixed in various later releases of the software, but, in some cases, a temporary patch was supplied prior to the next release. To summarize, the number of problems found during Phase I were much as expected. More faults were found in the newer devices (particularly the sixpac boards), and a major problem was the dearth of debugging tools or other diagnostic aids. Nevertheless, by the end of 1982, Phase I was deemed to have been concluded, and it was decided to implement Net/One as an operational
computer communications
network within the hospital.
Installation and commissioning All tests during Phase I had taken place in an experimental environment, namely in a corner of the computer room. Once it was clear that the product could operate successfully, the network cable was laid throughout the hospital (although a short piece of cable was retained in the computer room for testing p u r p o s e s - - f o r example, when a new release of the network software was received). Although this sounded a straightfowvard task, a number of problems were encountered. First, the cost of laying the cable was quite high and considerably more than laying the normal multiway data cables. This was undertaken by specialist outside contractors. Second, despite measuring the route of the cable as accurately as possible, the cable did not reach its destination; this was attributed to the fact that, owing to the rigidity of the cable, it was not possible to bend it and so significant amounts were 'lost' on corners. In fact, it was necessary to extend the cable by about 100 m to compensate for this (bringing its total length to 600 m which is more than the specification allows, although no problems have been noticed as a result). In addition, choosing sites for the NIUs was not as easy as expected. Various criteria had to be applied such as the site having power available (13 amp mains), being accessible to computer centre staff while being inacessible to the general public and being reasonably well ventilated. Nevertheless, within a relatively short time, a number of NIUs were distributed around the hospital. Some were retained in the computer room since all extant wiring terminated there, and there was little point in rerunning data cables (except where this would be cost-effective- for example, where the use of line drivers could be obviated). The first year of operation of the network within the district was designated Phase II of the Technical Study and, since the network operated successfully, there was a straight
vol 8 no 6 december 1985
transition to the full operational network (with the implication that, had Phase II not proved successful, other alternatives would have been explored).
Operational network The start of Phase II was timed to fit in with a programme of installing terminals on wards and the provision of a 24 h, 7 day enquiry service on the Sigma-6. So, as the network was extended, the major use was from these terminals accessing the enquiry service (the prototype bulletin board). However, the network was used for various other purposes such as providing a dedicated link (a Permannent Virtual Circuit) between a terminal on one ward and a microcomputer in the Department of Medicine. Therefore, as far as the operation of the hospital was concerned, implementation of Phase II provided, almost immediately, additional services. However, as faras the technical study was concerned, considerably more work was undertaken. In addition (and in parallel) work continued on implementing the strategic plan. Perhaps the first point to be noted is that, as the network was extended and more terminals added, these terminals were from a number of different manufacturers and had significantly different characteristics. The terminals included two models of Cifers, theTVI-925, Kokusai KDS-7362 (which is very similar to the TVl-950) and hardcopy terminals such as a DECwriter III and a Diablo. In all cases, no problems were encountered. Similarly, a number of microcomputers were attached (through RS232 connections), and, once again, no problems have been encountered. This is to be expected, since the nework has been used, in all these cases, merely as a replacement for a hardwired connection. Nevertheless, as mentioned above, one connection has been used in a live clinical environment on a 24 h a day,7 days a week basis since the middle of 1983. Connections to minicomputers have also proved trouble-free. A
Prime 300 computer was connected for early tests (but the machine was later sold). In addition, a twin GEC4190 configuration (bought as the shared services system and described below) was connected in late 1983. Finally, two mainframes have been connected. The first of these is the Sigma-@ and the work on this has been discussed above. This machine was replaced by the hardware-compatible Telefile T-85 in 1984. The ward terminals access the T-85 using a facility that was introduced into Net/One in 1983, known as a'demanded circuit'. This is similar to a Permanent Virtual Circuit but is not a permanent connection (and so does not use network resources when not being used). A port is configured for a demanded circuit to a particular destination (in this case, the Telefile), and, when any key is hit on the terminal connected to that port, a connection is made to the selected destination. The 'destination' is a network name and can therefore be a single port, or a group of ports, in which case, the first free one is used. Thus, the network may be used as a terminal multiplexer, and, at the time of writing, about 50 terminals have access to eight Telefile ports. In order to implement this satisfactorily, it is necessary to impose a relatively short time-out on these ports since, otherwise, they would rapidly become blocked. The second mainframe attached to the network temporarily was an ICL 2955. A large number of problems were encountered, mainly due to the fact that this machine used synchronous terminal protocols. Experiments to use the Net/One to connect the mainframe and its cluster controllers (DRS-20/33s) were partially successful. Connections were established but downloading the DRS-20s did not take place. The experiments were terminated but may be restarted at a later date. Similarly, access to the ICL machine using asynchronous links proved difficult, despite the installation of an ICL product known as a CMS-1 board. The major problems were concerned with flow control and it only proved possible to obtain reliable communications using 300 baud links. Again, these experiments
311
were terminated, but may be restarted. During Phase II, a number of hardware and software problems were encoutered. The number of hardware faults (mainly board failures) was considered reasonable and the revenue costs of the system were relatively low (first line maintenance is carded out at the hospital). Most of the software errors were still outstanding at the end of Phase II.
Network monitoring
A serious criticism of Net/One when first purchased was its dearth of monitoring and debugging tools. Indeed, the only two tools provided were the 'Network Debugger' which allowed a user to access memory locations in an NIU across the network and a debugging PROM on each NIU board. During Phase II, the network software improved and, in particular, a monitoring tool known as the Data Link Monitor (DIM) became available. A sample of its output is given in Figure 2 (see opposite). This enables traffic to be monitored and proved extremely valuable in pointing out a fault which might otherwise not have been discovered for some considerable time. When DLM was used initially, it was noted that the number of packet collisions was very high and this was traced to incorrect termination of the cable. Once rectified, the number of collisions dropped to almost zero (since, at present, the amount of network traffic is only a very small proportion of the total capacity). Within the last year, further monitoring tools have become available with the development of the NMC-1 (based on an IBM PC) and the NMC-2 (based on a Sun-l/20 mini-computer). Shared services system
The second major component of the strategic plan was a computer system to provide shared services on the network. An operational requirement was produced and over 50 companies requested copies. Eventually, four companies were short-listed, and,
312
after careful evaluation, it was decided to purchase a twin GEC-4190 configuration with two 275 Mbyte discs on each machine, an intercomputer link and a watchdog timer. The computers are thus interconnected, and each is connected to a number of NIUs, thus allowing relatively failsafe operation. The machines were delivered in mid1983 and it was anticipated that they would be in operation by mid1984. However, software that had been promised by the suppliers (in particular, Version 4 of 'Rapport', a relational database system, and Unix) were long delayed, and, in fact, were only delivered in mid1985. As a result, only prototype systems have been implemented on the GEC machines and it now seems unlikely that the operational systems will be implemented on these machines. Perhaps the most important system (especially from the point of view of network security) intended to be implemented on the shared sewices system was the Iogon sewer. A prototype was implemented on the GEC machines, but it was found that a number of problems existed, such as the speed of response. It was decided that it was unlikely that these could be overcome in the short term, and therefore a policy decision was taken to implement a similar, but much simpler, system in the network itself. This approach had been considered earlier but rejected owing to the relative instability of the network software-releases of the system were occurring at fairly frequent intervals and, although the suppliers guaranteed the interfaces to network sewices, the amount of spare memory was small and not guaranteed. However, in the meantime, the product had stabilized, and the combination of this and the problems discovered using the other approach led to the decision being taken to implement a simplified Iogon server within the network (on an NIU board). Each terminal that needs to use this sewice is a 'demanded circuit' to the Iogon sewer, and so, on hitting any key, a virtual circuit is set up to this sewice. In the current implementation, users are not required to give a password (since it is assumed that security will be handled in the target
system), but the Iogon server is aware of the location of the terminal. From this knowledge, it constructs a menu that lists all the network facilities to which that terminal has access (i.e. the user is not even made aware of sewices to which he has no access). The user selects the appropriate sewice by number and the Iogon sewer makes a 'third-party' connection to that sewice, disconnectingthe user from itself. When the user has finished with that sewice and disconnects (or is disconnected), the terminal reverts to the state of having a demanded circuit to the Iogon sewer. A prototype bulletin board was implemented on the GEC-4190s using Version 3 of Rapport, but, due to the delays in delivery of the next version, it was necessary to implement the operational bulletin board on the Telefile T-85. This system is currently being accessed by over 100 terminals and there are over 300 connections to the service each day. This figure is somewhat deceptive since certain terminals (such as the one in the Intensive Care Unit) are permanently connected whereas others (the majority) make a connection when it is necessary to make an enquiry. Perhaps a more pertinent figure is that the service makes more than a million disc accesses each week. The transfer of patient data from the Telefile 1--85 to the GEC-4190s was implemented using a simple end-to-end protocol which proved satisfactory. The general problem of connecting machines together through the network is still being examined and is discussed below. A prototype message board has been implemented using the proprietary electronic mall system Memo provided by Telefile. Work is proceeding on implementing the other shared sewices. External communications links
The need for external communications was recognized by the strategic plan and a number of these have been implemented. Most are straightforward and are point-to-point, although this is likely to change in the future.
computer communications
Data LinkMonitorl.0 NIU Send pkt 1933 10534359 1156 1971914 5634 1116138 Board Send pkt 1933A 5894289 1933B 7761244 1933C 4611906 1933D 3899152
Send pkt/s 21.0 1.6
Recvpkt 10919157 2258894 1228827
Recv pkt/s 21.0 1.6
2.8
2,8
Send pkt/s 1.6 0.6 13.4 4.8
Recvpkt 6667201
10091019
7604789 4372323
Recv pkt/s 1.6 0.6 13.4 4.7
Collisions 6697 2152
Tuesday August 13, 1985 15:54:29 CRC errors Active boards 0 ABCD 0 AB
1602
0
AB
CPU u t i l % 16.1 14.8 32.8 18.8
Response time (ms) 16 16 16 16
Net/One service VC VC VC VC
Command menu (Type ? for help) 1) Add line 4) Freezedisplay 2) Delete line 5) Unfreeze display 3) Clear data 6) Exit Select command number: 1 Address of NIU or board: 5634
Figure 2.
Sample DLM output
Telefile
T-85
300 bit/s [] Diol-up modem
c
4190
GEC 4190
EPS-8 link E3 To Finonce Dpt ---~
Minstrel- I
9600 bit/s D To Regional CC
9600 bills
~
__•
MicroAPL Spectrum
EPS-8 link
D To Tooting Bec __~
RCC
PLANET
districts
Network
IMS 8000
9600 bit/s D
0 bit/s
9600
To South Westn
Thomos' - ~
Hospitol Ethernet
Fortune --'-----~ ULCC
Word ond Clinic
Termino Is
I 6r~OOnmc0 bl !
bitls
,ooo
bitls
Repeoter
500 m spur
Figure 3.
,6oo
Current network schematic
vol 8 n o 6 d e c e m b e r 1985
Tooting 8ec Hospito I
Figure 4.
South Western Hospito I
Communications links
313
The first such link was to the Regional Computer Centre (at the site of the old Lambeth Hospital, some two miles away). Two ports are currently available, using an eightport statistical multiplexer (the remaining ports are available for expansion in the future) linked to a 9 600 bit/s modem and a leased line. The traffic on this link is relatively low, but nevertheless important. Additional links have been provided from St Thomas' Finance Department (just off the main site) to the Regional Computer Centre and to the Ethernet. The former is a 9 600 bit/s link, the latter an EPS-8 line. At present, links are being installed between St Thomas' and the two other hospitals in the district, South Western Hospital and Tooting Bec Hospital. Although it was originally intended to provide point-to-point links using statistical multiplexers and modems, it was finally decided to use Case DCX equipment, which provided similar facilities but with a number of additions. Many of the additions were already available on the Ethemet and therefore provided little additional functionality, but two were of particular use. The first was the ability to monitor line traffic; the second was the possibility of rerouting traffic from one port to another (even on the same site) using the DCX rather than using the Net/One facilities, and so, in the relatively unlikely event of the Net/One being unavailable, some work could continue. A final external link that should be discussed is that to the University of London Computer Centre (ULCC) network. This is provided by a link from the Net/One to a PAD in the Department of Community Medicine. This PAD is linked, not only to two local machines (aVAX and an LSI-11), but also to the ULCC network. Thus the link is simple, but effective. The questions of security and charging have not yet been addressed except to restrict access to the Community Medicine PAD to a very limited number of terminals and to rely on the security facilities of the connected systems. N e t w o r k protocols
divided into two areas - - t h e protocols used within the Net/One itself and the end-to-end protocols. Within the network, the protocols conform to the DEC/Intel/Xerox specification, although Ungermann-Bass intend moving over to IEEE-802.3 within the near future. The upper level protocols are proprietary, but it is intended to implement XNS protocols in the next release of the Network Operating System. These protocols provided an errorfree service from source NIU to destination NIU but not from end-toend (i.e. user system to user system). It should be pointed out that, in many cases, such a service is adequate and there is no need for further protocols. In the case of machine-to machine data transfers, it is necessary to implement a number of levels of protocol. First, it is necessary to implement an end-to-end transport service. Because of the intrinsic reliability of the network, this can be relatively simple. So far, such a service has been implemented in one case (transfer of data from the Telefile to the GEC-4190s) and is about to be implemented in another (Ferranti Argus to Telefile). In the former case, a simple, ad hoc, protocol was implemented with a I byte checksum and a simple acknowledgement. In the latter case, a similar, but different, protocol is being implemented. Clearly, the implementation of a number of different protocols is unsatisfactory and it is intended that international standard protocols6will be implemented in the near future, although this may not be possible in all cases if the manufacturer of the connected machine does not support such protocols. It is intended that such support be made
a mandatory provision of future operational requirements, but there are still a number of existing machines that raise problems. A yet higher-level protocol is being defined. This is an Application Level protocol which defines what data is being transmitted. Thus, for example, a record transmitted from the Haematology system needs to state that fact, then each individual test (together with its result and, possibly, reference ranges) needs to be defined. The design of such a protocol is based on each item carrying with it a descriptor of the item (thus implying a global data dictionary). The work on these protocols is expected to be complete in the near future and the connection of the Ferranti Argus system to the Telefile T85 should be operational soon afterwards. The initial phase of this will allow the transmission of completed Haematology results to the bulletin board. The second Phase (necessitating a two-way transfer) will allow the Haematology system to transmit a patient number to the bulletin board and to receive back the patient's details. The need for such machine to machine data transfers will become even more important with the advent of the 'KOrner recommendations' (requiring large amounts of management information to be transmitted) and 'management budgeting' (needing financial information to be transmitted).
Table 2. Devices attached to the network
Network configuration facility (Z-80 based) Sperry 'Personal Connection' NIU-2As (up to 24 ports each) NIU-1/50 (six ports) NIU-1/80s (eight ports) Ethemet cable
1 2 1 2 2 I 1 ~ 100
Telefile T-85 G EC-4190s Equinox IMS-8000 Fortunes HMS Minstrel-ls MicroAPL Spectrum CAMTEC PAD Ward/clinic terminals and printers
Current status The current status of the various Table 3. ment
1 9 1 7
Current network equip-
300 1 Repeater Providing a total of 218 serial (RS-232) and 8 parallel ports
The question of protocols can be
314
computer communications
aspects of the strategic plan have already been discussed and this section merely summarizes those. The Ungermann-Bass Net/One has been sucessfully installed within St Thomas' Hospital and consists of two segments of cable, one 600 m long, the other 300 m, interconnected by a local repeater. Connected to this network is a wide variety of devices, ranging from over 1OO dumb terminals (of many different types) through microcomputers and minis to the mainframe Telefile T-85. The current network schematic is shown in Figure 3 (see p 313) and a list of the attached devices is given in Table 2. Table 3 shows the network equipment currently in use. Currently, all connections are by means of RS-232 interfaces. The network is linked to a number of external sites and Figure 4 (see p 313) shows a schematic representation of those linkages. The shared services are currently not being provided as originally intended. A prototype Iogon server has been implemented within the network itself and prototype bulletin board and message board facilities are being provided by the Telefile T-85.
Future plans To a large extent, detailed future plans do not exist owing to the demand-led nature of the service (and the various uncertainties). However, in the short term, priority
vol 8 no 6 december 1985
will be given to examining other methods of connection to the network, ranging from the use of parallel ports to D/VIA linkages (Telefile have provided two Ethernet controller boards for the latter). The higher level network protocols will be fully defined and implemented. In the longer term, consideration will be given to the possibility of upgradingthe network to broadband, especially when requirements such as the transmission of digital images (e.g. X-ray) become clearer. External links will need to be reexamined as the need for such linkages increases (for example, by GPs) and the use of PSS considered.
Summary and conclusions The strategic plan for St Thomas' was technologically advanced but is being implemented successfully. There have been a number of problems, especially with the initial systems testing, but these have been overcome and the network is now in use on a 24 h, seven day a week basis with very few problems. The technical decisions taken appear to have been sound and the flexibility made available through this strategy will be to the considerable advantage of the Health Authority.
ledge the contribution of the Department of Health and Social Security who funded the Technical Study which has been described. He would also like to thank the various people who have contributed to the work on the network in St Thomas' Hospital, including the staff of the SharpeySchafer Centre, and the members of the Network Steering Group for their valuable suggestions.
References
2
3
4
5 6
Acknowledgements 7 The author would like to acknow-
Rosenthal, R M 'Network access techniques - - a review' Proc. NaL CompuL Vol 45 (1976) pp 495-500 Metcalfe, R M and Boggs, D R 'Ethernet: distributed packet switching for local computer networks' Commun. ACM Vol 19 (1976) pp 395-404 'The Ethernet: a local area network; data link layer and physical layer specifications (Version 2.0)', DEC/Intel/Xerox (1981) 'Local area network -- CSMA/CD Access Method and Physical Layer Specification' Recommendation 802.3, IEEE, USA (1984) 'CSMA/CD local area network: technical specification', DIS 8802/3, ISO, Switzerland (1984) 'lntemet transport protocols', XSIS 028112, Xerox Corporation, USA (December 1981) 'Transport Service Definition', DIS 8072, ISO, Switzerland (1984)
315
Adrian V Stokes describes the evolution and present day status of a network used by a London Health Authority to link three hospitals
Hospitals deal with large amounts of data, which need to be accesed by different internal departments, each with its own particular needs. When information is passed within a District Health Authority from hospital to hospital the situation is more complex. A network in a major London hospital with external links is described with particular emphasis on its evolution and the need for versatility with cost effectiveness. Future plans are also discussed. Keywords: computer networks, distributed processin& District Health Authority
Introduction St Thomas' Hospital (London, UK) has an extensive history of computing. Itwas one of the first hospitals in the UK to make major use of computers in a health care context. Its early computing capability (from the beginning of the 1970s) was based on a mainframe using an interactive system, but, towards the end of the 1970s, as a prelude to implementing its second generation of systems, a strategic plan was drawn up involving distributing functions to a decentralized network of various sized processors and interconnecting these processors by means of a comunications network. This paper gives the background to the computer project at St Thomas' Hospital, and indicates some of the problems of the District Health Authority which make a distributed strategy even more appealing. It also discusses the concepts of the strategic plan, and then outlines its implementation, from the initial selection of equipment to the present situation. The paper also discusses future plans.
The DHSS experimental projects
West Lambeth Health Authority, St Thomas' Hospital, Lambeth PalaceRoad, London SE1, UK
Followingthe Flowers Report in 1966, a Committee was set up at St Thomas' Hospital to look at the possible use of computers in the hospital, and this Committee recommended the setting up of a new Department within the
hospital for this purpose. The Departmerit of Health and Social Security (DHSS) supported this idea, and, in 1969, the 'ADP Project Team' was formed to conduct a feasibility study. In 1971, St Thomas' became part of the 'Coordinated Hospitals Project' in which four hospitals (Addenbrookes' Hospital Cambridge, Charing Cross Hospital, University College Hospital and St Thomas') were provided with the same computer with the intention of being able to exchange large amounts of software. The computer chosen was the Xerox Data Systems' Sigma-6 and it was installed at St Thomas' in late 1973. The initial configuration had 112 kbyte of main (core) memory and 200 Mbyte of disc storage, by 1976, it had 20 online terminals. Over the next few years, a large number of applications were written for the computer. Two major areas were c o v e r e d - patient administration and pathology laboratories - - as well as a number of smaller ones, such as nursing sickness and absence recording. In the late 1970s, the Sigma-6 was coming to the end of its useful life and a study was commissioned from external consultants to recommend the future of computing at St Thomas'. They produced a Strategic Plan for Computing which recommended a distributed approach. There were various reasons for this, including the uncertain financial climate in the National Health Service (NHS).
0140-3664/85/060306-10 $03.00 © 1985 Butterworth & Co (Publishers) Ltd 306
computer communications
Planning in an age of uncertainty St Thomas' is the largest hospital in the West Lambeth Health Authority (in London), which was created by the 1981 NHS reorganization. West Lambeth is one of 14 Health Authorities in the South East Thames Region and is at the far north-west of the Region, adjacent to the River Thames and opposite the Palace of Westminster. It is an inner city area and for a number of years has had a declining population. This trend is expected to slow down and the population at the end of the decade is expected to be the same as at present, although the age distribution will change. Within the District, there are now only three hospitals, St Thomas', the South Westem (at Stockwell) and Tooting Bec with approximately 900, 300 and 800 beds respectively. Although the District serves the needs of the local population of about 180 000, it also deals with a much wider catchment area, not only because of the traffic patterns in South London (especially with Waterloo railway station a few minutes walk from St Thomas'), but also because St Thomas' is a major teaching hospital, and as such has patients referred from all over the country and the world. Table 1 shows the workload of St Thomas' Hospital. It should be noted that there are records for about 750 000 patients, of which over 650 000 are recorded on the computer-based Patient Master Index. The latter is growing at the rate of about 30 000 per annum and the rate of increase is growing. There is very considerable uncertainty about the future. Although this applies generally to the NHS (and many other organizations), it applies especially to West Lambeth Health Authority. The Govemment has declared a policy of redistributing msources within the NHS, and, as a result, the South-East Thames Region will lose resources. Within the Region, West Lambeth will lose considerably more than a v e r a g e - about 15% of its revenue over the next ten years (i.e. over £1 M per year cumulatively). Similar considerations apply to capital
vol 8 no 6 december 1985
Table 1. Annual statistics of West Lambeth Health Authority 650 000 Patient records on computer (30 000 new each year) 290 000 Outpatient attendances (64 000 new each year) 14 000 Outpatient clinic sessions 35 000 Inpatient attendances 2 900 Day cases 64 000 A & E attendances 600 000 Pathology tests funding. The position is further complicated by the way in which funding occurs. In common with similar organizations, funding occurs on a yearby-year basis and there is no 'canyover' from year to year. Thus, a flexible strategy was required which took all these factors into account as well as the more obvious technical ones such as changes in computer technology and the emergence of local area networks and microcomputers. Such a strategy was drawn up in the late 1970s by a team consisting of external consultants, together with staff from St Thomas'. This was known as the Strategic Plan for Computing.
The strategic plan Since the plan was originally drawn up, it has evolved significantly. This section discusses the basic concepts of the plan, together with details of its different aspects. Where it is important, differences between the original plan and its current state are pointed out.
Basic concepts A hospital may be looked upon as a number of semiautonomous units which have considerable internal needs for computing and communication but with relatively few links to the outside. For example, from the point of view of a user, a pathology laboratory will receive a specimen, then produce a report. In fact, this report is the result of very extensive work, much of it involving computers.
Thus a model of hospital has a number of important features. First, there is a large number of semiautonomous activities which are physically distributed and which intercommunicate by means of a complex network, using media such as printed reports and the telephone. Second, there is the financial uncertainty referred to previously. There are two ways of implementing the model that has been outlined on a computer. First (classically), each separate activity may be a separate process on a large computer and these may be interconnected by interprocess calls and shared files. Second, each separate activity may take place on a different processor with these processors linked by a communications network. Both of these have the same end result, which can be summarized as the single objective of improving the planning, monitoring and management of the Health District. Of the two approaches mentioned, it was the second that was adopted by the plan. There were various justifications for this approach. In particular, it allowed an incremental changover, followed by incremental growth. Once the transition to a distributed network had occurred, there would be no need to replace a mainframe once every seven to ten years, but the various computers on the network could be replaced gradually, thus spreading out the capital costs. Furthermore, with such an approach, it is probably easier to integrate other machines into the system. Frequently, users within the hospital are able to purchase computers with, for example, research funds and then wish these computers to be connected to other systems (in particular, to the Patient Master Index). There is often little, if any, control over the computers chosen and so a highly flexible approach is needed and this can be provided by adopting a distributed strategy. It was realized that the strategy suggested had some disadvantages. In particular, the technology was considerably newer and problems could be envisaged as a result. Second, it was likely that the overall costs
307
Applications S~llam$ Shared Services
J
Off-site communications
Figure 1. Schematic representation of the strategic plan would be higher. Nevertheless, the advantages seemed to outweigh the disadvantages. The plan thus proposed that, at least in theory, each application should reside on a separate processor and that these processors should be interconnected. In practice, of course, it was realized that a number of different (but usually related) applications were likely to share the same machine but this did not detract from the general principle. However, using this approach, a number of problems are evident and these arise as a result of the (semi-) autonomy of the 'owners' of these applications. First, the data needed generally within the hospital was distributed over a number of processors which were not centrally controlled. Thus, it would be possible for data to be unavailable at critical times. Also, the method of accessing this data would vary from machine to machine (or, from the user's point of view, from application to application). In order to overcome these problems, the plan proposed that there should be a particular system on the network which provided any services which needed to be shared amongst the users of the network and this was called the shared services system. A final component of the plan was that of links to external systems (for example, the Regional Computer Centre). Figure 1 is a schematic representation of the strategic plan.
Communications network Perhaps the major component of the
308
plan is the main communications medium on the St Thomas' site. Without this, the plan could not operate. Clearly this communications medium had to be a local area network, but this term could include not only conventional LANs but also devices such as PACXs. The plan did not lay down specific criteria for such a network but made it clear that a high degree of flexibility and functionality was required. The selection of the network is discussed below.
Shared services system The shared services system was proposed to provide services that needed to be shared between all users within the District and various such functions were identified. First (and, probably, most important), it was necessary to provide a general enquiry service for all the commonly used data. This was designated the 'bulletin board' and was intended to provide a uniform method of access to all data required for the day-to-day running of the hospital. Clearly, different users (or classes of users) had different requirements for access to data and the bulletin board therefore had to implement a high degree of security. It could be characterized by the feature that the information providers were not able, at the time of providing the information, to specify to whom it was addressed but that this was to be done as a management function at an earlier stage. Thus, for example, it may be decided that all information from the nursing system sent to the bulletin board can only be accessed by nurses but no mechanism is provided for restricting access to information to a particular nurse. The second shared service was intended to provide that facility. It was designated the 'message board' and messages had to be addressed to specific recipients (i.e. an 'electronic mail' system). The 'message' could, in fact, be the same data as was sent to the bulletin board. If it were required to send a message to a number of recipients, one copy had to be sent to each recipient. The third service was called the 'logan server' and was intended to
provide a mechanism for controlling access to the network and for making the appropriate connections. When a single computer system (which may, in fact, comprise a number of different processors) is used, it is usual to 'log in' once only, then to access the particular facilities required. The user is unaware, if it is a rnultiprocessor system, on which processor his process is runnin~ nor does it make any difference. Just because the network is more loosely connected with dissimilar processors, there seems to be no reason why the user interface should be complicated. It was therefore decided to implement the logan server so as to emulate the interface that would be seen on a single machine. That is, the user would log in to the network and not to individual machines within that network. To some extent, the logan server was intended to perform the functions of a network access machine 1, but at a considerably less sophisticated level. More to the point, it was intended to be able to provide access control and security. Various other functions have since been identified for the shared services system but these are not relevant to the present discussion.
External communications Since the district is geographically widespread, there is a need to communicate between the main site and, not only the other hospitals in the District, but also with the smaller health clinics, and general practitioners etc. In addition, access is required to external facilities such as the regional computer centre. There are also desirable links - - for example, it would be helpful to be able to access external databases from terminals within the district (although such usage has considerable implications from the charging and secudty aspects).
Implementation of the plan Implementation of the plan started in 1981 and is still proceeding.
computer communications
Indeed, as the plan is evolutionary, it will never have a definite date of completion. The details of the implementation are discussed in the following sections, with the emphasis being placed on the internal and extemal communications links, although, for completeness, some attention is paid to the other aspects of the strategic plan, such as the shared services. The current status is also defined.
Installation and assessment Selection of equipment Following the drawing up of the strategic plan and its acceptance by hospital, work proceeded on implementation. The first stage in this was the selection and procurement of equipment. Two major items were required - - the shared services system and the network itself. A second consultants' report was commisioned and a steering group set up, consisting of the consultants together with representatives from St Thomas' Hospital. This group made various recommendations which were approved by the District Management Team and work began on the procurement of the equipment. An operational requirement was written and a number of selection criteria defined as follows: • closeness to the operational requirement • understanding of operational requirement • estimated costs • reputation of supplier • level of support offered • expected delivery dates • range of equipment available • expansion capability • performance of prototype systems Two basic solutions emerged; the first was a PACX, the second a LAN (all the proposals of this type suggested an Ethemet). After a detailed evaluation, it was decided that the communications network would be an Ethernet 2.
vol 8 no 6 december 1985
There were many reasons for this choice, the most important of which were that Ethemet was a relatively well proven technology and had reached the stage of a marketable product that could be bought 'offthe-shelf'. The transmission medium is passive and is also well shielded, both attributes that were likely to be necessary in a hospital environment. It conformed to a formal standard 3 which had been proposed as a national standard 4 (and, more recently, as an International Standard5). Finally, there was the possibility of an 'upgrade path' from baseband transmission to broadband should this be required in the future. The DHSS was approached and asked to fund a Technical Study into the implementation of an Ethemet and this was approved in January 1982. An order was placed for a Net/One system from Ungermann-Bass and this was delivered in March 1982. The Net/One system Net/One presents the user with considerably more facilities than the basic Ethemet. At the lowest level, the system consists of a cable to the full Ethernet2 specification 3 allowing data transfer at 10 Mbit/s. A transceiver is required to transmit data onto the cable and this is clamped directly onto the cable (since the transmission medium is passive, transceivers may be added or removed while the network is in use). The transceiver is connected, in turn, to a network interface unit (N IU) which must be located within 50 m of the transceiver. The early NIUs were based on the Z-80 processor (although more recent versions have used 16-bit processors). The N l Us purchased for the Technical Study were NIU-2As, each of which contained up to four boards. Each board (known as an application processor board (APB)) had a Z-80A processor, 64 kbyte RAM and 4 kbyte ROM. The APB is interfaced to an I/O board which may handle, for example, four serial (RS-232) and two parallel connections. In fact, a variety of different I/O interfaces are available, including IEEE-488. For the technical study, five NIU-
2As were bought, mostly with a full complement of APBs. The I/O boards chosen were '4+2's (i.e. 4 serial and 2 parallel) or 'sixpac's (6 serial) and so each NIU could handle a maximum of 24 devices. A later version of the NIU based on the Fujitsu Ethemet chip set (N IU-1/50) became available in 1983 and was evaluated during Phase II. This NIU had six I/O ports. More recently, a number of other NIUs have become available (with the main processing being handled by an Intel 80186 processor rather than a Z-80A, allowing much greater throughput). The NlUs implement the basic Ethemet protocols but, in addition, provide the user with higher level facilities enabling for example, connection to a named resource elsewhere on the network. The NIUs also guarantee error free transmission through the network. Various types of service may be obtained through an NIU. The most common is access to the command interpreter which allows functions such as Connect to another port, Disconnect and Examine the status of a connection. A user who has access to the command interpreter has access to all the ports on the network but has to issue specific commands to make and break connections. There is a special class of user who has access to various additional functions, such as the ability to disconnect circuits between two other ports and to make 'third party connections'. Needless to say, access to these services is normally heavily restricted. A second service is that of the permanent virtual circuit (PVC). The network can be configured so that one port appears to be permanently connected to another. The user has no need to (and, indeed, cannot) issue network commands. A PVC reserves the network resources required and they cannot be accessed by another user. A further facility, introduced later, is known as a'demanded circuit' and this is discussed below. Initial tasting The technical study for the evaluation
309
of the network was intended to take 18 months to complete, but, for reasons discussed later, was extended by a further six months. Thus it terminated at the end of March 1984. It consisted of two phases. The first was intended to be a thorough test of the system (both hardware and software) in an experimental environment. When the system had been proved to be acceptable, it would be installed and then used in an operational environment. The equipment purchased for Phase I consisted of one Network Configuration Facility (NCF-1) using twin 8in floppy discs as the storage medium, five NIUs providing a total of 102 ports (most RS-232 although there were a few parallel ports) together with 500 m Ethemet cable with all the appropriate fittings. In fact, the equipment delivered did not quite conform to the order. In particular, the NCF-1 had been dropped during transit and an NCF-2 (with a hard disc instead of one of the floppy drives) was supplied on loan until a new NCF-1 could be shipped from the USA. The equipment was powered up and a fault discovered on one NIU. This was due to some chips on one board becoming loose during transit and so the fault was quickly rectified. The initial tests were extremely simple and consisted of connecting a terminal to each (serial) port on each NIU to ensure that the command interpreter could be accessed. The next set of tests involved two VDUs connected to different ports. Since testing was intended to be exhaustive, as many combinations of ports as possible were checked (although it was decided that testing the 5 000 or so possible combinations was too extreme) and as many facilities as possible were used. Thus, for each pair of ports, tests were conducted with both ports atthe same baud rate (over all possible baud rates), at different baud rates, at the same and different parities etc. In general, these tests were successful but a few problems did materialize (thus demonstrating the need for such exhaustive tests). In particular, a fault was discovered on the sixpac boards (i.e. those supporting six serial ports rather than the '4+2' - -
310
four serial and two parallel boards). This manifested itself by the echoing of a spurious character when local echo had been selected and this was found to occur at regular intervals, equal to the input buffer size. This was clearly due to a software error concerned with buffer pointers. Having made successful connections, disconnection tests were performed and these were successful, but a problem was discovered later. For each port, a sequence of characters is defined to be that which performs the disconnect function and this must clearly be chosen to be a sequence that would not normally occur. For most devices, the sequence '@END' Was chosen. While this was found to work when typed in via a terminal, it was later discovered that it did not work when sent by a machine, unless the machine paused between characters! The explanation seemed obvious and was presumably due to the speed of transmission. The third series of tests was concerned with flow control. Net/One allows ports to be configured with one of a variety of flow control mechanisms such as XON/XOFF or RS-232 line signalling. While the network appeared to perform correctly with the various types of flow control, a problem was discovered when connecting the network to the mainframe computer, the Sigma-6. This arose since the Sigma did not implement XON/XOFF flow control but, in addition, although the documentation suggested that it did implement line signalling, this was found notto bethe case. The problem was eventually solved by Honeywell (who maintained the Sigma), who supplied a 'patch' to the system software to implement RTS/CTS line signalling correctly. A further problem occured when a port was established without flow control and data was sent too rapidly to the NIU; instead of discarding characters, it wrote beyond the end of the buffer and 'hung'. This problem was solved in a later software release. Having established that the system appeared to perform satisfactorily (with a few problems) under normal circumstances, tests were conducted to simulate failure conditions. For example, while in the middle of
transmitting data, an N IU was powered off. In all cases, the appropriate recovery procedures were effected. In the test just mentioned, the NIU performed the normal power-up procedure (self diagnosis, followed by requesting the NCF to down-load its configuration) with all circuits prior to the 'crash' being terminated. Clearly, in an operational environment, this is not convenient, and the users would have to reconnect to the particular destination. The alternative of some third party (or the network itself) performing the reconnections would imply more centralized control than presented by Net/One. In any case, it is expected that such occurrences would be rare. During the initial tests, various problems were discovered, as outlined above. Some hardware faults were noted, and one in particular took a long time to clear since it seemed to be due to a combination of faults. The symptoms were that some boards of an NIU kept ceasing operation. A number of different solutions were proposed - - for example, the Ethemet cable is marked at 2.5 m intervals and NIUs should be connected to a mark, but, in our case, this had not been done. Rectification of this produced no permanent solution. Another proposal was that it was caused by the Ethemet cable being looped around on a drum; it was unwound without effect. Eventually, the problem was solved by replacing the transmitter board and a new version of the network software. Various faults were noticed in the software and, during Phase I, about eight were serious enough to be reported to the supplier. All were fixed in various later releases of the software, but, in some cases, a temporary patch was supplied prior to the next release. To summarize, the number of problems found during Phase I were much as expected. More faults were found in the newer devices (particularly the sixpac boards), and a major problem was the dearth of debugging tools or other diagnostic aids. Nevertheless, by the end of 1982, Phase I was deemed to have been concluded, and it was decided to implement Net/One as an operational
computer communications
network within the hospital.
Installation and commissioning All tests during Phase I had taken place in an experimental environment, namely in a corner of the computer room. Once it was clear that the product could operate successfully, the network cable was laid throughout the hospital (although a short piece of cable was retained in the computer room for testing p u r p o s e s - - f o r example, when a new release of the network software was received). Although this sounded a straightfowvard task, a number of problems were encountered. First, the cost of laying the cable was quite high and considerably more than laying the normal multiway data cables. This was undertaken by specialist outside contractors. Second, despite measuring the route of the cable as accurately as possible, the cable did not reach its destination; this was attributed to the fact that, owing to the rigidity of the cable, it was not possible to bend it and so significant amounts were 'lost' on corners. In fact, it was necessary to extend the cable by about 100 m to compensate for this (bringing its total length to 600 m which is more than the specification allows, although no problems have been noticed as a result). In addition, choosing sites for the NIUs was not as easy as expected. Various criteria had to be applied such as the site having power available (13 amp mains), being accessible to computer centre staff while being inacessible to the general public and being reasonably well ventilated. Nevertheless, within a relatively short time, a number of NIUs were distributed around the hospital. Some were retained in the computer room since all extant wiring terminated there, and there was little point in rerunning data cables (except where this would be cost-effective- for example, where the use of line drivers could be obviated). The first year of operation of the network within the district was designated Phase II of the Technical Study and, since the network operated successfully, there was a straight
vol 8 no 6 december 1985
transition to the full operational network (with the implication that, had Phase II not proved successful, other alternatives would have been explored).
Operational network The start of Phase II was timed to fit in with a programme of installing terminals on wards and the provision of a 24 h, 7 day enquiry service on the Sigma-6. So, as the network was extended, the major use was from these terminals accessing the enquiry service (the prototype bulletin board). However, the network was used for various other purposes such as providing a dedicated link (a Permannent Virtual Circuit) between a terminal on one ward and a microcomputer in the Department of Medicine. Therefore, as far as the operation of the hospital was concerned, implementation of Phase II provided, almost immediately, additional services. However, as faras the technical study was concerned, considerably more work was undertaken. In addition (and in parallel) work continued on implementing the strategic plan. Perhaps the first point to be noted is that, as the network was extended and more terminals added, these terminals were from a number of different manufacturers and had significantly different characteristics. The terminals included two models of Cifers, theTVI-925, Kokusai KDS-7362 (which is very similar to the TVl-950) and hardcopy terminals such as a DECwriter III and a Diablo. In all cases, no problems were encountered. Similarly, a number of microcomputers were attached (through RS232 connections), and, once again, no problems have been encountered. This is to be expected, since the nework has been used, in all these cases, merely as a replacement for a hardwired connection. Nevertheless, as mentioned above, one connection has been used in a live clinical environment on a 24 h a day,7 days a week basis since the middle of 1983. Connections to minicomputers have also proved trouble-free. A
Prime 300 computer was connected for early tests (but the machine was later sold). In addition, a twin GEC4190 configuration (bought as the shared services system and described below) was connected in late 1983. Finally, two mainframes have been connected. The first of these is the Sigma-@ and the work on this has been discussed above. This machine was replaced by the hardware-compatible Telefile T-85 in 1984. The ward terminals access the T-85 using a facility that was introduced into Net/One in 1983, known as a'demanded circuit'. This is similar to a Permanent Virtual Circuit but is not a permanent connection (and so does not use network resources when not being used). A port is configured for a demanded circuit to a particular destination (in this case, the Telefile), and, when any key is hit on the terminal connected to that port, a connection is made to the selected destination. The 'destination' is a network name and can therefore be a single port, or a group of ports, in which case, the first free one is used. Thus, the network may be used as a terminal multiplexer, and, at the time of writing, about 50 terminals have access to eight Telefile ports. In order to implement this satisfactorily, it is necessary to impose a relatively short time-out on these ports since, otherwise, they would rapidly become blocked. The second mainframe attached to the network temporarily was an ICL 2955. A large number of problems were encountered, mainly due to the fact that this machine used synchronous terminal protocols. Experiments to use the Net/One to connect the mainframe and its cluster controllers (DRS-20/33s) were partially successful. Connections were established but downloading the DRS-20s did not take place. The experiments were terminated but may be restarted at a later date. Similarly, access to the ICL machine using asynchronous links proved difficult, despite the installation of an ICL product known as a CMS-1 board. The major problems were concerned with flow control and it only proved possible to obtain reliable communications using 300 baud links. Again, these experiments
311
were terminated, but may be restarted. During Phase II, a number of hardware and software problems were encoutered. The number of hardware faults (mainly board failures) was considered reasonable and the revenue costs of the system were relatively low (first line maintenance is carded out at the hospital). Most of the software errors were still outstanding at the end of Phase II.
Network monitoring
A serious criticism of Net/One when first purchased was its dearth of monitoring and debugging tools. Indeed, the only two tools provided were the 'Network Debugger' which allowed a user to access memory locations in an NIU across the network and a debugging PROM on each NIU board. During Phase II, the network software improved and, in particular, a monitoring tool known as the Data Link Monitor (DIM) became available. A sample of its output is given in Figure 2 (see opposite). This enables traffic to be monitored and proved extremely valuable in pointing out a fault which might otherwise not have been discovered for some considerable time. When DLM was used initially, it was noted that the number of packet collisions was very high and this was traced to incorrect termination of the cable. Once rectified, the number of collisions dropped to almost zero (since, at present, the amount of network traffic is only a very small proportion of the total capacity). Within the last year, further monitoring tools have become available with the development of the NMC-1 (based on an IBM PC) and the NMC-2 (based on a Sun-l/20 mini-computer). Shared services system
The second major component of the strategic plan was a computer system to provide shared services on the network. An operational requirement was produced and over 50 companies requested copies. Eventually, four companies were short-listed, and,
312
after careful evaluation, it was decided to purchase a twin GEC-4190 configuration with two 275 Mbyte discs on each machine, an intercomputer link and a watchdog timer. The computers are thus interconnected, and each is connected to a number of NIUs, thus allowing relatively failsafe operation. The machines were delivered in mid1983 and it was anticipated that they would be in operation by mid1984. However, software that had been promised by the suppliers (in particular, Version 4 of 'Rapport', a relational database system, and Unix) were long delayed, and, in fact, were only delivered in mid1985. As a result, only prototype systems have been implemented on the GEC machines and it now seems unlikely that the operational systems will be implemented on these machines. Perhaps the most important system (especially from the point of view of network security) intended to be implemented on the shared sewices system was the Iogon sewer. A prototype was implemented on the GEC machines, but it was found that a number of problems existed, such as the speed of response. It was decided that it was unlikely that these could be overcome in the short term, and therefore a policy decision was taken to implement a similar, but much simpler, system in the network itself. This approach had been considered earlier but rejected owing to the relative instability of the network software-releases of the system were occurring at fairly frequent intervals and, although the suppliers guaranteed the interfaces to network sewices, the amount of spare memory was small and not guaranteed. However, in the meantime, the product had stabilized, and the combination of this and the problems discovered using the other approach led to the decision being taken to implement a simplified Iogon server within the network (on an NIU board). Each terminal that needs to use this sewice is a 'demanded circuit' to the Iogon sewer, and so, on hitting any key, a virtual circuit is set up to this sewice. In the current implementation, users are not required to give a password (since it is assumed that security will be handled in the target
system), but the Iogon server is aware of the location of the terminal. From this knowledge, it constructs a menu that lists all the network facilities to which that terminal has access (i.e. the user is not even made aware of sewices to which he has no access). The user selects the appropriate sewice by number and the Iogon sewer makes a 'third-party' connection to that sewice, disconnectingthe user from itself. When the user has finished with that sewice and disconnects (or is disconnected), the terminal reverts to the state of having a demanded circuit to the Iogon sewer. A prototype bulletin board was implemented on the GEC-4190s using Version 3 of Rapport, but, due to the delays in delivery of the next version, it was necessary to implement the operational bulletin board on the Telefile T-85. This system is currently being accessed by over 100 terminals and there are over 300 connections to the service each day. This figure is somewhat deceptive since certain terminals (such as the one in the Intensive Care Unit) are permanently connected whereas others (the majority) make a connection when it is necessary to make an enquiry. Perhaps a more pertinent figure is that the service makes more than a million disc accesses each week. The transfer of patient data from the Telefile 1--85 to the GEC-4190s was implemented using a simple end-to-end protocol which proved satisfactory. The general problem of connecting machines together through the network is still being examined and is discussed below. A prototype message board has been implemented using the proprietary electronic mall system Memo provided by Telefile. Work is proceeding on implementing the other shared sewices. External communications links
The need for external communications was recognized by the strategic plan and a number of these have been implemented. Most are straightforward and are point-to-point, although this is likely to change in the future.
computer communications
Data LinkMonitorl.0 NIU Send pkt 1933 10534359 1156 1971914 5634 1116138 Board Send pkt 1933A 5894289 1933B 7761244 1933C 4611906 1933D 3899152
Send pkt/s 21.0 1.6
Recvpkt 10919157 2258894 1228827
Recv pkt/s 21.0 1.6
2.8
2,8
Send pkt/s 1.6 0.6 13.4 4.8
Recvpkt 6667201
10091019
7604789 4372323
Recv pkt/s 1.6 0.6 13.4 4.7
Collisions 6697 2152
Tuesday August 13, 1985 15:54:29 CRC errors Active boards 0 ABCD 0 AB
1602
0
AB
CPU u t i l % 16.1 14.8 32.8 18.8
Response time (ms) 16 16 16 16
Net/One service VC VC VC VC
Command menu (Type ? for help) 1) Add line 4) Freezedisplay 2) Delete line 5) Unfreeze display 3) Clear data 6) Exit Select command number: 1 Address of NIU or board: 5634
Figure 2.
Sample DLM output
Telefile
T-85
300 bit/s [] Diol-up modem
c
4190
GEC 4190
EPS-8 link E3 To Finonce Dpt ---~
Minstrel- I
9600 bit/s D To Regional CC
9600 bills
~
__•
MicroAPL Spectrum
EPS-8 link
D To Tooting Bec __~
RCC
PLANET
districts
Network
IMS 8000
9600 bit/s D
0 bit/s
9600
To South Westn
Thomos' - ~
Hospitol Ethernet
Fortune --'-----~ ULCC
Word ond Clinic
Termino Is
I 6r~OOnmc0 bl !
bitls
,ooo
bitls
Repeoter
500 m spur
Figure 3.
,6oo
Current network schematic
vol 8 n o 6 d e c e m b e r 1985
Tooting 8ec Hospito I
Figure 4.
South Western Hospito I
Communications links
313
The first such link was to the Regional Computer Centre (at the site of the old Lambeth Hospital, some two miles away). Two ports are currently available, using an eightport statistical multiplexer (the remaining ports are available for expansion in the future) linked to a 9 600 bit/s modem and a leased line. The traffic on this link is relatively low, but nevertheless important. Additional links have been provided from St Thomas' Finance Department (just off the main site) to the Regional Computer Centre and to the Ethernet. The former is a 9 600 bit/s link, the latter an EPS-8 line. At present, links are being installed between St Thomas' and the two other hospitals in the district, South Western Hospital and Tooting Bec Hospital. Although it was originally intended to provide point-to-point links using statistical multiplexers and modems, it was finally decided to use Case DCX equipment, which provided similar facilities but with a number of additions. Many of the additions were already available on the Ethemet and therefore provided little additional functionality, but two were of particular use. The first was the ability to monitor line traffic; the second was the possibility of rerouting traffic from one port to another (even on the same site) using the DCX rather than using the Net/One facilities, and so, in the relatively unlikely event of the Net/One being unavailable, some work could continue. A final external link that should be discussed is that to the University of London Computer Centre (ULCC) network. This is provided by a link from the Net/One to a PAD in the Department of Community Medicine. This PAD is linked, not only to two local machines (aVAX and an LSI-11), but also to the ULCC network. Thus the link is simple, but effective. The questions of security and charging have not yet been addressed except to restrict access to the Community Medicine PAD to a very limited number of terminals and to rely on the security facilities of the connected systems. N e t w o r k protocols
divided into two areas - - t h e protocols used within the Net/One itself and the end-to-end protocols. Within the network, the protocols conform to the DEC/Intel/Xerox specification, although Ungermann-Bass intend moving over to IEEE-802.3 within the near future. The upper level protocols are proprietary, but it is intended to implement XNS protocols in the next release of the Network Operating System. These protocols provided an errorfree service from source NIU to destination NIU but not from end-toend (i.e. user system to user system). It should be pointed out that, in many cases, such a service is adequate and there is no need for further protocols. In the case of machine-to machine data transfers, it is necessary to implement a number of levels of protocol. First, it is necessary to implement an end-to-end transport service. Because of the intrinsic reliability of the network, this can be relatively simple. So far, such a service has been implemented in one case (transfer of data from the Telefile to the GEC-4190s) and is about to be implemented in another (Ferranti Argus to Telefile). In the former case, a simple, ad hoc, protocol was implemented with a I byte checksum and a simple acknowledgement. In the latter case, a similar, but different, protocol is being implemented. Clearly, the implementation of a number of different protocols is unsatisfactory and it is intended that international standard protocols6will be implemented in the near future, although this may not be possible in all cases if the manufacturer of the connected machine does not support such protocols. It is intended that such support be made
a mandatory provision of future operational requirements, but there are still a number of existing machines that raise problems. A yet higher-level protocol is being defined. This is an Application Level protocol which defines what data is being transmitted. Thus, for example, a record transmitted from the Haematology system needs to state that fact, then each individual test (together with its result and, possibly, reference ranges) needs to be defined. The design of such a protocol is based on each item carrying with it a descriptor of the item (thus implying a global data dictionary). The work on these protocols is expected to be complete in the near future and the connection of the Ferranti Argus system to the Telefile T85 should be operational soon afterwards. The initial phase of this will allow the transmission of completed Haematology results to the bulletin board. The second Phase (necessitating a two-way transfer) will allow the Haematology system to transmit a patient number to the bulletin board and to receive back the patient's details. The need for such machine to machine data transfers will become even more important with the advent of the 'KOrner recommendations' (requiring large amounts of management information to be transmitted) and 'management budgeting' (needing financial information to be transmitted).
Table 2. Devices attached to the network
Network configuration facility (Z-80 based) Sperry 'Personal Connection' NIU-2As (up to 24 ports each) NIU-1/50 (six ports) NIU-1/80s (eight ports) Ethemet cable
1 2 1 2 2 I 1 ~ 100
Telefile T-85 G EC-4190s Equinox IMS-8000 Fortunes HMS Minstrel-ls MicroAPL Spectrum CAMTEC PAD Ward/clinic terminals and printers
Current status The current status of the various Table 3. ment
1 9 1 7
Current network equip-
300 1 Repeater Providing a total of 218 serial (RS-232) and 8 parallel ports
The question of protocols can be
314
computer communications
aspects of the strategic plan have already been discussed and this section merely summarizes those. The Ungermann-Bass Net/One has been sucessfully installed within St Thomas' Hospital and consists of two segments of cable, one 600 m long, the other 300 m, interconnected by a local repeater. Connected to this network is a wide variety of devices, ranging from over 1OO dumb terminals (of many different types) through microcomputers and minis to the mainframe Telefile T-85. The current network schematic is shown in Figure 3 (see p 313) and a list of the attached devices is given in Table 2. Table 3 shows the network equipment currently in use. Currently, all connections are by means of RS-232 interfaces. The network is linked to a number of external sites and Figure 4 (see p 313) shows a schematic representation of those linkages. The shared services are currently not being provided as originally intended. A prototype Iogon server has been implemented within the network itself and prototype bulletin board and message board facilities are being provided by the Telefile T-85.
Future plans To a large extent, detailed future plans do not exist owing to the demand-led nature of the service (and the various uncertainties). However, in the short term, priority
vol 8 no 6 december 1985
will be given to examining other methods of connection to the network, ranging from the use of parallel ports to D/VIA linkages (Telefile have provided two Ethernet controller boards for the latter). The higher level network protocols will be fully defined and implemented. In the longer term, consideration will be given to the possibility of upgradingthe network to broadband, especially when requirements such as the transmission of digital images (e.g. X-ray) become clearer. External links will need to be reexamined as the need for such linkages increases (for example, by GPs) and the use of PSS considered.
Summary and conclusions The strategic plan for St Thomas' was technologically advanced but is being implemented successfully. There have been a number of problems, especially with the initial systems testing, but these have been overcome and the network is now in use on a 24 h, seven day a week basis with very few problems. The technical decisions taken appear to have been sound and the flexibility made available through this strategy will be to the considerable advantage of the Health Authority.
ledge the contribution of the Department of Health and Social Security who funded the Technical Study which has been described. He would also like to thank the various people who have contributed to the work on the network in St Thomas' Hospital, including the staff of the SharpeySchafer Centre, and the members of the Network Steering Group for their valuable suggestions.
References
2
3
4
5 6
Acknowledgements 7 The author would like to acknow-
Rosenthal, R M 'Network access techniques - - a review' Proc. NaL CompuL Vol 45 (1976) pp 495-500 Metcalfe, R M and Boggs, D R 'Ethernet: distributed packet switching for local computer networks' Commun. ACM Vol 19 (1976) pp 395-404 'The Ethernet: a local area network; data link layer and physical layer specifications (Version 2.0)', DEC/Intel/Xerox (1981) 'Local area network -- CSMA/CD Access Method and Physical Layer Specification' Recommendation 802.3, IEEE, USA (1984) 'CSMA/CD local area network: technical specification', DIS 8802/3, ISO, Switzerland (1984) 'lntemet transport protocols', XSIS 028112, Xerox Corporation, USA (December 1981) 'Transport Service Definition', DIS 8072, ISO, Switzerland (1984)
315