- Email: [email protected]
Control Hierarchy in a Distributed Process Control System
Copyright © IFAC Software for Computer Control Madrid, Spain 1982
SOITW ARE FOR DISTRIBUTED CONTROL SYSTEMS I
CONTROL HIERARCHY IN A DISTRIBUTED PROCESS CONTROL SYSTEM M. J. Shah and F. Schneider IBM Corporation, San Jose, California, USA
Abstract: ~apid advances in microprocessor based controllers and minicomputers have presented opportun1t1es ~or tot~l control of a large plant complex cluster. A hardware/software architect~retf~r a h~era~ch1cal co~trol system for overall control and data base management of such a p a~ 1n ,a d~str~buted env1ronment, is presented in this paper. The software problems of funct1~n d1str1but10n, communication bottlenecks, data base responsibility and engineer/operator 1nterface are prese~ted , together with an approach for their resolut i on . Design approaches for software automat10n 1n this environment are presented. Keywords.Distributed control; hierarchical control; Software automation; Computer control .
I NTRODUCTI ON
interacting plant units, or overall reai time plant data base management and optimization.
For the past few years in process industry there has been a trend for clustering several plant units together in a single complex for the sake of economic efficiency as well as for better coordination between various plant units. Thus, for example, modern refineries are in reality a complex cluster of interacting individual units producing a multitude of products.
This paper, then, explores a software system architecture that addresses the challenges of optimal control of a large complex of interacting plant units whether they be continuous or discrete. The environment considered here is a complex of plant units each with its own control system .
Similarly, a fertilizer plant complex sequentially produces ammonia, urea, nitric acid, sodium nitrate, phosphates, etc .. Most manufacturing plants today consist of a mixture of continuous operations as well as discrete operations and they are both interacting in one form or another.
The units may be close or considerable distance apart, thus necessitating sophisticated communications with high and low speed lines, keeping in mind the problems of real time computer control of remote plant units and their dependence on the total distributed system.
As the plant cluster grows, the complexity of control of the interacting plant units increases and the need for an overall real time data base for the entire plant for reporting, trouble shooting, optimization and other uses, become more acute. The high cost of energy and competitive forces on product cost have forced the manufacturing industries to efficiency in design of individual plant units; however, the ' interacting control of individual plant units still remains a challenge :
GENERAL SYSTEM REQUIREMENTS It is worthwhile first to list system requirements for distributed control of a large plant complex. Compl ete operator control for a plant unit, uninterrupted even when system communication breakdown occurs. Engineer/operator access for plant information data as well as control of multiple units from a single supervisory control room.
Several vendors advertize distributed computer control in one form or another. In general, these approaches distribute computer workload, such as control, display processing and communication between processors, more or less for control of a single unit. However, to-date, there has been no published work on distributed systems which describe a cohesive software architecture that meets the challenges of
Sensor input /output connection to local measurements or remote attachment to microprocessor controllers. Unified software scheme for variable definition irrespective of hardware connection.
133
134
M. J. Shah and F. Schneider
Direct digital control and Supervisory control at plant unit level with communication for Supervisory and optimum set point calculation from a higher level system. For discrete and batch processes in a unit, access to a prepared data base to activate for example a new batch or test. Provide the same data base for transfer of discrete data logs. Access to a large data base for continuous transfer of plant operational data for each of the desired variables in the plant unit, for historic, trend and logging purposes. Mechanism for retrieving the transferred data for plots, as well as calculations suc~ ~s r~gression analysis, modeling, optlmlzatlon and production purposes. Ability to modify operational data base at the plant unit level with integrity checking for operator/engineer entries as well as ability to occasionally add and activate new variables without disturbing the rest of the plant unit control system. Ability to add a new unit control system (continuous or discrete) without affecting other units. Separable engineer/operator interface for accessing and modifying different parts of the control characteristics for each varia~e. For exam~e, only an engineer with proper password authorization may modify tuning constants, whereas an operator may vary setpoints, limits ro 11 over to ma nua 1 contro 1, etc. Tracking of all operator/engineer actions by logging them on a large data base. Provide all levels of control from the basic monitoring of variables to sophisticated interactive control such as cascade, feed forward and multivariable contro 1 . Minimize programming requirement by providing: a. Fill-in the blanks (FITB) interface for variables, for all modes of contro 1 . b. FITB interface for specifying different computer hardware configurations for automatic system generation, for each unit level and all communication systems between the distributed systems. c. A "Control Systems non programmers I language" for unique control algorithms/ calculations/actions with transparent access to the real time plant variable values.
d. Utilities which create for example a "wiring list" from the FITB which map a plant measurement location to a local/remote attachment to the computer sensor I/O point. The above is only a partial list of the needs one hears from the plant operational management. The list illustrates, however, the requirements of an orderly hierarchical distributed Control System which can meet the challenges posed in controlling a large plant complex. A HARDWARE APPROACH Figure 1 illustrates a hierarchical distributed system consisting of three or four level hierarchy in terms of computer hardware. The lowest level is either a microprocessor based single loop control card or a direct interface to an instrument such as temperature or pressure indicator, current output station for setpoint interface or an alarm relay input via Analog/ digital input/output hardware. The second level in the hierarchy is a unit level controller. The unit level has the complete set of storage table information for all the variables under control of/or attached to that computer. The unit system also has a bulk storage hardware for periodically checkpointing variable data as well as minimum program information required for cold, or warm restart in case of power or system failure. A minimum set of operator displays and operator communication console may also exist on the system in case the unit is in a remote control room. The third level in this hierarchy is an area level computer whose main function is to provide access to all the variables associated with all the units attached to the area computer. The area machine will have all the variable and history/log data base for the area complex on large disk storage devices. Further, it has the full display capability for engineers/operators to perform inter-unit control in a supervisory mode. The area level system, undoubtedly is in a large control room and if one chooses, it may also be connected to a unit machine monitoring plant laboratory instruments performing routine product analyses, or a unit monitoring discrete production data. The area level system will have thus real time laboratory or production data for the plant loops, if required. Next, a fourth level hierarchical system, which we will call a "host" level. This system may generally be a typical Data Processing System performing basically routine non-real time functions such as accounting, plant maintenance scheduling, and other commercial or scientific applic-
135
Contro l Hierarc hy in a Distrib uted Proces s Contro l System
ations . Because of the large disk data base capabi lity, this system provides facilit y for one or more area level systems to transm it log and plant production data to this level for data base management use for the entire plant complex. Some of the plant data may be used by engine ers for optimi zation and statist ical model buildin g or in some cases for model "fittin g". The versat ility of modern computer systems may allow a large system to perform two level functio ns of Figure 1 into one level. For example IBM's Advanced Control System can combine the area level with the host level, in a VM (virtua l machine) System environment, with contro l functio ns being in a OS/VSl partiti on. The separa tion of host and area, on the other hand, provide s the independence of the real time (proces s contro l) systems from the data proces sing systems, from safety and reliab ility consid eration s, and also control inform ation at the remote contro l room of the unit. The four level hierarc hy of Figure 1 provides distin ct separa tion of functio ns, and with proper software archite cture can allow independent operati on and system integr ity at each lower level when communication breakdown between systems occurs . In Figure 2 we show two typical hardware config uration s where the functio nal hierarc hy shown in Figure 1 is represe nted. In Figure 2a, 1evel three and 1evel four functio ns are service d by an IBM 370 (30XX) or 43XX whereas lev~l two functio n is provided by the IBM Series /l. Level 1 functio n is provided by remote terminal units (RTU) consis ting of one or more microp rocesso r operate d contro l loops found in modern plants or Series /l Analog /digita l interfa ces to plant contro llers. If the link between the 370 or 43XX and Series /l is a channel then it represe nts the IBM Advanced Control Systems' program (3) hardware system; with the proviso that all variab le control process ing is performed on level 3. Figure 2b on the other hand shows a hardware system where the area level functio n is performed by a large IBM Series /l (256-5l2 K memory, multip l e 64 megabyte disks, etc.)· and unit level . functio n performed by a Small IBM Serles /l, (128 K memory, 1.2 megabyte disket tes, etc.). All contro l and variab le proces sing is now on the unit level for the proces s variab les attache d to that unit. Note that the communication between the variou s Series /l systems is either via a high speed communication ring or, when distanc es are larger than a mile between units, via bisync lines. If the plant instrum ents are connected direct ly to the unit level Series /l analog and digital input/o utput, then level two and one functio ns are
combined in the Series /l, with the Software respon sible for level 1 functio ns suc~ as direct digital contro l. If level 1 functio n is performed by several microprocessor cards, the unit level Series /l interfa ces to the microp rocesso rs and provides the necess ary commands for monitoring and superv isory control of the loops under control of each microp rocesso r. The interfa ce is again dependent on the type of hardware at level 1 with communicationon a data bus or via a RS232/244 (asyncronous) line. To further complicate the require ments, level one devices may have multip le levels underneath them as shown in Figure 2b. For example, the unit level may addres s a device through a multip lexor, a submu ltiplexo r and remote device addres s (with device type and command). We see then, that the distrib uted computer control of an entire plant complex poses some interes ting challen ges for a software engine er. SOFTWARE REQUIREMENTS Process contro·l programming has always been a diffic ult area because of the real time functio n demands on the System. Several approaches have been made in the last twenty years to simplif y systems as well as applic ation programming to reduce large manpower requirements of a process contro l install ation. The most ideal situati on demands not only a "Fill-In -The-B lanks" (FITB) approach for applic ation definit ion but also an automatic (or prefera bly "canned") System genera tion procedure for a variety of hardware, communications between the variou s computer systems and variety of instrum ent interfa ces. In additio n, one
DATA BASE
LEVEL
4
COMPUTER
Figure 1: A multile vel hierarc hical contro l system.
M. J. Shah and F. Schneider
136
needs a user programming capability for non standard control functions in a "High Level/Engineer's Language". The introduction of a distributed System adds further complications. since one does not know whether to allow the operator/engineer of one remote unit to modify control parameters of another unit with "ease". One also needs to decide how many control loops can and need to be handl ed at a unit 1evel. Is it necessary. for example. to divide a large plant unit into multiple "sub-units" so that more than one level one System is required. thus restricting cascading of control within a plant unit. Conversely. some plant units may have sufficiently small number of loops/ variables so that one first level computer may be able to control multiple plant units. thus posing the problem (sometimes political) which control room has the first level machine. with the added respons- ibility of System maintenance and integrity . varia~es/
CENTRALIZED VERSUS DISTRIBUTED For several years debates between centralized and distributed control has gone on. As the computer hardware price/performance ratio continues to drop. and. as high speed communication between systems becomes possible. the Software cost has become the driving force in selecting the proper approach for control of a large plant complex. 4341
Centralized control has the advantages of having all the data base for the complex in one machine. providing instantaneous access to any plant variable for manipulation. management information or any interacting control required. Figure 2a represents such a system. taking advantage of very high speed capability of a large processor. With a channel attachment to Series/l. the problem of remote instrument/controller attachment is tackled by using remote terminal units (RTU). to the Series/l where the RTU's communicate measurement/control information to the Series/l via telecommunication. Note aqain that the control/ display information is ~ at the central location. The disadvantages of the requirement of a for duplexing. and a operator environment operation.
a central system is second large processor system programmer/ for large system
The distributed system (Figure 2b) allows independent operation at unit level. with its own auto fail/restart capability. Communication failure between area/unit disrupts mainly the historic/log data base for that unit and prevents interactive control at the area level between units. Duplexing requirements for system failure/ integrity are therefore financially less severe. The function distribution between units/area/host at the same time separates the functional authority between operators. engineers and management which mayor may not be desirable. depending upon the policy
( DATA ( BASE
370 DP SYSTEM
EJ~SE
S/SYNC
PLANT INSTRUMENTS
Figure 2: a) Centralized Control System
b) Distributed Control System.
Control Hierarchy in a Distributed Process Control System
in a plant. The disadvantage of such a system however is some loss in the control and immediate retrieval of the entire plant data base, which in essence is the purpose of a large computer control system for a large plant complex. A SOFTWARE APPROACH FOR
AHIERARCHICAL/DISTRIBUTED
137
l.level one is primarily the instrument or a microprocessor driver single loop controller, or an analog controller with a computer interface. In the manual mode this controller is the backup if the System at level 2 fails or if the operator puts the loop in manual.
SYSTEM
For the remainder of the paper we will present a software architecture that addresses some of the distributed system requirements, while maintaining advantages of a centralized data base on a large host. Suppose that the host system in Figure 2b is a standard large data processing system such as an IBM 370 (30XX) or 43XX operating with a standard software such as MVS/IMS, DOS/CICS with Dl/l for standard data base access techniques. This system is mainly used for non process control work. However, ar-the same time, it provides a large data base accessible to the area level machines to transmit plant data on a periodic basis via a bisync line, or if one chooses on a 370 channel. The plant data on the host is also accessible to engineers and management for information/ optimization/ model building, plant maintenance, discrete production data and report purposes using high level languages such as Pl/l, and for preparing the initial real time data base required for control at the levels below. The area level (level 3) and unit level (level 2) are Series/l computers with operating system of Event Driven Executive (EDX). The communication between Series/l's at the two levels as well as between levels 3 and 4 is using the Communications facilities (CF) program. The link between Series/l 's may be the high speed ring or for a remote unit via a bisync 1 i ne. The communication between level 2 Series/l and an instrument (1 evel 1) is via the standard EDX sensor input/output commands. If level 1 consists of a hierarchy of microprocessor controllers (with multiplexor/submultiplexor/ RTU combination), the inte~face is via a special program such as an I/O driver, with a generalized FITB established protocol for measurement/ control data transfer. Undoubtedly the design of I/O driver depends upon the level 1 hardware itself. The application program design then allows for intermix of different interfaces at level 1 and permits the engineer to define his instrument interface to the variable without ~ priori knowledge of the I/O driver program. The functional responsibility of each level in Figure 2b is summarized as follows:
2.level two is the Series/l which performs: a) PlO, ratio, supervisory, feed forward and cascaded control, and user specified control algorithms, b) limited operator display interface for variable data update, alarms and information, c) checkpointing variable tables for warm restart, d) transmission of log and history to area level, e) communicating to area current variable information and updating of tables by area command with integrity checking. f) Miscellaneous back up functions in case of communication fa il ures. 3.level three Series/l provides full operator/engineer display interface for variable update on all units attached to the area, for interactive control, agiin with integrity checking at both levels. It also logs the data transmitted from units on formatted files for access at this level for reports, optimization, model fitting and other user programs. Color graphics both for display and simulation resides at this level. Periodically or on demand, logged data is transmitted to level four, for management information system. 4.level four (host) is used for program and control data base preparation, total plant optimization (periodic or otherwise), scheduling, etc .. The importance of this level is not critical for plant operation. The host may occasionally be used for exp~nding the unit data base for new instruments/variables and writing and testing new user control strategies, again using the pass through capabilities of CF program (1) from the a rea 1evel . USER INTERFACE: If the plant units are sufficiently small, level 2 and 3 functions may be performed in one Ser i es/l. By the same to ken, if a plant unit has more variables than the capabil ity of 1evel 2 Series/l, mul tipl e Series/l processors may be required to handle variable processing load from a single control room. Clearly, cascading from control loops across Series/l 's pose tricky programming problems. Since the Series/l 's have different display terminal types (required in different plant control room environments) and since the terminals on the 370 (30XX) and 43XX are
138
M. J. Shah and F. Schneider
terminal, presents display information in identical form, thus e1 iminating user confusion as operators/engineers move from one level system to another. In addition the 3270 emulation capability of CF on Series/1 permits the engineer to switch the terminal on level 3 to execute programs on level 4, as if operating directly on level 4.
PROGRAM AUTOMATION The lessons from enormous software cost for impl ementing process control systems in the last decade led to use of so called "Fil1-In- The-B1 anks" approach to generate the data base for process control system. For smaller systems even program source modifications with FITB has been used (5) to generate flexible object modules for different hardware configurations. The FITB approach however must avoid overwhelming a user with numerous questions which may be unrelated or confusing, either with safe defaults, or via a gradual method, whereby more complex definitions are only required if a basic question is answered affirmatively. Often large software effort is involved in just generating the data base with FITB, since all answers must be checked for integrity and allow a user friendly recovery. The IBM ACS product demonstrates this approach and is described elsewhere (3). The problem with the FITB approach is that it is impossible to satisfy application requirements of all users so that some user programming capability must be provided. To minimize programming difficulty and isolating user errors from system functions, the FITB forms allow user exits at various stages of processing a variable for special functions. Furthermore, a general algorithm language facility as in ACS (3) can provide a language with access to the variable data base on line. This facility, allows a compile time integrity checking, while the user is programming, as well as trapping of execution errors from erroneous update of variable data base. The central ized system, such as ACS (3) allows more experimentation with user programs/ strategies than a distributed system, due to the centra 1 da ta ba se. In the latter case, dynamic expansion of variable data base on level 2 presents not only interesting software problems but also domain/responsibility considerations for the remote unit data base. Problems can also arise if the user program/ algorithm uses excessive number of machine cycles thus interfering with processing of variab1 es.
For the distributed system, the Series/1 EDX operating system (2) allows a powerful capability of adding system language (EDl) extensions; the interpretive nature of EDl then permits inclusion of high performance re-entrant assembler subroutines in the operatir.g system, to serv~ce t~e . . extensions, at the same tlme slmp11fYlng compiler construction for the extended language set. This approach has been successfully used in CF program (2) for communication extensions. For user programs that do not require in-line execution, the EDl facility of WAIT(POST command then permits the user to wrlte sophisticated programs for complex calculations, with easy access to ~he real time data base via language extenslons, without interrupting the processing of high speed control loops. Finally, the combination of EDl on Series/l and Pl/l on the host to create the required level 2 data base (as well as some special EDl source), allows the flexibility of expanding the form definitions with automatic program generation for handling ~ew items in the data base. In effect, thlS concept proposes an FITB for FITB for process control.
REFERENCES 1. IBM Series/l Event Driven Executive Communications facilit y, Introduction. Doc No. Gl2 3-007l-l, IBM Corp., Atlanta, Georgia. 2. IBM Series/l Event Dirven Executive Program Description/Operations Manual SB30-1053, IBM Corp., Atlanta, Georgia . 3.IBM Advanced Control System General Information Manual, GH20-2464, IBM Corporation, White Plains, NY . 4.Morris H. M., Control Engineering, 29, 1, pp 68-70 ; 1982. 5.Shah M. J., IBM Systems Journal, 18 , 3, 1979.
SOITW ARE FOR DISTRIBUTED CONTROL SYSTEMS I
CONTROL HIERARCHY IN A DISTRIBUTED PROCESS CONTROL SYSTEM M. J. Shah and F. Schneider IBM Corporation, San Jose, California, USA
Abstract: ~apid advances in microprocessor based controllers and minicomputers have presented opportun1t1es ~or tot~l control of a large plant complex cluster. A hardware/software architect~retf~r a h~era~ch1cal co~trol system for overall control and data base management of such a p a~ 1n ,a d~str~buted env1ronment, is presented in this paper. The software problems of funct1~n d1str1but10n, communication bottlenecks, data base responsibility and engineer/operator 1nterface are prese~ted , together with an approach for their resolut i on . Design approaches for software automat10n 1n this environment are presented. Keywords.Distributed control; hierarchical control; Software automation; Computer control .
I NTRODUCTI ON
interacting plant units, or overall reai time plant data base management and optimization.
For the past few years in process industry there has been a trend for clustering several plant units together in a single complex for the sake of economic efficiency as well as for better coordination between various plant units. Thus, for example, modern refineries are in reality a complex cluster of interacting individual units producing a multitude of products.
This paper, then, explores a software system architecture that addresses the challenges of optimal control of a large complex of interacting plant units whether they be continuous or discrete. The environment considered here is a complex of plant units each with its own control system .
Similarly, a fertilizer plant complex sequentially produces ammonia, urea, nitric acid, sodium nitrate, phosphates, etc .. Most manufacturing plants today consist of a mixture of continuous operations as well as discrete operations and they are both interacting in one form or another.
The units may be close or considerable distance apart, thus necessitating sophisticated communications with high and low speed lines, keeping in mind the problems of real time computer control of remote plant units and their dependence on the total distributed system.
As the plant cluster grows, the complexity of control of the interacting plant units increases and the need for an overall real time data base for the entire plant for reporting, trouble shooting, optimization and other uses, become more acute. The high cost of energy and competitive forces on product cost have forced the manufacturing industries to efficiency in design of individual plant units; however, the ' interacting control of individual plant units still remains a challenge :
GENERAL SYSTEM REQUIREMENTS It is worthwhile first to list system requirements for distributed control of a large plant complex. Compl ete operator control for a plant unit, uninterrupted even when system communication breakdown occurs. Engineer/operator access for plant information data as well as control of multiple units from a single supervisory control room.
Several vendors advertize distributed computer control in one form or another. In general, these approaches distribute computer workload, such as control, display processing and communication between processors, more or less for control of a single unit. However, to-date, there has been no published work on distributed systems which describe a cohesive software architecture that meets the challenges of
Sensor input /output connection to local measurements or remote attachment to microprocessor controllers. Unified software scheme for variable definition irrespective of hardware connection.
133
134
M. J. Shah and F. Schneider
Direct digital control and Supervisory control at plant unit level with communication for Supervisory and optimum set point calculation from a higher level system. For discrete and batch processes in a unit, access to a prepared data base to activate for example a new batch or test. Provide the same data base for transfer of discrete data logs. Access to a large data base for continuous transfer of plant operational data for each of the desired variables in the plant unit, for historic, trend and logging purposes. Mechanism for retrieving the transferred data for plots, as well as calculations suc~ ~s r~gression analysis, modeling, optlmlzatlon and production purposes. Ability to modify operational data base at the plant unit level with integrity checking for operator/engineer entries as well as ability to occasionally add and activate new variables without disturbing the rest of the plant unit control system. Ability to add a new unit control system (continuous or discrete) without affecting other units. Separable engineer/operator interface for accessing and modifying different parts of the control characteristics for each varia~e. For exam~e, only an engineer with proper password authorization may modify tuning constants, whereas an operator may vary setpoints, limits ro 11 over to ma nua 1 contro 1, etc. Tracking of all operator/engineer actions by logging them on a large data base. Provide all levels of control from the basic monitoring of variables to sophisticated interactive control such as cascade, feed forward and multivariable contro 1 . Minimize programming requirement by providing: a. Fill-in the blanks (FITB) interface for variables, for all modes of contro 1 . b. FITB interface for specifying different computer hardware configurations for automatic system generation, for each unit level and all communication systems between the distributed systems. c. A "Control Systems non programmers I language" for unique control algorithms/ calculations/actions with transparent access to the real time plant variable values.
d. Utilities which create for example a "wiring list" from the FITB which map a plant measurement location to a local/remote attachment to the computer sensor I/O point. The above is only a partial list of the needs one hears from the plant operational management. The list illustrates, however, the requirements of an orderly hierarchical distributed Control System which can meet the challenges posed in controlling a large plant complex. A HARDWARE APPROACH Figure 1 illustrates a hierarchical distributed system consisting of three or four level hierarchy in terms of computer hardware. The lowest level is either a microprocessor based single loop control card or a direct interface to an instrument such as temperature or pressure indicator, current output station for setpoint interface or an alarm relay input via Analog/ digital input/output hardware. The second level in the hierarchy is a unit level controller. The unit level has the complete set of storage table information for all the variables under control of/or attached to that computer. The unit system also has a bulk storage hardware for periodically checkpointing variable data as well as minimum program information required for cold, or warm restart in case of power or system failure. A minimum set of operator displays and operator communication console may also exist on the system in case the unit is in a remote control room. The third level in this hierarchy is an area level computer whose main function is to provide access to all the variables associated with all the units attached to the area computer. The area machine will have all the variable and history/log data base for the area complex on large disk storage devices. Further, it has the full display capability for engineers/operators to perform inter-unit control in a supervisory mode. The area level system, undoubtedly is in a large control room and if one chooses, it may also be connected to a unit machine monitoring plant laboratory instruments performing routine product analyses, or a unit monitoring discrete production data. The area level system will have thus real time laboratory or production data for the plant loops, if required. Next, a fourth level hierarchical system, which we will call a "host" level. This system may generally be a typical Data Processing System performing basically routine non-real time functions such as accounting, plant maintenance scheduling, and other commercial or scientific applic-
135
Contro l Hierarc hy in a Distrib uted Proces s Contro l System
ations . Because of the large disk data base capabi lity, this system provides facilit y for one or more area level systems to transm it log and plant production data to this level for data base management use for the entire plant complex. Some of the plant data may be used by engine ers for optimi zation and statist ical model buildin g or in some cases for model "fittin g". The versat ility of modern computer systems may allow a large system to perform two level functio ns of Figure 1 into one level. For example IBM's Advanced Control System can combine the area level with the host level, in a VM (virtua l machine) System environment, with contro l functio ns being in a OS/VSl partiti on. The separa tion of host and area, on the other hand, provide s the independence of the real time (proces s contro l) systems from the data proces sing systems, from safety and reliab ility consid eration s, and also control inform ation at the remote contro l room of the unit. The four level hierarc hy of Figure 1 provides distin ct separa tion of functio ns, and with proper software archite cture can allow independent operati on and system integr ity at each lower level when communication breakdown between systems occurs . In Figure 2 we show two typical hardware config uration s where the functio nal hierarc hy shown in Figure 1 is represe nted. In Figure 2a, 1evel three and 1evel four functio ns are service d by an IBM 370 (30XX) or 43XX whereas lev~l two functio n is provided by the IBM Series /l. Level 1 functio n is provided by remote terminal units (RTU) consis ting of one or more microp rocesso r operate d contro l loops found in modern plants or Series /l Analog /digita l interfa ces to plant contro llers. If the link between the 370 or 43XX and Series /l is a channel then it represe nts the IBM Advanced Control Systems' program (3) hardware system; with the proviso that all variab le control process ing is performed on level 3. Figure 2b on the other hand shows a hardware system where the area level functio n is performed by a large IBM Series /l (256-5l2 K memory, multip l e 64 megabyte disks, etc.)· and unit level . functio n performed by a Small IBM Serles /l, (128 K memory, 1.2 megabyte disket tes, etc.). All contro l and variab le proces sing is now on the unit level for the proces s variab les attache d to that unit. Note that the communication between the variou s Series /l systems is either via a high speed communication ring or, when distanc es are larger than a mile between units, via bisync lines. If the plant instrum ents are connected direct ly to the unit level Series /l analog and digital input/o utput, then level two and one functio ns are
combined in the Series /l, with the Software respon sible for level 1 functio ns suc~ as direct digital contro l. If level 1 functio n is performed by several microprocessor cards, the unit level Series /l interfa ces to the microp rocesso rs and provides the necess ary commands for monitoring and superv isory control of the loops under control of each microp rocesso r. The interfa ce is again dependent on the type of hardware at level 1 with communicationon a data bus or via a RS232/244 (asyncronous) line. To further complicate the require ments, level one devices may have multip le levels underneath them as shown in Figure 2b. For example, the unit level may addres s a device through a multip lexor, a submu ltiplexo r and remote device addres s (with device type and command). We see then, that the distrib uted computer control of an entire plant complex poses some interes ting challen ges for a software engine er. SOFTWARE REQUIREMENTS Process contro·l programming has always been a diffic ult area because of the real time functio n demands on the System. Several approaches have been made in the last twenty years to simplif y systems as well as applic ation programming to reduce large manpower requirements of a process contro l install ation. The most ideal situati on demands not only a "Fill-In -The-B lanks" (FITB) approach for applic ation definit ion but also an automatic (or prefera bly "canned") System genera tion procedure for a variety of hardware, communications between the variou s computer systems and variety of instrum ent interfa ces. In additio n, one
DATA BASE
LEVEL
4
COMPUTER
Figure 1: A multile vel hierarc hical contro l system.
M. J. Shah and F. Schneider
136
needs a user programming capability for non standard control functions in a "High Level/Engineer's Language". The introduction of a distributed System adds further complications. since one does not know whether to allow the operator/engineer of one remote unit to modify control parameters of another unit with "ease". One also needs to decide how many control loops can and need to be handl ed at a unit 1evel. Is it necessary. for example. to divide a large plant unit into multiple "sub-units" so that more than one level one System is required. thus restricting cascading of control within a plant unit. Conversely. some plant units may have sufficiently small number of loops/ variables so that one first level computer may be able to control multiple plant units. thus posing the problem (sometimes political) which control room has the first level machine. with the added respons- ibility of System maintenance and integrity . varia~es/
CENTRALIZED VERSUS DISTRIBUTED For several years debates between centralized and distributed control has gone on. As the computer hardware price/performance ratio continues to drop. and. as high speed communication between systems becomes possible. the Software cost has become the driving force in selecting the proper approach for control of a large plant complex. 4341
Centralized control has the advantages of having all the data base for the complex in one machine. providing instantaneous access to any plant variable for manipulation. management information or any interacting control required. Figure 2a represents such a system. taking advantage of very high speed capability of a large processor. With a channel attachment to Series/l. the problem of remote instrument/controller attachment is tackled by using remote terminal units (RTU). to the Series/l where the RTU's communicate measurement/control information to the Series/l via telecommunication. Note aqain that the control/ display information is ~ at the central location. The disadvantages of the requirement of a for duplexing. and a operator environment operation.
a central system is second large processor system programmer/ for large system
The distributed system (Figure 2b) allows independent operation at unit level. with its own auto fail/restart capability. Communication failure between area/unit disrupts mainly the historic/log data base for that unit and prevents interactive control at the area level between units. Duplexing requirements for system failure/ integrity are therefore financially less severe. The function distribution between units/area/host at the same time separates the functional authority between operators. engineers and management which mayor may not be desirable. depending upon the policy
( DATA ( BASE
370 DP SYSTEM
EJ~SE
S/SYNC
PLANT INSTRUMENTS
Figure 2: a) Centralized Control System
b) Distributed Control System.
Control Hierarchy in a Distributed Process Control System
in a plant. The disadvantage of such a system however is some loss in the control and immediate retrieval of the entire plant data base, which in essence is the purpose of a large computer control system for a large plant complex. A SOFTWARE APPROACH FOR
AHIERARCHICAL/DISTRIBUTED
137
l.level one is primarily the instrument or a microprocessor driver single loop controller, or an analog controller with a computer interface. In the manual mode this controller is the backup if the System at level 2 fails or if the operator puts the loop in manual.
SYSTEM
For the remainder of the paper we will present a software architecture that addresses some of the distributed system requirements, while maintaining advantages of a centralized data base on a large host. Suppose that the host system in Figure 2b is a standard large data processing system such as an IBM 370 (30XX) or 43XX operating with a standard software such as MVS/IMS, DOS/CICS with Dl/l for standard data base access techniques. This system is mainly used for non process control work. However, ar-the same time, it provides a large data base accessible to the area level machines to transmit plant data on a periodic basis via a bisync line, or if one chooses on a 370 channel. The plant data on the host is also accessible to engineers and management for information/ optimization/ model building, plant maintenance, discrete production data and report purposes using high level languages such as Pl/l, and for preparing the initial real time data base required for control at the levels below. The area level (level 3) and unit level (level 2) are Series/l computers with operating system of Event Driven Executive (EDX). The communication between Series/l's at the two levels as well as between levels 3 and 4 is using the Communications facilities (CF) program. The link between Series/l 's may be the high speed ring or for a remote unit via a bisync 1 i ne. The communication between level 2 Series/l and an instrument (1 evel 1) is via the standard EDX sensor input/output commands. If level 1 consists of a hierarchy of microprocessor controllers (with multiplexor/submultiplexor/ RTU combination), the inte~face is via a special program such as an I/O driver, with a generalized FITB established protocol for measurement/ control data transfer. Undoubtedly the design of I/O driver depends upon the level 1 hardware itself. The application program design then allows for intermix of different interfaces at level 1 and permits the engineer to define his instrument interface to the variable without ~ priori knowledge of the I/O driver program. The functional responsibility of each level in Figure 2b is summarized as follows:
2.level two is the Series/l which performs: a) PlO, ratio, supervisory, feed forward and cascaded control, and user specified control algorithms, b) limited operator display interface for variable data update, alarms and information, c) checkpointing variable tables for warm restart, d) transmission of log and history to area level, e) communicating to area current variable information and updating of tables by area command with integrity checking. f) Miscellaneous back up functions in case of communication fa il ures. 3.level three Series/l provides full operator/engineer display interface for variable update on all units attached to the area, for interactive control, agiin with integrity checking at both levels. It also logs the data transmitted from units on formatted files for access at this level for reports, optimization, model fitting and other user programs. Color graphics both for display and simulation resides at this level. Periodically or on demand, logged data is transmitted to level four, for management information system. 4.level four (host) is used for program and control data base preparation, total plant optimization (periodic or otherwise), scheduling, etc .. The importance of this level is not critical for plant operation. The host may occasionally be used for exp~nding the unit data base for new instruments/variables and writing and testing new user control strategies, again using the pass through capabilities of CF program (1) from the a rea 1evel . USER INTERFACE: If the plant units are sufficiently small, level 2 and 3 functions may be performed in one Ser i es/l. By the same to ken, if a plant unit has more variables than the capabil ity of 1evel 2 Series/l, mul tipl e Series/l processors may be required to handle variable processing load from a single control room. Clearly, cascading from control loops across Series/l 's pose tricky programming problems. Since the Series/l 's have different display terminal types (required in different plant control room environments) and since the terminals on the 370 (30XX) and 43XX are
138
M. J. Shah and F. Schneider
terminal, presents display information in identical form, thus e1 iminating user confusion as operators/engineers move from one level system to another. In addition the 3270 emulation capability of CF on Series/1 permits the engineer to switch the terminal on level 3 to execute programs on level 4, as if operating directly on level 4.
PROGRAM AUTOMATION The lessons from enormous software cost for impl ementing process control systems in the last decade led to use of so called "Fil1-In- The-B1 anks" approach to generate the data base for process control system. For smaller systems even program source modifications with FITB has been used (5) to generate flexible object modules for different hardware configurations. The FITB approach however must avoid overwhelming a user with numerous questions which may be unrelated or confusing, either with safe defaults, or via a gradual method, whereby more complex definitions are only required if a basic question is answered affirmatively. Often large software effort is involved in just generating the data base with FITB, since all answers must be checked for integrity and allow a user friendly recovery. The IBM ACS product demonstrates this approach and is described elsewhere (3). The problem with the FITB approach is that it is impossible to satisfy application requirements of all users so that some user programming capability must be provided. To minimize programming difficulty and isolating user errors from system functions, the FITB forms allow user exits at various stages of processing a variable for special functions. Furthermore, a general algorithm language facility as in ACS (3) can provide a language with access to the variable data base on line. This facility, allows a compile time integrity checking, while the user is programming, as well as trapping of execution errors from erroneous update of variable data base. The central ized system, such as ACS (3) allows more experimentation with user programs/ strategies than a distributed system, due to the centra 1 da ta ba se. In the latter case, dynamic expansion of variable data base on level 2 presents not only interesting software problems but also domain/responsibility considerations for the remote unit data base. Problems can also arise if the user program/ algorithm uses excessive number of machine cycles thus interfering with processing of variab1 es.
For the distributed system, the Series/1 EDX operating system (2) allows a powerful capability of adding system language (EDl) extensions; the interpretive nature of EDl then permits inclusion of high performance re-entrant assembler subroutines in the operatir.g system, to serv~ce t~e . . extensions, at the same tlme slmp11fYlng compiler construction for the extended language set. This approach has been successfully used in CF program (2) for communication extensions. For user programs that do not require in-line execution, the EDl facility of WAIT(POST command then permits the user to wrlte sophisticated programs for complex calculations, with easy access to ~he real time data base via language extenslons, without interrupting the processing of high speed control loops. Finally, the combination of EDl on Series/l and Pl/l on the host to create the required level 2 data base (as well as some special EDl source), allows the flexibility of expanding the form definitions with automatic program generation for handling ~ew items in the data base. In effect, thlS concept proposes an FITB for FITB for process control.
REFERENCES 1. IBM Series/l Event Driven Executive Communications facilit y, Introduction. Doc No. Gl2 3-007l-l, IBM Corp., Atlanta, Georgia. 2. IBM Series/l Event Dirven Executive Program Description/Operations Manual SB30-1053, IBM Corp., Atlanta, Georgia . 3.IBM Advanced Control System General Information Manual, GH20-2464, IBM Corporation, White Plains, NY . 4.Morris H. M., Control Engineering, 29, 1, pp 68-70 ; 1982. 5.Shah M. J., IBM Systems Journal, 18 , 3, 1979.