Distributed Control: Status and Opportunities

Copvright

IFAC Components and Instruments for

PLEl\;ARY PAPERS

Disnibuted Control Systems. Paris. France 1982

DISTRIBUTED CONTROL: STATUS AND OPPORTUNITIES I. Lefkowitz and M. R. Buchner Department of Systems Engineering, Case Western Reserve Universl"ty, Cleveland, Ohio, USA

Abstract. Goals of improved productivity, efficiency, and product quality have motivated a continuing development over the years of automatic control theory and practice in industrial applications. A brief historical account of this development leads to a discussion of the current status of distributed control as a natural stage in the evolution toward complete integrated control of an industrial plant. The historical perspective also provides background for an examination of trends and opportunities in distributed control. Present generation microprocessor-based distributed control systems are described in terms of their generic features and attributes. Reference to limitations of these systems with respect to the goal of achieving implementable, cost effective and reliable integrated systems control motivates discussion of some current research areas and expected future developments. Finally, a review of the hierarchical control approach is presented. This provides a conceptual framework for organizing the elements of the distributed system for integration of the many diverse information processing, decision-making and control functions that are involved in a total plant control. Keywords. Hierarchical control; computer control; distributed control systems; real-time systems; integrated systems control. INTRODUCTION The control of industrial processes has evolved very considerably over the past half century. Early objectives of automatic control were to relieve human operators of the tedium and drudgery of maintaining certain key variables of the plant at desired values through feedback actions. At the same time, control devices provided the means of better accuracy and more consistent and dependable results. The introduction of electronic instrumentation enabled remote sensing and actuation which led to the development of central control rooms where the operator could keep track of a large number of control loops. His role became increasingly one of supervision - to adjust controller setpoints whenever required by a change in the mode of operation or in the specifications of the product. The operator also had the responsibility of monitoring the performance of the various control loops to make sure that the plant was operating properly, to make changes whenever product quality or production efficienc y fell below tolerance limits and to respond to contingency events (e.g. a malfunction of a piece of equipment) with proper emergency actions.

It was soon recognized that some of the more elementary supervisory functions could be carried out automatically. For example, controllers were introduced that could maintain a fixed functional relationship among several process variables so as to improve process performance (e.g. yield, efficiency, product quality). At the same time, sophisticated monitoring and alarm systems were developed that automatically sensed the status of all the plant variables and alerted the operator if any of them exc e eded preset limits. The advent of the digital control computer in the 1950's initiated a revolution in the control of industrial plants. The computer made it possible to store and process large quantities of data and to implement complex algorithms in real-time so that we could advance from simple control objectives of maintaining process variables at fixed desired values to the more interesting objectives of determining how these variables should be changed with time or in relationship to other variables in order to optimize plant performance. The control computer also provided the capability of rapid switching from one computational task to another. Thus, one machine could handle a large number of

2

I. Lefkowitz and M.R. Buchner

control loops as well as various auxiliary tasks such as monitoring, start-up sequences, operational control, etc. unf o rtunately, the early process control computers were costly and had limited speed, memory capacity and software capabilities. Reliability was another problem, i.e. assuring safe and dependable performance of the system over months and even years of continuous plant operation. As a result, many of the initial attempts at computer control, while boldly conceived and implemented with great fervor and effort, fell dismally short of expectations. Developments in computer technology over the past fifteen years have resulted in tremendous reductions in hardware costs while computation speeds and storage capacities have increased dramatically. User-oriented programming languages have greatly eased the man-machine interaction problem, e.g. in programming, debugging and updating computer control algorithms. Also, reliability has improved substantially as a result of more reliable components and the increased feasibility of fault-tolerant designs, redundancy and diagnostic routines -- enhanced by low hardware costs and more sophisticated design techniques. More r ecent l y , advances in real-time applications of minicomputers and microprocessors have had a profound effect on the directions of current effort in industrial systems control. Specifically, these have opened up new opportunities for system configuration based on (i) distributed data acquisition and control and (ii) hierarchical computer control where each computer performs selected tasks appropriate to its position in the hierarchy. These approaches (in contrast with the initial idea of lumping all tasks in one giant control computer) have contributed to design flexibility, improved reliability and security, better performance, etc. Integrated Systems Control (See [1,2) A consequence of these developments has been a vast broadening of the domain of what is technologically and economically feasible to achieve in the application of computers to control of industrial systems. Now, all aspects of information processing, data gathering, process control, on-line optimization, operations control - advancing even to realtime schedu ling and production planning functions may be included in the range of tasks to be carried out by the computer control sys tem. This has made possible the realization of integrated systems control in which all factors influencing plant performance are taken into account in an integrated fashion-recognizing the couplings, interactions and complex feedback paths existing in the system -- to achieve an overall optimum performance. The analytical and technological advances in

control capabilities had been accompanied by a growing perception in industry of the need for applying more advanced and effective control, spurred on by such factors as: (a) the need for more efficient utilization of resources (e.g. energy, water, labor, materials) because of increasing cost, limited availability, or both; (b) demands for higher productivity to meet more intense competition; and (c) more stringent requirements concerning product quality, environment impact, and human safety because of government regulations and greater consumer awareness. The problems of realization and implementation of an integrated systems control are quite formidable because of the complexity of production processes, dynamic interactions among the production units, time-varying aspects of the system, various constraints to be satisfied, etc. Additional factors to be considered include multiobject ive decision-making under uncertainty and man-machine interactions. The hierarchical control approach provides a rational and systematic design procedure for addressing the problems of complexity and uncertainty. Modern distributed intelligence/ computer control systems provide the hardware and software capabilities for cost effective realization of the integrated system and for effective incorporation of the human (operator/ decision-maker) as an integral part of the system. With respect to decision-making and control, we distinguish the following basic elements: 1. The plant denotes the controlled system and may refer variously to a production unit, a processing complex or even an entire company depending on the level of control being considered. The plant is subject to a variety of disturbance inputs, i.e. the effects of interactions of the plant with other plant units and with the environment that cause th e system to deviate from desired or predicted behavior. A special type of disturbance is the discrete event or "contingency " occurrence, e . g. , failure of a component or taking a unit off line. A contingency event usually implies that th e system is no longer operating according to assumptions imbedded in the current control model and hence, it is necessary to modify the structure of the system, go into a new control mode or develop some other corrective action. 2. The controller generates the controlled inputs to the plant for the purpose of achieving a desired behavior or performance consistent with system constraints. The constraints define th e regions of f easible or acceptable plant operation; they are imposed to ensure safe ty of operating pe rsonnel, security of th e production means, and that various "qualit y" requirements are met, e.g. product specifications, effluent pollution restrictions, etc. The control functions may be performed by man, by machine (computer), or more generally, by an integration of human operators, schedulers, and planners with the computer control and data

Status and Opportunities management system. The functions performed by man include those based on judgment or experiences whose subleties or nonquantifiable attributes defy computer implementation. The functions performed by computers are essentially those where the tasks are routine and well-defined and where the operating standards are quantified and established. 3. The underlying assumption in the achievement of integrated control is that the controller acts on the basis of (real-time) information concerning the state of the plant, external inputs, etc. Major functions of the information processor include: (a) data gathering and processing, e.g. datasmoothing, noise filtering, prediction and extrapolation, etc. (b) the monitoring of system status for contingency events to determine whether diagnostic and/or corrective responses are to be initiated. (c) the storage and retrieval of operating instructions, standards, parameter values, and other information required for the functioning of the system. A block diagram of the relationship of the controller to the plant is given in Fig. 1. Current values of the output variables y (feedback action) and some of the disturbance variables z (for feedforward actions) are transmitted to the controller by means of sensors or measuring devices. The raw information set x may be further processed (filtered, smoothed, transformed, etc.) by the information processor. The controller generates its outputs according to current information concerning y and z, in relation to the input r, which defines the desired behavior of the plant, e.g. provides the setpoint values at which certain output variables are to be maintained. Finally, the controller must communicate its decisions/actions to the plant -and this is the role of the actuator. The elements represented in Fig. 1 are generic to all control and decision-making functions and are embedded within each of the hierarchical control structures described below. Hierarchical Control ([2,3]) The overall complex problem may be decomposed according to various criteria; these include: Ca) decomposition according to control function: functional multilayer control hierarchy Cb) decomposition according to subsystem classification or system structure: multilevel control hierarchy

3

1st Layer: The first or direct la y er function constitutes the interface between the controlled plant and the decision-making and control aspects of the system. An important characteristic of the first layer, therefore, is its ability to interact directly with the plant and in the same time scale. Typically, this layer incorporates the functions of data acquisition, event monitoring and direct control. 2nd Layer: The second-layer or supervisory function is concerned with the problem of defining the immediate target or task to be implemented by the first layer. In the normal mode, the objective may be control of the plant for optimum performance according to the assumed mathematical model. Under emergency conditions, different objectives may take precedence through implementation of the appropriate contingency plan. In general, there may be a number of operating modes or topologies identified for the system with each having a different mathematical model through which information describing the current state of the system is transformed into directives applied to the first layer function. In the conventional process control application, the second layer intervention takes the form of defining the set-point values for the first-layer controllers. In the discrete formulation, the output of the supervisory function may be a specified target or "next state" to be implemented by the direct controller through a predetermined sequence of actions. 3rd Layer: The third-layer or adaptive function is concerned with the problem of updating algorithms employed at the first and s e cond layers. The adaptive lay e r may intervene in the operation of the lower layers in the following ways: (a) updating of parameter values associated with the first and second laye r control algorithms, say b y least-squares fitting of the underlying mathematical models to observed plant behavior. (b) updating of parame ters associated with the event monitoring function . Of particular interest here is th e s e nsing of the transition of the plant from one operating mode to another. (c) development of contingenc y plans, i.e. alternative procedures for second-lay er implementation, to be invoked when the plant degenerates to an emergency mode.

(c) decomposition according to time scale: temporal multilayer control hierarchy.

A common and distinguishing feature of the third-layer function is that its actions are a reflection of operating e xperience over a period of time. The actions ar e discrete, taking place in response to e vent occurrences (e.g. operator inputs).

The functional multilayer control hierarchy is characterized by the diagram in Fig. 2 where four layers of control are identified:

4th Layer: The fourth-la y er or self-organizing function is concerned with decisions relevant to the choice of structure of the

4

I. Lefkowitz and

algorithms associated with the lower layers of the hierarchy . These decisions are based on overall considerations of performance objectives, assumption s of th e nature of the system relationships , coordination with other systems, etc. An e xample of the application of the foregoin g structure is provided by a catalytic r eac tor process. The proc ess inputs are controlled as continuous functions of t i me; however, at discrete points in time, the "normal" operating mode is disrupt e d to go into a "regeneration" mode for the purpose of r es toring catalyst activity. The direct control function is concerne d wi th the task of controlling th e process variables, e . g. pressures, tempe ratur es and flow rat es to the set- point values d efi n ed b y the supervisory control function. Th i s function is imp l e mented by means of c onventional feedbac k control loops, with perhaps some feed forward considerations. The determination of set-points at th e second layer depends on th e mode of operation. In the normal operating mod e the set - points may be d e termined to maximize product yie ld consistent with sys tem co n s traint s and specifications on product q ual i t y . In the catalyst r e generation mode, t he objective may be to minimize the r egen e ration period, i.e. th e time during which product is not being produc e d. Ther e are at l eas t two di s tin c t tasks that may be assigned to th e third laye r. The first r e lates to the updating of selected parame t e r s of the lower-layer control algorithms to take care of the ef fect s of normal variations in operating conditions, ca talys t activity, e tc. The second ta sk u p dat es th e c r i t e rion function for switching between operati n g and r egene rating mod es . The fourth control laye r has the r espo n sibi lit y of selecting th e operating mode and, co nseq u e ntl y , th e p r og rams to be use d by the lowe r-laye r control fu n c ti o n s. In general, a transf e r of mod e r eq uir es ex t ensive changes in the control s tructur e which are to be coordinat e d by fourth-la ye r int e rv e ntion. In t he multilevel co ntrol hie r a r chy , t he over all p lant system is d ecomposed into s u bsystems, each wi th its own local co ntr o ll e r. In thi s scheme (see Fig . 3): (a) The first-level co ntroll ers compensate for local effects of th e disturbances, e . g . maintain local performance c lo se to th e optimum while ensuring that lo ca l constraints are not violated. (b) The second-l eve l con tr o ll e r modifies t he criteria and /o r the constraints for th e fi r s tleve l controllers in r esponse to changing r e q uirements on the sys t em so that actions of th e local controll e r s a r e cons i s t en t wi th t he ove r al l objectives of the system .

~.R.

Buchner

In effect, th e subsystem p robl e ms are solved a t the first l eve l of control. However, s inc e the subsystems are co u p l ed and int eracting, these so luti o n s have no meaning unl ess th e interaction co n s traint s are simultan eo u s l y satisf ie d. This is the coo rdinati o n p r ob lem t ha t is so l ved a t t he seco nd l eve l of th e hierarchy. The d ecompo s ition of the overall system into s ub sys t ems ma y be based on geog raphical considerations ( e.g. relative proximity of different units), line s of managerial r es ponsibility (e .g. s t ee l-making shop and rollin g mill in a s t ee l work s) , or on th e t ype o f e quipme nt ( e . g. di s tillation tower and r eactor in a chemical plant). In general, howeve r, the plants are d esig n ed so that th ese divisions correspond t o l ines of weak interac tion, i .e . t h rou gh t he incorporation of va r io u s "buffer" or control mechanisms, the r es ulting subsystems are pa rtially decoupled so that interaction effec t s t end to be sma ll and/or only s lowl y varying wi th time. Advantages of multilevel decomposition include (i) reduction in comp utat io nal effort and data tran s mi ssio n r eq uir e me nt s because th e mor e comp lex coordination ta sks are handl ed a t a higher l evel at l owe r frequency, (ii) oppor tuniti es for increased system r e liabilit y beca u se most of the control ta sks are d es igned t o be ha ndled lo call y wi th s hort-t e rm ind e pendence of the other s ub sys t e ms , and (iii) r ed uctions in maint e nan ce and sys t em dev e l op ment cos ts by virtue of th e fact that mod e l s , control algorithms and compu t e r sof t wa r e may be d eve lop ed in a step-by-step, semi -ind epen d e nt fashion. A particularly r elevant applica tion of th e multil eve l approach is i n the e l ec tr ic power industry whe r e the powe r ge n eration and distribution s ys tem is designed as a n inter co nn ectio n of semi - independent s ubsys t ems . Thu s , t he r e is a natural decomposition induced by t ec hnological consid e rations a t the genera tin g unit l eve l, geographical conside rati o n s at t he ge n e ratin g s t ation l e v e l, ownership boundaries at th e company level, e tc. A seco nd application is sugges t ed by th e sys tem shown in Fig. 4 comprising severa l interconnecte d p rodu c ti on operations of a st ee l mi ll. In order t o maximi ze his perfo r mance (and s t i ll meet deliv e r y commitments) ,rolling mill operator may cal l for slabs of different sizes and g rad es . Howeve r, t he steel shop schedu l e r wants to minimize t he number of grade changes because of the increased l ike lihood of off-standard p r od uct during tran sition from one grade to another. Simi larl y , t he r e is a sig nificant se t-up cost associated with changing s lab dimensions on th e continuous cas tin g machine, hence , the s la bbi n g department wants to minimize t he frequency of s l ab changes . An alternative is t o p rovide mo r e s t o ra ge of s lab s in the s l ab ya rd but t his ma y increase s la b yard cos t s. Thus, we have a role for a highe r l eve l p r oduc ti on sched ul e r t hat reconciles the various con fl icting local

Status and Opportunities objectives of t hese interacting production unit s to satisfy ove rall objectives and cons traint s [4). The multi l eve l control hi e rarchy induces an orde ring with r espec t to time scale; specifically, the mean pe riod of control action t ends to incr ease as we proceed from a lower to a high e r l eve l of th e hie rarchy. In addition, any controller within the multilevel s tructur e ma y it se lf r ep r esent a series of co ntrol tasks that t e nd to be carried out with different fr eq uencies o r time priorities. This motivates th e concept of a temporal control hie rarchy wherei n the control or decis ion-making problem is partitioned into subprobl ems based on time scales which reflect (i) time requir e d to obtain th e information on which th e co ntrol action is based, (ii) bandwid th properties or mean time be tween discret e changes in disturbance inputs, (iii) time horizon associated with th e control problem, and (iv) cost-benef it trade-off consideratio ns . Th us , in e rar chy , decision time (on

th e multilaye r t empo ral control hikth la ye r con trolle r generates a or co ntrol action eve r y Tk units of average), wi th T + > T , k = 1,2, ... , k l k based on (a) th e input information currently avai la bl e , i. e . state of the plant and envi ronme ntal factors, (b) targ e t s and/or const raints provided by a (k+l)th layer controll er, (c) fe edback of pr io r experience provided by a (k-l)th l ayer controller. The temporal hie r a rchy app roach provid es a r a tional mecha ni sm for (i) reducing th e e ffec t s of unc e rt ai nt y , ( ii) introducing experien tial feedback, (iii) aggregating va ri ables and simplifying models, and (iv) impl eme nt ing systems in t eg ration through we lldefined assignments of t asks and res po nsi bili ties. An examp l e of a se t of control functions distinguished by their temporal attributes is provid ed by the r o llin g mill r efe rred to ea rli e r. Because of s ur face wea r, the rolls have t o be replaced at frequent intervals. Each r o ll cha ng e se t s in motion a sequence of eve nts by which t he mill goes from its normal opera tin g mode to a roll change mode and back again, wi t h t he attendant shut down and s tartup proced ur es. The roll change also affects the sequenci ng of s la bs ove r t he s ubsequ e nt ope r a tin g pe ri ods. The r eceip t of a new orde r, invo l ving pe rha ps a lar ge number of s l abs , r equires new mi ll instructions and setups determined by t he o rd e r speci fication s and othe r factors. As each individual slab enters th e mill it initiat es a se ri es of actions r e lating to roll se tt i ngs , spee ds , e tc. Finall y , various feedback mecha ni sms a pp l y in a l most con t inuous action to mai ntain th e t e nsion , thicknes s and t empera tur e of t he s t ee l strip at critical points in t he mi ll to pr ede t e r mined va lues . Thus, there is a broad spec trum of contro l and dec i sion-mak ing activities ranging in time scale from second s to weeks and th ese activi ti es inte ract in a

5

s peci al way because of the temporal relations. A second example is the ordering with respect to time scale of production planning and scheduling funct io ns, e.g. five year plan, annual plan, monthly schedule, daily schedule, e tc. He re, e ach la ye r of the hierarchy gene rates the targets/constraints for the layer below, with feedbacks induced to provide corrections for deviating from the prior plan or schedule. In summary of the foregoing, the hierarchical control approach addresses the problem of complexity by th e following means: 1. The overall complex control problem is reduc e d to a set of much simpler subproblems; the controllers assoc ia ted with the subproblems (subsystems) are coordinated by a higher l eve l controller so that overall goals and constraints are satisfied. 2. Each subsystem controller is concerned only with the local problem of satisfying loca l objectives and lo ca l constraints. 3. Effective action of th e lower level controller s helps reduce the complexity of the higher l eve l controller (and also the amount of information required) by permitting simplification and aggregation of the models associated with th e highe r level function. 4. Decomposition of control tasks according to function and time scale provides the rationale for allocating tasks to the various computing facilities within the system so as to make most effective utilization of reso urces. 5 . Distributed a r chi t ec tures for the informat io n/control/decision-making system fit in very naturally wi th th e decomposition of tasks induced by the various hierarchical control s tructur es . Although the concepts of hi e rarchical and distributed control have been discussed mostly in the context of process-type systems, e.g. chemical, steel, e l ec tri c power, they are r eadily extended to the class of discrete even t sys t ems , e . g . manufacturing or assembly t ype sys t e ms. As an example of the former, we may consider a job shop consisting of vario us computer controlled me tal working machi nes (e.g . lathes, milling machines, grinde rs). The first l evel is concerned with feedback control of tool pos ition relative to the wo rk s urface to follow a sp ecified trajectory. A second level def i nes the trajectory to be follow e d according to th e r e quirements imposed by product specifications (dimensions, smoothness of finish, etc.) . Optimization may be included here, e .g. minimizing the time required for a given operation consistent with product quality constraints. A third level may be associated with the tasks of determining the optimum seq uence of operations to be pe rf o rme d by th e machine including the switching of tools. A fou rt h l eve l may be concerned wi th multi-machine ope r a tions in producing the

6

I. Lefkowitz and M.R. Buchner

final product - this includes sequencing and scheduling of the machines. A fifth level then includes overall scheduling of the shop to maximize productivity and/or minimize costs. Consideration of maintenance and repair schedule s obviously fit in here. From the standpoint of the multilayer functional hierarchy, we can draw direct analogies to the implementation, supervisory, adaptive and self-organizing functions. In particular, the adaptive function is important for compensating for tool wear and variations in the physical properties of the metal being worked (e.g. hardness). The self-organizing function plays a dominant role in the application of computer-aided design and manufacturing methodology (CAD/CAM) in the se nse that each new job may involve a restructuring of the physical system in terms of the kinds of machines and tools (and their sequencing) used in the manufacture of the product .

offer a variet:· of features such as: redundant power supplies and controllers (usually hot standbys with automatic swi tching), basic controller functions in a read only memory, diagnostic testing of components, and colocated process input/output modules. The available controller functions may include, in addition to the standard algorithms (e.g. PlO and leadlag), remote/local set point capability, digital logic functions, time and/or logical based sequencing of functions, remote adjustment of tuning parameters, alarming, and basic arithmetic calculations . One can normally combine these functions to d eve lop relatively sophisticated control algorithms. In addition, the capability of local operator display of output, controlled variable, and setpoint may be present.

While the first-level microprocessor-based controllers may perform tasks of reasonable sophistication, there remain many more complex tasks such as optimization, adaptation, schedDistributed Intelligence Control Systems [S,6,7,8juling, data base management, event logging, and management reporting which are generally more suitable for execution on a higher level As noted, the continual improvement in comcomputer system (e.g. a minicomputer or large puter hardware from microprocessors and main-frame computer). A determining characdigital signal transducers to mainframe comteri s tic of these higher level control funcputers and sophisticated data transmission tions is that the y do not have the stringent techniques has led to the d esign and impleresponse time requir e ments of the direct conmentation of distributed computer control systrol functions. Th e refore, a larger degree of tems, or the realization of control tasks on operating system overhead costs may be toleran interconnected multiple computer system ated at this level in order to have a more for industrial control systems. These complex efficient allocation of computer resources systems have four general characteristics among the competing tasks. which serve to represent a large class of systems important in industrial applications: The addition of a high speed computer - to-computer communication network to a collection of (i) Microprocessor and analog based control autonomous control computers allows the reall e rs for first level control functions . ization of tru e concurrency and resource sharing in a control system. The communication (ii) Minicomputers and large mainframes for system is r espo n sib l e for p roc ess data, prohigher level control functions. gram, command, and status information transfer among the control computers and devices . Com(iii) Physically dispersed structure implying me rciall y available communication networks for th e n eed for a computer - to- compu ter communica industrial process control provide a number of tion system . important features: Data link level protocols, r ed undant communication channels with auto(iv) Centralized operator interfaces having matic live break detection, significant error access to all process variables, control vari detection/correction capabilities, restricted ables, and system sta tus indicators. graceful degradation in the event of local failures resultin g in a disconnect of the Primarily as a r esu lt of the low cost of failed unit or, in the eve nt of a global commicroprocessors, memory, and peripheral inter munication failur e , in con tinu ed operation of face chips and th e potential for integrated a local net\,'ork. The communication rates of control, there has bee n a large increase in available systems ran ge from 100,000 bits per the use of LSI and VLSI ci rcuits in industrial second to 10, 000,000 bits per seco nd and proco ntrol systems components. Microprocessorvide for reliable communication of information based controllers have not only mimicked th e from component to component in industrial profunctions of analog controllers, but the y cess envi ronments. present significant advantages with respect to the range of functions which are available, The management. parameters, and the structure the ease of altering the control algorithm, of the communication svstem p la y a critical compatibility with other computer-based inrol e in the overall performance of the dis dustrial control d evices and other features. tribut ed system [S). Consequently, the de However, if ther e is a ne ed for very short sign of a distributed system must carefully response times then, either a dedicated microtranslat e the ne eds of the specif ic applicap rocessor-based controller havi ng responsition into a communication topology, level and bi lit y for only one or two loops, or an location of redundanc v , set of operating analog-based controller i s indicated. rul es , i.e. protocol, for the commun ication Microprocessor-based co ntrollers typically system. Common l y used structures include the

Status and Opportunities loop, star , global bus, or hierarchical topologies. Two r ece nt texts [9,1 0 ) and the articles [5, 6 ,7) are relevant references for the concepts, methodologies, software, hardware and structure of distributed computer systems. Each struc ture can be accessed in a number of ways with each structure/access mode yieldi ng different performance characteristics [ 5 ). As a simple examp le, a star system has a reliabilit y measure that depends critically on the reliability of the central switching unit. On the other hand, a bi-directional loop need not n ecessa rily fail upon the failure of a single interface. Similarly, the communication delay of a loop s tructur e ca n be expected to be higher than the delay of a shared global bus because of th e additional delay imposed by information being routed through device interfaces along th e loop. Even though ther e are significant problems and questions to be addressed in the design of a communication syst em for a distribut e d control application, the potential benefit s are quite appealing from the viewpoints of modularity, ease of expansion, reliability, performance, and cost [6). The operator interface is an essential component of a control sys tem for a large industrial plant. The n eed for displaying process information in a timely and appropriate manner is critical to achieving plant efficiency, safety, security, and integrated control. A computer control sys t em having distributed int e lli ge nce and an associated computer-to-computer communication system can provide for an economically attractive solution for an operator interface system. In particular, the distributed system allows for a centralized display and command capability having access to all process variables, controller inputs and outputs, as we ll as system status indicators. Currently available interfaces have the capability of providing the operator with an extensive amount of information. This information may be prese nt ed in a variety of forms, including individual loop displays, graphical tr e ndin g of process variables, plant-wide alarm display, and dynamic process control flow diagrams. The information display is usually s tructur ed in a hie rarchical fashion permitting ra pid access to critical data. Both color graphics and hard copy capability of screen information may be provided to improve the qualit y of the information flow across the man-machine interface. Not onl y can a singl e unit of this type be incorporated in a di stribu ted system, but additional operator interfaces of varying complexity can be conveniently located with only minimal modification or extension of the original system . In this context, the management r epo rting function can be approached in the same manner and with the same hardware as the primary operator interface. One need only provide the appropriate software for the computation and display functions required of the management/supervisory system. Of course, consideration of mu lti p le access points to either a centralized or distributed data base

7

describing the state of the control/process system presents considerable data base management problems. This is particularly true when error recov e r y and consistent state recovery are required in a real-time control environment. The above characteristics are genera l in nature, possessed by most state-of-the-art commercial, general purpose process control systems. However, there are many distributed control systems which exhibit only a subset of these chracteristics because of their highly application-oriented design. The structure of a we ll-designed distributed system often relects the control philosophy of the plant. The partitioning of computer responsibility into the various functions of the control hierarchies discussed ear lier, such as dir ec t control, supervisory control, scheduling operations and management repQrting may reflect a natural association of control tasks with computer attributes. For example, a microprocessor having limited arithmetic capability, rapid response time and limited memory is we ll-suited for performing a PID control calculation for several loops but not wellsuited for an identification algorithm computation. On the other hand, a minicomputer having considerable floating point capability, a relatively long response time, and considerable main and secondary memory might be very appropriate for executing a sched uling task but not for res ponding to an interrupt indicating a dangerous condition. Thus, the distributed system lends itself to hierarchical control strategies in a natural way . In industrial control applications, cabling is often a significant factor affecting the overall project. A distribut ed con trol system incorporating a line -shared digital data communication network allows for considerable savings in cabling costs, increased security of data, and ease of system expansio n. Associated with this last attribute is the ability to implement th e system in s ta ges . In particular, the distribut ed system allows, to an extent depending on th e overall sys tem structur e and the characteristics of th e communication system , an economica ll y viable so lution to changes requir ed over time to meet changing needs and prioriti es in an installed and operating system. In summary, the proper design of a distributed system for a process/plant control application permits the realization of a \~ide variety of advantages which directly impact on plant performance, cost, and safety. Research Studies There are a number of cu rr ently active research areas, such as decentralized control, adaptive control, fault toleran ce, discr e t eevent system analysis, and communication system design which are relevant to the study of hierarchical/distributed control of industrial systems. These studies may be divided into two broad categories:

8

1. Lefkowi tz and

1) The devel op me nt and ana l ysis of moni toring/dia gnosti c/c ontrol algorithms whi ch ca n tak e advan t age of th e features and capabil iti es of pa rticular di s tribut ed architectures . 2) The analysis of pe r fo r mance , attributes, and cons traint s of distributed sys t ems to p rovid e a basi s for new approaches, t e chni q ues , and me t hodo lo gies for improving sys t em pe rforman ce . An examp l e of a c urr e nt r esea r c h study in each of the se cat ego ri es wi ll be d esc ribed. De ce ntraliz ed Se l f - Tuning Regulator Ther e have been many r ece nt exte n sio n s of th e self -tun ing r egu lat or algorithm developed by As trom and Wittenmark [11] t o multivariable systems [12,1 3 ,14]. What i s common to these approa c h es is th e centralized computing r e quirement impli e d by the se l f -tun ing function . In one new approach considered [15], advanta ge is taken of th e di s tribut ed intelligence comp utational framework . Given a sys t e m with multip l e outputs, a modified sing l e input single output se lf-tunin g r egu lator i s applied to each output. A modified cost criterion is us e d for th e d es ign of each r egulator so that it minimizes a weighted average of the vari ance s of all of the sys t e m outputs . Th e r e lativ e weighting of the s ub sys tem varian ces are det e rmined by th e d eg r ee of coupling between subsystems . Whil e the re s ulting controllers are r es tricte d complexity controllers [16], the decentralized controllers, as described, make use of information transmitt e d from o th e r subsystems and, as a r es ult, exhibi t several valuable prop e rties. In particular, si mulation r es ult s show r e ducti o ns of both the contro l and output variances of th e co ntroll e d p l an t as compared to a co mpl e t e l y decentralized controller . Furth e r, as a r es ult of th e s tru c tur e of th e algorit hm and the pa tt e rn of information ex change be tlvee n local contro ll e r s , communication requirements ar e d ec r ease d as compared to a fu ll multivariab l e minimum variance con troller. In addition, if t he computer -to-computer communication l ine fails, the resulting control st ructur e ca n maintain a degraded b ut still acceptable d ecent r a liz ed adaptive co ntrol. Anoth e r approach to t he se lf-tunin g regulator in a di s tri buted compu t er con tr o l envi ronme nt is base d on a t wo -time sca l e approach moti v at ed by th e fu ncti o n a l mu ltilaye r control hi e rar chy described in a n ea rli e r sec ti o n. Specifical l y , t he direct co ntrol function is impl e me nted by the microproces so r control l ers operating in th e time scale of the plant dy namic s . The identification and up datin g comp utation s are ca rri e d ou t on a highe r l eve l computer on a much s lowe r time sca l e . This ap p roach has a s po t en ti a l advantages: only pa rtiall y d eg rad ed pe r fo rmance in the event of a single p ro cesso r fai lur e , app r opriate matchi n g of control functions to processor capabilities , r eso ur ce sharing among many local direct

~.R.

Buchner

control functions, and r esultin g eco nomic savi n gs. Th e disadvantages of th e approach are impose d communication d e la vs , increased communications hardware/software cos t s , di s tribut ed data base p ro b l ems , a nd the possi bility of communication fau lt s a nd fai lur es. Distributed Control System De s i gn Methodology The d esign of a d i s tr ib ut ed computer control system is a comp l ic at e d, time- co n s umin g , and costly process . Probl e ms to be addressed include r es ourc e a llo cation schedules for th e control sof t wa r e , contingency p lans in th e eve nt of p roc essor or o th e r component failure, and meas ur es of sys t e m utilization and ex pected performance und e r diff e r e nt plant operating co nditions. One me thod b ei n g studied [171a ssumes that the control algorithms, th e d ec ision processes, and t he computations to be performed in i mplementing control of th e p lant a r e k nown a pri ori. These are d ec ompo sed into "ta s ks" which are co d e d into p rograms to be run on a comput e r processor. Th e anal ysis of t he p lant then ca n be broke n down int o seve ral r ela tion sh ip s which ca n be assemb l e d to analyze the effect of d es ign d ec i sio n s on the eve ntual performance of th e system . Th e following relationships ar e developed: 1. Effective ta s k times vs. plant performance 2. Ac tiv e control s tructure modes as a func tion of component failures. The ta s k s are pa rtitioned into two sets: pe r iodic and aperiodic. Each periodic task may be parameter iz e d b y two values: the time it tak es to execu te and th e period of exec ution. The ta sks in th e asynchronous task se t are not run perio dicall y , but on d emand. They ma y be pa rame t e ri zed by one tim e value - the time it take s for the ta sk to be invoked and run to completion after a tri ggeri n g eve nt has occurred. Th ese tim e parame t e r s a re d efi n e d as th e effec t ive task t i me (ETT) for the t asks . One expec ts the p lant to exhibit similar ETT performance s urf aces for a ran ge of s t a t e variable valu es , within some tol e ranc e . Reg ion s of th e p lant s tat e space which have simi lar pe rforman ce surfaces are ca ll ed ope ra ting modes of the plant. These operating modes are important because, inst ead of having to keep track of the ETT - perfo rmanc e s urfac e for each possible p lant s tat e vector , we need only do so f o r each ope rating mode . To determine feasib l e r anges of effective ta sk tim es for control s tru ct ur es , the sys t em r e so urc es required by each ta sk are identified . The term r eso ur ces , as u se d here, includ es information (s enso r readin gs , ta b l e e ntri es , and operator inputs) and capability (CPU time , array p roc esso r tim e , e tc.). Tasks also s u pp l y the system with resources (info r mation) in th e form of set-points, valve set tin gs , comp utat io nal r es ult s for u se bv other tasks, opera tor dis p la ys , etc. The p robl em of developing the r e l a tionshi ps

Status and Opportunities linkin g control s tructur e to ETT's can be formulated in th e context of a n e twork flow problem. In a distributed comput e r control sys t e m th e movement of r eso urces consisting of some piece of information (such as sensor readings, data base r eco rds, or intertask mess a ges) can be modeled as a flow of t he r e sou rc es from sources ( se nsor s , data bases , or task s) to users (in thi s case , a ta sk ) through s i gnal lines , processors, multiplexors, and op e rating sys t ems. If th e task is not exec uting on the same processor as th e so urc e of the information, the resourc e mu s t also flow throu gh communications programs and interproc esso r communication lines . Constraints on the flow rat es are impose d to prevent more capabi lit y being utiliz ed by th e ta sks than is present in th e svs t em . A subnetwork i s d eve loped for each processo r in th e control sys t em. To model th e e ntir e con trol s tructur e , th e n, we int e rconn ect these p roc essor s ubne ts with communication lines, modeled as n e twork arcs, and impos e s imilar corresponding constraints that r ep r ese nt capacity limitation s . Finding feasible resource allocations r e duc es then to so lving t he network flow problem of insuring that n ee d e d resources flow from their sources to th ei r u se r s wh il e conforming to th e impos e d capacity constraints of the individual network arc s and of th e arc collections. The sum of th e delays encountered during the moveme nt of r eso urces to a task becomes the ETT for that task. Th ese effective ta s k times are then u se d wi th th e ETT-plant performance surface s to es timate th e performance of th e plant in any of its operating modes. The control sys tem may be mod e led with certain components not pres e nt, such as would be the case if they had failed during operation, in order to obtain estimates of th e d eg raded p lant performance under th ese conditions. Using the se r e lationships and examining different structural configurations allows, th e n, for th e logic a l design of th e computer control system wi t h consideration of th e effec ts of various control s tructur es on sys t em performance and t he r esu lt s of component failures. Future Potential There is no question that t he trend to emp lo y computers in an incr easi ng variet y of functions in indus trial control systems, which began in the 19 50 ' s continued in the 60's, and which was accelerated in the 70's, will continue at its current ra pid pace . The problems and n ee ds of today ' s complex plant svstems focus attention on such areas as r e liabilit y , integrated control, operator display, and cost. As the sYstems become more s trongl y coupled, as demands for improved p roductivit y and p roduct quality increases, as the n ee d for higher l eve ls of r e lia bi lit y increases, and as th e in fo rmati on and data p roc essing r eq uirements g row , th e concomitant demands and requirements on multiple compute r- based cont rol sys t ems wi ll dlso g rm,'. Based on hi s close association and ex t ensive work with industry, partic ularl y t he s t ee l

indus tr y , Wi ll iams [ 1 8) has identified t he following trends in distributed computer control ove r th e n ea r t e r m ,vhich, he expec t s , "ill maintain the momen tum in indu s trial process applications: 1 . The d eve lo pme nt of h i gh l y r ed undant sys t ems including backup powe r s uppli es and failsafe p rovi sio ns within th e comp uters and processors themselves, t o sa tisf y th e growing ex pectations for "absolute" system r e liabilit y. 2. Increased use of dis tribut e d multiplexing and distributed lo ca l control un i t s to sho rt e n analog data pat h s . Al so , with continuation of present trends in increased capabilities and d ecreased costs of microprocessors, th e r e will be decid e d incr eases in distributed comp uting. 3 . Con~u n ica t ions among dis tribut ed multiplexors and processo r s and among u ppe r l eve l hierarchy comp ut e r systems wi ll be by lin e shared (co- axia l o r fiber-optics) data highway sys t ems. 4. Concentration of man/machine int e rfac es into minimal-sized CRT-based consoles with associated keyboard sys t ems . Th e re will be an increasing trend to ge n e rali ze d sys t e ms with n ee ded specific function gene ration and prese ntation sof twar e produced for th e application involved. 5.

Standardization of control sys t e m s oftware.

6. Exp ec ted reduction s in costs of electronic components wi ll be offset by greater complexity built into the control sys tem. This complexity ma y comprise r e dundanc y or other fail-safe techniques , self-contained diagnostics, packa gi ng for easy maintenance by larg e unit r ep lac e ment, e tc . 7. Acce l e rat e d r esea rch and development of on-line dia gnos tic techniques and se lf-d iagnostic sys t ems for easy maintenance. Cl ea rl y , the r e are many potential be nefit s of a well-designed dis tribut ed computer control sys t em. In par ticular, one can cite advanta ges s uch as: increased fault tol e ranc e , r e liabilit y , and availability; natural impl eme ntat io n of hierarchical co ntrol algorithms, modularity, and flexibility; ease of design, r ea lization and maintenance; evo lutionar y construction, and r ed uc ed wiring costs. Currently available distri bu ted control systems only partly r e aliz e the se po tentials due to a combination of eco nomic factors, technical constraints, and the current s tat e of understandin g of th ese complex sys tems . As noted ea rl ie r, while th e capabi lit y for h ig he r l eve l or advanced control functions is often provided, t he r e are t vp ically conside ra b l e difficulties in th ei r r ea li za tion, ran ging f rom a la ck of s u i ta b l e interfaces and/or app r op r itl t e higher-level protocols to a lack of vendor-supplied co ntrol analysis tools and sophisticated functions. Thus far, t he r e seems to have been only mi n ima l attention gi ven (as r ef l ected in current p rodu c ts) to extending system capabi lit ies beyond on-line

10

I. Lefkowitz and M.R. Buchner

control and monitoring of the plant. For example, computer-aided design tools, identification techniques, and simulation have the potential for significantly assisting a control engineer in the design and start-up of an on-line control system; however, industrial process control vendors have only started to provide these capabilities integrated into their distributed systems. It should be noted that the major process control vendors are currently actively involved in incorporating these capabilities into future products. Further, distributed architectures provide a natural environment for cost effective implementation of these functions. As noted, one of the significant advantages of a distributed system is the potential for increased "survivability." By this is meant the ability of the system to perform satisfactorily a given set of functions over a particular time horizon. This property is manifest in two distinct ways: fault avoidance and fault tolerance. In the context of fault avoidance, a distributed system allows the designer the flexibility of allocating redundancy, pre-testing, and quality control to the extent that is required for a given component of the overall system. Therefore, if a given regulatory control function requires rapid response and tight control, then it may be appropriate to provide redundancy, at the loop level, for this function. On the other hand, if a loop is not critical or can tolerate a moderate response time, then it may be appropriate to allow for backup in a higher level processor and accepting some degradation of performance. By appropriately allocating fault avoidance techniques, the designer can achieve an economic solution to the problem of achieving a sufficiently high availability. On the other hand, the dynamic reconfiguration of one component to perform the functions or tasks of a failed component can also yield increased survivability. Of course reconfiguration must be accomplished so as to assure adequate performance of basic functions. Clearly this dynamic aspect of survivability also affects the economics of the problem solution by providing a complementary alternative to fault avoidance. Although commercially available systems exploit some aspects of fault avoidance, fault tolerance and fault containment techniques (i.e. the isolation of the effects of a particular fault or failure), they have generally limited reconfiguration capabilities. It is expected that the use of these techniques will be enhanced with the continued advances in LSI (VLSI) techniques and technologies resulting in increased reliabilities of components, less costly redundancy, and further advances in digital communication, e.g. higher speeds. The potential thus exists for a wide range of choices available to the user with respect to reliability, resource sharing, and complexity. While there can be significant increases in reliability at the lower levels of control through increased use of redundancy, resource sharing at the higher levels of control can provide for reduced cost and increased

flexibility. Distributed systems offer a unique opportunity to provide for the coordinated use of a large collection of resources ranging from input/ output modules to microprocessor controllers to large mainframe computers. This potential has only been partially exploited to date. From the user's point of view, however, this capability offers significant economic advantages with respect to integrated on-line control and monitoring on the level of a production unit, a plant, or a company. What is required to fully realize this potential is the ability to provide communications between the various elements or resources. This must be at both a hardware level and a software level. Standardization of hardware interfaces and communication protocols at all levels or the availability of suitable gateways to provide for protocol translation for a set of required protocol levels can facilitate this communication for the coordination and use of resources. The development of these gateways or the interface standardization of hardware/software interfaces will allow a full realization of the potential of distributed systems for resource sharing and increased resource utilization. One notes that in the manufacturing area it is possible, with commercially available systems, to go through the process of design and optimization of a discrete part using computeraided design facilities, translate the design into an appropriate parts program for a machine operation, download into a numerically controlled machine, e.g. lathe or mill, and have the machine produce the part. In the process industry, this is not currently commercially possible. While a distributed system is not essential for providing the capabilities required for all the processes of design through to implementation, significant decreases in start-up times and problems associated with start-up of the system may be realized if design functions can be coupled easily with on-line operation. The flexibility, modularity and range of resources that a distributed control system may offer could facilitate this process. In addition to the above points, the capabilities of a lower level control computer (e.g. microcomputer-based controller), may be greatly enhanced over currently available models. Clearly with increased research and development efforts directed at higher level control functions coupled with the capacity and performance expected of future microcomputers, one would expect that these control functions, e.g. adaptive and supervisory control, will be moved closer to the plant, subject to appropriate communication and physical constraints. The use of advanced control algorithms at appropriate points in the distributed system offer the potential for improved control with minimal increase in cost. Also the need for improved control of complex, strongly coupled industrial process systems implies the use of hierarchical control

Status and Upportunities structures. This, in turn, can be reflected in the structure, topology, protocol, and capabilities of distributed control systems for industrial applications. As a result of the potential of these advanced control algorithms and control structures, they are currently being investigated by vendors, users, and university researchers. In this regard, one should cite the potential for a more complete integration of discrete and continuous control. This is particularly important both with respect to the lower levels of control and the coordination of upper and lower levels of control. Finally, one should cite the potential for the use of on-line diagnostic procedures for both the controlled system and the computer control system itself, "smart" sensors and actuators and a wide spectrum of appropriate operator input/display devices which can be easily interfaced to the distributed system. It should be emphasized that, while many of the potential benefits noted here can be associated with a centralized single processor control system, distributed systems offer the potential for a more attractive solution from the point of view of cost, reliability, modularity, flexibility, and ease of expansion. A very important additional factor is that this is the direction that the vast majority of vendors have chosen to pursue for industrial process control.

2.

Lefkm"itz, I., "Integrated Control of Industrial Svstems," Trans. Royal Society, London, vol. 287, 1977.

3.

Lefkowitz, I., "Hierarchical Control in Large Scale Industrial Systems," Chap. 4 in Large Scale Systems, Y. Y. Haimes, editor, North Holland, 1982.

4.

Lefkowitz, I.,and A. Cheluistkin, "Integrated Systems Control in the Steel Industry", Report of the International Institute of Applied Systems Anal y sis, Laxenburg, Austria, CP76-l3, 1976.

5.

Schoeffler, J. D., and C. W. Rose, "Distributed Computer Intelligence for Data Acquisition and Control," IEEE Trans.Nuc. Sci., vol. NS-23, No. 1, Feb. 1976.

6.

Fraade, D.J., and S. Gast, "A Survey of Computer Networks and Distributed Control," Digital Computer Applications in Process Control, IFAC, 1977.

7.

Kahne, S., T. Lefkowitz and C.W. Rose, "Automatic Control by Distributed Int e lligence," Scientific Ame rican,Jun e 1979.

8.

Buchner, M.R., and I. Lefkowitz, "Distributed Computer Control for Industrial Process Systems: Characteristics, Attributes and an Experimental Facility," Control Systems Mag., CSM-2, 8-14, 1982.

9.

Weitzman, C., Distributed Micro/Minicomputer Systems, Prentice Hall, 1980.

Summary Distributed control systems represent a relatively new approach to the control of complex industrial process systems. These control systems combine exciting possibilities for the realization of advanced control algorithms, hierarchical control and dynamic controller reconfiguration with attributes of flexibility, versatility, reliability, and maintainability. The realization of many potential advantages of distributed control depends critically on the proper design of the structure, component interactions, control tasks, and the communication system. In this regard, we have outlined the fundamental aspects and principles of hierarchical control and the distribution of intelligence for industrial control systems. We have also presented examples of the implementation of these current practice along with trends that we expect these systems to follow in coming years. The prospect and potential of these systems is exciting with respect to both the need for the development of analytical tools to deal with the problems and questions which are raised and also opportunities for the realization of truly integrated systems control in a variety of industrial applications.

10.

Bowen, B.A., and R.J.A. Buhr, The Logical Design of Multiple-Microprocessor Systems, Prentice Hall, 1980.

11.

Astrom, K.J. and B. Wittenmark, "On SelfTuning Regulators," Automatica, 9(2), 185-194, 1973.

12.

Borison, U., "Self-Tuning Regulators for a Class of Multivariable Systems," Automatica, 15, 209-216, 1979.

13.

Koivo, H.N., "A ~lultivariable Self-Tuning Controller," Automatica,16, 351-366, 1980.

14.

Bayoumi, M.M., K.Y. Wong and M.A.El-Bagoury . "A Self-Tuning Regulator for Hultivariable Systems," Automatica, 17, 572-592, 1981.

15.

Costin, ~!., and M. Buchner, "A Self-Tuning Regulator for Distributed Control Svstems," submitted to Automatica, 1982.

16.

Goodwin, G.C. and P.J. Ramadge, "Design of Restricted Complexity Adaptive Regulators," IEEE Trans. Auto. Cont., AC-24,583-588,1979.

17.

Buchner, ~1., and v. ~!atula, "A Framel,ork for the Design of Survivable Distributed Systems, Part I: Communication Systems," HIT/ONR Workshop on C3, San Diego, 1981.

18.

Hilliams, T.J., "Computer Integrated Control," Proc. ISA/82 International Conf., Philadelphia, October 1982.

References 1.

Lefkowitz, I., "Systems Control of Chemical and Related Process Systems," Proc.6th IFAC World Congress, Cambridge, HA, 1975.

JJ

I. Lefkowitz and M.R. Buchner

12

r-----------.

I

CO~TROLLER

Fig. 1

j)A'l'A

I

I!\FORMATION PROCESSOR

Basic Control Configuration

Fig. 3

Multilevel Control Hierarchy

!i,',.St::

Fig. 2

Functional Multilayer Control Hierarchy

Fig. 4

Multilevel Control Structure in Steel ~laking