- Email: [email protected]
G11.2 : The Impact of Distributed Computer Control Systems on Software
THE IMPACT OF DISTRIBUTED COMPUTER CONTROL SYSTEMS ON SOFTWARE H. U. Steusloff Fraunhofer-Institutefor Informution- and Data-Aocesszng, Sebastian-Kneipp-StraSe12-14,0-7500Karkruhe 1, Federal Republic of Genurny
Abstract. During the past years demands for automation of technical systems, qenerated the todays structural change from centralized, single comnuter control systems to distributed computer control systems. The results of this change are - on the one hand - improved capabilities and properties of the automation hardware system; on the other hand there is a risk of increasing software expenditure due to those structural properties of distributed com?uter systems. This paper starts from an anylysis of the impact of distributed computer control systems on software and shows, that - in the field of application programming only higher level languages or problem-oriented languages will cope with the arising problems. The necessary supporting system programs are introduced, but the main importance is attached to the design of a well suited application programming language. This language design is derived from two assessments using systems and decision theory and is classified into a hierarchy of application programming aids. The realization of the language design is based on the existing language P E A R L (Process and Experiment Automation Realtime Language), which has to be extended (but not altered) for the description of structural and fault tolerance features of distributed computer control systems. The implemented language elements are explained by examples. A final view of the utilization of higher level languages for distributed microcomputer systems considers the rapid development in this field of hardware. Keywords. Distributed computer control; microprocessors; programming languages; redundant systems; fault tolerant systems. INTRODUCTION
There are two trends determininq the todays situation in computer control, the increasing demands for automatin? technical systems and the economically suitable availability of V L S I based com?uting hardware. Considering those demands we have to care of
-
.
ten years (Kniffler, 1976) The only answer to this quantitative challenge is a qualitative change of automation systems; quantitatively oriented solutions, e.g. further increase in capacity of a central control computer, would only be a short term solution.
increasing complexity of production methods for higher product yuality and flexibility, - operation of plants very close to their technoloqical limits to save eneray and investments, - intensified safety and reliability regulations e.g. for invironmental protection.
The only possible qualitative change of computer control systems for plant automation is changing the structure of the computer system from a centralized design into a distributed, possibly decentralized system.
In electric power ?lants, this resulted in an increase of the number of measurement points and actuators by a factor of approx. five during the last
Vithout discussing in detail all the features of distributed computer control systems we have to mention them, as
far, as there exists an imgact on programminq these systems:
-
Physical distribution of the computer systems means communication between the single computers via an explicit communication system (Heger, 1979).
- Many computers ("computer stations") in the distributed system provide a multiplication of resources (e.?. CPU's, storage units, I/O-units), giving system redundancy in the sense of an improved availability of resources for the most important tasks of the control system and, by this, fault tolerance (Syrbe, 1975). This kind of redundancy is called "function-sharing-redundancy".
- Distributed, redundant resources to~etherwith a communication system allow the reconfiguration of the control system in the case of faults, and increase availability as well as the safety of critical control functions.
-
Decrease of the com~lexityof tasks in the distributed computer stations is accompanied by an increase of comnrehensability and imnroved modularity of the automatic control system. These properties will help to tackle with the increasing overall complexity of the entire control system.
All these features have to be supported by suitable programming aids, which will decisively affect the utilization of distributed computer systems. IJp to he nearest past, application
programming of realtime computer control systems was done by describinq the processing functions, using mostly an assembler as programming lanquage. This was sufficient, as long as the hardware structure of the comguter system was fixed, except for a variation of quantity parameters (storage capacity, number of 1/0 units etc.). Changes in hardware structure after system faults either meant total failure of the system or switching to identical redundancy, or had to be trbated by the procram as variants of the processing functions (e.g. processinq of error messaTes of the I/Ochannels). For complex computer systems like automatic control systems (e.g. in a star structure and with high availability) the error.processing (exception handling) part of the programs are often larger than the actual processing functions. A further part of error processina is frequently integrated into the operating system of the nrocess computer and cannot be
influenced by the user. Reactions to such errors often have to be formulated in another language, the control language of the operating system. Distributed computing systems with function-sharing redundancy are subject to structural variations due to extensions and to their actual operating state. Therefore the proqrams have also to describe - besides the processing functions - the structure of the system and its communication. The advantages of function-sharing redundancy can only be exploited if all significant states and structures of the system, as well as the corresponding exception handling, are explicitely described in user programs. It should be formulated, easily recognizable, and in % programming languaqe, under which operating conditions and states the distributed computing system exhibits a certain capacity.In the same way, it must be made clear, for which operating states there is no redundant system capacity available and how large the degradation in performance may become in those states. This should even be readable by technical supervision authorities. Within a uniform language design, the software aids for structural description should be separated from the means of describing processing functions, in order to make programming of distributed automatic control systems not substantially more complicated than that of centralized systems. All these necessary features of a programming language for application in computer control systems are required by the impact of system distribution. The only way for providing these features, we feel, is the use of higher level languages. Today there is an intensive discussion about the properties of higher level programming languages in realtime apnlications. To examine this class of languages more closely, fig. 1 shows a hierarchy of language levels. On top of this hierarchy there is the level of very high level languages (VHLL), which are dedicated to special problems; therefore they are often called problem-oriented languages (POL). Higher level lanauaqes like FORTRAN or PEARL are universal, operative languages, i.e. not problemoriented-Intermediate languages usually are machine-independent and either assembler-like or operator-languages. They can be intergreted, mapped to a machine-dependent assembler or directly translated to machine-code. The snecial class of system implementation lanquaqes (SPL), like CORAL or Ada, can be located between the HLL and
Impact of Distributed Computer Control Systems
and VHLL. The strong lines in fig. 1 indicate the use of a HLL as an intermediate language for VHLL, employing then an IL to serve heterogenous computing systems. It may be convenient to interpose an assembler language for easier testing and software portability (dashed lines) The dashed-dotted lines show the way via a SPL, which generally will produce machine-level programs but may also use the IL-way (dotted line) . The x-line is of some importance: In future, the SPL will probably be used to implement HLL as well as serving for the production of machine-level programs.
.
.
4
should be illustrated. These terms furtheron are used for language constructs which are syntactically & semantically completely defined in the language itself. A counterexample is given by the language element "CALL procedure-name (parameter, ) Neither the performed function of this procedure is defined by its name nor are the meanings and the dimensions of the parameters defined semantically in the language where such a CALL is used. For the purpose of portability, readability and self-documentation of programs it is very necessary to employ genuine language constructs.
... ".
LANGUAGE LEVEL GRAPHIC AND VHLL
" F I L L I N THE VHLL
0 SYMBOL LANGUAGES
BLANKS"
LANGUAGES
.-.-.
APPLICATION
./
LEVEL
PEARL FORTRAN
HLL
1-
PL/I
I
+ ADA
4
S L
CORAL SYSTEM LEVEL
IL 1
u
u
Fig.1: Lanquage Levels and Interdependencies It should be stated here, that only the HLL and VHLL are intrinsically able to provide the necessary features of application languages for distributed computer control systems. The general reason is the (required) support of those languages by program production systems, operating systems and a set of runtime procedures, usually called runtime system, which all perform a mapping of complex, machine-independent language constructs to machine level instructions. This is a reasonable way of integrating further special and qenuine languaqe constructs for the descriction of system structures, function-sharing redundancy, system faults, configuration and -reconfiguration, communication and synchronization in distributed computing systems. Concluding this introduction the terms "genuine language construct (elenent)"
DESIGN OF AN APPLICATION PROGiZAMMING LANGUAGE FOR DISTRIBUTED COMPUTER SYSTEMS To investigate the necessary features and elements of a realtime application language for distributed computer systems it is necessary to look for the features of the automated technical process. The structures of the technical process and the appropriated automating system are related by a principle of duality, derived from Kalman (1969) and formulated by Syrbe (1978). From this principle of duality, complemented by a principle duality between realtime automating systems and programming languages (Steusloff, 1980) it is possible to derive the necessary elements of a suited programming language. This way of language design guarantees the completeness of the design; it is given by Steusloff (1977) in some detail (e.g. properties
H.U. Steusloff
=
sets of I/O-valuesand types; functions = algorithms; structure = communication and data-way tables). Here it is not possible to explain these details but we will discuss the complete language design in the following.
All language elements not common in well known and wide-spread programming languages are double-framed in fig. 2. A dotted frame indicates that in some very modern languages there are some language elements of this type.
Concerning the desirable structure of such a language, it is reasonable to consider the process of programming as a decision process. Programming demands a sequence of decisions of the programmer,concerning the proper choice and use of language elements. This decision process will be easier - as it is known from decision theory if language elements are grouped together in a hierarchy of semantically related sets, e.g. functional description elements, structural description elements etc.
Functions and Algorithms Description This language division starts with the well known language elements for the definition of data and program objects. For distributed computing systems the existence of global objects is essential; these global objects (e.9. global semaphores) are necessary means for system-wide communication and synchronization. The language elements for formulating algorithms also are well known in all higher level languages, whereas language elements for input/output are usually only available for the input/ output from/to standard peripherals and not for process peripheral devices. Here CALLed are widely in use, i.e. non-genuine language elements.
Following these assessments we can now design an application language for distributed realtime computing systems (fi9.2) .The presented language consists of two divisions with language elements for the description of the system structure and the functions of the system, the latter ones containing the language elements for the description of system properties. Each of the two divisions comprises three subgroups which, on their part, are again subdivided down to the single language constructs.
I
Language elements for real-time program flow control and communication control are very new in higher level programming languages. Nevertheless, they are most important to achieve portable real-time automation programs, especially with resepct to
HIGHER PROCESSPROGRAMMING LANGUAGE
STRUCTURAL DESCRIPTION
FUNCTIONAL DESCRIPTION
D A T A AND PROGRAM
TO FAULTS
TO FAULTS
I
Fig.2: Basic Structure of an Application Pro~rammingLanguage for Distributed Automation Systems
Impact of Distributed Computer Control Systems
time-dependent pronrams, where the use of physical time-units instead of counter-intervals is required. Resources and Confiquration Description This language division comprises all the structural description elements. These are the description elements for the medium-term unchanged features of each computer system in a distributed system structure, and finally, language elements for the description of data ways and communication links. All these language elements today are not contained in computer programming languages, because their necessity. arises only with distributed computer systems, where the structure of a system cannot be predefined due to structural changes during the operation and the life-time of a system. Here the impact of distributed computer control systems on programming is evident and there is no doubt, that assembler-type programming aids are not capable of being extended towards the description of system structures. There is some discussion about the necessity of an explicit description of the program configuration within a distributed system versus an automatic production of such configuration by the system itself. At present, we feel that an automatic distribution of programs is not feasible due to the lack of a suitable method for the formal description of program semantics. This would be necessary to determine, for any system state, the optimal assignment of programs and processors. Today only the system designer will be able to perform this task, also and especially for nonregular system states. Reactions to Non-Regular System Operating States Most of the language element groups at the third level in fig. 2 contain elements for the reaction to non-regular system operating states, i.e. for exception handling. Exception handling too has to be formulated explicitely by the programmer, because today there is no possibility of determining automatically the reactions to system faults. The related language elements should be separated from other language parts to support transparency and, as far as possible, avoid influences from the change of other system features. On the other hand these language elements are part of the application programminq language in order to maintain the uniformity
of the description of automation systems by such a language. Conclusion. There is in fact an impact on software, generated by the structural and fault tolerance properties of distributed computing systems. On the one hand, this impact means a push-forward in programming by the arising requirements of strucbre describing language parts. On the other hand, according to fig. 2, there are many well proven programming aids for the function description in automation programs, which should be a basis to absorb the impact and direct it into an evolution. REALIZATION OF A PROGRAMMING LANGUAGE FOR DISTRIBUTED COMPUTER CONTROL Today there are many well established higher level languages for the description of mathematical an technical algorithms, but they all fail concerning structuredescription elements (cf. fig. 2, double framed boxes). In Steusloff (1980) a value analysis is carried out in order to find a most suitable candidate language as a basis for a language according to fig. 2. Candidates are Realtime - FORTRAN, RTL/~,BASEX, PROCOL, PEARL as application languages and JOVIAL, CORAL, Ada as system implementation languages. The candidate with the higheest value of utility comes out to be PEARL (Process and Experiment Automation Realtime Language). Furtheron - as an example, how to answer the impact of distributed systems on software - the language PEARL with extensions for distributed computing systems will be called "Multicomputer-PEARL"
.
The Programming Language PEARL The programming language PEARL has been defined and developed by a German group of people from industry and scientific institutions with the support of the German Ministry of Research and Technology (data-processing programs). The algorithmic part is related to ALGOL 68 and PL/I, among others comprising pointers and the definition of compound data structures and types. The programstructure is block-oriented; a MODULE is subdivided into TASKS and PROCEDURFs, on the MODULE-level as well as on the block level (Kappatsch and others, 1977) .
H.U.
Steusloff
There are genuine language elements for realtime proyram flow control and explicit synchronization of TASKS for event-controlled program flow and for the control of parallel activities. Besides the language elements for standard-I/@ and formating, there are special elements for process-I/@. This all is part of the PROB1,EPI-division of a PEARL-MODULE.
The STATIONS-division serves for the description of the static structure of a distributed computing system. It contains a hardware description (processor. storage, errorcodes and status identifiers, peripheral devices) and a description of the local operating system.
- The LOAD-division is used to describe the program-structure (confisuration and -reconfiguration) of the entire distributed computing system, depending on the hardwaredetected system status. By use of LOAD-statements, the functionsharing redundancy of the system is utilized and pre-planned.
In a second division, the SYSTEPI-division, a PEARL-PCODULE contains an I/O-data-way description. This is one of the outstandinq features of PEARL, giving means for some structural description of a single-computer system by a separate division of a MODULE. On the basis of these features, PEARL is the most suitable candidate for a language according to the design of fig.2. However, PEARL does not contain lanquage elements for the structural description of a distributed computing system and its prourams. Furthermore, exception handling has to be programmed within the alqorithm part, even for structural or I/Ofaults (double framed blocks in fig.2). At present there are two completely defined versions of PEARL, Basic PEARL (Hruschka and others, 1977) and Full PEARL (Kappatsch and others, 1977). The first one has been standardized in the FRG (DIN 66253, 1979) and has been submitted to IS0 for standardization, whereas Full PEARL is in the process of standardization in the FRG. There are at least six implementations in the FRG and one from abroad which are or will be available in 1980 at the latest. Therefore P!ulticomputer-PEARL must not change standard PEARL in any way, rather than literally extend PEARL in such a way that a standard PEARL compiler would understand the standard program parts of a proqram written in Plulticomputer-PEARL. This can be achieved, accordinq to the above mentioned structural design requirements, by adding on new language divisions or using the technique of "pragmats", i.e. special coments. PEARL Extensions towards "Multicomnuter-PEARL" As an example, how to realize a realtine language for distributed computing systems, PEARL extensions are mentioned, forming a Yulticomputer-PEARL. There are three extensions, two of them being additional language divisions and the third one beinq integrated into the already existing SYSTEM-division of PEARL:
-
Language elements for formulating the use of replacement processdata-ways and replacement values according to the status of dataways and measuring devices and/or actuators.
All these language elements are explained in some detail in Steusloff (1980). As an example, the LOADinstructions and their execution by a so-called "Dynamic Loader" shall be described here.
Language elements for the description of proaram confiauration and reconfiguration, denendina on the operatinn state of the computing svstem, may be called "load instructions" (fig.3). They consist of a snecification of a destination computer station, of the so-called load priority,of an operating status condition and optional supplements. The load priority determines the importance of a program module relatively to other modules within a computer station (functional priority);this is to be distinguished from the operational priority of individual subprograms. The operating state INITIAL denotes the initial (bootstrap) and normal status, while the logical expressions in parenthesis describe non-regular operating states by means of statusidentifiers from the STATIONS-division. The optional specification of a starting number for the initial status describes the bootstrap-sequence of a proaram s stem. The optional attribute RES [IDENTt causes the bootstrap-loading of programs into redundant computer stations, in order to be available there after failure of the program loadinq device or to shorten the loss of processinq functions after timecritical program reconfigurations.
Impact o f D i s t r i b u t e d Computer C o n t r o l Systems
MODULE MOD1; LOAD ; TO STAl LDPRIO 5 INITIAL STAFTMOI; TO STA3 LDPRIO 5 ON(STAlPR,AMD. .NOT. STA3PR)RES; TO STA2 LDPRIO 5 ON(STA1PR.AND. STA3PR); SYSTEM; data-ways, correction of data, description of replacement data
.
PROBLEM; description of algorithms MODEND ; Fig. 3:
LOAD-division within a PEARL MODULE
The Dynamic Loader The execution of LOAD-instructions by the Dynamic Loader shall be explained usins fig. 4. There are two MODULES, MOD1 and MOD 2, with their LOAD-divisions underlying this example. The compiler for the LOAD-instructions pre-sorts the information, contained in all the LOAD-divisions of a proqram system, into the so-called Rtables (reconfiguration-tables) in such a way, that each R-table contains the information necessary for the actions of the corresponding computer station.
During INITIAL load time MODULE MOD1 has been loaded to computer station STAl for operation during standard system operatinq status. Additionally MOD1 has been loaded to STA3 in the RESIDENT-mode to run there in the case, that the status condition "STA1 PR AND NOT STA3PR" becomes true. Let us assume, that the identifier "STnPR" is the name for a status code with the meaning of "nrocessor of computer station n is out of service". MODULES PIOD2 and PlODK have been loaded to computer station STA2. The LOAD-division of MOD1 contains a statement indicating, that MODl has to run in STA2 in case of the true condition "STA1PB AYD STA3PR" (cf fig. 3)
The Dynamic Loader consists of two parts, a centralized Global Loader and distributed Local Observers in each comnuter station. The Local Observers act upon system status messages which have to be mailed through the entire system by the communication line. In the example, a sequence of actions is assumed as follows:
Global Loader
.
As soon as the processor of STA1 fails, the condition "STAlPR AND NOT STA3PR" in the R-table of STA3 becomes true and the residing MODULE ElOD1 is activated in STA3 by the Local Observer. If the processor of
-
K - Table Global Observer
1
STAI 1
Status
Module
I N I T ~ M O D I ~MOD 1
Load Module Library
Communication Line
STAZI
STA3
1
LDPRIO
5
INIT (MOD21 MOD 2 INIT (MODKI MOD K STAlPR MOD1 8 STA3PR
.
10 1
5
/NIT( I STAlPR MOD1 8 STA3PR
-
Fig. 4: Software-Configuration and -Reconfiguration
5RES
H.U. Steusloff
STA3 also fails. the condition "STAlPR AND STA3PR1'in the R-table of STA2 becomes true. The Local Observer of STA2 now has to initialize the downline loading of POD1 from the load module librarv into STA2 by the Global Loader. Let us assume, that the working storage of STA2 is occupied by MOD2 and MODK. Now the load priority plays its role; since LDPRIO of MOD1 is higher (indicated by the smaller number) than that of MOD2, the latter MODULE is terminated and MOD1 loaded into the storage area previously possessed bu PIOD2. After the repair of STA3 and STAl the reverse sequence of actions takes place. This example shows the i~.pactof distributed computer systems on application proaramming as well as on the supportinrr software. There are further supporting software units, influenced by the distribution of computers, like the program production system and the operating system, which cannot be treated here but are of fundamental importance in tackling with the problems of distributed systems (Heqer, 1979). On the other hand, they are the tools for makins the applications of distributed systems possible and economically feasible. FINAL REMARKS Today there is some discussion,whether hisher level languages (HLL) are suited for microcomputers, beins the outstandins devices in distributed control svstems. The answer is, that HLL are suited if certain conditions are observed. Efficiency problems can be met e.g. by preparing and using prosrammer's lanauaqe hand books containina advices on efficiency properties of special language constructions. By this, efficiency factors - compared with assembled programs - of down to 1.2 have been reached with PEAFL. Runtime supporting software (operating systems, including communication systems, runtime procedures) for a PEARL system will require 6K of 16bit words or less, if standard-I/O is not included but centralized to some dedicated computer stations in a distributed system. It will be very necessary to improve the modularity of operating systems and to facilitate the generating of load-modules dedicated to the local requirements. Finally the compilation of HLL like PEARL will require a working storage of about 32K of 16-bit words and e.g. a Winchester-type 8" disk storage, which is possible with the new 16bit microprocessors.
The impact of distributed computer control systems on software has created a push-forward onto the development of software techniques, allowing to cope with the increasing complexity of automation systems and at the same time maintaining the software expenditures. This paper should have shown that the above mentioned impact will be the occasion of improving automatic control technology in an evolutionary manner. REFERENCES DIN 66253 (1979). Programmiersprache PEARL, Basic PEARL. Deutsches Institut fur Normunq (DIN), DIN 66253, Teil 1. Heger, D., H. Steusloff, and M. Syrbe (1979). Echtzeitrechnersystem mit verteilten Mikroprozessoren. Bundesministerium fur Forschung und Technologie, Forschungsbericht DV 79-01 Datenverarbeitunq. (Realtime Computer System with DistriButed Microprocessors. Federal Ministry of Research and Technology (FRG) Report DV 79-01 Data Processing (in German). 364 pp. Hruschka, P. et a1 (1977). Basic PEARL Language Description. Kernforschungszentrum Karlsruhe (FRG). Report KFK-PDV 120, 242 pp. Kalman, R.E., P.L. Falb, and M.A.Arbib (1969). Topics in Mathematical System Theory. McGraw-Hill, New York. Kappatsch, A., et a1 (1977) Full PEARL Lanauase Description. Kernforschungszentrum Karlsruhe (FRG) Report KFK PDV 130, 462 pp. Kniffler, U. and Denkes, R. Einsatz von EDV-Anlagen zur Planung und Abwicklung von leittechnischen Ausrustungen fur verfahrens- und -kraftwerkstechnische Projekte. berlag VDE, Bezirksverein Frankfurt ./ M . . . 1976. D R . 21 - 32. Steusloff, H.U. (1 977) . Zur Programmierung von raumlich verteilten, dezentralen ProzeRrechnersystemen. Dissertation-Fakultatfur Informatik der Universitat (TH) Karlsruhe (FRG) 180 DD. Steusloff, H.U. (1986) Programming Distributed Computer Systems with Higher Level Languages. Proceeding of the IFAC-Workshop on Distributed Computer Control Systems; Tampa, Florida, USA, Oct.2-4, 1979. Peraamon Press, 1980. (to appear) . Syrbe, M. (1978). Basic Principles of Advanced Process Control System Structures and a Realization with Optical Fibres-Coupled Distributed Ilicrocom~uters.Preprints of the 7th Triennial world Congress, IFAC Helsinki. Pergamon Press, New York, pp. 393 401.
.
.
.
.
.
Abstract. During the past years demands for automation of technical systems, qenerated the todays structural change from centralized, single comnuter control systems to distributed computer control systems. The results of this change are - on the one hand - improved capabilities and properties of the automation hardware system; on the other hand there is a risk of increasing software expenditure due to those structural properties of distributed com?uter systems. This paper starts from an anylysis of the impact of distributed computer control systems on software and shows, that - in the field of application programming only higher level languages or problem-oriented languages will cope with the arising problems. The necessary supporting system programs are introduced, but the main importance is attached to the design of a well suited application programming language. This language design is derived from two assessments using systems and decision theory and is classified into a hierarchy of application programming aids. The realization of the language design is based on the existing language P E A R L (Process and Experiment Automation Realtime Language), which has to be extended (but not altered) for the description of structural and fault tolerance features of distributed computer control systems. The implemented language elements are explained by examples. A final view of the utilization of higher level languages for distributed microcomputer systems considers the rapid development in this field of hardware. Keywords. Distributed computer control; microprocessors; programming languages; redundant systems; fault tolerant systems. INTRODUCTION
There are two trends determininq the todays situation in computer control, the increasing demands for automatin? technical systems and the economically suitable availability of V L S I based com?uting hardware. Considering those demands we have to care of
-
.
ten years (Kniffler, 1976) The only answer to this quantitative challenge is a qualitative change of automation systems; quantitatively oriented solutions, e.g. further increase in capacity of a central control computer, would only be a short term solution.
increasing complexity of production methods for higher product yuality and flexibility, - operation of plants very close to their technoloqical limits to save eneray and investments, - intensified safety and reliability regulations e.g. for invironmental protection.
The only possible qualitative change of computer control systems for plant automation is changing the structure of the computer system from a centralized design into a distributed, possibly decentralized system.
In electric power ?lants, this resulted in an increase of the number of measurement points and actuators by a factor of approx. five during the last
Vithout discussing in detail all the features of distributed computer control systems we have to mention them, as
far, as there exists an imgact on programminq these systems:
-
Physical distribution of the computer systems means communication between the single computers via an explicit communication system (Heger, 1979).
- Many computers ("computer stations") in the distributed system provide a multiplication of resources (e.?. CPU's, storage units, I/O-units), giving system redundancy in the sense of an improved availability of resources for the most important tasks of the control system and, by this, fault tolerance (Syrbe, 1975). This kind of redundancy is called "function-sharing-redundancy".
- Distributed, redundant resources to~etherwith a communication system allow the reconfiguration of the control system in the case of faults, and increase availability as well as the safety of critical control functions.
-
Decrease of the com~lexityof tasks in the distributed computer stations is accompanied by an increase of comnrehensability and imnroved modularity of the automatic control system. These properties will help to tackle with the increasing overall complexity of the entire control system.
All these features have to be supported by suitable programming aids, which will decisively affect the utilization of distributed computer systems. IJp to he nearest past, application
programming of realtime computer control systems was done by describinq the processing functions, using mostly an assembler as programming lanquage. This was sufficient, as long as the hardware structure of the comguter system was fixed, except for a variation of quantity parameters (storage capacity, number of 1/0 units etc.). Changes in hardware structure after system faults either meant total failure of the system or switching to identical redundancy, or had to be trbated by the procram as variants of the processing functions (e.g. processinq of error messaTes of the I/Ochannels). For complex computer systems like automatic control systems (e.g. in a star structure and with high availability) the error.processing (exception handling) part of the programs are often larger than the actual processing functions. A further part of error processina is frequently integrated into the operating system of the nrocess computer and cannot be
influenced by the user. Reactions to such errors often have to be formulated in another language, the control language of the operating system. Distributed computing systems with function-sharing redundancy are subject to structural variations due to extensions and to their actual operating state. Therefore the proqrams have also to describe - besides the processing functions - the structure of the system and its communication. The advantages of function-sharing redundancy can only be exploited if all significant states and structures of the system, as well as the corresponding exception handling, are explicitely described in user programs. It should be formulated, easily recognizable, and in % programming languaqe, under which operating conditions and states the distributed computing system exhibits a certain capacity.In the same way, it must be made clear, for which operating states there is no redundant system capacity available and how large the degradation in performance may become in those states. This should even be readable by technical supervision authorities. Within a uniform language design, the software aids for structural description should be separated from the means of describing processing functions, in order to make programming of distributed automatic control systems not substantially more complicated than that of centralized systems. All these necessary features of a programming language for application in computer control systems are required by the impact of system distribution. The only way for providing these features, we feel, is the use of higher level languages. Today there is an intensive discussion about the properties of higher level programming languages in realtime apnlications. To examine this class of languages more closely, fig. 1 shows a hierarchy of language levels. On top of this hierarchy there is the level of very high level languages (VHLL), which are dedicated to special problems; therefore they are often called problem-oriented languages (POL). Higher level lanauaqes like FORTRAN or PEARL are universal, operative languages, i.e. not problemoriented-Intermediate languages usually are machine-independent and either assembler-like or operator-languages. They can be intergreted, mapped to a machine-dependent assembler or directly translated to machine-code. The snecial class of system implementation lanquaqes (SPL), like CORAL or Ada, can be located between the HLL and
Impact of Distributed Computer Control Systems
and VHLL. The strong lines in fig. 1 indicate the use of a HLL as an intermediate language for VHLL, employing then an IL to serve heterogenous computing systems. It may be convenient to interpose an assembler language for easier testing and software portability (dashed lines) The dashed-dotted lines show the way via a SPL, which generally will produce machine-level programs but may also use the IL-way (dotted line) . The x-line is of some importance: In future, the SPL will probably be used to implement HLL as well as serving for the production of machine-level programs.
.
.
4
should be illustrated. These terms furtheron are used for language constructs which are syntactically & semantically completely defined in the language itself. A counterexample is given by the language element "CALL procedure-name (parameter, ) Neither the performed function of this procedure is defined by its name nor are the meanings and the dimensions of the parameters defined semantically in the language where such a CALL is used. For the purpose of portability, readability and self-documentation of programs it is very necessary to employ genuine language constructs.
... ".
LANGUAGE LEVEL GRAPHIC AND VHLL
" F I L L I N THE VHLL
0 SYMBOL LANGUAGES
BLANKS"
LANGUAGES
.-.-.
APPLICATION
./
LEVEL
PEARL FORTRAN
HLL
1-
PL/I
I
+ ADA
4
S L
CORAL SYSTEM LEVEL
IL 1
u
u
Fig.1: Lanquage Levels and Interdependencies It should be stated here, that only the HLL and VHLL are intrinsically able to provide the necessary features of application languages for distributed computer control systems. The general reason is the (required) support of those languages by program production systems, operating systems and a set of runtime procedures, usually called runtime system, which all perform a mapping of complex, machine-independent language constructs to machine level instructions. This is a reasonable way of integrating further special and qenuine languaqe constructs for the descriction of system structures, function-sharing redundancy, system faults, configuration and -reconfiguration, communication and synchronization in distributed computing systems. Concluding this introduction the terms "genuine language construct (elenent)"
DESIGN OF AN APPLICATION PROGiZAMMING LANGUAGE FOR DISTRIBUTED COMPUTER SYSTEMS To investigate the necessary features and elements of a realtime application language for distributed computer systems it is necessary to look for the features of the automated technical process. The structures of the technical process and the appropriated automating system are related by a principle of duality, derived from Kalman (1969) and formulated by Syrbe (1978). From this principle of duality, complemented by a principle duality between realtime automating systems and programming languages (Steusloff, 1980) it is possible to derive the necessary elements of a suited programming language. This way of language design guarantees the completeness of the design; it is given by Steusloff (1977) in some detail (e.g. properties
H.U. Steusloff
=
sets of I/O-valuesand types; functions = algorithms; structure = communication and data-way tables). Here it is not possible to explain these details but we will discuss the complete language design in the following.
All language elements not common in well known and wide-spread programming languages are double-framed in fig. 2. A dotted frame indicates that in some very modern languages there are some language elements of this type.
Concerning the desirable structure of such a language, it is reasonable to consider the process of programming as a decision process. Programming demands a sequence of decisions of the programmer,concerning the proper choice and use of language elements. This decision process will be easier - as it is known from decision theory if language elements are grouped together in a hierarchy of semantically related sets, e.g. functional description elements, structural description elements etc.
Functions and Algorithms Description This language division starts with the well known language elements for the definition of data and program objects. For distributed computing systems the existence of global objects is essential; these global objects (e.9. global semaphores) are necessary means for system-wide communication and synchronization. The language elements for formulating algorithms also are well known in all higher level languages, whereas language elements for input/output are usually only available for the input/ output from/to standard peripherals and not for process peripheral devices. Here CALLed are widely in use, i.e. non-genuine language elements.
Following these assessments we can now design an application language for distributed realtime computing systems (fi9.2) .The presented language consists of two divisions with language elements for the description of the system structure and the functions of the system, the latter ones containing the language elements for the description of system properties. Each of the two divisions comprises three subgroups which, on their part, are again subdivided down to the single language constructs.
I
Language elements for real-time program flow control and communication control are very new in higher level programming languages. Nevertheless, they are most important to achieve portable real-time automation programs, especially with resepct to
HIGHER PROCESSPROGRAMMING LANGUAGE
STRUCTURAL DESCRIPTION
FUNCTIONAL DESCRIPTION
D A T A AND PROGRAM
TO FAULTS
TO FAULTS
I
Fig.2: Basic Structure of an Application Pro~rammingLanguage for Distributed Automation Systems
Impact of Distributed Computer Control Systems
time-dependent pronrams, where the use of physical time-units instead of counter-intervals is required. Resources and Confiquration Description This language division comprises all the structural description elements. These are the description elements for the medium-term unchanged features of each computer system in a distributed system structure, and finally, language elements for the description of data ways and communication links. All these language elements today are not contained in computer programming languages, because their necessity. arises only with distributed computer systems, where the structure of a system cannot be predefined due to structural changes during the operation and the life-time of a system. Here the impact of distributed computer control systems on programming is evident and there is no doubt, that assembler-type programming aids are not capable of being extended towards the description of system structures. There is some discussion about the necessity of an explicit description of the program configuration within a distributed system versus an automatic production of such configuration by the system itself. At present, we feel that an automatic distribution of programs is not feasible due to the lack of a suitable method for the formal description of program semantics. This would be necessary to determine, for any system state, the optimal assignment of programs and processors. Today only the system designer will be able to perform this task, also and especially for nonregular system states. Reactions to Non-Regular System Operating States Most of the language element groups at the third level in fig. 2 contain elements for the reaction to non-regular system operating states, i.e. for exception handling. Exception handling too has to be formulated explicitely by the programmer, because today there is no possibility of determining automatically the reactions to system faults. The related language elements should be separated from other language parts to support transparency and, as far as possible, avoid influences from the change of other system features. On the other hand these language elements are part of the application programminq language in order to maintain the uniformity
of the description of automation systems by such a language. Conclusion. There is in fact an impact on software, generated by the structural and fault tolerance properties of distributed computing systems. On the one hand, this impact means a push-forward in programming by the arising requirements of strucbre describing language parts. On the other hand, according to fig. 2, there are many well proven programming aids for the function description in automation programs, which should be a basis to absorb the impact and direct it into an evolution. REALIZATION OF A PROGRAMMING LANGUAGE FOR DISTRIBUTED COMPUTER CONTROL Today there are many well established higher level languages for the description of mathematical an technical algorithms, but they all fail concerning structuredescription elements (cf. fig. 2, double framed boxes). In Steusloff (1980) a value analysis is carried out in order to find a most suitable candidate language as a basis for a language according to fig. 2. Candidates are Realtime - FORTRAN, RTL/~,BASEX, PROCOL, PEARL as application languages and JOVIAL, CORAL, Ada as system implementation languages. The candidate with the higheest value of utility comes out to be PEARL (Process and Experiment Automation Realtime Language). Furtheron - as an example, how to answer the impact of distributed systems on software - the language PEARL with extensions for distributed computing systems will be called "Multicomputer-PEARL"
.
The Programming Language PEARL The programming language PEARL has been defined and developed by a German group of people from industry and scientific institutions with the support of the German Ministry of Research and Technology (data-processing programs). The algorithmic part is related to ALGOL 68 and PL/I, among others comprising pointers and the definition of compound data structures and types. The programstructure is block-oriented; a MODULE is subdivided into TASKS and PROCEDURFs, on the MODULE-level as well as on the block level (Kappatsch and others, 1977) .
H.U.
Steusloff
There are genuine language elements for realtime proyram flow control and explicit synchronization of TASKS for event-controlled program flow and for the control of parallel activities. Besides the language elements for standard-I/@ and formating, there are special elements for process-I/@. This all is part of the PROB1,EPI-division of a PEARL-MODULE.
The STATIONS-division serves for the description of the static structure of a distributed computing system. It contains a hardware description (processor. storage, errorcodes and status identifiers, peripheral devices) and a description of the local operating system.
- The LOAD-division is used to describe the program-structure (confisuration and -reconfiguration) of the entire distributed computing system, depending on the hardwaredetected system status. By use of LOAD-statements, the functionsharing redundancy of the system is utilized and pre-planned.
In a second division, the SYSTEPI-division, a PEARL-PCODULE contains an I/O-data-way description. This is one of the outstandinq features of PEARL, giving means for some structural description of a single-computer system by a separate division of a MODULE. On the basis of these features, PEARL is the most suitable candidate for a language according to the design of fig.2. However, PEARL does not contain lanquage elements for the structural description of a distributed computing system and its prourams. Furthermore, exception handling has to be programmed within the alqorithm part, even for structural or I/Ofaults (double framed blocks in fig.2). At present there are two completely defined versions of PEARL, Basic PEARL (Hruschka and others, 1977) and Full PEARL (Kappatsch and others, 1977). The first one has been standardized in the FRG (DIN 66253, 1979) and has been submitted to IS0 for standardization, whereas Full PEARL is in the process of standardization in the FRG. There are at least six implementations in the FRG and one from abroad which are or will be available in 1980 at the latest. Therefore P!ulticomputer-PEARL must not change standard PEARL in any way, rather than literally extend PEARL in such a way that a standard PEARL compiler would understand the standard program parts of a proqram written in Plulticomputer-PEARL. This can be achieved, accordinq to the above mentioned structural design requirements, by adding on new language divisions or using the technique of "pragmats", i.e. special coments. PEARL Extensions towards "Multicomnuter-PEARL" As an example, how to realize a realtine language for distributed computing systems, PEARL extensions are mentioned, forming a Yulticomputer-PEARL. There are three extensions, two of them being additional language divisions and the third one beinq integrated into the already existing SYSTEM-division of PEARL:
-
Language elements for formulating the use of replacement processdata-ways and replacement values according to the status of dataways and measuring devices and/or actuators.
All these language elements are explained in some detail in Steusloff (1980). As an example, the LOADinstructions and their execution by a so-called "Dynamic Loader" shall be described here.
Language elements for the description of proaram confiauration and reconfiguration, denendina on the operatinn state of the computing svstem, may be called "load instructions" (fig.3). They consist of a snecification of a destination computer station, of the so-called load priority,of an operating status condition and optional supplements. The load priority determines the importance of a program module relatively to other modules within a computer station (functional priority);this is to be distinguished from the operational priority of individual subprograms. The operating state INITIAL denotes the initial (bootstrap) and normal status, while the logical expressions in parenthesis describe non-regular operating states by means of statusidentifiers from the STATIONS-division. The optional specification of a starting number for the initial status describes the bootstrap-sequence of a proaram s stem. The optional attribute RES [IDENTt causes the bootstrap-loading of programs into redundant computer stations, in order to be available there after failure of the program loadinq device or to shorten the loss of processinq functions after timecritical program reconfigurations.
Impact o f D i s t r i b u t e d Computer C o n t r o l Systems
MODULE MOD1; LOAD ; TO STAl LDPRIO 5 INITIAL STAFTMOI; TO STA3 LDPRIO 5 ON(STAlPR,AMD. .NOT. STA3PR)RES; TO STA2 LDPRIO 5 ON(STA1PR.AND. STA3PR); SYSTEM; data-ways, correction of data, description of replacement data
.
PROBLEM; description of algorithms MODEND ; Fig. 3:
LOAD-division within a PEARL MODULE
The Dynamic Loader The execution of LOAD-instructions by the Dynamic Loader shall be explained usins fig. 4. There are two MODULES, MOD1 and MOD 2, with their LOAD-divisions underlying this example. The compiler for the LOAD-instructions pre-sorts the information, contained in all the LOAD-divisions of a proqram system, into the so-called Rtables (reconfiguration-tables) in such a way, that each R-table contains the information necessary for the actions of the corresponding computer station.
During INITIAL load time MODULE MOD1 has been loaded to computer station STAl for operation during standard system operatinq status. Additionally MOD1 has been loaded to STA3 in the RESIDENT-mode to run there in the case, that the status condition "STA1 PR AND NOT STA3PR" becomes true. Let us assume, that the identifier "STnPR" is the name for a status code with the meaning of "nrocessor of computer station n is out of service". MODULES PIOD2 and PlODK have been loaded to computer station STA2. The LOAD-division of MOD1 contains a statement indicating, that MODl has to run in STA2 in case of the true condition "STA1PB AYD STA3PR" (cf fig. 3)
The Dynamic Loader consists of two parts, a centralized Global Loader and distributed Local Observers in each comnuter station. The Local Observers act upon system status messages which have to be mailed through the entire system by the communication line. In the example, a sequence of actions is assumed as follows:
Global Loader
.
As soon as the processor of STA1 fails, the condition "STAlPR AND NOT STA3PR" in the R-table of STA3 becomes true and the residing MODULE ElOD1 is activated in STA3 by the Local Observer. If the processor of
-
K - Table Global Observer
1
STAI 1
Status
Module
I N I T ~ M O D I ~MOD 1
Load Module Library
Communication Line
STAZI
STA3
1
LDPRIO
5
INIT (MOD21 MOD 2 INIT (MODKI MOD K STAlPR MOD1 8 STA3PR
.
10 1
5
/NIT( I STAlPR MOD1 8 STA3PR
-
Fig. 4: Software-Configuration and -Reconfiguration
5RES
H.U. Steusloff
STA3 also fails. the condition "STAlPR AND STA3PR1'in the R-table of STA2 becomes true. The Local Observer of STA2 now has to initialize the downline loading of POD1 from the load module librarv into STA2 by the Global Loader. Let us assume, that the working storage of STA2 is occupied by MOD2 and MODK. Now the load priority plays its role; since LDPRIO of MOD1 is higher (indicated by the smaller number) than that of MOD2, the latter MODULE is terminated and MOD1 loaded into the storage area previously possessed bu PIOD2. After the repair of STA3 and STAl the reverse sequence of actions takes place. This example shows the i~.pactof distributed computer systems on application proaramming as well as on the supportinrr software. There are further supporting software units, influenced by the distribution of computers, like the program production system and the operating system, which cannot be treated here but are of fundamental importance in tackling with the problems of distributed systems (Heqer, 1979). On the other hand, they are the tools for makins the applications of distributed systems possible and economically feasible. FINAL REMARKS Today there is some discussion,whether hisher level languages (HLL) are suited for microcomputers, beins the outstandins devices in distributed control svstems. The answer is, that HLL are suited if certain conditions are observed. Efficiency problems can be met e.g. by preparing and using prosrammer's lanauaqe hand books containina advices on efficiency properties of special language constructions. By this, efficiency factors - compared with assembled programs - of down to 1.2 have been reached with PEAFL. Runtime supporting software (operating systems, including communication systems, runtime procedures) for a PEARL system will require 6K of 16bit words or less, if standard-I/O is not included but centralized to some dedicated computer stations in a distributed system. It will be very necessary to improve the modularity of operating systems and to facilitate the generating of load-modules dedicated to the local requirements. Finally the compilation of HLL like PEARL will require a working storage of about 32K of 16-bit words and e.g. a Winchester-type 8" disk storage, which is possible with the new 16bit microprocessors.
The impact of distributed computer control systems on software has created a push-forward onto the development of software techniques, allowing to cope with the increasing complexity of automation systems and at the same time maintaining the software expenditures. This paper should have shown that the above mentioned impact will be the occasion of improving automatic control technology in an evolutionary manner. REFERENCES DIN 66253 (1979). Programmiersprache PEARL, Basic PEARL. Deutsches Institut fur Normunq (DIN), DIN 66253, Teil 1. Heger, D., H. Steusloff, and M. Syrbe (1979). Echtzeitrechnersystem mit verteilten Mikroprozessoren. Bundesministerium fur Forschung und Technologie, Forschungsbericht DV 79-01 Datenverarbeitunq. (Realtime Computer System with DistriButed Microprocessors. Federal Ministry of Research and Technology (FRG) Report DV 79-01 Data Processing (in German). 364 pp. Hruschka, P. et a1 (1977). Basic PEARL Language Description. Kernforschungszentrum Karlsruhe (FRG). Report KFK-PDV 120, 242 pp. Kalman, R.E., P.L. Falb, and M.A.Arbib (1969). Topics in Mathematical System Theory. McGraw-Hill, New York. Kappatsch, A., et a1 (1977) Full PEARL Lanauase Description. Kernforschungszentrum Karlsruhe (FRG) Report KFK PDV 130, 462 pp. Kniffler, U. and Denkes, R. Einsatz von EDV-Anlagen zur Planung und Abwicklung von leittechnischen Ausrustungen fur verfahrens- und -kraftwerkstechnische Projekte. berlag VDE, Bezirksverein Frankfurt ./ M . . . 1976. D R . 21 - 32. Steusloff, H.U. (1 977) . Zur Programmierung von raumlich verteilten, dezentralen ProzeRrechnersystemen. Dissertation-Fakultatfur Informatik der Universitat (TH) Karlsruhe (FRG) 180 DD. Steusloff, H.U. (1986) Programming Distributed Computer Systems with Higher Level Languages. Proceeding of the IFAC-Workshop on Distributed Computer Control Systems; Tampa, Florida, USA, Oct.2-4, 1979. Peraamon Press, 1980. (to appear) . Syrbe, M. (1978). Basic Principles of Advanced Process Control System Structures and a Realization with Optical Fibres-Coupled Distributed Ilicrocom~uters.Preprints of the 7th Triennial world Congress, IFAC Helsinki. Pergamon Press, New York, pp. 393 401.
.
.
.
.
.