- Email: [email protected]
Methodical and Management Experiences from an Extensive Software Project
Copyright © IFAC Experience with the Management of Software Projects, Sarajevo, Yugoslavia, 1988
EXPERIENCES I
METHODICAL AND MANAGEMENT EXPERIENCES FROM AN EXTENSIVE SOFTWARE PROJECT W. H. Koch Fachhochschule Ravensburg-Weingarten, D-7987 Weingarten, FRG
Abstract A project of software redevelopment related to durable investment goods is discussed . The major pre-design, design and implementation decisions are reflected . Important topics are the use of rapid prototyping as a design aid and the use of symbolic debuggers in a multitasking environment. Finally the most important methods used are reviewed and suggestions for future developments are made . Keywords Computer applications; computer programming; computer software; programming languages; rapid prototyping; software debugging; software engineering . means of analytical geometry and curve-fitting the desired measure is derived from the single points taken before. This measure may e . g . be the diameter of a cylinder, the distance of two planes, position and direction of an edge). The software connected to CMDs mainly has the following tasks:
1. Im'RODUCTION How to attack large software projects and how
these projects (hopefully) will proceed was the subject of many recent publications . Main keywords in these fields are
- modularization - top-dawn-approach - stepwise refinement
Reading of the POint coordinates (measured values) from the measuring device into the computer (including scaling and correcting of the CMD's reproducible errors). Deriving of geometrical values (measuring results) from probed points. User guidance in dialogue Direct and feedback control of the coordinate measurement device during automatic (CNC-) operation
Here a "real life project" is to be discussed. In terms of subject and field of application this is not a very prestigious project. Just in contrary: it reflects the average. &.it exactly this is why I think the experiences of this project are so important . Seen on the whole the majority of the projects qualify as average; rut just in that majority lies the enormous economic importance of these projects .
coordinate measuring device (portal type)
1 1
The Task
The project to be reviewed here is from the field of durable investment goods . The problem - as it was defined initially - was to transfer a program package from interpreted code on a desktop computer to compiled code on a mini (process) computer .
Soon it became apparent that simple transscription of the code, e .g . replacing each statement of the interpretative language by an equivalent statement of the compiled language would hardly do the job . Above all it would not be a future oriented solution. Instead we went to a completely new construction of the programs, adapted to the changed situation. 1 2
Field of Application
Figure 1 - 1: Coordinate measuring device with control unit and computer
31
32
W. H. Koch Display of the results on display (LED line / CRI' screen) or hardcopy (printer / plotter) devices .
These tasks were already known when the project started. When the project was in progress two more major tasks were added :
New hardware devices (significant advances every 2-3 years) had to be integrated with their full features as fast as possible. Operation with limited features would not (or only on short term) be accepted . 2 2
Connecting of the computer (jnitially working with a single connection to the Cl1D) to different computers in the factory lOO6tly for file exchanging purposes . Integration of the measuring device into flexible manufacturing systems.
2. SYSTEMS ANALYSIS AND CONCLUSIONS FOR THE NEW DEVELOPMENT 2 1
SysyYre
Analvsis
When the project started one found the following situation:
The software which had to be transferred had a volume of about 300 kB . It was implemented in HPL on the desktop computer HP 9825 A. The discrepancy of 300 kB programcode to 23 kB of main storage was COIQP9ll8ated by an extremly optimized and somewhat complicated overlay technique. The control of the measuring device was done in the mode of synchronous programming [LAUB76] and was implemented in the general part of the measuring software. Despite great efforts in time optimization, the Cl1D sometimes halted during automatic runs because the computer was still occupied with the longer lasting calculation of a geometrical measuring result or an output (e.g . a plot) and thus could not initialize the Cl1D ' s move to the next position . For different program versions the complete code always had to be maintained, as e . g . the partly device dependent sequences of lo-statements had not been centralized. and dialog and protocol texts had not been implemented as variables.
The software had been developed in an evolutionary manner. Time and storage optimization had been main criteria in program development . That naturally forced drawbacks on the program structure and those properties needed urgently in future developments as e .g . maintainability . Main documentation was - as not unusual in those days the actual program code by itself . For the different application areas the software solutions had been developed independently. So problems of the same type were solved differently in different places of the program . The development work was done on independent workstations. The means of COIJIDUllication between the different prograJJJDers were magnetic media (floppy discs and tape cassettes) . ce"" eoordtn... )(
•
20
mm
d ........
•
Figure 1 - 2: Two - point measurement and coordlna1e measurement
20 mm
Consegyences for
the
New Deyelogrent
fue to the technical limitations and the special features of the desktop software on the one hand and the capabilities of the process computer on the other, different demands and expectations arose for the development of the new software : The programs should run faster. This could be ensured by used of compiled code and basically faster operating computer hardware. Additionally it had to be ensured that the Cl1D (if the underlying causalities would allow it) would move permanently i f operated in CNC mode to get the maximum measurement speed the device' s mechanics allowed . Making the transition to the process computer (HP 1000, original operating system RTE lVB) the size of one program could be enlarged to 64 kB. Besides that it became possible to keep more than one of these programs in working storage at a time. A general reorganization was to ensure economical maintenance and extension on a long term basis. Especially the countless possible program variations should be produced and maintained easier when the program was constructed in an appropriate way. Distributed development should be made easier by a good modularization. Simple, understandable and exactly documented program and data interfaces were planned to allow even external developers (software houses, customers) to customize the measuring software without any problems. As during the time we expected a very large number of collaborators in our project we favoured unified understandable and transferable program schemes inatead of overoptimized specialized program structures program structures . 3 . THE CHOICE OF THE PROORAMMlOO LAOOUAGE
For speed reasons only a compiled language could be selected. Further postulations on the language were the following: - cooperating with the operating system it had to allow the implementation of the demands on the software. This required that the language had to be supported by a well defined operating system interface . - there had to exist reliable and approved implementations of the software. - knowledge of the language had to be wide spread (intended software maintenance in foreign offices, temporary recruiting of external personal resources) - transition for those prograJJJDers who have only HPL experience should be made easy . - the language should offer a reasonable storage and runtime efficiency . Judged on the basis of general technical criteria at that time, FORTRAN IV, PASCAL , (compilable) BASIC and PEARL were acceptable candidates for our use . From a purely technical point of view PEARL would have certainly been the ideal programming language for the task at hand . At the time in question, however, this language lacked the necessary broad acceptance . Also there was no PEARL-compiler avai-
Software Project Methodical and Management Experiences lable on the' 'candidate computers . rue to personal constrainte a al ternatively compilable or interpretable BASIC had been favored then. BecaU8e for this COllBtellation no compiler was available in time FOR'mAN became the final choice. This primarily meant FOR'mAN IV and later FOR'mAN 66 and FOR'mAN 77 respectively. In the be€inning of this project PEARL was U8ed as a design language to fonrulate problems when constructing the ml ti tasking program System (see sections 4 and 5).
4. BASELINES OF SYSTEl1 DESIGN The software was designed as a ml tiU8er and mlti tasking system. Cbe computer thus should be able to serve several coordinate measuring devices . The software for operating Qne shwle CMD was designed as mltitasking software . This allowed a (quasi-) parallel execution of all functionally parallel software activities. This mltitask construct formed the basis for a (besides minor exceptions) ever-moving operation of the CMD. This increased the throughput remarkably especially in demanding and time consuming measuring applications. For the whole prograDIDing concept this meant the change from synchronous to asynchronous prograDIDing [LAUB 76]. For the software this gave the following structure :
5 . RAPID PRaroI'YPIOO AS A MEANS OF FOUNDATION JUSTIFICATION OF MAJOR DESIGN DECISIONS
AND 5 1
All devices are virtualized . That means input or output statements referring to a certain physical device will only appear in one single device specific task . These device specific tasks are developed and optimized for one certain device each . When a new device is introduced (e. g . a new printer) only one task and maybe some clearly defined files have to be changed. The rest of the software remains untouched . As a tool for fonrulating the design, diagrams in hierarchical input/output presentation were used in part .
Later, for fonrulating the exact design for the tasking problems (scheduling, intertask COIIIWllication) , the prograDIDing language PEARL was a valuable help . Proceeding further it was seen that the transition from a design description in PEARL was not quite straightforward at some places of structural importance . Mainly from this the necessity arose to add a phase of prototype implementation into the design work (see section 5).
Reaaan.s for rapid prototvping
There were three reasons to implement a prototype of th.~ ~!anned measurement software as a means of further research : 1. Thoughts, backgrounds and possibilities of the current design had to be brought close to nonspecialist decisionmakers, progTaDJDers without a deep training in software engineering and systems engineers (software consultants) of the computer manufacturer in a most descriptive way. The alternatively available graphical methods (SArfI', HIPO, Jackson etc.) proved to be too weak for this purpose . There is more willingness to spent some time on looking at a presentation of a runnable program and then to discuss, than to di6CU6s on some graphics drawn on a sheet of paper . Maybe in the latter case there is also an overdemand of imagination. 2 . The extremely difficult jobs of task synchroni-
sation and task COIIIWllication should be examined and designed perfectly in a realistic environment rut without overhead. examples : - Parallel input via a standard te=inal (RS 232 C - interface) and the CMD had to be provided. The program had to react on the device operated first. Potential operation of the other device shortly thereafter has to be ignored. - For implementation of the intertask COIlIIlUIlication a solution had to be found, which at least allowed the logical separation of the multi tasking program systems dedicated to single CMDs that are hooked up on one computer of the ml ti tasking systems dedicated to the single CMDs hooked up on one computer . This separation was expected to need no special effort by the application prograDJDer .
- control program for the coordinate measurement device - dialog program - filehandling program - output program (one per device, easily allowing parallel output) - calculation programs This list gives the miniJWm number of programs which is detennined by the number of activities that could be executed in parallel. Because of the different sizes of the programs and the decentralized development a more detailed subdivision was chosen. This meant that flmctions that by themselves ran synchronuously had to be separated into different tasks due to their volume or staffing problems .
33
3 . The prototype was planned to be used as convincing and infornJative material for the development team working on the project futurely. This was necessary to motivate the prograDJDers that had been highly experienced and extremly successful on the desktop to migrate to the new technology . The prototype on the one hand was planned for research of program construction on the other side the documentation of the prototype was intended to be documentation protoptype for the development follwing . 5 2
Volume and Structure of the Program Prototype
The program prototype consisted of - task for program control - task for simple geometric calculations - task for CMD control - task for (asynchronuous) data output The research on the prototype lead to the basic structure for the tasks as shown in figure 5-1 . First all tasks are sequentially transferred into the "active" state and then (except for one task the so called "start task") and then into a waiting state ("wait for roailoox info rnJation . sent from another task") . If one task now receives information sent by another task, then it is transferred from the "waiting" state into the "active" or into the "running" state. At the end of the task (renewed) roailbox information is sent to another task . The data passed per mailbox between the single tasks
34
W. H . Koch program activation and program termination conlrolled by operatirr;l system
results of developnent so a drifting away from each other by UBing different stages of software was prevented .
preparation phase 10 nt'l'Illlem'lllllllofl
mlllbQl ruel .taltmm
g.t InfoUIlUOII froll! &noliif' progrul pnlll"Uon. •• I.~tlon of function
runello. ~
function
, 1
~
funcllon
21
~
11
funcllon ~
n1
lu cul, lI'cutl luellle IIlCut, IIn (lIon 1 III/lcllon 2 lunctlon 1 (Uletlo" n
.f.
m,lIbol wrlle ItlllII.
IlIformaUont
program.
To allow not only single module developnent, rut also have system test and integration of several complete 1W1titasking systems from 1W1tiple spots in the 1W1 tiuser environment, these program systems had to be completely isolated. JUBt for this purpose the decision to go on mailbox and not on system COIIIOOll for intertask coummication proved to be extremly positive . The excellent working conditions created by this design decision fully JUBtified the additional cost of runtime and storage space .
pallId 10 oiler
Figure 5- 1: Basic structure of the programs ""lIbol-lnp,fa from dtfftflnl program. (ta.b)
6 2
Development and ])xumentation Gujdelines
For all modules (pograms 1 , subroutines, functions, dataregions (block-data-statements)) a program frame was prescribed . This consisted of a program header, an extended program description and guidelines for coding . In the program header the following information put together : - name of module, parameter list if applicable - short description of the module ' s function - author - changes (date and reason) - classification code (for archiving purposes) Following the header , information on the parameters and possible error states of the module was documented . In addition detailed rules on error tracing and error handling were set up for all modules . This made available to obtain full information on the type and location of the error, and on the full calling path whenever an err occured . This was important for reconstruction of sporadic errors .
Figure 5- 2 : Use of monitors in more complicated cases of program - toprogram communication
are mainly global information for the software operating together with a CMD . These are freely availabe in the 1W1titasking system dedicated to one CMD rut protected from the other systems running on the same hardware . Minor disadvantages in terms of runtime and storage space were accepted to get more reliabil ty and an easier error handling . If a task may get information from more than one scurce at the same time the basic concept of mailbox COIIIIllIlication had to be added either by semaphors or by mailbox (Figure 5 - 2). For the tasks program frames were developed, which served the prograJJJDers to insert their specific code (see 6 . 2) . This allowed concentration of the work on the difficult and far reaching multitasking problems onto just a few specialists. The most important result of the prototyping phase
was the developnent of a unified, reliable and easily maintainable concept for task control and a small scale proof of feasibilty in the same step .
6 . MANAGEliEN!' OF SOFIWARE DESIGN AND DEVELOFt!ENT 6 1
Deyelopment on a Multiuser System
The new development was immediately started on a 1W1tiUBer System . Each user got on-line access to controlled libraries containing the most recent
6 3
Job sharing develogpent
The module concept and especially the clear separation of tasks implementing organisational functions from those implementing applicational functions led to a change in the method of job sharing in the laboratories . Formerly the development was only divided by application categories (each programmer fully developed a measurement application from beginning to the end including dialog, coordinate acquisition , control of the measuring device calculation and output), rut now there were specialists needed for user dialog, the different application dependent calculations , track control of the CMD, graphics and so on . There were no technical arguments to oppose this solution but for a phase of transition two problems had to be solved : - The limitation of solving only parts of the measuring problem made prograuiDers feel a certain degree of discontent, they felt their compentencies cut. The profits of the new way of job sharing and the following release from routine work and the chance of concentrating on the important parts of organizing and prograJlllling a measuring function was only acknowledged slowly in the beginning . - The new way of job sharing increased the demand of coammication between the development people. In regions where one formerly 1 "Program"
in the sense of FORTRAN tenninolOjiy, respectively in the COIIlPlter manufacturer's tenninology . Looking at the implemented program system , "task" would be the better tenn.
Software Project Methodical and Management Experiences could decide on one's own (e.g. how to transfer a result from the calculation to the ootpxt DXlule) now the application of some interfacing conventions was COIIlP-tlsory.
,..t drMr ('~CItc'Of'" tined
J.... _l I J InpuI Oala
Test and Pel.,gging l1ethode
6 4
Initially the Black-Box-Test (picture 6-1) was the only way for testing . That meant that each designed DXlule had. to be examined in its f.nput-outwtbehavioor by UBe of a individually designed test driver. This way of testing was not easily accepted by those progr8l!lDers who had their main experience on interpreting systems. As they were used a working cycle like - correction - interpretation - test (having access to all variables without any special handling) - correction having the program under develoFmeIlt IIlO6tly permanent in working storage they now had to get used to the cycle - correction (editing) - compilation - link /load - test which was more time consuming and by far less comfortable .
:--------------1 I I I I I I I
The increasing number of progr8l!lDers involved in the project on one side and the develoFmeIlts on the other led to the necessity and possibility to distribute the develoFmeIlt activities on several computers:
--
terminal I:
Ul.ra turflCa
I
I
1 I
:
terminal 2:
I
.." Of one tull: by meaN Of • • ttdrtoW
I I
I I I I I I I I
1
t.....
I
I I
l---7------ J mu.... kSng program tyttem uno..... et
6-2) .
Networking and distributed development activities
I I
I
Figure 6 - 2: use of the symbolic debuggers to test a tesk In the mUtItaskIng program system
The debugger was especially helpful and absolutely
6 5
--
Figure 6 - 1: black - box - test of a program by meens of a test drtver
ments. On the whole the early introduction of the symbolic debugger proved to be very successful (figure
Intensive internal diBCU6sions and external inwt [FOGE 86] led to development of a testframe for single tasks. All inwt and outwt was of program version n was recorded . In the test of version n the task is supplied automatically with all inwt data of version n . Afterwards all outwt data of version n are compared with those of version n+l. Differences between the two data sets IWBt be correlated to program modifications from version n to version n+1. [KOCH88]
~
o ~ 11
Later the introduction of a symbolic del:x.lgger . The availability of this tool fully changed the testing strategy and demanded big hardware invest-
unreplaceable when synchronization and communication of several tasks had to be tested in the multitasking environment. It offered a "view" into one or more programs at the same time. Disadvantageous was the favouring of trial-anderror-programming. Also the the system test using the debugger demanded vast hardware resources. This demand could be optiJnized but needed very detailed and highly specialized operating system know-how. Main test before delivery was a complete functional test at the aID. It was disadvantageous that this test originally could be only automated for the CNC mode, whereas for manual operation which proved to be the more change-sensi tive operational mode no possibility for automatic testing existed. The main problem in automatic testing of the manual mode was to reproduce the often very long inwt dialogs . The demand of exact repetition of even extended inwt sequences required almost superhuman concentrational Bkills of the program tester .
35
- central comwter for stable software and marketed program versions - computers for the different application prograllJlling groups - computers operating on the test sites together with CMDs Different external locations were connected to the central comwter by a WAN (Datex-P / X.25). They consisted of external development locations (even in the USA) or pilot customers . In the first case the WAN accelerated the cooperation in development remarkably, in the other it allowed qualified software service while avoiding travel expenses . The network software in use allowed file transfer and even terminal emulation on a remote host. In terms of cost (FR of Germany, 1986) the WAN was an extremly good choice for dialog operations or transfer of minor corrections . For transfer of complete software revisions (several MegaBytes) the use of the traditional mail services was more acceptable, unless there was extrem time pressure (picture 6-3) . 6 6
Software Prodyction
In the central computer the software is generated due to customers' orders . Relevant parameters in this process are : - operating system (selection from pregenerated standard systems related to the order's volume) . In the operating system variants the variants, of the CPUs and the peripherals are respected . - software for measuring applications (as ordered)
- universal software - curve measuring software - gear measuring software
etc.
36
W. H. Koch - software to control the CMD (specific for each OO>-type) - (national) language used on the customer's site
should be considered . Today even the availability of software tools could to a high level influence the selection of the computer hardware. 7 2
What RhOllld be attacked in the
MD"
way
?
Program structure
It is absolutely necessary to use large-scale, clear and standardized program structures instead of detailed pre-optimization . Optimization not only coeta development time, but also tends to deterioriate quality parameters such as maintenability and clarity . Therefore optimization ~ld be used only, where time or ,storage limit absolutely demand it . Beal10ble program to program cggglnication instead
of sYStem qgmxms Starting development from the beginning with reliable COIIIWllication mechanisms promoted systematic development. The errors coming. up through the use of this mechanism should be seen in a p:JSitive light. I f COIIIWllication mechanisms with a less controlled data exchange had been used such errors would probably have arisen later and more sporadic so that correction would have been more difficult.
ampute< c:onn~
IoCMD
figure 6 - 3: Software development In a network environment
7 . COOCLUSIONS 7 1
What would be 891ved in
a
different
Way today
Pxwra!ll!llng laruruage Today the trend to Pascal or PEARL (or partly even C) would be stronger. as the general distribution • the system softwal~ market (compilers). the acceptance and the situation on the personal market (there are more engineers knowing these languages) have developed in favour of these languages . The use of these languages would move forward the detection of many errors from the test phase to the coding phase and by that accelerate development. Whether these technical arguments would suffice for those languages to compete successfully with FORTRAN JIl.lSt be questioned . FORTRAN's strengths continue to lie in the situation in the personal market. international acceptance. and last but not least ongoing adapt ion of the language to software technological trenda.
System organioatign The development of multiuser systems would be discontinued due to the changed capabilities and pricing of computer hardware and networking equipment. Instead one would connect several multitaslting ptqt'am systems - each running on a separate computer - by a network . In an analoguos way in development, one would form BJDaller "development islands" in the laboratory which would be connected by a network . As the prograDlll size / memory size hardly is a problem any longer (at least in the dimensions we need here), the number of tasks will be limited to that one what is necessary to fulfil the demand for actions running in parallel .
Use of a symbolic dehlgger Even if perhaps questionable from the theoretical point of view, the symbolic debugger has become a JIl.lSt. EspeCially appreciated is ita usefulness when testing the whole integrated lI1Il ti tssking program . By use of several tenninals one can at the same time have a look at the ORIGINAL user's surface of the program product on the one hand, and look into one or several programs on the other. The application of the symbolic debugger JIl.lSt be combined with restructuring phases when the ad-hoc corrections made in the debugging session are restructured to reduce the program's entropy . Organization Job sharing as described above IWBt not be changed . It avoida lI1Iltiple developments and produces solutions of more professional competence . By appropriate staffing one has to ensure multiple availability of key know-how . A WAN should be used support distributed development activities and pilot customers. Rapid prototYPing The DX>St effective tool for resolving difficult and vital structural problems is the prototype. Defining and building of nxxiels are well introduced ways of experimenting and COIIIWllicating in many other technologies. This methodology IWBt also find ita place in software technology . Especially as a means of COIIIWllication the prototype is of extreme value. Program graphs - as technical drawings in mechanical or civil engineering - are intended to be COIIIWllication media as well . fut their standards ~ not well established and the ability to produce or read !Such graphs unambijUously is not wide spread.
[LAUB76] [KUHN88]
Program structure
Software tools Well adapted use of software tools to free the ptqt'ammer from desktop routine work
[FUGE86]
Lauber, R. : Proze/3automatisierung I, Springer Verlag, Berlin, Heidelberg, New York, 1976 . KUhn, E.: Stand, Erfahrungen und Trends fUr die Anwendung rechnergestutzter Werkzeuge bei der Entwicklung von Proze!3rechnersystemen, Fachtagung Prozel3rechnersysteme , Stut~art 1988. Fuget, C. D.; Scott , B. J .: Tools for Automating Software Test Package Execution; Hewlett-Packard-Journal; Vol. 37, March 1986, S. 24 ff.
EXPERIENCES I
METHODICAL AND MANAGEMENT EXPERIENCES FROM AN EXTENSIVE SOFTWARE PROJECT W. H. Koch Fachhochschule Ravensburg-Weingarten, D-7987 Weingarten, FRG
Abstract A project of software redevelopment related to durable investment goods is discussed . The major pre-design, design and implementation decisions are reflected . Important topics are the use of rapid prototyping as a design aid and the use of symbolic debuggers in a multitasking environment. Finally the most important methods used are reviewed and suggestions for future developments are made . Keywords Computer applications; computer programming; computer software; programming languages; rapid prototyping; software debugging; software engineering . means of analytical geometry and curve-fitting the desired measure is derived from the single points taken before. This measure may e . g . be the diameter of a cylinder, the distance of two planes, position and direction of an edge). The software connected to CMDs mainly has the following tasks:
1. Im'RODUCTION How to attack large software projects and how
these projects (hopefully) will proceed was the subject of many recent publications . Main keywords in these fields are
- modularization - top-dawn-approach - stepwise refinement
Reading of the POint coordinates (measured values) from the measuring device into the computer (including scaling and correcting of the CMD's reproducible errors). Deriving of geometrical values (measuring results) from probed points. User guidance in dialogue Direct and feedback control of the coordinate measurement device during automatic (CNC-) operation
Here a "real life project" is to be discussed. In terms of subject and field of application this is not a very prestigious project. Just in contrary: it reflects the average. &.it exactly this is why I think the experiences of this project are so important . Seen on the whole the majority of the projects qualify as average; rut just in that majority lies the enormous economic importance of these projects .
coordinate measuring device (portal type)
1 1
The Task
The project to be reviewed here is from the field of durable investment goods . The problem - as it was defined initially - was to transfer a program package from interpreted code on a desktop computer to compiled code on a mini (process) computer .
Soon it became apparent that simple transscription of the code, e .g . replacing each statement of the interpretative language by an equivalent statement of the compiled language would hardly do the job . Above all it would not be a future oriented solution. Instead we went to a completely new construction of the programs, adapted to the changed situation. 1 2
Field of Application
Figure 1 - 1: Coordinate measuring device with control unit and computer
31
32
W. H. Koch Display of the results on display (LED line / CRI' screen) or hardcopy (printer / plotter) devices .
These tasks were already known when the project started. When the project was in progress two more major tasks were added :
New hardware devices (significant advances every 2-3 years) had to be integrated with their full features as fast as possible. Operation with limited features would not (or only on short term) be accepted . 2 2
Connecting of the computer (jnitially working with a single connection to the Cl1D) to different computers in the factory lOO6tly for file exchanging purposes . Integration of the measuring device into flexible manufacturing systems.
2. SYSTEMS ANALYSIS AND CONCLUSIONS FOR THE NEW DEVELOPMENT 2 1
SysyYre
Analvsis
When the project started one found the following situation:
The software which had to be transferred had a volume of about 300 kB . It was implemented in HPL on the desktop computer HP 9825 A. The discrepancy of 300 kB programcode to 23 kB of main storage was COIQP9ll8ated by an extremly optimized and somewhat complicated overlay technique. The control of the measuring device was done in the mode of synchronous programming [LAUB76] and was implemented in the general part of the measuring software. Despite great efforts in time optimization, the Cl1D sometimes halted during automatic runs because the computer was still occupied with the longer lasting calculation of a geometrical measuring result or an output (e.g . a plot) and thus could not initialize the Cl1D ' s move to the next position . For different program versions the complete code always had to be maintained, as e . g . the partly device dependent sequences of lo-statements had not been centralized. and dialog and protocol texts had not been implemented as variables.
The software had been developed in an evolutionary manner. Time and storage optimization had been main criteria in program development . That naturally forced drawbacks on the program structure and those properties needed urgently in future developments as e .g . maintainability . Main documentation was - as not unusual in those days the actual program code by itself . For the different application areas the software solutions had been developed independently. So problems of the same type were solved differently in different places of the program . The development work was done on independent workstations. The means of COIJIDUllication between the different prograJJJDers were magnetic media (floppy discs and tape cassettes) . ce"" eoordtn... )(
•
20
mm
d ........
•
Figure 1 - 2: Two - point measurement and coordlna1e measurement
20 mm
Consegyences for
the
New Deyelogrent
fue to the technical limitations and the special features of the desktop software on the one hand and the capabilities of the process computer on the other, different demands and expectations arose for the development of the new software : The programs should run faster. This could be ensured by used of compiled code and basically faster operating computer hardware. Additionally it had to be ensured that the Cl1D (if the underlying causalities would allow it) would move permanently i f operated in CNC mode to get the maximum measurement speed the device' s mechanics allowed . Making the transition to the process computer (HP 1000, original operating system RTE lVB) the size of one program could be enlarged to 64 kB. Besides that it became possible to keep more than one of these programs in working storage at a time. A general reorganization was to ensure economical maintenance and extension on a long term basis. Especially the countless possible program variations should be produced and maintained easier when the program was constructed in an appropriate way. Distributed development should be made easier by a good modularization. Simple, understandable and exactly documented program and data interfaces were planned to allow even external developers (software houses, customers) to customize the measuring software without any problems. As during the time we expected a very large number of collaborators in our project we favoured unified understandable and transferable program schemes inatead of overoptimized specialized program structures program structures . 3 . THE CHOICE OF THE PROORAMMlOO LAOOUAGE
For speed reasons only a compiled language could be selected. Further postulations on the language were the following: - cooperating with the operating system it had to allow the implementation of the demands on the software. This required that the language had to be supported by a well defined operating system interface . - there had to exist reliable and approved implementations of the software. - knowledge of the language had to be wide spread (intended software maintenance in foreign offices, temporary recruiting of external personal resources) - transition for those prograJJJDers who have only HPL experience should be made easy . - the language should offer a reasonable storage and runtime efficiency . Judged on the basis of general technical criteria at that time, FORTRAN IV, PASCAL , (compilable) BASIC and PEARL were acceptable candidates for our use . From a purely technical point of view PEARL would have certainly been the ideal programming language for the task at hand . At the time in question, however, this language lacked the necessary broad acceptance . Also there was no PEARL-compiler avai-
Software Project Methodical and Management Experiences lable on the' 'candidate computers . rue to personal constrainte a al ternatively compilable or interpretable BASIC had been favored then. BecaU8e for this COllBtellation no compiler was available in time FOR'mAN became the final choice. This primarily meant FOR'mAN IV and later FOR'mAN 66 and FOR'mAN 77 respectively. In the be€inning of this project PEARL was U8ed as a design language to fonrulate problems when constructing the ml ti tasking program System (see sections 4 and 5).
4. BASELINES OF SYSTEl1 DESIGN The software was designed as a ml tiU8er and mlti tasking system. Cbe computer thus should be able to serve several coordinate measuring devices . The software for operating Qne shwle CMD was designed as mltitasking software . This allowed a (quasi-) parallel execution of all functionally parallel software activities. This mltitask construct formed the basis for a (besides minor exceptions) ever-moving operation of the CMD. This increased the throughput remarkably especially in demanding and time consuming measuring applications. For the whole prograDIDing concept this meant the change from synchronous to asynchronous prograDIDing [LAUB 76]. For the software this gave the following structure :
5 . RAPID PRaroI'YPIOO AS A MEANS OF FOUNDATION JUSTIFICATION OF MAJOR DESIGN DECISIONS
AND 5 1
All devices are virtualized . That means input or output statements referring to a certain physical device will only appear in one single device specific task . These device specific tasks are developed and optimized for one certain device each . When a new device is introduced (e. g . a new printer) only one task and maybe some clearly defined files have to be changed. The rest of the software remains untouched . As a tool for fonrulating the design, diagrams in hierarchical input/output presentation were used in part .
Later, for fonrulating the exact design for the tasking problems (scheduling, intertask COIIIWllication) , the prograDIDing language PEARL was a valuable help . Proceeding further it was seen that the transition from a design description in PEARL was not quite straightforward at some places of structural importance . Mainly from this the necessity arose to add a phase of prototype implementation into the design work (see section 5).
Reaaan.s for rapid prototvping
There were three reasons to implement a prototype of th.~ ~!anned measurement software as a means of further research : 1. Thoughts, backgrounds and possibilities of the current design had to be brought close to nonspecialist decisionmakers, progTaDJDers without a deep training in software engineering and systems engineers (software consultants) of the computer manufacturer in a most descriptive way. The alternatively available graphical methods (SArfI', HIPO, Jackson etc.) proved to be too weak for this purpose . There is more willingness to spent some time on looking at a presentation of a runnable program and then to discuss, than to di6CU6s on some graphics drawn on a sheet of paper . Maybe in the latter case there is also an overdemand of imagination. 2 . The extremely difficult jobs of task synchroni-
sation and task COIIIWllication should be examined and designed perfectly in a realistic environment rut without overhead. examples : - Parallel input via a standard te=inal (RS 232 C - interface) and the CMD had to be provided. The program had to react on the device operated first. Potential operation of the other device shortly thereafter has to be ignored. - For implementation of the intertask COIlIIlUIlication a solution had to be found, which at least allowed the logical separation of the multi tasking program systems dedicated to single CMDs that are hooked up on one computer of the ml ti tasking systems dedicated to the single CMDs hooked up on one computer . This separation was expected to need no special effort by the application prograDJDer .
- control program for the coordinate measurement device - dialog program - filehandling program - output program (one per device, easily allowing parallel output) - calculation programs This list gives the miniJWm number of programs which is detennined by the number of activities that could be executed in parallel. Because of the different sizes of the programs and the decentralized development a more detailed subdivision was chosen. This meant that flmctions that by themselves ran synchronuously had to be separated into different tasks due to their volume or staffing problems .
33
3 . The prototype was planned to be used as convincing and infornJative material for the development team working on the project futurely. This was necessary to motivate the prograDJDers that had been highly experienced and extremly successful on the desktop to migrate to the new technology . The prototype on the one hand was planned for research of program construction on the other side the documentation of the prototype was intended to be documentation protoptype for the development follwing . 5 2
Volume and Structure of the Program Prototype
The program prototype consisted of - task for program control - task for simple geometric calculations - task for CMD control - task for (asynchronuous) data output The research on the prototype lead to the basic structure for the tasks as shown in figure 5-1 . First all tasks are sequentially transferred into the "active" state and then (except for one task the so called "start task") and then into a waiting state ("wait for roailoox info rnJation . sent from another task") . If one task now receives information sent by another task, then it is transferred from the "waiting" state into the "active" or into the "running" state. At the end of the task (renewed) roailbox information is sent to another task . The data passed per mailbox between the single tasks
34
W. H . Koch program activation and program termination conlrolled by operatirr;l system
results of developnent so a drifting away from each other by UBing different stages of software was prevented .
preparation phase 10 nt'l'Illlem'lllllllofl
mlllbQl ruel .taltmm
g.t InfoUIlUOII froll! &noliif' progrul pnlll"Uon. •• I.~tlon of function
runello. ~
function
, 1
~
funcllon
21
~
11
funcllon ~
n1
lu cul, lI'cutl luellle IIlCut, IIn (lIon 1 III/lcllon 2 lunctlon 1 (Uletlo" n
.f.
m,lIbol wrlle ItlllII.
IlIformaUont
program.
To allow not only single module developnent, rut also have system test and integration of several complete 1W1titasking systems from 1W1tiple spots in the 1W1 tiuser environment, these program systems had to be completely isolated. JUBt for this purpose the decision to go on mailbox and not on system COIIIOOll for intertask coummication proved to be extremly positive . The excellent working conditions created by this design decision fully JUBtified the additional cost of runtime and storage space .
pallId 10 oiler
Figure 5- 1: Basic structure of the programs ""lIbol-lnp,fa from dtfftflnl program. (ta.b)
6 2
Development and ])xumentation Gujdelines
For all modules (pograms 1 , subroutines, functions, dataregions (block-data-statements)) a program frame was prescribed . This consisted of a program header, an extended program description and guidelines for coding . In the program header the following information put together : - name of module, parameter list if applicable - short description of the module ' s function - author - changes (date and reason) - classification code (for archiving purposes) Following the header , information on the parameters and possible error states of the module was documented . In addition detailed rules on error tracing and error handling were set up for all modules . This made available to obtain full information on the type and location of the error, and on the full calling path whenever an err occured . This was important for reconstruction of sporadic errors .
Figure 5- 2 : Use of monitors in more complicated cases of program - toprogram communication
are mainly global information for the software operating together with a CMD . These are freely availabe in the 1W1titasking system dedicated to one CMD rut protected from the other systems running on the same hardware . Minor disadvantages in terms of runtime and storage space were accepted to get more reliabil ty and an easier error handling . If a task may get information from more than one scurce at the same time the basic concept of mailbox COIIIIllIlication had to be added either by semaphors or by mailbox (Figure 5 - 2). For the tasks program frames were developed, which served the prograJJJDers to insert their specific code (see 6 . 2) . This allowed concentration of the work on the difficult and far reaching multitasking problems onto just a few specialists. The most important result of the prototyping phase
was the developnent of a unified, reliable and easily maintainable concept for task control and a small scale proof of feasibilty in the same step .
6 . MANAGEliEN!' OF SOFIWARE DESIGN AND DEVELOFt!ENT 6 1
Deyelopment on a Multiuser System
The new development was immediately started on a 1W1tiUBer System . Each user got on-line access to controlled libraries containing the most recent
6 3
Job sharing develogpent
The module concept and especially the clear separation of tasks implementing organisational functions from those implementing applicational functions led to a change in the method of job sharing in the laboratories . Formerly the development was only divided by application categories (each programmer fully developed a measurement application from beginning to the end including dialog, coordinate acquisition , control of the measuring device calculation and output), rut now there were specialists needed for user dialog, the different application dependent calculations , track control of the CMD, graphics and so on . There were no technical arguments to oppose this solution but for a phase of transition two problems had to be solved : - The limitation of solving only parts of the measuring problem made prograuiDers feel a certain degree of discontent, they felt their compentencies cut. The profits of the new way of job sharing and the following release from routine work and the chance of concentrating on the important parts of organizing and prograJlllling a measuring function was only acknowledged slowly in the beginning . - The new way of job sharing increased the demand of coammication between the development people. In regions where one formerly 1 "Program"
in the sense of FORTRAN tenninolOjiy, respectively in the COIIlPlter manufacturer's tenninology . Looking at the implemented program system , "task" would be the better tenn.
Software Project Methodical and Management Experiences could decide on one's own (e.g. how to transfer a result from the calculation to the ootpxt DXlule) now the application of some interfacing conventions was COIIlP-tlsory.
,..t drMr ('~CItc'Of'" tined
J.... _l I J InpuI Oala
Test and Pel.,gging l1ethode
6 4
Initially the Black-Box-Test (picture 6-1) was the only way for testing . That meant that each designed DXlule had. to be examined in its f.nput-outwtbehavioor by UBe of a individually designed test driver. This way of testing was not easily accepted by those progr8l!lDers who had their main experience on interpreting systems. As they were used a working cycle like - correction - interpretation - test (having access to all variables without any special handling) - correction having the program under develoFmeIlt IIlO6tly permanent in working storage they now had to get used to the cycle - correction (editing) - compilation - link /load - test which was more time consuming and by far less comfortable .
:--------------1 I I I I I I I
The increasing number of progr8l!lDers involved in the project on one side and the develoFmeIlts on the other led to the necessity and possibility to distribute the develoFmeIlt activities on several computers:
--
terminal I:
Ul.ra turflCa
I
I
1 I
:
terminal 2:
I
.." Of one tull: by meaN Of • • ttdrtoW
I I
I I I I I I I I
1
t.....
I
I I
l---7------ J mu.... kSng program tyttem uno..... et
6-2) .
Networking and distributed development activities
I I
I
Figure 6 - 2: use of the symbolic debuggers to test a tesk In the mUtItaskIng program system
The debugger was especially helpful and absolutely
6 5
--
Figure 6 - 1: black - box - test of a program by meens of a test drtver
ments. On the whole the early introduction of the symbolic debugger proved to be very successful (figure
Intensive internal diBCU6sions and external inwt [FOGE 86] led to development of a testframe for single tasks. All inwt and outwt was of program version n was recorded . In the test of version n the task is supplied automatically with all inwt data of version n . Afterwards all outwt data of version n are compared with those of version n+l. Differences between the two data sets IWBt be correlated to program modifications from version n to version n+1. [KOCH88]
~
o ~ 11
Later the introduction of a symbolic del:x.lgger . The availability of this tool fully changed the testing strategy and demanded big hardware invest-
unreplaceable when synchronization and communication of several tasks had to be tested in the multitasking environment. It offered a "view" into one or more programs at the same time. Disadvantageous was the favouring of trial-anderror-programming. Also the the system test using the debugger demanded vast hardware resources. This demand could be optiJnized but needed very detailed and highly specialized operating system know-how. Main test before delivery was a complete functional test at the aID. It was disadvantageous that this test originally could be only automated for the CNC mode, whereas for manual operation which proved to be the more change-sensi tive operational mode no possibility for automatic testing existed. The main problem in automatic testing of the manual mode was to reproduce the often very long inwt dialogs . The demand of exact repetition of even extended inwt sequences required almost superhuman concentrational Bkills of the program tester .
35
- central comwter for stable software and marketed program versions - computers for the different application prograllJlling groups - computers operating on the test sites together with CMDs Different external locations were connected to the central comwter by a WAN (Datex-P / X.25). They consisted of external development locations (even in the USA) or pilot customers . In the first case the WAN accelerated the cooperation in development remarkably, in the other it allowed qualified software service while avoiding travel expenses . The network software in use allowed file transfer and even terminal emulation on a remote host. In terms of cost (FR of Germany, 1986) the WAN was an extremly good choice for dialog operations or transfer of minor corrections . For transfer of complete software revisions (several MegaBytes) the use of the traditional mail services was more acceptable, unless there was extrem time pressure (picture 6-3) . 6 6
Software Prodyction
In the central computer the software is generated due to customers' orders . Relevant parameters in this process are : - operating system (selection from pregenerated standard systems related to the order's volume) . In the operating system variants the variants, of the CPUs and the peripherals are respected . - software for measuring applications (as ordered)
- universal software - curve measuring software - gear measuring software
etc.
36
W. H. Koch - software to control the CMD (specific for each OO>-type) - (national) language used on the customer's site
should be considered . Today even the availability of software tools could to a high level influence the selection of the computer hardware. 7 2
What RhOllld be attacked in the
MD"
way
?
Program structure
It is absolutely necessary to use large-scale, clear and standardized program structures instead of detailed pre-optimization . Optimization not only coeta development time, but also tends to deterioriate quality parameters such as maintenability and clarity . Therefore optimization ~ld be used only, where time or ,storage limit absolutely demand it . Beal10ble program to program cggglnication instead
of sYStem qgmxms Starting development from the beginning with reliable COIIIWllication mechanisms promoted systematic development. The errors coming. up through the use of this mechanism should be seen in a p:JSitive light. I f COIIIWllication mechanisms with a less controlled data exchange had been used such errors would probably have arisen later and more sporadic so that correction would have been more difficult.
ampute< c:onn~
IoCMD
figure 6 - 3: Software development In a network environment
7 . COOCLUSIONS 7 1
What would be 891ved in
a
different
Way today
Pxwra!ll!llng laruruage Today the trend to Pascal or PEARL (or partly even C) would be stronger. as the general distribution • the system softwal~ market (compilers). the acceptance and the situation on the personal market (there are more engineers knowing these languages) have developed in favour of these languages . The use of these languages would move forward the detection of many errors from the test phase to the coding phase and by that accelerate development. Whether these technical arguments would suffice for those languages to compete successfully with FORTRAN JIl.lSt be questioned . FORTRAN's strengths continue to lie in the situation in the personal market. international acceptance. and last but not least ongoing adapt ion of the language to software technological trenda.
System organioatign The development of multiuser systems would be discontinued due to the changed capabilities and pricing of computer hardware and networking equipment. Instead one would connect several multitaslting ptqt'am systems - each running on a separate computer - by a network . In an analoguos way in development, one would form BJDaller "development islands" in the laboratory which would be connected by a network . As the prograDlll size / memory size hardly is a problem any longer (at least in the dimensions we need here), the number of tasks will be limited to that one what is necessary to fulfil the demand for actions running in parallel .
Use of a symbolic dehlgger Even if perhaps questionable from the theoretical point of view, the symbolic debugger has become a JIl.lSt. EspeCially appreciated is ita usefulness when testing the whole integrated lI1Il ti tssking program . By use of several tenninals one can at the same time have a look at the ORIGINAL user's surface of the program product on the one hand, and look into one or several programs on the other. The application of the symbolic debugger JIl.lSt be combined with restructuring phases when the ad-hoc corrections made in the debugging session are restructured to reduce the program's entropy . Organization Job sharing as described above IWBt not be changed . It avoida lI1Iltiple developments and produces solutions of more professional competence . By appropriate staffing one has to ensure multiple availability of key know-how . A WAN should be used support distributed development activities and pilot customers. Rapid prototYPing The DX>St effective tool for resolving difficult and vital structural problems is the prototype. Defining and building of nxxiels are well introduced ways of experimenting and COIIIWllicating in many other technologies. This methodology IWBt also find ita place in software technology . Especially as a means of COIIIWllication the prototype is of extreme value. Program graphs - as technical drawings in mechanical or civil engineering - are intended to be COIIIWllication media as well . fut their standards ~ not well established and the ability to produce or read !Such graphs unambijUously is not wide spread.
[LAUB76] [KUHN88]
Program structure
Software tools Well adapted use of software tools to free the ptqt'ammer from desktop routine work
[FUGE86]
Lauber, R. : Proze/3automatisierung I, Springer Verlag, Berlin, Heidelberg, New York, 1976 . KUhn, E.: Stand, Erfahrungen und Trends fUr die Anwendung rechnergestutzter Werkzeuge bei der Entwicklung von Proze!3rechnersystemen, Fachtagung Prozel3rechnersysteme , Stut~art 1988. Fuget, C. D.; Scott , B. J .: Tools for Automating Software Test Package Execution; Hewlett-Packard-Journal; Vol. 37, March 1986, S. 24 ff.