Modern software development methodologies and their environments

Computer Physics Communications 38 (1985) 119—134 North-Holland, Amsterdam

119

MODERN SOFTWARE DEVELOPMENT METHODOLOGIES AND THEIR ENVIRONMENTS Anthony I. WASSERMAN Medical Information Science, University of California, San Francisco, San Francisco, CA 94143, USA

Progress in software engineering has come in a sequence of overlapping stages. The first stage, circa 1965—1970, was “recognition of the problem”. This stage was followed by “structured programming solutions”, from 1971 to about 1974. A third phase was “life cycle models”, from 1974 until 1976. The next step was “analysis, specification and design methods”, from about 1976 to 1980. The fifth stage was “software development methodology”, beginning in about 1980 and extending to the present. A sixth stage, “software engineering environments”, has recently begun and addresses automated tools and programming environments to aid the software development process. This paper gives an overview of key issues and developments in software development, emphasizing the latter two stages and their role in scientific programming.

I. Historical background The nature of scientific programming has changed drastically over four decades. The earliest days of computing were devoted exclusively to numerical, scientific computation, and were characterized by large, expensive, and unreliable hardware, relatively slow processing speeds, severe limitations on both central memory and “backing store”, and the total lack of programming support tools. All programming was done in “machine language”, using binary or octal numbers. Even the simplest programs by today’s standards were a major undertaking under those conditions. The development of assemblers and symbolic instructions in the early 1950’s represented a significant breakthrough and the introduction of high level languages such as ALGOL and FORTRAN was truly revolutionary! The arrival of such highlevel languages was greeted with debate and skepticism by some. It was argued that high-level languages were inefficient (in their use of machine resources) by comparison with assembler language. In retrospect, of course, this argument was strongly outweighed by several factors, of which the most important is the suitability of high level languages for expressing algorithmic solutions to problems. The key result was a change in the programmer’s cognitive process, and the problemoriented approach replaced the machine-oriented approach.

Along with problem-oriented languages came the subprogram, regarded by Dijkstra [1] and others as one of the most critical advances in programming. While the ability to compile subprograms separately led to savings in processing time, the big contribution of subprograms is its support for problem solving and decomposition. Virtually every programming language supports some form of subprogram capability, and it is at the heart of many current concerns in programming methodology. Without subprograms, it would remain nearly impossible to understand and to modify large programs. At this stage, though, little attention was given to matters of programming style and the overall development process. It was often a significant achievement simply to make a program work at all! This statement was especially true for many scientific experiments where the programming task included the construction of an interface between the scientific equipment and the computer. Any concern about the programming process was directed to the discovery of efficient algorithms for computation. 1.1. Recognition of the problem

With each successful system, though, came an effort to implement larger and more sophisticated systems. Gradually, the complexity of the software

0010-4655/85/$03.30 © Elsevier Science Publishers B.V. (North-Holland Physics Publishing Division)

120

A.!. Wasserman

/

Modern software development methodologies

systems increased. Furthermore, it was neccessary to modify such systems, often continually. As the price of hardware decreased and as hardware became more reliable, the problems of software development received increased attention. In scientific computing, this attention was first directed towards issues of algorithms and performance improvements, as well as to analyzing the precision of computations. These themes have continued to be important in scientific computing. At the same time, though, work began on issues of program correctness and software reliability, Clearly, speed of computation was irrelevant if the program failed to dependably produce results that were sufficiently accurate. Also, the cost of cornputing, the cost of the scientific equipment, and the cost of performing experiments were high, making software errors and program crashes very expensive. Along the same lines, the process of software development began to receive attention, not so much in scientific computing, but in a broad range of computing applications. As the cost of computing equipment came down and the complexity of programs went up, the cost emphasis shifted from hardware to software. Software became a significant and measurable cost item, requiring management, planning, and control. At that time (the late 1960’s), though, there were few, if any, effective techniques for management of software development, as the process was highly dependent on the skills of individual programmers. As a result, many projects that depended upon successful software production encountered serious problems, including lengthy delays, cost overruns, performance inefficiencies, and worse. In many instances, expensive projects were abandoned or restarted. Furthermore, it was often hard to tell until a very late stage in the project whether or not the software would be properly built and delivered. Even when the software functioned, the documentation was incomplete, and many systems went through a seemingly endless series of refinements and new versions, termed “maintenance”. Because many of these problems occurred on important and highly visible projects, such as radar systems, operating systems, and programming language compilers, people spoke of a “software cri-

sis”, and sought means to remedy the crisis. The most significant of the meetings held to address these topics were two meetings sponsored by the NATO Science Committee in 1968 (Garmisch) and 1969 (Rome) [2]. The meetings covered a broad range of concerns in the software area, extending from management through programming techniques and testing. “Software engineering” was chosen as the theme of these meetings. The term was intended to be provocative, asking if it were possible to apply the methods of traditional engineering disciplines to the production of software as a way to overcome problems with reliability, correctness, problem complexity, and the development process. It was well understood that software engineering was very much a wish, rather than a reality. 1.2. Structured programming solutions This concept of software engineering was the principal thrust for work in the 1970’s, as researchers and developers sought to turn the goal of software engineering into reality. In the early 1970’s, software engineering was largely a research activity with little impact on day-to-day software development practices. From this foundation, though came many of the most important notions of modern software development techniques, including top-down design, modularity [3,4], stepwise refinement, [5] and structured programming [6]. The relationship between programming languages, problem-solving, and high-quality software was also recognized at this time, leading to the design of many different programming languages, most notably Pascal [7].The principal focus was at the code-writing stage, with attention concentrated on good programming style. The elimination of unconditional branching (goto) and the use of regular control structures (sequencing, if—then—else, and while—do) in programming had favorable implications for program readability and verification of program correctness. These concerns had relatively little effect in the scientific programming community, where FORTRAN IV and ALGOL 60 remained the languages of choice. From the standpoint of structured programming and program readability, FORTRAN

A.1. Wasserman

/

Modern software development methodologies

IV was especially weak. Most programs made extensive use of the goto statement, modularity was compromised by widespread use of blank or labeled COMMON, and program comprehensibility was often hampered by the limit on the length of variable names and the use of EQUIVALENCE. Furthermore, FORTRAN’s poor data structuring capabilities forced programmers to devise very complicated data structures to represent records and/or pointers, so that it was often difficult to see the intended logical data structure. Some preprocessors were developed for FORTRAN to provide the programmer with a better set of control structures. However, these systems often forced the programmer to refer to the unattractive code produced with the preprocessor as the only way to understand diagnostic messages and to fix bugs. In addition, these preprocessors were only partial solutions and failed to address the data structuring and modularization issues, Even with languages such as Pascal, though, it was clear that programming played only a small role in the overall programming process. Instead, writing executable code was but one step in an entire sequence of steps, termed the “software life cycle”, 1.3. Life cycle concepts

The unifying concept of the “life cycle” was adapted from other engineering disciplines, and refers to a sequence of phases that comprise the software development and evolution activities, These phases include, but are not limited to, requirements analysis and definition, functional specification, design, manufacture (coding), quality assurance, operation and evolution, Throughout the entire process, there are aspects of management and communication, including documentation, budgeting, personnel deployment, project review, scheduling, and product control, that serve to tie the stages together and to provide the organizational environment that surrounds the technical process. Life cycle considerations may be seen clearly when considering other manufacturing disciplines, Consider the process of developing a new automobile. At first, the company begins with some gen-

121

eral idea of the requirements: size, price range, competitive products, performance goals, and other such requirements or wishes. Once these goals are established, a designer may take one of several steps. First, the designer may select a design that is very similar to an existing design, in the hope of simplifying the design activity and reducing costs, as well as reducing the overall design time. Alternatively, the designer may produce some original pictures as a way of studying possible choices. Having done this, the designer will proceed with more detailed design of the automobile, including the dimensions, the wheelbase, the engine type(s) and the brakes. This process includes integration of the constraints, so that the resulting design is unified. When this is done, the result is an architectural rendering of the car and a detailed blueprint from which a prototype version can be built. This prototype version can be tested and compared against the original requirements and constraints to see if the result is satisfactory. If so, then the prototype can be used as the basis for continued production. More likely, though, the prototype will be refined, either because of errors in the original requirement, or because the resulting automobile could be improved. In any event, the design process is controlled carefully, and the standard engineering notation of the blueprint plays a key role in this process. The blueprint can be understood and followed easily, with little danger of misinterpretation. This step is valuable, since it allows validation of the design before a large investment is made in the production of the automobile itself. The automobile design analogy is a good one for software development in the setting of scientific experiments, since many of the same issues come into play. With respect to software, then, the life cycle provides a framework for the type of engineering process being sought. The various steps of the engineering process then serve to delineate the activities of software development. Discussion of life issues makes it clear that there are two distinct, yet equally important, major areas in software engineering: technical and management issues. These issues are often closely

122

A.!. Wasserman

/

Modern software development methodologies

related, as each method used in software development should have both a technical and a managerial justification. Thus, we may say that software engineering is concerned with both the process of software development and the products that result from that process. The life cycle is a useful framework for discussing the entire software development process. The various steps may be put into a project plan, with separate schedules and budgets for each phase. Furthermore, documents and review procedures may be associated with each phase. Thus, the life cycle is an organizational structure that supports management of the software development process. A possible disadvantage is that rigid adherence to the steps of the life cycle is not always desirable, since it does not apply equally well to every project within an organization. The life cycle does not prescribe how to carry out the activity at each phase. In particular, it provides no guidance as to the appropriate technical methods or supporting tools that are to be used. Also, there is no agreement on the exact set of distinct phases that comprise a life cycle. In summary, there is little standardization among the steps of the life cycle or among the specific technical methods used at each step. As the concept of the life cycle became accepted in software engineering, it became apparent that there were very few methods and/or tools for addressing phases other than the code writing stage. The result of this awareness, beginning in the mid-1970’s, was to focus attention on analysis, specification and design methods. 1.4. Analysis, specification and design methods

Analysis of the problem at hand is the essential first step of any software development activity, Without such analysis, it is impossible to proceed; furthermore, an inadequate job of analysis is virtually certain to lead to project failure, since poor understanding of the problem makes it impossible to produce a good specification. Successful analysis involves communication with users and customers for the system, who can describe their needs. Analysis also involves cornmunication with the eventual developers of the

system, who must be able to evaluate implementation feasibility and to describe any design or implementation constraints. Because of the complexity of systems, key tools for analysis support problem decomposition, through any of a variety of schemes, including: procedural decomposition, data abstraction, dataflow, processing sequence, or transactions. Graphical notations help to show the interrelationships of system components to one another and facilitate the communication process. User involvement is particularly important in this phase, since users possess substantial expertise about the problem domain. There are a variety of approaches that can be used effectively to improve problem understanding and communication between the analyst(s) and user(s). These approaches include interviewing (in the journalistic sense), doing the user’s job (where feasible and permissible), building mockups or prototypes of the systern, and/or writing user level documentation for the proposed system. The principal product of the analysis process is a specification, which is the first complete description of the functions to be performed by the system. Different forms of specification are best suited to different types of systems, implying that no single specification method is appropriate for all classes of programs. The precise mathematical description that may be given for numerical scientific software cannot be applied to a commercial data processing system or to a highly interactive color graphics system. The methods that are used to describe multiprocessing and real-time control systems are inappropriate to describe small, single process sequential programs. There are also tradeoffs between using formal and informal specification techniques. In general, though, both formal and informal approaches are useful. The informal approach is used to develop an initial understanding of the system and to communicate that understanding to users and others without a technical background, while the formal approach provides an unambiguous and verifiable description of the expected system behavior. The process of software design permits the developer to determine the feasibility of implement-

A.!. Wasserman

/

Modern software development methodologies

ing the specification, to analyze alternative system structures, to develop algorithms for the solution of the problem, and to define any constraints upon the implementation. In summary, it is a stage at which the primary concern is with the means by which the system will be built. The design activity was originally largely implicit in the system development process, and any design was done either in conjunction with the specification process or with code production: In the former case, design issues often became entangled with more basic issues concerning the functions to be performed by the system; in the latter case, it was often impossible to make global design decisions. More recently, though, the design process has become more explicit. Indeed, the design activity can be separated into two phases: architectural design and detailed design. Architectural design is concerned with recognition of the overall program structure and the interconnection of program pieces, while detailed design is more concerned with the selection of algorithms and data structures that are appropriate to the fulfillment of specific system functions. One of the key goals of the design process is to simplify the subsequent stages of coding and testing. At the end of the design phase, virtually all of the key decisions concerning program organization, logical data structures, and processing algorithms will have been made, with the intent of making code production into a straightforward activity. The primary concerns of coding then focus on proper programming style, and the mapping of logical data structures to physical data structures. Many different techniques have been developed to aid the activities of analysis, specification, and design. Among the better known methods that are widely used for analysis are Structured Systems Analysis [8,9], SADTTM [10], and ISAC [11]. Specification methods include PSL/PSA [12], SREM [13], HIPO [14], Jackson System Design (JSD) [15,16] and algebraic specifications [17,18]. Well known methods for software design include Warnier’s LCP [19], Structured Design [20,21], Jackson Structured Programming [22] program design languages [23], and structured flowcharts [24]. The interested reader is also referred to the

123

collections by Freeman and Wasserman [25] and by Bergland and Gordon [26], which contain additional information on these and other techniques. Each of these methods provided an approach for handling a specific phase of the software life cycle. However, as noted, none of the methods were universally well-suited and none of them were intended to handle the separate phases of analysis, specification, and design. As a result, it was necessary to select different methods for different aspects of the pre-implementation stage. This process was particularly difficult, since the various methods used different representations and often did not combine well with one another. Furthermore, there was little, if anything, available to provide automated assistance in the use of these methods. As a result, these different methods received relatively little use, despite their apparent value in representing aspects of software function and logical structure. The major shortcomings, namely the need for an integrated series of steps and the need for automated support, have been addressed in more recent work in software engineering. The remainder of this paper emphasizes that work.

2. Software development methodologies This fifth stage addressed the problems of integrating methods for each of the various tasks of the software development process. A visible result of this activity is the emergence of a large number of software development methodologies, many of which purport to include all of the steps necessary for successful completion of a software development project [27,28]. Our notion of a methodology is that of a system of methods and tools, carefully chosen and integrated to support both the development and evolution processes *, Work on the development of methodologies is still underway, and there are TMSADT is “methodology” a trademark of Sofrech, Inc. American usage for * The term is the accepted .

.

.

this concept, refernog to a collection of methods. Strictly speaking, the term “method” is more accurate to express this notion.

124

A.!. Wasserman

/

Modern software development methodologies

few, if any, stable and widely used approaches. Instead, organizations are seeking to evolve their own approaches, building upon their present tools and techniques [29]. 2.1. Goals for a software development methodology One important step in this process is the establishment of goals for a software development methodology. The following idealized set of goals, based on efforts to define requirements to be used when developing embedded systems in Ada, can be used as the basis for evaluating the software development process in an organization and for modifying that process to improve both the process and the resulting software systems [30]. (1) Entire development process A methodology should cover the entire development process. It does little good to have a methodology for software design if there is no systematic procedure to produce the functional specification used for the design and the program(s) that must be created from the design. The methodology should facilitate transitions between phases of the development cycle. When a developer is working on a particular phase of a project (other than requirements analysis), it is important to be able to refer to the previous phase and to trace one’s work. At the design stage, for example, one must make certain that the architecture of the software system provides for all of the specified functions; one should be able to identify the software module(s) that fulfill each system function. During implementation, it should be easy to establish a correspondence between modules in the system design and program units, and between the logical data objects from the design stage and the physical data objects in the program. It is important to note that one must be able to proceed not only forward to the next phase of the life cycle, but also backward to a previous phase so that work can be checked and any necessary corrections can be made. This phased approach to software development makes it clear that information lost at a particular phase in generally lost forever, with an impact on the resulting system. Eventually, during acceptance testing (or perhaps

during system operation), that failure will be recognized and it will be neccessary to make modifications to the system. (2) Effective communication A methodology should enhance communication among all of the different persons involved in a software development project at all of the different stages of development. Written documents serve to support this communication by providing a record of decisions and agreements. At times, such documents may be informal, simply recording a piece of information or an agreement. This approach is particularly useful when a document is intended for someone without specialized knowledge of software development. However, precise communication among technically knowledgeable individuals is enhanced by the use of formal notations, including graphical representations and specialized languages. Examples of such notations are data flow diagrams, HIPO diagrams, algebraic specification of data types, and FORTRAN. The methodology should prescribe the forms of documentation and representation that will be used for this technical communication at all stages of the development process. (3) Support for problem analysis A methodology should support effective problem-solving techniques. It should encompass intellectual processes to support problem decomposition. Modelling is a particularly important aspect of this objective. There exist techniques for activity and data modelling. Formal modelling techniques should be based upon a suitable set of primitives for the application domain. For embedded systems, concepts such as external interfaces and concurrent processes should be included. The resulting model should serve to answer questions about the problem domain and can be used to define the scope of interest for the system. Unless one has suitable tools for analysis of the problem, one cannot easily produce a functional specification or a satisfactory system. Problem solving techniques are used to decompose the problem and to create the model. Techniques such as data abstraction, data flow diagrams, functional decomposition, transaction

A.!. Wasserman

/

Modern software development methodologies

modelling, and state machines have been effectively used for this purpose in the past. The goal of this phase is to determine the structure of the problem the interrelation of the parts of the problem. —

(4) Support for top-down and bottom-up development The methodology should support a variety of different approaches to system design and dëvelopment. While systems are often modelled or analyzed using methods of top-down decomposition, there are typically low-level constraints, such as interface requirements, for which a bottom-up approach is necessary. Efforts to reuse software designs and/or code, including the use of packages, also leads to the use of bottom-up techniques to develop software. Finally, design constraints, such as performance requirements, necessitate the creation and testing of low-level aspects of a system before many of the high-level aspects have been developed. Thus, the methodology should not constrain the developer to follow either a pure bottom-up or a pure top-down approach to software development, but should allow any combination of the two in system design and implementation. (5) Support for validation and verification The methodology must support determination of system correctness throughout the life cycle. System correctness encompasses many issues, including not only the correspondence between the results of one stage of development and the previous stage, but also the extent to which the system meets user needs. For example, the methods used for analysis and specification of the system should make it possible to trace later system development back to the requirements and functional specification. The methodology must prescribe a strategy for assurance of system quality. Test planning, document review, design and code walkthroughs and/or inspections should be integrated in the methodology, with emphasis given to early error detection and correction. A test plan should establish standards for test coverage, a means for measuring that coverage, and the acceptable quality criterion.

125

(6) Support for constraints A methodology should facilitate the inclusion of design, implementation, and performance constraints in the system requirements. Systems often have severe requirements on memory utilization, real time response, use of specific machine-dependent features, or integration with other hardware or software systems. Experience has shown that severe constraints have a major detrimental effect on system development. It should be possible to state these requirements using the methodology. Furthermore, it should allow them to be incorporated into each subsequent stage and verified. This requirement may necessitate the use of analytical tools and/or construction of prototype systems. (7) Support for management of software development The methodology must, above all, support the intellectual efforts of the designers and other technical people. Beyond this, it should support the software development organization that has been chosen for the project at hand. It must be possible to manage the developers and the developers must be able to work together. This requirement implies the need for effective communication among analysts, developers, and managers, with well-defined steps for making progress visible throughout the development activity. The intermediate products generated by the methods and tools, such as a detailed design or an acceptance test plan, can be reviewed by the organization so that progress can be effectively measured and so that quality can be assured. The methodology must support the management of the project. It should include methods for cost estimation, project planning (scheduling), and staffing. It should also specify methods for ongoing review of project progress, such as design walkthroughs and code inspections. The management procedures should maintain a project handbook and library, showing project history, project plans, and the evolving software product(s). Finally, the management procedures should identify an evolving set of standards and conventions that can be applied to all projects.

126

A.!. Wasserman

/

Modern software development methodologies

(8) Support for system evolution The methodology should support the eventual evolution of the system. Systems typically go through many versions during their lifetimes, which may last eight to ten years or more. New requirements arise from changes in technology, usage patterns, or user needs, and these changes or additional requirements must be reflected in a modified system. In many ways, the evolution activity is a microcosm of the development process itself, The development methodology can assist this evolutionary activity by providing accurate external and internal system documentation, and a well structured software system that is easily prototyped and modified by those making the system changes. (9) Automated support Wherever possible, the methodology should be supported by automated tools that improve the productivity of both the individual developer and the development team. This collection of tools, and the way in which they are used, constitute a “programming suppbrt environment.” (Note that the word “programming” is not used in the sense of “coding” here, but rather in a more general sense.) The tools should be integrated so that they may communicate via a common database and so that they can work effectively with one another. Furthermore, these tools should be linked to the methods so that the tools support the management methods and technical practices. (10) Support for software configuration The methodology should maintain the visibility of the emerging and evolving software product and its supporting work products. All of these items should be placed in a project database. The underlying notion is that of software configuration management. All of the components of a software development project must be identified, collected, and controlled in order to assure proper distribution of the finished system and its supporting work products, as well as to assist in evolution of the product and version control, (11) Open -endedness A methodology should be open-ended. It should

be possible to introduce new technical and managerial methods, as well as to create new tools, and thereby modify the methodology. Techniques and tools evolve through a process of “natural selection”, whereby newer, more effective techniques and tools supplement or replace existing ones. Clearly, this set of goals applies most directly to highly complex systems requiring at least two person years of effort (several people over several months). Furthermore, it is unrealistic to expect any but the most highly structured organizations to achieve all of these goals. Nonetheless, inspection of the list and evaluation of one’s own practices against these goals is instructive, since it illustrates relative strengths and weaknesses, suggesting the most likely areas for improvement. 2.2. An example.’ User Software Engineering Many of the concepts of software development methodologies can be illustrated by a brief overview of the User Software Engineering (USE) methodology, developed by the author at the University of California, San Francisco [31,32]. User Software Engineering is a methodology, supported by automated tools, the Unified Support Environment, for the specification, design, and development of interactive information systems. An interactive information system (IIS) may be characterized as a program that provides its users with conversational access to data. An IIS is seen to consist of three parts: the user-program interaction, the database design, and the operations (transactions) associated with various user inputs. Most of the operations, though not all, involves access to and/or modifications of the database. In addition to the general goals of software development methodologies described above, a principal goal of the USE methodology is support for effective user involvement in the development process. The user involvement is largely achieved through the early specification of the external (user) interface to the system, combined with the construction of easily modified prototypes of user interfaces. The steps of the USE methodology can be

A.1. Wasserman

/

Modern software development methodologies

summarized as follows: (1) Requirements analysis—activity modelling, data modelling, analysis of user characteristics and expected usage characteristics (2) External design, including terminal selection and user interface(s) (3) Informal specification of the system operations using narrative text (4) Relational database design (5) Addition of system functions, providing at least some, and possibly all, of the system’s functions (6) Formal specification of the system operations (7) Architectural and detailed design (8) Implementation (9) Testing and/or verification Each of these stages has both associated documents (and/or executable programs) and methods for review. For example, the second stage, external design, yields a set of executable USE state transition diagrams, which have been validated through user experience and feedback [33]. The interaction is described in a set of augmented state transition diagrams, each of which is termed a conversation. Nodes in the diagrams are associated with messages and system inputs, while arcs are associated with user inputs. Actions may be associated with a transition along an arc, so that all of the operations may be attached to transitions. The transition diagrams are encoded and compiled by an automated tool, the Transition Diagram Interpreter (TDI), that “executes” the diagrams, thereby providing the user with the user interface to the program at a terminal, The Transition Diagram Interpreter is part of the RAPID/USE application development system [34,35,36].RAPID/USE also contains an Action Linker that allows the transition diagram description to be compiled along with programs written in C, Pascal, and/or FORTRAN 77. Both TDI and RAPID/USE provide direct access to the Troll/USE relational database management systern [37,38]. The combination of the dialogue management facilities of RAPID/USE with the database management facilities of Troll/USE is especially well suited for the rapid creation of interactive information systems. It can be seen that the Transition Diagram Interpreter supports

i27

the second stage of the methodology, while Troll/USE supports the process of relational database design, and RAPID/USE supports the creation of a functioning system (step 6). Other support for the USE methodology includes the BASIS method for formal specification of the operations [39] and the PLAIN programming language, a Pascal-based language designed especially to meet the requirements of a systematic, wellstructured modern programming language for the development of interactive information systems [40,41]. Thus, there are not only well-defined steps in the USE methodology, but also a set of tools that support the methodology. In many ways, USE is typical of current methodologies, addressing some aspects of the development process, while ignoring others. Little is said about process modelling or testing, and procedures for project management are only lightly sketched. At the same time, the specification method and the use of prototypes are carefully prescribed. When evaluated against the stated goals for methodologies, USE rates moderately well on the most of them, except that it provides almost no support for system constraints. 2.3. Directions in methodologies Currently used methodologies may be regarded as being relatively rigid and incomplete. In general, they are only a few years old and have only been applied to a handful of systems, typically of small-to-moderate size. Thus, there is relatively little experience with them, and one can expect them to undergo changes as this experience is gained. There are several directions where additional work is clearly needed: (1) Support for variations There are many differences in development settings and many different types of systems. The approaches that will work successfully in a highly structured, tightly managed organization involved in building precisely defined systems to contract specification may not work at all in academic or research settings where the management structure is much less formal. Similarly, some complex

128

A.!. Wasserman

/

Modern software development methodologies

methodologies introduce so much overhead that they are only suitable for extremely large systems. Finally, much more rigor is required for large and critical systems than for smaller or less critical systems. At present, though, most methodologies do not make this distinction, and prescribe only a single form of their methodology, (2) Support for reusability Although there has been some mention of software reusability, it is expected that many of the major forthcoming improvements in developer productivity will come from the ability to reuse specifications, designs, and code that were developed for previous projects. To do so effectively requires not only access to the documents and code for a particular system, but also access to an “intelligent” database from which information about reusable components may be extracted. Work in this area is very preliminary, and must be integrated into software development methodologies so that one’s attention is automatically drawn to candidates for reuse [42]. It should be noted that much of the effective reuse of software has been in numerical scientific applications. Although most of the reuse is achieved through the use of code (SUBROUTINE and FUNCTION libraries), existing documentation about such libraries can be stored in machine processable format to allow a developer to browse through information about the library and to locate modules that may be reused.

need for more tools to assist with software specification, design, and testing, as well as with management of software projects. The entire subject of software tools and programming environments is now receiving significant attention, and has now become a major research direction in software engineering [43,44].

3. Software engineering environments An important consideration for the software development organization is the entire development environment. In its most general sense, the development environment includes the technical methods, the management procedures, the computing equipment, the automated tools to support development, and the physical workspace. With this broad definition, the software development environment is different for every organization, as it is dependent upon the individuals who comprise that organization. More importantly, it can be seen that the environment is affected by every change, no matter how minor, ranging from the use of new computer equipment to new hirings. Improvements in developer productivity may therefore originate from many different changes in the developer’s environment. The present concerns include the following.: (1) programming support tools improvements to traditional programming tools, such as editors, compilers, debuggers, linkers, loaders, and operating systems support for those tools; tool integration through sharing of common data structures and communication protocols among tools; use of project databases; development of tools closely linked to a software development methodology; (2) hardware support the advantages and disadvantages of using a network of personal development systems, e.g., workstations, in place of traditional time-shared computer systems; (3) physical workspace the integration of tools and workstations into an overall working environment, with emphasis on ergonomic considerations in the workplace. —

(3) Integrated tools

While many tools exist to support the software development process, particularly coding and debugging, the relationship between tools and methodologies is weak. There are few settings in which the tools actually work effectively in harmony with one another and in support of a software development methodology. There are two significant problems with most tool sets. First, they fail to support a software development methodology or to capture any data that assists in control of the software development process. Second, present tools primarily support coding activities and fail to support the software development life cycle in its entirety. There is a

—

—

A.!. Wasserman

/

Modern software development methodologies

3.1. Software tools

Modern development environments are typically implemented on a time-sharing system and provide the developer with a collection of tools. This tool collection usually takes on of two forms: a tool system or a tool kit. In a tool system, such as INTERLISP [45] or Smalltalk-80 [46], all of the tools are organized in support of a development task, typically in support of a single programming language. Each tool has a single purpose and is expected to be used in a certain way. In a tool kit, as best exemplified by UnixTM [47], the environment contains a large number of tools, possibly in support of a variety of programming languages, but the developer has considerable freedom in deciding when and how to use the various tools. In either case, though, the key objective is to provide programmers with tools and techniques that can both improve the process of software production and the quality of the resulting software. Among the most common examples of such tools are text editors and compilers; file copying and printing routines are among the most common utilities. Examination of a number of tools, development environments, and proposed environments is useful in enumerating some specific tools that should be part of an information system development environment. These tools include the following: An operating system that provides access to shared resources, standard interfaces to I/O devices, and program linkage and loading facilities is basic to any system development environment.

i 29

in the information system development environment. Text editor. A text editor is essential for all of the documentation associated with the system development, as well as for the program code. Some programming environments may provide syntax-directed editors that are able to check program input for conformity to the syntax rules of one or more languages that are used during the development process [48]. Formatters. Formatting programs are useful both

for documentation and for programs. Formatting programs support processing of tables, equations, and text, providing a means to produce documents ~or reproduction and/or phototypesetting. A “pretty printer” accepts program text as input and produces a reformatted program text as output, performing any needed indenting or line splitting to enhance the readability of the program text. A static analyzer is able to examine program text to determine whether there are any program variables that are used before they are assigned a value, any program variables that are assigned a value and never used, or any segments of “dead code” that cannot be reached through the program’s control flow. Such a tool is valuable in uncovering errors and in improving program structures. Static analyzer.

Operating system level support.

A database management system can be used both by the tools in the development environment and by the systems developed in the environment, Database management system.

Compilers and/or interpreters, along with runtime support are needed for all of the programming languages that are to be used Language processors.

TM

Unix is a trademark of AT&T Bell Laboratories.

Dynamic analyzer. A dynamic analyzer is able to provide information about a program during its execution, One can obtain a shapshot giving the values of program variables, a trace that shows the program units that have been invoked, an analysis of the execution time spent in each program unit, or a count of the number of times that specific lines in the program have been executed. With a suitable graphics-based tool, one could observe the dynamic execution of a program seeing accesses to program units or data objects. All of this information is useful for uncovering errors and for tuning the system to enhance performance. Such dynamic analyzers are often incorporated into sophisticated source-level debugging tools [49]. Similar dynamic analysis tools exist for data-

130

A. I. Wasserman

/

Modern software development methodologies

base management systems, making it possible to count the number of I/O operations, the number of pages fetched, or the elapsed time for various database operations. management. A configuration management tool may be used to keep track of the myriad documentation associated with an system development project, including one or more versions of the source and object code. Such configuration management, as noted above, is an important aspect in controlling changes to the emerging system. Configuration

Logging aids may be used for system auditing and as a way of determining the use of various programs in the development environment, Logs may keep track of user access to the system and can assist in uncovering any security violations of the system. Logging tools.

Development dictionary. A development dictionary

be used to store information on data items, data structures, functions, and other information gathered through the development process. Information can be entered into a dictionary beginning at the analysis phase, as one identifies data elements and data flows. System functions specified early in the development process can be linked to modules in the design phase and procedures or functions in the implementation phase through the development dictionary. may

This list is far complete, since one could provide useful tools for specification, architectural design, detailed design, database design, program verification, project management, and library maintenance, but it is intended to be suggestive of the types of tools that can aid the developer in the effective production of high quality software systems. If such a collection of tools can be designed in a harmonious way, then the automated development support system should have a significant favorable impact on the development process. In order to meet this goal, the tools that are created for this environment will have to possess certain characteristics, including: (1) ease of use the user must not need elaborate —

knowledge in order to be able to use a tool. (2) levels of use the casual user of a tool should be able to gain a substantial portion of the benefits of the tool with only a small subset of its available facilities, while the sophisticated user of the same tool may be able to gain additional benefits through the complete set of features. (3) consistency with other tools each tool should have a standard interface that enables it to be used in conjunction with other tools, and should have a set of usage conventions that conforms to those of other tools in the environment. (The Unix “pipe” mechanism is a good example of this consistency.) (4) adaptability tools should be adaptable to specific user requirements, and should support different patterns of use by different classes of users; furthermore, tools should contain a meaningful set of defaults that can be altered by the user in order to tailor it to individual preferences and needs. (5) local intelligence tools should be designed to collect information in a private database concerning profiles of usage of the tool; the data thereby captured could assist in the evolution of that tool and in the development of new tools. The value of such improved tools and tool integration can be seen in a program development setting such as Pecan, which supports the development of Pascal programs [50]. Pecan employs a multiple-window workstation environment. A structured editor is used to create the program text, and a pictorial representation of the program structure is created in another window. If a runtime error occurs in a program, debugging tools are immediately available to assist the programmer to help find the source(s) of the error; the program can be easily edited to change the program and rerun it. The programmer has access to the current value of program variables and can examine the runtime stack for the program as well. —

—

—

—

3.2. Hardware support

Just as there was a transition from batch processing to time-sharing in the 1970’s, a transi-

A.1. Wasserman

/

Modern software development methodologies

tion from timesharing to personal computing is presently underway. The goal is to create a hardware/software development environment in which the individual software developer can “live and work” via a personal machine that provides the maximum possible support for the developer’s various activities. In addition to the obvious system development work, the machine can serve as a vehicle for a number of important tasks: (1) Text processing, including the preparation of system documentation, technical papers and reports, and correspondence, as well as editing or reviewing material typed by others. (2) Information storage and retrieval, including documents associated with various software development projects, telephone directories, mailing lists, and personal data bases. (3) Communication, including the sending and receiving of mail throughout both a local network and a larger network involving remote access, and conferencing with one or more persons via these networks. (4) Self-management, including a long-term diary, short-term (day or week) scheduling, and a “to-do” list, with reminders that can be automatically triggered by a system clock, We may call such a development system a “personal development system” (PDS) [51]. The typical configuration for a PDS provides the programmer with at least the computing power available in a present-day medium scale time-sharing system. The components of this configuration typically include: (1) an intelligent terminal having approximately 2 Mbytes of primary memory and the computing power and address space of today’s 32-bit minicomputers; (2) local secondary storage (e.g., a Winchester disk) with upwards of 40 Mbyte of storage capacity as well as some removable media (e.g., floppy disk or tape cartridge); (3) graphics capability, including multiple character fonts, reverse video, variable intensity, split screen, color, and the ability to display a standard page of text completely on the screen; (4) networking capability, to connect with other PDS’s in a local network and to geographically dispersed systems.

131

Within this framework, it should be possible to add additional primary and secondary storage, as well as other kinds of peripheral devices, e.g., a print-quality hardcopy device. This configuration not only provides powerful local processing, but opens up a broad range of possibilities for user interaction, permitting the use of windows and menus for task selection and sequence control, as well as incorporating pictures and eventually audio input and output. Such a PDS is presently available in the price range of 10—20 k$, a small sum when compared to the costs of skilled software developers. The potential benefits of the PDS are substantial, and can not only make the development organization more productive, but can also improve software quality. Among the advantages that the PDS can provide over the time sharing approach presently used are improved reliability of hardware, improved hardware performance, private mass storage, and support for non-keyboard input. 3.3. Physical workspace

Studies indicate that most programmers spend less than a third of their time actually doing programming, i.e., writing, debugging, testing, or documenting code. Much of the rest of the time is occupied with mundane chores, meetings, or vanous kinds of breaks and interruptions. Additional time is taken in becoming familiar with new applications or new development environments. While it is unreasonable to expect a programmer to spend all of the time performing programming tasks, it is clear that there is much room for improvement in programmer effectiveness. Some gain can be brought about by the improvement and standardization of tools previously discussed; additional gains can be achieved through environmental changes. For example, the programmer’s physical work environment seems to relate closely to productivity. Such factors as office characteristics, noise levels, smoke, terminal type and speed, lighting, and access to library materials and documentation can all affect the programmer, although different programmers will be affected to varying degrees by the different factors.

132

Al. Wasserman

/

Modern software development methodologies

Also, a large portion of programmer overhead is attributable to the time associated with routine chores and clerical tasks. Advances in office automation, including word processing systems for documentation, automated mailing systems, and the like, can substantially reduce the time needed for these activities. The software development activity can be integrated with the capabilities of the “electronic office”, which offers the developer many features to organize and utilize time more effectively, as well as to help facilitate better cornmunication and information sharing among peers and management. An especially important consideration in the physical workspace is the developer’s computer terminal. Because so much time is spent at the terminal, it must be designed to minimize the occurrence of errors, to reduce visual fatigue (a cause of eye strain and headaches), and to increase the effective user input and system output speeds. There are a number of ergonomic considerations in the design of the visual display and the keyboard that can affect the developer, The first set of criteria apply to the visual display. Most alphanumeric terminals use a dotmatrix approach for display, ranging from a 5 X 7 minimum upward, with 7 >< 9 being the most cornmon format for character generation. The higher the dot density, the more readable the character will appear. The use of ascenders for tall letters and descenders for letters below the base line, such as “g” and “p”, is particularly helpful. Character size and spacing is equally important. Character width should be 70—80% of character height, and line thickness should be 10—12% of character height. Assuming that one views a terminal from an average distance of 50—60 cm, the minimum acceptable character height is about 0.3 cm, although 0.4 cm procides substantially better reliability. There are also important minimum levels for horizontal and vertical character separation. Another characteristic of visual display units is the refresh rate. It is important to maintain a refresh rate that eliminates flicker. While the minimal refresh rate is dependent upon the type of phosphor and the number of lines on the screen, the normal North American refresh rate of 60 Hz

is quite good, and the European 50 Hz rate is only minimally acceptable. Finally, chromaticity is important, with the ideal screen color falling into the range where the eye is most sensitive. Studies have indicated that greens are the best colors, but the color may be less important than the level of contrast, making dark colors such as red and blue less suitable for screens. The next set of criteria apply to the keyboard. In addition to the positioning of various keys, a number of physical parameters influence user performance, including: (1) the shape of the keys, (2) the glare or reflection from the keys, (3) the force required to displace the key (bounce), (4) the key displacement (travel). As might be expected, there are significant individual differences in keyboard preferences. The last set of terminal-related criteria relate to its placement in the working environment. It is important to strike a balance between the proper amount of lighting for work and the reduction of bothersome glare on the terminal. Glare can often be reduced by adjustment of the task lighting or by coating the screen. The angle of the screen and the keyboard is important to support readability and to prevent poor posture leading to backache and headache. 3.4. Summary: the physical environment

This discussion only scratches the surface of the ergonomic and human factors aspects of the physical workspace, but it serves to highlight many of the important considerations and to point toward needed future improvements, some of which are discussed elsewhere [52]. This entire area is in need of a great deal of additional study and experimentation so that suitable workspaces may be devised and used in conjunction with modern software development methodologies.

4. Conclusion This paper has presented a historically-oriented overview of the development of software engineering and related topics. The field has expanded

A.1. Wasserman

/

Modern software development methodologies

greatly over the past fifteen years from a point at which there was little or no understanding of the nature of the problems of software development to the present, where several major problem areas have been successfully addressed, The nature of computing and the nature of software is continuing to change rapidly [53].As a result, new challenges arise as rapidly as old problems are solved, Active research in areas such as expert systems, automation of the programming process, and highly interactive systems both present new approaches to software creation and require the development of more sophisticated tools

133

methodologies and software engineering environments is still in its infancy, with a vast amount of work yet to be done. Among the most critical needs in this area is that of evaluation, being able to determine quantitatively the benefits gained by the use of various methodologies, tools, and environments. Until now, there have been very few scientifically controlled studies that establish the value of any of the methods and tools discussed here. It is not until such data exists that the concepts of software engineering can be widely accepted and used.

[54].

Work is continuing in each of the six areas described in this paper. Problems of software development are now being recognized by people working in personal computing, where many programs are widely distributed. Developers recognize the high cost of errors and the difficulty of software production. Structured programming considerations may be found in programming language design and programming methodology. The focus now is not so much on control structures, but rather on data structuring, with particular attention on language constructs for data abstraction and modularization. The design of Ada addressed many of these issues, and they are central to current discussions involving a successor to FORTRAN 77 Life cycle models are being analyzed and revised to incorporate prototyping of systems and reuse of software. It is now recognized that many classes of systems do not fit the life cycle model very well, especially those where it is difficult, if not impossible, to specify the system functions in the absence of some experimentation with a partial system. Many new approaches are being devised for analysis, specification, and design. Shortcomings of existing methods often include poor support for transitions between informal and formal understanding of the problem and difficulties in going from specification to design. Also, there were few techniques for distributed systems, expert systems, and other types are systems that are now becoming widely used. Of course, the work on software development

References [1] E.W. Dijkstra, Commun. ACM 15 (1972) 859. [2] J. Buxton, P. Naur and B. Randell, Software Engineering Concepts and Techniques (Petrocelli/Charter, New York, 1976) (Reprint of original NATO Science Committee reports). [3] D.L. Parnas, Commun. ACM 15 (1972) 1053. [4] B. Liskov, Proc. AFIPS 1972 FJCC 41 (part 1) (1972) 191. [5] N. Wirth, Commun. ACM 14 (1971) 221. [6] O.-J. Dahl, E.W. Dijkstra and C.A.R. Hoare, Structured Programming (Academic Press, London, 1972). [7] N. Wirth, Acta Informatica 1(1971) 35. [8] C. Gane and T. Sarson, Structured Systems Analysis (Prentice-Hall, Englewood Cliffs, 1979). [9] T. DeMarco, Structured Analysis and System Specification (Prentice-Hall, Englewood Cliffs. 1979). [10] D.T. Ross and K.E. Schoman, Jr., IEEE Trans. on Software Eng. SE-3 (1977) 6. [11] M. Lundeberg, G. Goldkuhl and A. Nilsson, Information Systems Development — a Systematic Approach (Prentice-Hall, Englewood Cliffs, 1981). [12] D. Teichroew and E.A. Hershey, IEEE Trans. on Software Eng. SE-3 (1977) 41. [13] C.G. Davis and C.R. Vick. IEEE Trans. on Software Eng. SE3 (1977) 69. [14] J.F. Stay, IBM Systems J. 15 (1976). [15] M. Jackson, System Development (Prentice-Hall International, London, 1982). [16] J. Cameron, JSP&JSD: Jackson Society, Approach Software Development. IEEEThe Computer LostoAlamitos, CA (1983). [17] H.K. Berg, W.E. Boebert, W.R. Franta and TO. Moher. Formal Methods of Program Verification and Specification (Prentice-Hall, Englewood Cliffs. 1982). [18] J.V. Guttag and J.J. Horning, Acta Informatica 10 (1978). [19] J.-D. Warnier, Logical Construction of Programs (Van Nostrand Reinhold, New York, 1974). [20] M. Page-Jones, The Practical Guide to Structured Systems Design (Yourdon Press, New York. 1980).

134

Al. Wasserman

/

Modern software development methodologies

[21) E. Yourdon and L.L. Constantine, Structured Design (Prentice-Hall, Englewood Cliffs, 1979). [22] MA. Jackson, Principles of Program Design (Academic Press, London, 1975). [23] S.H. Caine and E.K. Gordon, Proc. AFIPS 1975 NCC 44 (1975) 271. [24] I. Nassi and B. Shneiderman, ACM SIGPLAN Notices 8 (8) (1973) 12. [25] P. Freeman and Al. Wasserman, eds., Tutorial: Software Design Techniques, 4th edition. IEEE Computer Society, Los Alamitos, CA (1983). [26] GD. Bergland and RD. Gordon, Tutorial: Software Design Strategies (2nd edition), IEEE Computer Society, Los Alamitos, CA (1981). [27] T.W. Olle, H.G. Sol and A.A. Verrijn-Stuart, eds., Information System Design Methodologies: a Comparative Review (North-Holland, Amsterdam, 1982). [28] T.W. Olle, HG. Sol and C.J. Tully, eds., Information System Design Methodologies: a Feature Analysis (North-Holland, Amsterdam, 1983). [29] Al. Wasserman, J. Am. Soc. Information Sci. 31(1980) 5. [30] Ada Joint Program Office, ACM Software Engineering Notes 8 (1) (1983) 33. [31] Al. Wasserman, in: Information System Design Methodologies — a Comparative Review, eds. T.W. Olle, HG. Sol and A.A. Verrijn-Stuart (North-Holland, Amsterdam, 1982) pp. 591—628. [32] Al. Wasserman, PA. Pircher, D.T. Shewmate and ML. Kersten, IEEE Trans. on Software Eng., SE-12 (1986), in press. [33] Al. Wasserman, Extending State Transition Diagrams for the Specification of Human—Computer Interaction. [34] Al. Wasserman and D.T. Shewmake, ACM Software Engineering Notes 7 (5) (1982) 171. [35] Al. Wasserman, D.T. Shewmake and P.A. Pircher, RAPID/USE Reference Manual, Laboratory of Medical Information Science, University of California, San Francisco (1984). [36] Al. Wasserman, The Role of Prototypes in the User Software Engineering (USE) Methodolog~’,in: Advances in Human Computer Interaction, ed. HR. Hartson (Ablex Publishing, Norwood, 1985), in press.

[37] ML. Kersten and Al. Wasserman, Software — Practice and Experience 11(1981)175. [38] M.L. Kersten, Al. Wasserman and R.P. van de Riet, Troll/USE Reference Manual, Laboratory of Medical Information Science, University of California, San Francisco (1984). [39] N.G. Leveson, Al. Wasserman and D.M. Berry, lnformation Systems 8 (1983) 15. [40] Al. Wasserman, D.D. Sherertz, ML. Kersten, R.P. van de Riet and M.D. Dippé ACM SIGPLAN Notices 16 (5) (1981) 59. [41] Al. Wasserman, R.P. van de Riet and M.L. Kersten, in: Algorithmic Languages, eds. J.C. van Vliet and J.W. de Bakker (North-Holland, Amsterdam, 1981) pp. 29—47. [42] Proc.: Workshop on Reusability in Programming, ITT Programming, Stratford, CT (September, 1983). [43] D. Barstow, H. Shrobe and E. Sandewall, eds., Interactive Programming Environments (McGraw-Hill, New York, 1984). [44] P. Henderson, ed., ACM SIGSOFT/SIGPLAN Software Eng. Symp. on Practical Software Development Environments, ACM Software Eng. Notes 9 (3) (May, 1984). [45] W. Teitelman and L. Masinter, Computer 14(4) (1981) 25. [46] A. Goldberg and D. Robson, Smalltalk-80: the Language and its Implementation (Addison-Wesley, Reading, 1983). [47] D.M. Ritchie and K. Thompson, Commun. ACM 17 (1974) 365. [48] T. Teitelbaum and T. Reps, Commun. ACM 24 (1981) 563. [49] M. Johnson, ed., ACM Software Eng. Notes 8 (1983). [50] S.P. Reiss, IEEE Trans. on Software Eng. SE-li (1985) 276. [51] S. Gutz, Al. Wasserman and M.J. Spier, Computer 14 (4) (1981) 45. [52] Al. Wasserman, in: Advances in Computers, vol. 22, ed. M. Yovits (Academic Press, New York, 1983). [53] Al. Wasserman and S. Gutz, Commun. ACM 25 (1982) 196. [54] R. Balzer, T. Cheatham and C. Green, Computer 16 (11) (1983) 39.