- Email: [email protected]
Computer program testing
EUROMICROReports In his lecture, "Automatic Synthesis, Verification and Testing", he also gave a method for partitioning a logic design which converts a VLSI diagnosis job into an LSI diagnosis job. A formal method for the design of testable hardware was proposed by .4.C. Parker (University of Southern California, U.S.A.) and L.J. Hafer (Carnegie-Mellon University, U.S.A.) in their lecture, "Automating the Design of Testable Hardware". The method uses an algebraic model of resister-transfer (RT) behavior which the lecturers have developed. The proceedings of VLSI 81 have been edited by John P. Gray and are available from Academic Press Inc., Ltd., 24-28 Oval Road, London NWl 7DX, England (1981. xiv + 364 pages. ISBN 0-12-296860-3. Price: £15/$36).
Computer Program Testing The Sogesta Summer School, held this year from June 29 - July 3, 1981 in Urbino, Italy, concentrated on Computer Program Testing. The lectures delivered at the conference provided a comprehensive, tutorial discussion of the current state of the art as well as research directions in the area of testing computer programs. They covered a wide spectrum of topics: from theoretical notions through practical issues in testing programs and large software systems to integrated environments and tools for performing a variety of tests. The lecturers were all active and recognized researchers and practitioners in the field. A report on the Summer School on Computer Program Testing is featured below.
Introductory Concepts In the first lecture of the conference, entitled "Computer Program Testing - An Introduction", S.H. Zweben (The Ohio State University, U.S.A.) presented a general overview of basic terminology and philosophies in computer program testing. Some problems associated with many of the commonly suggested strategies were also discussed. L.J. White (The Ohio State University, U.S.A.) surveyed some of the "Basic Mathematical Definitions and Results in Testing". In order to study concepts of program structure, digraphs were introduced as a potential model. White also reviewed the papers of Goodenough/Gerhart and Howden, which most researchers in the area of program testing agree comprise the basis for a theory of testing.
Aspects of Program Testing A tutorial introduction to static program checking was given by C. Ghezzi (Universit~t di Padova and Politec-
49
nico di Milano, Italy). He showed how programming languages influence the amount of checks that can be done statically on programs and also how recent programming languages are designed with the express goal of supporting extensive static checking. In his lecture, "Levels of Static Program Validation", he also introduced higher order forms of static checking provided by data flow analyzers and symbolic executors. Ghezzi finally discussed how static checking tools can be integrated in a coherent development system. Data flow analysis has been shown to be a useful tool in demonstrating the presence or absence of certain significant classes of programming errors. According to L.J. Osterweil, L.D. Fosdick (University of Colorado, U.S.A.) and R.N. Taylor (University of Victoria, Canada), it is an important software verification technique, as it is inexpensive and dependably detects a well defined and useful class of anomalies. Work to this point has been directed at the analysis of a small but diverse assortment of errors and anomalies. The lecturers believe, however, that a larger assortment could be studied and they presented a conceptual framework for doing so in their lecture entitled "Error and Anomaly Diagnosis through Data Flow Analysis". In the lecture, "Symbolic Evaluation Methods - Implementations and Applications", L.,4. Clarke and D.J. Richardson (University of Massachusetts, U.S.A.) described symbolic evaluation, a program analysis method that concisely represents a program's computations and input domain by symbolic expressions. The general concepts were explained and three related methods of symbolic evaluation were elaborately described. Examples of all three methods were given and each method's implementation approach, applications and limitations were explained, as well as the status of current research in the area. L.J. White, E.I. Cohen and S.J. Zeil (The Ohio State University, U.S.A.) presented a testing strategy designed to detect errors in the control flow of a computer program, and gave and characterized the conditions under which this strategy is reliable. In their lecture, " A Domain Strategy for Computer Program Testing", they described a new method to decide whether an additional path should be tested when a number of paths have already been tested, and whether no additional information can be gained by testing this path. According to W.E. Howden (University of California, San Diego, U.S.A.), the basic steps in functional testing are the identification of the functions that are supposed to be computed by a program or system, the identification of the important classes of input and output data for the functions and the selection of test data. In his lecture, "Errors, Design Properties and Functional Program Tests", he made an attempt to set firm guidelines for the application of the method. T.A. Budd (University of Arizona, U.S.A.) gave an introduction to the ideas of mutation analysis, which is a method for measuring test data quality. The method was
50
EUROMICRO Reports
first described in general terms and then a specific system to implement mutation analysis of FORTRAN programs was introduced. After discussing some of the theory behind the method and overviewing related research, Budd mentioned several problems encountered in the application of the method in practice.
Tools and Environments In the first lecture of this session, R.E. Fairley (Colorado State University, U.S.A.) discussed the role that "Software Testing Tools" can play in increasing the quality of software and the productivity of programmers as the software evolves through various phases of the software life cycle. Requirements testing, design testing, source code debugging and testing, acceptance testing, and maintenance testing were all reviewed. In his lecture entitled " A Strategy for Integrating Program Testing and Analysis", L. Osterweil (University of Colorado, U.S.A.) presented a view of how various testing and analysis techniques can be integrated into a tool supported methodology. Techniques were described, compared and contrasted, and TOOLPACK, a specific integrating methodology, was dealt with. An outline of some general considerations leading to the development of an integrated automated test system for computer software was presented by E.F. Miller, Jr., J.B. Henderson and T.E. Mapp (Software Research Associates, U.S.A.). In their lecture, " A Software Test Bed: Philosophy, Implementation and Application", an example of the finished system, called ITB (Interactive Test Bed), was given.
Aspects of System Testing A. Cicu (Honeywell Information Systems Italia, Italy) gave a general definition of quality for an industrial product, and developed a definition of it specifically for computer programs, based on the analysis of purposes and structure of this type of product. The different interests, on quality attributes so defined, by user and by software engineers, were analyzed, together with the items which allow the user and the engineer to evaluate and/or to build quality in computer programs. The lecture "Software Development and Testing in an Integrated Programming Environment", presented by P. Degano and G. Levi (Universit~ di Pisa, Italy), was concerned with integrated programming environments and, more specifically, with the set of basic tools related to the testing activity. E.F. Miller, Jr. (Software Research Associates, U.S.A.) outlined some aspects of software quality testing that are of importance to contemporary industrial quality assurance. He further presented in his lecture, "Experience with Industrial Software Quality Testing", contemporary results that indicate the relative effectiveness of such quality assurance analysis. " A Systematic Approach to System and Acceptance Testing" was presented by A. Celentano (Politecnico di Milano and CNR, Italy), C. Ghezzi (CNR and Universit~t
di Padova, Italy) and F. Liguori (ARG, Italy). The lecturers provided some guidelines that can be followed to design system and acceptance testing methodically and illustrated their approach with the aid of an example. A. Haley and S. Zweben (The Ohio State University, U.S.A.) discussed the problem of integrating a previously validated module into a software system. They showed that, when doing integration testing, it is not enough to treat the previously validated module as a "black box", for otherwise certain integration errors may go undetected. Their results indicate that such errors can be detected through a retesting (at integration time) of a selected set of paths through the validated module. " A Methodology for Accurate Software Test Specification and Auditing" was presented by M. Ceriani (Etnoteam Srl, Italy), A. Cicu (Honeywell Information Systems Italia, Italy), and M. Maiocchi (Universith di Milano, Italy). The method they described allows the automatic checklist construction and maintenance and a very high cost reduction in the verification of checklist completeness and conformance to functional specifications. The Proceedings of this conference have been edited by B. Chandrasekaran and S. Radicchi and published by North-Holland Publishing Company under the title Computer Program Testing. (1981. viii+326 pages. ISBN 0-444-86292-7. Price: US $42.50/Dfl. 100).
Second Vacation School on "Advanced Techniques for Microprocessor Systems" The second school "Advanced Techniques for Microprocessor Systems" took place at the University of Manchester Institute of Technology (UMIST) from 30th March to 3rd April 1981. Its objectives were similar to those of the first school in that it set out to introduce a series of topics of interest to engineers who had overcome the initial hurdles of working with microprocessors and who wished to advance their knowledge further. It attracted fifty-nine attendees - a pleasing result in a year when Industry has been cutting back severely in its training support. The scene was set initially by Derek Roberts, Director of GEC Hirst Research Centre, who suggested that the microprocessor was not the end of the road in electronic sophistication, nor was it the ultimate weapon in the engineer's artillery and that it may be but part of a future technology. The major themes covered by the other fourteen lectures were firstly, software - its management, requirements definition and design languages, validation and a study of PASCAL as a High Level Programming Language. Secondly, aspects of "the chip" - a survey of device technologies, an update on VLSI, hardware start-
Computer Program Testing The Sogesta Summer School, held this year from June 29 - July 3, 1981 in Urbino, Italy, concentrated on Computer Program Testing. The lectures delivered at the conference provided a comprehensive, tutorial discussion of the current state of the art as well as research directions in the area of testing computer programs. They covered a wide spectrum of topics: from theoretical notions through practical issues in testing programs and large software systems to integrated environments and tools for performing a variety of tests. The lecturers were all active and recognized researchers and practitioners in the field. A report on the Summer School on Computer Program Testing is featured below.
Introductory Concepts In the first lecture of the conference, entitled "Computer Program Testing - An Introduction", S.H. Zweben (The Ohio State University, U.S.A.) presented a general overview of basic terminology and philosophies in computer program testing. Some problems associated with many of the commonly suggested strategies were also discussed. L.J. White (The Ohio State University, U.S.A.) surveyed some of the "Basic Mathematical Definitions and Results in Testing". In order to study concepts of program structure, digraphs were introduced as a potential model. White also reviewed the papers of Goodenough/Gerhart and Howden, which most researchers in the area of program testing agree comprise the basis for a theory of testing.
Aspects of Program Testing A tutorial introduction to static program checking was given by C. Ghezzi (Universit~t di Padova and Politec-
49
nico di Milano, Italy). He showed how programming languages influence the amount of checks that can be done statically on programs and also how recent programming languages are designed with the express goal of supporting extensive static checking. In his lecture, "Levels of Static Program Validation", he also introduced higher order forms of static checking provided by data flow analyzers and symbolic executors. Ghezzi finally discussed how static checking tools can be integrated in a coherent development system. Data flow analysis has been shown to be a useful tool in demonstrating the presence or absence of certain significant classes of programming errors. According to L.J. Osterweil, L.D. Fosdick (University of Colorado, U.S.A.) and R.N. Taylor (University of Victoria, Canada), it is an important software verification technique, as it is inexpensive and dependably detects a well defined and useful class of anomalies. Work to this point has been directed at the analysis of a small but diverse assortment of errors and anomalies. The lecturers believe, however, that a larger assortment could be studied and they presented a conceptual framework for doing so in their lecture entitled "Error and Anomaly Diagnosis through Data Flow Analysis". In the lecture, "Symbolic Evaluation Methods - Implementations and Applications", L.,4. Clarke and D.J. Richardson (University of Massachusetts, U.S.A.) described symbolic evaluation, a program analysis method that concisely represents a program's computations and input domain by symbolic expressions. The general concepts were explained and three related methods of symbolic evaluation were elaborately described. Examples of all three methods were given and each method's implementation approach, applications and limitations were explained, as well as the status of current research in the area. L.J. White, E.I. Cohen and S.J. Zeil (The Ohio State University, U.S.A.) presented a testing strategy designed to detect errors in the control flow of a computer program, and gave and characterized the conditions under which this strategy is reliable. In their lecture, " A Domain Strategy for Computer Program Testing", they described a new method to decide whether an additional path should be tested when a number of paths have already been tested, and whether no additional information can be gained by testing this path. According to W.E. Howden (University of California, San Diego, U.S.A.), the basic steps in functional testing are the identification of the functions that are supposed to be computed by a program or system, the identification of the important classes of input and output data for the functions and the selection of test data. In his lecture, "Errors, Design Properties and Functional Program Tests", he made an attempt to set firm guidelines for the application of the method. T.A. Budd (University of Arizona, U.S.A.) gave an introduction to the ideas of mutation analysis, which is a method for measuring test data quality. The method was
50
EUROMICRO Reports
first described in general terms and then a specific system to implement mutation analysis of FORTRAN programs was introduced. After discussing some of the theory behind the method and overviewing related research, Budd mentioned several problems encountered in the application of the method in practice.
Tools and Environments In the first lecture of this session, R.E. Fairley (Colorado State University, U.S.A.) discussed the role that "Software Testing Tools" can play in increasing the quality of software and the productivity of programmers as the software evolves through various phases of the software life cycle. Requirements testing, design testing, source code debugging and testing, acceptance testing, and maintenance testing were all reviewed. In his lecture entitled " A Strategy for Integrating Program Testing and Analysis", L. Osterweil (University of Colorado, U.S.A.) presented a view of how various testing and analysis techniques can be integrated into a tool supported methodology. Techniques were described, compared and contrasted, and TOOLPACK, a specific integrating methodology, was dealt with. An outline of some general considerations leading to the development of an integrated automated test system for computer software was presented by E.F. Miller, Jr., J.B. Henderson and T.E. Mapp (Software Research Associates, U.S.A.). In their lecture, " A Software Test Bed: Philosophy, Implementation and Application", an example of the finished system, called ITB (Interactive Test Bed), was given.
Aspects of System Testing A. Cicu (Honeywell Information Systems Italia, Italy) gave a general definition of quality for an industrial product, and developed a definition of it specifically for computer programs, based on the analysis of purposes and structure of this type of product. The different interests, on quality attributes so defined, by user and by software engineers, were analyzed, together with the items which allow the user and the engineer to evaluate and/or to build quality in computer programs. The lecture "Software Development and Testing in an Integrated Programming Environment", presented by P. Degano and G. Levi (Universit~ di Pisa, Italy), was concerned with integrated programming environments and, more specifically, with the set of basic tools related to the testing activity. E.F. Miller, Jr. (Software Research Associates, U.S.A.) outlined some aspects of software quality testing that are of importance to contemporary industrial quality assurance. He further presented in his lecture, "Experience with Industrial Software Quality Testing", contemporary results that indicate the relative effectiveness of such quality assurance analysis. " A Systematic Approach to System and Acceptance Testing" was presented by A. Celentano (Politecnico di Milano and CNR, Italy), C. Ghezzi (CNR and Universit~t
di Padova, Italy) and F. Liguori (ARG, Italy). The lecturers provided some guidelines that can be followed to design system and acceptance testing methodically and illustrated their approach with the aid of an example. A. Haley and S. Zweben (The Ohio State University, U.S.A.) discussed the problem of integrating a previously validated module into a software system. They showed that, when doing integration testing, it is not enough to treat the previously validated module as a "black box", for otherwise certain integration errors may go undetected. Their results indicate that such errors can be detected through a retesting (at integration time) of a selected set of paths through the validated module. " A Methodology for Accurate Software Test Specification and Auditing" was presented by M. Ceriani (Etnoteam Srl, Italy), A. Cicu (Honeywell Information Systems Italia, Italy), and M. Maiocchi (Universith di Milano, Italy). The method they described allows the automatic checklist construction and maintenance and a very high cost reduction in the verification of checklist completeness and conformance to functional specifications. The Proceedings of this conference have been edited by B. Chandrasekaran and S. Radicchi and published by North-Holland Publishing Company under the title Computer Program Testing. (1981. viii+326 pages. ISBN 0-444-86292-7. Price: US $42.50/Dfl. 100).
Second Vacation School on "Advanced Techniques for Microprocessor Systems" The second school "Advanced Techniques for Microprocessor Systems" took place at the University of Manchester Institute of Technology (UMIST) from 30th March to 3rd April 1981. Its objectives were similar to those of the first school in that it set out to introduce a series of topics of interest to engineers who had overcome the initial hurdles of working with microprocessors and who wished to advance their knowledge further. It attracted fifty-nine attendees - a pleasing result in a year when Industry has been cutting back severely in its training support. The scene was set initially by Derek Roberts, Director of GEC Hirst Research Centre, who suggested that the microprocessor was not the end of the road in electronic sophistication, nor was it the ultimate weapon in the engineer's artillery and that it may be but part of a future technology. The major themes covered by the other fourteen lectures were firstly, software - its management, requirements definition and design languages, validation and a study of PASCAL as a High Level Programming Language. Secondly, aspects of "the chip" - a survey of device technologies, an update on VLSI, hardware start-