- Email: [email protected]
The METKIT industrial package
Special section on M E T K I T
The METKIT Industrial Package N Ashley
The METKIT btdustrial Package provides trahzhlg materials for software managers and software enghleers on how to use measurement to understand, control and then bnprove software development, hz the paper the structure and content of the materials are briefly explahted. software engh2eerhzg measurement, trahlhzg, ESPRIT
THE REQUIREMENTS As was explained in Ashley ~, the M E T K I T project was formed because of a perceived lack of educational materials to train software de~blopers in how to use software engineering measurement. In order to develop training materials that industry would really use, M E T K I T first carried out a survey of 400 European IT departments 2 covering major sectors (e.g. finance, chemical, telecommunications, computer manufacturers) to identify their major problems and to find out answers to the following questions: 9 Who needs to be trained in using measurement? 9 What needs to be taught? 9 What media should be used? The survey revealed that there was a need to train both managers and software engineers. The main topics that needed to be covered were cost estimation, software reliability, and process assessment and improvement. Regarding the media to be used, the survey revealed that most software developers preferred the traditional approach of a teacher presenting to an audience using slides on an overhead projector. The survey also revealed that less than 5% of IT departments used measurement to improve their software development process. The reasons given included that it would not be worth the effort; that the benefits of measurement were oversold; the development team would object; and that guidance on how to set up a measurement programme was missing.
STRUCTURE OF THE INDUSTRIAL PACKAGE The Industrial Package is a set df 17 modules which form a hierarchy as shown in Figure 1. The hierarchy is based BRAMEUR Ltd, 237 High Street, Aldershot,HampshireGUi I ITJ, UK Vol 35 No 2 February 1993
upon simple-to-complex sequencing of the subject matter. Each module is a self-contained teaching unit that can be studied relatively independently, as long as students have the necessary prior knowledge and skills. A module can be tailored to suit the needs of a particular organization. For example, slides can be inserted into a module to reflect the practices (culture) of an organization or to present an example tailored to that organization. Slides that have been categorized as optional can be omitted without affecting the main objectives of the module. M E T K I T decided to develop a set of modules rather than developing complete courses. However, these modules can be put together to form different courses. In addition, a module may easily be integrated into existing courses. This provides flexibility for tho'se organizations who may not need to teach their staff a whole course but rather want to select more specific topics to teach.
Materials The modules are designed to be delivered by a teacher in a classroom, using overhead transparencies. They are of varying length and take between one and eight hours to teach. Each industrial module consists of several documents for teacher and student respectively. The materials for the teacher include those for the student. The Teacher Guide provides information for the teacher about the module and its supporting materials. It includes such items as objectives for the module and its sections, a teacher summary and didactic guidelines. The Slides With Teacher Notes contains the slides annotated with notes to assist the teacher explaining, for instance~ the objective of the slide and points to emphasize. The Overhead Projector Slides contains the full size paper version of the slides. Teacher Guidelines For Makh~g Module Interactive helps the teacher to make the module interactive with the audience. For each slide it contains questions to ask the audience, discussion points, and exercises where appropriate. The Set o f Questions & Answers lists typical questions that members of the audience may raise together with suggested answers. The Set o f Anecdotes & Quotes contains a set of anecdotes and quotes that the teacher can use to liven up a presentation.
0950--5849/93/030111-O49 1993 Butterworth-Heinemann Ltd
111
The METKIT [ndustrial Package
_•
modulelIMP Howto implement a measurement programme
moduleICSP J Casestudy- setting ,upa measurement J [ programme
moduleIISM ~1 Introductionto i ] software engineering measurement _~ Estimatingthecost of software development J Whatcan we measure / moduleIWDH
moduleIMMT Measurementas a managementtool t
J
in software
--~[
j enginee.ngho? J _ ~ Procuringsoftware systems
moduleIPRB I
Process benchmarking
moduleISMI _ ~ Soltwareengineering
measurementin industry
moduleICEM ~_~ Establishinga cost estimation measurement programme _~ moduleICES
moduleIWBS Theeasefora standardwork breakdown structure moduleIFPA Principlesof functionpoint analysis
Costestimation strategy
1~
moduleIPSS
--~
moduleIPOM ~-
Processoptimization measures moduleIUSA
moduleISMQ I Specifying& r- measuringsoftware quality
Usability assessment moduleIDFA Defectanalysis
Figure 1. The structure of the METKIT Industrial Package
The Student Notes is a self-study booklet for the students to study outside the presentation. The Glossary of Abbreviations provides a list of abbreviations used in the module.
Contents of the Industrial
Package
The Industrial Package starts off by making managers and software engineers aware of the important role measurement plays in planning, controlling and optimizing the software development process. It also trains them in how to use measurement as a management tool by providing them with the means to apply measurement using a step-by-step approach. The areas covered include: 9 Procurement of software systems from defining the i-equirements in testable terms to contract negotiations with the chosen bidder. 9 Cost estimation, covering the four main methods currently used (decomposition, expert opinion, ana!ogy and using tools that are based on an equation relating effort to factors such as size, complexity and quality) together with a detailed tutorial on Function Point Analysis. 9 Defect analysis, providing software engineers with a working knowledge of the purposes and types of defect analysis. In particular, it covers the following analysis techniques: classification by severity, classifi112
9
9 9
,,
cation by type of defect, stage defect analysis, defect density, defect detection profile. Usability, providing an introduction of the main aspects surrounding usability for software engineers. It covers a range of techniques which can be used to assess the usability of different kinds of systems at different stages of a software development process. Process assessment based on the SEI five level maturity model developed by Watts Humphrey 3. Setting up and running a measurement programme which is supported by a case study where the METKIT materials were used to implement an evolutionary measurement programme into a major financial institution.
We will now briefly describe each of the 17 Industrial Modules~ A more detailed description of each module is contained in Ashley4. The objective of Module Measurement as a Management Tool (IMMT) is to make managers aware of the need to use measurement as a management tool to help to understand, control and then improve software development. The main message of the module is that measurement is a vital tool for management in helping to improve their business. Module hltroduction to Software Enghwerhlg Measurement (IISM) gives software engineers an overview of the software measurement process. On Information and Software Technology
N AStlLEY
completion ol~ the module, students should understand that measurement can prevent many of the problems found in software development. Module What Can We Measure In Software Enghleerhlg & How? (IWDH) gives software engineers a background to software measurement to enable them to understand and take part in a software measurement programme. Module Procurhlg Software Systems (IPSS) develops a strategy to support the procurement o f software systems which uses measurement to help ensure the customer gets what he wants. It covers the following activities: determining the requirements; identifying and selecting bidders; producing the Invitation To Tender; requesting and evaluating proposals; and finally conducting contract negotiations with the preferred bidder. Module Software Enghleerfllg Measurement hi hldustry (ISMI) provides the audience with an appreciation of the current industrial practices in software engineering measurement and their relationship to internationally accepted standards. It details the historical development of research in the area, the level of uptake in industry worldwide, and the support in terms of standards and tools. ,, Module How To hnplentent a Measurement Programme (lIMP) provides managers with a practical step-by-step guide on how to introduce and run a measurement programme using a phased implementation. It covers such issues as obtaining management commitment, training staff and how to benefit from the programme. Module Estimathlg the Cost of Software Developnlent (IECD) provides managers with an introduction to cost estimation. It describes what is meant by a cost estimate, the common methods of cost estimation, and the main aspects o f a cost estimation strategy. The module demonstrates that it takes more than an educated guess to generate an estimate from the requirements that is correct to within 15% of the actual case in 80% of the cases. The objective of Module Process Benchmarking (IPRB) is to provide senior managers with a method to measure, compare, improve and then optimize their software engineering process. The module shows that quantitative measures can be applied to a development process to assist organizations in applying industry 'best practices'. Module Specifying & Measuring Software Quali O, (ISMQ) helps users and specifiers to state the quality attributes o f a software system in testable terms. This will help to ensure that customers and developers are in agreement on what is'required and designers know the constraints. In addition, it will facilitate the task of acceptance testing. Module Case Stud),--Setthlg up a Measurement Programme (ICSP)describes how a measurement programme was set up for a major financial institution. It provides a step-by-step guide on setting up a phased implementation using M E T K I T materials. It Vol 35 No 2 February 1993
addresses the type of infrastructure required, the training the staff involved in the measurement programme needs, and how to assess the success of the measurement programme. Module Establishhlg a Cost Esthnation Measurement Programme (ICEM) provides managers with a practical guide to the processes necessary to establish and maintain a cost estimation programme. It uses the G Q M approach 5 to identify data that can be used as a predictor of cost and covers the steps needed to set in place the collection, validation and use of this data. Module Cost Estintation Strategy (ICES) shows why an estimating strategy is needed for deriving a software cost estimate and provides an example of one such strategy. The module also covers what is required of an estimate and procedures for producing an initial estimate, a detailed estimate and how to perform risk analysis. Module Process Opthnization Measures (IPOM) provides comparative measures to evaluate and improve the effectiveness of a software development process. The module is based on the principle that process improvement is a continuous activity. Module Usability Assessment (IIUSA) provides an introduction to the main aspects surrounding.usability for software engineers. It covers a range of techniques which can be used to assess the usability of different kinds of systems at different stages of a software development process. The objective of Module Defect Analysis (IDFA) is to provide software engineers with a step-by-step guide on how to start using defect analysis as a tool to improve software development. For example, it shows how to use defect analysis to plan the effort for correcting defects found during the test phase of development, to identify the most effective test techniques for a particular type of defect, and to estimate the defect density of a system on release to the customer. The objective of Module The Case For a Standard Work Breakdown Structure (IIVBS) is to demonstrate the benefits of using a standard WBS (Work Breakdown Structure). It provides practical guidance as to how a WBS can be used by projects and its role in the collection of software measures. It then explains how to construct a WBS alild provides an example of a standard WBS. Finally, it shows, using a step-by-step approach, how an organization can tailor the standard WBS to its own needs. Module Functi~t Pohtt Analysis (IFPA) provides a step-by-step guide to measuring the size o f a software system using Function Point Analysis. Different courses can be constructed by combining selections o f modules. An industrial course on software quality, for instance, could be formed by modules I M M T (Measurement as a Management Tool), I W H D
(What Can We Measure 111 Software Engbleerfllg & How?), ISMQ (Specifyhtg & Measurhlg Software Quality), and IDFA (Defect Analysis). This selection relates to a path in the module hierarchy shown in Figure 1. 113
Tile METKIT hldustrial Package EXPERIENCE MATERIALS
IN USING
THE
The M E T K I T industrial modules have been used by a major U K bank, the Civil Aviation Authority and a major telecommunications company to implement measurement programmes. The response of all three organizations was very positive, reporting that the materials helped to successfully introduce software engineering measurement to their staff and to set up and run their measurement programmes. To give the reader an idea of the success of the M E T K I T Industrial Package we have the following statistics that were compiled up to 31 October 1992:267 M E T K I T industrial modules had been sold to 22 organizations in Europe, USA, Japan and Australia, and over 4000 software managers and software engineers have attended at least one M E T K I T module.
Availability The M E T K I T industrial modules are now available to be purchased either individually or as a complete
114
package. Those interested can obtain more details from the author at the address given at the start of the paper.
ACKNOWLEDGEMENT The work described in this paper has been supported by the Commission o f the European Communities under their ESPRIT II programme.
REFERENCES 1 Ashley, N 'Introduction to METKIT' Information and Software Technology (1993) (This issue) 2 Drummond Tyler, E hldustrialneeds and practices METKIT Consortium, (1990) 3 Humphrey, W S Managhlg the software process AddisonWesley (1989) 4 Ashley, N Description of the METKIT hzdustrial modules METKIT Consortium (1992) 5 BasUi, V 'Software development: a paradigm for the future' Proc. 13th Ann. bzt. Computer & Software Applications Col~ (1989)
Information and Software Technology
The METKIT Industrial Package N Ashley
The METKIT btdustrial Package provides trahzhlg materials for software managers and software enghleers on how to use measurement to understand, control and then bnprove software development, hz the paper the structure and content of the materials are briefly explahted. software engh2eerhzg measurement, trahlhzg, ESPRIT
THE REQUIREMENTS As was explained in Ashley ~, the M E T K I T project was formed because of a perceived lack of educational materials to train software de~blopers in how to use software engineering measurement. In order to develop training materials that industry would really use, M E T K I T first carried out a survey of 400 European IT departments 2 covering major sectors (e.g. finance, chemical, telecommunications, computer manufacturers) to identify their major problems and to find out answers to the following questions: 9 Who needs to be trained in using measurement? 9 What needs to be taught? 9 What media should be used? The survey revealed that there was a need to train both managers and software engineers. The main topics that needed to be covered were cost estimation, software reliability, and process assessment and improvement. Regarding the media to be used, the survey revealed that most software developers preferred the traditional approach of a teacher presenting to an audience using slides on an overhead projector. The survey also revealed that less than 5% of IT departments used measurement to improve their software development process. The reasons given included that it would not be worth the effort; that the benefits of measurement were oversold; the development team would object; and that guidance on how to set up a measurement programme was missing.
STRUCTURE OF THE INDUSTRIAL PACKAGE The Industrial Package is a set df 17 modules which form a hierarchy as shown in Figure 1. The hierarchy is based BRAMEUR Ltd, 237 High Street, Aldershot,HampshireGUi I ITJ, UK Vol 35 No 2 February 1993
upon simple-to-complex sequencing of the subject matter. Each module is a self-contained teaching unit that can be studied relatively independently, as long as students have the necessary prior knowledge and skills. A module can be tailored to suit the needs of a particular organization. For example, slides can be inserted into a module to reflect the practices (culture) of an organization or to present an example tailored to that organization. Slides that have been categorized as optional can be omitted without affecting the main objectives of the module. M E T K I T decided to develop a set of modules rather than developing complete courses. However, these modules can be put together to form different courses. In addition, a module may easily be integrated into existing courses. This provides flexibility for tho'se organizations who may not need to teach their staff a whole course but rather want to select more specific topics to teach.
Materials The modules are designed to be delivered by a teacher in a classroom, using overhead transparencies. They are of varying length and take between one and eight hours to teach. Each industrial module consists of several documents for teacher and student respectively. The materials for the teacher include those for the student. The Teacher Guide provides information for the teacher about the module and its supporting materials. It includes such items as objectives for the module and its sections, a teacher summary and didactic guidelines. The Slides With Teacher Notes contains the slides annotated with notes to assist the teacher explaining, for instance~ the objective of the slide and points to emphasize. The Overhead Projector Slides contains the full size paper version of the slides. Teacher Guidelines For Makh~g Module Interactive helps the teacher to make the module interactive with the audience. For each slide it contains questions to ask the audience, discussion points, and exercises where appropriate. The Set o f Questions & Answers lists typical questions that members of the audience may raise together with suggested answers. The Set o f Anecdotes & Quotes contains a set of anecdotes and quotes that the teacher can use to liven up a presentation.
0950--5849/93/030111-O49 1993 Butterworth-Heinemann Ltd
111
The METKIT [ndustrial Package
_•
modulelIMP Howto implement a measurement programme
moduleICSP J Casestudy- setting ,upa measurement J [ programme
moduleIISM ~1 Introductionto i ] software engineering measurement _~ Estimatingthecost of software development J Whatcan we measure / moduleIWDH
moduleIMMT Measurementas a managementtool t
J
in software
--~[
j enginee.ngho? J _ ~ Procuringsoftware systems
moduleIPRB I
Process benchmarking
moduleISMI _ ~ Soltwareengineering
measurementin industry
moduleICEM ~_~ Establishinga cost estimation measurement programme _~ moduleICES
moduleIWBS Theeasefora standardwork breakdown structure moduleIFPA Principlesof functionpoint analysis
Costestimation strategy
1~
moduleIPSS
--~
moduleIPOM ~-
Processoptimization measures moduleIUSA
moduleISMQ I Specifying& r- measuringsoftware quality
Usability assessment moduleIDFA Defectanalysis
Figure 1. The structure of the METKIT Industrial Package
The Student Notes is a self-study booklet for the students to study outside the presentation. The Glossary of Abbreviations provides a list of abbreviations used in the module.
Contents of the Industrial
Package
The Industrial Package starts off by making managers and software engineers aware of the important role measurement plays in planning, controlling and optimizing the software development process. It also trains them in how to use measurement as a management tool by providing them with the means to apply measurement using a step-by-step approach. The areas covered include: 9 Procurement of software systems from defining the i-equirements in testable terms to contract negotiations with the chosen bidder. 9 Cost estimation, covering the four main methods currently used (decomposition, expert opinion, ana!ogy and using tools that are based on an equation relating effort to factors such as size, complexity and quality) together with a detailed tutorial on Function Point Analysis. 9 Defect analysis, providing software engineers with a working knowledge of the purposes and types of defect analysis. In particular, it covers the following analysis techniques: classification by severity, classifi112
9
9 9
,,
cation by type of defect, stage defect analysis, defect density, defect detection profile. Usability, providing an introduction of the main aspects surrounding usability for software engineers. It covers a range of techniques which can be used to assess the usability of different kinds of systems at different stages of a software development process. Process assessment based on the SEI five level maturity model developed by Watts Humphrey 3. Setting up and running a measurement programme which is supported by a case study where the METKIT materials were used to implement an evolutionary measurement programme into a major financial institution.
We will now briefly describe each of the 17 Industrial Modules~ A more detailed description of each module is contained in Ashley4. The objective of Module Measurement as a Management Tool (IMMT) is to make managers aware of the need to use measurement as a management tool to help to understand, control and then improve software development. The main message of the module is that measurement is a vital tool for management in helping to improve their business. Module hltroduction to Software Enghwerhlg Measurement (IISM) gives software engineers an overview of the software measurement process. On Information and Software Technology
N AStlLEY
completion ol~ the module, students should understand that measurement can prevent many of the problems found in software development. Module What Can We Measure In Software Enghleerhlg & How? (IWDH) gives software engineers a background to software measurement to enable them to understand and take part in a software measurement programme. Module Procurhlg Software Systems (IPSS) develops a strategy to support the procurement o f software systems which uses measurement to help ensure the customer gets what he wants. It covers the following activities: determining the requirements; identifying and selecting bidders; producing the Invitation To Tender; requesting and evaluating proposals; and finally conducting contract negotiations with the preferred bidder. Module Software Enghleerfllg Measurement hi hldustry (ISMI) provides the audience with an appreciation of the current industrial practices in software engineering measurement and their relationship to internationally accepted standards. It details the historical development of research in the area, the level of uptake in industry worldwide, and the support in terms of standards and tools. ,, Module How To hnplentent a Measurement Programme (lIMP) provides managers with a practical step-by-step guide on how to introduce and run a measurement programme using a phased implementation. It covers such issues as obtaining management commitment, training staff and how to benefit from the programme. Module Estimathlg the Cost of Software Developnlent (IECD) provides managers with an introduction to cost estimation. It describes what is meant by a cost estimate, the common methods of cost estimation, and the main aspects o f a cost estimation strategy. The module demonstrates that it takes more than an educated guess to generate an estimate from the requirements that is correct to within 15% of the actual case in 80% of the cases. The objective of Module Process Benchmarking (IPRB) is to provide senior managers with a method to measure, compare, improve and then optimize their software engineering process. The module shows that quantitative measures can be applied to a development process to assist organizations in applying industry 'best practices'. Module Specifying & Measuring Software Quali O, (ISMQ) helps users and specifiers to state the quality attributes o f a software system in testable terms. This will help to ensure that customers and developers are in agreement on what is'required and designers know the constraints. In addition, it will facilitate the task of acceptance testing. Module Case Stud),--Setthlg up a Measurement Programme (ICSP)describes how a measurement programme was set up for a major financial institution. It provides a step-by-step guide on setting up a phased implementation using M E T K I T materials. It Vol 35 No 2 February 1993
addresses the type of infrastructure required, the training the staff involved in the measurement programme needs, and how to assess the success of the measurement programme. Module Establishhlg a Cost Esthnation Measurement Programme (ICEM) provides managers with a practical guide to the processes necessary to establish and maintain a cost estimation programme. It uses the G Q M approach 5 to identify data that can be used as a predictor of cost and covers the steps needed to set in place the collection, validation and use of this data. Module Cost Estintation Strategy (ICES) shows why an estimating strategy is needed for deriving a software cost estimate and provides an example of one such strategy. The module also covers what is required of an estimate and procedures for producing an initial estimate, a detailed estimate and how to perform risk analysis. Module Process Opthnization Measures (IPOM) provides comparative measures to evaluate and improve the effectiveness of a software development process. The module is based on the principle that process improvement is a continuous activity. Module Usability Assessment (IIUSA) provides an introduction to the main aspects surrounding.usability for software engineers. It covers a range of techniques which can be used to assess the usability of different kinds of systems at different stages of a software development process. The objective of Module Defect Analysis (IDFA) is to provide software engineers with a step-by-step guide on how to start using defect analysis as a tool to improve software development. For example, it shows how to use defect analysis to plan the effort for correcting defects found during the test phase of development, to identify the most effective test techniques for a particular type of defect, and to estimate the defect density of a system on release to the customer. The objective of Module The Case For a Standard Work Breakdown Structure (IIVBS) is to demonstrate the benefits of using a standard WBS (Work Breakdown Structure). It provides practical guidance as to how a WBS can be used by projects and its role in the collection of software measures. It then explains how to construct a WBS alild provides an example of a standard WBS. Finally, it shows, using a step-by-step approach, how an organization can tailor the standard WBS to its own needs. Module Functi~t Pohtt Analysis (IFPA) provides a step-by-step guide to measuring the size o f a software system using Function Point Analysis. Different courses can be constructed by combining selections o f modules. An industrial course on software quality, for instance, could be formed by modules I M M T (Measurement as a Management Tool), I W H D
(What Can We Measure 111 Software Engbleerfllg & How?), ISMQ (Specifyhtg & Measurhlg Software Quality), and IDFA (Defect Analysis). This selection relates to a path in the module hierarchy shown in Figure 1. 113
Tile METKIT hldustrial Package EXPERIENCE MATERIALS
IN USING
THE
The M E T K I T industrial modules have been used by a major U K bank, the Civil Aviation Authority and a major telecommunications company to implement measurement programmes. The response of all three organizations was very positive, reporting that the materials helped to successfully introduce software engineering measurement to their staff and to set up and run their measurement programmes. To give the reader an idea of the success of the M E T K I T Industrial Package we have the following statistics that were compiled up to 31 October 1992:267 M E T K I T industrial modules had been sold to 22 organizations in Europe, USA, Japan and Australia, and over 4000 software managers and software engineers have attended at least one M E T K I T module.
Availability The M E T K I T industrial modules are now available to be purchased either individually or as a complete
114
package. Those interested can obtain more details from the author at the address given at the start of the paper.
ACKNOWLEDGEMENT The work described in this paper has been supported by the Commission o f the European Communities under their ESPRIT II programme.
REFERENCES 1 Ashley, N 'Introduction to METKIT' Information and Software Technology (1993) (This issue) 2 Drummond Tyler, E hldustrialneeds and practices METKIT Consortium, (1990) 3 Humphrey, W S Managhlg the software process AddisonWesley (1989) 4 Ashley, N Description of the METKIT hzdustrial modules METKIT Consortium (1992) 5 BasUi, V 'Software development: a paradigm for the future' Proc. 13th Ann. bzt. Computer & Software Applications Col~ (1989)
Information and Software Technology