- Email: [email protected]
Requirements engineering by prototyping: experiences in development of estimating system
Requirementsengineeringby pr0t0typing: experiences in developmentof estimating system M A Stephens and P E Bates
Rapid software prototyping is" used in a complex costing and estimating project within the metal-finishing industry. Structured design methods proved unsuccessful in defining the nature of the problem. Interface and functional prototypes are used to gain a better understanding and to help define the boundaries of the system. The importance of user evaluation and feedback is acknowledged as a valuable means of increasing accuracy and improving validation. The investigation reinforces much of the literature but raises further questions of prototyping management, maintenance, and installation. case study, system design, prototyping, rapid prototyping, requirements engineering
A specialist metal-plating company identified the need for a more rigorous method of pricing and estimating. Their established pricing and estimating techniques, and hence the quotations produced, relied heavily on the experience of particular specialists. These individuals subjectively interpreted a pricing policy formulated by the company. Inconsistencies in pricing occurred due to different interpretations of the policy and complexity of the processes involved. Complexity arises because of the diversity of components to be finished and the large number of subprocesses carried out (several hundred possible combinations). Pricing and estimating are further complicated by constant changes in trading conditions, particularly metal price fluctuations. The open-ended brief was reduce the incidence of erroneous and inconsistent estimates. Current practices in pricing and estimating were to be analysed, the cause of problems reported, and an automated system to streamline processing specified. It was discovered that such open-ended briefs are indicative of systems that are particularly suited to prototyping. Initially the problem was approached using a traditional structured design method. This was found to be difficult because the users were uncertain of details of the present and proposed systems. Hence rapid software prototyping was used to formulate (engineer) system
School of Computing and Information Technology,Wolverhampton Polytechnic,Wulfruna Street, WolverhamptonWVI ILY, UK
vol 32 no 4 may 1990
requirements by stimulating a joint learning process between users and developers.
Rapid software prototyping Prototypes may be built for a variety of purposes. These include identifying, learning about, and clarifying user requirements (requirements engineering). They involve the building of working 'mock-up' versions of parts of, or the whole, system. These are demonstrated to users with the intention of maximizing product quality; the assumption being that it is easier to focus on a working version rather than on paper documents. For this to be successful, developers must encourage users to be constructively critical. The literature suggests prototyping as a technique that promotes better user/developer communication~.L While it is difficult to formulate a universal definition of prototypes, they do exhibit some common characteristics in that they: • involve a high degree of user evaluation, which substantially affects requirements, specification, or design • are analogous to experiments built to test hypotheses • stimulate a joint learning process for users and developers There are many approaches to prototyping. Hekmatpour and Ince 3 and Ince 4 classify prototyping into three categories: • throw-away • evolutionary • incremental Throw-away prototyping is often used for requirements identification and clarification. As the name suggests the prototype built is 'discarded' once the developers and users have evaluated it. It is not intended to be used as a direct implementation but may be kept as a reference point. With incremental prototyping, a complete system design is specified, subtasks identified, and prototypes built, evaluated, and modified according to user feed-
0950-5849/90/040253~)5 © 1990 Butterworth-Heinemann Ltd
253
back. The overall design, however, remains unaltered. This is really incremental development with an emphasis on frequent user feedback about the working parts. Evolutionary prototyping allows a system to evolve, according to user feedback, so that dynamic change is allowed. At the onset of prototyping no system specification need exist; the prototyping process produces the specification. An alternative classification can be used dependent on the purpose of the prototype. If it is concerned with system function it is a functional prototype. If it concentrates on user interface it is an interface prototype. However, interface prototyping can also reveal much about function. This paper outlines the use of prototypes that conform to both of the above classifications. Evolutionary and throw-away prototypes are used for both interface design and the discovery of functional requirements. This work has identified a close correspondence between interface and function, thus reinforcing the view that the major purpose of interface prototyping is to gain insight into users' functional requirements. Management of software projects tends to depend on a traditional life-cycle approach 5. The management of prototyping, however, is not a well researched or documented topic. Particular issues are: • developing aims and objectives (type of prototyping and what is to be learnt) • selection of users • budgeting (both developer and user time) • when to start prototyping • when to stop prototyping (limiting user change and enthusiasm!) • how to document prototyping and record user changes Mayhew e t al. 6 discuss these issues and propose strategies for control, some of which have been adapted by this project.
Because the necessary information could not be elicited from the estimators, computerization of current manual procedures was not feasible. The analysis did, however, provide a partial view of current procedures. It increased both the users' and developers' understanding, and in particular it identified the main determinants for estimating. It was also evident that because the estimators produced inconsistent estimates any computerized system that mimicked their reasoning would itself be inconsistent and hence fail to address one of the major problems. This work formed a basis for discussion with management and helped identify the main problems: • Quotations were inconsistent (most significant). • Cross-referencing of previous quotations was too difficult, because of insufficient indexing and no standard layout. • Manual quotations were expensive to produce and modify. Furthermore, the relationships between costs and prices remained unclear and no rules could be identified for the transformation of determinants into quotations. Following these discussions, management, who were now more aware of the extent of the problems, wished to reformulate the requirements of the proposed system. However, they were not fully aware of what they actually wanted. This phenomenon is recognised by Yourdon 7, who states that one of the good candidates for prototyping is where: 'The user is unable to articulate (or "prespecify") his or her requirements in any form and can only determine the requirements through a process of trial and error.'
TRADITIONAL APPROACH (USE OF STRUCTURED ANALYSIS)
PROTOTYPING APPROACH
During the early stages of the project it was anticipated that analysis of the existing pricing procedures and practices (including documentation) using structured analysis7 would identify the areas causing difficulty. Hence a series of interviews with estimating staff and management were conducted. During interviews with estimators it was found that some subjects had difficulty in explaining the factors they were considering when calculating a price; attempts to decompose parts of the system often ended with a series of examples of individual circumstances cited, with no easily identifiable underlying procedures. These difficulties may have been due to an unwillingness to reveal the procedures used or an inability to verbalize them, or the questions asked may have been inappropriate or misleading. Whatever the reason, it meant that there was inadequate information for the creation of a complete specification that modelled the existing system.
At this stage it was agreed to build prototypes to solve some of the identified problems. As prototyping was relatively novel to the authors, planning and control were limited. As stated above, however, the structured analysis work helped identify the parts of the system to be prototyped and the aims and objectives of the exercise. Additionally, appropriate users were identified in advance of the prototyping sessions. Evolutionary interface prototyping was selected to help define the boundaries of the specification and to provide easy access to quotations. This was also intended to improve user/developer understanding of the pricing process. The prototypes were designed to be used as the interfaces of the final complete system. On completion of the interface prototypes it was hoped that the further information gained from them would enable functional prototypes of the pricing process to be built.
254
information and software technology
Interface prototyping An evolutionary approach was adopted with interfaces developed on the target machine. This gave the company a working system for quotation production, modification, and retrieval and hence a faster payback on some of their investment. It was intended to try to achieve several objectives: • To produce user-acceptable input screens. • To clarify the information on quotations and identify the key fields used in historical cross-referencing. • To identify management information reports. • To consolidate understanding of determinants. • To help identify the boundaries of the system. This last aim requires explanation. The users were not clear about the level of expertise that the system would exhibit in automating pricing. This had not been finalised. Prototypes were designed and implemented on the company's IBM minicomputer using RPG2 and the screen painter facilities available. Interface prototyping was conducted informally, with estimating staff and company management deciding how the system evolved. User involvement maintained and stimulated interest in the project. Screen designs and quotation and report layouts were changed directly as a result of criticism from both estimators and management. The approach brought a usable system to completion within a reasonable time-scale, undergoing the first trials within three months of the exercise starting. This would have been faster if the developers had been more familiar with RPG2 at the outset. The use of particular tools can require a time investment at the beginning of the project. This should be appreciated before prototyping takes place. The difficulties of interface prototyping and the need for designers to be both skilled programmers as well as have expertise in human factors is a current research area 8. The 'interim' system built as a result of the interface prototyping was given a trial of several months in the factory. This system is now being used by the estimators for the production of all quotations, facilitating a more streamlined operation. The data captured will be used in future research for comparison with prices produced by prototypes. There is no doubt that the function of the final system was developed from an exercise that was only considered to be interface prototyping. By eliciting the requirements from users, using only screen designs and report layouts, the final system design was developed and implemented. This type of exercise raises the question of where the boundary between the two types of prototyping should be drawn. Post-delivery modifications to the system have been few (mainly cosmetic changes to some of the printed output), and it has met with enthusiastic user response despite the lack of a well defined specification. Compared with the authors' experience with other systems, this was
vol 32 no 4 may 1990
a positive outcome, tending to reinforce the claims of prototyping in the literature. However the exercise raised as many questions about prototyping as it answered. The developers felt that the users were tending to control the process rather than the developers. The need for a strategy to handle differences of opinion between users was identified, though fortunately these were few and minor this time around. It was decided to research further into, and give further thought to, the management of prototyping before the next exercise. The relationship between prototypes, documentation, and specification was not clear. Ince 4 suggests a prototyping plan where: ~The prototyping objectives, the prototyping model adopted, a description of how prototyping is to be achieved, and the use that is to be made of the prototype, should all be written into a document known as the prototyping plan.' Analysis of the outcome of interface prototyping and a review with senior management led to a change of system boundary. The project was no longer trying to produce a fully automated pricing system. Instead, an automated costing system that would provide a consistent base for manual pricing was adopted. Interface prototyping enabled both developers and users to get a better understanding of the type of problem being investigated. This has been noted by other authors. For example, Buckley e t al. 9 go further than this and put forward the idea that the major purpose of interface prototyping is to gain insight into functional requirements: 'While a good user interface is desirable, the real value in using the user interface in this manner is to enable the end-users to talk about what they want the system to do.' Three components were identified as essential to a consistent costing and pricing process: • For costing, identify the 'route' through the factory; and apply an up-to-date cost on a 'route'. • To extend into the area of pricing, manually adjust a cost to yield a price. As a result of the analysis of interface prototyping it was decided to proceed with further prototyping. This pointed to the need for functional prototyping.
Functional prototyping The 'route' through the factory is particularly significant because costs associated with apparently similar components can vary by a factor of up to 10. Provided a subsystem could be developed to assist in the accurate prediction of 'route' and another subsystem to maintain the cost of routes, then the cost-estimation problem could be solved. The adjustment of costs to give prices is complex and dynamic, depending on market forces. In
255
some cases the actual adjustment is applied by a member of the board. It was felt that it would not be practical to try to extend automation into this area. Functional prototypes for the prediction of route have been built with the aim of discovering the relationship between determinants of process and route. Due to the complexity of this problem, current work has been restricted to silver finishes to evaluate feasibility. The prototypes were constructed using a 'throw-away' approach, the intention being to enhance the system already installed on the IBM minisystem once the requirements could be specified. The production staff in the factory were identified as the target user group to maximize the validity of predicted routes. These staff decide the actual route that a job takes when it arrives at the factory gate. Prototypes to tackle the route problem were built using a commercially available package. The expertsystem shell Crystal was chosen for this task. It is easy to learn, and once some basic system knowledge is acquired, prototypes can be quickly built and easily modified. The aim was not to construct an expert system per se, but to investigate further the decisions taken when routing a job through the factory. With the hindsight o f the earlier prototyping exercise, the developers wished to maximize the control and monitoring of the exercise. This was to be improved in three ways: • Extensive records were to be kept of prototyping sessions. • Some prototyping sessions were to be attended by more than one developer to enhance critical analysis. • A strategy was to be developed to control change. The latter was inspired by the control strategy advocated by Mayhew et al. 6. Using this method, all changes to prototypes suggested by users are recorded and classified into three categories (cosmetic, local, and global) according to their impact on the system. It is recommended that both cosmetic and local changes are made immediately, with the latter being subject to later scrutiny. Global changes that have an impact beyond the parts of the system currently under review are recorded, but change is delayed until all the implications have been researched. Hence the developers retain control over changes that have wide-reaching effects, while still leaving users with the impression that it is their system. The prototyping sessions did not produce any global changes (this was expected given that the main boundaries had now been determined). Cosmetic changes were few, again not surprising as it was throw-away functional prototyping, rather than interface prototyping, that was being carried out. It was not possible to make functional changes during sessions; alterations to one rule in an expert system can have a ripple effect on other rules. The strategy was to confirm the change with the user and return with an updated system quickly (normally a few days). Otherwise the suggestions made by Mayhew were found to be most pertinent. The results of the exercise can be provisionally des-
256
Table I. Summary of time spent in prototype sessions (silver department only; functional prototyping) Meeting no
Time (h)
Introductory meeting Pre-prototype meeting 1 2 Review meeting 3 4 5 6 Totals (excluding reviews) Changes per hour
Changes
2 1.5
18 11
1.25 1 1.3 0.63 ~8 7
8 6 7 5 55
Note: Times exclude introductory and review meetings where changes were not suggested.
cribed as a success. After about six sessions with users, plus three review sessions and non-prototype meetings, the prototype was accepted. Table 1 summarizes the time spent and the changes suggested. Management are impressed with the outcome to the extent that they have requested that it be developed into a usable system. They would like it to be used as a sales support system as well as being used by the estimating staff. This is in spite of the fact that it was not intended to be an evolutionary prototype. The developers paid particular attention to educating the users of the dangers of adopting rapidly built throw-away prototypes as fully implemented systems. This is leading to problems that were not anticipated. Because the prototyping tool was not chosen with the intention of building a finished system, installation and maintenance were not considered relevant at the time. It is difficult to deny access to a prototype, even though it was never intended for everyday use.
FUTURE
WORK
There are three areas that require further work as a result of the investigation carried out so far: • Can the methods used in developing the limited domain system be extended so that the whole problem can be tackled? • Will the use of prototype systems as installed products cause maintenance problems? • Can the control of prototyping be enhanced by the possible development of a metric? Currently, the rule-based system uses well over 150 rules for a production area, representing about 20 per cent of factory finishes. Research is needed to discover whether such prototypes and the associated management and control strategies are suitable for extending the approach from the particular to the general. Additionally, documentation would be required for the guidance of developers with less experience of the problem. If the decision is made to remain with the rule-based system, the use of rapid prototyping may lead to future maintenance problems. Diaper ~° states:
information and software technology
'such expert systems are likely to be expensive and difficult, if not impossible, to maintain and up-date.' Yourdon 7 also draws attention to this problem and writes: 'There is a significant danger that either the user or the development team may try to turn the prototype into a production system. This usually turns out to be a disaster ...' Further work on the installation, maintenance, and restructuring or re-specifying of the installed system will thus be required. This is important when it is considered that the rule-based approach was not intended to be installed as part of a finished system. It has been suggested 6 that the formulation of a metric linking number and type of change during a set period could help determine when to cease prototyping. Further analysis of the results will be necessary before comment is possible on this point.
DISCUSSION The paper has attempted to show how interface and functional prototyping have contributed to the success, so far, of the development of a costing system. The experiences of the authors back up research done by other workers and suggest that this type of problem, with difficult open-ended initial requirements, can be developed successfully by planned user participation centred on prototypes. The approach provided a good vehicle for the education of both the developers and users. It forced the designer to take account of the users' wishes at every point. A designer must be prepared to encourage users by considering all comments and criticisms, taking care not to be defensive or negative. This enhanced communication between developers and users was found in both interface and functional prototyping. In fact, one perceived advantage of prototyping is the focus on the problem and communication with users at the expense of the technology. The user-interface prototyping allowed a payback early on in the project by evolution of a usable system, albeit with limited function. This was despite the extra time required by the developers to learn about the software tools and how to conduct prototype evaluation. For prototyping to be adopted, it is necessary for project managers to recognise the potential payback and also be prepared to budget for prototyping activities. A dynamic process such as prototyping requires careful planning and control. The developers found it highly iterative and difficult to estimate the time and effort involved in the different activities. Identifying milestones was also found to be a challenging problem. This reinforces the comments 6 that the management of prototyping requires more research, development, and publication. The classification of approaches was found to be useful in the planning of the two prototyping exercises. It was a definite aid to think in terms of throw-away versus
vol 32 no 4 may 1990
evolutionary, or interface versus function. As with all theory, however, the practice is not so clear. For example, interface prototyping provided communication about functional requirements as well as inspiring major user management decisions. Another example was the strong user reluctance to throw away a prototype that was designed to be discarded. The users wish to change the prototype to evolutionary for good reason. The developers could accept this provided doubts about maintainability can be resolved.
CONCLUSION The authors are encouraged by the results of the use of prototyping in the project. It was felt that progress was more effective than using a conventional structured design method where user and developer perceptions were uncertain. More extensive testing is planned for functional prototypes currently under construction. The route problem is being investigated. The prototypes built have been enthusiastically received. Despite the use of control strategies, however, additional questions have arisen about the installation and maintenance of the systems developed so far. It is felt that more work is required to develop this type of prototyping into a controllable design methodology.
ACKNOWLEDGEMENTS The authors thank the staff at Frost Electroplating for their collaboration in this project and the personnel at JBS Computers for their advice.
REFERENCES 1 Cerveny, R P, Garrity, E J and Sanders G L 'The application of prototyping to systems development: a rationale and model' J. Manage. Inf. Syst. Vol 3 No 2 (1986) 2 Parbst, F 'Experience with prototyping in an IBM based installation' in Budde, R et al. Approaches to prototyping Springer-Verlag, Berlin, FRG (1983) 3 Hekmatpour, S and Ince, D C 'Rapid software prototyping' Technical report 86/4 Open University, Milton Keynes, UK (1986) 4 Inee, D C Software development. Fashioning the baroque Oxford University Press, Oxford, UK (1988) 5 Lehman, M M and Belady, L A Program ew~lution. Process of software change Academic Press, New York, NY, USA (1985) 6 Mayhew, P J, Worsley, a and Dearnley, P A 'Control of software prototyping process: change classification approach' Inf. Soft. Technol. Vol 31 No 2 (March 1989) pp 59 66 7 Yourdon, E Modern structured analysis Prentice Hall, Englewood Cliffs, NJ, USA (1989) 8 Gray, P D, Kilgour, A C and Wood, C A 'Dynamic reconfigurability for fast prototyping of user interfaces' SOL/?.Eng. J. Vol 3 No 6 (November 1988) 9 Buekley, M, Candy, L and Edmonds, E A 'Determining requirements and prototyping the user interface module. Methodology for expert system specification' in Human and organisational issues of expert systems Conference proceedings, Stratford, UK, May 1988 10 Diaper, D 'The promise of POMESS: a people oriented methodology for expert system specification' in Human and organisational issues of expert systems Conference proceedings, Stratford, UK, May 1988
257
Rapid software prototyping is" used in a complex costing and estimating project within the metal-finishing industry. Structured design methods proved unsuccessful in defining the nature of the problem. Interface and functional prototypes are used to gain a better understanding and to help define the boundaries of the system. The importance of user evaluation and feedback is acknowledged as a valuable means of increasing accuracy and improving validation. The investigation reinforces much of the literature but raises further questions of prototyping management, maintenance, and installation. case study, system design, prototyping, rapid prototyping, requirements engineering
A specialist metal-plating company identified the need for a more rigorous method of pricing and estimating. Their established pricing and estimating techniques, and hence the quotations produced, relied heavily on the experience of particular specialists. These individuals subjectively interpreted a pricing policy formulated by the company. Inconsistencies in pricing occurred due to different interpretations of the policy and complexity of the processes involved. Complexity arises because of the diversity of components to be finished and the large number of subprocesses carried out (several hundred possible combinations). Pricing and estimating are further complicated by constant changes in trading conditions, particularly metal price fluctuations. The open-ended brief was reduce the incidence of erroneous and inconsistent estimates. Current practices in pricing and estimating were to be analysed, the cause of problems reported, and an automated system to streamline processing specified. It was discovered that such open-ended briefs are indicative of systems that are particularly suited to prototyping. Initially the problem was approached using a traditional structured design method. This was found to be difficult because the users were uncertain of details of the present and proposed systems. Hence rapid software prototyping was used to formulate (engineer) system
School of Computing and Information Technology,Wolverhampton Polytechnic,Wulfruna Street, WolverhamptonWVI ILY, UK
vol 32 no 4 may 1990
requirements by stimulating a joint learning process between users and developers.
Rapid software prototyping Prototypes may be built for a variety of purposes. These include identifying, learning about, and clarifying user requirements (requirements engineering). They involve the building of working 'mock-up' versions of parts of, or the whole, system. These are demonstrated to users with the intention of maximizing product quality; the assumption being that it is easier to focus on a working version rather than on paper documents. For this to be successful, developers must encourage users to be constructively critical. The literature suggests prototyping as a technique that promotes better user/developer communication~.L While it is difficult to formulate a universal definition of prototypes, they do exhibit some common characteristics in that they: • involve a high degree of user evaluation, which substantially affects requirements, specification, or design • are analogous to experiments built to test hypotheses • stimulate a joint learning process for users and developers There are many approaches to prototyping. Hekmatpour and Ince 3 and Ince 4 classify prototyping into three categories: • throw-away • evolutionary • incremental Throw-away prototyping is often used for requirements identification and clarification. As the name suggests the prototype built is 'discarded' once the developers and users have evaluated it. It is not intended to be used as a direct implementation but may be kept as a reference point. With incremental prototyping, a complete system design is specified, subtasks identified, and prototypes built, evaluated, and modified according to user feed-
0950-5849/90/040253~)5 © 1990 Butterworth-Heinemann Ltd
253
back. The overall design, however, remains unaltered. This is really incremental development with an emphasis on frequent user feedback about the working parts. Evolutionary prototyping allows a system to evolve, according to user feedback, so that dynamic change is allowed. At the onset of prototyping no system specification need exist; the prototyping process produces the specification. An alternative classification can be used dependent on the purpose of the prototype. If it is concerned with system function it is a functional prototype. If it concentrates on user interface it is an interface prototype. However, interface prototyping can also reveal much about function. This paper outlines the use of prototypes that conform to both of the above classifications. Evolutionary and throw-away prototypes are used for both interface design and the discovery of functional requirements. This work has identified a close correspondence between interface and function, thus reinforcing the view that the major purpose of interface prototyping is to gain insight into users' functional requirements. Management of software projects tends to depend on a traditional life-cycle approach 5. The management of prototyping, however, is not a well researched or documented topic. Particular issues are: • developing aims and objectives (type of prototyping and what is to be learnt) • selection of users • budgeting (both developer and user time) • when to start prototyping • when to stop prototyping (limiting user change and enthusiasm!) • how to document prototyping and record user changes Mayhew e t al. 6 discuss these issues and propose strategies for control, some of which have been adapted by this project.
Because the necessary information could not be elicited from the estimators, computerization of current manual procedures was not feasible. The analysis did, however, provide a partial view of current procedures. It increased both the users' and developers' understanding, and in particular it identified the main determinants for estimating. It was also evident that because the estimators produced inconsistent estimates any computerized system that mimicked their reasoning would itself be inconsistent and hence fail to address one of the major problems. This work formed a basis for discussion with management and helped identify the main problems: • Quotations were inconsistent (most significant). • Cross-referencing of previous quotations was too difficult, because of insufficient indexing and no standard layout. • Manual quotations were expensive to produce and modify. Furthermore, the relationships between costs and prices remained unclear and no rules could be identified for the transformation of determinants into quotations. Following these discussions, management, who were now more aware of the extent of the problems, wished to reformulate the requirements of the proposed system. However, they were not fully aware of what they actually wanted. This phenomenon is recognised by Yourdon 7, who states that one of the good candidates for prototyping is where: 'The user is unable to articulate (or "prespecify") his or her requirements in any form and can only determine the requirements through a process of trial and error.'
TRADITIONAL APPROACH (USE OF STRUCTURED ANALYSIS)
PROTOTYPING APPROACH
During the early stages of the project it was anticipated that analysis of the existing pricing procedures and practices (including documentation) using structured analysis7 would identify the areas causing difficulty. Hence a series of interviews with estimating staff and management were conducted. During interviews with estimators it was found that some subjects had difficulty in explaining the factors they were considering when calculating a price; attempts to decompose parts of the system often ended with a series of examples of individual circumstances cited, with no easily identifiable underlying procedures. These difficulties may have been due to an unwillingness to reveal the procedures used or an inability to verbalize them, or the questions asked may have been inappropriate or misleading. Whatever the reason, it meant that there was inadequate information for the creation of a complete specification that modelled the existing system.
At this stage it was agreed to build prototypes to solve some of the identified problems. As prototyping was relatively novel to the authors, planning and control were limited. As stated above, however, the structured analysis work helped identify the parts of the system to be prototyped and the aims and objectives of the exercise. Additionally, appropriate users were identified in advance of the prototyping sessions. Evolutionary interface prototyping was selected to help define the boundaries of the specification and to provide easy access to quotations. This was also intended to improve user/developer understanding of the pricing process. The prototypes were designed to be used as the interfaces of the final complete system. On completion of the interface prototypes it was hoped that the further information gained from them would enable functional prototypes of the pricing process to be built.
254
information and software technology
Interface prototyping An evolutionary approach was adopted with interfaces developed on the target machine. This gave the company a working system for quotation production, modification, and retrieval and hence a faster payback on some of their investment. It was intended to try to achieve several objectives: • To produce user-acceptable input screens. • To clarify the information on quotations and identify the key fields used in historical cross-referencing. • To identify management information reports. • To consolidate understanding of determinants. • To help identify the boundaries of the system. This last aim requires explanation. The users were not clear about the level of expertise that the system would exhibit in automating pricing. This had not been finalised. Prototypes were designed and implemented on the company's IBM minicomputer using RPG2 and the screen painter facilities available. Interface prototyping was conducted informally, with estimating staff and company management deciding how the system evolved. User involvement maintained and stimulated interest in the project. Screen designs and quotation and report layouts were changed directly as a result of criticism from both estimators and management. The approach brought a usable system to completion within a reasonable time-scale, undergoing the first trials within three months of the exercise starting. This would have been faster if the developers had been more familiar with RPG2 at the outset. The use of particular tools can require a time investment at the beginning of the project. This should be appreciated before prototyping takes place. The difficulties of interface prototyping and the need for designers to be both skilled programmers as well as have expertise in human factors is a current research area 8. The 'interim' system built as a result of the interface prototyping was given a trial of several months in the factory. This system is now being used by the estimators for the production of all quotations, facilitating a more streamlined operation. The data captured will be used in future research for comparison with prices produced by prototypes. There is no doubt that the function of the final system was developed from an exercise that was only considered to be interface prototyping. By eliciting the requirements from users, using only screen designs and report layouts, the final system design was developed and implemented. This type of exercise raises the question of where the boundary between the two types of prototyping should be drawn. Post-delivery modifications to the system have been few (mainly cosmetic changes to some of the printed output), and it has met with enthusiastic user response despite the lack of a well defined specification. Compared with the authors' experience with other systems, this was
vol 32 no 4 may 1990
a positive outcome, tending to reinforce the claims of prototyping in the literature. However the exercise raised as many questions about prototyping as it answered. The developers felt that the users were tending to control the process rather than the developers. The need for a strategy to handle differences of opinion between users was identified, though fortunately these were few and minor this time around. It was decided to research further into, and give further thought to, the management of prototyping before the next exercise. The relationship between prototypes, documentation, and specification was not clear. Ince 4 suggests a prototyping plan where: ~The prototyping objectives, the prototyping model adopted, a description of how prototyping is to be achieved, and the use that is to be made of the prototype, should all be written into a document known as the prototyping plan.' Analysis of the outcome of interface prototyping and a review with senior management led to a change of system boundary. The project was no longer trying to produce a fully automated pricing system. Instead, an automated costing system that would provide a consistent base for manual pricing was adopted. Interface prototyping enabled both developers and users to get a better understanding of the type of problem being investigated. This has been noted by other authors. For example, Buckley e t al. 9 go further than this and put forward the idea that the major purpose of interface prototyping is to gain insight into functional requirements: 'While a good user interface is desirable, the real value in using the user interface in this manner is to enable the end-users to talk about what they want the system to do.' Three components were identified as essential to a consistent costing and pricing process: • For costing, identify the 'route' through the factory; and apply an up-to-date cost on a 'route'. • To extend into the area of pricing, manually adjust a cost to yield a price. As a result of the analysis of interface prototyping it was decided to proceed with further prototyping. This pointed to the need for functional prototyping.
Functional prototyping The 'route' through the factory is particularly significant because costs associated with apparently similar components can vary by a factor of up to 10. Provided a subsystem could be developed to assist in the accurate prediction of 'route' and another subsystem to maintain the cost of routes, then the cost-estimation problem could be solved. The adjustment of costs to give prices is complex and dynamic, depending on market forces. In
255
some cases the actual adjustment is applied by a member of the board. It was felt that it would not be practical to try to extend automation into this area. Functional prototypes for the prediction of route have been built with the aim of discovering the relationship between determinants of process and route. Due to the complexity of this problem, current work has been restricted to silver finishes to evaluate feasibility. The prototypes were constructed using a 'throw-away' approach, the intention being to enhance the system already installed on the IBM minisystem once the requirements could be specified. The production staff in the factory were identified as the target user group to maximize the validity of predicted routes. These staff decide the actual route that a job takes when it arrives at the factory gate. Prototypes to tackle the route problem were built using a commercially available package. The expertsystem shell Crystal was chosen for this task. It is easy to learn, and once some basic system knowledge is acquired, prototypes can be quickly built and easily modified. The aim was not to construct an expert system per se, but to investigate further the decisions taken when routing a job through the factory. With the hindsight o f the earlier prototyping exercise, the developers wished to maximize the control and monitoring of the exercise. This was to be improved in three ways: • Extensive records were to be kept of prototyping sessions. • Some prototyping sessions were to be attended by more than one developer to enhance critical analysis. • A strategy was to be developed to control change. The latter was inspired by the control strategy advocated by Mayhew et al. 6. Using this method, all changes to prototypes suggested by users are recorded and classified into three categories (cosmetic, local, and global) according to their impact on the system. It is recommended that both cosmetic and local changes are made immediately, with the latter being subject to later scrutiny. Global changes that have an impact beyond the parts of the system currently under review are recorded, but change is delayed until all the implications have been researched. Hence the developers retain control over changes that have wide-reaching effects, while still leaving users with the impression that it is their system. The prototyping sessions did not produce any global changes (this was expected given that the main boundaries had now been determined). Cosmetic changes were few, again not surprising as it was throw-away functional prototyping, rather than interface prototyping, that was being carried out. It was not possible to make functional changes during sessions; alterations to one rule in an expert system can have a ripple effect on other rules. The strategy was to confirm the change with the user and return with an updated system quickly (normally a few days). Otherwise the suggestions made by Mayhew were found to be most pertinent. The results of the exercise can be provisionally des-
256
Table I. Summary of time spent in prototype sessions (silver department only; functional prototyping) Meeting no
Time (h)
Introductory meeting Pre-prototype meeting 1 2 Review meeting 3 4 5 6 Totals (excluding reviews) Changes per hour
Changes
2 1.5
18 11
1.25 1 1.3 0.63 ~8 7
8 6 7 5 55
Note: Times exclude introductory and review meetings where changes were not suggested.
cribed as a success. After about six sessions with users, plus three review sessions and non-prototype meetings, the prototype was accepted. Table 1 summarizes the time spent and the changes suggested. Management are impressed with the outcome to the extent that they have requested that it be developed into a usable system. They would like it to be used as a sales support system as well as being used by the estimating staff. This is in spite of the fact that it was not intended to be an evolutionary prototype. The developers paid particular attention to educating the users of the dangers of adopting rapidly built throw-away prototypes as fully implemented systems. This is leading to problems that were not anticipated. Because the prototyping tool was not chosen with the intention of building a finished system, installation and maintenance were not considered relevant at the time. It is difficult to deny access to a prototype, even though it was never intended for everyday use.
FUTURE
WORK
There are three areas that require further work as a result of the investigation carried out so far: • Can the methods used in developing the limited domain system be extended so that the whole problem can be tackled? • Will the use of prototype systems as installed products cause maintenance problems? • Can the control of prototyping be enhanced by the possible development of a metric? Currently, the rule-based system uses well over 150 rules for a production area, representing about 20 per cent of factory finishes. Research is needed to discover whether such prototypes and the associated management and control strategies are suitable for extending the approach from the particular to the general. Additionally, documentation would be required for the guidance of developers with less experience of the problem. If the decision is made to remain with the rule-based system, the use of rapid prototyping may lead to future maintenance problems. Diaper ~° states:
information and software technology
'such expert systems are likely to be expensive and difficult, if not impossible, to maintain and up-date.' Yourdon 7 also draws attention to this problem and writes: 'There is a significant danger that either the user or the development team may try to turn the prototype into a production system. This usually turns out to be a disaster ...' Further work on the installation, maintenance, and restructuring or re-specifying of the installed system will thus be required. This is important when it is considered that the rule-based approach was not intended to be installed as part of a finished system. It has been suggested 6 that the formulation of a metric linking number and type of change during a set period could help determine when to cease prototyping. Further analysis of the results will be necessary before comment is possible on this point.
DISCUSSION The paper has attempted to show how interface and functional prototyping have contributed to the success, so far, of the development of a costing system. The experiences of the authors back up research done by other workers and suggest that this type of problem, with difficult open-ended initial requirements, can be developed successfully by planned user participation centred on prototypes. The approach provided a good vehicle for the education of both the developers and users. It forced the designer to take account of the users' wishes at every point. A designer must be prepared to encourage users by considering all comments and criticisms, taking care not to be defensive or negative. This enhanced communication between developers and users was found in both interface and functional prototyping. In fact, one perceived advantage of prototyping is the focus on the problem and communication with users at the expense of the technology. The user-interface prototyping allowed a payback early on in the project by evolution of a usable system, albeit with limited function. This was despite the extra time required by the developers to learn about the software tools and how to conduct prototype evaluation. For prototyping to be adopted, it is necessary for project managers to recognise the potential payback and also be prepared to budget for prototyping activities. A dynamic process such as prototyping requires careful planning and control. The developers found it highly iterative and difficult to estimate the time and effort involved in the different activities. Identifying milestones was also found to be a challenging problem. This reinforces the comments 6 that the management of prototyping requires more research, development, and publication. The classification of approaches was found to be useful in the planning of the two prototyping exercises. It was a definite aid to think in terms of throw-away versus
vol 32 no 4 may 1990
evolutionary, or interface versus function. As with all theory, however, the practice is not so clear. For example, interface prototyping provided communication about functional requirements as well as inspiring major user management decisions. Another example was the strong user reluctance to throw away a prototype that was designed to be discarded. The users wish to change the prototype to evolutionary for good reason. The developers could accept this provided doubts about maintainability can be resolved.
CONCLUSION The authors are encouraged by the results of the use of prototyping in the project. It was felt that progress was more effective than using a conventional structured design method where user and developer perceptions were uncertain. More extensive testing is planned for functional prototypes currently under construction. The route problem is being investigated. The prototypes built have been enthusiastically received. Despite the use of control strategies, however, additional questions have arisen about the installation and maintenance of the systems developed so far. It is felt that more work is required to develop this type of prototyping into a controllable design methodology.
ACKNOWLEDGEMENTS The authors thank the staff at Frost Electroplating for their collaboration in this project and the personnel at JBS Computers for their advice.
REFERENCES 1 Cerveny, R P, Garrity, E J and Sanders G L 'The application of prototyping to systems development: a rationale and model' J. Manage. Inf. Syst. Vol 3 No 2 (1986) 2 Parbst, F 'Experience with prototyping in an IBM based installation' in Budde, R et al. Approaches to prototyping Springer-Verlag, Berlin, FRG (1983) 3 Hekmatpour, S and Ince, D C 'Rapid software prototyping' Technical report 86/4 Open University, Milton Keynes, UK (1986) 4 Inee, D C Software development. Fashioning the baroque Oxford University Press, Oxford, UK (1988) 5 Lehman, M M and Belady, L A Program ew~lution. Process of software change Academic Press, New York, NY, USA (1985) 6 Mayhew, P J, Worsley, a and Dearnley, P A 'Control of software prototyping process: change classification approach' Inf. Soft. Technol. Vol 31 No 2 (March 1989) pp 59 66 7 Yourdon, E Modern structured analysis Prentice Hall, Englewood Cliffs, NJ, USA (1989) 8 Gray, P D, Kilgour, A C and Wood, C A 'Dynamic reconfigurability for fast prototyping of user interfaces' SOL/?.Eng. J. Vol 3 No 6 (November 1988) 9 Buekley, M, Candy, L and Edmonds, E A 'Determining requirements and prototyping the user interface module. Methodology for expert system specification' in Human and organisational issues of expert systems Conference proceedings, Stratford, UK, May 1988 10 Diaper, D 'The promise of POMESS: a people oriented methodology for expert system specification' in Human and organisational issues of expert systems Conference proceedings, Stratford, UK, May 1988
257