- Email: [email protected]
Prototyping of computer-based training materials
Computers Educ. Vol. 22, No. 3, pp. 251-256, 1994 Copyright 0 1994 Elsevier Science Ltd
Pergamon
Printed in Great Britain. All rights reserved 0360-1315/94 $6.00 + 0.00
PROTOTYPING
OF COMPUTER-BASED MATERIALS D. E. GRAY
Department
of Educational
Studies,
and T. R.
TRAINING
BLACK
University of Surrey, Guildford [Fax: +44 483 3008031
GU2 5XH, Surrey,
England
(Received IO March 1993; accepted 22 July 1993) Abstract-This article proposes that prototyping would enhance the quality of both the computer-based training courseware development process and the final product. Too often a communication gap exists within a courseware design team or between designers and program sponsors or end-users. Prototyping uses common software utilities as design tools to more clearly convey such aspects as learner-courseware interactivity, screen design, and navigation strategies, enhancing the understanding of package functionality early in development, to enhance communication with sponsors as well as fellow developers. Through spending modest amounts of time and resources on a prototype, considerable savings can be made by avoiding costly errors which then have to be rectified. In some cases, it may significantly reduce the risk of complete project failure.
INTRODUCTION
One of the perennial problems with computer-based training development is that the courseware which finally emerges from the development process is not what its users or those who commission the courseware wanted at the outset. This is one aspect of a wider problem that Pressman[l] calls the “software crisis”, epitomized by grossly inadequate schedule and cost estimates, poor productivity levels for software development, and poor quality software. One potential cause of this crisis is the difficulty in communication within the development lifetime of a package. First, designers have to communicate to the commissioning group, who increasingly are more sophisticated in their expectations, just what the final courseware will look like simply to ensure that their interpretation of aims is acceptable. Plans presented solely on paper do not always give the true flavour of what is envisaged or desired for this highly visual medium, a problem that has been exacerbated by developments in technology (graphical user interfaces, increased screen resolution, multimedia), offering greater choice. There is a need to return to the commissioning group at an early stage before too much is invested in a potentially misguided endeavour, with a clear picture of the design team’s perception of the outcome. Secondly, there is the design team’s difficulty in communicating among themselves during the design and development stage. Packages are increasing in size, not only from the viewpoint of quantity of learning, but also in demands on disc space with graphics, animation and a desire to provide users with more open-ended opportunities, employing add-on software such as tools, working models or simulations. There needs to be a way of trying out ideas and presenting them visually without having to generate a fully-working program. Finally, at the implementation stage comes the designer’s problem of communicating the content and interaction of the final courseware package to those who actually produce the software, such as graphic artists and programmers. With a need to communicate across three groups, the potential exists for the final package not to correspond to what the commissioning group originally wanted. It is proposed that prototyping can contribute significantly to enhancing communication all around and consequently enhance the probability of producing a high quality product, as well as enhancing the efficiency of the development process. WHAT
IS
PROTOTYPING?
Prototyping has been an important issue in the software development and automation realm for a number of years. Yet, despite the attention given to the subject, there is still a lack of 251
252
D. E. GRAY and T. R. BLACK
consensus as:
as to what prototyping
an original
version
actually
or model
is or should be. King[2] succinctly
on which a completed
software
describes
prototyping
system is formed.
Note the use of the word “model”: there is no indication of what its size should be, or how much functionality compared to the final version it should have. Certainly in mechanical engineering, a prototype (for example, of a bridge or an aircraft) can be either a scaled-down model or a full sized version. In software engineering, however, there is no suggestion of a complete version of the system being produced since the functions will be to illustrate specific important aspects of the final system. Just what aspects are to be included will vary depending on the intended function of the prototype and it is these that need to be clarified if this tool is to make a contribution to enhancing courseware development. One way of viewing the process is to consider the successful development of a package as the product of the interaction of three technologies: (a) educational technology, focusing on what is to be learned and facilitating learning through interaction with the courseware, (b) software engineering, ensuring a smooth running package with quality graphics, animation and interfaces with other media, and (c) project management, coordinating the resources to maximize quality while ensuring efficient use of resources. The following sections will examine assist in facilitating communication
the development process across these technologies.
and focus on how prototyping
can
Meeting requirement speciJications A prototype produced to communicate with the program’s sponsors may be built as a working model of part of the program providing examples of functionality that would include demonstrations of interactivity, samples of learner routes, and specific screen designs to establish a house style. Above all, it should convey the cognitive emphasis of the package: whether it intends merely to transmit verbal information and concepts or whether it seeks to put across higher level cognitive skills such as problem solving. The relative success in stimulating thinking at the desired level may not come through clearly from paper-based designs but should be more obvious when observing a sample from the functional courseware. Vonk[3] believes that prototyping is only valid as part of the requirements phase. As such, it has two main characteristics-iteration and user participation. Through iteration, the prototype is continually passed back and forth between the developer and the end-user with changes being made until the user’s requirements are fully reflected in the prototype itself. This might be at two different levels: through modelling the user interface of the system (user-interface prototyping, such as menus or navigational aids); or through modelling some or all of the system’s functions (functional prototyping, such as emulating the way the system stores and retrieves data). The second characteristic of prototyping--user participation-means that not only do users become actively involved, through being asked to evaluate prototypes, they are encouraged to think about their requirements. This may entail users focusing less on what they want than on what they do not want, still a useful and valid function. Requirements prototyping, then, is an example of communication between designers and program sponsors or end-users. The importance of user-interface prototyping is confirmed by Wasserman and Shewmake [4] who highlight five important benefits. Ideally a prototype: (a) enables those who commission the courseware to evaluate the system in practice and to suggest changes; (b) enables the developers to evaluate user performance and to modify the system to reduce user errors and to improve user satisfaction; (c) facilitates experimentation with alternative approaches and modifications; (d) gives the user a more immediate sense of the proposed system and thereby encourages those who have provided the original specifications to think more carefully about the needed and desirable characteristics of the system; (e) reduces the likelihood of project failure.
Prototyping CBT
253
The feedback collected through such processes should actually encourage creativity in the development team by allowing the testing of new ideas. Also learners do not always respond as predicted by existing models or predetermined strategies, thus there is the need for early testing of proposed approaches on learners. For example, Alessi and Trollip[5] outline five main types of strategy for learner instruction. A prototype can emulate each of these and allow an early consideration of structure and appropriateness for a given target audience. (a) Tutorial: if the training is to use significant amounts of text, the designer might wish to prototype use of graphics, font sizes, typefaces, colour, text positions and size of blocks of text on the screen. Do users feel comfortable with the amount of text presented? How should the screen be replenished with text through scrolling, overlays, wipes, or fades? Would an alternative medium be preferable, such as paper-based formats? Is the cognitive emphasis of the questions consistent with that of the package objectives? (b) Drill: does the user understand how to use the drill? How are items selected (by the program or by the user)? How are user responses to be judged and the answers logged? Is feedback to be given and if so, when and how? How is the drill to be terminated? (c) Game: do users find this an acceptable way of learning the subject matter? Is it instructive as well as being fun? Is it simple enough to actually build within your budget/time scales? (d) Simulation: are instructions on how the user is to respond clear and is the computergenerated output meaningful? Does the user have sufficient control over its operation? Do users actually learn anything, as opposed to merely being entertained by a lot of graphics and animation on the screen? Is there help or guidance available when needed? Is the routing (e.g., through selection of options) clear and not too complex, so that the learner does not get lost? (e) Test: what sort of format is most appropriate: fill in the missing word, yes/no, multiplechoice? Is it clear to users how to enter or change answers before the system evaluates or marks them? Does the system log user inputs effectively? If summative scores are required, does the system present them in an appropriate way for the particular learning context? Clearly, in acquiring useful information on these aspects, the process of user evaluation must be planned in advance. Will verbal reactions be adequate, or is a more formal methodology of acquiring feedback needed? If so, an instrument must be designed and the approach tested using the prototype in advance. It also might be prudent to prototype aspects of a courseware management system, that is, a system which logs student names and the courses they have completed, questions answered correctly and incorrectly, what level of mastery has been achieved, etc. Evaluating a prototype may be an instructive process for managers, getting them to think about their requirements for such a management system. Hansen[6] points out that with long projects, user requirements may change over time between the various stages of the software development cycle. If those who have commissioned the package have been involved in examining prototypes from early on in the project there may be fewer costly changes at the testing and acceptance stage. So ultimately, by spending money on the model, an organization can save money on the final product, whether this is an applications package, computer system or CBT courseware. Design and development Another purpose of prototyping which possibly results in a different type of product, may involve using it as a kind of discussion tool between the various members of a design team such as needs analysts, subject matter experts and instructional designers. The prototype may be used to try out new ideas and to illustrate them to colleagues in the sort of precise way that paper-based designs could not achieve. In this case, the prototype need not be fully functional and only the primarily visual aspects would need to be presented. This avoids any need for programming by the design team and focuses attention on evaluating user-courseware interaction. It can also facilitate communication on house style and screen design encouraging new ideas from graphically orientated members to be demonstrated.
254
D. E. GRAK and T. R. BLACK
Humancomputer interaction (HCI), an important by Baecker and Buxton[7] as the set of processes, a computer.
dialogues
and actions through
consideration
which a human
in any design, has been defined
user employs and interacts
with
The implications of HCI for courseware development are many and varied. It is important, for example, that the screen design (colours, resolution, layout, etc.) is acceptable and in particular, not alienating, to learners. The methods for navigating around the course should be accessible and easily identified with the functions they perform. Only by prototyping these and trying them out will designers know they are valid in the context in which they will be used. This can also help to avoid all packages looking the same, having established a “perfect” house style. Of course, prototyping will only be cost-effective if the costs of developing the prototype are kept to a minimum. Birrell and Ould[8] argue that one of the essential aspects of a prototype is its “throwaway” nature-once it has been analysed for information it will be discarded and the main developmental path started or re-entered. It is at this stage that designers may wish to try out ideas for enhancing the quality of learning by experimenting with new strategies. For example, they may wish to employ as an alternative to a tutorial approach, one that encourages exploration and discovery, providing software tools for the user to try ideas. Such an approach would need to be tested to ensure that the courseware does not leave users unsupported or under supported at any time. Implementation If the prototype is used to convey ideas from the design team to the programmer, then its main purposes may be to communicate unambiguously branching and routing schemes as well as the learner-courseware interactivity necessary to achieve the objectives. As Black[9] points out, this may be particularly important for more complex routing such as that required for judging student responses to multiple-choice questions and higher level skills such as problem solving. The results then may include fairly complete text and graphics, navigational aids, and questions to stimulate user responses with appropriate courseware responses (either feedback in a tutorial or reactions from a simulation), providing complete screen designs. Using the prototype combined with routing maps, the programmer’s task is to produce a fully functional program. without any need to make educational decisions. Such an approach frees the designers from being programmers and programmers from being designers, but it does allow the group to exchange ideas, educating each other on not only what is possible but what is desirable. It is often essential for the designer to communicate detailed and complex ideas clearly to programmers. Specifications passed to programmers in written form can fail to explain all the functions of the system in adequate detail and consequently it is all too easy for errors and misinterpretations to occur. When problematic aspects of the system can be included as a prototype, then there is less risk of programmers misinterpreting the design. Consequently, this compels the designers to consider and resolve all issues related to learning, interactivity and navigation. leaving no ambiguities. It may be at this stage that a prototype will also assist programmers and project managers to predict time and resource needs for programming and fully implementing the package more accurately. Anticipating resource requirements can ensure that the project is not only completed on time but that quality is maintained. SOFTWARE
TOOLS
There is some difficulty in identifying appropriate prototyping tools. If the prototype is implemented in the language or system chosen for creating the final version, this can be time-consuming and therefore expensive, since there is a tendency to try to make a fully functional version. In a series of limited exercises at the University of Surrey, it was found that when experienced developers engaged in a prototyping activity, those using their standard authoring system tended to produce half to one-third the number of frames as those using a hypermedia package where sophisticated programming was virtually impossible. Software used to develop the prototype does not have to be the same as that used to implement the final package. The final
255
Prototyping CBT
distributed version system requires the use of a flexible language that demands structured programs which are complex, but must be easily debugged and designed to be maintained. By contrast, the prototype must be produced cheaply and quickly (minor flaws are acceptable) with maintenance not an issue since the prototype will, in all likelihood, be discarded after use. Today software is already available which can offer the development team many useful prototyping facilities. Hypermedia, e.g. HyperCard for the Apple, Linkway or Guide for the IBM PC, is one type of system that has successfully been used. Care should be taken, however, that the structure of the prototype is similar to that of the intended final system. One of the dangers of hype~edia is the ease with which buttons can be created which, with lack of careful planning, can encourage not only “spaghetti” programming which is so difficult to debug, but also “spaghetti” routing, which is confusing to learners trying to navigate through a package. Some practical advantages to be gained from prototyping with hypermedia are provided by Fitzgibbon [ lo] who describes the difficulties faced in producing an interactive video for airline pilot training with Scandinavian Airlines System. He maintains that the problem is that while users and subject matter experts have very definite opinions on the content, format and feel of the final course, they tend to voice these opinions effectively only after seeing the finished product. By this time it is too late, since the designs are “set in concrete”. Fitzgibbon puts it clearly: It seems that no amount of specifications on paper, reviews and signatures can bridge the communications gap between what was originally envisaged and what was initially produced. In fact, too many intermediate stages and too much delay can often be counter-productive. By the time the lesson finally emerges everybody has forgotten what was intended. Fitzgibbon recommends the production of a prototype early in the development process. This model acted as a communication vehicle between the designer and subject matter experts, between the subject matter experts and secondary parties (validators, quality assurance supervisors, managers, students), and between the designer and programmer. Changes were made to the model at a fraction of the cost and in a fraction of the time than what would have resulted from making changes on the finished courseware. This is because three new items of technology were used, making prototyping a viable proposition: (a) digital scanners to scan existing diagrams into the lesson without the need to draw graphics; (b) phonetic audio which will speak a word processed file without the need to record provisional audio at a studio; (c) card index programs which speed the process of creating storyboards (hypermedia). WHY
IS SOFTWARE
PROTOTYPING
NOT
THE
NORM?
A range of reasons why software prototyping is important have been considered, yet it seems that although prototyping is used in software development, it is also undoubtedly the case that this is the exception rather than the rule. Birreli and Ould[8] suggest two reasons for the lack of use of prototyping. Firstly, there is a widespread conception that software prototyping is not cost-effective. Secondly, the tools for software prototyping are not readily available. Given the diversity of development schemes, one could ask how true these assumptions are. The belief that software prototyping is not cost-effective may have partial validity in organizations that produce near-perfect software every time. Some projects, however, have run into expensive difficulties which could have been avoided through a small expenditure on prototyping. It is too late when the project’s sponsors finally get to see the system, and it is not what they expected or desired. Certainly, in the case of CBT courseware development, prototyping is comparatively rare. Examine most textbooks or guides to CBT design and it is difficult to discover either a recommendation to use prototype modelling or the methodology for doing it. Perhaps one reason is that, in general, the scale of CBT development is usually smaller than for that of other software projects. Thus the resulting software does not have the updating and maintenance problems of typical utility software packages whose lifecycle may cover a number of years as they are continuously improved. If a design team is only producing a 30 minute course, project management
256
D. E.
GRAY
and T. R. BLACK
considerations may militate against prototyping when it can get sponsors or end-users to evaluate the final product and make any necessary modifications. Another argument concerns human resources and organization. Very experienced CBT development teams may operate according to a strictly defined methodology which is tried and tested. Why prototype, when users know exactly what they are going to get because they have seen it all before? Both these arguments have some validity, but they should not obscure the very real benefits that prototyping can bring to CBT courseware development through providing a cost-effective way of considering new ideas. No matter how small the project, it is surely better to get matters such as styles, formats, and functionality agreed well before the final product is completed. Even the small project cited above will cost tens of thousands of pounds. As far as the well tried approach is concerned, this presents problems as well as advantages. If the end product is so predictable should not the development team be trying something different? Think of the unfortunate end-users who have to plough through hours of the same old courseware. Standardization can be an argument for avoiding imagination, experimentation and innovation in ways of enhancing learning. CONCLUSIONS It has been noted that prototyping is an important element in traditional hardware and software engineering and proposed that there is an important role for it in courseware development. The production of CBT prototypes can enhance the interaction of the three technologies to encourage creative development: educational/learning design, software engineering and project management. While the scale of CBT development is, in general, smaller than that of software development as a whole, the difficulties inherent in CBT production are not insignificant. Indeed, the move towards a multimedia approach (often including a video and audio component such as CD-I and interactive video) means that CBT development costs may be quite considerable for a single package. As in general software development, CBT prototyping may be of most value at either the requirement or specification stage. The requirement stage is vital for finalizing preferences of those who commission the package, which can be as much an education and learning process for them as it is for designers. The next stage, design and development, requires continual communication among members of the design team: subject matter experts, courseware designers, graphics artists, video production team, etc. A set of throw-away versions for trying out ideas can be invaluable. Finally, completing the final specifications, when various aspects of the final system must be modelled and tried out, requires a flexible approach. Prototyping can be especially useful for designers to communicate the results to those responsible for producing and programming the final courseware package. Hence, development tools such as hypermedia software are very useful for producing and linking together large numbers of screens with relative ease. In the final analysis, the aim of prototyping is to not only reduce the risks associated with the courseware development but also enhance its quality as well. If used wisely, it can be a cost-effective addition to the development process. REFERENCES Engineering: A Practitioner’s Approach. McGraw-Hill International Editions. New York I. Pressman R. S., S@nzre (1988). 2. King D., Current Pructices in Sofzware Dei~eiopment. Yourdon Press, New York (1984). Use of CASE Technology. Prentice Hail. Hemei Hemnstead (1990). 3. Vonk R.. Prototvpina: The Efietive A. and Shewmake D., The role of prototypes inthe User Software Engineering (USE) methodology. In 4. Wasserman Human-Comauter Inferaction (Edited bv Preece J. and Keller L.). Prentice Hall. Englewood Cliffs, N.J. (1990). 5. Alessi S. and’Trollip S., Compiter Ea.& Instruction: Method.7 and Development. Prentice Hall, Englewood Cliffs, N.J. (1991). 6. Hansen H., Up clt Running: A Case Study of Succrssfui Systems Development. Yourdon Press, New York (1984). Los Altos, Cafif. 7. Baecker R. M. and Buxton W. A. S., Readings in Human- Computer Interaction. Morgan Kaufmann, (1987). England 8. Birreli N. and Ould M., A Practical Handbook fir Sqftware Development. Cambridge Univ. Press, Cambridge, (1985). CAL courseware: a role for computer-shy subject experts. In Designing New Systems and 9. Black T. R., Prototyping Technolonies for Lenrnina (Edited bv Mathias H., Rushbv N. and Budgett R.). Konan Pane, London (1988). 10. Fitzgibbon d., Methodology for CBT design: courseware modelling. In Symposium-on Cohputer Assisted Learning: Handbook and Abstracts. Univ. of Surrey, Guildford, England (1989).
Pergamon
Printed in Great Britain. All rights reserved 0360-1315/94 $6.00 + 0.00
PROTOTYPING
OF COMPUTER-BASED MATERIALS D. E. GRAY
Department
of Educational
Studies,
and T. R.
TRAINING
BLACK
University of Surrey, Guildford [Fax: +44 483 3008031
GU2 5XH, Surrey,
England
(Received IO March 1993; accepted 22 July 1993) Abstract-This article proposes that prototyping would enhance the quality of both the computer-based training courseware development process and the final product. Too often a communication gap exists within a courseware design team or between designers and program sponsors or end-users. Prototyping uses common software utilities as design tools to more clearly convey such aspects as learner-courseware interactivity, screen design, and navigation strategies, enhancing the understanding of package functionality early in development, to enhance communication with sponsors as well as fellow developers. Through spending modest amounts of time and resources on a prototype, considerable savings can be made by avoiding costly errors which then have to be rectified. In some cases, it may significantly reduce the risk of complete project failure.
INTRODUCTION
One of the perennial problems with computer-based training development is that the courseware which finally emerges from the development process is not what its users or those who commission the courseware wanted at the outset. This is one aspect of a wider problem that Pressman[l] calls the “software crisis”, epitomized by grossly inadequate schedule and cost estimates, poor productivity levels for software development, and poor quality software. One potential cause of this crisis is the difficulty in communication within the development lifetime of a package. First, designers have to communicate to the commissioning group, who increasingly are more sophisticated in their expectations, just what the final courseware will look like simply to ensure that their interpretation of aims is acceptable. Plans presented solely on paper do not always give the true flavour of what is envisaged or desired for this highly visual medium, a problem that has been exacerbated by developments in technology (graphical user interfaces, increased screen resolution, multimedia), offering greater choice. There is a need to return to the commissioning group at an early stage before too much is invested in a potentially misguided endeavour, with a clear picture of the design team’s perception of the outcome. Secondly, there is the design team’s difficulty in communicating among themselves during the design and development stage. Packages are increasing in size, not only from the viewpoint of quantity of learning, but also in demands on disc space with graphics, animation and a desire to provide users with more open-ended opportunities, employing add-on software such as tools, working models or simulations. There needs to be a way of trying out ideas and presenting them visually without having to generate a fully-working program. Finally, at the implementation stage comes the designer’s problem of communicating the content and interaction of the final courseware package to those who actually produce the software, such as graphic artists and programmers. With a need to communicate across three groups, the potential exists for the final package not to correspond to what the commissioning group originally wanted. It is proposed that prototyping can contribute significantly to enhancing communication all around and consequently enhance the probability of producing a high quality product, as well as enhancing the efficiency of the development process. WHAT
IS
PROTOTYPING?
Prototyping has been an important issue in the software development and automation realm for a number of years. Yet, despite the attention given to the subject, there is still a lack of 251
252
D. E. GRAY and T. R. BLACK
consensus as:
as to what prototyping
an original
version
actually
or model
is or should be. King[2] succinctly
on which a completed
software
describes
prototyping
system is formed.
Note the use of the word “model”: there is no indication of what its size should be, or how much functionality compared to the final version it should have. Certainly in mechanical engineering, a prototype (for example, of a bridge or an aircraft) can be either a scaled-down model or a full sized version. In software engineering, however, there is no suggestion of a complete version of the system being produced since the functions will be to illustrate specific important aspects of the final system. Just what aspects are to be included will vary depending on the intended function of the prototype and it is these that need to be clarified if this tool is to make a contribution to enhancing courseware development. One way of viewing the process is to consider the successful development of a package as the product of the interaction of three technologies: (a) educational technology, focusing on what is to be learned and facilitating learning through interaction with the courseware, (b) software engineering, ensuring a smooth running package with quality graphics, animation and interfaces with other media, and (c) project management, coordinating the resources to maximize quality while ensuring efficient use of resources. The following sections will examine assist in facilitating communication
the development process across these technologies.
and focus on how prototyping
can
Meeting requirement speciJications A prototype produced to communicate with the program’s sponsors may be built as a working model of part of the program providing examples of functionality that would include demonstrations of interactivity, samples of learner routes, and specific screen designs to establish a house style. Above all, it should convey the cognitive emphasis of the package: whether it intends merely to transmit verbal information and concepts or whether it seeks to put across higher level cognitive skills such as problem solving. The relative success in stimulating thinking at the desired level may not come through clearly from paper-based designs but should be more obvious when observing a sample from the functional courseware. Vonk[3] believes that prototyping is only valid as part of the requirements phase. As such, it has two main characteristics-iteration and user participation. Through iteration, the prototype is continually passed back and forth between the developer and the end-user with changes being made until the user’s requirements are fully reflected in the prototype itself. This might be at two different levels: through modelling the user interface of the system (user-interface prototyping, such as menus or navigational aids); or through modelling some or all of the system’s functions (functional prototyping, such as emulating the way the system stores and retrieves data). The second characteristic of prototyping--user participation-means that not only do users become actively involved, through being asked to evaluate prototypes, they are encouraged to think about their requirements. This may entail users focusing less on what they want than on what they do not want, still a useful and valid function. Requirements prototyping, then, is an example of communication between designers and program sponsors or end-users. The importance of user-interface prototyping is confirmed by Wasserman and Shewmake [4] who highlight five important benefits. Ideally a prototype: (a) enables those who commission the courseware to evaluate the system in practice and to suggest changes; (b) enables the developers to evaluate user performance and to modify the system to reduce user errors and to improve user satisfaction; (c) facilitates experimentation with alternative approaches and modifications; (d) gives the user a more immediate sense of the proposed system and thereby encourages those who have provided the original specifications to think more carefully about the needed and desirable characteristics of the system; (e) reduces the likelihood of project failure.
Prototyping CBT
253
The feedback collected through such processes should actually encourage creativity in the development team by allowing the testing of new ideas. Also learners do not always respond as predicted by existing models or predetermined strategies, thus there is the need for early testing of proposed approaches on learners. For example, Alessi and Trollip[5] outline five main types of strategy for learner instruction. A prototype can emulate each of these and allow an early consideration of structure and appropriateness for a given target audience. (a) Tutorial: if the training is to use significant amounts of text, the designer might wish to prototype use of graphics, font sizes, typefaces, colour, text positions and size of blocks of text on the screen. Do users feel comfortable with the amount of text presented? How should the screen be replenished with text through scrolling, overlays, wipes, or fades? Would an alternative medium be preferable, such as paper-based formats? Is the cognitive emphasis of the questions consistent with that of the package objectives? (b) Drill: does the user understand how to use the drill? How are items selected (by the program or by the user)? How are user responses to be judged and the answers logged? Is feedback to be given and if so, when and how? How is the drill to be terminated? (c) Game: do users find this an acceptable way of learning the subject matter? Is it instructive as well as being fun? Is it simple enough to actually build within your budget/time scales? (d) Simulation: are instructions on how the user is to respond clear and is the computergenerated output meaningful? Does the user have sufficient control over its operation? Do users actually learn anything, as opposed to merely being entertained by a lot of graphics and animation on the screen? Is there help or guidance available when needed? Is the routing (e.g., through selection of options) clear and not too complex, so that the learner does not get lost? (e) Test: what sort of format is most appropriate: fill in the missing word, yes/no, multiplechoice? Is it clear to users how to enter or change answers before the system evaluates or marks them? Does the system log user inputs effectively? If summative scores are required, does the system present them in an appropriate way for the particular learning context? Clearly, in acquiring useful information on these aspects, the process of user evaluation must be planned in advance. Will verbal reactions be adequate, or is a more formal methodology of acquiring feedback needed? If so, an instrument must be designed and the approach tested using the prototype in advance. It also might be prudent to prototype aspects of a courseware management system, that is, a system which logs student names and the courses they have completed, questions answered correctly and incorrectly, what level of mastery has been achieved, etc. Evaluating a prototype may be an instructive process for managers, getting them to think about their requirements for such a management system. Hansen[6] points out that with long projects, user requirements may change over time between the various stages of the software development cycle. If those who have commissioned the package have been involved in examining prototypes from early on in the project there may be fewer costly changes at the testing and acceptance stage. So ultimately, by spending money on the model, an organization can save money on the final product, whether this is an applications package, computer system or CBT courseware. Design and development Another purpose of prototyping which possibly results in a different type of product, may involve using it as a kind of discussion tool between the various members of a design team such as needs analysts, subject matter experts and instructional designers. The prototype may be used to try out new ideas and to illustrate them to colleagues in the sort of precise way that paper-based designs could not achieve. In this case, the prototype need not be fully functional and only the primarily visual aspects would need to be presented. This avoids any need for programming by the design team and focuses attention on evaluating user-courseware interaction. It can also facilitate communication on house style and screen design encouraging new ideas from graphically orientated members to be demonstrated.
254
D. E. GRAK and T. R. BLACK
Humancomputer interaction (HCI), an important by Baecker and Buxton[7] as the set of processes, a computer.
dialogues
and actions through
consideration
which a human
in any design, has been defined
user employs and interacts
with
The implications of HCI for courseware development are many and varied. It is important, for example, that the screen design (colours, resolution, layout, etc.) is acceptable and in particular, not alienating, to learners. The methods for navigating around the course should be accessible and easily identified with the functions they perform. Only by prototyping these and trying them out will designers know they are valid in the context in which they will be used. This can also help to avoid all packages looking the same, having established a “perfect” house style. Of course, prototyping will only be cost-effective if the costs of developing the prototype are kept to a minimum. Birrell and Ould[8] argue that one of the essential aspects of a prototype is its “throwaway” nature-once it has been analysed for information it will be discarded and the main developmental path started or re-entered. It is at this stage that designers may wish to try out ideas for enhancing the quality of learning by experimenting with new strategies. For example, they may wish to employ as an alternative to a tutorial approach, one that encourages exploration and discovery, providing software tools for the user to try ideas. Such an approach would need to be tested to ensure that the courseware does not leave users unsupported or under supported at any time. Implementation If the prototype is used to convey ideas from the design team to the programmer, then its main purposes may be to communicate unambiguously branching and routing schemes as well as the learner-courseware interactivity necessary to achieve the objectives. As Black[9] points out, this may be particularly important for more complex routing such as that required for judging student responses to multiple-choice questions and higher level skills such as problem solving. The results then may include fairly complete text and graphics, navigational aids, and questions to stimulate user responses with appropriate courseware responses (either feedback in a tutorial or reactions from a simulation), providing complete screen designs. Using the prototype combined with routing maps, the programmer’s task is to produce a fully functional program. without any need to make educational decisions. Such an approach frees the designers from being programmers and programmers from being designers, but it does allow the group to exchange ideas, educating each other on not only what is possible but what is desirable. It is often essential for the designer to communicate detailed and complex ideas clearly to programmers. Specifications passed to programmers in written form can fail to explain all the functions of the system in adequate detail and consequently it is all too easy for errors and misinterpretations to occur. When problematic aspects of the system can be included as a prototype, then there is less risk of programmers misinterpreting the design. Consequently, this compels the designers to consider and resolve all issues related to learning, interactivity and navigation. leaving no ambiguities. It may be at this stage that a prototype will also assist programmers and project managers to predict time and resource needs for programming and fully implementing the package more accurately. Anticipating resource requirements can ensure that the project is not only completed on time but that quality is maintained. SOFTWARE
TOOLS
There is some difficulty in identifying appropriate prototyping tools. If the prototype is implemented in the language or system chosen for creating the final version, this can be time-consuming and therefore expensive, since there is a tendency to try to make a fully functional version. In a series of limited exercises at the University of Surrey, it was found that when experienced developers engaged in a prototyping activity, those using their standard authoring system tended to produce half to one-third the number of frames as those using a hypermedia package where sophisticated programming was virtually impossible. Software used to develop the prototype does not have to be the same as that used to implement the final package. The final
255
Prototyping CBT
distributed version system requires the use of a flexible language that demands structured programs which are complex, but must be easily debugged and designed to be maintained. By contrast, the prototype must be produced cheaply and quickly (minor flaws are acceptable) with maintenance not an issue since the prototype will, in all likelihood, be discarded after use. Today software is already available which can offer the development team many useful prototyping facilities. Hypermedia, e.g. HyperCard for the Apple, Linkway or Guide for the IBM PC, is one type of system that has successfully been used. Care should be taken, however, that the structure of the prototype is similar to that of the intended final system. One of the dangers of hype~edia is the ease with which buttons can be created which, with lack of careful planning, can encourage not only “spaghetti” programming which is so difficult to debug, but also “spaghetti” routing, which is confusing to learners trying to navigate through a package. Some practical advantages to be gained from prototyping with hypermedia are provided by Fitzgibbon [ lo] who describes the difficulties faced in producing an interactive video for airline pilot training with Scandinavian Airlines System. He maintains that the problem is that while users and subject matter experts have very definite opinions on the content, format and feel of the final course, they tend to voice these opinions effectively only after seeing the finished product. By this time it is too late, since the designs are “set in concrete”. Fitzgibbon puts it clearly: It seems that no amount of specifications on paper, reviews and signatures can bridge the communications gap between what was originally envisaged and what was initially produced. In fact, too many intermediate stages and too much delay can often be counter-productive. By the time the lesson finally emerges everybody has forgotten what was intended. Fitzgibbon recommends the production of a prototype early in the development process. This model acted as a communication vehicle between the designer and subject matter experts, between the subject matter experts and secondary parties (validators, quality assurance supervisors, managers, students), and between the designer and programmer. Changes were made to the model at a fraction of the cost and in a fraction of the time than what would have resulted from making changes on the finished courseware. This is because three new items of technology were used, making prototyping a viable proposition: (a) digital scanners to scan existing diagrams into the lesson without the need to draw graphics; (b) phonetic audio which will speak a word processed file without the need to record provisional audio at a studio; (c) card index programs which speed the process of creating storyboards (hypermedia). WHY
IS SOFTWARE
PROTOTYPING
NOT
THE
NORM?
A range of reasons why software prototyping is important have been considered, yet it seems that although prototyping is used in software development, it is also undoubtedly the case that this is the exception rather than the rule. Birreli and Ould[8] suggest two reasons for the lack of use of prototyping. Firstly, there is a widespread conception that software prototyping is not cost-effective. Secondly, the tools for software prototyping are not readily available. Given the diversity of development schemes, one could ask how true these assumptions are. The belief that software prototyping is not cost-effective may have partial validity in organizations that produce near-perfect software every time. Some projects, however, have run into expensive difficulties which could have been avoided through a small expenditure on prototyping. It is too late when the project’s sponsors finally get to see the system, and it is not what they expected or desired. Certainly, in the case of CBT courseware development, prototyping is comparatively rare. Examine most textbooks or guides to CBT design and it is difficult to discover either a recommendation to use prototype modelling or the methodology for doing it. Perhaps one reason is that, in general, the scale of CBT development is usually smaller than for that of other software projects. Thus the resulting software does not have the updating and maintenance problems of typical utility software packages whose lifecycle may cover a number of years as they are continuously improved. If a design team is only producing a 30 minute course, project management
256
D. E.
GRAY
and T. R. BLACK
considerations may militate against prototyping when it can get sponsors or end-users to evaluate the final product and make any necessary modifications. Another argument concerns human resources and organization. Very experienced CBT development teams may operate according to a strictly defined methodology which is tried and tested. Why prototype, when users know exactly what they are going to get because they have seen it all before? Both these arguments have some validity, but they should not obscure the very real benefits that prototyping can bring to CBT courseware development through providing a cost-effective way of considering new ideas. No matter how small the project, it is surely better to get matters such as styles, formats, and functionality agreed well before the final product is completed. Even the small project cited above will cost tens of thousands of pounds. As far as the well tried approach is concerned, this presents problems as well as advantages. If the end product is so predictable should not the development team be trying something different? Think of the unfortunate end-users who have to plough through hours of the same old courseware. Standardization can be an argument for avoiding imagination, experimentation and innovation in ways of enhancing learning. CONCLUSIONS It has been noted that prototyping is an important element in traditional hardware and software engineering and proposed that there is an important role for it in courseware development. The production of CBT prototypes can enhance the interaction of the three technologies to encourage creative development: educational/learning design, software engineering and project management. While the scale of CBT development is, in general, smaller than that of software development as a whole, the difficulties inherent in CBT production are not insignificant. Indeed, the move towards a multimedia approach (often including a video and audio component such as CD-I and interactive video) means that CBT development costs may be quite considerable for a single package. As in general software development, CBT prototyping may be of most value at either the requirement or specification stage. The requirement stage is vital for finalizing preferences of those who commission the package, which can be as much an education and learning process for them as it is for designers. The next stage, design and development, requires continual communication among members of the design team: subject matter experts, courseware designers, graphics artists, video production team, etc. A set of throw-away versions for trying out ideas can be invaluable. Finally, completing the final specifications, when various aspects of the final system must be modelled and tried out, requires a flexible approach. Prototyping can be especially useful for designers to communicate the results to those responsible for producing and programming the final courseware package. Hence, development tools such as hypermedia software are very useful for producing and linking together large numbers of screens with relative ease. In the final analysis, the aim of prototyping is to not only reduce the risks associated with the courseware development but also enhance its quality as well. If used wisely, it can be a cost-effective addition to the development process. REFERENCES Engineering: A Practitioner’s Approach. McGraw-Hill International Editions. New York I. Pressman R. S., S@nzre (1988). 2. King D., Current Pructices in Sofzware Dei~eiopment. Yourdon Press, New York (1984). Use of CASE Technology. Prentice Hail. Hemei Hemnstead (1990). 3. Vonk R.. Prototvpina: The Efietive A. and Shewmake D., The role of prototypes inthe User Software Engineering (USE) methodology. In 4. Wasserman Human-Comauter Inferaction (Edited bv Preece J. and Keller L.). Prentice Hall. Englewood Cliffs, N.J. (1990). 5. Alessi S. and’Trollip S., Compiter Ea.& Instruction: Method.7 and Development. Prentice Hall, Englewood Cliffs, N.J. (1991). 6. Hansen H., Up clt Running: A Case Study of Succrssfui Systems Development. Yourdon Press, New York (1984). Los Altos, Cafif. 7. Baecker R. M. and Buxton W. A. S., Readings in Human- Computer Interaction. Morgan Kaufmann, (1987). England 8. Birreli N. and Ould M., A Practical Handbook fir Sqftware Development. Cambridge Univ. Press, Cambridge, (1985). CAL courseware: a role for computer-shy subject experts. In Designing New Systems and 9. Black T. R., Prototyping Technolonies for Lenrnina (Edited bv Mathias H., Rushbv N. and Budgett R.). Konan Pane, London (1988). 10. Fitzgibbon d., Methodology for CBT design: courseware modelling. In Symposium-on Cohputer Assisted Learning: Handbook and Abstracts. Univ. of Surrey, Guildford, England (1989).