- Email: [email protected]
Designing a Decision Support System: How Can The Designer Fit the User’s Needs?
Co p yright © I FAC \I
DESIGNING A DECISION SUPPORT SYSTEM: HOW CAN THE DESIGNER FIT THE USER'S NEEDS? D. Ackermann IlIdu strial alld Orgalli4atiu llal PHrhology L'I/It , Sll'iss Fpderal IlIstitllt p of T echnology, CH -8092 Z urirh. Sll'itu rlalld
Abstract. Based on the principles of differential and dynamic work design we developed a decision support system for evacuation planning. That system should be easy to learn. offer different ways of problem solving to prevent difficulties and delays in the evacuation planning. We started with an analysis of the scope of action and involved different samples of prospective user in the development of the program. We conceived tools and methods for the user's participation, Our experiences show that the designer should have good communicative skills and must be willing to learn form the user. It is not always easy to judge the relevancy of the prospective user's statements. A very good acceptance and the efficiency in the use of the program (and also the resulting adaptability of it) will compensate for the slightly higher expenditure in the developing process. Keywords. Decision Support Systems. user participation, system design. work psychology. problem solving. individual differences.
demonstrated by the example of the desicion support system EPILOG (Ivacuation flanning in the DialQg).
INTRODUCTION An optimal development of personality in working life is only possible if (1) individual differences in action-regulation are taken into account and if (2) possibilities exist to adapt existing worksystems or to create new worksystems. Ulich (1978. 1987a.b» stated this in his principles of differential and dynam i c work design. According to these principles dialogues have to be tailored for and by the individual to his or her own individual demands and abilities in order to improve efficiency and to give optimal chances of task orientation and achievment motivation.
STARTING POINT AND TASK Luthi et al. (1982) developed the shelter allocation planning system CASA for the Swiss Civil Defen se Organisation in Zuric~ Later on. Betschart & Kalchofner (1985) developed a first version of a program for the local evacuation planning in emergency cases. This solution was a first step but could not meet the requirements of militiamen. Therefore we tried to find a solution to make the problem solving routines of the program accessible for mi 1itiamen which have no computer or mathematical experiences. The basic idea was to construct an interactive desicion support system which enables the user to model different solutions and to compare their properties. The system should also facilitate the instruction and training of militiamen in evacuation plannin~
How can we match these requirements in dialogue design? Do these principles not only add new problems to the difficulties of designing software? Are design criteria and guidelines in combination with the developers experiences not sufficient? Many examples show that it is very puzzling to understand the prospective user's way of thinking and it is almost impossible to predict it without a profound analysis of the user's thinking process. From previous experiments we know that difficulties in human-computer interaction during task solving processes are caused by the discrepancy between individual mental representation and goal setting on the one hand and the scope of action prescribed by the dialog on the other hand (Ackermann. 1983, 1986. 1987). One possible way to solve this problem seems to be the participation of the user i n the development and design process by uSing his specific knowledge about his task and by observing his way of working. Despite the fact that the need for user participation has been recognized and accepted for a long time there is still small knowledge about how to establish effectiveley a cooperation between designers and users.
Design objectives Based on the above-mentioned concepts of work psychology and considerations of the related DSS literature (Sprague and Carl son. 1982. Benbasat & Wand. 1982. Pfeifer & Luthi. 1987. Ulich.1978) we formulated some design objectives for the development process: a) The goal of the Decision Support System is not to automatize decision making but rather to support the intuition of the desicion maker. b) The system must allow the user to confront a problem in a flexible, individual way by prOViding the possibility to manipulate the data and models in a variety of ways while going through the desicion making proceSL The system shall facilitate the potential use of methods. models and data on an individual basiL c) The system has to be easy to learn and to support training in evacuation planing. d) The dialogue has to be compatible with the individual users thinking process and has to meet
Starting from our demand to individualize work activities and worksystems we will present a possible way of solution. The approach of user participation is discussed as the basis of a paradigmatic example. Some methods and tools developped in the laboratory and tested in a field work will be 29 1
292
D, Ackermann
the criteria of user-oriented dialogue design (Ulich, 1987b): Transparency Consistency Tolerance Compat i b li ty Support Flexibility/User-Defineability Participation e) The problem should be represented in its real context and not in abstract forms. Context-relevant marks and information should be represented in the dialogue as good as possibl~ f) The main topic of this work should be the development of the user interface. I n consequence a 1gorithms were taken from the program developped by Betschart & Kalchofner (1985) and data about shelters and the population were taken from the CASA program. g) The program should be developed as a prototype to evaluate the questions (1) if it would be possible to implement the task on personal computers and (2) if it is possible to realize evacuation planning by militiamen. Causal constraints The dialogue was not intended for research in the laboratory, in contrary it had to fit the requirements of the Swiss Civil Defense Organization. We had to use the data of this institution in a compatible format and the program had to be developed for the most common accessible computers. This led to severe causal constraints: MS-DOS standard with 640 Kbytes memory and high resolution screens (640 x 400 pixels) without colours. The program and the necessary data should be restricted to two floppy discs. The use of a mouse to select menu items and other objects on the screen was excluded because it could not be assured that mice will be available in any case. The hierarchical structures of the civil defense organization had to be taken into account and we had no chances to change environmental circumstances such as abbreviations, rules documented by law or by custom. Who is the actual user of this program? One group of intended users consists of the staff of the organization. This group of users is stable in the sense that they work on their job in a professional way. The other group is formed by militiamen. People with different professional education have to be integrated in this group. They work only temporarily in the organization and on the task. That means the membership in this group is not stable, members change very often. Participating the whole staff group in the design was easy, for the militiamen we had to restrict us to representative samples.
Table
Steps of the intended developmental process with user participation
Step
Concepts/ MethodslT oals
Problem setting
ParUciD'lnts responsible Staff
Analysis of problem
Interview Cbservation Analysis of docunents
Desi~
Psychol. theory OSS theory
- outline
comic strips
Staff, selected militiamen
- dialogue
screen layout
Staff, students of the computer science department
- prototyping
pilot studies
Students, selected militiamen
- implementation
experiments
Students, militiamen
Staff, selected militiamen
Because we have already discussed the topic of problem setting, we will continue with step 2: The analysis of the problem. Analysis of the problem Usually the designer can go and have a look on the workplace to analyze the working process which he has to translate into a dialogue. In our case only two specialist were able to run the evacuation planing program CASA on the computer and to interpret the data. There was no "paper and pencil solutio~' to be observed which could have given any hints how to design the dialogue and the militiamen had up to that time never been confronted with the problem. Therefore we decided to collect documents and data about the problem and the problem solving process. We got a chart with abbreviation~ a map of the town with special marks of the organization and a dictionnary of shelters and the distribution of the population. In addition we got some instructions about how the regulations in shelter allocation should be applied. As many participants referred to the map of the town as a possible basis for evacuation planning, we concluded that this instrument could be a good representation of the contex~
PLANNING THE DEVELOPMENT In designing a program especially with user participation we are confronted with two problems: (1) The designer has to understand the task to be solved and the user has to learn which options the new tool "computer" offers for performing his specific kind of task. Designer and user have to communicate their problems of understanding and their expectations about the outcome. It is important that they learn from each other during and by the development process. This process has to be supported by suitable tools and methods to guarantee an optimal agreement as a basis for an optimal resul~ Thus we conceived a timetable for the developmental ~ rocess (Table 1). (2) Designers may not assume that an existing worksystem fit the requirements of work psychology (see Uli ch, 1987 a, b). Because our intent i on was to create a new work system we will not discuss this topic here.
As we had no information how an evacuation planning would have been done by human subjects without the existing program on the mainframe, we decided to base the dialogue on evidence from psychological research on problem solVing (D6rner, 1974). The design of the scope of action was grounded in the cycle of problem solving formulated by Daenzer (1978) and is depicted in Table 2. Outline of the Dialogue Proceeding from this outline of the scope of action we draw comic-strips for the different dialogue steps (Fig. 1). We used the map of the town as an essential basis of planning. We presented this comic-strips to some staff members and to some selected militiamen. They had to evacuate an area with the presented information and dialogue com-
293
Designin g a Decisio n Suppo rt SI'stem
Phase
Stage in the process
Intended scope of action/ Dialogue-structure
Target definition
Analysis of situation
define areas to be evacuated define prC>'1ibited areas define accessible shelter cat (quality of shelters)
Fornulation of objectives
primary 92!!.1 (minimum requirement) evacuate all endangered pers. secllldary ~ (exaJlllles) I shortest ways optimize shelter quality minimize collecting paints
Synthesis of passible solutions
Define collecting paints shelters within easy reaching distances degree of overcrow ding of the shelters
Test if solution corrputable
Test if enou9'l shelterplaces are defined
ly. Confronted with the problem that in some cases poeple could not discriminate the cursor from the surrounding fields, we experimented with different cursor shapes. We got evidence, that it is possible to present the map of the town on screen if we provide comfortable means to zoom in and out and to move the map arround in the screen window. Implementation
Solution
H
Calrpute solution by transhipment algorithn
!selection
Evaluation of corrputed solution Decision
Corrpare solution according secondary goals All goals met? If not, try to change the formulation of
objectives and calculate new solution or adapt your intentions to reality
The implementation was done as a diploma thesis in computer science by Bosze (1986). He conceived a very flexible software architecture enabling the deSigner to adapt and change the system without great effor~ The architecture is based on a modular design. The user-interface was implemented firs~ Each module was tested after its implementation together with students of the computer science department and afterwards with samples of militiamen delegated by the Civil Defense Organization. The pilot study with the interface-module lead to some small improvements. The selection of the area to be zoomed out caused some problems due to cursor shapes. A rectangular rubber-band cursor which embraced the selected area on screen solved the proble~ Some partiCipants suggested the use of function keys for selecting menu items instead of selection by cursor. The speed of cursor movement puzzled some partiCipants. Therefore the speed of cursor movement is now selectable (fast/slow). We observed some errors in conducting the displacement of the selected area of the map. We think this was due to different mental representations: Is the window moved over the map or is the map moved under the window? Now, when the user starts the program he can select between the two displacement modes •
...... 1 • • ~ M.nu.
she lt ers
=;»
(s elect f ro m ",)
)
b~;"
~.v,"\cua.tO::td ,.
,
"
,
,'. ,) ,~) .•.!~~: :, '
2;?:. ?....J" .. .J 1/ ~/.r"~.s 2¥.-(. 7"'/50 .. 5
Fig. 1. Example of the comic strip series. The user has defined some blocks to be evacuated. mands. Every step was discussed with them and we evaluated problems and uncertainties together. This approach rendered evidence which kind of information the user requires. We tried to formulate the steps of the dialogue according to these results. Furthermore we learned a lot about the presentation of information. For example one partiCipant asked if the computer could not show him the distribution of free shelter places on the map of the town. Such an overview would facilitate his search for possible solutions. Later on this feature proved to be very helpful in the planning proceSL Screen Layout of Dialogue In the next step we tested the feasibility of our intentions with a drawing programm. Some screen layouts have been delineated and tested to see whether all details of the town-map are visible and if people will get the necessary information easi-
\~- - ~"-~c '
' ~... "
-.\ .
\
I:
\ ,.
,'.
Fig. 2. Picture of the Screen: The blocks to be evacuated and the prohibited areas are defined " . .. We •• ay., -) Sst lYe varIant)
·. .'D·· . ·····,_.
IoIlU.t .......
"-" ...
(selt:ct fr o m "'- )
) ( menu)
:iYY,
az ,h ..... }, !
.. a t
runk' .
T .. s ~ ..
1 2; T ..... II .. ...
(ref e r e nce varlant)
-:::·D'·'·"·& ...,-..
r .. at
:iez
-..rd....... rohv.tw-..-.
O'l,
a
, (DE: t
i s o f ac tIv e var 13ntj
Fig. 3. Picture of the screen: Some detai ls of two solutions are represented
294
D. Ackcrmann
Menu
Structure and Manual
The screen shows the menue in the upper right corner of the screen (e. g. Fig. 2) with the relevant items according to the stage of the process. Without a plan of the menue structure the user has to memorize the dependences and pathes between the displayed menues. In an ideal design, this plan should be represented on screen but we had no place on the screen to implement such a feature. Furthermore we were afraid that an additional pop up window with this information would lead to information overload and therefore disturb the user. We solved this problem by designing a manual, This manual includes a plan of the menue hierarchy with the possible pathes among the menues. From the begining to the end the manual serves as a tutorial. The prospective users also participated in writing this manual. They read draft versions and told us what they didn't understand and which kind of information they would prefer. When the novices have become experts. an index at the end of the manual will help them to find selected topics.
not use any psychological tests.
The results are reported here only as far as they are relevant to the topic of design and individual differences. Each participant was able to solve the first evacuation task. The second one was much more difficult and we observed many differencies in the solutions. Two participants reached the goal only with the he 1p of the experi menter.
Benefits and Problems in User Participation User participation proved to be a very valuable tool in the presented design process. We already discussed the importance of learning from each other during the development. Sometimes the designer tries to convince the user that the designers solution is best as he doesn't like to change his program. On the other side a user may stick to his unefficient way of thinking which is not even understood by his colleagues. Beside the importance of the learning process and the need of good communicative skills for the designer, he has also to clarify the relevance of the user's statements. Sometimes the user mixes some other problems up with the presented dialog. We got a lot of statements about organizational problems in the evacuation process which were not relevant to the design of the program. We got also some unrealistic suggestions such as to implement a command language instead of a menu. This was proposed by a man who had once heard about UNIX but never seen such a system. A suggestion from one single user may be a valuable hint. In some cases the designer may decide on his own or in doubt he has to ask other users or specialists. We like to stress here again that an existing worksystem is no guarantee that it will fit the criteria of work psychology (see e. g. Spinas, Troy, Ulich, 1983, Ulich, 1987).
Fig. 4. Problem solving process: The different selected areas of the map are reconstructed and the displacement of the map is depicted (arrow)
EXPERIMENTAL EVALUATION After the debugging of the program and the pilot studies, we tested it experimentally. The Swiss Civil Defense organisation defined two evacuation tasks for this experiment. One was considered to be easy and the other was difficult to solve. The 10 participants have been selected by the organisation from obligatory training courses. The organization agreed to evaluate the participants solutions after the experi ments. Fig. 5. Reconstruction of the problem solving process: Displacement and zooming in and out of the map by trial and error The participants were instructed by the experimenter. They had to get acquainted with the manual and the program. They solved a first preliminary task designed by the experimenter in order to learn the use of the program. When they had solved this first task, they were confronted with the evacuation orders of the Swiss Civil Defense Organisation. They were told to think aloud if possible and tell the experimenter their intentions. Afterwards we made an interview about their task solving proces~ As the participants came not voluntary we di~
Fig. 4 and Fig. 5 show typical differences in the search strategies of two participants. One (Fig. 4) tries to solve the problem deliberately. His second step is to select the map in overview mode to make use of the facility of program to depict the distribution of free shelters graphically. He gets a rat her optimal solution, The other participant (Fig. 5) moves the map around and around and selects different extensions of the map to be represented. The impress i on is, that he gets 1os tin the
295
Designing a Decision Support Svstem
map and loses overview. In his solution the persons to be evacuated are scattered "randomly" all over the area. The prob 1em of "1 os i ng overvi ew" can be solved in two ways: (1) One can try to implement a knowledge-based system which is able to realize if the user does not make any progress or (2) one can instruct the user that he has to use first the overview facility of the map which will indicate areas with free shelters. The second suggestion solves the problem in a very easy way and saves a lot of work and memory.
main menu
~
Discussion From these results and the comments of the staff we concluded that the developed program met the abovementioned requirements. Unfortunately the organization's commanders renounced to evaluate and judge the solutions of the experiment as they realized that they had a lot of organizational problems not defined such as means for transportation. number of persons looking after the people to be evacuated and so on. They greatly appreciate the possibility to model the evacuation planning but the program also pointed out some organizational problems which have to be sol ved fi rst.
The students were able to solve the rather easy problem faster with the representation in tabular form although they had no previous experience in evacuation planning. For the more difficult task some get lost in the tables. We concluded that for certain users and easy problems with only a few places to be evacuated the problem solving process
change representation of map
~
/
select / deselect bl ocks to be evacuated
~ define prohibited areas
/~ define prohibited
select accessible shelter categories
areas by distance
defi ne prohtited areas by distance
abe,f1teidneaPreroahsi~
define prohibited areas
se l ect accessible shelter categories
l~~k' for free she 1te.r places define causal
-
r[>- - - I
--'~]""
determine factor of allowed overc rowding of shelters ~ change representation of map
\ '"c
10 ..... I~
lE ..... \
I
I
depict distribution of free shelters graphical l y in the map
r-{>- - - - - - - -
I
- -,.--.-.select / deselct collection points
lock up number of free shelters of a block in table
selec~lect
: '" \ 1:5 collection points
---.
.... I~
select blocks with free shelters
select blocks with free she I ters
....,
I'~
\
Our design was based on the map of the town due to our intention to support the evacuation planning in an appropriate context. Starting our project we missed the possibility to observe people doing the task by paper and pencil. Therefore we made a short follow up study in the laboratory with our students. Six students had to solve the abovementioned evacuation problems formulated by the Swiss Civil Defense Organisation by paper and pencil. The necessary data were provided by the computer as tables.
I
problem definition
\~
A FOLLOW-UP STUDY: I S OU R DESIGN OPTIHAL?
definition
I
There was also another problem caused by a well known phenomenon of perception: The participants prefered larger blocks with shelters on the map to smaller ones if they were marked in the same way. This marks indicate the aproximate amount of free shelters. The participants have been misleaded by the "1 aw of the bi gger shape". one of the we 11 known "Gestalt Principles". We see two possible solutions: (1) The participants get experienced about this misleading indicator or (2) we depict the number of free shelters as bars in the blocks and not by di fferent shadows. We were also interested in investigating how the scope of action implemented in the menu structure was used. Fig. 6 presents this "diagram of navigation in the dialogue". The first part shows the phase of the problem definition and the different ways to do it. To our surprise we had to realize that the menu "search for free she 1ters" was not often used. The participants told us afterwards that they could not figure out what the title (in German "SP suchen") of the menu shou 1d mean. The second part of the diagram shows different iterations in defining and evaluating possible solutions. There are two apparently different ways in defining solutions: One group prefers to have control and looks for free shelter places in advance and select suitable places as collection points. The other group picked the collection points from the map by geographical reasons.
~problem
change representation of map
test i f su f ficient shelter places for calculating the solution ? • ________ calculate solution
-
1 I
I._<}-
I I I"'c:
IS
act i ve variant
I~ I~ I~
1_<]- ___ _
~ .----
save / res tore .... reference vari ant decision possible?
Fig. 6. "Menu-navigation Diagram" showing individual differences in using the dialogue
296
D. Ackermann
should better be based on tables than on a map. The user should have the possibility to select between tables and the map as support of the planning process. CONCLUSION The results show that it is possible to realize a flexible structure of the software which can be easily adapted to individual differences in the users working styles even within the very limited capacities of a personal computer. Furthermore the results show the importance of user participation for the design of the dialog. This approach assures that individual differences are detected and taken into cons i derat ion. The "corn i c-stri ps" of the di alog turned out to be a valuable tool in designing the dialog and in stimulating the discussion with the prospective users. The designer should have good communicative skills and must be willing to learn form the use~ It is not always easy to judge the relevancy of the prospective user's statements. A good acceptance and the efficiency in the use of the program (and also the resulting adaptability of it) will compensate for the (slightly) higher expenditure in the developing proceSL REFEREl-JCES Ackermann, O. (1983). Robi Otter oder die Suche nach dem operativen Abbildsystem. Interner Bericht uber die Studienarbeiten SS 1983. Ackermann, D. (1986). A pilot study on the effects of individualization in man-computer-interaction. In: Manci ni, G. Johannsen, G. & Martenson, L. (Eds.): Analysis, Design and Evaluation of ManMachine-SystemL Proceedings of the 2nd IFAC/IFIP /lFORS/IEA Conference, Varese, Italy, 10. - 12. September 1985. Oxford/New York: Pergamon Press, 293 - 297. Ackermann, D. (1987). Handlungsspielraum, lllentale Reprasentation und Handlungsregulation am Beispiel der l~ensch-Compute- I nterakt ion. Doctora 1 thes is, University of Bern, Switzerland. Betschart, 1·1. & Ka lchofner, S. (1985). Evakuationsplanung im Zivilschutz. Unpublished Semester Termwork. Institut fur Operations Research ETH Zuri ch. Basze, J. Z.: EPILOG: Evakuationsplanung im Dialog. Unpublished Diploma Thesis, Department of Computer Science, Swiss Federal Institute of Technology, 1986. Benbasat, I. & Wand, Y. (1982). A dialogue generator and its use in DSS Design. Information & l~anagement, 2, 231 - 241. Daenzer, W. F. (1978). Systems engineering. Kaln: Hanstein. Darner, D. (1974). Die kognitive Organisation beim Prob 1em1asen. Bern: Huber. Luthi, H.-J.,
J., Frehner, R., Polymeris, R. (1982). Computerunterstutzte Pl anung im Zivilschutz - Zuweisungsplanung mit CASA. Output, 11, 19 - 22. l~ayer,
& Fi sch, A.
Pfeifer, R. & Luthi, H.-J. (1987). Desicion support systems and expert systems: A complemantary relationship? In: Tackenberg, C. A. (Ed.): Expert systems and artificial intelligence in desicion support systems. Amsterdam: Reidel, 41 - 51. Spinas, P. Troy, N. & Ulich, E. (1983). Leitfaden zur Einfuhrung und Gestaltung von Arbeit an Bild-
schirmsystemen. Munchen: VW Publikationen. Spi nas, P. (1987). Arbei tspsycho logi sche Aspekte der Benutzerfreundlichkeit von Bildschirmsystemen. Doctora 1 Thes i s, Uni vers i ty of Bern Swi tzer 1and. Sprague, R. H. & Carlson, E. 0.: Building effective decision support systems. Englewood Cliffs: Prentice Hall, 1982. Ulich, E. (1978). Ueber das Prinzip der differentiellen Arbeitsgestaltung. Industrielle Organisat ion, 47, 566 - 568. Ulich, E. (1987a): Individual differences in human -computer interaction: concepts and research findings. In: Salvendy. G. (Ed): Cognitive engineering in the design of human-computer interaction and expert systems. Amsterdam: Elsevier, 1987. Ulich, E. (1987b): Some aspects of user-oriented dialogue design. In: Docherty, P., Fuchs-Kittowski, K.. Kolm, P. & l~athiassen, L. (Eds): System design for human development and productivity: Participation and Beyond. 33 - 47.
I thank PO Dr. H.-J. Luthi for his cooperation, J. Z. Boesze for implementing the interface and H. Schmi tz for ass i st i ng in the experi ments. Prof. Dr. E. Ulich supported this work and gave many helpful comments on this manuscript. IBM Switzerland supported this work with a PC for the development.
Authors present adress: International Business Machines Corporation Thomas J. Watson Research Center P. O. Box 218 Yorktown Heights, New York 10598
DESIGNING A DECISION SUPPORT SYSTEM: HOW CAN THE DESIGNER FIT THE USER'S NEEDS? D. Ackermann IlIdu strial alld Orgalli4atiu llal PHrhology L'I/It , Sll'iss Fpderal IlIstitllt p of T echnology, CH -8092 Z urirh. Sll'itu rlalld
Abstract. Based on the principles of differential and dynamic work design we developed a decision support system for evacuation planning. That system should be easy to learn. offer different ways of problem solving to prevent difficulties and delays in the evacuation planning. We started with an analysis of the scope of action and involved different samples of prospective user in the development of the program. We conceived tools and methods for the user's participation, Our experiences show that the designer should have good communicative skills and must be willing to learn form the user. It is not always easy to judge the relevancy of the prospective user's statements. A very good acceptance and the efficiency in the use of the program (and also the resulting adaptability of it) will compensate for the slightly higher expenditure in the developing process. Keywords. Decision Support Systems. user participation, system design. work psychology. problem solving. individual differences.
demonstrated by the example of the desicion support system EPILOG (Ivacuation flanning in the DialQg).
INTRODUCTION An optimal development of personality in working life is only possible if (1) individual differences in action-regulation are taken into account and if (2) possibilities exist to adapt existing worksystems or to create new worksystems. Ulich (1978. 1987a.b» stated this in his principles of differential and dynam i c work design. According to these principles dialogues have to be tailored for and by the individual to his or her own individual demands and abilities in order to improve efficiency and to give optimal chances of task orientation and achievment motivation.
STARTING POINT AND TASK Luthi et al. (1982) developed the shelter allocation planning system CASA for the Swiss Civil Defen se Organisation in Zuric~ Later on. Betschart & Kalchofner (1985) developed a first version of a program for the local evacuation planning in emergency cases. This solution was a first step but could not meet the requirements of militiamen. Therefore we tried to find a solution to make the problem solving routines of the program accessible for mi 1itiamen which have no computer or mathematical experiences. The basic idea was to construct an interactive desicion support system which enables the user to model different solutions and to compare their properties. The system should also facilitate the instruction and training of militiamen in evacuation plannin~
How can we match these requirements in dialogue design? Do these principles not only add new problems to the difficulties of designing software? Are design criteria and guidelines in combination with the developers experiences not sufficient? Many examples show that it is very puzzling to understand the prospective user's way of thinking and it is almost impossible to predict it without a profound analysis of the user's thinking process. From previous experiments we know that difficulties in human-computer interaction during task solving processes are caused by the discrepancy between individual mental representation and goal setting on the one hand and the scope of action prescribed by the dialog on the other hand (Ackermann. 1983, 1986. 1987). One possible way to solve this problem seems to be the participation of the user i n the development and design process by uSing his specific knowledge about his task and by observing his way of working. Despite the fact that the need for user participation has been recognized and accepted for a long time there is still small knowledge about how to establish effectiveley a cooperation between designers and users.
Design objectives Based on the above-mentioned concepts of work psychology and considerations of the related DSS literature (Sprague and Carl son. 1982. Benbasat & Wand. 1982. Pfeifer & Luthi. 1987. Ulich.1978) we formulated some design objectives for the development process: a) The goal of the Decision Support System is not to automatize decision making but rather to support the intuition of the desicion maker. b) The system must allow the user to confront a problem in a flexible, individual way by prOViding the possibility to manipulate the data and models in a variety of ways while going through the desicion making proceSL The system shall facilitate the potential use of methods. models and data on an individual basiL c) The system has to be easy to learn and to support training in evacuation planing. d) The dialogue has to be compatible with the individual users thinking process and has to meet
Starting from our demand to individualize work activities and worksystems we will present a possible way of solution. The approach of user participation is discussed as the basis of a paradigmatic example. Some methods and tools developped in the laboratory and tested in a field work will be 29 1
292
D, Ackermann
the criteria of user-oriented dialogue design (Ulich, 1987b): Transparency Consistency Tolerance Compat i b li ty Support Flexibility/User-Defineability Participation e) The problem should be represented in its real context and not in abstract forms. Context-relevant marks and information should be represented in the dialogue as good as possibl~ f) The main topic of this work should be the development of the user interface. I n consequence a 1gorithms were taken from the program developped by Betschart & Kalchofner (1985) and data about shelters and the population were taken from the CASA program. g) The program should be developed as a prototype to evaluate the questions (1) if it would be possible to implement the task on personal computers and (2) if it is possible to realize evacuation planning by militiamen. Causal constraints The dialogue was not intended for research in the laboratory, in contrary it had to fit the requirements of the Swiss Civil Defense Organization. We had to use the data of this institution in a compatible format and the program had to be developed for the most common accessible computers. This led to severe causal constraints: MS-DOS standard with 640 Kbytes memory and high resolution screens (640 x 400 pixels) without colours. The program and the necessary data should be restricted to two floppy discs. The use of a mouse to select menu items and other objects on the screen was excluded because it could not be assured that mice will be available in any case. The hierarchical structures of the civil defense organization had to be taken into account and we had no chances to change environmental circumstances such as abbreviations, rules documented by law or by custom. Who is the actual user of this program? One group of intended users consists of the staff of the organization. This group of users is stable in the sense that they work on their job in a professional way. The other group is formed by militiamen. People with different professional education have to be integrated in this group. They work only temporarily in the organization and on the task. That means the membership in this group is not stable, members change very often. Participating the whole staff group in the design was easy, for the militiamen we had to restrict us to representative samples.
Table
Steps of the intended developmental process with user participation
Step
Concepts/ MethodslT oals
Problem setting
ParUciD'lnts responsible Staff
Analysis of problem
Interview Cbservation Analysis of docunents
Desi~
Psychol. theory OSS theory
- outline
comic strips
Staff, selected militiamen
- dialogue
screen layout
Staff, students of the computer science department
- prototyping
pilot studies
Students, selected militiamen
- implementation
experiments
Students, militiamen
Staff, selected militiamen
Because we have already discussed the topic of problem setting, we will continue with step 2: The analysis of the problem. Analysis of the problem Usually the designer can go and have a look on the workplace to analyze the working process which he has to translate into a dialogue. In our case only two specialist were able to run the evacuation planing program CASA on the computer and to interpret the data. There was no "paper and pencil solutio~' to be observed which could have given any hints how to design the dialogue and the militiamen had up to that time never been confronted with the problem. Therefore we decided to collect documents and data about the problem and the problem solving process. We got a chart with abbreviation~ a map of the town with special marks of the organization and a dictionnary of shelters and the distribution of the population. In addition we got some instructions about how the regulations in shelter allocation should be applied. As many participants referred to the map of the town as a possible basis for evacuation planning, we concluded that this instrument could be a good representation of the contex~
PLANNING THE DEVELOPMENT In designing a program especially with user participation we are confronted with two problems: (1) The designer has to understand the task to be solved and the user has to learn which options the new tool "computer" offers for performing his specific kind of task. Designer and user have to communicate their problems of understanding and their expectations about the outcome. It is important that they learn from each other during and by the development process. This process has to be supported by suitable tools and methods to guarantee an optimal agreement as a basis for an optimal resul~ Thus we conceived a timetable for the developmental ~ rocess (Table 1). (2) Designers may not assume that an existing worksystem fit the requirements of work psychology (see Uli ch, 1987 a, b). Because our intent i on was to create a new work system we will not discuss this topic here.
As we had no information how an evacuation planning would have been done by human subjects without the existing program on the mainframe, we decided to base the dialogue on evidence from psychological research on problem solVing (D6rner, 1974). The design of the scope of action was grounded in the cycle of problem solving formulated by Daenzer (1978) and is depicted in Table 2. Outline of the Dialogue Proceeding from this outline of the scope of action we draw comic-strips for the different dialogue steps (Fig. 1). We used the map of the town as an essential basis of planning. We presented this comic-strips to some staff members and to some selected militiamen. They had to evacuate an area with the presented information and dialogue com-
293
Designin g a Decisio n Suppo rt SI'stem
Phase
Stage in the process
Intended scope of action/ Dialogue-structure
Target definition
Analysis of situation
define areas to be evacuated define prC>'1ibited areas define accessible shelter cat (quality of shelters)
Fornulation of objectives
primary 92!!.1 (minimum requirement) evacuate all endangered pers. secllldary ~ (exaJlllles) I shortest ways optimize shelter quality minimize collecting paints
Synthesis of passible solutions
Define collecting paints shelters within easy reaching distances degree of overcrow ding of the shelters
Test if solution corrputable
Test if enou9'l shelterplaces are defined
ly. Confronted with the problem that in some cases poeple could not discriminate the cursor from the surrounding fields, we experimented with different cursor shapes. We got evidence, that it is possible to present the map of the town on screen if we provide comfortable means to zoom in and out and to move the map arround in the screen window. Implementation
Solution
H
Calrpute solution by transhipment algorithn
!selection
Evaluation of corrputed solution Decision
Corrpare solution according secondary goals All goals met? If not, try to change the formulation of
objectives and calculate new solution or adapt your intentions to reality
The implementation was done as a diploma thesis in computer science by Bosze (1986). He conceived a very flexible software architecture enabling the deSigner to adapt and change the system without great effor~ The architecture is based on a modular design. The user-interface was implemented firs~ Each module was tested after its implementation together with students of the computer science department and afterwards with samples of militiamen delegated by the Civil Defense Organization. The pilot study with the interface-module lead to some small improvements. The selection of the area to be zoomed out caused some problems due to cursor shapes. A rectangular rubber-band cursor which embraced the selected area on screen solved the proble~ Some partiCipants suggested the use of function keys for selecting menu items instead of selection by cursor. The speed of cursor movement puzzled some partiCipants. Therefore the speed of cursor movement is now selectable (fast/slow). We observed some errors in conducting the displacement of the selected area of the map. We think this was due to different mental representations: Is the window moved over the map or is the map moved under the window? Now, when the user starts the program he can select between the two displacement modes •
...... 1 • • ~ M.nu.
she lt ers
=;»
(s elect f ro m ",)
)
b~;"
~.v,"\cua.tO::td ,.
,
"
,
,'. ,) ,~) .•.!~~: :, '
2;?:. ?....J" .. .J 1/ ~/.r"~.s 2¥.-(. 7"'/50 .. 5
Fig. 1. Example of the comic strip series. The user has defined some blocks to be evacuated. mands. Every step was discussed with them and we evaluated problems and uncertainties together. This approach rendered evidence which kind of information the user requires. We tried to formulate the steps of the dialogue according to these results. Furthermore we learned a lot about the presentation of information. For example one partiCipant asked if the computer could not show him the distribution of free shelter places on the map of the town. Such an overview would facilitate his search for possible solutions. Later on this feature proved to be very helpful in the planning proceSL Screen Layout of Dialogue In the next step we tested the feasibility of our intentions with a drawing programm. Some screen layouts have been delineated and tested to see whether all details of the town-map are visible and if people will get the necessary information easi-
\~- - ~"-~c '
' ~... "
-.\ .
\
I:
\ ,.
,'.
Fig. 2. Picture of the Screen: The blocks to be evacuated and the prohibited areas are defined " . .. We •• ay., -) Sst lYe varIant)
·. .'D·· . ·····,_.
IoIlU.t .......
"-" ...
(selt:ct fr o m "'- )
) ( menu)
:iYY,
az ,h ..... }, !
.. a t
runk' .
T .. s ~ ..
1 2; T ..... II .. ...
(ref e r e nce varlant)
-:::·D'·'·"·& ...,-..
r .. at
:iez
-..rd....... rohv.tw-..-.
O'l,
a
, (DE: t
i s o f ac tIv e var 13ntj
Fig. 3. Picture of the screen: Some detai ls of two solutions are represented
294
D. Ackcrmann
Menu
Structure and Manual
The screen shows the menue in the upper right corner of the screen (e. g. Fig. 2) with the relevant items according to the stage of the process. Without a plan of the menue structure the user has to memorize the dependences and pathes between the displayed menues. In an ideal design, this plan should be represented on screen but we had no place on the screen to implement such a feature. Furthermore we were afraid that an additional pop up window with this information would lead to information overload and therefore disturb the user. We solved this problem by designing a manual, This manual includes a plan of the menue hierarchy with the possible pathes among the menues. From the begining to the end the manual serves as a tutorial. The prospective users also participated in writing this manual. They read draft versions and told us what they didn't understand and which kind of information they would prefer. When the novices have become experts. an index at the end of the manual will help them to find selected topics.
not use any psychological tests.
The results are reported here only as far as they are relevant to the topic of design and individual differences. Each participant was able to solve the first evacuation task. The second one was much more difficult and we observed many differencies in the solutions. Two participants reached the goal only with the he 1p of the experi menter.
Benefits and Problems in User Participation User participation proved to be a very valuable tool in the presented design process. We already discussed the importance of learning from each other during the development. Sometimes the designer tries to convince the user that the designers solution is best as he doesn't like to change his program. On the other side a user may stick to his unefficient way of thinking which is not even understood by his colleagues. Beside the importance of the learning process and the need of good communicative skills for the designer, he has also to clarify the relevance of the user's statements. Sometimes the user mixes some other problems up with the presented dialog. We got a lot of statements about organizational problems in the evacuation process which were not relevant to the design of the program. We got also some unrealistic suggestions such as to implement a command language instead of a menu. This was proposed by a man who had once heard about UNIX but never seen such a system. A suggestion from one single user may be a valuable hint. In some cases the designer may decide on his own or in doubt he has to ask other users or specialists. We like to stress here again that an existing worksystem is no guarantee that it will fit the criteria of work psychology (see e. g. Spinas, Troy, Ulich, 1983, Ulich, 1987).
Fig. 4. Problem solving process: The different selected areas of the map are reconstructed and the displacement of the map is depicted (arrow)
EXPERIMENTAL EVALUATION After the debugging of the program and the pilot studies, we tested it experimentally. The Swiss Civil Defense organisation defined two evacuation tasks for this experiment. One was considered to be easy and the other was difficult to solve. The 10 participants have been selected by the organisation from obligatory training courses. The organization agreed to evaluate the participants solutions after the experi ments. Fig. 5. Reconstruction of the problem solving process: Displacement and zooming in and out of the map by trial and error The participants were instructed by the experimenter. They had to get acquainted with the manual and the program. They solved a first preliminary task designed by the experimenter in order to learn the use of the program. When they had solved this first task, they were confronted with the evacuation orders of the Swiss Civil Defense Organisation. They were told to think aloud if possible and tell the experimenter their intentions. Afterwards we made an interview about their task solving proces~ As the participants came not voluntary we di~
Fig. 4 and Fig. 5 show typical differences in the search strategies of two participants. One (Fig. 4) tries to solve the problem deliberately. His second step is to select the map in overview mode to make use of the facility of program to depict the distribution of free shelters graphically. He gets a rat her optimal solution, The other participant (Fig. 5) moves the map around and around and selects different extensions of the map to be represented. The impress i on is, that he gets 1os tin the
295
Designing a Decision Support Svstem
map and loses overview. In his solution the persons to be evacuated are scattered "randomly" all over the area. The prob 1em of "1 os i ng overvi ew" can be solved in two ways: (1) One can try to implement a knowledge-based system which is able to realize if the user does not make any progress or (2) one can instruct the user that he has to use first the overview facility of the map which will indicate areas with free shelters. The second suggestion solves the problem in a very easy way and saves a lot of work and memory.
main menu
~
Discussion From these results and the comments of the staff we concluded that the developed program met the abovementioned requirements. Unfortunately the organization's commanders renounced to evaluate and judge the solutions of the experiment as they realized that they had a lot of organizational problems not defined such as means for transportation. number of persons looking after the people to be evacuated and so on. They greatly appreciate the possibility to model the evacuation planning but the program also pointed out some organizational problems which have to be sol ved fi rst.
The students were able to solve the rather easy problem faster with the representation in tabular form although they had no previous experience in evacuation planning. For the more difficult task some get lost in the tables. We concluded that for certain users and easy problems with only a few places to be evacuated the problem solving process
change representation of map
~
/
select / deselect bl ocks to be evacuated
~ define prohibited areas
/~ define prohibited
select accessible shelter categories
areas by distance
defi ne prohtited areas by distance
abe,f1teidneaPreroahsi~
define prohibited areas
se l ect accessible shelter categories
l~~k' for free she 1te.r places define causal
-
r[>- - - I
--'~]""
determine factor of allowed overc rowding of shelters ~ change representation of map
\ '"c
10 ..... I~
lE ..... \
I
I
depict distribution of free shelters graphical l y in the map
r-{>- - - - - - - -
I
- -,.--.-.select / deselct collection points
lock up number of free shelters of a block in table
selec~lect
: '" \ 1:5 collection points
---.
.... I~
select blocks with free shelters
select blocks with free she I ters
....,
I'~
\
Our design was based on the map of the town due to our intention to support the evacuation planning in an appropriate context. Starting our project we missed the possibility to observe people doing the task by paper and pencil. Therefore we made a short follow up study in the laboratory with our students. Six students had to solve the abovementioned evacuation problems formulated by the Swiss Civil Defense Organisation by paper and pencil. The necessary data were provided by the computer as tables.
I
problem definition
\~
A FOLLOW-UP STUDY: I S OU R DESIGN OPTIHAL?
definition
I
There was also another problem caused by a well known phenomenon of perception: The participants prefered larger blocks with shelters on the map to smaller ones if they were marked in the same way. This marks indicate the aproximate amount of free shelters. The participants have been misleaded by the "1 aw of the bi gger shape". one of the we 11 known "Gestalt Principles". We see two possible solutions: (1) The participants get experienced about this misleading indicator or (2) we depict the number of free shelters as bars in the blocks and not by di fferent shadows. We were also interested in investigating how the scope of action implemented in the menu structure was used. Fig. 6 presents this "diagram of navigation in the dialogue". The first part shows the phase of the problem definition and the different ways to do it. To our surprise we had to realize that the menu "search for free she 1ters" was not often used. The participants told us afterwards that they could not figure out what the title (in German "SP suchen") of the menu shou 1d mean. The second part of the diagram shows different iterations in defining and evaluating possible solutions. There are two apparently different ways in defining solutions: One group prefers to have control and looks for free shelter places in advance and select suitable places as collection points. The other group picked the collection points from the map by geographical reasons.
~problem
change representation of map
test i f su f ficient shelter places for calculating the solution ? • ________ calculate solution
-
1 I
I._<}-
I I I"'c:
IS
act i ve variant
I~ I~ I~
1_<]- ___ _
~ .----
save / res tore .... reference vari ant decision possible?
Fig. 6. "Menu-navigation Diagram" showing individual differences in using the dialogue
296
D. Ackermann
should better be based on tables than on a map. The user should have the possibility to select between tables and the map as support of the planning process. CONCLUSION The results show that it is possible to realize a flexible structure of the software which can be easily adapted to individual differences in the users working styles even within the very limited capacities of a personal computer. Furthermore the results show the importance of user participation for the design of the dialog. This approach assures that individual differences are detected and taken into cons i derat ion. The "corn i c-stri ps" of the di alog turned out to be a valuable tool in designing the dialog and in stimulating the discussion with the prospective users. The designer should have good communicative skills and must be willing to learn form the use~ It is not always easy to judge the relevancy of the prospective user's statements. A good acceptance and the efficiency in the use of the program (and also the resulting adaptability of it) will compensate for the (slightly) higher expenditure in the developing proceSL REFEREl-JCES Ackermann, O. (1983). Robi Otter oder die Suche nach dem operativen Abbildsystem. Interner Bericht uber die Studienarbeiten SS 1983. Ackermann, D. (1986). A pilot study on the effects of individualization in man-computer-interaction. In: Manci ni, G. Johannsen, G. & Martenson, L. (Eds.): Analysis, Design and Evaluation of ManMachine-SystemL Proceedings of the 2nd IFAC/IFIP /lFORS/IEA Conference, Varese, Italy, 10. - 12. September 1985. Oxford/New York: Pergamon Press, 293 - 297. Ackermann, D. (1987). Handlungsspielraum, lllentale Reprasentation und Handlungsregulation am Beispiel der l~ensch-Compute- I nterakt ion. Doctora 1 thes is, University of Bern, Switzerland. Betschart, 1·1. & Ka lchofner, S. (1985). Evakuationsplanung im Zivilschutz. Unpublished Semester Termwork. Institut fur Operations Research ETH Zuri ch. Basze, J. Z.: EPILOG: Evakuationsplanung im Dialog. Unpublished Diploma Thesis, Department of Computer Science, Swiss Federal Institute of Technology, 1986. Benbasat, I. & Wand, Y. (1982). A dialogue generator and its use in DSS Design. Information & l~anagement, 2, 231 - 241. Daenzer, W. F. (1978). Systems engineering. Kaln: Hanstein. Darner, D. (1974). Die kognitive Organisation beim Prob 1em1asen. Bern: Huber. Luthi, H.-J.,
J., Frehner, R., Polymeris, R. (1982). Computerunterstutzte Pl anung im Zivilschutz - Zuweisungsplanung mit CASA. Output, 11, 19 - 22. l~ayer,
& Fi sch, A.
Pfeifer, R. & Luthi, H.-J. (1987). Desicion support systems and expert systems: A complemantary relationship? In: Tackenberg, C. A. (Ed.): Expert systems and artificial intelligence in desicion support systems. Amsterdam: Reidel, 41 - 51. Spinas, P. Troy, N. & Ulich, E. (1983). Leitfaden zur Einfuhrung und Gestaltung von Arbeit an Bild-
schirmsystemen. Munchen: VW Publikationen. Spi nas, P. (1987). Arbei tspsycho logi sche Aspekte der Benutzerfreundlichkeit von Bildschirmsystemen. Doctora 1 Thes i s, Uni vers i ty of Bern Swi tzer 1and. Sprague, R. H. & Carlson, E. 0.: Building effective decision support systems. Englewood Cliffs: Prentice Hall, 1982. Ulich, E. (1978). Ueber das Prinzip der differentiellen Arbeitsgestaltung. Industrielle Organisat ion, 47, 566 - 568. Ulich, E. (1987a): Individual differences in human -computer interaction: concepts and research findings. In: Salvendy. G. (Ed): Cognitive engineering in the design of human-computer interaction and expert systems. Amsterdam: Elsevier, 1987. Ulich, E. (1987b): Some aspects of user-oriented dialogue design. In: Docherty, P., Fuchs-Kittowski, K.. Kolm, P. & l~athiassen, L. (Eds): System design for human development and productivity: Participation and Beyond. 33 - 47.
I thank PO Dr. H.-J. Luthi for his cooperation, J. Z. Boesze for implementing the interface and H. Schmi tz for ass i st i ng in the experi ments. Prof. Dr. E. Ulich supported this work and gave many helpful comments on this manuscript. IBM Switzerland supported this work with a PC for the development.
Authors present adress: International Business Machines Corporation Thomas J. Watson Research Center P. O. Box 218 Yorktown Heights, New York 10598