- Email: [email protected]
A Formal Method for Designing the Interface of Interactive Systems
Co pyri ght
©
I F:\ ( : \LtJl · \1ac hilll" S\ "l elll ... .
~L\ " - " L\ C Hl "E
\'tl n 'st'. I t;! I ~· . I ~'K:-)
l :-':TERFACE
A FORMAL METHOD FOR DESIGNING THE INTERF ACE OF INTERACTIVE SYSTEMS J.-M. Robert .\'o l io//o/ U{'\/'{/rrh , ~ .I.l tlCi(III', .\"ASA -.-\ IIII'.1 U 1'.11'1IIr!1 CA 'J -IIJ J5, ['SA
CI' It/N.
,\/. S. 2 J'J. ,\I ot/i'lf Fir/ri,
A..!2illJCt. Tills re",ci.lrL"11 in\ · C',tigi.1te~ the u ...e of iJ !"prmal methou as a tool fu r uesigntng i.1nU the inter!"Jce or interal"t i\'e (\' mputer s.\"stems, The meth ou enahles ('ne {(l m;lk.e iJ f\"lrmal Je'\c npt i{ln IJf the user ;Il"tit'ns and system re"rvn'-oes, to iJ entify fli.JW S <.Inu incons i... ten · t:ie~ of the s~' stem. h ) 111~Ik.e predicti{m~ t.thout the u~er liltticult ies and errors with \"arious parts of the ill terfat:e. 1<..' te"-'l the<..;e preolL"li\,n'i. anti Ohx!iJ\ tht: lnttrfat:e act:oruing to the preuictlons that are confirmed. The ne\\ interfl.lce \ers ion th at L'omes out i'i expected to be more ctln'iistent anu predictahl e. ;m u tilt'refllre. easier t() lear n anu use, It ca n be impro\"eu Ite rati\el.\". An experiment \l."~lS ru n w test ('ur predictl\)ns <.I1x)Ut suhjects' errors and diffi culties \1.:ith a gmphics S\'stem. I"g ht sub",cts With kn,' w leJ ge about computers part icipated in the studv. In J first sessinn. the.\" were ll"i ked to learn how to use the s~'stem . In a second session ( tw o da~'s later). they hau hi L:\)mp lete ;1 questH.mn<.!ire about this s.\"ste m. Resu lts confi rmeu ('Iur predictions (with \er.\ · fe\\" t':\Lepti("\ns ). ~e\eral re( ommendations are pro posed for impro\"ing the user Interface . De"i.plt e a fe\\ o..;hnrt(('m in gs. th is formal methou great l.\ he lps in testing and impro\" IJ1g th e c(' n~lqenL'y and rredill ~lhil it~· of a s~'stem,
e\ ' ;du~' tln g
~~ J"or mtd meth pu : l"lerfale: i:rg()Jl{'mu.:s,
('u n siste n c~' :
Human -Computer Inter 'lI.. ti lm: Human !";:Iu\irs: L"ser
I\ T IWDLCrH )\
require the use of t he .olphan umer ic keyboa rd (e.g., '1'1:\'1'). \ Ienus are uynamic, I.e., t hey ch an ge as one se lects comm an d s and g(>es to different s tates of the mac h ine. T h e system prompts the user with he l pful com ments before the e n try of eac h com mand (e,g.," P ick a symbo l to <;1'1 \. ·'" new com mand en ds the SPI\. E\ j) en ds the Fl lIT co mmand". )
This research investiga tes the use of a forma l me t hod for the design and e\·alua tion of human machine interfaces. It IS consistent wit h the w or ks of differe nt auth ors on I\ )fmal dest:ription anu user in t erfaces (e ,g., l.eJgard &. Singer. 1978: \ Ioran, 1981: Reisner, 1981 : Shnelderman , 1982 ). Its pur pose is ll' develop a n analvtical a nd p reJictiw wol to .lId In designing easier -t0- lea rn -<.I nd -use i n ter~lct i \-e computer systems. A t the prese nt time. there a re re\\ tools t o drilw upon for suc h a task. T he desi ~ n (\1 good interfaces is often an int ui tive process. it I ~ '\till an art, not a SCien ce (Smith et aL, 1982 ).
Drawi ngs are prese nt eu on 0 \' erla~l s which are lik e tra nspare nt s heets of paper or pl astic, Subjects crea te and name (werla\s. up W maximum of 20; o\" erlays' ntlmes appe~lr in ,} speLial section of the sc r ee n . An ove rl ay can be manipulated (i.e .. rotated, t rans lated , sca led, etL): al l dr"wings on it wi ll m ove as one. Each in di\' idua l dra\\·lng on an oYerlay is ca lled a symbol. S, ' m bo ls ca n a lso be individually manipulated. A symbol is composed of one or se ve r a l ele me nts w h ich a re any geo metr ica l construct ions ( i,e .. dot , line, circle),
,I
Am ong the criteria that can be useu to e\·t.t lu;lte the ease o f lea rnin g and use of a 'i,\"stem , consisten(,\" is ce rtainly the m ost reported. It is generally .,c k nowledged to be basic and useful (Barnard et a I., 198 1). It all ows users to f or m expectat ions arn)u t " system. tp m'lJ..e rredJ("tkln<.; ahout Ito..; beh ;I\· ll)f.
The system is organized in f0 ur leve ls and five mroes (Fig, 2 ). The Displav mode, at the top le\'e l. corresponds to t he s\stem entrance, and is, therefore, th" first seec bv the user. It IS com prised of twO com mands: \E\\·D iSpl. and ()l.lJlJlSPL. It pe r mits one to de"~nale t he dravdnr (calleJ a dlSpla,·) t o be edited: elth~r an e\ i<.;ting nn. .e. or a new one to be cre.Jted. T he Statement m~xle enables one to CRE.-\TF, \ A\1E. S.-\ \ .1:. R1.\!( )\T. or RLSTORE (we r lav,. It als0
drawing BreB
list :of ouertB
I
The sYstem.
prompt
Prior to th e presentat ion of the fNmal m et hod, we shall in troduce the syste m useu in this experi ment.
display nllme --'_ __ --'
Fig. 1 s hows the sc reen and input d evi ces of the computer graph ics syste m . The system permits users to create and manipulate 2-d im ensional "wire -fra me" d r awings. It is menu -dr,,·en: menus a re ((' mposed of command names. C('m mands ... re selected \\'ith an electron1 c ren th<.lt (me mo\"es across a tahl et: some
fig. 1: T he screen a nd Inpu t devices of the graphics system.
25 5
J. M. Robert
256
Global Edit Mode
Fig. 2: The four lenls and fiv'e modes of the s,'stem, enables to EDIT an entire onrlay (considered as a unit) or a sing le overlay !to deal with indiv'idu al symhols on the onrlavl. hnal1\'. it permits one w change some DlTALLT "alues, define a GRID, and E'.:IT from the s,'stem. The Global EUil mode allows
SPIN
or SCALE
one to m(Jnipulate one entire overlay at a time; it is
or
compflsed of SIX commands: E" I), SIZE. GRID, SPI", SCALE, DRAG. The Edit mooe allows one to manipu· late a single symbol; it is comprised of the same commands as the preceding mooe, pi us the commands ERASE, GATHER, EPLODE, DLPE, DRAW (to get into the Dra,,' mooe), a nd EDIT (to edit a svmbol or another overla\). Finall';, t he Draw mooe IS' used for drawing and wflling characters. It is comprised of these commands: ["D, CA"CEI.. ARC, CIRCLE, OOT, L1"E, POLY , DELETE. and \10\'1:.
DRAG
Fig. 4: Schema of the user's actions and system's responses v,ith commands of the Global Edit mode. recursive. genenlte
The Formal \Iethoo. This method is impl emented in five steps. It can be applied to a real system, or to one that is still at the pro.)tct level. First, a s\,stemaliC, complete, detailed, and simultaneous description of the user's actions and system's responses is made (see Fig. 3). The command menu, the message to users (prompt), and the ne" informatIOn appearing on the screen at each step of the execution of a command are taken into consider ation. It shows ",hlch and how many commands are /'
~
I'
H IP21~H
pick CAll/tU
r ;;.
pick GRID
M4
~
r-
-7
4
~
pick END
type n:mber
I
I-
~
Global Edit mode P 21 M 11
type It + number
PlO M6
-----1
L r
tUDe It
PlO
-
number
-'
type PI9 M4/6 letter decimal ?
I
(no grid)
M6
batlc- spate to rid
01 ?
M6
~
I
1 (I.egend: P ~ Prompt: \1 ~ \Ienu;
=
Pie
bltclcspete + Ret,
Ib8tlc-1
PI9
.~P::t~ ,M4/6 I
0
~ ill egal character t\'ped hy the user )
Fig. 3: Formal description of the user's actions and system's responses for the (;RID command.
I
.-\ Formal Met hod necting them. \odes correspond to states of the interaction between the user and the machine. and the arcs represent transitions from state to state (Parnas, 1969, \1" oods, 1970). This notation was preferred to the formally equivalent Backus-\aur form (Jl\F) used by different authors (Reisner, 1981. Shnelderman. 1982 ) for describing user interfaces. It has the advantage of showing more explicit Iv the conce pt of a state and the transitions associated with it. This is fund a men tal for representlOg sequen ces or actions and responses. \l oreove r. it better highlights the tim -
'257
- Wording: the w ording varies from one prompt to another, a nd the information IS not presented in the same order. - Use of certain words: Select, Pick , Enter, "Key", \1enu , etc. Prompts do not use these words rigorouslv (e.g., "Select" fo r menus, "Pick" for an object 10 the dra wing area ). - Capitalization: some words are upper case in the prompts to draw the attention of the reader. There are some inconsistencies in the use of upper case.
ing of the use r actions and system responses. Fm.:dly .
it seems more convenie nt for captuTlng error han · dling. As seen in Ftg. 1. one's erroneous actions (e.g .. misundersta ndings, inadvertent mO\' es) are represented as well as the valid ones: the error recovery procedure for the user a nd the system's responses are ind icated. A good survey ~f the specification techniques for user interfaces was made by Jacob (1981 . 1983 ). The second step of the forma l method consists of ident ifyin g ftaws and incon sisten cies of the system. Thev a re based on close examination and comparison of users' actions and svstem's beha v iors for all the different arguments of ~ single command. all the commands " 'ithin a mode. and all the modes of the system. They deal with the exceptions to the ru les that one learns about the machine. Three topics are considered: the command language, the informa tion presented to users (prompts), and the location of information on the screen. For each of these topics, one or several issues can be analyzed. Command Language: - Recur'sion: (A command is recursive if, once selected. it can be used with several objects without hav ing to be reselected fo r each indepen dent object. ) Within each mode. some comm a nds are recursi v e and some are not.
- Menu: commands may give rise to a full, or restricted , or empty menu. (A full menu is comprised of all the commands a\'a ilable when a mode is ente red, plus the C,.\\CEL command; a restricted menu contains one or a few commands
only .) - Time al which a command appears in the menu and lime al which it vanishes: inconsiste ncies in the timing of certain commands were identified. - Number of steps to execule commands: same commands used in different modes may require different numbers of steps to be executed. - Default option: (The default opti on en ables one to skip a step in the execution of a command: the system chooses a value by default for this step.) Only one comm and, CIRCLE, is provided with this facility; it seems that other commands cou Id benefit from a default opt ion as well. - Use of key commands: ['iD. CA\CEL, C\!Xl. These co mmands are not used ncorousl v. (The\, permit the user. respectl\ely , to cflange ;"odes. to revoke a com mand. or delete some w orld - f eedback: a fev, LOmm"nds do not pro\'id e the user with feedhach. - Command nesting: (Th iS corresponds t('
Information Presen ted to Lsers (Prompts): - in forma tion conveyed: th IS Information is about the new mode the user is in. or the next action to be done, or what v.-as wrong with the last act ion. Pro mpts do not present the same information in identical situations.
Location of Information on the Screen. - Display and overlays' names: since a displa\' is formed of overlay(s), the display name should appear in the same section of the screen as the overlay names.
The third step of the formal method consists of making predictions about users' errors or difficulties with the system's features that are inconsistent or fault~'. Predictions come out directly from the descripti on presented above. They are concerned with any violation of the expectations that users form when operat ing the svste m. To avoid repetitions, predIctions will be presented with the results. Here are just a few exampl es: use rs will have a problem remembering or figu rin g out 1. which commands are not recursive: 2. \\'hich commands generate a restricted or empt\' menu as oppo;ed to a full menu: 3. whIch commands do not generate the CA\CEL com mand; 4. which comm ands do not give rise to C\D(): ete. Fourth step: these predictions are tested with users in order to measure their val ue a nd strength . This is the pu rpose of the expe riment presented be low . Fifth step: system's features causing problems to users are modified . A new system version is proposed, h opefull~' more consistent and better su ited to users' expectations and behaviors. \1odifications are tested repeatedl\'. Ite rative design is a ke y principle for user interfaces (Good et al .. 1984, Gou ld & Lewis, 1984). \1ETIIODOLOGY Subjects. Eight suhjects with knowledge about computers pa rticipated in the experiment. They were all unfamiliar wi th the graphics system described above.
Procedu,·e. The experiment ""as conducted in two sessi ons. During the first one, subjects learned how to use the graphics S\'stem. They were given a user ma nu al that they read in f ron t o f the svstem
(wi th the Disp lay m'ooe appearing on the sc'reen). Th e\' were asked to lea rn the svstem as well as they could oUTIng th e 2 hour sessl~n: the\' knew that 'a questionnaJre about the commands, thei r orga nization . a nd their operation would be completed afterwards. Subjects w ere required to read the manual entJrel\' (x ~ 36 mln). and then explore the svstem In order to learn Its operation . On ce th e readIng was completed , they were required to inform the experimenter. They were then informed to pa\" attention to the commands showing on the sc reen and to the s\'stem response obtained after each action. They were asked to think a loud while using the system: their \'erbal comments were tape-recorded , their actions and the system responses were \·ideo-taped . Protocols (the chronological history of all the subject actions and verbal comments, and al l the system responses) were transcribed for a later a nal ysis. During the session, subjects did not benefit from an\' external assistance other than the user manual. Thi-s manual (19 pages) has been written b\' the experimenter. The first part presents the basic conce pts of the system: functional ity. hardware. sections of the screen, organiza tion and components of a drawing, and levels and modes of the system. The second pa rt focuses on how to get started, how to draw. and how to manipulate draw ings. It gives a list of a ll the comm ands in each mode, and describes their role and the procedure to execute them.
J.
\1. RoiJcrt
In the second session (2 days lated. subiects WeTe allowed to read the user manual and pla~' ~'ith the S\'stem for 10 minutes. Thev then had to com plete a l-hou r questionnaire about the s~'stem operation. The questlOns were about fttlWS and inconsIste ncies that ha"e been id entlfieel on the basIS of the formal descTlptlon. SubJects ',",re provided with info rmati on about the name l.lnU l)f~i.lnIZation of the mode ... and levels of the s\,stem. and- about the components of a drawing (dTawlng. oH rl a\, . s\,mbol. e lement ). When necessar~', they we re reminded of the tasks that can be done in a mode. In each questi on. the use of Tel,,'a nt commands was defined : there were multiple -choice answeTS, and an opt ion "I d" not know" was included, This questionnalTe had been tested with fuur pilot sub.iects. Pro,'isions had so been made to insu re that the subJect understood the conte xt of a question . the mod~ or le vel of the system that " 'as conce rned. a nd the vocabulary used. RESLI.TS This section presents the results of 14 predicti ons tha t we Te made about the subjects' errors a nd difficulties with the ,,'stem . The Learning session, Before exposing these results, the learnin g session has to be examined. It is important to know if sub.iects succeeded in using the system , to give some ideas of what happened dUTing this sesslOn, and look at the indi\'idual differen ces in learning. Protocol analysis of the learning session indicates that all subiects were able to use the system within the first hou r of the session, and perform some real tasks of their chOice. The\' went back and forth between the Statement. Edit, 'a nd DTaw modes of the system, and they used all the commands but a few ones (different for each subjectl in each of these modes. Only one subject explored the Global Edit mode OT 36 seconds ). The da ta from t he few q uestions about the comma nds of this mode were included in the ana l\'sis. The answers about thiS mode were easily predictable from the ot heT modes.
er
TheTe were importa nt individual differences in learning. They appear in the number of times each command was selected, in the duration of use of each command. in the use of a command recursi\'el y or not, and in the few unexplored commands. FurthermOTe, there were diffeTences in the understanding of s~'mbol independence (one must exit from the Draw mode to CTeate two independent s~' mbols) and in the understanding of the role of the EDIT commanu in the Statement and the Edit modes. It ca n be concluded that each sUb.iect w as proficient enough with the system to ha ve a good idea of its ope ration. and be familiar with the menus. the commands. th e prompts. and the \·oca bular~·. Resu I ts of the VueslIon nai res. Sinct' the s\"<.;tem is menu -dri\"en. suhje cts did n(H rXl\" mUlh attentiun tp the t:om mand menu. Therefore . learning about the co mmand menu "'-as "in cide ntal". as one subiect put it. \10reo\'e r. the questionn aire was gi\"en two dJ .vs after the learning session, and asked man\" detai led qU"'tions. ,-\s s,,'eral subjects wrote on the questionna ires. a lot 01' "reasoning out" , or "l ogical understanding" of the s\,stem was necessan', This situation was createu purposel\' to lead subiects (w ho are knowledgeable with computers) to figure out the different rules on which the s\,stem was based . The sub.iects performar,ce in remembeTlng or figurlllg out a feature of the system is expected to be pvorer "ith the exceptions than with the rules,
T,,'o POlllts ought to be mentioned for the presentati on 01' the results: con.mands that are concerned are t he ones sh()wlng when a mode is entered, and for each question, there " 'as an "I do not kno"'" optIon .
Prediction J: Recursion. Because not all commands a re recurS] ,," ( 23°" of exceptions), sUb.jects will ha\'e mo re difficult~· rememberin g or figuring
out whIch commands are not recursive. E1,,'en questions were asked: six about recurSIve commands, and fi ve about non recursi"e ones. For each command . there was a choice between recuTsive or not. Resu lts show that the subJ"cts' success rate is 2(),;,c higher with recursive commands than \vith non recursive com mands (65<;, v's. 45C7,). The prediction that sub.Jects have more trouble with the except ion com mands IS confirmed,
1'I'ediction 2: Menu . \ot all commands give rise to a full menu 06% of commands bring a restricted
menu, anu 7%. an empt~' one). Therefore, SUbjects ",,11 ha ,," more difficulty remembering or figuring out "'hich C0m mands do not generate a full menu. Five questions were askeel, o ne about each mode of the "'stem. Each question waS concerned with a representatiw group of commands from a mode, and offered fi\'e possible chOices: 1. a full menu, 2. a full menu + CA\CEL. 3. a restTlcted menu comprising E\D and CA\CEL. 4. a res tricted menu comprising only CA\CEI., and 5. an empt\' menu. Results show that the subiects' success rate is much higher fo r commands giving a full menu (wi th or without CA\CELl than for commands giving a restricted or empty menu (88% \'S. 25%). In each mode where there is a mixing of commands giving rise to a full menu and comma nds gi\'in g rise to a restricted one. the difference in the success rates is Yery large. Most subJects th ought that exceptions ga\'e f ull menus. For example, seven out of eight subiects knew th at th e ARC command (,n the Draw mod e) ga\'e access to a full menu. where as on l\' 12 an d 0% of sub.iec ts w u lu contend that the TL:\ T and GRJJ) commands. r espect l\ · el~· . brought a restrlcteu menu.
,se"
Predictioll J: C J\ NC };J .. \ot all commands gen erate CA\CE I. when the\' are selected ( 19ck of exceptlons). Therefore. subiects should have more difficulty re membe rin g or figuring out which com mands do nnt generate C-\\CEJ.. Sennteen questions were asked: elewn about the commands generating CA\CEI., and six about the exception commands, Each question offered two choices (Yes or ,"0). Results show that t he subjects' success rate is much higher with the normal commands than with the exception commands (60Cf, \'5. 26%). ,-\11 subiects but one stressed within-a-mode consisten c~' for every mode. and man :v stressed between - modes consjstenc~· . Predictioll 4: C l RCLF The CIRCLE command does not beha"e like the ot her insertion commands from the Draw mode kg .. ,-\Rc:. DOT, TE\T) since it does n()t ge nera te C'.-\\CEJ.. Therefore, subiects shou ld ha ve more difficult\' remem ber ing or figurin g out that CIRCLE is an exce ption and does not gen erate CA\CEJ.. Onl\' two questions weTe asked about insert ion commands fr nm the Draw mode and CA\ CEL: one was about TE\T and the other, about C IR CU:. Resul ts show that 880" of subjects knew that TE\T was gene rating CA\CEI. ,,'hereas no o ne knew that CIRCLE was not generatin g c.-\\CU .. Prediction 5: CRI D, The GR]]) command from the Draw mode does not beho\'e like GR ID from the t hree other modes beca use it does not generates a menu with CA\CTL. Therefore, subjec ts should h3\'e more difficu lt\' remembering or figuring out the mode wherein GR ID does not generate CA\CEL. Three questi ons were asked abou t GR ID in the Edit, Global Edit. and Ora,,' modes. Eac h question offe red two choices ( Yes or \ ('). Results ind ica te that 50% of subiects knew that GRID from the Edit mode ge nerated a menu with C.'" \(:1: 1., 25<;,., that GRID from the Gl oba l Edit genera ted a menu with CA\CEL. and 25'"" that GRID from the Draw mode did not give rise to C-\\CEL. The prediction was confirmed with th e Fd it mode bu t not with the Global Edi t mode ("hlch was not exp lored). Pl'ediction 6: DRAW MOD!:'. Since one mode (Draw ) is com prised of CA\CEL in its first menu. subjects will ha\'e more difficult\' remembering or figuring OU t which mode is exception and includes C-\\CEL in its first menu. F,w questions were asked about the presence of C.-\\CEJ. In the first menu of modes, For each question. there was a chOICe between
.'\ Forlllal \kt!toc! Yes 0 r \0. Results sh0w that 87.S'~ of subjects answered that CA\CEI. was appearing In the Draw m ode. 75" ,. that it was appearing In the Edit mode. and 62 .5(",.. that is was appeanng In the Global Edit nlllde. The prediction that the 1)ra" m ode would not be seen as an exception wnh respect to CA\CEL was c()nhrmed.
F,edictiofl 7: };,ND CANCFL. [\D is the main command used to go into a higher In'e! mode. There fore. \'en' rew sub.!ects among those wh o think that CA\CEI. appears In the first menu of a mode. should be able to remember or figure out the use of the C,-\\CEL command appearing in the first menu of the Draw mode. ri,'e questions were asked about the use of C,-\\CEL ("' hen one answered th', t CA\CEI. appeared in the first menu of a mode). For each question . there were multiple choices: C'.\CEL allows to: go into a higher Ie'el mode: go into a lower le\'el mode: sta\ In the same mode: get a message in the prompt about the impos.sibilitv of us ing CA\CEL to exit from a mode. \\'hen asked about the use of CA\CEI. from the Draw mode. onl\' two subjects (out of se\'en wh o knew that C.-\\CEL " 'as showing in the Draw mode) correctl, contended that c.-\\CEl provoked a mode cha~ge (go into the higher level mode): four tl10ught that it was impo;sible to go into a higher le\'e l mode with C\\CEL. When asked the same question in the Edit and the Global Edit modes. among subjects (\ ~ 6 ) who erroneousl\' thought that CA\CEl appeared in the first menu of these modes. onlv one said that it ,,'ou ld lead into a higher le\'el mode. F0ur subjects contended that it was impossible to go int0 a higher le\'el mode with CA\CEL. This shows that there were more subjects thinking that CA\CEL ca nn ot lead into a h ig her le"el mode than there were subjects thinking the opposite. Subjects were confused about the role of CA\CEL. P,'edictiofl 11: UNDO. Ilecause not all commands give rise to an L'\ D() command when the,' are selected (16% of exceptions). subjects should have more difficult,' remembering or figuring out which commands d o not generate L\DO. For SUbjects knowledgeable with computers. it is postulated that L l\ DO is expected \\'ith commands allowing to create. delete , or modify. Fourteen questions were asked, three about the exception commands generating L'\DO, and eleven about the other commands. Each question had two choices (Yes or \0). Results show that the subjects' success rate was much higher with commands generating L' \DO than with commands not generating it (710/(. vs. 360(. ). Twice as man:,; suh:!ects knev.; about commands gi\'in g rise to C\f)() than about commands not generating it.
F, ediction Y: UNIX). Subltcts will h,,'e more difficult\' rememberin g or figuring out which com mands d0 not giH rise to L\IX) among those that a re d0Sely rebted to the ones bringing C\IXl. Commands such as DEI.ITE. .\10\T I b()th from Dra\\' mode). ERASE (from Edit mode). and RL'vl0\T (from Statement mode ) should be expected to beha\'e sim llarl\ because the ,Y have the same functionality of deletion or modification. Simil"rlv. commands such as CIRCl.E. ARC. DOT. Ll\E. POl.'y. and TE\T (from the Draw mode) should be expected to beha"e similarly because they are all concerned with creatl(' n, It was expected that subjects knew that RDIO\T did not comprise an L\[)() command because of the presen ce of the RESTORE com mand in the menu (RESTORE is equi"alen t to C\lX». Eight questions were asked. f0ur about the c reati on commands and f our about the deleti on or modification commands. Resul ts ab0ut the c reati on commands show that the subltcts' success rate was high for commands generating L \IX land , 'en' 10W for the exception CIRCLE command I 75'~,. ' ·s. 12.5';-,')' 110,,'e\·e r. as opposed w ClRCI.E. the success rate was fairl,' high f or the TF\T wmm"nd (SIl'i,.J. This result was wrongl~.. predicted: on ly 25C', of subjec ts th ou ght t hat L\D() wl' uld appear. Results about the delet ion or modif1ciJtlOn commi.lnds sh ow th<.Lt the success rate was birl." high f0f the w mm a nd I LRASE ) generat ing C\])() (c2.Sc-,). and "erv' It) w ror the com man ds
(DELETE ,!Od .\!
Prediction 10: PROMPTS fOl' EDIT. The prumpts aCCllmpanving the EDIT command in the Statement and the Edit modes are misleading. Therefore. all the sub.ltcts or a large majority of them shou Id make the same mistake when interpreting these prompts. "Edit the entire displa,'" should cause them to erroneousl v think that all the o\'erlavs forming a display c~n be edited at once (one overl~,· at a time can be edited). "Edit an oHrlay" should cause them to think that an entire oyerlav can be edited at once (one symbol at a time can be ~dited on the o\,er lay). Three questions were asl-ed. tw o about the prompts that appear when EDIT is selected from the Statement mode (two pos.sible argumen ts). and one about the prompt that appears when EDIT is selected from the Edit m ode ("' ith the argument O\TRI..-\ Y l. Results sh ow that all subjects got con fused when Interpreting the prompt" Edit the entire display". The\' all made the mistake that was predlcteJ. 7':'C, or suh.)ects g\){ (0nfused when inter preting the prompt" edit an overla\''' : they made the mistake that was predicted. Prediction 11: Number of steps for /:.'DI'J'. W e postulated that the same command appearing in different modes is expected to require the same number of steps to be executed. The EDIT command requires two steps in the Statement mode, and three steps in the Edit mode. Our predict ion is that all sub.ietts o r a great maJOrit,' of them should make the same mistake when tn'ing to remember or figure out the number of steps: the\' should think that this nu mber is the same in bot h modes. Two questions were asked: one abo ut the EDIT command from the Statement mode. and one about the EDIT command from the Edit mode. Resu lts show that onl v one subject th ought that the number or steps f or executing EDIT was the same in the t"'o modes. Therefore. our prediction was not canfi rmed. The reason cou id be that subjects m ay have simpl\ s uspected that there " 'as an exception. and that the tw o answers had to be different. Prediction 12: Statement mode. Because there is no need of a GRID command in the Statement mode, sublects should have m ore difficulty than in the ot her modes (Edit, Global Edit, and Draw) remembering or figuring out if GRID appears in the Statement m ode. ri"e questions were asked about the presence "I' the (jRID command in each mode. For each question. there was a choice between Yes or \ 0. Resul ts SI10W that subjects had m ore trouble with the Statement mode th~n \\'ith the other modes when dealing with GRID: 5('" of subje((, thou2ht that there " 'as no GRID in the Stat~ment mOde, whereas 25",. thought that there was n0 ne in the Edit o r the Gl ohal Edit mode . 37.5<;. that there " 'as none in the Draw mode. P,'edictioll J.j: De fault option, There is at least one command us ing the default option : CIRCLE (one can skip the ch oice or "radius" or "diameter" when drawing circles. and radius will be chosen by def'lult ), There should be some subjects thinking that other comm a nds use a default o ption as we ll. Fi ve questions were asked. one about CIRCLE and four about ot her candidate commands: SPI\. SCALE. DRAG , EDIT. These comman ds lend themsel"es t o the use of a defa ult option because on e step cou ld be easily s kipped. and because theY reqUired m ore steps (4) than ('ther cnmmands tl' be exec uted. Each question had tw o c hOi ces ( Yes or \ 0). Resu lts show that 25';-, of the subltcts th ourht that the commands SI'I\. SeALF. a nd DRAt; used-a default o ption. and 37.5(", . that FIJI')' ( fT(\m the Edit mOde) also used a default optI on.
J.
2(iO
~1.
P"edic/ion 1.J: LX IT. Because one can expect to get out of the S\'stem from the same mode as the entrance mode. some subjects should have difficulty remembering or figuring out which mode of the SYStem the E,\IT command appears m. and should thin).. that [,\IT appears in the Display mode. Presentlv. E,\IT shoves In the Statement mode. One question was asked ahout the mode wherein E,\IT was showIng. There were fiw chOices. corresponding to each mode of the system. Results show that 25% of the sub.Jfcts got confused and thought that E,\IT appeared In the Display mode.
Roben Finally, it would be useful to have some relative order of importance of the effect of various system's inconsistencies on the users' performance. The formal method is already a useful tool for designing human-machine i'nterfaces, but it needs some improvement to be used more rIgorously. REFERE'\CES Barnard. PJ" lIammond. ,\.\,,, \lorton, 1., Long, 1.B. (1981). ConSistency and compatibilIty in human computer dialogue: Int.crnational Journal of Man \lachine Studies Vol. 15 pp.
Reco mmendatillns.
The design of user Interfaces is the result of man.\" compromises:
11
Almost alw;):vs, there are trade -
offs." (Smith et al" 1982) The path is not always clear-cut. Recommendations proposed to Improve the consIStencY and predictabilit\· of the interface studied here should be con"dered in this perspectin. Exceptions to rules should be banIShed entirely to simplify the system. Consequently. all commands should be recursi\·e. They should all give rise to a full menu and to CA\CTL when selected. C.'..'\CEL should be strictlv resened for canceling a command that was Just selected. v,hereas E,\D should be strictly used for between -modes changes. L'\l)() should be a\'ailable with all commands allOWing one to create. delete . or modif,·. The same command used in different modes should always be comprised of the same number of steps to be executed. The default option should be used with all commands that lend themselyes to it: for example. commands requiring more steps than the average to be executed (e.g., SPI'\, SCALE, DRAG). or having steps that can easily be skipped for most of the tasks performed with these commands. Feedback should always be provided with commands to inform users of success, failure, or partial success. There were several complaints ahout the lack of feedback with various commands (e.g., GATHER, ['\PLODE, SIZE). FinallY. since the entrance and the exit modes are expected to be the same in a system, E,\IT should appear in the Display mode instead of the Statement mode. Similar recommendations ahout the presentation and location of information on the screen could be proposed. These modifications should impro\'e the Interface consistency and predictability. and make the
s~'stem
easier to learn and use.
CO,\CI.LSIO,\ This research examined the use of a formal method to design and eyaluate user interf<.Jces. It focused on conSistency and predictability. The method allowed one to make a s,'stematic description of the users' actions and system's resJX>nses. to iden tiCy flaws ..md inconsistencies of the s,ystem, and
make predictions ahout users errors and dIfficulties with the SYstem. These predictions were tested with users and were confirmed ( with few exceptions ). \lany modifica tions to improve the interface consistenc\' and predictability. and make the system easier to learn and use "'ere proposed. These modifications can be tested iteratively. The formal method turns out to be a powerful tool for analyzIng and predicting the consistency of interfaces. lluw eyer. it still has some shortcomings that will have to be tackled. First, despite the precI sion and completeness of the formal description. there
IS
no guarantee that different persons \vould
identify the same Inconsistencies and make the same predicti ons. The prediction process has to be more explicit. For example. there is a need for a list of factors to be checked for consistency of the user interface. The lIst presented in this paper can certainiv' serve as a good basis, but it still has to be extended. Second, it IS difficult to measure the scope of each prediction. Is It affecling all, or several, or a fe"" users1 We certainly had some intuition ahout it, but this is not sufficient. Third, we would be unable to make different predicti ons for users with different levels of qualifications. Some work has been done on thiS topic by Reisner (1982); it should be pursued.
Gould. 1.0., I.ewis, C. (1984). Designing for usabilItynl\C)' principles and what designers tb.ink., Res. Rep. 10317, 1B\1 \\'atson Research Center, Yorktown Heights, '\.1'. 25 pp. Good , \1.1)" Whiteside. 1.A .. \\·ixon. D.R .. lones. SJ. (1984). flulldlng a Lser -Derind Interface. Communi cations of the -\('\1 \01. 27 \0 10 pp. 1032- ]()43. lacob, RJ.I\. (1981). Survey o[ SpeCification Tech niques for Lser Interfaces" Technical \1emoTaouum
7590-.'()3:Rl:rl. \a\al Research I.ahorator,·. Washing ton. D,C
lacob, RJ.I\. (1983). LSlng Formal Specifications in the Design of a Human -Computer Interface. Cilln:: munications of the .'..C\1 \'01. 26 \0..1. pp. 259-264. Ledgard, H.F .. Singer. A. (1978). Formal definition and design. COI,\S Tech. Rep. 78-01, Feb., Lni\'. of Massachusetts. \1.'... 6 p. Parnas, D.L (1969). Un the use of transition dIagrams in the design of a user interface [or an interactlw computer svstem. In Proceedings of the 24th \ational AC\1 Conference. .'..C\l. \ew York. pp. 379-385. Reisner, P. (1981). Formal Grammar and Human Factors Design of an Interactive Graphics System. Il.:EE Transactions on SQftware Fngineering Vol. SE-~ 2. \la rch , pp. 229-24(). Reisner, P. (1982). Further de\elopments toward using f,)rmal grammar as a deSign tool. In E=dings of Human Factors ID Computer Systems \Iarch 1517. (jalthershu rg, \1arvland. pp. 30-1-308. Shneiderman , ll. (1982). \lultlparty Grammars and Related Features for Defining Interactive Systems. IEEE Transactions on Systems \lan and Cybernetics \,QI. S\1C- 12 \Q) \larch ·April. pp. 148-15-1. Smith. D.e.. IrbY. C, I\imball, R.. \'erplank, fl., lIarslem, E. (1982). Designing the Star Lser Interface. Byte \'01. 7 \04 '\pril pp. 242 -2 82. Woods, W ..-\. ( 1970). Transition '\etwork Grammars for '\atural Language AnalysIS. Communications of tb..e.....:\.Qi \'01. 13 \0 J U. Oct.. pp. 591-606. Acknowledgement.
I want to thank Robin Wise who spent much time transcrIhlng the protocol, and helped to analyze the data.
©
I F:\ ( : \LtJl · \1ac hilll" S\ "l elll ... .
~L\ " - " L\ C Hl "E
\'tl n 'st'. I t;! I ~· . I ~'K:-)
l :-':TERFACE
A FORMAL METHOD FOR DESIGNING THE INTERF ACE OF INTERACTIVE SYSTEMS J.-M. Robert .\'o l io//o/ U{'\/'{/rrh , ~ .I.l tlCi(III', .\"ASA -.-\ IIII'.1 U 1'.11'1IIr!1 CA 'J -IIJ J5, ['SA
CI' It/N.
,\/. S. 2 J'J. ,\I ot/i'lf Fir/ri,
A..!2illJCt. Tills re",ci.lrL"11 in\ · C',tigi.1te~ the u ...e of iJ !"prmal methou as a tool fu r uesigntng i.1nU the inter!"Jce or interal"t i\'e (\' mputer s.\"stems, The meth ou enahles ('ne {(l m;lk.e iJ f\"lrmal Je'\c npt i{ln IJf the user ;Il"tit'ns and system re"rvn'-oes, to iJ entify fli.JW S <.Inu incons i... ten · t:ie~ of the s~' stem. h ) 111~Ik.e predicti{m~ t.thout the u~er liltticult ies and errors with \"arious parts of the ill terfat:e. 1<..' te"-'l the<..;e preolL"li\,n'i. anti Ohx!iJ\ tht: lnttrfat:e act:oruing to the preuictlons that are confirmed. The ne\\ interfl.lce \ers ion th at L'omes out i'i expected to be more ctln'iistent anu predictahl e. ;m u tilt'refllre. easier t() lear n anu use, It ca n be impro\"eu Ite rati\el.\". An experiment \l."~lS ru n w test ('ur predictl\)ns <.I1x)Ut suhjects' errors and diffi culties \1.:ith a gmphics S\'stem. I"g ht sub",cts With kn,' w leJ ge about computers part icipated in the studv. In J first sessinn. the.\" were ll"i ked to learn how to use the s~'stem . In a second session ( tw o da~'s later). they hau hi L:\)mp lete ;1 questH.mn<.!ire about this s.\"ste m. Resu lts confi rmeu ('Iur predictions (with \er.\ · fe\\" t':\Lepti("\ns ). ~e\eral re( ommendations are pro posed for impro\"ing the user Interface . De"i.plt e a fe\\ o..;hnrt(('m in gs. th is formal methou great l.\ he lps in testing and impro\" IJ1g th e c(' n~lqenL'y and rredill ~lhil it~· of a s~'stem,
e\ ' ;du~' tln g
~~ J"or mtd meth pu : l"lerfale: i:rg()Jl{'mu.:s,
('u n siste n c~' :
Human -Computer Inter 'lI.. ti lm: Human !";:Iu\irs: L"ser
I\ T IWDLCrH )\
require the use of t he .olphan umer ic keyboa rd (e.g., '1'1:\'1'). \ Ienus are uynamic, I.e., t hey ch an ge as one se lects comm an d s and g(>es to different s tates of the mac h ine. T h e system prompts the user with he l pful com ments before the e n try of eac h com mand (e,g.," P ick a symbo l to <;1'1 \. ·'" new com mand en ds the SPI\. E\ j) en ds the Fl lIT co mmand". )
This research investiga tes the use of a forma l me t hod for the design and e\·alua tion of human machine interfaces. It IS consistent wit h the w or ks of differe nt auth ors on I\ )fmal dest:ription anu user in t erfaces (e ,g., l.eJgard &. Singer. 1978: \ Ioran, 1981: Reisner, 1981 : Shnelderman , 1982 ). Its pur pose is ll' develop a n analvtical a nd p reJictiw wol to .lId In designing easier -t0- lea rn -<.I nd -use i n ter~lct i \-e computer systems. A t the prese nt time. there a re re\\ tools t o drilw upon for suc h a task. T he desi ~ n (\1 good interfaces is often an int ui tive process. it I ~ '\till an art, not a SCien ce (Smith et aL, 1982 ).
Drawi ngs are prese nt eu on 0 \' erla~l s which are lik e tra nspare nt s heets of paper or pl astic, Subjects crea te and name (werla\s. up W maximum of 20; o\" erlays' ntlmes appe~lr in ,} speLial section of the sc r ee n . An ove rl ay can be manipulated (i.e .. rotated, t rans lated , sca led, etL): al l dr"wings on it wi ll m ove as one. Each in di\' idua l dra\\·lng on an oYerlay is ca lled a symbol. S, ' m bo ls ca n a lso be individually manipulated. A symbol is composed of one or se ve r a l ele me nts w h ich a re any geo metr ica l construct ions ( i,e .. dot , line, circle),
,I
Am ong the criteria that can be useu to e\·t.t lu;lte the ease o f lea rnin g and use of a 'i,\"stem , consisten(,\" is ce rtainly the m ost reported. It is generally .,c k nowledged to be basic and useful (Barnard et a I., 198 1). It all ows users to f or m expectat ions arn)u t " system. tp m'lJ..e rredJ("tkln<.; ahout Ito..; beh ;I\· ll)f.
The system is organized in f0 ur leve ls and five mroes (Fig, 2 ). The Displav mode, at the top le\'e l. corresponds to t he s\stem entrance, and is, therefore, th" first seec bv the user. It IS com prised of twO com mands: \E\\·D iSpl. and ()l.lJlJlSPL. It pe r mits one to de"~nale t he dravdnr (calleJ a dlSpla,·) t o be edited: elth~r an e\ i<.;ting nn. .e. or a new one to be cre.Jted. T he Statement m~xle enables one to CRE.-\TF, \ A\1E. S.-\ \ .1:. R1.\!( )\T. or RLSTORE (we r lav,. It als0
drawing BreB
list :of ouertB
I
The sYstem.
prompt
Prior to th e presentat ion of the fNmal m et hod, we shall in troduce the syste m useu in this experi ment.
display nllme --'_ __ --'
Fig. 1 s hows the sc reen and input d evi ces of the computer graph ics syste m . The system permits users to create and manipulate 2-d im ensional "wire -fra me" d r awings. It is menu -dr,,·en: menus a re ((' mposed of command names. C('m mands ... re selected \\'ith an electron1 c ren th<.lt (me mo\"es across a tahl et: some
fig. 1: T he screen a nd Inpu t devices of the graphics system.
25 5
J. M. Robert
256
Global Edit Mode
Fig. 2: The four lenls and fiv'e modes of the s,'stem, enables to EDIT an entire onrlay (considered as a unit) or a sing le overlay !to deal with indiv'idu al symhols on the onrlavl. hnal1\'. it permits one w change some DlTALLT "alues, define a GRID, and E'.:IT from the s,'stem. The Global EUil mode allows
SPIN
or SCALE
one to m(Jnipulate one entire overlay at a time; it is
or
compflsed of SIX commands: E" I), SIZE. GRID, SPI", SCALE, DRAG. The Edit mooe allows one to manipu· late a single symbol; it is comprised of the same commands as the preceding mooe, pi us the commands ERASE, GATHER, EPLODE, DLPE, DRAW (to get into the Dra,,' mooe), a nd EDIT (to edit a svmbol or another overla\). Finall';, t he Draw mooe IS' used for drawing and wflling characters. It is comprised of these commands: ["D, CA"CEI.. ARC, CIRCLE, OOT, L1"E, POLY , DELETE. and \10\'1:.
DRAG
Fig. 4: Schema of the user's actions and system's responses v,ith commands of the Global Edit mode. recursive. genenlte
The Formal \Iethoo. This method is impl emented in five steps. It can be applied to a real system, or to one that is still at the pro.)tct level. First, a s\,stemaliC, complete, detailed, and simultaneous description of the user's actions and system's responses is made (see Fig. 3). The command menu, the message to users (prompt), and the ne" informatIOn appearing on the screen at each step of the execution of a command are taken into consider ation. It shows ",hlch and how many commands are /'
~
I'
H IP21~H
pick CAll/tU
r ;;.
pick GRID
M4
~
r-
-7
4
~
pick END
type n:mber
I
I-
~
Global Edit mode P 21 M 11
type It + number
PlO M6
-----1
L r
tUDe It
PlO
-
number
-'
type PI9 M4/6 letter decimal ?
I
(no grid)
M6
batlc- spate to rid
01 ?
M6
~
I
1 (I.egend: P ~ Prompt: \1 ~ \Ienu;
=
Pie
bltclcspete + Ret,
Ib8tlc-1
PI9
.~P::t~ ,M4/6 I
0
~ ill egal character t\'ped hy the user )
Fig. 3: Formal description of the user's actions and system's responses for the (;RID command.
I
.-\ Formal Met hod necting them. \odes correspond to states of the interaction between the user and the machine. and the arcs represent transitions from state to state (Parnas, 1969, \1" oods, 1970). This notation was preferred to the formally equivalent Backus-\aur form (Jl\F) used by different authors (Reisner, 1981. Shnelderman. 1982 ) for describing user interfaces. It has the advantage of showing more explicit Iv the conce pt of a state and the transitions associated with it. This is fund a men tal for representlOg sequen ces or actions and responses. \l oreove r. it better highlights the tim -
'257
- Wording: the w ording varies from one prompt to another, a nd the information IS not presented in the same order. - Use of certain words: Select, Pick , Enter, "Key", \1enu , etc. Prompts do not use these words rigorouslv (e.g., "Select" fo r menus, "Pick" for an object 10 the dra wing area ). - Capitalization: some words are upper case in the prompts to draw the attention of the reader. There are some inconsistencies in the use of upper case.
ing of the use r actions and system responses. Fm.:dly .
it seems more convenie nt for captuTlng error han · dling. As seen in Ftg. 1. one's erroneous actions (e.g .. misundersta ndings, inadvertent mO\' es) are represented as well as the valid ones: the error recovery procedure for the user a nd the system's responses are ind icated. A good survey ~f the specification techniques for user interfaces was made by Jacob (1981 . 1983 ). The second step of the forma l method consists of ident ifyin g ftaws and incon sisten cies of the system. Thev a re based on close examination and comparison of users' actions and svstem's beha v iors for all the different arguments of ~ single command. all the commands " 'ithin a mode. and all the modes of the system. They deal with the exceptions to the ru les that one learns about the machine. Three topics are considered: the command language, the informa tion presented to users (prompts), and the location of information on the screen. For each of these topics, one or several issues can be analyzed. Command Language: - Recur'sion: (A command is recursive if, once selected. it can be used with several objects without hav ing to be reselected fo r each indepen dent object. ) Within each mode. some comm a nds are recursi v e and some are not.
- Menu: commands may give rise to a full, or restricted , or empty menu. (A full menu is comprised of all the commands a\'a ilable when a mode is ente red, plus the C,.\\CEL command; a restricted menu contains one or a few commands
only .) - Time al which a command appears in the menu and lime al which it vanishes: inconsiste ncies in the timing of certain commands were identified. - Number of steps to execule commands: same commands used in different modes may require different numbers of steps to be executed. - Default option: (The default opti on en ables one to skip a step in the execution of a command: the system chooses a value by default for this step.) Only one comm and, CIRCLE, is provided with this facility; it seems that other commands cou Id benefit from a default opt ion as well. - Use of key commands: ['iD. CA\CEL, C\!Xl. These co mmands are not used ncorousl v. (The\, permit the user. respectl\ely , to cflange ;"odes. to revoke a com mand. or delete some w orld - f eedback: a fev, LOmm"nds do not pro\'id e the user with feedhach. - Command nesting: (Th iS corresponds t('
Information Presen ted to Lsers (Prompts): - in forma tion conveyed: th IS Information is about the new mode the user is in. or the next action to be done, or what v.-as wrong with the last act ion. Pro mpts do not present the same information in identical situations.
Location of Information on the Screen. - Display and overlays' names: since a displa\' is formed of overlay(s), the display name should appear in the same section of the screen as the overlay names.
The third step of the formal method consists of making predictions about users' errors or difficulties with the system's features that are inconsistent or fault~'. Predictions come out directly from the descripti on presented above. They are concerned with any violation of the expectations that users form when operat ing the svste m. To avoid repetitions, predIctions will be presented with the results. Here are just a few exampl es: use rs will have a problem remembering or figu rin g out 1. which commands are not recursive: 2. \\'hich commands generate a restricted or empt\' menu as oppo;ed to a full menu: 3. whIch commands do not generate the CA\CEL com mand; 4. which comm ands do not give rise to C\D(): ete. Fourth step: these predictions are tested with users in order to measure their val ue a nd strength . This is the pu rpose of the expe riment presented be low . Fifth step: system's features causing problems to users are modified . A new system version is proposed, h opefull~' more consistent and better su ited to users' expectations and behaviors. \1odifications are tested repeatedl\'. Ite rative design is a ke y principle for user interfaces (Good et al .. 1984, Gou ld & Lewis, 1984). \1ETIIODOLOGY Subjects. Eight suhjects with knowledge about computers pa rticipated in the experiment. They were all unfamiliar wi th the graphics system described above.
Procedu,·e. The experiment ""as conducted in two sessi ons. During the first one, subjects learned how to use the graphics S\'stem. They were given a user ma nu al that they read in f ron t o f the svstem
(wi th the Disp lay m'ooe appearing on the sc'reen). Th e\' were asked to lea rn the svstem as well as they could oUTIng th e 2 hour sessl~n: the\' knew that 'a questionnaJre about the commands, thei r orga nization . a nd their operation would be completed afterwards. Subjects w ere required to read the manual entJrel\' (x ~ 36 mln). and then explore the svstem In order to learn Its operation . On ce th e readIng was completed , they were required to inform the experimenter. They were then informed to pa\" attention to the commands showing on the sc reen and to the s\'stem response obtained after each action. They were asked to think a loud while using the system: their \'erbal comments were tape-recorded , their actions and the system responses were \·ideo-taped . Protocols (the chronological history of all the subject actions and verbal comments, and al l the system responses) were transcribed for a later a nal ysis. During the session, subjects did not benefit from an\' external assistance other than the user manual. Thi-s manual (19 pages) has been written b\' the experimenter. The first part presents the basic conce pts of the system: functional ity. hardware. sections of the screen, organiza tion and components of a drawing, and levels and modes of the system. The second pa rt focuses on how to get started, how to draw. and how to manipulate draw ings. It gives a list of a ll the comm ands in each mode, and describes their role and the procedure to execute them.
J.
\1. RoiJcrt
In the second session (2 days lated. subiects WeTe allowed to read the user manual and pla~' ~'ith the S\'stem for 10 minutes. Thev then had to com plete a l-hou r questionnaire about the s~'stem operation. The questlOns were about fttlWS and inconsIste ncies that ha"e been id entlfieel on the basIS of the formal descTlptlon. SubJects ',",re provided with info rmati on about the name l.lnU l)f~i.lnIZation of the mode ... and levels of the s\,stem. and- about the components of a drawing (dTawlng. oH rl a\, . s\,mbol. e lement ). When necessar~', they we re reminded of the tasks that can be done in a mode. In each questi on. the use of Tel,,'a nt commands was defined : there were multiple -choice answeTS, and an opt ion "I d" not know" was included, This questionnalTe had been tested with fuur pilot sub.iects. Pro,'isions had so been made to insu re that the subJect understood the conte xt of a question . the mod~ or le vel of the system that " 'as conce rned. a nd the vocabulary used. RESLI.TS This section presents the results of 14 predicti ons tha t we Te made about the subjects' errors a nd difficulties with the ,,'stem . The Learning session, Before exposing these results, the learnin g session has to be examined. It is important to know if sub.iects succeeded in using the system , to give some ideas of what happened dUTing this sesslOn, and look at the indi\'idual differen ces in learning. Protocol analysis of the learning session indicates that all subiects were able to use the system within the first hou r of the session, and perform some real tasks of their chOice. The\' went back and forth between the Statement. Edit, 'a nd DTaw modes of the system, and they used all the commands but a few ones (different for each subjectl in each of these modes. Only one subject explored the Global Edit mode OT 36 seconds ). The da ta from t he few q uestions about the comma nds of this mode were included in the ana l\'sis. The answers about thiS mode were easily predictable from the ot heT modes.
er
TheTe were importa nt individual differences in learning. They appear in the number of times each command was selected, in the duration of use of each command. in the use of a command recursi\'el y or not, and in the few unexplored commands. FurthermOTe, there were diffeTences in the understanding of s~'mbol independence (one must exit from the Draw mode to CTeate two independent s~' mbols) and in the understanding of the role of the EDIT commanu in the Statement and the Edit modes. It ca n be concluded that each sUb.iect w as proficient enough with the system to ha ve a good idea of its ope ration. and be familiar with the menus. the commands. th e prompts. and the \·oca bular~·. Resu I ts of the VueslIon nai res. Sinct' the s\"<.;tem is menu -dri\"en. suhje cts did n(H rXl\" mUlh attentiun tp the t:om mand menu. Therefore . learning about the co mmand menu "'-as "in cide ntal". as one subiect put it. \10reo\'e r. the questionn aire was gi\"en two dJ .vs after the learning session, and asked man\" detai led qU"'tions. ,-\s s,,'eral subjects wrote on the questionna ires. a lot 01' "reasoning out" , or "l ogical understanding" of the s\,stem was necessan', This situation was createu purposel\' to lead subiects (w ho are knowledgeable with computers) to figure out the different rules on which the s\,stem was based . The sub.iects performar,ce in remembeTlng or figurlllg out a feature of the system is expected to be pvorer "ith the exceptions than with the rules,
T,,'o POlllts ought to be mentioned for the presentati on 01' the results: con.mands that are concerned are t he ones sh()wlng when a mode is entered, and for each question, there " 'as an "I do not kno"'" optIon .
Prediction J: Recursion. Because not all commands a re recurS] ,," ( 23°" of exceptions), sUb.jects will ha\'e mo re difficult~· rememberin g or figuring
out whIch commands are not recursive. E1,,'en questions were asked: six about recurSIve commands, and fi ve about non recursi"e ones. For each command . there was a choice between recuTsive or not. Resu lts show that the subJ"cts' success rate is 2(),;,c higher with recursive commands than \vith non recursive com mands (65<;, v's. 45C7,). The prediction that sub.Jects have more trouble with the except ion com mands IS confirmed,
1'I'ediction 2: Menu . \ot all commands give rise to a full menu 06% of commands bring a restricted
menu, anu 7%. an empt~' one). Therefore, SUbjects ",,11 ha ,," more difficulty remembering or figuring out "'hich C0m mands do not generate a full menu. Five questions were askeel, o ne about each mode of the "'stem. Each question waS concerned with a representatiw group of commands from a mode, and offered fi\'e possible chOices: 1. a full menu, 2. a full menu + CA\CEL. 3. a restTlcted menu comprising E\D and CA\CEL. 4. a res tricted menu comprising only CA\CEI., and 5. an empt\' menu. Results show that the subiects' success rate is much higher fo r commands giving a full menu (wi th or without CA\CELl than for commands giving a restricted or empty menu (88% \'S. 25%). In each mode where there is a mixing of commands giving rise to a full menu and comma nds gi\'in g rise to a restricted one. the difference in the success rates is Yery large. Most subJects th ought that exceptions ga\'e f ull menus. For example, seven out of eight subiects knew th at th e ARC command (,n the Draw mod e) ga\'e access to a full menu. where as on l\' 12 an d 0% of sub.iec ts w u lu contend that the TL:\ T and GRJJ) commands. r espect l\ · el~· . brought a restrlcteu menu.
,se"
Predictioll J: C J\ NC };J .. \ot all commands gen erate CA\CE I. when the\' are selected ( 19ck of exceptlons). Therefore. subiects should have more difficulty re membe rin g or figuring out which com mands do nnt generate C-\\CEJ.. Sennteen questions were asked: elewn about the commands generating CA\CEI., and six about the exception commands, Each question offered two choices (Yes or ,"0). Results show that t he subjects' success rate is much higher with the normal commands than with the exception commands (60Cf, \'5. 26%). ,-\11 subiects but one stressed within-a-mode consisten c~' for every mode. and man :v stressed between - modes consjstenc~· . Predictioll 4: C l RCLF The CIRCLE command does not beha"e like the ot her insertion commands from the Draw mode kg .. ,-\Rc:. DOT, TE\T) since it does n()t ge nera te C'.-\\CEJ.. Therefore, subiects shou ld ha ve more difficult\' remem ber ing or figurin g out that CIRCLE is an exce ption and does not gen erate CA\CEJ.. Onl\' two questions weTe asked about insert ion commands fr nm the Draw mode and CA\ CEL: one was about TE\T and the other, about C IR CU:. Resul ts show that 880" of subjects knew that TE\T was gene rating CA\CEI. ,,'hereas no o ne knew that CIRCLE was not generatin g c.-\\CU .. Prediction 5: CRI D, The GR]]) command from the Draw mode does not beho\'e like GR ID from the t hree other modes beca use it does not generates a menu with CA\CTL. Therefore, subjec ts should h3\'e more difficu lt\' remembering or figuring out the mode wherein GR ID does not generate CA\CEL. Three questi ons were asked abou t GR ID in the Edit, Global Edit. and Ora,,' modes. Eac h question offe red two choices ( Yes or \ ('). Results ind ica te that 50% of subiects knew that GRID from the Edit mode ge nerated a menu with C.'" \(:1: 1., 25<;,., that GRID from the Gl oba l Edit genera ted a menu with CA\CEL. and 25'"" that GRID from the Draw mode did not give rise to C-\\CEL. The prediction was confirmed with th e Fd it mode bu t not with the Global Edi t mode ("hlch was not exp lored). Pl'ediction 6: DRAW MOD!:'. Since one mode (Draw ) is com prised of CA\CEL in its first menu. subjects will ha\'e more difficult\' remembering or figuring OU t which mode is exception and includes C-\\CEL in its first menu. F,w questions were asked about the presence of C.-\\CEJ. In the first menu of modes, For each question. there was a chOICe between
.'\ Forlllal \kt!toc! Yes 0 r \0. Results sh0w that 87.S'~ of subjects answered that CA\CEI. was appearing In the Draw m ode. 75" ,. that it was appearing In the Edit mode. and 62 .5(",.. that is was appeanng In the Global Edit nlllde. The prediction that the 1)ra" m ode would not be seen as an exception wnh respect to CA\CEL was c()nhrmed.
F,edictiofl 7: };,ND CANCFL. [\D is the main command used to go into a higher In'e! mode. There fore. \'en' rew sub.!ects among those wh o think that CA\CEI. appears In the first menu of a mode. should be able to remember or figure out the use of the C,-\\CEL command appearing in the first menu of the Draw mode. ri,'e questions were asked about the use of C,-\\CEL ("' hen one answered th', t CA\CEI. appeared in the first menu of a mode). For each question . there were multiple choices: C'.\CEL allows to: go into a higher Ie'el mode: go into a lower le\'el mode: sta\ In the same mode: get a message in the prompt about the impos.sibilitv of us ing CA\CEL to exit from a mode. \\'hen asked about the use of CA\CEI. from the Draw mode. onl\' two subjects (out of se\'en wh o knew that C.-\\CEL " 'as showing in the Draw mode) correctl, contended that c.-\\CEl provoked a mode cha~ge (go into the higher level mode): four tl10ught that it was impo;sible to go into a higher le\'e l mode with C\\CEL. When asked the same question in the Edit and the Global Edit modes. among subjects (\ ~ 6 ) who erroneousl\' thought that CA\CEl appeared in the first menu of these modes. onlv one said that it ,,'ou ld lead into a higher le\'el mode. F0ur subjects contended that it was impossible to go int0 a higher le\'el mode with CA\CEL. This shows that there were more subjects thinking that CA\CEL ca nn ot lead into a h ig her le"el mode than there were subjects thinking the opposite. Subjects were confused about the role of CA\CEL. P,'edictiofl 11: UNDO. Ilecause not all commands give rise to an L'\ D() command when the,' are selected (16% of exceptions). subjects should have more difficult,' remembering or figuring out which commands d o not generate L\DO. For SUbjects knowledgeable with computers. it is postulated that L l\ DO is expected \\'ith commands allowing to create. delete , or modify. Fourteen questions were asked, three about the exception commands generating L'\DO, and eleven about the other commands. Each question had two choices (Yes or \0). Results show that the subjects' success rate was much higher with commands generating L' \DO than with commands not generating it (710/(. vs. 360(. ). Twice as man:,; suh:!ects knev.; about commands gi\'in g rise to C\f)() than about commands not generating it.
F, ediction Y: UNIX). Subltcts will h,,'e more difficult\' rememberin g or figuring out which com mands d0 not giH rise to L\IX) among those that a re d0Sely rebted to the ones bringing C\IXl. Commands such as DEI.ITE. .\10\T I b()th from Dra\\' mode). ERASE (from Edit mode). and RL'vl0\T (from Statement mode ) should be expected to beha\'e sim llarl\ because the ,Y have the same functionality of deletion or modification. Simil"rlv. commands such as CIRCl.E. ARC. DOT. Ll\E. POl.'y. and TE\T (from the Draw mode) should be expected to beha"e similarly because they are all concerned with creatl(' n, It was expected that subjects knew that RDIO\T did not comprise an L\[)() command because of the presen ce of the RESTORE com mand in the menu (RESTORE is equi"alen t to C\lX». Eight questions were asked. f0ur about the c reati on commands and f our about the deleti on or modification commands. Resul ts ab0ut the c reati on commands show that the subltcts' success rate was high for commands generating L \IX land , 'en' 10W for the exception CIRCLE command I 75'~,. ' ·s. 12.5';-,')' 110,,'e\·e r. as opposed w ClRCI.E. the success rate was fairl,' high f or the TF\T wmm"nd (SIl'i,.J. This result was wrongl~.. predicted: on ly 25C', of subjec ts th ou ght t hat L\D() wl' uld appear. Results about the delet ion or modif1ciJtlOn commi.lnds sh ow th<.Lt the success rate was birl." high f0f the w mm a nd I LRASE ) generat ing C\])() (c2.Sc-,). and "erv' It) w ror the com man ds
(DELETE ,!Od .\!
Prediction 10: PROMPTS fOl' EDIT. The prumpts aCCllmpanving the EDIT command in the Statement and the Edit modes are misleading. Therefore. all the sub.ltcts or a large majority of them shou Id make the same mistake when interpreting these prompts. "Edit the entire displa,'" should cause them to erroneousl v think that all the o\'erlavs forming a display c~n be edited at once (one overl~,· at a time can be edited). "Edit an oHrlay" should cause them to think that an entire oyerlav can be edited at once (one symbol at a time can be ~dited on the o\,er lay). Three questions were asl-ed. tw o about the prompts that appear when EDIT is selected from the Statement mode (two pos.sible argumen ts). and one about the prompt that appears when EDIT is selected from the Edit m ode ("' ith the argument O\TRI..-\ Y l. Results sh ow that all subjects got con fused when Interpreting the prompt" Edit the entire display". The\' all made the mistake that was predlcteJ. 7':'C, or suh.)ects g\){ (0nfused when inter preting the prompt" edit an overla\''' : they made the mistake that was predicted. Prediction 11: Number of steps for /:.'DI'J'. W e postulated that the same command appearing in different modes is expected to require the same number of steps to be executed. The EDIT command requires two steps in the Statement mode, and three steps in the Edit mode. Our predict ion is that all sub.ietts o r a great maJOrit,' of them should make the same mistake when tn'ing to remember or figure out the number of steps: the\' should think that this nu mber is the same in bot h modes. Two questions were asked: one abo ut the EDIT command from the Statement mode. and one about the EDIT command from the Edit mode. Resu lts show that onl v one subject th ought that the number or steps f or executing EDIT was the same in the t"'o modes. Therefore. our prediction was not canfi rmed. The reason cou id be that subjects m ay have simpl\ s uspected that there " 'as an exception. and that the tw o answers had to be different. Prediction 12: Statement mode. Because there is no need of a GRID command in the Statement mode, sublects should have m ore difficulty than in the ot her modes (Edit, Global Edit, and Draw) remembering or figuring out if GRID appears in the Statement m ode. ri"e questions were asked about the presence "I' the (jRID command in each mode. For each question. there was a choice between Yes or \ 0. Resul ts SI10W that subjects had m ore trouble with the Statement mode th~n \\'ith the other modes when dealing with GRID: 5('" of subje((, thou2ht that there " 'as no GRID in the Stat~ment mOde, whereas 25",. thought that there was n0 ne in the Edit o r the Gl ohal Edit mode . 37.5<;. that there " 'as none in the Draw mode. P,'edictioll J.j: De fault option, There is at least one command us ing the default option : CIRCLE (one can skip the ch oice or "radius" or "diameter" when drawing circles. and radius will be chosen by def'lult ), There should be some subjects thinking that other comm a nds use a default o ption as we ll. Fi ve questions were asked. one about CIRCLE and four about ot her candidate commands: SPI\. SCALE. DRAG , EDIT. These comman ds lend themsel"es t o the use of a defa ult option because on e step cou ld be easily s kipped. and because theY reqUired m ore steps (4) than ('ther cnmmands tl' be exec uted. Each question had tw o c hOi ces ( Yes or \ 0). Resu lts show that 25';-, of the subltcts th ourht that the commands SI'I\. SeALF. a nd DRAt; used-a default o ption. and 37.5(", . that FIJI')' ( fT(\m the Edit mOde) also used a default optI on.
J.
2(iO
~1.
P"edic/ion 1.J: LX IT. Because one can expect to get out of the S\'stem from the same mode as the entrance mode. some subjects should have difficulty remembering or figuring out which mode of the SYStem the E,\IT command appears m. and should thin).. that [,\IT appears in the Display mode. Presentlv. E,\IT shoves In the Statement mode. One question was asked ahout the mode wherein E,\IT was showIng. There were fiw chOices. corresponding to each mode of the system. Results show that 25% of the sub.Jfcts got confused and thought that E,\IT appeared In the Display mode.
Roben Finally, it would be useful to have some relative order of importance of the effect of various system's inconsistencies on the users' performance. The formal method is already a useful tool for designing human-machine i'nterfaces, but it needs some improvement to be used more rIgorously. REFERE'\CES Barnard. PJ" lIammond. ,\.\,,, \lorton, 1., Long, 1.B. (1981). ConSistency and compatibilIty in human computer dialogue: Int.crnational Journal of Man \lachine Studies Vol. 15 pp.
Reco mmendatillns.
The design of user Interfaces is the result of man.\" compromises:
11
Almost alw;):vs, there are trade -
offs." (Smith et al" 1982) The path is not always clear-cut. Recommendations proposed to Improve the consIStencY and predictabilit\· of the interface studied here should be con"dered in this perspectin. Exceptions to rules should be banIShed entirely to simplify the system. Consequently. all commands should be recursi\·e. They should all give rise to a full menu and to CA\CTL when selected. C.'..'\CEL should be strictlv resened for canceling a command that was Just selected. v,hereas E,\D should be strictly used for between -modes changes. L'\l)() should be a\'ailable with all commands allOWing one to create. delete . or modif,·. The same command used in different modes should always be comprised of the same number of steps to be executed. The default option should be used with all commands that lend themselyes to it: for example. commands requiring more steps than the average to be executed (e.g., SPI'\, SCALE, DRAG). or having steps that can easily be skipped for most of the tasks performed with these commands. Feedback should always be provided with commands to inform users of success, failure, or partial success. There were several complaints ahout the lack of feedback with various commands (e.g., GATHER, ['\PLODE, SIZE). FinallY. since the entrance and the exit modes are expected to be the same in a system, E,\IT should appear in the Display mode instead of the Statement mode. Similar recommendations ahout the presentation and location of information on the screen could be proposed. These modifications should impro\'e the Interface consistency and predictability. and make the
s~'stem
easier to learn and use.
CO,\CI.LSIO,\ This research examined the use of a formal method to design and eyaluate user interf<.Jces. It focused on conSistency and predictability. The method allowed one to make a s,'stematic description of the users' actions and system's resJX>nses. to iden tiCy flaws ..md inconsistencies of the s,ystem, and
make predictions ahout users errors and dIfficulties with the SYstem. These predictions were tested with users and were confirmed ( with few exceptions ). \lany modifica tions to improve the interface consistenc\' and predictability. and make the system easier to learn and use "'ere proposed. These modifications can be tested iteratively. The formal method turns out to be a powerful tool for analyzIng and predicting the consistency of interfaces. lluw eyer. it still has some shortcomings that will have to be tackled. First, despite the precI sion and completeness of the formal description. there
IS
no guarantee that different persons \vould
identify the same Inconsistencies and make the same predicti ons. The prediction process has to be more explicit. For example. there is a need for a list of factors to be checked for consistency of the user interface. The lIst presented in this paper can certainiv' serve as a good basis, but it still has to be extended. Second, it IS difficult to measure the scope of each prediction. Is It affecling all, or several, or a fe"" users1 We certainly had some intuition ahout it, but this is not sufficient. Third, we would be unable to make different predicti ons for users with different levels of qualifications. Some work has been done on thiS topic by Reisner (1982); it should be pursued.
Gould. 1.0., I.ewis, C. (1984). Designing for usabilItynl\C)' principles and what designers tb.ink., Res. Rep. 10317, 1B\1 \\'atson Research Center, Yorktown Heights, '\.1'. 25 pp. Good , \1.1)" Whiteside. 1.A .. \\·ixon. D.R .. lones. SJ. (1984). flulldlng a Lser -Derind Interface. Communi cations of the -\('\1 \01. 27 \0 10 pp. 1032- ]()43. lacob, RJ.I\. (1981). Survey o[ SpeCification Tech niques for Lser Interfaces" Technical \1emoTaouum
7590-.'()3:Rl:rl. \a\al Research I.ahorator,·. Washing ton. D,C
lacob, RJ.I\. (1983). LSlng Formal Specifications in the Design of a Human -Computer Interface. Cilln:: munications of the .'..C\1 \'01. 26 \0..1. pp. 259-264. Ledgard, H.F .. Singer. A. (1978). Formal definition and design. COI,\S Tech. Rep. 78-01, Feb., Lni\'. of Massachusetts. \1.'... 6 p. Parnas, D.L (1969). Un the use of transition dIagrams in the design of a user interface [or an interactlw computer svstem. In Proceedings of the 24th \ational AC\1 Conference. .'..C\l. \ew York. pp. 379-385. Reisner, P. (1981). Formal Grammar and Human Factors Design of an Interactive Graphics System. Il.:EE Transactions on SQftware Fngineering Vol. SE-~ 2. \la rch , pp. 229-24(). Reisner, P. (1982). Further de\elopments toward using f,)rmal grammar as a deSign tool. In E=dings of Human Factors ID Computer Systems \Iarch 1517. (jalthershu rg, \1arvland. pp. 30-1-308. Shneiderman , ll. (1982). \lultlparty Grammars and Related Features for Defining Interactive Systems. IEEE Transactions on Systems \lan and Cybernetics \,QI. S\1C- 12 \Q) \larch ·April. pp. 148-15-1. Smith. D.e.. IrbY. C, I\imball, R.. \'erplank, fl., lIarslem, E. (1982). Designing the Star Lser Interface. Byte \'01. 7 \04 '\pril pp. 242 -2 82. Woods, W ..-\. ( 1970). Transition '\etwork Grammars for '\atural Language AnalysIS. Communications of tb..e.....:\.Qi \'01. 13 \0 J U. Oct.. pp. 591-606. Acknowledgement.
I want to thank Robin Wise who spent much time transcrIhlng the protocol, and helped to analyze the data.