- Email: [email protected]
A comparison of simple hierarchy and grid metaphors for option layouts on small-size screens
ARTICLE IN PRESS
Int. J. Human-Computer Studies 60 (2004) 564–584
A comparison of simple hierarchy and grid metaphors for option layouts on small-size screens John Christiea, Raymond M. Kleina, Carolyn Wattersb,* b
a Department of Psychology, Dalhousie University, Halifax, NS, Canada Faculty of Computer Science, Dalhousie University, 6050 University Avenue, Halifax, NS, Canada B3H 3W5
Received 26 March 2003; received in revised form 8 October 2003; accepted 20 October 2003
Abstract Modern, technologically driven society is characterized by an increase in the rate of mobile device use and an increase in the extent to which these devices are used for more complex tasks than search for phone numbers. While direct consequences of screen-size reduction on task performance are well known, data are lacking on the impact of layout of multiple options in a complex task environment. In this paper we describe the results of an end-user study in which we compared two basic interface designs for the layout of multiple options: a simple hierarchy and a single layer grid. These two designs were presented to users on two screen sizes; a larger size approximating the size of a standard laptop or desktop screen and a smaller size approximating the size of a PDA screen. This study illustrates that while task performance in accessing information was superior using the grid interface rather than the simple hierarchy interface, users preferred the simple hierarchy interface. Even as the complexity of the task increased, the physical size of the screen had a significantly bigger impact on task performance than did the task complexity. These findings indicate that the grid layout should be used when task performance is of the paramount concern and the complexity of choices is not expected to be large. When user preference/satisfaction is more important than task performance the appeal of the simple hierarchy layout may supersede the cost in performance it entails. r 2003 Elsevier Ltd. All rights reserved.
*Corresponding author. Tel.: +902-494-6721; fax: +902-492-1517. E-mail address: [email protected] (C. Watters). 1071-5819/$ - see front matter r 2003 Elsevier Ltd. All rights reserved. doi:10.1016/j.ijhcs.2003.10.003
ARTICLE IN PRESS J. Christie et al. / Int. J. Human-Computer Studies 60 (2004) 564–584
565
1. Introduction The use of mobile devices, such as Personal Data Assistants (PDAs) and smart phones, has increased dramatically in recent years. Not only has usage increased, but so has the complexity of the tasks for which these devices are being used. The increased capability of these appliances supports applications for tasks that may include a series of activities such as lookups, selections, comparisons, and data input. Furthermore, many users of mobile appliances also use desktop and laptop applications and may switch or migrate between these devices frequently. It has become important, therefore, to understand the impact that screen reduction has on both task performance and user preference so that interfaces can be designed for use on small appliances, such as those that may be deployed in education (Fulp and Fulp, 2002) or telehealth (Rodger and Pendharkar, 2000). Interface designers for such applications should aim to optimize performance by minimizing confusion, learning time, and error rates. In this paper we concentrate on two interfaces for the presentation and navigation of options across screen sizes, the simple hierarchy and the single layer grid. We use the terms interface and layout interchangeably. The simple hierarchical layout presents the options at the end of ‘‘paths’’ through which the user proceeds by a sequence of choices, while the grid layout presents all of the options to the user at once. The simple hierarchy layout initially presents the user with fewer but higher level abstractions from which a choice is required to get to a subset of the individual options and it may be necessary for the user to backup and repeat choices before the correct option is found. The grid layout presents a larger array of choices from which the user needs to find the appropriate option but the complete set of individual options are available at once. A trade-off then exists in layout choice between the time needed to perform a search through many options in the grid layout and the time needed to perform multiple selections among fewer options at each step of the simple hierarchy layout. We are interested in finding whether the simple hierarchical layout or the single layer grid layout provides better task performance in a search task using two different screen sizes. We are also interested in finding out which interface most users prefer. Our experimental design to test for performance effectiveness and user preferences across the two screen sizes is outlined in Table 1.
Table 1 Outline of experimental design PDA-size screen Performance (grid vs. hierarchy) Preference (grid vs. hierarchy)
Laptop-size screen
ARTICLE IN PRESS 566
J. Christie et al. / Int. J. Human-Computer Studies 60 (2004) 564–584
2. Background Over the past 20 years there have been many studies on the performance of users reading from screens and the effects of small screen size on performance. Two issues from this research are directly relevant to the current study: screen size and interface design or layout. Screen size. On small screens the frequency of scrolling can be expected to be higher simply because less data are presented on the screen. Research has shown (Sekey and Tietz, 1982) that left–right or horizontal scrolling degrades reading performance and that vertical scrolling, while acceptable for skimming or scanning tasks, degrades performance on actual reading or comparison types of tasks. Studies by Jones et al. (1999) indicate that scrolling on small screens leads to decreases in both speed and effectiveness in performing tasks. A related finding is that the width of lines of text and screen size do not affect the users ability to comprehend text, although a reduction in the size of the screen leads to an increase in scrolling activities by the reader (Dillon et al., 1990). The readability of options has also been shown to affect task performance and must be considered in any screen-size reductions. Clearly, it is harder for users to make correct choices when the font or icon is too small to be easily recognizable. Even if the text or image is readable, performance may be undermined if the user has difficulty activating or indicating a choice because the size of the option is too small to target. Fitts’ (1954) law indicates that as the size of the buttons become smaller they will be more difficult to target accurately. There will clearly be a trade-off between presenting all of the options to the user, when the options must be reduced to fit into the smaller space, and the users’ ability to identify and select the correct option. Overall, research indicates that reducing screen size negatively affects user performance for a variety of tasks but that some features can be incorporated that mitigate the effect. One such feature is the use of a search function, particularly with lists and text (Reseil and Shneiderman, 1987). Another feature, for tabular data, is the inclusion of contextual information so that the user can manipulate the cell information within its larger context (Watters et al., 2003). Option layout patterns. One of the appeals of handheld devices is that they are very portable. Unfortunately, this increased portability means that the environment in which they are used is often distracting. The competing demands on perception and working memory, simplicity and directness of the navigational model, and fitness of a particular task become important performance effectiveness factors (Jameson et al., 1999). The choice of layout for multiple options may also be an important factor in improving performance effectiveness for certain tasks on small screens. 2.1. Full size screen factors The use of menus to organize options has become nearly ubiquitous in interface design. Menus have obvious appeal on small screens because they can remain hidden much of the time. Menus tend to be organized either in a linear fashion or in a
ARTICLE IN PRESS J. Christie et al. / Int. J. Human-Computer Studies 60 (2004) 564–584
567
hierarchic fashion. Menus allow a user to scan and find the feature of interest or make an initial choice followed by one or more narrowly focused choices. A study by Resiel and Shneiderman (1987) tested user performance on lists of different lengths and concluded that users performed the task equally well whether they saw the entire list or only a few sections at a time. A study by Duchnicky and Kolers (1983), however, showed that the amount of scrolling by users increased as lists got longer than the screen and further that effectiveness decreased as the length of the visible portion of the list was reduced. Swierenga (1990) found no difference in performance in using hierarchical menus as the list got longer but Han and Kwahk (1994) did find a difference in performance in using hierarchical menus as the list got longer. Studies by Wallace et al. (1987) indicate that users perform better on shallow hierarchies than on deep hierarchies. Frey et al. (1992) indicate that a hierarchical menu gives better performance than scrolling through a list of choices. The dashboard metaphor (Lundell and Anderson, 1995) collocates options and views on a single screen for ready access by the user to multiple functional options. Desktops and common tab bars on web site pages are modern versions of the dashboard. This layout design provides consistency for the user and a one-stop-shop that reduces the cognitive effort of the user to remember where a user is in relation to the decision points. The number of choices is of course limited by screen size and recognition factors. A study by Mitchell and Miller (1986) indicated that, when choices are grouped according to their relationship to the given task, performance improved, and Staggers (1993) found that a single display showing the whole system also led to improved performance.
2.2. Small screen factors Jones et al. (1999) found that users preferred focused navigation and direct access strategies when using small screens. Simple hierarchical layouts are a commonly used method of representing choices on small screens. News sites, for example, replace the original data with a short list of categories of news types that fits on the screen. The user selects one category, such as sports, and a list of sports article titles is presented on the screen. From this list of titles, the user can select a title and the full text or a summary of the article will be presented. While studies have shown (Watters et al., 1997) that scanning the representative chunks of news data is more effective than using a hierarchy of headings on the larger screens where the task is related to browsing or scanning, Jones et al. (1999) found that scrolling on small screens reduces the task effectiveness for the user. The small size of screens means that even small lists may require hierarchies of several levels to accommodate a few sets of choices. We expect that the effect of navigation options and layout will be exaggerated on the small-size screen and possibly further hindered by the limited input capability of handheld devices.
ARTICLE IN PRESS 568
J. Christie et al. / Int. J. Human-Computer Studies 60 (2004) 564–584
3. Study In this paper we examine two layouts for presenting a relatively small and finite number of options (categories), 16 or 25, to users with lookup tasks. One layout is a shallow (two-level) hierarchy and the other is the grid layout, a variation of the dashboard metaphor. These are shown in Fig. 1 with the grid layout on the left and
Fig. 1. Screenshots of the experiment. All of the grid layouts are on the left and the top level of the simple hierarchy layouts are on the right. The first row is handheld and the last two are desktop. The first two rows are for 16 item complexity and the last two are for 25 item complexity.
ARTICLE IN PRESS J. Christie et al. / Int. J. Human-Computer Studies 60 (2004) 564–584
569
the first level of the simple hierarchy on the right. With the simple hierarchy layout, when one item is selected the array of items would be replaced with the second level or actual options. The grid layout presents all options to the user and is constrained as the number of options increases. We did not test the linear dropdown menu model in this study, as there is no literature supporting its use on small-size screens for lists longer than the screen size. 3.1. Goals of the study The primary goals of this study were to assess the relative performance with, and user preferences for, simple hierarchy and grid layouts for use in search tasks on smaller screens. The simple hierarchy is a layout in which the choices are organized into a shallow hierarchy. The simple hierarchy layouts are often found in GUI based applications. In the simple hierarchy layout, a few choices are presented on each level of the interface; choices at the first level of the interface lead to deeper levels until the appropriate function is found. Any one selection is not necessarily an end in itself, but a step along a path to the final goal. In the grid layout all of the options are presented to the user at the same time. In this case, each choice is an end in itself. It is interesting to note that the grid layout is commonly found in devices of relatively low complexity or with actual buttons, such as a telephone or simple calculator while more complex devices driven by software, such as the VCR remote or PDA, generally use a hierarchy of options. In a previous study by Christie and Klein (1997), comparing human performance measured in reaction time and accuracy, it was shown that for simple interfaces with nine possible options the grid layout was superior than the simple hierarchy layout. In that study, a simple hierarchy of three buttons on the first level led to three target items accessible on each of the three possible second levels. In the grid layout all nine buttons were simultaneously presented to the participants. Participants were faster to access target items using the grid layout than the simple hierarchy layout. In the current study, we question whether the findings of Christie and Klein (1997) study will hold true for more complex interfaces, which include more target items, and more complex tasks. A complex task requires a two-step answer over simple recognition. Since the increased response time using the simple hierarchy layout can be attributed to the extra cognitive and motor step of selecting a choice to move to the second level of the interface, we predicted that the performance difference between the grid layout and the simple hierarchy layout might be reversed when the interface became sufficiently complex. Consider a simple complexity metric based on time to support this argument. Additional time is required to access a subsequent level and this time would be constant time factor, Y. Further we assume that there is a fixed amount of time, X, that is needed per button to search through each item in a display to find any one button (Wolfe, 1998). As an example, consider the earlier study, which had nine buttons. If a user checks each of the possible buttons before making a choice, the time to find a button in the grid is the number of buttons plus a constant (k), in this
ARTICLE IN PRESS 570
J. Christie et al. / Int. J. Human-Computer Studies 60 (2004) 564–584
case 9X+k. In the hierarchy, the time to find a button depends on the number of items on the first level plus the number of items on the second level, 3X + k per level in this case, and the constant Y to change level, for a total of 6X+Y+2k units of time. Of course, it is likely that searches would actually be self-terminating rather than exhaustive, i.e., the user will stop examining buttons when the target is found. On average the user finds the button in half the time, in both the simple hierarchy and grid layouts. Therefore, it takes 5X+k for the grid layout and 3X+Y+2k for the simple hierarchy layout. As device complexity increases the multiplier for X increases faster for the grid layout than for the simple hierarchy layout. For example, at 25 items search time for the grid will be 13X+k, but search time for the hierarchy will only be 6X+Y+2k. Let us call the difference in the number of options inspected (NX) between the grid layout and the simple hierarchy layout, DT. In the present experiment where 16 items are used DT= 8X 4X=4X. Using low complexity interfaces, performance using the simple hierarchy layout will be worse than using the grid layout because of the additional Y+k constant. But, because the number of items to be searched in the grid layout rises much more steeply than for the simple hierarchy layout, at some point the performance advantage should shift to the simple hierarchy layout as the time represented by DT exceeds Y+k. An additional factor in performance is that option selection likely follows Fitt’s law. As the number of options increases passed certain threshold, the size of some or all of the items on the display must decrease. This will occur faster for the grid layout than for the simple hierarchy layout. In the current experiment for 16-item complexity, the maximum size of buttons in the grid will be 1/16th the size of the display while those in the simple hierarchy layout will be 1/4th the size of the display. But, in the 25-item complexity condition, the grid buttons will reduce to 1/25th of the size of the display while the buttons in the simple hierarchy layout will be 1/5th of the size of the display. Therefore, as complexity increases it may be hard for users to target buttons the grid layout. When the user is in motion or in a distracting environment small-size buttons may not be desirable. 3.2. Experiment In the current study we follow up the effect of layout of choices (Christie and Klein, 1997) in the context of the small screens typical of handheld devices. We also increased the complexity of the task and layout by increasing the number of options from which the user had to make a selection. We wanted to see both how performance is affected by the layout of the options and different layouts influenced participant’s subjective impressions. If a difference in performance can be demonstrated then understanding the relationship of given tasks to the organization of choices should permit optimization of screen layout for given tasks. However, the usefulness of performance measures may be mediated by user preferences. In order to test the impact of task complexity, two complexity scenarios were used. The first scenario consisted of 16-items, and the second scenario consisted of 25items. The number of levels for each scenario for the simple hierarchy layout was two, and the number of levels for each scenario for the grid layout was one. The
ARTICLE IN PRESS J. Christie et al. / Int. J. Human-Computer Studies 60 (2004) 564–584
571
complexity levels have an average DT values of 4X and 7X for the simple hierarchy and grid layouts. It is anticipated that higher DT will approach or exceed Y+k. In this within subject experiment participants were required to find a target word based on a two-word clue using both layouts. For example, given a clue makes holes the target would be Drill. In the grid layout shown on the left in Fig. 1, Drill can be accessed directly by the user, while in the simple hierarchy layout shown on the right in Fig. 1, the user would first select Tool and subsequently Drill. Table 2 provides a list of the clues and solutions for the 16 and 25 item conditions. This task mimics the user learning of interfaces that are used to translate concepts into actions. For example, when a user wants to see what is in a file in most computer operating systems, the concept must be mapped to the action File:Open. The table includes the top-level headers used in the Simple Hierarchy version and the corresponding second level target item groups. In the grid condition all of the target items would be presented.
Table 2 A list of clues and solutions for 25-item complexity Header
Target
Clue
Baseball Football Golf Hockey Running
home run touch down hole in one fast skaters pound pavement
Tool
Chisel Drill Hammer Sander Saw
wood shaper makes holes drives nails surface cuts stuff
Vegetable
Carrot Corn Onion pea Potato
orange root yellow kernel tear maker in a pod french fry
Vehicle
Bicycle Boat Carriage Plane Train
two wheeler water craft horse drawn high flyer rail racer
Weather
Rain Snow Sunshine Tornado Wind
falling drops drifting flakes warm brightness fast funnel blowing air
(Simple Path only) Sport
ARTICLE IN PRESS 572
J. Christie et al. / Int. J. Human-Computer Studies 60 (2004) 564–584
In order to test the effect of screen size the 16-item scenario was used both in a small screen size, similar to that of a handheld device, and a large screen size, similar to that of a desktop computer. The ratio of small screen size and large screen size is similar to the ratio of the size of upper panes and the size of lower two panes illustrated in Fig. 1. The use of different screen sizes allowed for the assessment of the possibility that, in addition to the performance differences due to the complexity of an interface, perhaps screen size may play a part in determining the user task performance. The screen-size variable was not used for the 25-item complexity scenario because for the small screen size, the grid layout with 25-item complexity was already unreadable. Reaction time and accuracy measures were recorded automatically. In addition, a questionnaire was presented following the performance experiment in which participants subjectively compared the graphical user interface metaphors by rating their perceived performance and preferred layout.
4. Methodology Thirty-six undergraduate psychology students participated in the experiment. Participants were randomly assigned to one of three groups, which experienced the following combinations of display size and display complexity: large (desktop size) with 25 options; large (desktop size) with 16 options; small (handheld size) with 16 options. Each individual participated in both simple hierarchy layout, called the Simple Path, and grid layout, called the Gateway, conditions, with the order counterbalanced across participants. Each trial consisted of a search for the solution to one clue, shown at the bottom of each screen. A block of trials for each participant consisted of the complete set of clues for one interface complexity, i.e., 16 or 25 trials. There were 12 blocks per layout, per participant making 24 blocks in total with block 13 representing a shift from one layout to the other. Half of the participants used simple hierarchy layout followed by grid layout while the rest used the grid layout first and then the simple hierarchy layout. In the experiment, the two screen sizes were presented in grey in the centre of the monitor while the rest of the monitor area was black. The small size, meant to represent a handheld device, measured 41 mm wide and 83 mm high; and the large size, meant to represent a desktop screen, measured 122 mm wide and 86 mm high. Participants were requested to sit at a comfortable distance from the machine while viewing the text displayed in Chicago font. The text ranged from 9-point font size for the grid layout on the handheld size to 32 point for the simple hierarchy layout at the desktop device size and 16-item complexity. Example pictures of each possible display size and complexity are in Fig. 1. Table 2 contains a listing of all of the clue strings and the corresponding target word. Items in Table 2 shown in italics were not used in the lower complexity version, i.e., 16 items rather than 25 items. Participants were instructed that they would be playing a multiple choice word clue game. During the experiment the participant first viewed a black display with a grey rectangle in the centre that was either the small or large size display size. A small
ARTICLE IN PRESS J. Christie et al. / Int. J. Human-Computer Studies 60 (2004) 564–584
573
square in the centre of that rectangle was clicked to begin the experiment. In the grid condition the display was replaced by a grid layout of 16 or 25 target buttons ordered alphabetically. The clue appeared at the bottom of the display area. The participant would click on target buttons to try to select the correct one (up to five errors were permitted). After selecting the correct response, or five tries, the sequence would cycle on to the next clue. Each participant would repeat this for all 12 clues in random order. In the simple hierarchy layout, the options were not presented all at the same time. Rather there would be a presentation of words naming the categories of potential targets as well as the clue presented beneath the display. The subject would select a category that the answer to the clue should be in and the display would be replaced with a set of 4 or 5 targets within that category. The user could then select one of those target items or move back up to the first level by pressing a button above the items. The subject navigated the display until the correct target was selected or until five incorrect target items were selected. All subjects completed one session using the simple hierarchy layout and one session using the grid layout and then completed a simple questionnaire, shown in Fig. 2. Due to the researcher oversight two participants in each of two of the groups were not given the questionnaire. 5. Results 5.1. Performance All participants made very few errors and there were no significant accuracy effects that differentiated complexity, size, or layout condition. A block of trials
Fig. 2. Questionnaire for each participant after completing the experiment.
ARTICLE IN PRESS J. Christie et al. / Int. J. Human-Computer Studies 60 (2004) 564–584
574
consisted of the complete set of clues for one interface complexity. There were 12 such blocks for each layout, for a total of 24 blocks. For block one the errors were 9% for the grid layout, and 7% for the simple hierarchy layout. By block 2 these had dropped to less than 5%, and in all blocks after that the means were 3% or less. A failure resulted when a total of five errors occurred during one trial. In block one there was an average of one failure per person, by block 2 there was an average of 0.5 failures, and by block 7 there was an average of 0. Reaction time, shown in Fig. 3, improved consistently in all conditions until around block 7 by which time asymptotic levels of performance had been achieved. In Block 13 the layout was switched from one layout to the other; this dropped performance significantly. Performance reached asymptote after the layout switch in about six blocks. Performance was better for the grid layout than for the simple hierarchy layout in both display sizes and both complexities. Four analyses of variance (ANOVAs) were carried out on the reaction time data. In the first ANOVA all three groups were included (Appendix A). The second and third ANOVAs (Appendices B and C) were focused on interface complexity (16 or 25 options, with the large display size), and the interface sizes (large or small, with 16 options), respectively. In all of the analyses, block (1–12), layout condition (hierarchy vs. grid), and order of layout presentation were within-subject factors and group was a between-subject factor. A final analysis was carried out on the 16 item HandHeld
25 item Desktop
16 item Desktop
6500
Simple path Gatewa
6000 5500 5000 4500 4000 3500 3000 2500 2000 1500 2
6 10 14 18 22
2
6 10 14 18 22 Blocks
2
6 10 14 18 22
Fig. 3. The mean reaction time performance for all subjects in both simple hierarchy and grid layouts. Each graph represents a separate group of participants.
ARTICLE IN PRESS J. Christie et al. / Int. J. Human-Computer Studies 60 (2004) 564–584
575
simple hierarchy layout to examine the effect, if any, of the relative position of buttons in subsequent layers in the interface (Appendix D). One might note that font size perfectly covaried with condition and screen size. Although one might attribute some performance changes to changes in font size, this was not varied independently. It is clear from the data that diminishing font size had little negative effect because the smallest font size for each screen size was in the grid layout, where performance was best. 5.2. Analysis across three groups There was a marginal effect of group, F(2,30)=3.065, po0.07, and only one interaction involving group: group, layout, and block, F(22,330)=2.80, po0.001. Both of these effects will be discussed in the next section where the present group variable is broken out into display size and complexity. Performance improved with practice (main effect of block: F(11, 330) =102.89, po0.001) and was better with the grid layout than with the simple hierarchy layout (F(1, 30)=353.13, po0.001). These main effects are qualified by several interactions. Layout interacted with the order in which the layouts were presented, F(1, 30)=47.14, po0.001, and also separately with block, F(11, 330)=2.70, p=0.02. In addition, there were two higher order interactions with layout, the aforementioned interaction with block and group, and the other with block and order, F(11, 330)=57.53, po0.001. There were no higher order interactions. The nature of the layout block order interaction is fairly straight-forward. When looking at Fig. 3 it can be seen that the order of presentation modifies the effects of the condition on the performance at the block level. If the grid layout is used first, performance improves faster over the initial few blocks than if the simple hierarchy layout is used first. It may be tempting to conclude that the grid layout is learned faster. However, the interaction could be caused by the generally better performance using the grid layout being achieved at the same time as best performance in the simple hierarchy layout, resulting in steeper reaction time slopes over the first few blocks, but not faster learning. 5.3. Analysis of display size and layout complexity An analysis of the 16-item scenario using the desktop and handheld sizes (the top and center panels of Fig. 1) revealed that participants were significantly faster at finding targets using the desktop rather than the handheld display sizes, F(1,20)=4.15, p=0.05. Contrary to predictions, an analysis of the desktop size using layouts with 16 and 25 options (the centre and bottom panels of Fig. 1) revealed no effect of complexity, F(1,20)=1.075, p=0.31. Thus the marginal effect of group reported above was driven by the screen-size variable and not by the task complexity variable. There were no interactions between the screen size and other variables. There was one higher order interaction among layout, block, and complexity, F(11,220)=4.07, po0.001. This effect is confined to the first two blocks of trials. In
ARTICLE IN PRESS 576
J. Christie et al. / Int. J. Human-Computer Studies 60 (2004) 564–584
block 1 of the 25-item desktop size trial, performance is poorer using the grid layout than simple hierarchy. This is the only time that this happens across comparable blocks in the entire experiment. It is rather surprising to us that this effect goes away within only one block of practice. Given that it does not happen at all if one has practice with the other interface first, it appears it is not simply because the layout of the options is learned. Finally, an analysis of the simple hierarchy layout data was conducted in order to see if an additional effect from Christie and Klein (1997) was replicated. In designing the simple hierarchy layout, target items in the second level can be positioned in the same location as the button that was used to access that level, they can be in an adjacent location, or they can be in a non-adjacent location. In this earlier study with nine items using the simple hierarchy layout, when the option selected in the second level was in the same location as the previously selected first level option there was a 557 ms benefit in performance compared to targets that were not in the same location. This effect was found to also be significant in the present study across all blocks, F(2,70)=22.68, po0.001, and is illustrated in Fig. 3. (ANOVA results are presented in Appendix D.) The reaction times, in milliseconds, are shown for the simple hierarchy layout as a function of whether the item in the subcategory was in the same location as the item in the initial interface, an adjacent location, or whether the items was in a non-adjacent location. Block 1 is plotted separate from the other blocks because the performance pattern was significantly different. A test of adjacent items vs. non-adjacent items was also carried out. As can be seen in Fig. 4 there was a significant advantage for non-adjacent location over adjacent in the first block (5066 ms vs. 3978 ms), F(1,35)=7.23, po0.01; but in subsequent blocks this advantage was not present, Fo1. One might note that the 16-item data in Fig. 4 comes from the same source as the desktop data from Fig. 3. 5.4. Subjective ratings The subjective ratings from the questionnaire are shown in Table 3 where the numbers represent the proportion of respondents who indicated that the simple hierarchy layout was Better than, Easier than, and Preferred to the grid layout. With the desktop screen size and 16 items in the interface, participants found that the simple hierarchy layout was easier to use and preferred it. None of the other values differed from chance (0.5). However, chance may not be the proper comparison. It can be argued that the proportion of participants that performed better in the simple hierarchy layout is the correct ratio. In this instance that is 0.0, and all of these numbers are greater than that by a Sign test, po0.001. Subjective impressions of the interface did not appear to be affected by performance and significantly deviated from what would be predicted if preferences did follow performance. Intercorrelations among the various subjective ratings revealed that, individuals prefer the layout on which they believe they performed better, r=0.57, po0.01. There was a moderate correlation between what was preferred and what was considered easier, r=0.35, p=0.05. But, no correlation, r= 0.11, p=0.54, was found between the layout that they believed gave the best performance and the one
ARTICLE IN PRESS J. Christie et al. / Int. J. Human-Computer Studies 60 (2004) 564–584
577
Simple Path Position
5250 5000
Block1
4750
Block 2-12
4500 4250
RT
4000 3750 3500 3250 3000 2750 2500 2250 2000 Repeat
Adjacent
Non-Adjacent
Fig. 4. Reaction time in milliseconds.
Table 3 The subjective rating data Questions
Handheld with 16 items N=10 Desktop with 16 items N=10 Desktop with 25 items N=12
Better
Easier
Preferred
0.40
0.50
0.50
0.60
0.75
0.80
0.58
0.33
0.58
that was considered easier. The correlation between the magnitude of the performance benefit for the grid layout over the simple hierarchy layout, and the degree of preference for the simple hierarchy layout was negative, and significant, r= 0.52, po0.01. The performance advantage for the grid layout was also negatively correlated with the layout that the individuals believed they performed better with, r= 0.36, po0.05. These results imply that the subjective measures were sensitive to the performance. Participants were less likely to prefer the simple
ARTICLE IN PRESS 578
J. Christie et al. / Int. J. Human-Computer Studies 60 (2004) 564–584
hierarchy layout as the performance advantage for the grid layout increased. However, the fact that performance was always better with the grid layout was not sufficient for it to be preferred, just more likely to be preferred if the advantage was very large.
6. Discussion In this study we found that the screen size of an interface significantly influences performance in finding target items. However, contrary to our predictions, increasing the complexity of the interface, from 16 to 25 items, had no effect on performance after the interface was learned. Furthermore, it was discovered that, even though individuals were consistently much more efficient at finding targets using the grid layout in our study, their subjective judgments of the interface did not show a preference for that interface, and sometimes showed just the opposite. While there was a performance advantage in using the grid layout, participants preferred the simple hierarchy layout. Extending previous conclusions (Christie and Klein, 1997), these findings lead us to suggest that in situations where an interface is static and not likely to significantly increase in complexity, a grid layout is the most efficient for the user as long as the buttons are large enough to read and use. The grid layout has been shown to be the better choice, from the point of view of performance, and useful when the number of potential options can fit on the screen (or device, if hard wired). If maximum performance is required, the grid layout is the preferred interface given that above conditions are satisfied. The pragmatic argument that says ‘‘if the interface is subject to potential higher complexity a simple hierarchy should be used’’ is true. A grid-based interface is limited by how many buttons can be presented in a clear and accessible fashion; a hierarchical interface is not. We did not present the 25-option condition using the small screen size because the option labels were not readable. In spite of the reduced performance in the use of the simple hierarchy layout, subjective perceptions of the interface are positive. It is rated as easy or easier to use than the grid layout. Therefore, it is believed that, although performance in tasks like the one we have explored here will be reduced, a simple hierarchy is an excellent substitute for the grid layout in interfaces, and users may actually prefer it. Thus, when user preference is more important than performance, the simple hierarchy appears to be the layout of choice. One of the more interesting and novel facts that arise from this experiment is that the physical display size affects performance much more than the complexity of the display. This was true for both hierarchy and grid cases. A small display hindered performance while a more complex display had no effect on performance for the tasks studied. The fundamental message here is that efforts to increase portability and compactness should be tempered with an understanding that the size of a display is very critical. For any specific device and task, optimal size parameters should be investigated.
ARTICLE IN PRESS J. Christie et al. / Int. J. Human-Computer Studies 60 (2004) 564–584
579
Recognizing other factors found in the results can also facilitate the use of simple hierarchies in interface design. Performance would be improved if the most frequently used button on the subsequent level in a hierarchy is in the same location as the button used on the preceding level, based on our finding that successive responses in the same location are made much faster than in different locations. It was also found that participants were poorer in selecting the target item adjacent to the location of the item required to get to that level of the hierarchy, but only in the first block. Therefore, cognitive factors that might limit performance in items immediately adjacent to the location selected in the hierarchy are not so much an issue in use but perhaps are an issue in learning. There are two hypothetical cognitive mechanisms that may impede performance when selecting items near one that had been recently selected. The first, inhibition of return, assists in the efficiency of search and causes one to have difficulty in returning to previously visited locations that were already determined to be non-target (Klein, 2000). This effect is greatest for recently attended items, which would likely be the adjacent locations in the interface. The second mechanism, negative priming, is one of attentional selection where it is believed that items that directly interfere with target selection may be suppressed (Fox, 1995). In the present study it was found that, during the learning phase, performance was hindered when one was required to select a location immediately adjacent to the previously selected one. Since the effect was temporary one can guess that the selection mechanism was not operating because it would be used all of the time to select the target. However, the mechanism for increasing search efficiency may be necessary during learning phase because, after the interface is learned, one need not ever attend those adjacent areas and decide that they were non-target. This hypothesis is confirmed in the present study that complexity impacted performance primarily in the first block. It also supports the suggestion of Christie and Klein (2001) that the suppression of items to assist selection may not occur, or at least not be observable, in the localization tasks. 6.1. Limitations of the study Although our sample sizes (12 per group) are moderate and samples are convenience samples, they were considered enough for our experiments. It should be noted that our sample consisted entirely of college students. Almost all students, however, use computers and most are now familiar with handheld devices. Moreover, a large number of consumers who might use these devices were once undergraduates. The task used in the study reflects the most common uses of handheld devices. Tasks performed on handhelds typically require the user to make a short sequence of choices to find a name in an address book, to select a day and time in a calendar, or select a favourite web site. There may, however, be limits to our findings rooted in the possibility that data collected using the task we designed may not be generalizable to other tasks used by mobile users. This is the second study by the authors with similar findings. Since we are not testing theories related to hypothetical constructs for which converging evidence is generally considered necessary (Garner et al., 1956), we are confident that further
ARTICLE IN PRESS J. Christie et al. / Int. J. Human-Computer Studies 60 (2004) 564–584
580
triangulation of results is not warranted. Rather, we are generating parametric data about the relative efficacy of two different interface layouts when size and complexity are varied.
7. Conclusion In conclusion, we have confirmed that screen display size is of considerably more importance than display complexity. Second, we have shown that users prefer the simple hierarchy layout even if there is a performance disadvantage incurred. Finally, a simple hierarchy layout will be most efficient if the most common path is determined and the choice options for that path are located so that the mouse movements are the shortest. While user performance in accessing information was superior using the grid layout rather than the simple hierarchy layout, users preferred the simple hierarchy layout. These findings indicate that the grid layout should be preferred when performance effectiveness is the paramount concern and the complexity of choices is not expected to be large. When user preference is more important than performance effectiveness the appeal of the simple hierarchical layout may supersede the cost in performance it entails.
Acknowledgements This work was supported by Nortel Networks and the NSERC research council of Canada. Appreciation is also given to Jeff Fairless and the Design Interpretive Group for encouragement and support. William Matheson is also thanked for his assistance in recruiting participants.
Appendix A ANOVA results for across three groups (large 25 item, large 16 item, and small 16 item)
Group Order Group Order Subject(Group) Layout Layout layout
group Order
DF
Mean square
F-value
P-value
2 1 2 30
14807997.62 1179158.422 715142.31 4832065.085
3.065 0.244 0.148
0.0615 0.6249 0.8631
1 2 1
151946503.3 1201734.76 20282532.75
353.133 2.793 47.138
o0.0001 0.0772 o0.0001
ARTICLE IN PRESS J. Christie et al. / Int. J. Human-Computer Studies 60 (2004) 564–584
group Order Subject(Group)
581
2 30
713927.421 430280.992
1.659
0.2073
11 22 11 22 330
22573696.7 168950.819 95840.609 341548.827 219402.249
102.887 0.77 0.437 1.557
o0.0001 0.7625 0.9389 0.0544
layout Block 11 layout Block group 22 layout Block Order 11 layout Block group 22 order layout Block Subject 330 (Group)
340060.459 352471.57 7240520.097 72266.301
2.702 2.8 57.526 0.574
0.0024 o0.0001 o0.0001 0.9394
layout layout Block Block Block Block Block
group Order group Order Subject(Group)
125865.681
Appendix B ANOVA results for complexity effect DF
Mean square F-value P-value
complexity Order complexity Order Subject(Group)
1 1 1 20
1943294.084 266112.399 1017884.255 1807045.81
1.075 0.147 0.563
0.3121 0.7052 0.4617
layout layout complexity layout Order layout complexity Order layout Subject(Group)
1 1 1 1 20
84092644.46 3239.033 12492330.91 1366734.407 318235.006
264.247 0.01 39.255 4.295
o0.0001 0.9206 o0.0001 0.0514
11 11 11 11 220
14248853.79 85904.252 162813.822 351940.673 171811.864
82.933 0.5 0.948 2.048
o0.0001 0.9021 0.4956 0.0253
11 11
251348.158 473564.376
2.159 4.067
0.0177 o0.0001
Block Block Block Block Block
complexity Order complexity Order Subject(Group)
layout Block layout Block complexity
ARTICLE IN PRESS J. Christie et al. / Int. J. Human-Computer Studies 60 (2004) 564–584
582
layout Block Order 11 layout Block layout Block complexity 11 Order layout Block Subject 220 (Group)
4519662.079
38.814
o0.0001
55093.824
0.473
0.9185
116443.652
Appendix C ANOVA results for size effect DF
Mean square
F-value
P-value
size Order size Order Subject(Group)20
1 1 1 20
27591108.98 322121.279 1124870.15 6642145.338
4.154 0.048 0.169
0.055 0.8279 0.6851
GUI GUI GUI GUI GUI
1 1 1 1 20
109902789.4 1877342.422 10010994.88 637826.848 561242.994
195.82 3.345 17.837 1.136
o0.0001 0.0824 0.0004 0.2991
11 11 11 11 220
15641209.64 244894.04 298760.714 86576.355 277558.605
56.353 0.882 1.076 0.312
o0.0001 0.5585 0.3815 0.9829
GUI Block 11 GUI Block size 11 GUI Block Order 11 GUI Block size Order 11 GUI Block Subject 220 (Group)
524404.637 139023.18 4870593.094 105119.59 134562.702
3.897 1.033 36.196 0.781
o0.0001 0.418 o0.0001 0.6587
size Order size Order Subject(Group)20
Block Block Block Block Block
size Order size Order Subject(Group)
Appendix D ANOVA results for the block position interactions in the analysis of repeated adjacent vs. non-adjacent cell positions
ARTICLE IN PRESS J. Christie et al. / Int. J. Human-Computer Studies 60 (2004) 564–584
DF Subject Block Block Subject Position Position Subject Block position
35 11 385 2 70 22
583
Mean square F-value P-value 11424043.96 35121059.21 871817.632 16658155.6 734642.65 1124988.964
40.285
o0.0001
22.675
o0.0001
2.468
0.0002
References Christie, J., Klein, R.M., 1997. Simple path versus gateway GUI: a preliminary human performance exploration. Nortel Technologies report, March 3. Christie, J., Klein, R.M., 2001. Negative priming for spatial location. Canadian Journal of Experimental Psychology 55 (1), 24–38. Dillon, A., Richardson, J., McKnight, C., 1990. The effect of display size and text splitting on reading lengthy text from the screen. Behavior and Information Technology 9 (3), 215–227. Duchnicky, R.L., Kolers, P.A., 1983. Readability of text scrolled on visual display terminals as a function of window size. Human Factors 25, 683–692. Fitts, P.M., 1954. The information capacity if the human motor system in controlling the amplitude of movement. Journal of Experimental Psychology 47, 381–391. Fox, E., 1995. Negative priming from ignored distractors in visual selection: a review. Psychonomic Bulletin and Review 2 (2), 145–173. Frey, P.R., Rouse, W.B., Garris, R.D., 1992. Big graphics and little screens: designing graphical displays for maintenance tasks. IEEE Transactions On Systems, Man, and Cybernetics 22 (1), 10–20. Fulp, C.D., Fulp, E.W., 2002. A wireless handheld system for interactive multimedia-enhanced instruction. 32nd ASEE/IEEE Frontiers in Education Conference, November 6–9, Boston, T3F-6-9. Garner, W.R., Hake, H.W., Eriksen, C.W., 1956. Operationism and the concept of perception. The Psychological Review 63 (3), 149–159. Han, S.H., Kwahk, J., 1994. Design of a menu for small displays presenting a single item at a time. Proceedings of the Human factors and Ergonomics Society 38th Annual Meeting, Santa Monica, CA, pp. 360–364. Jameson, A., Schafer, R., Weiss, T., Berthold, A., Weyraith, T., 1999 Making systems sensitive to the users time and working memory constraints. IUI99: International Conference on Intelligent User Interfaces, Los Angeles, CA, pp. 79–86. Jones, M., Marsden, G., Mohd-Nasir, N., Boone, K., Buchanan, G., 1999. Improving web interaction on small displays. Proceedings of the WWW8 Conference, Toronto, Canada, May 11–14, pp. 51–59. Klein, R.M., 2000. Inhibition of return. Trends Cognitive Sci 4 (4), 138–147. Lundell, J., Anderson, S., 1995. Designing a ‘‘Front Panel’’ for Unix: the evolution of a metaphor. ACM Proceedings of Human Factors in Computing Systems, May, Denver, pp. 573–80. Mitchell, C.M., Miller, R.A., 1986. Discrete control model of operator function: a methodology for information display design. IEEE Transactions On Systems, Man, and Cybernetics 16 (3), 343–357. Resiel, J.F., Shneiderman, B., 1987. Is bigger better? The effects of display size on program reading. In: Salvendy, G. (Ed.), Social Ergonomic and Stress Aspects of Work with Computers. Elsevier, Amsterdam, pp. 113–122. Rodger, J.A., Pendharkar, P.C., 2000. Using telemedicine in the department of defense. Communication of the ACM 43 (3), 19–20. Sekey, A., Tietz, J., 1982. Text display by saccadic scrolling. Visible Language 17, 62–77.
ARTICLE IN PRESS 584
J. Christie et al. / Int. J. Human-Computer Studies 60 (2004) 564–584
Staggers, N., 1993. Impact of screen density on clinical nurses’ computer task performance and subjective screen satisfaction. International Journal of Man–Machine Studies 39 (5), 775–792. Swierenga, S.J., 1990. Menuing and scrolling as alternative information access techniques for computer systems; interfacing with the user. Proceedings of the Human Factors Society 34th Annual Meeting, Santa Monica, CA, pp. 356–359. Wallace, D., Anderson, N., Shneiderman, B., 1987. Time stress effects on two menu selection systems. Proceedings of the Human Factors Society, Thirty-first Annual Meeting. Santa Monica, CA, pp. 727–731. Watters, C., Shepherd, M., Manchester, L., Chaisson, T., 1997. An evaluation of two metaphors for the presentation of electronic news. Accepted for Electronic Publishing. Online at [www.cs.dal.ca/wifl/ evalNews.doc]. Watters, C., Duffy, J., Duffy, K., 2003. Using large tables on small screen display devices. International Journal of Human Computer Studies 58, 21–37. Wolfe, J.M., 1998. What can 1 million trials tell us about visual search? Psychological Science 9 (1), 33–39.
Int. J. Human-Computer Studies 60 (2004) 564–584
A comparison of simple hierarchy and grid metaphors for option layouts on small-size screens John Christiea, Raymond M. Kleina, Carolyn Wattersb,* b
a Department of Psychology, Dalhousie University, Halifax, NS, Canada Faculty of Computer Science, Dalhousie University, 6050 University Avenue, Halifax, NS, Canada B3H 3W5
Received 26 March 2003; received in revised form 8 October 2003; accepted 20 October 2003
Abstract Modern, technologically driven society is characterized by an increase in the rate of mobile device use and an increase in the extent to which these devices are used for more complex tasks than search for phone numbers. While direct consequences of screen-size reduction on task performance are well known, data are lacking on the impact of layout of multiple options in a complex task environment. In this paper we describe the results of an end-user study in which we compared two basic interface designs for the layout of multiple options: a simple hierarchy and a single layer grid. These two designs were presented to users on two screen sizes; a larger size approximating the size of a standard laptop or desktop screen and a smaller size approximating the size of a PDA screen. This study illustrates that while task performance in accessing information was superior using the grid interface rather than the simple hierarchy interface, users preferred the simple hierarchy interface. Even as the complexity of the task increased, the physical size of the screen had a significantly bigger impact on task performance than did the task complexity. These findings indicate that the grid layout should be used when task performance is of the paramount concern and the complexity of choices is not expected to be large. When user preference/satisfaction is more important than task performance the appeal of the simple hierarchy layout may supersede the cost in performance it entails. r 2003 Elsevier Ltd. All rights reserved.
*Corresponding author. Tel.: +902-494-6721; fax: +902-492-1517. E-mail address: [email protected] (C. Watters). 1071-5819/$ - see front matter r 2003 Elsevier Ltd. All rights reserved. doi:10.1016/j.ijhcs.2003.10.003
ARTICLE IN PRESS J. Christie et al. / Int. J. Human-Computer Studies 60 (2004) 564–584
565
1. Introduction The use of mobile devices, such as Personal Data Assistants (PDAs) and smart phones, has increased dramatically in recent years. Not only has usage increased, but so has the complexity of the tasks for which these devices are being used. The increased capability of these appliances supports applications for tasks that may include a series of activities such as lookups, selections, comparisons, and data input. Furthermore, many users of mobile appliances also use desktop and laptop applications and may switch or migrate between these devices frequently. It has become important, therefore, to understand the impact that screen reduction has on both task performance and user preference so that interfaces can be designed for use on small appliances, such as those that may be deployed in education (Fulp and Fulp, 2002) or telehealth (Rodger and Pendharkar, 2000). Interface designers for such applications should aim to optimize performance by minimizing confusion, learning time, and error rates. In this paper we concentrate on two interfaces for the presentation and navigation of options across screen sizes, the simple hierarchy and the single layer grid. We use the terms interface and layout interchangeably. The simple hierarchical layout presents the options at the end of ‘‘paths’’ through which the user proceeds by a sequence of choices, while the grid layout presents all of the options to the user at once. The simple hierarchy layout initially presents the user with fewer but higher level abstractions from which a choice is required to get to a subset of the individual options and it may be necessary for the user to backup and repeat choices before the correct option is found. The grid layout presents a larger array of choices from which the user needs to find the appropriate option but the complete set of individual options are available at once. A trade-off then exists in layout choice between the time needed to perform a search through many options in the grid layout and the time needed to perform multiple selections among fewer options at each step of the simple hierarchy layout. We are interested in finding whether the simple hierarchical layout or the single layer grid layout provides better task performance in a search task using two different screen sizes. We are also interested in finding out which interface most users prefer. Our experimental design to test for performance effectiveness and user preferences across the two screen sizes is outlined in Table 1.
Table 1 Outline of experimental design PDA-size screen Performance (grid vs. hierarchy) Preference (grid vs. hierarchy)
Laptop-size screen
ARTICLE IN PRESS 566
J. Christie et al. / Int. J. Human-Computer Studies 60 (2004) 564–584
2. Background Over the past 20 years there have been many studies on the performance of users reading from screens and the effects of small screen size on performance. Two issues from this research are directly relevant to the current study: screen size and interface design or layout. Screen size. On small screens the frequency of scrolling can be expected to be higher simply because less data are presented on the screen. Research has shown (Sekey and Tietz, 1982) that left–right or horizontal scrolling degrades reading performance and that vertical scrolling, while acceptable for skimming or scanning tasks, degrades performance on actual reading or comparison types of tasks. Studies by Jones et al. (1999) indicate that scrolling on small screens leads to decreases in both speed and effectiveness in performing tasks. A related finding is that the width of lines of text and screen size do not affect the users ability to comprehend text, although a reduction in the size of the screen leads to an increase in scrolling activities by the reader (Dillon et al., 1990). The readability of options has also been shown to affect task performance and must be considered in any screen-size reductions. Clearly, it is harder for users to make correct choices when the font or icon is too small to be easily recognizable. Even if the text or image is readable, performance may be undermined if the user has difficulty activating or indicating a choice because the size of the option is too small to target. Fitts’ (1954) law indicates that as the size of the buttons become smaller they will be more difficult to target accurately. There will clearly be a trade-off between presenting all of the options to the user, when the options must be reduced to fit into the smaller space, and the users’ ability to identify and select the correct option. Overall, research indicates that reducing screen size negatively affects user performance for a variety of tasks but that some features can be incorporated that mitigate the effect. One such feature is the use of a search function, particularly with lists and text (Reseil and Shneiderman, 1987). Another feature, for tabular data, is the inclusion of contextual information so that the user can manipulate the cell information within its larger context (Watters et al., 2003). Option layout patterns. One of the appeals of handheld devices is that they are very portable. Unfortunately, this increased portability means that the environment in which they are used is often distracting. The competing demands on perception and working memory, simplicity and directness of the navigational model, and fitness of a particular task become important performance effectiveness factors (Jameson et al., 1999). The choice of layout for multiple options may also be an important factor in improving performance effectiveness for certain tasks on small screens. 2.1. Full size screen factors The use of menus to organize options has become nearly ubiquitous in interface design. Menus have obvious appeal on small screens because they can remain hidden much of the time. Menus tend to be organized either in a linear fashion or in a
ARTICLE IN PRESS J. Christie et al. / Int. J. Human-Computer Studies 60 (2004) 564–584
567
hierarchic fashion. Menus allow a user to scan and find the feature of interest or make an initial choice followed by one or more narrowly focused choices. A study by Resiel and Shneiderman (1987) tested user performance on lists of different lengths and concluded that users performed the task equally well whether they saw the entire list or only a few sections at a time. A study by Duchnicky and Kolers (1983), however, showed that the amount of scrolling by users increased as lists got longer than the screen and further that effectiveness decreased as the length of the visible portion of the list was reduced. Swierenga (1990) found no difference in performance in using hierarchical menus as the list got longer but Han and Kwahk (1994) did find a difference in performance in using hierarchical menus as the list got longer. Studies by Wallace et al. (1987) indicate that users perform better on shallow hierarchies than on deep hierarchies. Frey et al. (1992) indicate that a hierarchical menu gives better performance than scrolling through a list of choices. The dashboard metaphor (Lundell and Anderson, 1995) collocates options and views on a single screen for ready access by the user to multiple functional options. Desktops and common tab bars on web site pages are modern versions of the dashboard. This layout design provides consistency for the user and a one-stop-shop that reduces the cognitive effort of the user to remember where a user is in relation to the decision points. The number of choices is of course limited by screen size and recognition factors. A study by Mitchell and Miller (1986) indicated that, when choices are grouped according to their relationship to the given task, performance improved, and Staggers (1993) found that a single display showing the whole system also led to improved performance.
2.2. Small screen factors Jones et al. (1999) found that users preferred focused navigation and direct access strategies when using small screens. Simple hierarchical layouts are a commonly used method of representing choices on small screens. News sites, for example, replace the original data with a short list of categories of news types that fits on the screen. The user selects one category, such as sports, and a list of sports article titles is presented on the screen. From this list of titles, the user can select a title and the full text or a summary of the article will be presented. While studies have shown (Watters et al., 1997) that scanning the representative chunks of news data is more effective than using a hierarchy of headings on the larger screens where the task is related to browsing or scanning, Jones et al. (1999) found that scrolling on small screens reduces the task effectiveness for the user. The small size of screens means that even small lists may require hierarchies of several levels to accommodate a few sets of choices. We expect that the effect of navigation options and layout will be exaggerated on the small-size screen and possibly further hindered by the limited input capability of handheld devices.
ARTICLE IN PRESS 568
J. Christie et al. / Int. J. Human-Computer Studies 60 (2004) 564–584
3. Study In this paper we examine two layouts for presenting a relatively small and finite number of options (categories), 16 or 25, to users with lookup tasks. One layout is a shallow (two-level) hierarchy and the other is the grid layout, a variation of the dashboard metaphor. These are shown in Fig. 1 with the grid layout on the left and
Fig. 1. Screenshots of the experiment. All of the grid layouts are on the left and the top level of the simple hierarchy layouts are on the right. The first row is handheld and the last two are desktop. The first two rows are for 16 item complexity and the last two are for 25 item complexity.
ARTICLE IN PRESS J. Christie et al. / Int. J. Human-Computer Studies 60 (2004) 564–584
569
the first level of the simple hierarchy on the right. With the simple hierarchy layout, when one item is selected the array of items would be replaced with the second level or actual options. The grid layout presents all options to the user and is constrained as the number of options increases. We did not test the linear dropdown menu model in this study, as there is no literature supporting its use on small-size screens for lists longer than the screen size. 3.1. Goals of the study The primary goals of this study were to assess the relative performance with, and user preferences for, simple hierarchy and grid layouts for use in search tasks on smaller screens. The simple hierarchy is a layout in which the choices are organized into a shallow hierarchy. The simple hierarchy layouts are often found in GUI based applications. In the simple hierarchy layout, a few choices are presented on each level of the interface; choices at the first level of the interface lead to deeper levels until the appropriate function is found. Any one selection is not necessarily an end in itself, but a step along a path to the final goal. In the grid layout all of the options are presented to the user at the same time. In this case, each choice is an end in itself. It is interesting to note that the grid layout is commonly found in devices of relatively low complexity or with actual buttons, such as a telephone or simple calculator while more complex devices driven by software, such as the VCR remote or PDA, generally use a hierarchy of options. In a previous study by Christie and Klein (1997), comparing human performance measured in reaction time and accuracy, it was shown that for simple interfaces with nine possible options the grid layout was superior than the simple hierarchy layout. In that study, a simple hierarchy of three buttons on the first level led to three target items accessible on each of the three possible second levels. In the grid layout all nine buttons were simultaneously presented to the participants. Participants were faster to access target items using the grid layout than the simple hierarchy layout. In the current study, we question whether the findings of Christie and Klein (1997) study will hold true for more complex interfaces, which include more target items, and more complex tasks. A complex task requires a two-step answer over simple recognition. Since the increased response time using the simple hierarchy layout can be attributed to the extra cognitive and motor step of selecting a choice to move to the second level of the interface, we predicted that the performance difference between the grid layout and the simple hierarchy layout might be reversed when the interface became sufficiently complex. Consider a simple complexity metric based on time to support this argument. Additional time is required to access a subsequent level and this time would be constant time factor, Y. Further we assume that there is a fixed amount of time, X, that is needed per button to search through each item in a display to find any one button (Wolfe, 1998). As an example, consider the earlier study, which had nine buttons. If a user checks each of the possible buttons before making a choice, the time to find a button in the grid is the number of buttons plus a constant (k), in this
ARTICLE IN PRESS 570
J. Christie et al. / Int. J. Human-Computer Studies 60 (2004) 564–584
case 9X+k. In the hierarchy, the time to find a button depends on the number of items on the first level plus the number of items on the second level, 3X + k per level in this case, and the constant Y to change level, for a total of 6X+Y+2k units of time. Of course, it is likely that searches would actually be self-terminating rather than exhaustive, i.e., the user will stop examining buttons when the target is found. On average the user finds the button in half the time, in both the simple hierarchy and grid layouts. Therefore, it takes 5X+k for the grid layout and 3X+Y+2k for the simple hierarchy layout. As device complexity increases the multiplier for X increases faster for the grid layout than for the simple hierarchy layout. For example, at 25 items search time for the grid will be 13X+k, but search time for the hierarchy will only be 6X+Y+2k. Let us call the difference in the number of options inspected (NX) between the grid layout and the simple hierarchy layout, DT. In the present experiment where 16 items are used DT= 8X 4X=4X. Using low complexity interfaces, performance using the simple hierarchy layout will be worse than using the grid layout because of the additional Y+k constant. But, because the number of items to be searched in the grid layout rises much more steeply than for the simple hierarchy layout, at some point the performance advantage should shift to the simple hierarchy layout as the time represented by DT exceeds Y+k. An additional factor in performance is that option selection likely follows Fitt’s law. As the number of options increases passed certain threshold, the size of some or all of the items on the display must decrease. This will occur faster for the grid layout than for the simple hierarchy layout. In the current experiment for 16-item complexity, the maximum size of buttons in the grid will be 1/16th the size of the display while those in the simple hierarchy layout will be 1/4th the size of the display. But, in the 25-item complexity condition, the grid buttons will reduce to 1/25th of the size of the display while the buttons in the simple hierarchy layout will be 1/5th of the size of the display. Therefore, as complexity increases it may be hard for users to target buttons the grid layout. When the user is in motion or in a distracting environment small-size buttons may not be desirable. 3.2. Experiment In the current study we follow up the effect of layout of choices (Christie and Klein, 1997) in the context of the small screens typical of handheld devices. We also increased the complexity of the task and layout by increasing the number of options from which the user had to make a selection. We wanted to see both how performance is affected by the layout of the options and different layouts influenced participant’s subjective impressions. If a difference in performance can be demonstrated then understanding the relationship of given tasks to the organization of choices should permit optimization of screen layout for given tasks. However, the usefulness of performance measures may be mediated by user preferences. In order to test the impact of task complexity, two complexity scenarios were used. The first scenario consisted of 16-items, and the second scenario consisted of 25items. The number of levels for each scenario for the simple hierarchy layout was two, and the number of levels for each scenario for the grid layout was one. The
ARTICLE IN PRESS J. Christie et al. / Int. J. Human-Computer Studies 60 (2004) 564–584
571
complexity levels have an average DT values of 4X and 7X for the simple hierarchy and grid layouts. It is anticipated that higher DT will approach or exceed Y+k. In this within subject experiment participants were required to find a target word based on a two-word clue using both layouts. For example, given a clue makes holes the target would be Drill. In the grid layout shown on the left in Fig. 1, Drill can be accessed directly by the user, while in the simple hierarchy layout shown on the right in Fig. 1, the user would first select Tool and subsequently Drill. Table 2 provides a list of the clues and solutions for the 16 and 25 item conditions. This task mimics the user learning of interfaces that are used to translate concepts into actions. For example, when a user wants to see what is in a file in most computer operating systems, the concept must be mapped to the action File:Open. The table includes the top-level headers used in the Simple Hierarchy version and the corresponding second level target item groups. In the grid condition all of the target items would be presented.
Table 2 A list of clues and solutions for 25-item complexity Header
Target
Clue
Baseball Football Golf Hockey Running
home run touch down hole in one fast skaters pound pavement
Tool
Chisel Drill Hammer Sander Saw
wood shaper makes holes drives nails surface cuts stuff
Vegetable
Carrot Corn Onion pea Potato
orange root yellow kernel tear maker in a pod french fry
Vehicle
Bicycle Boat Carriage Plane Train
two wheeler water craft horse drawn high flyer rail racer
Weather
Rain Snow Sunshine Tornado Wind
falling drops drifting flakes warm brightness fast funnel blowing air
(Simple Path only) Sport
ARTICLE IN PRESS 572
J. Christie et al. / Int. J. Human-Computer Studies 60 (2004) 564–584
In order to test the effect of screen size the 16-item scenario was used both in a small screen size, similar to that of a handheld device, and a large screen size, similar to that of a desktop computer. The ratio of small screen size and large screen size is similar to the ratio of the size of upper panes and the size of lower two panes illustrated in Fig. 1. The use of different screen sizes allowed for the assessment of the possibility that, in addition to the performance differences due to the complexity of an interface, perhaps screen size may play a part in determining the user task performance. The screen-size variable was not used for the 25-item complexity scenario because for the small screen size, the grid layout with 25-item complexity was already unreadable. Reaction time and accuracy measures were recorded automatically. In addition, a questionnaire was presented following the performance experiment in which participants subjectively compared the graphical user interface metaphors by rating their perceived performance and preferred layout.
4. Methodology Thirty-six undergraduate psychology students participated in the experiment. Participants were randomly assigned to one of three groups, which experienced the following combinations of display size and display complexity: large (desktop size) with 25 options; large (desktop size) with 16 options; small (handheld size) with 16 options. Each individual participated in both simple hierarchy layout, called the Simple Path, and grid layout, called the Gateway, conditions, with the order counterbalanced across participants. Each trial consisted of a search for the solution to one clue, shown at the bottom of each screen. A block of trials for each participant consisted of the complete set of clues for one interface complexity, i.e., 16 or 25 trials. There were 12 blocks per layout, per participant making 24 blocks in total with block 13 representing a shift from one layout to the other. Half of the participants used simple hierarchy layout followed by grid layout while the rest used the grid layout first and then the simple hierarchy layout. In the experiment, the two screen sizes were presented in grey in the centre of the monitor while the rest of the monitor area was black. The small size, meant to represent a handheld device, measured 41 mm wide and 83 mm high; and the large size, meant to represent a desktop screen, measured 122 mm wide and 86 mm high. Participants were requested to sit at a comfortable distance from the machine while viewing the text displayed in Chicago font. The text ranged from 9-point font size for the grid layout on the handheld size to 32 point for the simple hierarchy layout at the desktop device size and 16-item complexity. Example pictures of each possible display size and complexity are in Fig. 1. Table 2 contains a listing of all of the clue strings and the corresponding target word. Items in Table 2 shown in italics were not used in the lower complexity version, i.e., 16 items rather than 25 items. Participants were instructed that they would be playing a multiple choice word clue game. During the experiment the participant first viewed a black display with a grey rectangle in the centre that was either the small or large size display size. A small
ARTICLE IN PRESS J. Christie et al. / Int. J. Human-Computer Studies 60 (2004) 564–584
573
square in the centre of that rectangle was clicked to begin the experiment. In the grid condition the display was replaced by a grid layout of 16 or 25 target buttons ordered alphabetically. The clue appeared at the bottom of the display area. The participant would click on target buttons to try to select the correct one (up to five errors were permitted). After selecting the correct response, or five tries, the sequence would cycle on to the next clue. Each participant would repeat this for all 12 clues in random order. In the simple hierarchy layout, the options were not presented all at the same time. Rather there would be a presentation of words naming the categories of potential targets as well as the clue presented beneath the display. The subject would select a category that the answer to the clue should be in and the display would be replaced with a set of 4 or 5 targets within that category. The user could then select one of those target items or move back up to the first level by pressing a button above the items. The subject navigated the display until the correct target was selected or until five incorrect target items were selected. All subjects completed one session using the simple hierarchy layout and one session using the grid layout and then completed a simple questionnaire, shown in Fig. 2. Due to the researcher oversight two participants in each of two of the groups were not given the questionnaire. 5. Results 5.1. Performance All participants made very few errors and there were no significant accuracy effects that differentiated complexity, size, or layout condition. A block of trials
Fig. 2. Questionnaire for each participant after completing the experiment.
ARTICLE IN PRESS J. Christie et al. / Int. J. Human-Computer Studies 60 (2004) 564–584
574
consisted of the complete set of clues for one interface complexity. There were 12 such blocks for each layout, for a total of 24 blocks. For block one the errors were 9% for the grid layout, and 7% for the simple hierarchy layout. By block 2 these had dropped to less than 5%, and in all blocks after that the means were 3% or less. A failure resulted when a total of five errors occurred during one trial. In block one there was an average of one failure per person, by block 2 there was an average of 0.5 failures, and by block 7 there was an average of 0. Reaction time, shown in Fig. 3, improved consistently in all conditions until around block 7 by which time asymptotic levels of performance had been achieved. In Block 13 the layout was switched from one layout to the other; this dropped performance significantly. Performance reached asymptote after the layout switch in about six blocks. Performance was better for the grid layout than for the simple hierarchy layout in both display sizes and both complexities. Four analyses of variance (ANOVAs) were carried out on the reaction time data. In the first ANOVA all three groups were included (Appendix A). The second and third ANOVAs (Appendices B and C) were focused on interface complexity (16 or 25 options, with the large display size), and the interface sizes (large or small, with 16 options), respectively. In all of the analyses, block (1–12), layout condition (hierarchy vs. grid), and order of layout presentation were within-subject factors and group was a between-subject factor. A final analysis was carried out on the 16 item HandHeld
25 item Desktop
16 item Desktop
6500
Simple path Gatewa
6000 5500 5000 4500 4000 3500 3000 2500 2000 1500 2
6 10 14 18 22
2
6 10 14 18 22 Blocks
2
6 10 14 18 22
Fig. 3. The mean reaction time performance for all subjects in both simple hierarchy and grid layouts. Each graph represents a separate group of participants.
ARTICLE IN PRESS J. Christie et al. / Int. J. Human-Computer Studies 60 (2004) 564–584
575
simple hierarchy layout to examine the effect, if any, of the relative position of buttons in subsequent layers in the interface (Appendix D). One might note that font size perfectly covaried with condition and screen size. Although one might attribute some performance changes to changes in font size, this was not varied independently. It is clear from the data that diminishing font size had little negative effect because the smallest font size for each screen size was in the grid layout, where performance was best. 5.2. Analysis across three groups There was a marginal effect of group, F(2,30)=3.065, po0.07, and only one interaction involving group: group, layout, and block, F(22,330)=2.80, po0.001. Both of these effects will be discussed in the next section where the present group variable is broken out into display size and complexity. Performance improved with practice (main effect of block: F(11, 330) =102.89, po0.001) and was better with the grid layout than with the simple hierarchy layout (F(1, 30)=353.13, po0.001). These main effects are qualified by several interactions. Layout interacted with the order in which the layouts were presented, F(1, 30)=47.14, po0.001, and also separately with block, F(11, 330)=2.70, p=0.02. In addition, there were two higher order interactions with layout, the aforementioned interaction with block and group, and the other with block and order, F(11, 330)=57.53, po0.001. There were no higher order interactions. The nature of the layout block order interaction is fairly straight-forward. When looking at Fig. 3 it can be seen that the order of presentation modifies the effects of the condition on the performance at the block level. If the grid layout is used first, performance improves faster over the initial few blocks than if the simple hierarchy layout is used first. It may be tempting to conclude that the grid layout is learned faster. However, the interaction could be caused by the generally better performance using the grid layout being achieved at the same time as best performance in the simple hierarchy layout, resulting in steeper reaction time slopes over the first few blocks, but not faster learning. 5.3. Analysis of display size and layout complexity An analysis of the 16-item scenario using the desktop and handheld sizes (the top and center panels of Fig. 1) revealed that participants were significantly faster at finding targets using the desktop rather than the handheld display sizes, F(1,20)=4.15, p=0.05. Contrary to predictions, an analysis of the desktop size using layouts with 16 and 25 options (the centre and bottom panels of Fig. 1) revealed no effect of complexity, F(1,20)=1.075, p=0.31. Thus the marginal effect of group reported above was driven by the screen-size variable and not by the task complexity variable. There were no interactions between the screen size and other variables. There was one higher order interaction among layout, block, and complexity, F(11,220)=4.07, po0.001. This effect is confined to the first two blocks of trials. In
ARTICLE IN PRESS 576
J. Christie et al. / Int. J. Human-Computer Studies 60 (2004) 564–584
block 1 of the 25-item desktop size trial, performance is poorer using the grid layout than simple hierarchy. This is the only time that this happens across comparable blocks in the entire experiment. It is rather surprising to us that this effect goes away within only one block of practice. Given that it does not happen at all if one has practice with the other interface first, it appears it is not simply because the layout of the options is learned. Finally, an analysis of the simple hierarchy layout data was conducted in order to see if an additional effect from Christie and Klein (1997) was replicated. In designing the simple hierarchy layout, target items in the second level can be positioned in the same location as the button that was used to access that level, they can be in an adjacent location, or they can be in a non-adjacent location. In this earlier study with nine items using the simple hierarchy layout, when the option selected in the second level was in the same location as the previously selected first level option there was a 557 ms benefit in performance compared to targets that were not in the same location. This effect was found to also be significant in the present study across all blocks, F(2,70)=22.68, po0.001, and is illustrated in Fig. 3. (ANOVA results are presented in Appendix D.) The reaction times, in milliseconds, are shown for the simple hierarchy layout as a function of whether the item in the subcategory was in the same location as the item in the initial interface, an adjacent location, or whether the items was in a non-adjacent location. Block 1 is plotted separate from the other blocks because the performance pattern was significantly different. A test of adjacent items vs. non-adjacent items was also carried out. As can be seen in Fig. 4 there was a significant advantage for non-adjacent location over adjacent in the first block (5066 ms vs. 3978 ms), F(1,35)=7.23, po0.01; but in subsequent blocks this advantage was not present, Fo1. One might note that the 16-item data in Fig. 4 comes from the same source as the desktop data from Fig. 3. 5.4. Subjective ratings The subjective ratings from the questionnaire are shown in Table 3 where the numbers represent the proportion of respondents who indicated that the simple hierarchy layout was Better than, Easier than, and Preferred to the grid layout. With the desktop screen size and 16 items in the interface, participants found that the simple hierarchy layout was easier to use and preferred it. None of the other values differed from chance (0.5). However, chance may not be the proper comparison. It can be argued that the proportion of participants that performed better in the simple hierarchy layout is the correct ratio. In this instance that is 0.0, and all of these numbers are greater than that by a Sign test, po0.001. Subjective impressions of the interface did not appear to be affected by performance and significantly deviated from what would be predicted if preferences did follow performance. Intercorrelations among the various subjective ratings revealed that, individuals prefer the layout on which they believe they performed better, r=0.57, po0.01. There was a moderate correlation between what was preferred and what was considered easier, r=0.35, p=0.05. But, no correlation, r= 0.11, p=0.54, was found between the layout that they believed gave the best performance and the one
ARTICLE IN PRESS J. Christie et al. / Int. J. Human-Computer Studies 60 (2004) 564–584
577
Simple Path Position
5250 5000
Block1
4750
Block 2-12
4500 4250
RT
4000 3750 3500 3250 3000 2750 2500 2250 2000 Repeat
Adjacent
Non-Adjacent
Fig. 4. Reaction time in milliseconds.
Table 3 The subjective rating data Questions
Handheld with 16 items N=10 Desktop with 16 items N=10 Desktop with 25 items N=12
Better
Easier
Preferred
0.40
0.50
0.50
0.60
0.75
0.80
0.58
0.33
0.58
that was considered easier. The correlation between the magnitude of the performance benefit for the grid layout over the simple hierarchy layout, and the degree of preference for the simple hierarchy layout was negative, and significant, r= 0.52, po0.01. The performance advantage for the grid layout was also negatively correlated with the layout that the individuals believed they performed better with, r= 0.36, po0.05. These results imply that the subjective measures were sensitive to the performance. Participants were less likely to prefer the simple
ARTICLE IN PRESS 578
J. Christie et al. / Int. J. Human-Computer Studies 60 (2004) 564–584
hierarchy layout as the performance advantage for the grid layout increased. However, the fact that performance was always better with the grid layout was not sufficient for it to be preferred, just more likely to be preferred if the advantage was very large.
6. Discussion In this study we found that the screen size of an interface significantly influences performance in finding target items. However, contrary to our predictions, increasing the complexity of the interface, from 16 to 25 items, had no effect on performance after the interface was learned. Furthermore, it was discovered that, even though individuals were consistently much more efficient at finding targets using the grid layout in our study, their subjective judgments of the interface did not show a preference for that interface, and sometimes showed just the opposite. While there was a performance advantage in using the grid layout, participants preferred the simple hierarchy layout. Extending previous conclusions (Christie and Klein, 1997), these findings lead us to suggest that in situations where an interface is static and not likely to significantly increase in complexity, a grid layout is the most efficient for the user as long as the buttons are large enough to read and use. The grid layout has been shown to be the better choice, from the point of view of performance, and useful when the number of potential options can fit on the screen (or device, if hard wired). If maximum performance is required, the grid layout is the preferred interface given that above conditions are satisfied. The pragmatic argument that says ‘‘if the interface is subject to potential higher complexity a simple hierarchy should be used’’ is true. A grid-based interface is limited by how many buttons can be presented in a clear and accessible fashion; a hierarchical interface is not. We did not present the 25-option condition using the small screen size because the option labels were not readable. In spite of the reduced performance in the use of the simple hierarchy layout, subjective perceptions of the interface are positive. It is rated as easy or easier to use than the grid layout. Therefore, it is believed that, although performance in tasks like the one we have explored here will be reduced, a simple hierarchy is an excellent substitute for the grid layout in interfaces, and users may actually prefer it. Thus, when user preference is more important than performance, the simple hierarchy appears to be the layout of choice. One of the more interesting and novel facts that arise from this experiment is that the physical display size affects performance much more than the complexity of the display. This was true for both hierarchy and grid cases. A small display hindered performance while a more complex display had no effect on performance for the tasks studied. The fundamental message here is that efforts to increase portability and compactness should be tempered with an understanding that the size of a display is very critical. For any specific device and task, optimal size parameters should be investigated.
ARTICLE IN PRESS J. Christie et al. / Int. J. Human-Computer Studies 60 (2004) 564–584
579
Recognizing other factors found in the results can also facilitate the use of simple hierarchies in interface design. Performance would be improved if the most frequently used button on the subsequent level in a hierarchy is in the same location as the button used on the preceding level, based on our finding that successive responses in the same location are made much faster than in different locations. It was also found that participants were poorer in selecting the target item adjacent to the location of the item required to get to that level of the hierarchy, but only in the first block. Therefore, cognitive factors that might limit performance in items immediately adjacent to the location selected in the hierarchy are not so much an issue in use but perhaps are an issue in learning. There are two hypothetical cognitive mechanisms that may impede performance when selecting items near one that had been recently selected. The first, inhibition of return, assists in the efficiency of search and causes one to have difficulty in returning to previously visited locations that were already determined to be non-target (Klein, 2000). This effect is greatest for recently attended items, which would likely be the adjacent locations in the interface. The second mechanism, negative priming, is one of attentional selection where it is believed that items that directly interfere with target selection may be suppressed (Fox, 1995). In the present study it was found that, during the learning phase, performance was hindered when one was required to select a location immediately adjacent to the previously selected one. Since the effect was temporary one can guess that the selection mechanism was not operating because it would be used all of the time to select the target. However, the mechanism for increasing search efficiency may be necessary during learning phase because, after the interface is learned, one need not ever attend those adjacent areas and decide that they were non-target. This hypothesis is confirmed in the present study that complexity impacted performance primarily in the first block. It also supports the suggestion of Christie and Klein (2001) that the suppression of items to assist selection may not occur, or at least not be observable, in the localization tasks. 6.1. Limitations of the study Although our sample sizes (12 per group) are moderate and samples are convenience samples, they were considered enough for our experiments. It should be noted that our sample consisted entirely of college students. Almost all students, however, use computers and most are now familiar with handheld devices. Moreover, a large number of consumers who might use these devices were once undergraduates. The task used in the study reflects the most common uses of handheld devices. Tasks performed on handhelds typically require the user to make a short sequence of choices to find a name in an address book, to select a day and time in a calendar, or select a favourite web site. There may, however, be limits to our findings rooted in the possibility that data collected using the task we designed may not be generalizable to other tasks used by mobile users. This is the second study by the authors with similar findings. Since we are not testing theories related to hypothetical constructs for which converging evidence is generally considered necessary (Garner et al., 1956), we are confident that further
ARTICLE IN PRESS J. Christie et al. / Int. J. Human-Computer Studies 60 (2004) 564–584
580
triangulation of results is not warranted. Rather, we are generating parametric data about the relative efficacy of two different interface layouts when size and complexity are varied.
7. Conclusion In conclusion, we have confirmed that screen display size is of considerably more importance than display complexity. Second, we have shown that users prefer the simple hierarchy layout even if there is a performance disadvantage incurred. Finally, a simple hierarchy layout will be most efficient if the most common path is determined and the choice options for that path are located so that the mouse movements are the shortest. While user performance in accessing information was superior using the grid layout rather than the simple hierarchy layout, users preferred the simple hierarchy layout. These findings indicate that the grid layout should be preferred when performance effectiveness is the paramount concern and the complexity of choices is not expected to be large. When user preference is more important than performance effectiveness the appeal of the simple hierarchical layout may supersede the cost in performance it entails.
Acknowledgements This work was supported by Nortel Networks and the NSERC research council of Canada. Appreciation is also given to Jeff Fairless and the Design Interpretive Group for encouragement and support. William Matheson is also thanked for his assistance in recruiting participants.
Appendix A ANOVA results for across three groups (large 25 item, large 16 item, and small 16 item)
Group Order Group Order Subject(Group) Layout Layout layout
group Order
DF
Mean square
F-value
P-value
2 1 2 30
14807997.62 1179158.422 715142.31 4832065.085
3.065 0.244 0.148
0.0615 0.6249 0.8631
1 2 1
151946503.3 1201734.76 20282532.75
353.133 2.793 47.138
o0.0001 0.0772 o0.0001
ARTICLE IN PRESS J. Christie et al. / Int. J. Human-Computer Studies 60 (2004) 564–584
group Order Subject(Group)
581
2 30
713927.421 430280.992
1.659
0.2073
11 22 11 22 330
22573696.7 168950.819 95840.609 341548.827 219402.249
102.887 0.77 0.437 1.557
o0.0001 0.7625 0.9389 0.0544
layout Block 11 layout Block group 22 layout Block Order 11 layout Block group 22 order layout Block Subject 330 (Group)
340060.459 352471.57 7240520.097 72266.301
2.702 2.8 57.526 0.574
0.0024 o0.0001 o0.0001 0.9394
layout layout Block Block Block Block Block
group Order group Order Subject(Group)
125865.681
Appendix B ANOVA results for complexity effect DF
Mean square F-value P-value
complexity Order complexity Order Subject(Group)
1 1 1 20
1943294.084 266112.399 1017884.255 1807045.81
1.075 0.147 0.563
0.3121 0.7052 0.4617
layout layout complexity layout Order layout complexity Order layout Subject(Group)
1 1 1 1 20
84092644.46 3239.033 12492330.91 1366734.407 318235.006
264.247 0.01 39.255 4.295
o0.0001 0.9206 o0.0001 0.0514
11 11 11 11 220
14248853.79 85904.252 162813.822 351940.673 171811.864
82.933 0.5 0.948 2.048
o0.0001 0.9021 0.4956 0.0253
11 11
251348.158 473564.376
2.159 4.067
0.0177 o0.0001
Block Block Block Block Block
complexity Order complexity Order Subject(Group)
layout Block layout Block complexity
ARTICLE IN PRESS J. Christie et al. / Int. J. Human-Computer Studies 60 (2004) 564–584
582
layout Block Order 11 layout Block layout Block complexity 11 Order layout Block Subject 220 (Group)
4519662.079
38.814
o0.0001
55093.824
0.473
0.9185
116443.652
Appendix C ANOVA results for size effect DF
Mean square
F-value
P-value
size Order size Order Subject(Group)20
1 1 1 20
27591108.98 322121.279 1124870.15 6642145.338
4.154 0.048 0.169
0.055 0.8279 0.6851
GUI GUI GUI GUI GUI
1 1 1 1 20
109902789.4 1877342.422 10010994.88 637826.848 561242.994
195.82 3.345 17.837 1.136
o0.0001 0.0824 0.0004 0.2991
11 11 11 11 220
15641209.64 244894.04 298760.714 86576.355 277558.605
56.353 0.882 1.076 0.312
o0.0001 0.5585 0.3815 0.9829
GUI Block 11 GUI Block size 11 GUI Block Order 11 GUI Block size Order 11 GUI Block Subject 220 (Group)
524404.637 139023.18 4870593.094 105119.59 134562.702
3.897 1.033 36.196 0.781
o0.0001 0.418 o0.0001 0.6587
size Order size Order Subject(Group)20
Block Block Block Block Block
size Order size Order Subject(Group)
Appendix D ANOVA results for the block position interactions in the analysis of repeated adjacent vs. non-adjacent cell positions
ARTICLE IN PRESS J. Christie et al. / Int. J. Human-Computer Studies 60 (2004) 564–584
DF Subject Block Block Subject Position Position Subject Block position
35 11 385 2 70 22
583
Mean square F-value P-value 11424043.96 35121059.21 871817.632 16658155.6 734642.65 1124988.964
40.285
o0.0001
22.675
o0.0001
2.468
0.0002
References Christie, J., Klein, R.M., 1997. Simple path versus gateway GUI: a preliminary human performance exploration. Nortel Technologies report, March 3. Christie, J., Klein, R.M., 2001. Negative priming for spatial location. Canadian Journal of Experimental Psychology 55 (1), 24–38. Dillon, A., Richardson, J., McKnight, C., 1990. The effect of display size and text splitting on reading lengthy text from the screen. Behavior and Information Technology 9 (3), 215–227. Duchnicky, R.L., Kolers, P.A., 1983. Readability of text scrolled on visual display terminals as a function of window size. Human Factors 25, 683–692. Fitts, P.M., 1954. The information capacity if the human motor system in controlling the amplitude of movement. Journal of Experimental Psychology 47, 381–391. Fox, E., 1995. Negative priming from ignored distractors in visual selection: a review. Psychonomic Bulletin and Review 2 (2), 145–173. Frey, P.R., Rouse, W.B., Garris, R.D., 1992. Big graphics and little screens: designing graphical displays for maintenance tasks. IEEE Transactions On Systems, Man, and Cybernetics 22 (1), 10–20. Fulp, C.D., Fulp, E.W., 2002. A wireless handheld system for interactive multimedia-enhanced instruction. 32nd ASEE/IEEE Frontiers in Education Conference, November 6–9, Boston, T3F-6-9. Garner, W.R., Hake, H.W., Eriksen, C.W., 1956. Operationism and the concept of perception. The Psychological Review 63 (3), 149–159. Han, S.H., Kwahk, J., 1994. Design of a menu for small displays presenting a single item at a time. Proceedings of the Human factors and Ergonomics Society 38th Annual Meeting, Santa Monica, CA, pp. 360–364. Jameson, A., Schafer, R., Weiss, T., Berthold, A., Weyraith, T., 1999 Making systems sensitive to the users time and working memory constraints. IUI99: International Conference on Intelligent User Interfaces, Los Angeles, CA, pp. 79–86. Jones, M., Marsden, G., Mohd-Nasir, N., Boone, K., Buchanan, G., 1999. Improving web interaction on small displays. Proceedings of the WWW8 Conference, Toronto, Canada, May 11–14, pp. 51–59. Klein, R.M., 2000. Inhibition of return. Trends Cognitive Sci 4 (4), 138–147. Lundell, J., Anderson, S., 1995. Designing a ‘‘Front Panel’’ for Unix: the evolution of a metaphor. ACM Proceedings of Human Factors in Computing Systems, May, Denver, pp. 573–80. Mitchell, C.M., Miller, R.A., 1986. Discrete control model of operator function: a methodology for information display design. IEEE Transactions On Systems, Man, and Cybernetics 16 (3), 343–357. Resiel, J.F., Shneiderman, B., 1987. Is bigger better? The effects of display size on program reading. In: Salvendy, G. (Ed.), Social Ergonomic and Stress Aspects of Work with Computers. Elsevier, Amsterdam, pp. 113–122. Rodger, J.A., Pendharkar, P.C., 2000. Using telemedicine in the department of defense. Communication of the ACM 43 (3), 19–20. Sekey, A., Tietz, J., 1982. Text display by saccadic scrolling. Visible Language 17, 62–77.
ARTICLE IN PRESS 584
J. Christie et al. / Int. J. Human-Computer Studies 60 (2004) 564–584
Staggers, N., 1993. Impact of screen density on clinical nurses’ computer task performance and subjective screen satisfaction. International Journal of Man–Machine Studies 39 (5), 775–792. Swierenga, S.J., 1990. Menuing and scrolling as alternative information access techniques for computer systems; interfacing with the user. Proceedings of the Human Factors Society 34th Annual Meeting, Santa Monica, CA, pp. 356–359. Wallace, D., Anderson, N., Shneiderman, B., 1987. Time stress effects on two menu selection systems. Proceedings of the Human Factors Society, Thirty-first Annual Meeting. Santa Monica, CA, pp. 727–731. Watters, C., Shepherd, M., Manchester, L., Chaisson, T., 1997. An evaluation of two metaphors for the presentation of electronic news. Accepted for Electronic Publishing. Online at [www.cs.dal.ca/wifl/ evalNews.doc]. Watters, C., Duffy, J., Duffy, K., 2003. Using large tables on small screen display devices. International Journal of Human Computer Studies 58, 21–37. Wolfe, J.M., 1998. What can 1 million trials tell us about visual search? Psychological Science 9 (1), 33–39.