- Email: [email protected]
An experiment with content distribution methods in touchscreen mobile devices
Applied Ergonomics 50 (2015) 79e86
Contents lists available at ScienceDirect
Applied Ergonomics journal homepage: www.elsevier.com/locate/apergo
An experiment with content distribution methods in touchscreen mobile devices Eva Garcia-Lopez*, Antonio Garcia-Cabot, Luis de-Marcos , Spain Computer Science Department, University of Alcala
a r t i c l e i n f o
a b s t r a c t
Article history: Received 12 May 2014 Accepted 27 February 2015 Available online
This paper compares the usability of three different content distribution methods (scrolling, paging and internal links) in touchscreen mobile devices as means to display web documents. Usability is operationalized in terms of effectiveness, efficiency and user satisfaction. These dimensions are then measured in an experiment (N ¼ 23) in which users are required to find words in regular-length web documents. Results suggest that scrolling is statistically better in terms of efficiency and user satisfaction. It is also found to be more effective but results were not significant. Our findings are also compared with existing literature to propose the following guideline: “try to use vertical scrolling in web pages for mobile devices instead of paging or internal links, except when the content is too large, then paging is recommended”. With an ever increasing number of touchscreen web-enabled mobile devices, this new guideline can be relevant for content developers targeting the mobile web as well as institutions trying to improve the usability of their content for mobile platforms. © 2015 Elsevier Ltd and The Ergonomics Society. All rights reserved.
Keywords: Mobile usability Empirical study Scrolling
1. Introduction Several aspects of the usability of interfaces have already been addressed in Personal Computers (PC) including characters, formatting, contrast and color, and dynamic text (Mills and Weldon, 1987) or line length (Dyson and Kipping, 1998). However, it is also necessary to study similar and other usability aspects in mobile devices because of the substantial differences that exist between them and PCs, especially concerning the size of the screen. Such differences may have a negative influence in the efficiency of users (Jones et al., 1999; Parush and Yuviler-Gavish, 2004). The degree of interaction is usually higher when using a small screen (Dillon et al., 1990) because of the limited amount of information that can be displayed. Therefore smaller screens require that the user scrolls more times and more often to go through all the content. Several studies have compared different aspects of usability when performing tasks with screens of different sizes, where more or less information can be showed at a glance. Although Kim and
n, Edificio Polite cnico, Ctra * Corresponding author. Dpto Ciencias Computacio de Henares, Madrid, Spain. Tel.: þ34 918856836; Barcelona km 33.1, 28871 Alcala fax: þ34 918856646. E-mail addresses: [email protected] (E. Garcia-Lopez), [email protected] (A. Garcia-Cabot), [email protected] (L. de-Marcos). http://dx.doi.org/10.1016/j.apergo.2015.02.009 0003-6870/© 2015 Elsevier Ltd and The Ergonomics Society. All rights reserved.
Albers (2001) found that there was no difference between a small screen (a PDA in their study) and a larger one (PC) in terms of the number of errors made to complete a task and time taken; most studies found differences in effectiveness (Jones et al., 1999), efficiency (Lai and Wu, 2014; Parush and Yuviler-Gavish, 2004) and satisfaction (Dillon et al., 1990). Such differences evidence the necessity to study independently the interfaces of mobile devices. For instance, adaptive systems that accommodate web pages to mobile interfaces have been developed to try to improve the usability of mobile navigation (Ahmadi and Kong, 2012; Lee and Bahn, 2005). One of the major issues concerning usability for mobile devices is the scroll. Scrolling happens when the information that has to be displayed does not fit in a single screen, “overflowing” off the screen out of immediate view (Sanchez and Wiley, 2009). This results in a serious usability problem (Nielsen, 1997) when the user is reading since it is necessary to handle the device and the information being displayed simultaneously, and this can interfere reading (W€ astlund et al., 2008). The problem is aggravated when the user is in front of a small screen as she will have to go through more pages to read the same amount of text (Dillon et al., 1990). Nowadays, with an ever increasing number of accesses to the mobile web, it is also becoming increasingly important to study the usability of web interfaces in mobile devices. Research to date has studied mobile devices with stylus, keyboards or even with
80
E. Garcia-Lopez et al. / Applied Ergonomics 50 (2015) 79e86
simulated devices in computers, but today the most common mobile device is a touchscreen smartphone. The objective of this paper is to analyze the usability of methods to present information for users of web content in mobile devices with touchscreen. The most common content distribution method is vertical scrolling. It is used for texts (Andersen, 2005), documents and web pages (Yin and Liu, 2010). This paper sets out to test whether vertical scrolling in mobile devices is more usable in webpages than other methods like paging or internal links in terms of effectiveness, efficiency and user satisfaction. Results of our study can be relevant for mobile web developers who are involved in designing interfaces, and also for companies and organizations interested in increasing the readability of their web sites and applications. We provide a new usability guideline to improve the usability of regular-length web documents displayed in touchscreen mobile devices. The paper is structured as follows. Section 2 presents related work pointing the necessity of studying scrolling in web interfaces for touchscreen mobile devices. Section 3 presents the method and experimental design. Section 4 presents the results of experimentation. Section 5 presents discussion comparing our findings with previous studies. Finally, Section 6 outlines conclusions.
2. Related work Scrolling in mobile devices was initially studied for the native menus of mobile devices and in simple web menus displayed in such devices. Different studies about device menus found similar results. For example, Wang et al. (2004) suggested using hierarchical menus with a maximum of two levels, which inevitably increases vertical scrolling. Similarly, Buchanan et al. (2001) found that vertical scrolling is the best method for most users. Contrastingly, results of studies dealing with web menus were mostly contradictory. For example, Geven et al. (2006) concluded that using deeper hierarchies is better than using wider menus, thus suggesting using more levels instead of having too many options per level; while Parush and Yuviler-Gavish (2004) found that limiting depth rather than width is preferable. In order to try to mitigate the negative effects of scrolling, different hardware and software alternatives have been devised both for PC and mobile devices. Studies on hardware solutions for PC compared diverse gadgets like (1) a wheel mouse, a touchpad and a scroll ring (Wherry, 2003), or the combination of (2) a standard mouse, a wheel mouse, a mouse with an integrated joystick, and the simultaneous use of a mouse with an integrated joystick and a standard mouse (using both hands) (Zhai and Smith, 1999). Although in mobile devices it is harder to integrate and test similar solutions, Fallman et al. (2004) presented and tested a prototype with a sensor connected to a PDA. Their approach was unrealistic because the mobile device should be laid flat to operate properly. Software scrolling techniques for PC include accelerated scrolling (Hinckley et al., 2002), animated scrolling (Klein and Bederson, 2005), the combination of a rougher scrolling with one mouse and precise scrolling with another mouse (Yin and Liu, 2010) and Speed Dependent Automatic Zooming (SDAZ) (Igarashi and Hinckley, 2000). SDAZ combines scrolling and automatic zooming thus facilitating the movement of the scroll while the user is searching for information on the screen. SDAZ has also been tested in mobile devices but results were not satisfactory (Jones et al., 2005). SDAZ has even been tested in devices with a tilt sensor but users still preferred the traditional method using the stylus of the device instead of the sensor (Eslambolchilar and Murray-Smith, 2008).
Other solutions that try to mitigate the negative effects of scrolling in mobile devices include (1) different implementations of multi-flick for pointing devices (Aliakseyeu et al., 2008); (2) using a double scrollbar (horizontal and vertical); (3) grab & drag, which enables navigation in the information space by dragging the portion currently being displayed; (4) ZEN (Zoom-Enhanced Navigator), which is an adaptation for mobile maps and mobile web pages of Overview&Detail approaches (Burigat et al., 2008); (5) RSVP (Rapid Serial Visual Presentation) in which the text is displayed on the screen word by word (Hedin and Lindgren, 2007; € Juola et al., 1995; Oquist and Lundin, 2007); and (6) leading, which automatically moves the text from right to left, pixel by pixel € (Juola et al., 1995; Oquist and Lundin, 2007). However, all these techniques require special software in order to operate and this software is usually not readily available on consumer devices. Scrolling has also been studied as a method to display textual information, and it is usually compared with paging. Paging consists in dividing the whole content in several chunks that fit in the screen allowing the user to navigate between them. In terms of efficiency, results on PC are contradictory since Baker (2003) found that users performed tasks more efficiently with scrolling while Dyson and Kipping (1998, p. 171) found that users did it faster using paging. In terms of comprehension of the text, results also differ significantly. Studies that concluded that scrolling provided better text comprehension (Sahin and Alsancak, 2011; Sanchez and Wiley, 2009) contrast with other findings that suggest that users had a better “sense of the text” when using paging (Piolat et al., 1997, p. 583). Positive aspects of scrolling are that it reduces manipulation €stlund (van Nimwegen et al., 1999) and user's mental workload (Wa et al., 2008). Scrolling and paging have also been compared as content distribution methods in mobile devices, but there are far fewer studies in this area. Findings on opinions of users about both methods differed, since in one study scrolling was preferred (Fukaya et al., 2011), whereas in another study it was not (Costa et al., 2007). Contrasting results have also been found in terms of efficiency or reading speed because results showed that scrolling was better € (Costa et al., 2007) but also that paging was better (Oquist and Lundin, 2007). Furthermore, we think that the results of these studies should be taken with care in relation to web text displayed on touchscreen devices. The study of Costa et al. (2007) was conducted with PDAs, so the interaction method was a stylus. The € study of Oquist and Lundin (2007) was carried out with a device with keyboard, while Fukaya et al. (2011) studied applications that display PDF documents. Summarizing, studies about scrolling in mobile devices compare or propose techniques that cannot be run on current devices because they are not natively supported and require installing additional software. Also, to the best of authors' knowledge, there are no works that study vertical scrolling in web pages displayed on touchscreen mobile devices, which are common today. This paper presents an experiment that compares scrolling with alternative content distribution methods (paging and internal links) that can be natively run in web pages displayed on mobile devices. Currently, there are no guidelines or recommendations concerning vertical scrolling in touchscreen mobile devices. Existing guidelines are for PC (ISO, 2008) or date back to 2008 (Rabin and McCathieNevile, 2008), when most mobile devices were not tactile. This paper then contributes to knowledge by suggesting a new guideline about content distribution in touchscreen smartphones. Next section describes the experimental design to test and compare the effectiveness, efficiency and satisfaction provided by different content distribution methods in order to determine the most usable for displaying web documents in touchscreen mobile devices.
E. Garcia-Lopez et al. / Applied Ergonomics 50 (2015) 79e86
3. Method Users commonly employ a specific scrolling strategy for each different type of document (Aliakseyeu et al., 2008), so a different analysis is probably required for each type. This paper focuses on the impact on usability when reading text from web pages displayed on a touchscreen mobile device. This section describes the experimental design to evaluate and compare the usability of three different content distribution methods (scrolling, paging and internal links) for displaying web information natively on mobile devices. 3.1. Hypotheses According to ISO 9241-11 (ISO, 1998), usability can be measured in three dimensions: effectiveness, efficiency and satisfaction. Effectiveness can be measured by the number of errors, efficiency can be measured by the time taken to perform a task, and satisfaction can be measured using a questionnaire. In our experiment we compared the usability of three different content distribution methods along these dimensions. The null hypotheses were: H1. There is no difference in errors between content distribution methods. H2. There is no difference in time between content distribution methods. H3. There is no difference in satisfaction between content distribution methods.
81
webcam. Recordings were later analyzed and comments provided valuable information about user interaction when performing different tasks. 3.4. Tasks In order to compare the effectiveness, efficiency and satisfaction of the different content distribution methods, participants were asked to find specific words in a fragment of text of 1263 words extracted from the Darwin's Theory of Evolution. Each participant was asked to find two words for each content distribution method. One word was near to the beginning of the text (hereinafter, “nearby” word), and the other word was closer to the end of the text (hereinafter, “distant” word). Andersen (2005) found that when scrolling the movement time increases linearly with the distance to the target. As the position of the word in the text potentially affects the time to locate it, we decided to use similar conditions for each content distribution method to get comparable results. Two test cases (nearby and distant) were introduced to provide variability in the experimental conditions. If the distance to the target is always similar then the users may learn this and improve their efficiency over time and do better with the content distribution methods that are assessed in a later stage of each user testing. Two test cases per content distribution method and the correct randomization of experimental conditions produced the necessary variability to get comparable results. Participants were not given information about the position of the word to be found in each test case. 3.5. Design
3.2. Participants Twenty-three subjects voluntarily participated in the experiment (sixteen were male, seven were female). Sixteen participants were between 18 and 24 years old, three were between 25 and 34 years, and four were aged 35 or more. Four of them were undergraduate students, eight had an Associate Degree, five had a Degree, five had a Master Degree and one had a PhD. Participants were asked about their use and expertise with mobile devices. All participants were regular users of mobile devices: Eleven were experts, ten acknowledged to have an intermediate level and two were novice users. None of the participants had problems reading text on the screen or had musculoskeletal disorders in their hands or arms. Twenty-one were right-handed and only two were left-handed. The right-handed participants used their left hand to grab the device and interacted with their right hand, whereas left-handed used their right hand to grab the device and interacted with their left hand. 3.3. Apparatus The experiment was conducted using an iPhone 4 with a 3.5inch touch-screen (960 640 pixels of resolution). The device was 115.2 mm 58.6 mm 9.3 mm and weighed 137 g. Only one device was selected because we considered that it was a good representation of the general characteristics of touch devices. Besides, we considered that it was not necessary to use additional devices on the study because scrolling and paging interactions do not depend on the operating system and therefore they do not vary significantly between different touch devices. A webcam was used to record the interaction of the user with the smartphone. The webcam was attached to a metal structure, and the structure was joined to a soft housing specific for this kind of device (Fig. 1). The webcam was connected to a laptop where videos were recorded. Audio was also recorded using the microphone integrated in the
The text was adapted to the width of the screen of the device so that only vertical scrolling was allowed. Text was not justified since this has been found to be better in mobile devices (Garcia-Lopez et al., 2013). The font type was Arial and had a 14-point size, large enough so that the participants could read it without having to zoom (to prevent additional scrolling). The font color was black on a white background, thus having the highest value of contrast (Nielsen and Loranger, 2006, p. 241). A fully factorial within-subject experimental design was used with two independent variables: distance to the word and content distribution method. Distance to the word, as we have just described, had two treatments: nearby or distant. Content distribution methods were paging (Fig. 2(a)), internal links (Fig. 2(b)) and vertical scrolling (Fig. 2(c)). When paging was used as a content distribution method, the main page displayed the title of the sections and an initial fragment of text for each section. Participants had to press on “More info” to continue reading the corresponding text of the section. As for internal links, the content was presented on a single web page but links were added at the top of the page to facilitate access to sections. As for vertical scrolling, all content was presented in a single web page with no links. Dependent variables were (1) effectiveness, as measured by the number of errors in finding the word; (2) efficiency, as measured by the time to find the word; and (3) satisfaction, which was measured using a questionnaire of five questions in which participants were asked about the ease of use, comfort and appropriateness of each content distribution method. Three different content distribution methods and two distances to the target produced six experimental conditions per participant. Therefore 138 videos were recorded. Results of 23 questionnaires were also collected. External factors that could affect scrolling performance were the distance to the target (Andersen, 2005; Hinckley et al., 2002), user familiarity with the document and the order in which experimental
82
E. Garcia-Lopez et al. / Applied Ergonomics 50 (2015) 79e86
Fig. 1. Apparatus used to record user interactions.
conditions were tested. To try to minimize external factors, the following considerations were taken into account: (1) The effect of the distance to the target was minimized by adding experimental conditions to find two words in the text, (2) the text was unfamiliar to all participants to mitigate knowledge biases, and (3) the order of the experimental conditions was randomly decided for each participant to mitigate the order effect. 3.6. Procedure The experiment with each participant took place in three stages: preparation, testing and subjective rating. During the initial stage (preparation), the experimenter collected the demographic information of the participant and described to her the purpose,
procedure and task to be performed. Participants were allowed to ask any question at this point. The main testing stage took place in a usability lab that provided the appropriate environmental conditions. For each experimental condition, the experimenter loaded the text in the web browser of the mobile device. Then the participant was asked to find the word in the text and inform the experimenter by pointing to it. Participants went through all six experimental conditions. During this stage, the experimenter did not answer any question about the interface. Participants were required to interact with the device in an intuitive way as they would do if they were working on their own. The third and final stage (subjective rating) was conducted immediately after testing. Participants had to complete a questionnaire in which they were asked to rate the content distribution methods according to their
Fig. 2. Prototypes of web pages designed for the experiment: (a) paging, (b) internal links and (c) vertical scrolling.
E. Garcia-Lopez et al. / Applied Ergonomics 50 (2015) 79e86
comfort, efficiency and appropriateness to perform the task. They were also asked about the appropriateness of distribution methods for all kinds of web content and about what method they found more convenient for specific contents. 4. Results After completing the experiments, quantitative information was extracted from video recordings for further statistical analysis. Questionnaires also provided quantitative and qualitative information that was appraised. Results are presented along the three dimensions of usability (dependent variables in our study). In what follows, statistical significance means p < 0.05. In several cases pvalues equal or less than 0.1 are also pointed out. 4.1. Effectiveness For the first hypothesis (H1-There is no difference in errors between content distribution methods), errors were measured considering how many times the user had to go through the text to find the word. If the user found the word the first time she read the text then the number of errors was zero. If she needed to go through the text four times then the number of errors was three as the three initial readings produced no result. Results are presented in Table 1. On average vertical scrolling resulted in fewer errors than the other content distribution methods. However results from a Friedman test (c22 ¼ 2.11, p ¼ 0.348) returned that no content distribution method was better than any other in terms of the number of errors made. Therefore, H1 was accepted. 4.2. Efficiency Efficiency was measured as the time to complete the task in seconds. On average, vertical scrolling was the most efficient content distribution method, followed by paging and then internal links (Table 2). Results of a Friedman test returned that there was statistical difference between groups (c22 ¼ 6, p ¼ 0.049). Pairwise comparisons showed significance between vertical scrolling and internal links (z ¼ 2.04, p ¼ 0.041), and between vertical scrolling and paging (z ¼ 1.66, p ¼ 0.1). As participants took on average less time to find a word with vertical scrolling, the second hypothesis (H2-There is no difference in time between content distribution methods) was rejected. 4.3. Satisfaction Satisfaction was measured using a questionnaire of five items. Questions 1 to 3 rated the comfort, ease of use, efficiency and appropriateness of each content distribution method. Participants answered using a 6-point Likert scale (0 ¼ strongly disagree, 5 ¼ strongly agree). All statements were scored in a positive scale. The fourth question surveyed participants' opinion on whether the most appropriate content distribution method depends on the type of web content delivered. It was also answered on a 6-point Likert positive scale. The fifth and final question was an open question in which participants were asked to provide their opinion on what
Table 1 Number of errors of content distribution methods.
Paging Internal links Vertical scrolling
AVG (errors)
Std. Dev.
Median
Std. Error
0.348 0.348 0.130
0.487 0.629 0.270
0.000 0.000 0.000
0.102 0.131 0.056
83
Table 2 Efficiency (time) of content distribution methods.
Paging Internal links Vertical scrolling
AVG (secs)
Std. Dev.
Median
Std. Error
146.1 171.5 118.7
67.4 111.5 62.7
134.0 132.0 114.5
14.1 23.3 13.1
content distribution method is the most appropriate for each type of web page. Results of questions 1e3 are summarized in Table 3. Participants found vertical scrolling more comfortable and efficient but paging was preferred as a general method for content distribution. In terms of comfort and ease of use (Q1), a Friedman test returned that there was statistical difference in responses (c22 ¼ 11.37, p ¼ 0.00). A pairwise comparison (Wilcoxon test) returned significant differences in two cases (Table 4), suggesting that participants found vertical scrolling to be the most comfortable and easy to use method of content distribution. In terms of perceived efficiency (Q2), there was also statistical significance (c22 ¼11.08, p ¼ 0.00) and pairwise comparison also returned that participants found vertical scrolling as the most efficient method. When participants were asked about what content distribution method they found more appropriate for all kinds of web content (Q3), they preferred paging and this result had a 90% confidence level (c22 ¼4.58, p ¼ 0.10). Pairwise comparison returned that participants found paging more appropriate than vertical scrolling for all kinds of web contents. When participants were asked to rate the statement ‘the most appropriate content distribution method depends on the type of web content’ (Q4), results (AVG ¼ 4.30, SD ¼ 1.02) suggest that in their opinion the best content distribution method strongly depends on the type of content to be delivered. Q5 asked participants to write which content distribution method was more convenient for each type of web content. An analysis of their answers returned that 21.74% of participants found that paging was more appropriate for web pages that have much content, while 34.78% found vertical scrolling more convenient for web pages with not that much content. The difference found between the answers given to Q1eQ2, in which vertical scrolling was preferred for the task at hand, and the answers given to Q3, in which paging was preferred for generic content, can possibly be explained in terms of the answers given to Q4 and Q5. Since the first two questions were related to the experiments, participants probably found that the text used in the experiment was not very long and vertical scrolling was preferred. Although the text of the experiment had 1263 words and it was necessary to swipe many times, when answering to Q3 participants may had in mind complete web sites with lots of contents or very long documents and therefore paging was preferred.
Table 3 Satisfaction of content distribution methods. Paging
Q1. It is comfortable and easy to use for the experiment Q2. It is efficient for the experiment Q3. In general, it is appropriate for all kinds of web pages
Internal links
Vertical scrolling
AVG (0e5)
Std. Dev.
AVG (0e5)
Std. Dev.
AVG (0e5)
Std. Dev.
2.652
1.584
2.609
1.588
4.217
1.043
2.783
1.594
2.304
1.490
3.696
1.222
3.261
1.685
2.826
1.435
2.565
1.273
84
E. Garcia-Lopez et al. / Applied Ergonomics 50 (2015) 79e86
Table 4 Pairwise comparison of comfort of content distribution methods (Wilcoxon tests).
Q1. It is comfortable and easy to use for the experiment Q2. It is efficient for the experiment Q3. In general, it is appropriate for all kinds of web pages
Paging vs internal links
Paging vs vertical scrolling
Internal links vs vertical scrolling
z
p
z
p
z
p
0.08 1.01 1.25
0.94 0.31 0.21
2.73 1.95 1.89
0.01 0.05 0.06
2.90 2.65 0.85
0.00 0.01 0.39
To sum up, participants found scrolling more comfortable, easy to use and efficient in the experiment, but they also found paging to be more appropriate as a general content distribution method. These differences were statistically significant so the third hypothesis (H3-There is no difference in satisfaction between content distribution methods) was rejected. 5. Discussion The main finding of the present study is that scrolling is more usable than paging and internal links as a content distribution method for displaying web-documents in touchscreen smartphones. In our experiment, usability was operationalized in terms of effectiveness, efficiency and user satisfaction. Scrolling resulted more effective, although results were not statistically significant. Similarly, scrolling was the most efficient content distribution method, as measured by the time that participants needed to complete the given tasks. Scrolling was also the content distribution method preferred by participants; although participants also pointed that paging may be a better option for substantially large contents. Given the increasing consumption of information in mobile devices, results of our study can be relevant for designing more usable web documents. Furthermore, providing that current usability guidelines do not address mobile devices or are outdated, we also propose the following guideline for designing interfaces for the mobile web: “try to use vertical scrolling in web pages for mobile devices instead of paging or internal links, except when the content is too large, and then paging is recommended”. For this guideline, our definition of large is based on users' opinion (user satisfaction) and it means complete web sites or very long documents. The text used in the experiment (1263 words) displayed along six screens (on an iPhone4) and the participants did not consider that it was large. Only when asked about the most convenient content distribution method, users found scrolling to be more appropriate for web content like the one presented in the experiment, but they also pointed out that paging may be more appropriate for very long documents and web sites. Our results in terms of efficiency support previous research performed by Baker (2003) with PC mouse, and Costa et al. (2007) with PDA and stylus. They also found that scrolling was more efficient than paging, and the results of our study confirm that their findings can also be generalized to touchscreen mobile devices. In terms of user satisfaction, our findings support previous work by Fukaya et al. (2011) as participants also preferred scrolling to other content distribution methods. We used the same device (touchscreen smartphone), but the kind of document was different: we focused on web pages with text while Fukaya et al. work focused on PDF documents. So apparently the type of document does not influence user's satisfaction and in touchscreen mobile devices users also prefer scrolling. Our findings differ from results presented in studies using PC € with keyboard. Oquist and Lundin (2007) and Dyson and Kipping (1998) found paging to be faster (more efficient) than scrolling. However, as in their experiments they used devices with keyboard, we argue that the type of interaction may influence efficiency.
Users perform faster when reading from a touchscreen in mobile device if they use scrolling, but paging is faster for devices with keyboard. In terms of satisfaction, our results differ from Costa et al. (2007) as in our experiment users found scrolling more comfortable, easy to use and efficient than the other content distribution methods while Costa et al. found that links were preferred. They used a PDA with stylus for their experiments and this suggests that the kind of device and type of interaction also influences user's satisfaction about content distribution methods. Therefore, the guideline that we have suggested could also be applied to other mobile devices and also to any web interface that uses a pointing device (i.e. PC with mouse). The new guideline can also be applicable to other types of documents in touchscreen mobile devices. Nevertheless, our results cannot be extended to other devices with different interaction methods like keyboards. We conjecture that in intuitive interfaces, like touchscreens, scrolling may be more usable but with indirect interaction methods, like keyboards, paging is faster. Further research needs to be conducted to corroborate or refute this hypothesis. Finally, in terms of user satisfaction our results differ from other pointing interaction methods even for mobile devices (namely, PDA with stylus). As user satisfaction involves participants' subjective opinion we suggest to be careful when interpreting results as different studies usually employ very different instruments to operationalize this construct. In our opinion, further research is needed to develop standardized methods that can produce comparable results in terms of user satisfaction for mobile interfaces. 5.1. Limitations of the study Our experiment was only carried out on a touchscreen smartphone. Research so far has found that the interaction method impacts efficiency (Shi and Wu, 2015) so results may not be generalized to all types of devices and further research needs to be conducted to evaluate their usability. Similarly, screen size and resolution may influence results since wider screens require less interaction to display the same amount of information and higher resolution may produce more readable fonts that also reduce the amount of interaction needed. However we consider that our main results and the guideline suggested can be generalized across touchscreen mobile devices. As the main factors that can have effect on its applicability are the number of interactions (swipes) required to go through the content and the amount of information displayed on the screen of the device, any change in screen resolution or font size will only impact on the length of the web page that is suitable to be displayed using scrolling for each device. Sample size of our experiment was small and homogenous in statistical terms. Although the well-known 10 ± 2 rule (Hwang and Salvendy, 2010) may provide validity to our results, we also have to acknowledge that most participants were young, possibly digital natives, and therefore we have to be careful in terms of the generality of results. Although this target is important, results may be biased and more representative populations would be required to get a bigger picture.
E. Garcia-Lopez et al. / Applied Ergonomics 50 (2015) 79e86
Finally, the apparatus that was used to record user interactions could have biased results since it may obstruct natural interaction with the smartphone. However we think that after a few moments, participants get familiar with it and it had a minimal, if any, influence since it did not interfere in the view and it was light. In any case, developing less invasive methods to record interaction will be required to abate any threat to validity that this and similar external factors may entail. 6. Conclusion and future work This paper presented an experiment to evaluate the usability of three different content distribution methods for displaying web documents in touchscreen mobile devices. Following the ISO 924111 recommendation, we measured the three dimensions of usability: effectiveness, efficiency and user satisfaction. Participants (N ¼ 23) were asked to find words in average-sized web documents presented on a smartphone. Results suggested that in terms of effectiveness vertical scrolling was on average better than paging and internal links. This means that users made fewer errors when the content was displayed in a web page with vertical scrolling, although the differences found were not statistically significant. In terms of efficiency, findings suggested that when using vertical scrolling users employed less time to read texts in web pages displayed on mobile devices. In terms of satisfaction, users also preferred vertical scrolling over paging or internal links. However, there was one exception in this case as results also suggested that if the content to be displayed was rather large then users preferred paging. In the light of our results and previous studies, we suggest the following guideline for mobile web interfaces: “try to use vertical scrolling in web pages for mobile devices instead of paging or internal links, except when the content is too large, and then paging is recommended”. In our opinion, this guideline is applicable to mobile devices as well as to any device that uses a pointing interaction method. In terms of efficiency, existing literature as well as the results presented here suggest that the new guideline may result in more usable web content. An analysis of existing results for touchscreen mobile devices also suggests that this new guideline can even be applied to other types of contents like documents. In terms of user satisfaction, we have found contrasting results and existing literature is, in our opinion, both scarce and differing. In our study participants pointed that they preferred paging for large web contents, so it would be interesting to further investigate what they mean by large or for what kind of content paging would be preferred. Research community should also strive to develop standardized methods to assess user satisfaction that produce comparable results across platforms and across interaction methods. The guideline that we have introduced here could, in our opinion, be generalized across the multiplicity of touchscreen mobile devices with different screen resolution and font sizes. However, it can also be interesting to study the impact of screen size and even screen resolution because the former may influence the number of interactions to go through the content while the later may influence the readability of the content, therefore both can potentially influence usability. As additional future work we plan to address other mobile devices with different interaction methods (e.g. keyboard, stylus) because existing literature has found that they differ in terms of usability. Acknowledgments The first author thanks the Spanish Government (Ministerio de n, Cultura y Deporte) for a FPU grant (number AP2010Educacio
85
0499). The second author thanks the Junta de Comunidades de Castilla-La Mancha for a FPI grant. References Ahmadi, H., Kong, J., 2012. User-centric adaptation of web information for small screens. J. Vis. Lang. Comput. 23, 13e28. Aliakseyeu, D., Irani, P., Lucero, A., Subramanian, S., 2008. Multi-flick: an evaluation of flick-based scrolling techniques for pen interfaces. In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems. ACM, pp. 1689e1698. Andersen, T.H., 2005. A Simple Movement Time Model for Scrolling, CHI'05 Extended Abstracts on Human Factors in Computing Systems. ACM, pp. 1180e1183. Baker, J.R., 2003. The Impact of Paging vs. Scrolling on Reading Online Text Passages. Usability News 5. Buchanan, G., Farrant, S., Jones, M., Thimbleby, H., Marsden, G., Pazzani, M., 2001. Improving mobile internet usability. In: Proceedings of the 10th International Conference on World Wide Web. ACM, pp. 673e680. Burigat, S., Chittaro, L., Gabrielli, S., 2008. Navigation techniques for small-screen devices: an evaluation on maps and web pages. Int. J. Hum. Comput. Stud. 66, 78e97. Costa, C.J., Silva, J., Aparício, M., 2007. Evaluating web usability using small display devices. In: Proceedings of the 25th Annual ACM International Conference on Design of Communication. ACM, pp. 263e268. Dillon, A., Richardson, J., McKnight, C., 1990. The effects of display size and text splitting on reading lengthy text from screen. Behav. Inform. Technol. 9, 215e227. Dyson, M.C., Kipping, G.J., 1998. The effects of line length and method of movement on patterns of reading from screen. Visible Lang. 32, 150e181. Eslambolchilar, P., Murray-Smith, R., 2008. Control centric approach in designing scrolling and zooming user interfaces. Int. J. Hum. Comput. Stud. 66, 838e856. Fallman, D., Lund, A., Wiberg, M., 2004. Scrollpad: tangible scrolling with mobile devices. In: Proceedings of the 37th Annual Hawaii International Conference on System Sciences. IEEE. Fukaya, T.Y., Ono, S., Minakuchi, M., Nakashima, S., Hayashi, M., Ando, H., 2011. Reading text on a smart phone: scrolling vs. paging: toward designing effective electronic manuals. In: 2011 International Conference on User Science and Engineering (i-USEr). IEEE, pp. 59e63. Garcia-Lopez, E., Garcia-Cabot, A., de-Marcos, L., Hilera, J.-R., 2013. Usability Guidelines for Mobile Devices: Length of Hyperlinks, Human-computer InteractioneINTERACT 2013. Springer, pp. 705e712. Geven, A., Sefelin, R., Tscheligi, M., 2006. Depth and breadth away from the desktop: the optimal information hierarchy for mobile use. In: Proceedings of the 8th Conference on Human-computer Interaction with Mobile Devices and Services. ACM, pp. 157e164. Hedin, B., Lindgren, E., 2007. A comparison of presentation methods for reading on mobile phones. IEEE Distrib. Syst. Online 8. Hinckley, K., Cutrell, E., Bathiche, S., Muss, T., 2002. Quantitative analysis of scrolling techniques. In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems. ACM, pp. 65e72. Hwang, W., Salvendy, G., 2010. Number of people required for usability evaluation: the 10 ± 2 rule. Commun. ACM 53, 130e133. Igarashi, T., Hinckley, K., 2000. Speed-dependent automatic zooming for browsing large documents. In: Proceedings of the 13th Annual ACM Symposium on User Interface Software and Technology. ACM, pp. 139e148. ISO, 1998. Ergonomic Requirements for Office Work with Visual Display Terminals (VDTs) e Part 11: Guidance on Usability. ISO, 2008. Ergonomics of Human-system Interaction e Part 151: Guidance on World Wide Web User Interfaces. Jones, M., Marsden, G., Mohd-Nasir, N., Boone, K., Buchanan, G., 1999. Improving web interaction on small displays. Comput. Netw. 31, 1129e1137. Jones, S., Jones, M., Marsden, G., Patel, D., Cockburn, A., 2005. An evaluation of integrated zooming and scrolling on small screens. Int. J. Hum. Comput. Stud. 63, 271e303. Juola, J.F., Tiritoglu, A., Pleunis, J., 1995. Reading text presented on a small display. Appl. Ergon. 26, 227e229. Kim, L., Albers, M.J., 2001. Web design issues when searching for information in a small screen display. In: Proceedings of the 19th Annual International Conference on Computer Documentation. ACM, pp. 193e200. Klein, C., Bederson, B.B., 2005. Benefits of Animated Scrolling, CHI'05 Extended Abstracts on Human Factors in Computing Systems. ACM, pp. 1965e1968. Lai, C.-C., Wu, C.-F., 2014. Display and device size effects on the usability of mininotebooks (netbooks)/ultraportables as small form-factor mobile PCs. Appl. Ergon. 45, 1106e1115. Lee, K., Bahn, H., 2005. A compressed display technique for 240 320 resolution personal digital assistants (PDAs). IEEE Trans. Consum. Electron. 51, 1268e1272. Mills, C.B., Weldon, L.J., 1987. Reading text from computer screens. ACM Comput. Surv. 19, 329e357. Nielsen, J., 1997. Changes in Web Usability Since 1994. useit.com. http://www. nngroup.com/articles/changes-in-web-usability-since-1994/. Nielsen, J., Loranger, H., 2006. Prioritizing Web Usability. Pearson Education.
86
E. Garcia-Lopez et al. / Applied Ergonomics 50 (2015) 79e86
€ Oquist, G., Lundin, K., 2007. Eye movement study of reading text on a mobile phone using paging, scrolling, leading, and RSVP. In: Proceedings of the 6th International Conference on Mobile and Ubiquitous Multimedia. ACM, pp. 176e183. Parush, A., Yuviler-Gavish, N., 2004. Web navigation structures in cellular phones: the depth/breadth trade-off issue. Int. J. Hum. Comput. Stud. 60, 753e770. Piolat, A., Roussey, J.-Y., Thunin, O., 1997. Effects of screen presentation on text reading and revising. Int. J. Hum. Comput. Stud. 47, 565e589. Rabin, J., McCathieNevile, C., 2008. Mobile Web Best Practices 1.0: Basic Guidelines. W3C. Sahin, A., Alsancak, D., 2011. The effect of hypertexts with different lengths on reading and comprehension skills of the students. Procedia-Social Behav. Sci. 28, 347e354. Sanchez, C.A., Wiley, J., 2009. To scroll or not to scroll: scrolling, working memory capacity, and comprehending complex texts. Hum. Factors: J. Hum. Factors Ergon. Soc. 51, 730e738.
Shi, W.-Z., Wu, F.-G., 2015. An investigation of the performance of novel chorded keyboards in combination with pointing input devices. Appl. Ergon. 46, 1e7. van Nimwegen, C., Pouw, M., Van Oostendorp, H., 1999. The influence of structure and reading-manipulation on usability of hypertexts. Interact. Comput. 12, 7e21. Wang, A.-H., Lai, Y.-Y., Sun, C.-T., 2004. Effect of PDA scrolling-and hierarchy-menu design on users' operating performance. Displays 25, 109e114. €stlund, E., Norlander, T., Archer, T., 2008. The effect of page layout on mental Wa workload: a dual-task experiment. Comput. Hum. Behav. 24, 1229e1245. Wherry, E., 2003. Scroll Ring Performance Evaluation, CHI'03 Extended Abstracts on Human Factors in Computing Systems. ACM, pp. 758e759. Yin, J., Liu, H., 2010. The empirical study of a novel bimanual scrolling technique. In: 2010 IEEE International Conference on Information and Automation (ICIA). IEEE, pp. 110e113. Zhai, S., Smith, B.A., 1999. Multistream input: an experimental study of document scrolling methods. IBM Syst. J. 38, 642e651.
Contents lists available at ScienceDirect
Applied Ergonomics journal homepage: www.elsevier.com/locate/apergo
An experiment with content distribution methods in touchscreen mobile devices Eva Garcia-Lopez*, Antonio Garcia-Cabot, Luis de-Marcos , Spain Computer Science Department, University of Alcala
a r t i c l e i n f o
a b s t r a c t
Article history: Received 12 May 2014 Accepted 27 February 2015 Available online
This paper compares the usability of three different content distribution methods (scrolling, paging and internal links) in touchscreen mobile devices as means to display web documents. Usability is operationalized in terms of effectiveness, efficiency and user satisfaction. These dimensions are then measured in an experiment (N ¼ 23) in which users are required to find words in regular-length web documents. Results suggest that scrolling is statistically better in terms of efficiency and user satisfaction. It is also found to be more effective but results were not significant. Our findings are also compared with existing literature to propose the following guideline: “try to use vertical scrolling in web pages for mobile devices instead of paging or internal links, except when the content is too large, then paging is recommended”. With an ever increasing number of touchscreen web-enabled mobile devices, this new guideline can be relevant for content developers targeting the mobile web as well as institutions trying to improve the usability of their content for mobile platforms. © 2015 Elsevier Ltd and The Ergonomics Society. All rights reserved.
Keywords: Mobile usability Empirical study Scrolling
1. Introduction Several aspects of the usability of interfaces have already been addressed in Personal Computers (PC) including characters, formatting, contrast and color, and dynamic text (Mills and Weldon, 1987) or line length (Dyson and Kipping, 1998). However, it is also necessary to study similar and other usability aspects in mobile devices because of the substantial differences that exist between them and PCs, especially concerning the size of the screen. Such differences may have a negative influence in the efficiency of users (Jones et al., 1999; Parush and Yuviler-Gavish, 2004). The degree of interaction is usually higher when using a small screen (Dillon et al., 1990) because of the limited amount of information that can be displayed. Therefore smaller screens require that the user scrolls more times and more often to go through all the content. Several studies have compared different aspects of usability when performing tasks with screens of different sizes, where more or less information can be showed at a glance. Although Kim and
n, Edificio Polite cnico, Ctra * Corresponding author. Dpto Ciencias Computacio de Henares, Madrid, Spain. Tel.: þ34 918856836; Barcelona km 33.1, 28871 Alcala fax: þ34 918856646. E-mail addresses: [email protected] (E. Garcia-Lopez), [email protected] (A. Garcia-Cabot), [email protected] (L. de-Marcos). http://dx.doi.org/10.1016/j.apergo.2015.02.009 0003-6870/© 2015 Elsevier Ltd and The Ergonomics Society. All rights reserved.
Albers (2001) found that there was no difference between a small screen (a PDA in their study) and a larger one (PC) in terms of the number of errors made to complete a task and time taken; most studies found differences in effectiveness (Jones et al., 1999), efficiency (Lai and Wu, 2014; Parush and Yuviler-Gavish, 2004) and satisfaction (Dillon et al., 1990). Such differences evidence the necessity to study independently the interfaces of mobile devices. For instance, adaptive systems that accommodate web pages to mobile interfaces have been developed to try to improve the usability of mobile navigation (Ahmadi and Kong, 2012; Lee and Bahn, 2005). One of the major issues concerning usability for mobile devices is the scroll. Scrolling happens when the information that has to be displayed does not fit in a single screen, “overflowing” off the screen out of immediate view (Sanchez and Wiley, 2009). This results in a serious usability problem (Nielsen, 1997) when the user is reading since it is necessary to handle the device and the information being displayed simultaneously, and this can interfere reading (W€ astlund et al., 2008). The problem is aggravated when the user is in front of a small screen as she will have to go through more pages to read the same amount of text (Dillon et al., 1990). Nowadays, with an ever increasing number of accesses to the mobile web, it is also becoming increasingly important to study the usability of web interfaces in mobile devices. Research to date has studied mobile devices with stylus, keyboards or even with
80
E. Garcia-Lopez et al. / Applied Ergonomics 50 (2015) 79e86
simulated devices in computers, but today the most common mobile device is a touchscreen smartphone. The objective of this paper is to analyze the usability of methods to present information for users of web content in mobile devices with touchscreen. The most common content distribution method is vertical scrolling. It is used for texts (Andersen, 2005), documents and web pages (Yin and Liu, 2010). This paper sets out to test whether vertical scrolling in mobile devices is more usable in webpages than other methods like paging or internal links in terms of effectiveness, efficiency and user satisfaction. Results of our study can be relevant for mobile web developers who are involved in designing interfaces, and also for companies and organizations interested in increasing the readability of their web sites and applications. We provide a new usability guideline to improve the usability of regular-length web documents displayed in touchscreen mobile devices. The paper is structured as follows. Section 2 presents related work pointing the necessity of studying scrolling in web interfaces for touchscreen mobile devices. Section 3 presents the method and experimental design. Section 4 presents the results of experimentation. Section 5 presents discussion comparing our findings with previous studies. Finally, Section 6 outlines conclusions.
2. Related work Scrolling in mobile devices was initially studied for the native menus of mobile devices and in simple web menus displayed in such devices. Different studies about device menus found similar results. For example, Wang et al. (2004) suggested using hierarchical menus with a maximum of two levels, which inevitably increases vertical scrolling. Similarly, Buchanan et al. (2001) found that vertical scrolling is the best method for most users. Contrastingly, results of studies dealing with web menus were mostly contradictory. For example, Geven et al. (2006) concluded that using deeper hierarchies is better than using wider menus, thus suggesting using more levels instead of having too many options per level; while Parush and Yuviler-Gavish (2004) found that limiting depth rather than width is preferable. In order to try to mitigate the negative effects of scrolling, different hardware and software alternatives have been devised both for PC and mobile devices. Studies on hardware solutions for PC compared diverse gadgets like (1) a wheel mouse, a touchpad and a scroll ring (Wherry, 2003), or the combination of (2) a standard mouse, a wheel mouse, a mouse with an integrated joystick, and the simultaneous use of a mouse with an integrated joystick and a standard mouse (using both hands) (Zhai and Smith, 1999). Although in mobile devices it is harder to integrate and test similar solutions, Fallman et al. (2004) presented and tested a prototype with a sensor connected to a PDA. Their approach was unrealistic because the mobile device should be laid flat to operate properly. Software scrolling techniques for PC include accelerated scrolling (Hinckley et al., 2002), animated scrolling (Klein and Bederson, 2005), the combination of a rougher scrolling with one mouse and precise scrolling with another mouse (Yin and Liu, 2010) and Speed Dependent Automatic Zooming (SDAZ) (Igarashi and Hinckley, 2000). SDAZ combines scrolling and automatic zooming thus facilitating the movement of the scroll while the user is searching for information on the screen. SDAZ has also been tested in mobile devices but results were not satisfactory (Jones et al., 2005). SDAZ has even been tested in devices with a tilt sensor but users still preferred the traditional method using the stylus of the device instead of the sensor (Eslambolchilar and Murray-Smith, 2008).
Other solutions that try to mitigate the negative effects of scrolling in mobile devices include (1) different implementations of multi-flick for pointing devices (Aliakseyeu et al., 2008); (2) using a double scrollbar (horizontal and vertical); (3) grab & drag, which enables navigation in the information space by dragging the portion currently being displayed; (4) ZEN (Zoom-Enhanced Navigator), which is an adaptation for mobile maps and mobile web pages of Overview&Detail approaches (Burigat et al., 2008); (5) RSVP (Rapid Serial Visual Presentation) in which the text is displayed on the screen word by word (Hedin and Lindgren, 2007; € Juola et al., 1995; Oquist and Lundin, 2007); and (6) leading, which automatically moves the text from right to left, pixel by pixel € (Juola et al., 1995; Oquist and Lundin, 2007). However, all these techniques require special software in order to operate and this software is usually not readily available on consumer devices. Scrolling has also been studied as a method to display textual information, and it is usually compared with paging. Paging consists in dividing the whole content in several chunks that fit in the screen allowing the user to navigate between them. In terms of efficiency, results on PC are contradictory since Baker (2003) found that users performed tasks more efficiently with scrolling while Dyson and Kipping (1998, p. 171) found that users did it faster using paging. In terms of comprehension of the text, results also differ significantly. Studies that concluded that scrolling provided better text comprehension (Sahin and Alsancak, 2011; Sanchez and Wiley, 2009) contrast with other findings that suggest that users had a better “sense of the text” when using paging (Piolat et al., 1997, p. 583). Positive aspects of scrolling are that it reduces manipulation €stlund (van Nimwegen et al., 1999) and user's mental workload (Wa et al., 2008). Scrolling and paging have also been compared as content distribution methods in mobile devices, but there are far fewer studies in this area. Findings on opinions of users about both methods differed, since in one study scrolling was preferred (Fukaya et al., 2011), whereas in another study it was not (Costa et al., 2007). Contrasting results have also been found in terms of efficiency or reading speed because results showed that scrolling was better € (Costa et al., 2007) but also that paging was better (Oquist and Lundin, 2007). Furthermore, we think that the results of these studies should be taken with care in relation to web text displayed on touchscreen devices. The study of Costa et al. (2007) was conducted with PDAs, so the interaction method was a stylus. The € study of Oquist and Lundin (2007) was carried out with a device with keyboard, while Fukaya et al. (2011) studied applications that display PDF documents. Summarizing, studies about scrolling in mobile devices compare or propose techniques that cannot be run on current devices because they are not natively supported and require installing additional software. Also, to the best of authors' knowledge, there are no works that study vertical scrolling in web pages displayed on touchscreen mobile devices, which are common today. This paper presents an experiment that compares scrolling with alternative content distribution methods (paging and internal links) that can be natively run in web pages displayed on mobile devices. Currently, there are no guidelines or recommendations concerning vertical scrolling in touchscreen mobile devices. Existing guidelines are for PC (ISO, 2008) or date back to 2008 (Rabin and McCathieNevile, 2008), when most mobile devices were not tactile. This paper then contributes to knowledge by suggesting a new guideline about content distribution in touchscreen smartphones. Next section describes the experimental design to test and compare the effectiveness, efficiency and satisfaction provided by different content distribution methods in order to determine the most usable for displaying web documents in touchscreen mobile devices.
E. Garcia-Lopez et al. / Applied Ergonomics 50 (2015) 79e86
3. Method Users commonly employ a specific scrolling strategy for each different type of document (Aliakseyeu et al., 2008), so a different analysis is probably required for each type. This paper focuses on the impact on usability when reading text from web pages displayed on a touchscreen mobile device. This section describes the experimental design to evaluate and compare the usability of three different content distribution methods (scrolling, paging and internal links) for displaying web information natively on mobile devices. 3.1. Hypotheses According to ISO 9241-11 (ISO, 1998), usability can be measured in three dimensions: effectiveness, efficiency and satisfaction. Effectiveness can be measured by the number of errors, efficiency can be measured by the time taken to perform a task, and satisfaction can be measured using a questionnaire. In our experiment we compared the usability of three different content distribution methods along these dimensions. The null hypotheses were: H1. There is no difference in errors between content distribution methods. H2. There is no difference in time between content distribution methods. H3. There is no difference in satisfaction between content distribution methods.
81
webcam. Recordings were later analyzed and comments provided valuable information about user interaction when performing different tasks. 3.4. Tasks In order to compare the effectiveness, efficiency and satisfaction of the different content distribution methods, participants were asked to find specific words in a fragment of text of 1263 words extracted from the Darwin's Theory of Evolution. Each participant was asked to find two words for each content distribution method. One word was near to the beginning of the text (hereinafter, “nearby” word), and the other word was closer to the end of the text (hereinafter, “distant” word). Andersen (2005) found that when scrolling the movement time increases linearly with the distance to the target. As the position of the word in the text potentially affects the time to locate it, we decided to use similar conditions for each content distribution method to get comparable results. Two test cases (nearby and distant) were introduced to provide variability in the experimental conditions. If the distance to the target is always similar then the users may learn this and improve their efficiency over time and do better with the content distribution methods that are assessed in a later stage of each user testing. Two test cases per content distribution method and the correct randomization of experimental conditions produced the necessary variability to get comparable results. Participants were not given information about the position of the word to be found in each test case. 3.5. Design
3.2. Participants Twenty-three subjects voluntarily participated in the experiment (sixteen were male, seven were female). Sixteen participants were between 18 and 24 years old, three were between 25 and 34 years, and four were aged 35 or more. Four of them were undergraduate students, eight had an Associate Degree, five had a Degree, five had a Master Degree and one had a PhD. Participants were asked about their use and expertise with mobile devices. All participants were regular users of mobile devices: Eleven were experts, ten acknowledged to have an intermediate level and two were novice users. None of the participants had problems reading text on the screen or had musculoskeletal disorders in their hands or arms. Twenty-one were right-handed and only two were left-handed. The right-handed participants used their left hand to grab the device and interacted with their right hand, whereas left-handed used their right hand to grab the device and interacted with their left hand. 3.3. Apparatus The experiment was conducted using an iPhone 4 with a 3.5inch touch-screen (960 640 pixels of resolution). The device was 115.2 mm 58.6 mm 9.3 mm and weighed 137 g. Only one device was selected because we considered that it was a good representation of the general characteristics of touch devices. Besides, we considered that it was not necessary to use additional devices on the study because scrolling and paging interactions do not depend on the operating system and therefore they do not vary significantly between different touch devices. A webcam was used to record the interaction of the user with the smartphone. The webcam was attached to a metal structure, and the structure was joined to a soft housing specific for this kind of device (Fig. 1). The webcam was connected to a laptop where videos were recorded. Audio was also recorded using the microphone integrated in the
The text was adapted to the width of the screen of the device so that only vertical scrolling was allowed. Text was not justified since this has been found to be better in mobile devices (Garcia-Lopez et al., 2013). The font type was Arial and had a 14-point size, large enough so that the participants could read it without having to zoom (to prevent additional scrolling). The font color was black on a white background, thus having the highest value of contrast (Nielsen and Loranger, 2006, p. 241). A fully factorial within-subject experimental design was used with two independent variables: distance to the word and content distribution method. Distance to the word, as we have just described, had two treatments: nearby or distant. Content distribution methods were paging (Fig. 2(a)), internal links (Fig. 2(b)) and vertical scrolling (Fig. 2(c)). When paging was used as a content distribution method, the main page displayed the title of the sections and an initial fragment of text for each section. Participants had to press on “More info” to continue reading the corresponding text of the section. As for internal links, the content was presented on a single web page but links were added at the top of the page to facilitate access to sections. As for vertical scrolling, all content was presented in a single web page with no links. Dependent variables were (1) effectiveness, as measured by the number of errors in finding the word; (2) efficiency, as measured by the time to find the word; and (3) satisfaction, which was measured using a questionnaire of five questions in which participants were asked about the ease of use, comfort and appropriateness of each content distribution method. Three different content distribution methods and two distances to the target produced six experimental conditions per participant. Therefore 138 videos were recorded. Results of 23 questionnaires were also collected. External factors that could affect scrolling performance were the distance to the target (Andersen, 2005; Hinckley et al., 2002), user familiarity with the document and the order in which experimental
82
E. Garcia-Lopez et al. / Applied Ergonomics 50 (2015) 79e86
Fig. 1. Apparatus used to record user interactions.
conditions were tested. To try to minimize external factors, the following considerations were taken into account: (1) The effect of the distance to the target was minimized by adding experimental conditions to find two words in the text, (2) the text was unfamiliar to all participants to mitigate knowledge biases, and (3) the order of the experimental conditions was randomly decided for each participant to mitigate the order effect. 3.6. Procedure The experiment with each participant took place in three stages: preparation, testing and subjective rating. During the initial stage (preparation), the experimenter collected the demographic information of the participant and described to her the purpose,
procedure and task to be performed. Participants were allowed to ask any question at this point. The main testing stage took place in a usability lab that provided the appropriate environmental conditions. For each experimental condition, the experimenter loaded the text in the web browser of the mobile device. Then the participant was asked to find the word in the text and inform the experimenter by pointing to it. Participants went through all six experimental conditions. During this stage, the experimenter did not answer any question about the interface. Participants were required to interact with the device in an intuitive way as they would do if they were working on their own. The third and final stage (subjective rating) was conducted immediately after testing. Participants had to complete a questionnaire in which they were asked to rate the content distribution methods according to their
Fig. 2. Prototypes of web pages designed for the experiment: (a) paging, (b) internal links and (c) vertical scrolling.
E. Garcia-Lopez et al. / Applied Ergonomics 50 (2015) 79e86
comfort, efficiency and appropriateness to perform the task. They were also asked about the appropriateness of distribution methods for all kinds of web content and about what method they found more convenient for specific contents. 4. Results After completing the experiments, quantitative information was extracted from video recordings for further statistical analysis. Questionnaires also provided quantitative and qualitative information that was appraised. Results are presented along the three dimensions of usability (dependent variables in our study). In what follows, statistical significance means p < 0.05. In several cases pvalues equal or less than 0.1 are also pointed out. 4.1. Effectiveness For the first hypothesis (H1-There is no difference in errors between content distribution methods), errors were measured considering how many times the user had to go through the text to find the word. If the user found the word the first time she read the text then the number of errors was zero. If she needed to go through the text four times then the number of errors was three as the three initial readings produced no result. Results are presented in Table 1. On average vertical scrolling resulted in fewer errors than the other content distribution methods. However results from a Friedman test (c22 ¼ 2.11, p ¼ 0.348) returned that no content distribution method was better than any other in terms of the number of errors made. Therefore, H1 was accepted. 4.2. Efficiency Efficiency was measured as the time to complete the task in seconds. On average, vertical scrolling was the most efficient content distribution method, followed by paging and then internal links (Table 2). Results of a Friedman test returned that there was statistical difference between groups (c22 ¼ 6, p ¼ 0.049). Pairwise comparisons showed significance between vertical scrolling and internal links (z ¼ 2.04, p ¼ 0.041), and between vertical scrolling and paging (z ¼ 1.66, p ¼ 0.1). As participants took on average less time to find a word with vertical scrolling, the second hypothesis (H2-There is no difference in time between content distribution methods) was rejected. 4.3. Satisfaction Satisfaction was measured using a questionnaire of five items. Questions 1 to 3 rated the comfort, ease of use, efficiency and appropriateness of each content distribution method. Participants answered using a 6-point Likert scale (0 ¼ strongly disagree, 5 ¼ strongly agree). All statements were scored in a positive scale. The fourth question surveyed participants' opinion on whether the most appropriate content distribution method depends on the type of web content delivered. It was also answered on a 6-point Likert positive scale. The fifth and final question was an open question in which participants were asked to provide their opinion on what
Table 1 Number of errors of content distribution methods.
Paging Internal links Vertical scrolling
AVG (errors)
Std. Dev.
Median
Std. Error
0.348 0.348 0.130
0.487 0.629 0.270
0.000 0.000 0.000
0.102 0.131 0.056
83
Table 2 Efficiency (time) of content distribution methods.
Paging Internal links Vertical scrolling
AVG (secs)
Std. Dev.
Median
Std. Error
146.1 171.5 118.7
67.4 111.5 62.7
134.0 132.0 114.5
14.1 23.3 13.1
content distribution method is the most appropriate for each type of web page. Results of questions 1e3 are summarized in Table 3. Participants found vertical scrolling more comfortable and efficient but paging was preferred as a general method for content distribution. In terms of comfort and ease of use (Q1), a Friedman test returned that there was statistical difference in responses (c22 ¼ 11.37, p ¼ 0.00). A pairwise comparison (Wilcoxon test) returned significant differences in two cases (Table 4), suggesting that participants found vertical scrolling to be the most comfortable and easy to use method of content distribution. In terms of perceived efficiency (Q2), there was also statistical significance (c22 ¼11.08, p ¼ 0.00) and pairwise comparison also returned that participants found vertical scrolling as the most efficient method. When participants were asked about what content distribution method they found more appropriate for all kinds of web content (Q3), they preferred paging and this result had a 90% confidence level (c22 ¼4.58, p ¼ 0.10). Pairwise comparison returned that participants found paging more appropriate than vertical scrolling for all kinds of web contents. When participants were asked to rate the statement ‘the most appropriate content distribution method depends on the type of web content’ (Q4), results (AVG ¼ 4.30, SD ¼ 1.02) suggest that in their opinion the best content distribution method strongly depends on the type of content to be delivered. Q5 asked participants to write which content distribution method was more convenient for each type of web content. An analysis of their answers returned that 21.74% of participants found that paging was more appropriate for web pages that have much content, while 34.78% found vertical scrolling more convenient for web pages with not that much content. The difference found between the answers given to Q1eQ2, in which vertical scrolling was preferred for the task at hand, and the answers given to Q3, in which paging was preferred for generic content, can possibly be explained in terms of the answers given to Q4 and Q5. Since the first two questions were related to the experiments, participants probably found that the text used in the experiment was not very long and vertical scrolling was preferred. Although the text of the experiment had 1263 words and it was necessary to swipe many times, when answering to Q3 participants may had in mind complete web sites with lots of contents or very long documents and therefore paging was preferred.
Table 3 Satisfaction of content distribution methods. Paging
Q1. It is comfortable and easy to use for the experiment Q2. It is efficient for the experiment Q3. In general, it is appropriate for all kinds of web pages
Internal links
Vertical scrolling
AVG (0e5)
Std. Dev.
AVG (0e5)
Std. Dev.
AVG (0e5)
Std. Dev.
2.652
1.584
2.609
1.588
4.217
1.043
2.783
1.594
2.304
1.490
3.696
1.222
3.261
1.685
2.826
1.435
2.565
1.273
84
E. Garcia-Lopez et al. / Applied Ergonomics 50 (2015) 79e86
Table 4 Pairwise comparison of comfort of content distribution methods (Wilcoxon tests).
Q1. It is comfortable and easy to use for the experiment Q2. It is efficient for the experiment Q3. In general, it is appropriate for all kinds of web pages
Paging vs internal links
Paging vs vertical scrolling
Internal links vs vertical scrolling
z
p
z
p
z
p
0.08 1.01 1.25
0.94 0.31 0.21
2.73 1.95 1.89
0.01 0.05 0.06
2.90 2.65 0.85
0.00 0.01 0.39
To sum up, participants found scrolling more comfortable, easy to use and efficient in the experiment, but they also found paging to be more appropriate as a general content distribution method. These differences were statistically significant so the third hypothesis (H3-There is no difference in satisfaction between content distribution methods) was rejected. 5. Discussion The main finding of the present study is that scrolling is more usable than paging and internal links as a content distribution method for displaying web-documents in touchscreen smartphones. In our experiment, usability was operationalized in terms of effectiveness, efficiency and user satisfaction. Scrolling resulted more effective, although results were not statistically significant. Similarly, scrolling was the most efficient content distribution method, as measured by the time that participants needed to complete the given tasks. Scrolling was also the content distribution method preferred by participants; although participants also pointed that paging may be a better option for substantially large contents. Given the increasing consumption of information in mobile devices, results of our study can be relevant for designing more usable web documents. Furthermore, providing that current usability guidelines do not address mobile devices or are outdated, we also propose the following guideline for designing interfaces for the mobile web: “try to use vertical scrolling in web pages for mobile devices instead of paging or internal links, except when the content is too large, and then paging is recommended”. For this guideline, our definition of large is based on users' opinion (user satisfaction) and it means complete web sites or very long documents. The text used in the experiment (1263 words) displayed along six screens (on an iPhone4) and the participants did not consider that it was large. Only when asked about the most convenient content distribution method, users found scrolling to be more appropriate for web content like the one presented in the experiment, but they also pointed out that paging may be more appropriate for very long documents and web sites. Our results in terms of efficiency support previous research performed by Baker (2003) with PC mouse, and Costa et al. (2007) with PDA and stylus. They also found that scrolling was more efficient than paging, and the results of our study confirm that their findings can also be generalized to touchscreen mobile devices. In terms of user satisfaction, our findings support previous work by Fukaya et al. (2011) as participants also preferred scrolling to other content distribution methods. We used the same device (touchscreen smartphone), but the kind of document was different: we focused on web pages with text while Fukaya et al. work focused on PDF documents. So apparently the type of document does not influence user's satisfaction and in touchscreen mobile devices users also prefer scrolling. Our findings differ from results presented in studies using PC € with keyboard. Oquist and Lundin (2007) and Dyson and Kipping (1998) found paging to be faster (more efficient) than scrolling. However, as in their experiments they used devices with keyboard, we argue that the type of interaction may influence efficiency.
Users perform faster when reading from a touchscreen in mobile device if they use scrolling, but paging is faster for devices with keyboard. In terms of satisfaction, our results differ from Costa et al. (2007) as in our experiment users found scrolling more comfortable, easy to use and efficient than the other content distribution methods while Costa et al. found that links were preferred. They used a PDA with stylus for their experiments and this suggests that the kind of device and type of interaction also influences user's satisfaction about content distribution methods. Therefore, the guideline that we have suggested could also be applied to other mobile devices and also to any web interface that uses a pointing device (i.e. PC with mouse). The new guideline can also be applicable to other types of documents in touchscreen mobile devices. Nevertheless, our results cannot be extended to other devices with different interaction methods like keyboards. We conjecture that in intuitive interfaces, like touchscreens, scrolling may be more usable but with indirect interaction methods, like keyboards, paging is faster. Further research needs to be conducted to corroborate or refute this hypothesis. Finally, in terms of user satisfaction our results differ from other pointing interaction methods even for mobile devices (namely, PDA with stylus). As user satisfaction involves participants' subjective opinion we suggest to be careful when interpreting results as different studies usually employ very different instruments to operationalize this construct. In our opinion, further research is needed to develop standardized methods that can produce comparable results in terms of user satisfaction for mobile interfaces. 5.1. Limitations of the study Our experiment was only carried out on a touchscreen smartphone. Research so far has found that the interaction method impacts efficiency (Shi and Wu, 2015) so results may not be generalized to all types of devices and further research needs to be conducted to evaluate their usability. Similarly, screen size and resolution may influence results since wider screens require less interaction to display the same amount of information and higher resolution may produce more readable fonts that also reduce the amount of interaction needed. However we consider that our main results and the guideline suggested can be generalized across touchscreen mobile devices. As the main factors that can have effect on its applicability are the number of interactions (swipes) required to go through the content and the amount of information displayed on the screen of the device, any change in screen resolution or font size will only impact on the length of the web page that is suitable to be displayed using scrolling for each device. Sample size of our experiment was small and homogenous in statistical terms. Although the well-known 10 ± 2 rule (Hwang and Salvendy, 2010) may provide validity to our results, we also have to acknowledge that most participants were young, possibly digital natives, and therefore we have to be careful in terms of the generality of results. Although this target is important, results may be biased and more representative populations would be required to get a bigger picture.
E. Garcia-Lopez et al. / Applied Ergonomics 50 (2015) 79e86
Finally, the apparatus that was used to record user interactions could have biased results since it may obstruct natural interaction with the smartphone. However we think that after a few moments, participants get familiar with it and it had a minimal, if any, influence since it did not interfere in the view and it was light. In any case, developing less invasive methods to record interaction will be required to abate any threat to validity that this and similar external factors may entail. 6. Conclusion and future work This paper presented an experiment to evaluate the usability of three different content distribution methods for displaying web documents in touchscreen mobile devices. Following the ISO 924111 recommendation, we measured the three dimensions of usability: effectiveness, efficiency and user satisfaction. Participants (N ¼ 23) were asked to find words in average-sized web documents presented on a smartphone. Results suggested that in terms of effectiveness vertical scrolling was on average better than paging and internal links. This means that users made fewer errors when the content was displayed in a web page with vertical scrolling, although the differences found were not statistically significant. In terms of efficiency, findings suggested that when using vertical scrolling users employed less time to read texts in web pages displayed on mobile devices. In terms of satisfaction, users also preferred vertical scrolling over paging or internal links. However, there was one exception in this case as results also suggested that if the content to be displayed was rather large then users preferred paging. In the light of our results and previous studies, we suggest the following guideline for mobile web interfaces: “try to use vertical scrolling in web pages for mobile devices instead of paging or internal links, except when the content is too large, and then paging is recommended”. In our opinion, this guideline is applicable to mobile devices as well as to any device that uses a pointing interaction method. In terms of efficiency, existing literature as well as the results presented here suggest that the new guideline may result in more usable web content. An analysis of existing results for touchscreen mobile devices also suggests that this new guideline can even be applied to other types of contents like documents. In terms of user satisfaction, we have found contrasting results and existing literature is, in our opinion, both scarce and differing. In our study participants pointed that they preferred paging for large web contents, so it would be interesting to further investigate what they mean by large or for what kind of content paging would be preferred. Research community should also strive to develop standardized methods to assess user satisfaction that produce comparable results across platforms and across interaction methods. The guideline that we have introduced here could, in our opinion, be generalized across the multiplicity of touchscreen mobile devices with different screen resolution and font sizes. However, it can also be interesting to study the impact of screen size and even screen resolution because the former may influence the number of interactions to go through the content while the later may influence the readability of the content, therefore both can potentially influence usability. As additional future work we plan to address other mobile devices with different interaction methods (e.g. keyboard, stylus) because existing literature has found that they differ in terms of usability. Acknowledgments The first author thanks the Spanish Government (Ministerio de n, Cultura y Deporte) for a FPU grant (number AP2010Educacio
85
0499). The second author thanks the Junta de Comunidades de Castilla-La Mancha for a FPI grant. References Ahmadi, H., Kong, J., 2012. User-centric adaptation of web information for small screens. J. Vis. Lang. Comput. 23, 13e28. Aliakseyeu, D., Irani, P., Lucero, A., Subramanian, S., 2008. Multi-flick: an evaluation of flick-based scrolling techniques for pen interfaces. In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems. ACM, pp. 1689e1698. Andersen, T.H., 2005. A Simple Movement Time Model for Scrolling, CHI'05 Extended Abstracts on Human Factors in Computing Systems. ACM, pp. 1180e1183. Baker, J.R., 2003. The Impact of Paging vs. Scrolling on Reading Online Text Passages. Usability News 5. Buchanan, G., Farrant, S., Jones, M., Thimbleby, H., Marsden, G., Pazzani, M., 2001. Improving mobile internet usability. In: Proceedings of the 10th International Conference on World Wide Web. ACM, pp. 673e680. Burigat, S., Chittaro, L., Gabrielli, S., 2008. Navigation techniques for small-screen devices: an evaluation on maps and web pages. Int. J. Hum. Comput. Stud. 66, 78e97. Costa, C.J., Silva, J., Aparício, M., 2007. Evaluating web usability using small display devices. In: Proceedings of the 25th Annual ACM International Conference on Design of Communication. ACM, pp. 263e268. Dillon, A., Richardson, J., McKnight, C., 1990. The effects of display size and text splitting on reading lengthy text from screen. Behav. Inform. Technol. 9, 215e227. Dyson, M.C., Kipping, G.J., 1998. The effects of line length and method of movement on patterns of reading from screen. Visible Lang. 32, 150e181. Eslambolchilar, P., Murray-Smith, R., 2008. Control centric approach in designing scrolling and zooming user interfaces. Int. J. Hum. Comput. Stud. 66, 838e856. Fallman, D., Lund, A., Wiberg, M., 2004. Scrollpad: tangible scrolling with mobile devices. In: Proceedings of the 37th Annual Hawaii International Conference on System Sciences. IEEE. Fukaya, T.Y., Ono, S., Minakuchi, M., Nakashima, S., Hayashi, M., Ando, H., 2011. Reading text on a smart phone: scrolling vs. paging: toward designing effective electronic manuals. In: 2011 International Conference on User Science and Engineering (i-USEr). IEEE, pp. 59e63. Garcia-Lopez, E., Garcia-Cabot, A., de-Marcos, L., Hilera, J.-R., 2013. Usability Guidelines for Mobile Devices: Length of Hyperlinks, Human-computer InteractioneINTERACT 2013. Springer, pp. 705e712. Geven, A., Sefelin, R., Tscheligi, M., 2006. Depth and breadth away from the desktop: the optimal information hierarchy for mobile use. In: Proceedings of the 8th Conference on Human-computer Interaction with Mobile Devices and Services. ACM, pp. 157e164. Hedin, B., Lindgren, E., 2007. A comparison of presentation methods for reading on mobile phones. IEEE Distrib. Syst. Online 8. Hinckley, K., Cutrell, E., Bathiche, S., Muss, T., 2002. Quantitative analysis of scrolling techniques. In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems. ACM, pp. 65e72. Hwang, W., Salvendy, G., 2010. Number of people required for usability evaluation: the 10 ± 2 rule. Commun. ACM 53, 130e133. Igarashi, T., Hinckley, K., 2000. Speed-dependent automatic zooming for browsing large documents. In: Proceedings of the 13th Annual ACM Symposium on User Interface Software and Technology. ACM, pp. 139e148. ISO, 1998. Ergonomic Requirements for Office Work with Visual Display Terminals (VDTs) e Part 11: Guidance on Usability. ISO, 2008. Ergonomics of Human-system Interaction e Part 151: Guidance on World Wide Web User Interfaces. Jones, M., Marsden, G., Mohd-Nasir, N., Boone, K., Buchanan, G., 1999. Improving web interaction on small displays. Comput. Netw. 31, 1129e1137. Jones, S., Jones, M., Marsden, G., Patel, D., Cockburn, A., 2005. An evaluation of integrated zooming and scrolling on small screens. Int. J. Hum. Comput. Stud. 63, 271e303. Juola, J.F., Tiritoglu, A., Pleunis, J., 1995. Reading text presented on a small display. Appl. Ergon. 26, 227e229. Kim, L., Albers, M.J., 2001. Web design issues when searching for information in a small screen display. In: Proceedings of the 19th Annual International Conference on Computer Documentation. ACM, pp. 193e200. Klein, C., Bederson, B.B., 2005. Benefits of Animated Scrolling, CHI'05 Extended Abstracts on Human Factors in Computing Systems. ACM, pp. 1965e1968. Lai, C.-C., Wu, C.-F., 2014. Display and device size effects on the usability of mininotebooks (netbooks)/ultraportables as small form-factor mobile PCs. Appl. Ergon. 45, 1106e1115. Lee, K., Bahn, H., 2005. A compressed display technique for 240 320 resolution personal digital assistants (PDAs). IEEE Trans. Consum. Electron. 51, 1268e1272. Mills, C.B., Weldon, L.J., 1987. Reading text from computer screens. ACM Comput. Surv. 19, 329e357. Nielsen, J., 1997. Changes in Web Usability Since 1994. useit.com. http://www. nngroup.com/articles/changes-in-web-usability-since-1994/. Nielsen, J., Loranger, H., 2006. Prioritizing Web Usability. Pearson Education.
86
E. Garcia-Lopez et al. / Applied Ergonomics 50 (2015) 79e86
€ Oquist, G., Lundin, K., 2007. Eye movement study of reading text on a mobile phone using paging, scrolling, leading, and RSVP. In: Proceedings of the 6th International Conference on Mobile and Ubiquitous Multimedia. ACM, pp. 176e183. Parush, A., Yuviler-Gavish, N., 2004. Web navigation structures in cellular phones: the depth/breadth trade-off issue. Int. J. Hum. Comput. Stud. 60, 753e770. Piolat, A., Roussey, J.-Y., Thunin, O., 1997. Effects of screen presentation on text reading and revising. Int. J. Hum. Comput. Stud. 47, 565e589. Rabin, J., McCathieNevile, C., 2008. Mobile Web Best Practices 1.0: Basic Guidelines. W3C. Sahin, A., Alsancak, D., 2011. The effect of hypertexts with different lengths on reading and comprehension skills of the students. Procedia-Social Behav. Sci. 28, 347e354. Sanchez, C.A., Wiley, J., 2009. To scroll or not to scroll: scrolling, working memory capacity, and comprehending complex texts. Hum. Factors: J. Hum. Factors Ergon. Soc. 51, 730e738.
Shi, W.-Z., Wu, F.-G., 2015. An investigation of the performance of novel chorded keyboards in combination with pointing input devices. Appl. Ergon. 46, 1e7. van Nimwegen, C., Pouw, M., Van Oostendorp, H., 1999. The influence of structure and reading-manipulation on usability of hypertexts. Interact. Comput. 12, 7e21. Wang, A.-H., Lai, Y.-Y., Sun, C.-T., 2004. Effect of PDA scrolling-and hierarchy-menu design on users' operating performance. Displays 25, 109e114. €stlund, E., Norlander, T., Archer, T., 2008. The effect of page layout on mental Wa workload: a dual-task experiment. Comput. Hum. Behav. 24, 1229e1245. Wherry, E., 2003. Scroll Ring Performance Evaluation, CHI'03 Extended Abstracts on Human Factors in Computing Systems. ACM, pp. 758e759. Yin, J., Liu, H., 2010. The empirical study of a novel bimanual scrolling technique. In: 2010 IEEE International Conference on Information and Automation (ICIA). IEEE, pp. 110e113. Zhai, S., Smith, B.A., 1999. Multistream input: an experimental study of document scrolling methods. IBM Syst. J. 38, 642e651.