- Email: [email protected]
Effects of button position on a soft keyboard: Muscle activity, touch time, and discomfort in two-thumb text entry
Applied Ergonomics 60 (2017) 282e292
Contents lists available at ScienceDirect
Applied Ergonomics journal homepage: www.elsevier.com/locate/apergo
Effects of button position on a soft keyboard: Muscle activity, touch time, and discomfort in two-thumb text entry Joonho Chang a, Bori Choi b, Amir Tjolleng b, Kihyo Jung b, * a b
Department of Industrial and Manufacturing Engineering, Pennsylvania State University, University Park, PA 16802, USA University of Ulsan, 93 Daehak-ro, Nam-gu, Ulsan, 680-749, Republic of Korea
a r t i c l e i n f o
a b s t r a c t
Article history: Received 17 April 2016 Received in revised form 12 December 2016 Accepted 16 December 2016
Intensive use of the thumbs for text entry on smartphones may contribute to discomfort, pain, or musculoskeletal disorders. This study investigated the effect of twenty-five button positions (5 rows 5 columns) on a soft keyboard for two-thumb entry. Two experiments measured muscle activity, touch time, and discomfort as a function of the button positions. In Phase I, the muscle activities of two intrinsic (abductor pollicis brevis and first dorsal interossei) and two extrinsic (abductor pollicis longus and extensor digitorum communis) muscles associated with thumb motions were observed for ten college students (age: 24.2). In Phase II, touch time and discomfort were measured for 40 college students (age: 23.6). The results demonstrated that the %MVCs of the intrinsic muscles significantly increased when the thumbs flexed and abducted. Also, the button positions near the rest positions of the thumbs resulted in significantly shorter touch times (0.66 s) and lower discomfort ratings (0.70 pt) than their peripheral buttons (0.76 s; 2.29 pt). © 2016 Elsevier Ltd. All rights reserved.
Keywords: Soft keyboard Two-thumb input Smartphone
1. Introduction Frequency of text entry on mobile phones has been increasing with the popularity of mobile networks and messaging services. eMarketer (2015) reported that more than 1.4 billion consumers used mobile phone messaging apps worldwide in 2015, which indicated that 75% of smartphone users used mobile phone messaging apps. Furthermore, it was informed that approximately half of emails were opened on mobile phones (Jordan, 2015) and 52.7% of mobile phone users accessed the internet through their smartphones (Statista, 2016). These statistics imply that text entry on smartphones has become an important communication tool worldwide (Hsiao et al., 2014; Xiong and Muraki, 2014; Park et al., 2015). A soft keyboard (or on-screen keyboard) is widely adapted for text entry on smartphones. A soft keyboard is a graphical keyboard displayed on a touch-screen instead of a conventional hard keyboard (Kim et al., 2014; Ryu et al., 2013; Yin and Su, 2011). This soft keyboard can be hidden when a user wants to utilize a full screen in mobile apps and appear spontaneously when text entry is
* Corresponding author. E-mail addresses: [email protected] (J. Chang), [email protected] (B. Choi), [email protected] (A. Tjolleng), [email protected] (K. Jung). http://dx.doi.org/10.1016/j.apergo.2016.12.008 0003-6870/© 2016 Elsevier Ltd. All rights reserved.
needed (Lee and Zhai, 2009). In addition, the layout of a soft keyboard can be modified according to users' preferences, which improves its usability and the user experience. However, intensive and excessive use of the thumbs while texting may induce discomfort, pain, or musculoskeletal disorders on the thumbs and upper extremities (Berolo et al., 2011; Korpinen and Paakkonen, 2010). Two representative soft keyboards (a telephone keyboard and a QWERTY keyboard) have been popularly used for text entry on smartphones, but they have distinct pros and cons in terms of user experience. A telephone keyboard was inherited from a keypad employed on a traditional wired telephone, and it has been the de facto standard for mobile phones (Silfverberg et al., 2000). Since the telephone keyboard consists of a fewer number of buttons (n ¼ 12) than the number of alphabet letters (n ¼ 26), two or more letters are assigned to each button; consequently, the telephone keyboard layout allows for relatively large buttons which enable users to find and press them easily. However, using a standard telephone keyboard is often inefficient because the arrangement of the letters is unfamiliar to users as well as multiple tapping is required to toggle between letters (Yin and Su, 2011). On the other hand, a QWERTY keyboard on smartphones employs a very similar layout to traditional QWERTY keyboards used for personal computers. It consists of the same number of buttons as the traditional QWERTY
J. Chang et al. / Applied Ergonomics 60 (2017) 282e292
keyboards, so users could feel more familiar with the keyboard because their prior knowledge can be transferred from personal computers to smartphones. However, the button sizes on the QWERTY keyboard are relatively small since more than 30 buttons are assigned to a small space. Therefore, using a QWERTY keyboard on smartphones is prone to cause unintentional touch errors and may require more precise touch interactions (Jung and Jang, 2015). Many researchers have examined the effect of button positions on entire touch-screens of smartphones for single-handed mobile phone use; however, the results are limited in their direct application to soft keyboard designs. Park et al. (2008) and Kim et al. (2011) analyzed touch time and discomfort for buttons which were randomly displayed on an entire touch-screen. Park and Han (2010b) observed input accuracy and pressing patterns of the thumb as a function of different button positions. More recently, Trudeau et al. (2012a, 2012b) investigated kinematic thumb motions on different button positions using a 3D motion capture system. However, these studies may have limited impacts on the design of soft keyboards because the hand grip position to hold an entire touch-screen (called the middle point grip) is different from the grip position required to use the lower part of a touch-screen where soft keyboards are generally placed; therefore, corresponding thumb motions could have different patterns and performance. A limited number of studies have investigated the effect of button positions on a soft keyboard (the lower part of touchscreen); however, they failed to examine a comprehensive effect of various button positions, which could be useful for designing soft keyboards. Jonsson et al. (2011) observed thumb movements and muscle activities during two-thumb text entry on a soft keyboard and found that the extensor digitorum communis (ED) and first dorsal interosseous (FDI) were associated with thumb movements. Choi and Jung (2013) investigated touch discomfort and muscle activity for various button positions (5 5) on a soft keyboard and found that buttons that were closer to the initial position of the thumb produced relatively lower discomfort and muscle activities. Xiong and Muraki (2014) also observed thumb motions for different button positions (2 2) on a soft keyboard and reported that the adduction-abduction movements of the thumb showed better motor performance than its flexion-extension movements. Although the aforementioned studies have scientifically investigated the effect of button positions by considering thumb text entry on soft keyboards, a comprehensive map of muscle activity, touch time, and discomfort for various button positions has not yet been provided for two-thumb text entry. This study analyzed muscle activity, touch time, and discomfort as a function of various button positions on a soft keyboard for twothumb text entry. Two research questions were tested: (1) buttons positions on a soft keyboard significantly affect muscle activities in two-thumb text entry and (2) buttons positions influence significantly to touch performance and discomfort. To test the two research questions, twenty-five buttons displayed on the lower part of a touch-screen were prepared by separating rows (n ¼ 5) and columns (n ¼ 5). In Phase I, Electromyography (EMG) was measured for 10 participants to observe the muscle activity of the thumb during a touch motion. In Phase II, touch time and discomfort were measured for 40 participants under the same experimental conditions. Lastly, EMG, touch time, and discomfort were statistically tested by the rows and columns of buttons.
3.9 years) were recruited for this EMG experiment. They were all right-handed males with normal vision and their average age was 24.2 years (SD: 1.4). No participants reported any musculoskeletal pain or discomfort on their thumbs and upper limbs on the experiment day. They agreed with an informed consent form and were given a description of the study procedures. 2.1.2. Touch-screen device A small touch-screen device (MiMo UM-720S, Mimo monitors, USA) was employed in the experiment. The resolution was 800 480 pixels. Its overall size and touch-screen size were 18 cm (height) 12 cm (width) 2.2 cm (thickness) and 15.2 cm (height) 9.1 cm (width), respectively. The touch-screen device was linked to a desktop computer (OptiPlex 980, Dell, South Korea) in order to display and control an experimental screen on the touch-screen. This study developed an experimental software (Fig. 1) using Visual Basic 6.0 (Microsoft, USA). An experimental screen was divided into two sections: (1) instruction section (upper) and (2) touch section (lower). The instruction section was designed to provide experimental instructions and help participants proceed the present experiment. The touch section consisted of 25 buttons (5 rows 5 columns) and was used for text entry. The touch section was programmed to randomly display one target button among the 25 buttons at a time; then either the letter L (left thumb) or R (right thumb) appeared on the button after counting down numbers from 5 to 0 (pre-signal), which indicated a designated thumb work to use. 2.1.3. EMG measurement EMG data were measured on the abductor pollicis brevis (APB), abductor pollicis longus (APL), first dorsal interossei (FDI), and extensor digitorum communis (ED) of both the hands and forearms
2. Phase I e muscle activity 2.1. Method and materials 2.1.1. Participants Ten college students who experienced smartphone-use (average
283
Fig. 1. An experimental screen used in the experiment.
284
J. Chang et al. / Applied Ergonomics 60 (2017) 282e292
Fig. 2. Example of EMG data processing for a touch action.
which functioned the abduction, extension, and adduction of the thumb and the extension of the fingers, respectively (Gustafsson et al., 2010; Jonsson et al., 2011; Xiong and Muraki, 2014). The Telemyo DTS surface EMG system (Noraxon, USA) was used for EMG measurements using 10 mm diameter disposable EMG electrodes (3M, Korea) with 25 mm inter-electrode spacing e the surface EMG electrodes were placed as recommended by Perotto and Delagi (1994). A sampling rate was set to 1000 Hz, and its bandwidth filters ranged from 10 to 500 Hz. In addition, all raw EMG data were root mean squared (RMS) with window 100 ms and quantified by means. The EMG system and experimental software were synchronized based on an internal time clock of their master computer. The raw EMG data were processed from the beginning to the end of a muscle contraction in a touch action, which includes the movement and press motions of the thumb (Fig. 2); the duration between the beginning and the end of a muscle contraction was detected by the experimental software. Maximum voluntary contraction (MVC) was measured based on the Caldwell protocol (Chaffin et al., 1999) to calculate %MVC. MVCs for each muscle were recorded while the participants were maintaining their maximum thumb forces. For example, the MVC of the APB muscle (abductor) in the right thumb was measured when a participant gradually pressed the left side of his right thumb toward the wall (abduction direction) in static posture. Each participant was asked to maintain his maximum force for 5 s, and the EMG data for 3 s in the middle was employed for the computation of the MVC, as illustrated in Fig. 3.
2.1.4. Experimental design A two-factor within-subject design was used in the Phase I experiment. In the two-factor within-subject design, button rows (5 levels) and columns (6 levels) were employed as independent variables; column 3 had two levels for the left and right thumbs. An interactive thumb for each button was designated as follows: (1) the left thumb was only used for the buttons in the left two columns (columns 1 and 2); (2) the right thumb was only used for the buttons in the right two columns (columns 4 and 5); and (3) both thumbs were used to tap the buttons in the middle column (column 3). Therefore, the number of button alternatives was 30 (left thumb: left 3 columns 5 rows; right thumb: right 3 columns 5 rows). The participants were instructed to sit on a chair and rest their elbows naturally on a desk as shown in Fig. 4. EMG was measured for each button while the participants were holding naturally a touch-screen device (portrait way) using both hands. The participants were required to place their thumbs around the initial positions (near the buttons (row #, column #; 3, 2) for the left thumb
Fig. 3. Example of MVC data processing.
Fig. 4. Experimental posture.
and (3, 4) for the right thumb), before every touching trial. A target button was randomly generated every 5 s on the touch-screen, and the participants were asked to accurately press a target button with a designated thumb in about 1 s (the EMG measurement). Two measurements for each button were made for a reliable analysis. If the experimental software failed to recognize the press, the participants were asked to press the target again. In addition, an instructor monitored whether the participants properly used the designated thumb in touch action; if the press was not completed by the designated thumb, they needed to perform the press again with the designated thumb. 2.1.5. Statistical analysis Statistical analysis was conducted using Minitab v16.0 (Minitab Inc., USA) with a ¼ 0.10 d we used a relatively liberal significant level due to small sample size. To examine the effect of button rows and columns, a two-factor (row and column) within-subject analysis of variance (ANOVA) was conducted on the %MVC of each muscle. The ANOVA tests were performed separately on the left and right hands because the homogeneity of variance assumption
J. Chang et al. / Applied Ergonomics 60 (2017) 282e292
between both the left and right sides was violated on three muscles (APB: Levene's test statistics ¼ 33.6, p < 0.001; FDI: Levene's test statistics ¼ 9.10, p ¼ 0.003; ED: Levene's test statistics ¼ 5.78, p ¼ 0.017). In addition, to test the effect of the individual button position, a one-factor (button position) within-subject ANOVA test for the left and right sides was conducted on the %MVC of each muscle. Lastly, the Tukey tests were conducted as a post-hoc analysis on the significant independent variables and their interactions with the same confidence level. 2.2. Results The analyses demonstrated that EMG values varied significantly only for the intrinsic muscles such as the APB and the FDI (Fig. 5); the %MVC values on the APB were significant for both button rows (left side: F(4, 36) ¼ 13.50, p < 0.001, partial eta2 ¼ 0.61; right side: F(4, 36) ¼ 7.64, p < 0.001, partial eta2 ¼ 0.45) and columns (left side: F(2, 18) ¼ 9.18, p ¼ 0.002, partial eta2 ¼ 0.48; right side: F(2, 18) ¼ 5.26, p ¼ 0.015, partial eta2 ¼ 0.35), but those on the FDI were significant only for button rows (left side: F(4, 36) ¼ 2.14, p ¼ 0.094, partial eta2 ¼ 0.18; right side: F(4, 36) ¼ 2.41, p ¼ 0.065, partial eta2 ¼ 0.21). The %MVC on the APB increased when the thumbs pressed buttons from both the end columns (column 1 for the left side and column 5 for the right side) to the middle column (column 3). The Tukey test for the APB showed that both the end columns were statistically classified into a lower group than the middle column. The %MVC on both the APB and the FDI increased when the thumbs moved from the top row to the bottom row of buttons. The Tukey test showed that the top and bottom rows were statistically categorized into lower and higher groups respectively on both the APB and the FDI. The interactions for APB (left side: F(8, 72) ¼ 0.51, p ¼ 0.845, partial eta2 ¼ 0.03; right side: F(8, 72) ¼ 1.24, p ¼ 0.289, partial eta2 ¼ 0.07) and FDI (left side: F(8, 72) ¼ 1.14, p ¼ 0.348, partial eta2 ¼ 0.07; right side: F(8, 72) ¼ 0.78, p ¼ 0.621, partial eta2 ¼ 0.05) were not significant. The maps of the %MVCs for each muscle as a function of button positions were prepared in Table 1Ae1D. The %MVCs for APB varied significantly (range ¼ 8.7%e29.0%, max/min ratio ¼ 3.3) by button positions (left side: F(14, 126) ¼ 6.15, p < 0.001, partial eta2 ¼ 0.32; right side: F(14, 126) ¼ 5.16, p < 0.001, partial eta2 ¼ 0.28); the Tukey test showed that the buttons in the middle bottom area were statistically classified into a higher group than those in the other positions. However, the %MVCs on the other muscles varied relatively little by button positions (FDI: range ¼ 11.0%e17.3%, max/min ratio ¼ 1.6; APL: range ¼ 8.3%e14.4%, max/min ratio ¼ 1.7; ED: range ¼ 11.4%e18.0%, max/min ratio ¼ 1.6). 3. Phase II e touch time and discomfort rating 3.1. Method and materials 3.1.1. Participants Forty college students (male: 21, female: 19) who experienced smartphone-use (average 4.2 years) were involved in this experiment. All participants were right-handers, and their average age was 23.6 years (SD: 1.6). The participants did not have any pain or discomfort on the thumbs and upper extremities as well as any vision problem on the experimental day. Before the test, the descriptions of the study purpose and procedure were provided for all participants, and they signed an informed consent form. 3.1.2. Touch-screen device Phase II employed the same touch-screen device and experimental software which were used in the Phase I experiment. The experimental software was programmed to automatically record
285
touch time (unit: sec) from the beginning of a touch signal to the moment that the thumb touched a target button; the beginning signal was provided by the letter (L or R) to inform a designated thumb which was displayed on a target button, after counting down numbers from 5 to 0 (pre-signal). Also, the end moment was detected by the experimental software when the thumb reached a target button. 3.1.3. Experimental design A two-factor within-subject design was employed in the Phase II experiment. As in the Phase I experiment, the two-factor withinsubject design employed button rows (5 levels) and columns (6 levels) as independent variables. The two dependent variables (touch time and subjective discomfort) were measured for each button while the participants naturally grasped the touch-screen device; an interactive thumb for each button was designated as in the Phase I experiment. The participants were required to locate their thumbs around the initial positions (near the buttons (3, 2) for the left thumb and (3, 4) for the right thumb), before every touching action. A target button randomly appeared on the touchscreen device, and the participants were asked to tap a target button as accurately and fast as possible. Touch time was automatically recorded by the experimental software, and two measurements for each button were made. A short break (about one minute) was provided, and discomfort ratings for each button were verbally asked using Borg CR-10 scale (Borg, 1998; Kwon et al., 2009; Jung, 2014) which varies between 0 (no discomfort) and 10 (extremely strong discomfort). The experiment was conducted in 4 steps. First, the study purpose and experimental procedure were explained to the participants, and informed consent was obtained. Second, practice trials were provided for the participants to help not only become familiarized to use of the soft keyboard but also find their preferred grip positions and postures. During the practice trials, the participants were allowed to see their touch times and locations on the touchscreen, in order to give touch feedback. Third, a main experiment was performed; touch times and subjective discomfort ratings were collected for all button positions. Lastly, a debriefing was conducted regarding experiment results. 3.1.4. Statistical analysis Statistical analysis was conducted using Minitab v16.0 (Minitab Inc., USA) with a ¼ 0.05. To test the effect of button rows and columns, a two-factor (row and column) within-subject ANOVA was conducted on the results of each dependent variable. Since the homogeneity of variance assumption between the thumbs is valid (touch time: Levene's test statistics ¼ 0.28, p ¼ 0.594; discomfort rating: Levene's test statistics ¼ 0.07, p ¼ 0.793), we did not separately analyzed the data for each thumb. In addition, to investigate the effect of an individual button position, a one-factor (button position) within-subject ANOVA was performed. Lastly, the Tukey tests were conducted as a post-hoc analysis on the significant independent variables and their interactions with the same confidence level. 3.2. Results 3.2.1. Touch time Touch times were significantly different by button rows (F(4, 156) ¼ 5.87, p < 0.001, partial eta2 ¼ 0.13) and columns (F(5, 195) ¼ 5.75, p < 0.001, partial eta2 ¼ 0.13) as shown in Fig. 6. The touch times in the middle rows (rows 2, 3, and 4; 0.71 ± 0.36) were significantly faster than those of the other rows (0.78 ± 0.43). In addition, the touch times in the middle columns (0.69 ± 0.36) for each thumb (left thumb: column 2; right thumb: column 4) were
286
J. Chang et al. / Applied Ergonomics 60 (2017) 282e292
30
SE
25
20 %MVC 15 10 5 0
A
A
1
2
B A
C B
C
c b
c
3 4 Right side
5
b
3 4 Left side
5
a
a
1
2
a
Row number
(a) %MVC of the APB by row 30
SE
25 20 %MVC 15
10 5
C
B
b
b
A
0
a
1
2 Left side
3
3
4 Right side
5
Column number
(b) %MVC of the APB by column 20
SE
15
%MVC 10
5
0
A
A
1
2
B A
C B
3 4 Left side
C
5
a
b a
1
2
b a
b
3 4 Right side
5
b
a
Row number
(c) %MVC of the FDI by row Fig. 5. %MVC of the APB and the FDI by rows and columns (upper- and lower-case letters indicate statistically different groups for the left side and right side of thumb/forearm, respectively).
J. Chang et al. / Applied Ergonomics 60 (2017) 282e292
287
Table 1 %MVC (SE) for various button positions on a soft keyboard (unit: %). A. %MVC (SE) of APB for button positions (unit: %)a Row
Column Left side
1 2 3 4 5
Right side
1
2
3
3
4
5
9.0D (1.3) 8.7D (1.4) 8.8CD (1.2) 12.7CD (1.7) 16.5BD (2.3)
10.2CD (1.6) 12.0CD (2.1) 14.7BD (2.3) 16.5BD (2.4) 18.8AC (2.6)
16.1BD (3.5) 15.2BD (2.7) 18.3AC (3.8) 21.8AB (3.7) 26.5A (3.7)
15.3b-e (3.5) 14.1c-e (2.8) 23.6a-d (4.5) 25.8a-c (4.7) 29.0a (5.5)
15.1b-e (3.5) 10.9d-e (1.8) 14.7b-e (3.5) 22.1a-e (4.5) 27.2a-b (6.2)
10.6e (2.7) 11.8d-e (2.8) 9.9e (2.0) 14.5b-e (3.7) 21.6a-e (5.1)
B. %MVC (SE) of FDI for button positions (unit: %) Row
Column Left side 1
1 2 3 4 5
14.6 11.7 13.0 15.7 15.6
Right side 2
(3.2) (2.7) (3.1) (3.7) (3.8)
11.0 12.0 15.1 14.4 17.3
3 (2.3) (2.4) (3.4) (3.1) (5.0)
12.6 11.9 13.5 16.7 16.9
3 (2.6) (2.7) (3.0) (4.3) (4.3)
12.3 13.4 14.5 14.4 16.1
4 (2.2) (2.4) (2.3) (2.7) (3.3)
15.1 12.1 14.0 15.1 16.9
5 (3.0) (1.8) (3.3) (2.6) (3.2)
12.2 14.7 13.6 14.5 14.7
(1.6) (3.2) (2.1) (2.8) (2.9)
C. %MVC (SE) of APL for button positions (unit: %) Row
Column Left side
1 2 3 4 5
Right side
1
2
3
3
11.0 (2.2) 8.9 (1.9) 8.6 (1.6) 8.3 (1.2) 8.7 (1.8)
9.7 (1.9) 9.0 (1.6) 11.9 (2.9) 9.3 (2.1) 8.8 (1.4)
14.4 (3.5) 9.5 (1.9) 10.0 (1.7) 10.5 (1.7) 11.6 (2.4)
12.0 10.1 10.4 12.3 11.1
(2.3) (1.7) (1.5) (1.7) (1.7)
4
5
13.1 (2.5) 9.5 (1.8) 10.2 (1.5) 11.1 (1.9) 13.8 (3.0)
12.1 (2.7) 9.8 (1.7) 10.7 (2.1) 11.3 (2.2) 11.6 (2.5)
D. %MVC (SE) of ED for button positions (unit: %) Row
Column Left side 1
1 2 3 4 5
17.0 17.5 16.1 18.0 17.0
Right side 2
(2.4) (2.3) (2.0) (2.4) (2.1)
16.7 17.1 16.2 15.2 16.6
3 (2.7) (2.8) (2.3) (1.9) (1.9)
15.3 14.6 14.3 15.6 16.4
3 (2.0) (2.1) (2.0) (1.9) (1.9)
14.3 12.6 13.1 13.2 15.2
4 (1.4) (1.3) (1.5) (1.3) (1.5)
11.6 11.4 13.0 14.6 14.9
5 (1.5) (1.7) (2.0) (2.3) (1.9)
11.6 11.6 13.6 13.2 14.4
(1.1) (1.2) (1.5) (1.6) (1.6)
a Upper- and lower-case letters indicate statistically different groups for the left and right sides of thumb/forearm, respectively. Multiple groups were simultaneously expressed using two letters with a hyphen (e.g., the cell with A-C is belonging to three groups (A, B, and C)). The cells including the group A and a (the worst group) were shaded.
significantly faster than those of the other columns (0.76 ± 0.40). The interaction between rows and columns was not significant (F(20, 780) ¼ 1.51, p ¼ 0.07, partial eta2 ¼ 0.04). The map of the touch times as a function of button positions was prepared in Table 2. Touch times significantly varied from 0.58 to 0.91 (F(29, 1131) ¼ 3.19, p < 0.001, partial eta2 ¼ 0.08), and the max/ min ratio was 1.6. The Tukey test showed that touch times tended to increase when movement distances between the buttons and the initial positions of the thumbs (around (3, 2) for the left thumb and (3, 4) for the right thumb) increased. For example, the touch times (mean ± SD; 0.66 ± 0.32) of the buttons near the initial positions (left thumb: (2, 2), (3, 2), (4, 2); right thumb: (2, 4), (3, 4), (4, 4)) were relatively shorter than those (0.76 ± 0.40) on the peripheral buttons. 3.2.2. Discomfort rating Discomfort ratings were significantly different by button rows (F(5, 195) ¼ 55.3, p < 0.001, partial eta2 ¼ 0.59) and columns (F(4, 156) ¼ 13.55, p < 0.001, partial eta2 ¼ 0.26), as shown in Fig. 7. The discomfort ratings in middle rows (row 2, 3, and 4; 1.44 ± 1.32)
were significantly lower than those of other rows (2.71 ± 1.51). In addition, the discomfort ratings in the middle columns (1.28 ± 1.22) for each thumb (left thumb: column 2, right thumb: column 4) were significantly lower than those of other columns (2.28 ± 1.59). Although the interaction between the button rows and columns was statistically significant (F(20, 780) ¼ 5.27, p < 0.001, partial eta2 ¼ 0.12), the interaction effect was ordinal as illustrated in Fig. 8. The map of the subjective discomfort ratings as a function of the button positions was described in Table 3. The discomfort ratings significantly varied from 0.30 (extremely weak discomfort) to 3.69 (moderate discomfort) (F(29, 1131) ¼ 23.41, p < 0.001), and the max/min ratio was 12.3. The Tukey test showed that discomfort ratings (0.70 ± 0.83) for the buttons (left thumb: (2, 2), (3, 2), (4, 2); right thumb: (2, 4), (3, 4), (4, 4)) near the initial positions of the thumbs were remarkably smaller than those (2.29 ± 1.54) for the peripheral buttons. 4. Discussion Left and right thumbs increased their intrinsic muscle efforts to
288
J. Chang et al. / Applied Ergonomics 60 (2017) 282e292
0.85
SE
0.80 0.75
Touch time (sec)
0.70 0.65 B
A
A
A
B A
2
3 Row number
4
5
0.60 1
(a) Touch time by row. 0.85
SE
0.80
Touch time (sec)
0.75 0.70 0.65
C B
0.60 1
C
B A 2 Left thumb
C B
3 3(L)
A
3 (R) 3
4 Right thumb
C B A 5
Column number
(b) Touch time by column. Fig. 6. Touch times for rows and columns (Letters indicate statistically different groups).
Table 2 Touch times (SE) for various button positions on a soft keyboard (unit: sec).a Row
Column Left thumb 1
1 2 3 4 5
Right thumb 2
AB
0.84 0.71AC 0.70AC 0.74AC 0.79AC
(0.06) (0.04) (0.04) (0.03) (0.05)
3 AB
0.81 (0.05) 0.70BeC (0.04) B, C 0.70 (0.04) 0.65BeC (0.03) 0.72AC (0.03)
3 A
0.91 (0.03) 0.77AC (0.04) 0.74 AC (0.04) 0.74 AC (0.03) 0.83AB (0.04)
4 AB
0.79 (0.05) 0.80AB(0.05) AC 0.70 (0.05) 0.69BeC(0.04) 0.84AB (0.06)
5 AC
0.75 (0.04) 0.58C (0.02) BeC 0.67 (0.05) 0.66BeC (0.03) 0.68BeC (0.06)
0.74AC (0.05) 0.68BeC (0.04) 0.70AC (0.04) 0.81AB (0.05) 0.73AC (0.04)
a Letters indicate statistically different groups, and multiple groups were simultaneously expressed using two letters with a hyphen (e.g., the cell with AeC is belong to three groups (A, B, and C)). The cells including the group A (the slowest group) were shaded.
press buttons when they were abducted and flexed. The %MVC of the APB increased gradually from adduction to abduction movements of the thumbs. Also, the %MVC of the FDI increased gradually from extension to flexion movements of the thumbs. These propensities can be explained from a biomechanical point of view. First, the APB is an agonist muscle of thumb abduction. This means the APB contracts to abduct the thumb in the palmar or radial
direction on the carpometacarpal (CMC) and metacarpophalangeal (MCP) joints (Perotto and Delagi, 1994; Schmidt and Lanz, 2004). For example, when the thumb is abducted, muscle contraction naturally occurs in the APB. Therefore, the %MVC of the APB must increase during thumb abduction, as the present study showed. Second, tapping motions during thumb flexion require more effort from the FDI. When the thumb interphalangeal (IP) and MCP joints
J. Chang et al. / Applied Ergonomics 60 (2017) 282e292
289
4
SE
3
Discomfort 2
1
D
C
B
1
B
A
0 2
3 Row number
4
5
(a) Discomfort by row. 4
SE
3
Discomfort
2
1 B 0 1
A 2 Left thumb
B
B
3
3
A 4 Right thumb
B 5
Column number
(b) Discomfort by column. Fig. 7. Discomfort levels for rows and columns (Letters indicate statistically different groups).
are fully flexed, the thumb adjusts its posture to vertically touch buttons on the touch-screen due to the anatomy of the hand (Park and Han, 2010b). This motion could require more accurate thumb control to correctly press a button which can increase the efforts of the FDI. Xiong and Muraki (2014) proved that the FDI is more involved and is contracted longer when a button is pressed using a fully flexed thumb. The muscle activities of the APB and the FDI showed inconsistent patterns with the results of touch time and discomfort ratings; the APB and the FDI did not demonstrate that the closer to the initial positions of the thumbs the buttons are, the lower %MVC they show. However, this difference is very natural because the APB is a typical thumb abductor (agonist) and the FDI is strongly associated with thumb flexion (Xiong and Muraki, 2014). Therefore, the EMG amplitudes of the APB and the FDI must either increase or decrease monotonously along with button rows or columns in one direction; for example, in the present study, the %MVC of the APB significantly increased when the thumbs moved from button row 1 through 5 and also decreased from button column 3 through both end columns (1 and 5) for each thumb. Also, the %MVC of the FDI significantly increased when the thumbs moved from button row 1 through 5.
Touch time and discomfort increased as the movement distances of the thumbs lengthened away from their initial positions (near (row #, column #; 3, 2) for the left thumb and (3, 4) for the right thumb). Touch times for the buttons near the initial positions were disproportionally shorter than those of the peripheral buttons. Similarly, the discomfort ratings of the buttons near the initial positions were remarkably smaller than those on the peripheral buttons. For example, they both increased when the Euclidean distances between the buttons and the initial positions increased; the correlations between the touch times and the Euclidean distances were 0.75 (p < 0.001) and 0.55 (p ¼ 0.03) for the left and right thumbs, and the correlations between the discomfort ratings and the Euclidean distances were 0.90 (p < 0.001) and 0.91 (p < 0.001) for the left and right thumbs, respectively. In fact, these were expected because the peripheral buttons are located away from the initial positions of the thumbs and therefore generally require not only a larger amount of thumb joint displacements but also a greater perceived effort, which may increase both the transition time and physical/cognitive discomfort. Previous studies also revealed consistent results; Park and Han (2010a) and Kim and Myung (2013) identified positive correlations between the touch time and the movement distance of the thumb. In addition, Choi
290
J. Chang et al. / Applied Ergonomics 60 (2017) 282e292
Fig. 8. Interaction plot of rows and columns on discomfort.
Table 3 Discomfort levels (SE) for various button positions on a soft keyboard.a Row
Column Left thumb
1 2 3 4 5
Right thumb
1
2
3
3
4
5
2.48CG (0.18) 1.80FK (0.14) 1.53HL (0.14) 2.35CH (0.14) 3.69A (0.15)
1.99EI (0.14) 0.78LM (0.09) 0.36M (0.06) 1.03JM (0.11) 2.36CH (0.12)
2.91AE (0.28) 2.04DI (0.24) 1.56GL (0.23) 2.01EI (0.24) 3.03AC (0.25)
2.74BF (0.27) 1.85FK (0.23) 1.39IL (0.21) 1.84FK (0.21) 2.96AD (0.23)
1.95FJ (0.14) 0.74LM (0.08) 0.30M (0.05) 0.96KM (0.11) 2.33CI (0.13)
2.49CG (0.18) 1.74GK (0.14) 1.49HL (0.15) 2.20CI (0.15) 3.60AB (0.15)
a Letters indicate statistically different groups, and multiple groups were simultaneously expressed using two letters with a hyphen (e.g., the cell with A-C is belong to three groups (A, B, and C)). The cells including the group M (the worst group) were shaded.
J. Chang et al. / Applied Ergonomics 60 (2017) 282e292
et al. (2013) reported a positive relationship between the discomfort and the amount of thumb joint motions. The effects of outward (from adduction to abduction) and inward (from extension to flexion) thumb motions were observed in the subjective discomfort ratings. The inward (left thumb: (1, 3), (3, 2), (5, 1); right thumb: (1, 3), (3, 4), (5, 5)) movements (2.27 ± 1.92) of the thumbs had higher discomfort ratings than the outward (left thumb: (1, 1), (3, 2), (5, 3); right thumb: (1, 5), (3, 4), (5, 3)) movements (1.94 ± 1.75; F(1, 39) ¼ 22.8, p < 0.001); especially, the highest discomfort ratings were observed for the left and right bottom buttons ((5, 1) and (5, 5)), which required extremely flexed thumb postures. These propensities were consistent with previous studies for one-thumb text entry (Parhi et al., 2006; Park and Han, 2010a). Plausible reasons have been introduced from the diverse points of view; Karlson et al. (2006) explained that the inward motions (especially flexion) of the thumb require a greater perceived effort. Trudeau et al. (2012a) estimated that some inward actions of the thumb require a larger amount of thumb joint displacement (closer to the range of motion limit). Keir et al. (1996) also informed that during outward motions, the thumb could be easily returned to its initial position due to the motor control and passive joint force in comparison with its inward movements. In sum, the results of the present study could offer a good evidence to support that the inward motions of the thumbs are still unwanted motions for two-thumb text entry. The touch times measured in this study may be affected by two aspects: (1) thumb choice and (2) visual occlusion. First, the participants might hesitate to choose a corresponding interactive thumb before tapping buttons in column 3 although the designated thumb was instructed during the experiments. This corresponds to the findings of the present study where the touch times on column 3 (0.78 s) were relatively slower than those of other peripheral columns such as columns 1 (0.75 s) and 5 (0.72 s) (F(1, 39) ¼ 3.28, p ¼ 0.078). Second, the touch times on the left and right bottom corners might be affected by the visual occlusion due to the thumbs; in the present study, we did not control the issue of the visual occlusion caused by the thumbs, in order to avoid any artificial biases in the experiment data. The touch times on the left bottom ((4, 1), (4, 2), (5, 1), (5, 2)) and right-bottom ((4, 4), (4, 5), (5, 4), (5, 5)) corners were 0.73 and 0.72 s, respectively. In the observation on the results, these values were a little slower than those on the left middle (0.7 s; (3, 1), (3, 2)) and right middle (0.68 s; (3, 4), (3, 5)) area where the visual occlusion did not occur. Although the little differences were found, we cannot scientifically confirm whether the visual occlusion significantly affect the touch time during the study. Considering the results of this study and practicality, the following recommendations can be provided for the design of soft keyboards to improve the motion efficiency of two-thumb entry. First, buttons need to be primarily assigned on the outward (adduction-abduction) trajectory of the thumbs rather than the inward (extension-flexion) trajectory. Second, the most heavily used buttons need to be assigned around the initial (rest) positions of the thumbs. Third, the button assignment on the both bottom corners of touch-screens (left-bottom for the left thumb and rightbottom for the right thumb) needs to be limited to reduce the adverse effects of awkward (flexed) thumb motion and visual occlusion that may degrade touch performance. For example, a Vshape key assignment could be effective for two-thumb text entry. Further consideration is needed to generalize the results obtained in this study. First, an expansion of the experiment including other age groups is necessary. This study employed participants who were in 20s, because they were considered major consumers (95.8%) of smartphones (Lim and Park, 2013). However, smartphone-use is rapidly growing up in other age groups
291
including 30s (90.3%) and 10s (87.1%). It has also been found that age is strongly related to tactile performance and response time (Jung et al., 2011; Stevens, 1992; Verrillo and Gesheider, 1992). Thus, further studies that include other age groups could be useful for observing different touch performances among these groups. Second, this study was conducted with a fixed button/touch-screen size. However, the sizes of buttons and touch-screens are strongly associated with users' touch performance and subjective ratings (Sesto et al., 2013; Trudeau et al., 2012a). Therefore, the effect of different sizes of buttons/touch-screens during two-thumb text entry needs to be investigated in follow-up studies. Third, this study recruited only right-handers, and the touch-screen device was used only in portrait mode. However, the handiness and the orientation of a touch-screen device may affect muscle activity, touch time, and discomfort in text entry. Further experiments to examine how the handiness and touch-screen orientation affect touch performance in two-thumb text entry would be useful for better generalization of the study. 5. Conclusion The present study investigated the effect of various button positions on a soft keyboard for two-thumb entry. Twenty-five soft buttons (5 rows 5 columns) were placed on the lower part of a touch-screen, and muscle activity, touch time, and subjective discomfort were measured as a function of the button positions. The intrinsic muscles (APB and FDI) were more sensitive to touch actions of the thumb than the extrinsic muscles (APL and ED). Touch time and discomfort increased as the movement distances of the thumbs lengthened away from their initial positions. In addition, button pressing at the flexed thumb postures tended to have poorer subjective ratings than the abducted thumb postures. We expect that the study would be helpful in ergonomic design of soft keyboards to improve motion efficiency as well as to reduce physical load on the thumbs. Acknowledgements This work was supported by the National Research Foundation of Korea grant funded by the Korea government (MSIP, NRF2016R1C1B1008150). References Berolo, S., Wells, R.P., Amik III, B.C., 2011. Musculoskeletal symptoms among mobile hand-held device users and their relationship to device use: a preliminary study in a Canadian university population. Appl. Ergon. 42, 371e378. Borg, G., 1998. Borg's Perceived Exertion and Pain Scales. Human Kinetics, Champaign, IL. Chaffin, D.B., Andersson, G.B.J., Martin, B.J., 1999. Occupational Biomechanics, third ed. John Wiley and Sons, New York. Choi, B., Jung, K., 2013. Analysis of perceived discomfort and EMG for touch locations of a soft keyboard. J. Korean Inst. Ind. Eng. 39 (2), 99e104. Choi, B., Park, S., Jung, K., 2013. Analysis of perceived discomfort and EMG for touch locations of a soft keyboard. Int. Conf. Hum. Comput. Interact. 518e522. eMarketer, 2015. Mobile messaging to Reach 1.4 Billion Worldwide in 2015. http:// www.emarketer.com/Article/Mobile-Messaging-Reach-14-Billion-Worldwide2015/1013215. Gustafsson, E., Johnson, P.W., Hagberg, M., 2010. Thumb postures and physical loads during mobile phone useeA comparison of young adults with and without musculoskeletal symptoms. J. Electromyogr. Kinesiol. 20 (1), 127e135. Hsiao, H., We, F., Chen, C., 2014. Design and evaluation of small, linear QWERTY keyboards. Appl. Ergon. 45, 655e662. Jonsson, P., Johnson, P.W., Hagberg, M., Forsman, M., 2011. Thumb joint movement and muscular activity during mobile phone textingeA methodological study. J. Electromyogr. Kinesiol. 21 (2), 363e370. Jordan, J., 2015. 53% of Emails Opened on Mobile; Outlook Opens Decrease 33%. https://litmus.com/blog/53-of-emails-opened-on-mobile-outlook-opensdecrease-33. Jung, K., Jang, J., 2015. Development of a two-step touch method for website navigation on smartphones. Appl. Ergon. 48, 148e153.
292
J. Chang et al. / Applied Ergonomics 60 (2017) 282e292
Jung, K., 2014. Effects of slanted ergonomic mice on task performance and subjective responses. Appl. Ergon. 45, 450e455. Jung, K., Lee, M., Chang, J., You, H., 2011. Evaluation of information presentation methods for a wearable braille display. J. Ergon. Soc. Korea 30 (6), 739e747. Karlson, A., Bederson, B., Contreras-Vidal, J., 2006. Understanding single-handed mobile device interaction. In: Lumsden, J. (Ed.), Handbook of Research on User Interface Design and Evaluation for Mobile Technology. National Research Council of Canada Institute for Information Technology, Hershey, pp. 86e101. Keir, P.J., Wells, R.P., Ranney, D.A., 1996. Passive properties of the forearm musculature with reference to hand and finger postures. Clin. Biomech. 11 (7), 401e409. Kim, J.H., Aulck, L., Bartha, M.C., Harper, C.A., 2014. Differences in typing forces, muscle activity, comfort, and typing performance among virtual, notebook, and desktop keyboards. Appl. Ergon. 45, 1406e1413. Kim, K., Park, J., Kim, G., Son, K., 2011. A Comparative evaluation about force and fatigue on thumb according to input type for repetitive use of mobile phone. J. Biomed. Eng. Res. 32 (4), 312e317. Kim, I., Myung, R., 2013. Hybrid evaluation method for Korean Character input system in mobile phones. Int. J. Ind. Ergon. 43, 137e145. Korpinen, L., Paakkonen, R., 2010. Self-reported use of ICT (Information and communication technology) uptake in 2002 and discomfort amongst Finns aged 45-66. Appl. Ergon. 42, 85e90. Kwon, O., You, H., Jung, K., 2009. Identification of the maximum acceptable frequencies of upper-extremity motions in the sagittal plane. Hum. Factors Ergon. Manuf. 19 (3), 212e222. Lee, S.C., Zhai, S., 2009. The performance of touch screen soft buttons. Proc. SIGCHI Conf. Hum. Factors Comput. Syst. 309e318. Lim, J., Park, J., 2013. Major Changes in the Era of Smartphones. DIGIECO. Parhi, P., Karlson, A.K., Bederson, B.B., 2006. Target size study for one-handed thumb use on small touchscreen devices. Proc. 8th Conf. Human-computer Interact. Mob. Devices Serv. 203e210. Park, Y.S., Han, S.H., 2010a. Touch key design for one-handed thumb interaction with a mobile phone: effects of touch key size and touch key location. Int. J. Ind. Ergon. 40 (1), 68e76. Park, Y.S., Han, S.H., 2010b. One-handed thumb interaction of mobile devices from
the input accuracy perspective. Int. J. Ind. Ergon. 40, 746e756. Park, Y.S., Han, S.H., Park, J., Cho, Y., 2008. Touch key design for target selection on a mobile phone. Proc. 10th Int. Conf. Hum. Comput. Interact. Mob. Devices Serv. 423e426. Park, Y., Heo, H., Lee, K., 2015. Enhanced auditory feedback for Korean touch screen keyboards. Int. J. Hum. Comput. Stud. 73, 1e11. Perotto, A., Delagi, E.F., 1994. Anatomical Guide for the Electromyographer: the Limbs and Trunk, third ed. Charles C Thomas Publisher, Springfield. Ryu, T., Lim, J., Song, J., Yun, M.H., Shin, M., 2013. Performance comparison between the preferred right and preferred left hands in text entry using Qwerty touch keyboard smartphones. Int. J. Ind. Ergon. 43, 400e405. Schmidt, H., Lanz, U., 2004. Surgical Anatomy of the Hand. Thieme Medical Publishers, Thieme. Sesto, M.E., Irwin, C.B., Chen, K.B., Chourasia, A.O., Wiegmann, D.A., 2013. Effect of touch screen button size and spacing on touch characteristics of users with and without disabilities. Hum. Factors 54 (3), 425e436. Silfverberg, M., MacKenzie, I.S., Korhonen, P., 2000. Predicting text entry speed on mobile phones. Proc. Computer-Human Interact. CHI 2000. Statista, 2016. Mobile phone Internet User Penetration Worldwide from 2014 to 2019. http://www.statista.com/statistics/284202/mobile-phone-internet-userpenetration-worldwide/. Stevens, J.C., 1992. Aging and spatial acuity of touch. J. Gerontol. 47 (1), 35e40. Trudeau, M.B., Udtamadilok, T., Karlson, A.K., Dennerlein, J.T., 2012a. Thumb motor performance varies by movement orientation, direction, and device size during single-handed mobile phone use. Hum. Factors 54 (1), 52e59. Trudeau, M.B., Young, J.G., Jindrich, D.L., Dennerlein, J.T., 2012b. Thumb motor performance varies with thumb and wrist posture during single-handed mobile phone use. J. Biomech. 45 (14), 2349e2354. Verrillo, R.T., Gesheider, G.A., 1992. Perception via the sense of touch in tactile aids for the hearing impaired. In: Summers, I.R. (Ed.), Tactile Aids for the Hearing Impaired. Willey, John & Sons. Xiong, J., Muraki, S., 2014. An ergonomics study of thumb movements on smartphone touch screen. Ergonomics 57 (6), 943e955. Yin, P., Su, E., 2011. Cyber Swarm optimization for general keyboard arrangement problem. Int. J. Ind. Ergon. 41, 43e52.
Contents lists available at ScienceDirect
Applied Ergonomics journal homepage: www.elsevier.com/locate/apergo
Effects of button position on a soft keyboard: Muscle activity, touch time, and discomfort in two-thumb text entry Joonho Chang a, Bori Choi b, Amir Tjolleng b, Kihyo Jung b, * a b
Department of Industrial and Manufacturing Engineering, Pennsylvania State University, University Park, PA 16802, USA University of Ulsan, 93 Daehak-ro, Nam-gu, Ulsan, 680-749, Republic of Korea
a r t i c l e i n f o
a b s t r a c t
Article history: Received 17 April 2016 Received in revised form 12 December 2016 Accepted 16 December 2016
Intensive use of the thumbs for text entry on smartphones may contribute to discomfort, pain, or musculoskeletal disorders. This study investigated the effect of twenty-five button positions (5 rows 5 columns) on a soft keyboard for two-thumb entry. Two experiments measured muscle activity, touch time, and discomfort as a function of the button positions. In Phase I, the muscle activities of two intrinsic (abductor pollicis brevis and first dorsal interossei) and two extrinsic (abductor pollicis longus and extensor digitorum communis) muscles associated with thumb motions were observed for ten college students (age: 24.2). In Phase II, touch time and discomfort were measured for 40 college students (age: 23.6). The results demonstrated that the %MVCs of the intrinsic muscles significantly increased when the thumbs flexed and abducted. Also, the button positions near the rest positions of the thumbs resulted in significantly shorter touch times (0.66 s) and lower discomfort ratings (0.70 pt) than their peripheral buttons (0.76 s; 2.29 pt). © 2016 Elsevier Ltd. All rights reserved.
Keywords: Soft keyboard Two-thumb input Smartphone
1. Introduction Frequency of text entry on mobile phones has been increasing with the popularity of mobile networks and messaging services. eMarketer (2015) reported that more than 1.4 billion consumers used mobile phone messaging apps worldwide in 2015, which indicated that 75% of smartphone users used mobile phone messaging apps. Furthermore, it was informed that approximately half of emails were opened on mobile phones (Jordan, 2015) and 52.7% of mobile phone users accessed the internet through their smartphones (Statista, 2016). These statistics imply that text entry on smartphones has become an important communication tool worldwide (Hsiao et al., 2014; Xiong and Muraki, 2014; Park et al., 2015). A soft keyboard (or on-screen keyboard) is widely adapted for text entry on smartphones. A soft keyboard is a graphical keyboard displayed on a touch-screen instead of a conventional hard keyboard (Kim et al., 2014; Ryu et al., 2013; Yin and Su, 2011). This soft keyboard can be hidden when a user wants to utilize a full screen in mobile apps and appear spontaneously when text entry is
* Corresponding author. E-mail addresses: [email protected] (J. Chang), [email protected] (B. Choi), [email protected] (A. Tjolleng), [email protected] (K. Jung). http://dx.doi.org/10.1016/j.apergo.2016.12.008 0003-6870/© 2016 Elsevier Ltd. All rights reserved.
needed (Lee and Zhai, 2009). In addition, the layout of a soft keyboard can be modified according to users' preferences, which improves its usability and the user experience. However, intensive and excessive use of the thumbs while texting may induce discomfort, pain, or musculoskeletal disorders on the thumbs and upper extremities (Berolo et al., 2011; Korpinen and Paakkonen, 2010). Two representative soft keyboards (a telephone keyboard and a QWERTY keyboard) have been popularly used for text entry on smartphones, but they have distinct pros and cons in terms of user experience. A telephone keyboard was inherited from a keypad employed on a traditional wired telephone, and it has been the de facto standard for mobile phones (Silfverberg et al., 2000). Since the telephone keyboard consists of a fewer number of buttons (n ¼ 12) than the number of alphabet letters (n ¼ 26), two or more letters are assigned to each button; consequently, the telephone keyboard layout allows for relatively large buttons which enable users to find and press them easily. However, using a standard telephone keyboard is often inefficient because the arrangement of the letters is unfamiliar to users as well as multiple tapping is required to toggle between letters (Yin and Su, 2011). On the other hand, a QWERTY keyboard on smartphones employs a very similar layout to traditional QWERTY keyboards used for personal computers. It consists of the same number of buttons as the traditional QWERTY
J. Chang et al. / Applied Ergonomics 60 (2017) 282e292
keyboards, so users could feel more familiar with the keyboard because their prior knowledge can be transferred from personal computers to smartphones. However, the button sizes on the QWERTY keyboard are relatively small since more than 30 buttons are assigned to a small space. Therefore, using a QWERTY keyboard on smartphones is prone to cause unintentional touch errors and may require more precise touch interactions (Jung and Jang, 2015). Many researchers have examined the effect of button positions on entire touch-screens of smartphones for single-handed mobile phone use; however, the results are limited in their direct application to soft keyboard designs. Park et al. (2008) and Kim et al. (2011) analyzed touch time and discomfort for buttons which were randomly displayed on an entire touch-screen. Park and Han (2010b) observed input accuracy and pressing patterns of the thumb as a function of different button positions. More recently, Trudeau et al. (2012a, 2012b) investigated kinematic thumb motions on different button positions using a 3D motion capture system. However, these studies may have limited impacts on the design of soft keyboards because the hand grip position to hold an entire touch-screen (called the middle point grip) is different from the grip position required to use the lower part of a touch-screen where soft keyboards are generally placed; therefore, corresponding thumb motions could have different patterns and performance. A limited number of studies have investigated the effect of button positions on a soft keyboard (the lower part of touchscreen); however, they failed to examine a comprehensive effect of various button positions, which could be useful for designing soft keyboards. Jonsson et al. (2011) observed thumb movements and muscle activities during two-thumb text entry on a soft keyboard and found that the extensor digitorum communis (ED) and first dorsal interosseous (FDI) were associated with thumb movements. Choi and Jung (2013) investigated touch discomfort and muscle activity for various button positions (5 5) on a soft keyboard and found that buttons that were closer to the initial position of the thumb produced relatively lower discomfort and muscle activities. Xiong and Muraki (2014) also observed thumb motions for different button positions (2 2) on a soft keyboard and reported that the adduction-abduction movements of the thumb showed better motor performance than its flexion-extension movements. Although the aforementioned studies have scientifically investigated the effect of button positions by considering thumb text entry on soft keyboards, a comprehensive map of muscle activity, touch time, and discomfort for various button positions has not yet been provided for two-thumb text entry. This study analyzed muscle activity, touch time, and discomfort as a function of various button positions on a soft keyboard for twothumb text entry. Two research questions were tested: (1) buttons positions on a soft keyboard significantly affect muscle activities in two-thumb text entry and (2) buttons positions influence significantly to touch performance and discomfort. To test the two research questions, twenty-five buttons displayed on the lower part of a touch-screen were prepared by separating rows (n ¼ 5) and columns (n ¼ 5). In Phase I, Electromyography (EMG) was measured for 10 participants to observe the muscle activity of the thumb during a touch motion. In Phase II, touch time and discomfort were measured for 40 participants under the same experimental conditions. Lastly, EMG, touch time, and discomfort were statistically tested by the rows and columns of buttons.
3.9 years) were recruited for this EMG experiment. They were all right-handed males with normal vision and their average age was 24.2 years (SD: 1.4). No participants reported any musculoskeletal pain or discomfort on their thumbs and upper limbs on the experiment day. They agreed with an informed consent form and were given a description of the study procedures. 2.1.2. Touch-screen device A small touch-screen device (MiMo UM-720S, Mimo monitors, USA) was employed in the experiment. The resolution was 800 480 pixels. Its overall size and touch-screen size were 18 cm (height) 12 cm (width) 2.2 cm (thickness) and 15.2 cm (height) 9.1 cm (width), respectively. The touch-screen device was linked to a desktop computer (OptiPlex 980, Dell, South Korea) in order to display and control an experimental screen on the touch-screen. This study developed an experimental software (Fig. 1) using Visual Basic 6.0 (Microsoft, USA). An experimental screen was divided into two sections: (1) instruction section (upper) and (2) touch section (lower). The instruction section was designed to provide experimental instructions and help participants proceed the present experiment. The touch section consisted of 25 buttons (5 rows 5 columns) and was used for text entry. The touch section was programmed to randomly display one target button among the 25 buttons at a time; then either the letter L (left thumb) or R (right thumb) appeared on the button after counting down numbers from 5 to 0 (pre-signal), which indicated a designated thumb work to use. 2.1.3. EMG measurement EMG data were measured on the abductor pollicis brevis (APB), abductor pollicis longus (APL), first dorsal interossei (FDI), and extensor digitorum communis (ED) of both the hands and forearms
2. Phase I e muscle activity 2.1. Method and materials 2.1.1. Participants Ten college students who experienced smartphone-use (average
283
Fig. 1. An experimental screen used in the experiment.
284
J. Chang et al. / Applied Ergonomics 60 (2017) 282e292
Fig. 2. Example of EMG data processing for a touch action.
which functioned the abduction, extension, and adduction of the thumb and the extension of the fingers, respectively (Gustafsson et al., 2010; Jonsson et al., 2011; Xiong and Muraki, 2014). The Telemyo DTS surface EMG system (Noraxon, USA) was used for EMG measurements using 10 mm diameter disposable EMG electrodes (3M, Korea) with 25 mm inter-electrode spacing e the surface EMG electrodes were placed as recommended by Perotto and Delagi (1994). A sampling rate was set to 1000 Hz, and its bandwidth filters ranged from 10 to 500 Hz. In addition, all raw EMG data were root mean squared (RMS) with window 100 ms and quantified by means. The EMG system and experimental software were synchronized based on an internal time clock of their master computer. The raw EMG data were processed from the beginning to the end of a muscle contraction in a touch action, which includes the movement and press motions of the thumb (Fig. 2); the duration between the beginning and the end of a muscle contraction was detected by the experimental software. Maximum voluntary contraction (MVC) was measured based on the Caldwell protocol (Chaffin et al., 1999) to calculate %MVC. MVCs for each muscle were recorded while the participants were maintaining their maximum thumb forces. For example, the MVC of the APB muscle (abductor) in the right thumb was measured when a participant gradually pressed the left side of his right thumb toward the wall (abduction direction) in static posture. Each participant was asked to maintain his maximum force for 5 s, and the EMG data for 3 s in the middle was employed for the computation of the MVC, as illustrated in Fig. 3.
2.1.4. Experimental design A two-factor within-subject design was used in the Phase I experiment. In the two-factor within-subject design, button rows (5 levels) and columns (6 levels) were employed as independent variables; column 3 had two levels for the left and right thumbs. An interactive thumb for each button was designated as follows: (1) the left thumb was only used for the buttons in the left two columns (columns 1 and 2); (2) the right thumb was only used for the buttons in the right two columns (columns 4 and 5); and (3) both thumbs were used to tap the buttons in the middle column (column 3). Therefore, the number of button alternatives was 30 (left thumb: left 3 columns 5 rows; right thumb: right 3 columns 5 rows). The participants were instructed to sit on a chair and rest their elbows naturally on a desk as shown in Fig. 4. EMG was measured for each button while the participants were holding naturally a touch-screen device (portrait way) using both hands. The participants were required to place their thumbs around the initial positions (near the buttons (row #, column #; 3, 2) for the left thumb
Fig. 3. Example of MVC data processing.
Fig. 4. Experimental posture.
and (3, 4) for the right thumb), before every touching trial. A target button was randomly generated every 5 s on the touch-screen, and the participants were asked to accurately press a target button with a designated thumb in about 1 s (the EMG measurement). Two measurements for each button were made for a reliable analysis. If the experimental software failed to recognize the press, the participants were asked to press the target again. In addition, an instructor monitored whether the participants properly used the designated thumb in touch action; if the press was not completed by the designated thumb, they needed to perform the press again with the designated thumb. 2.1.5. Statistical analysis Statistical analysis was conducted using Minitab v16.0 (Minitab Inc., USA) with a ¼ 0.10 d we used a relatively liberal significant level due to small sample size. To examine the effect of button rows and columns, a two-factor (row and column) within-subject analysis of variance (ANOVA) was conducted on the %MVC of each muscle. The ANOVA tests were performed separately on the left and right hands because the homogeneity of variance assumption
J. Chang et al. / Applied Ergonomics 60 (2017) 282e292
between both the left and right sides was violated on three muscles (APB: Levene's test statistics ¼ 33.6, p < 0.001; FDI: Levene's test statistics ¼ 9.10, p ¼ 0.003; ED: Levene's test statistics ¼ 5.78, p ¼ 0.017). In addition, to test the effect of the individual button position, a one-factor (button position) within-subject ANOVA test for the left and right sides was conducted on the %MVC of each muscle. Lastly, the Tukey tests were conducted as a post-hoc analysis on the significant independent variables and their interactions with the same confidence level. 2.2. Results The analyses demonstrated that EMG values varied significantly only for the intrinsic muscles such as the APB and the FDI (Fig. 5); the %MVC values on the APB were significant for both button rows (left side: F(4, 36) ¼ 13.50, p < 0.001, partial eta2 ¼ 0.61; right side: F(4, 36) ¼ 7.64, p < 0.001, partial eta2 ¼ 0.45) and columns (left side: F(2, 18) ¼ 9.18, p ¼ 0.002, partial eta2 ¼ 0.48; right side: F(2, 18) ¼ 5.26, p ¼ 0.015, partial eta2 ¼ 0.35), but those on the FDI were significant only for button rows (left side: F(4, 36) ¼ 2.14, p ¼ 0.094, partial eta2 ¼ 0.18; right side: F(4, 36) ¼ 2.41, p ¼ 0.065, partial eta2 ¼ 0.21). The %MVC on the APB increased when the thumbs pressed buttons from both the end columns (column 1 for the left side and column 5 for the right side) to the middle column (column 3). The Tukey test for the APB showed that both the end columns were statistically classified into a lower group than the middle column. The %MVC on both the APB and the FDI increased when the thumbs moved from the top row to the bottom row of buttons. The Tukey test showed that the top and bottom rows were statistically categorized into lower and higher groups respectively on both the APB and the FDI. The interactions for APB (left side: F(8, 72) ¼ 0.51, p ¼ 0.845, partial eta2 ¼ 0.03; right side: F(8, 72) ¼ 1.24, p ¼ 0.289, partial eta2 ¼ 0.07) and FDI (left side: F(8, 72) ¼ 1.14, p ¼ 0.348, partial eta2 ¼ 0.07; right side: F(8, 72) ¼ 0.78, p ¼ 0.621, partial eta2 ¼ 0.05) were not significant. The maps of the %MVCs for each muscle as a function of button positions were prepared in Table 1Ae1D. The %MVCs for APB varied significantly (range ¼ 8.7%e29.0%, max/min ratio ¼ 3.3) by button positions (left side: F(14, 126) ¼ 6.15, p < 0.001, partial eta2 ¼ 0.32; right side: F(14, 126) ¼ 5.16, p < 0.001, partial eta2 ¼ 0.28); the Tukey test showed that the buttons in the middle bottom area were statistically classified into a higher group than those in the other positions. However, the %MVCs on the other muscles varied relatively little by button positions (FDI: range ¼ 11.0%e17.3%, max/min ratio ¼ 1.6; APL: range ¼ 8.3%e14.4%, max/min ratio ¼ 1.7; ED: range ¼ 11.4%e18.0%, max/min ratio ¼ 1.6). 3. Phase II e touch time and discomfort rating 3.1. Method and materials 3.1.1. Participants Forty college students (male: 21, female: 19) who experienced smartphone-use (average 4.2 years) were involved in this experiment. All participants were right-handers, and their average age was 23.6 years (SD: 1.6). The participants did not have any pain or discomfort on the thumbs and upper extremities as well as any vision problem on the experimental day. Before the test, the descriptions of the study purpose and procedure were provided for all participants, and they signed an informed consent form. 3.1.2. Touch-screen device Phase II employed the same touch-screen device and experimental software which were used in the Phase I experiment. The experimental software was programmed to automatically record
285
touch time (unit: sec) from the beginning of a touch signal to the moment that the thumb touched a target button; the beginning signal was provided by the letter (L or R) to inform a designated thumb which was displayed on a target button, after counting down numbers from 5 to 0 (pre-signal). Also, the end moment was detected by the experimental software when the thumb reached a target button. 3.1.3. Experimental design A two-factor within-subject design was employed in the Phase II experiment. As in the Phase I experiment, the two-factor withinsubject design employed button rows (5 levels) and columns (6 levels) as independent variables. The two dependent variables (touch time and subjective discomfort) were measured for each button while the participants naturally grasped the touch-screen device; an interactive thumb for each button was designated as in the Phase I experiment. The participants were required to locate their thumbs around the initial positions (near the buttons (3, 2) for the left thumb and (3, 4) for the right thumb), before every touching action. A target button randomly appeared on the touchscreen device, and the participants were asked to tap a target button as accurately and fast as possible. Touch time was automatically recorded by the experimental software, and two measurements for each button were made. A short break (about one minute) was provided, and discomfort ratings for each button were verbally asked using Borg CR-10 scale (Borg, 1998; Kwon et al., 2009; Jung, 2014) which varies between 0 (no discomfort) and 10 (extremely strong discomfort). The experiment was conducted in 4 steps. First, the study purpose and experimental procedure were explained to the participants, and informed consent was obtained. Second, practice trials were provided for the participants to help not only become familiarized to use of the soft keyboard but also find their preferred grip positions and postures. During the practice trials, the participants were allowed to see their touch times and locations on the touchscreen, in order to give touch feedback. Third, a main experiment was performed; touch times and subjective discomfort ratings were collected for all button positions. Lastly, a debriefing was conducted regarding experiment results. 3.1.4. Statistical analysis Statistical analysis was conducted using Minitab v16.0 (Minitab Inc., USA) with a ¼ 0.05. To test the effect of button rows and columns, a two-factor (row and column) within-subject ANOVA was conducted on the results of each dependent variable. Since the homogeneity of variance assumption between the thumbs is valid (touch time: Levene's test statistics ¼ 0.28, p ¼ 0.594; discomfort rating: Levene's test statistics ¼ 0.07, p ¼ 0.793), we did not separately analyzed the data for each thumb. In addition, to investigate the effect of an individual button position, a one-factor (button position) within-subject ANOVA was performed. Lastly, the Tukey tests were conducted as a post-hoc analysis on the significant independent variables and their interactions with the same confidence level. 3.2. Results 3.2.1. Touch time Touch times were significantly different by button rows (F(4, 156) ¼ 5.87, p < 0.001, partial eta2 ¼ 0.13) and columns (F(5, 195) ¼ 5.75, p < 0.001, partial eta2 ¼ 0.13) as shown in Fig. 6. The touch times in the middle rows (rows 2, 3, and 4; 0.71 ± 0.36) were significantly faster than those of the other rows (0.78 ± 0.43). In addition, the touch times in the middle columns (0.69 ± 0.36) for each thumb (left thumb: column 2; right thumb: column 4) were
286
J. Chang et al. / Applied Ergonomics 60 (2017) 282e292
30
SE
25
20 %MVC 15 10 5 0
A
A
1
2
B A
C B
C
c b
c
3 4 Right side
5
b
3 4 Left side
5
a
a
1
2
a
Row number
(a) %MVC of the APB by row 30
SE
25 20 %MVC 15
10 5
C
B
b
b
A
0
a
1
2 Left side
3
3
4 Right side
5
Column number
(b) %MVC of the APB by column 20
SE
15
%MVC 10
5
0
A
A
1
2
B A
C B
3 4 Left side
C
5
a
b a
1
2
b a
b
3 4 Right side
5
b
a
Row number
(c) %MVC of the FDI by row Fig. 5. %MVC of the APB and the FDI by rows and columns (upper- and lower-case letters indicate statistically different groups for the left side and right side of thumb/forearm, respectively).
J. Chang et al. / Applied Ergonomics 60 (2017) 282e292
287
Table 1 %MVC (SE) for various button positions on a soft keyboard (unit: %). A. %MVC (SE) of APB for button positions (unit: %)a Row
Column Left side
1 2 3 4 5
Right side
1
2
3
3
4
5
9.0D (1.3) 8.7D (1.4) 8.8CD (1.2) 12.7CD (1.7) 16.5BD (2.3)
10.2CD (1.6) 12.0CD (2.1) 14.7BD (2.3) 16.5BD (2.4) 18.8AC (2.6)
16.1BD (3.5) 15.2BD (2.7) 18.3AC (3.8) 21.8AB (3.7) 26.5A (3.7)
15.3b-e (3.5) 14.1c-e (2.8) 23.6a-d (4.5) 25.8a-c (4.7) 29.0a (5.5)
15.1b-e (3.5) 10.9d-e (1.8) 14.7b-e (3.5) 22.1a-e (4.5) 27.2a-b (6.2)
10.6e (2.7) 11.8d-e (2.8) 9.9e (2.0) 14.5b-e (3.7) 21.6a-e (5.1)
B. %MVC (SE) of FDI for button positions (unit: %) Row
Column Left side 1
1 2 3 4 5
14.6 11.7 13.0 15.7 15.6
Right side 2
(3.2) (2.7) (3.1) (3.7) (3.8)
11.0 12.0 15.1 14.4 17.3
3 (2.3) (2.4) (3.4) (3.1) (5.0)
12.6 11.9 13.5 16.7 16.9
3 (2.6) (2.7) (3.0) (4.3) (4.3)
12.3 13.4 14.5 14.4 16.1
4 (2.2) (2.4) (2.3) (2.7) (3.3)
15.1 12.1 14.0 15.1 16.9
5 (3.0) (1.8) (3.3) (2.6) (3.2)
12.2 14.7 13.6 14.5 14.7
(1.6) (3.2) (2.1) (2.8) (2.9)
C. %MVC (SE) of APL for button positions (unit: %) Row
Column Left side
1 2 3 4 5
Right side
1
2
3
3
11.0 (2.2) 8.9 (1.9) 8.6 (1.6) 8.3 (1.2) 8.7 (1.8)
9.7 (1.9) 9.0 (1.6) 11.9 (2.9) 9.3 (2.1) 8.8 (1.4)
14.4 (3.5) 9.5 (1.9) 10.0 (1.7) 10.5 (1.7) 11.6 (2.4)
12.0 10.1 10.4 12.3 11.1
(2.3) (1.7) (1.5) (1.7) (1.7)
4
5
13.1 (2.5) 9.5 (1.8) 10.2 (1.5) 11.1 (1.9) 13.8 (3.0)
12.1 (2.7) 9.8 (1.7) 10.7 (2.1) 11.3 (2.2) 11.6 (2.5)
D. %MVC (SE) of ED for button positions (unit: %) Row
Column Left side 1
1 2 3 4 5
17.0 17.5 16.1 18.0 17.0
Right side 2
(2.4) (2.3) (2.0) (2.4) (2.1)
16.7 17.1 16.2 15.2 16.6
3 (2.7) (2.8) (2.3) (1.9) (1.9)
15.3 14.6 14.3 15.6 16.4
3 (2.0) (2.1) (2.0) (1.9) (1.9)
14.3 12.6 13.1 13.2 15.2
4 (1.4) (1.3) (1.5) (1.3) (1.5)
11.6 11.4 13.0 14.6 14.9
5 (1.5) (1.7) (2.0) (2.3) (1.9)
11.6 11.6 13.6 13.2 14.4
(1.1) (1.2) (1.5) (1.6) (1.6)
a Upper- and lower-case letters indicate statistically different groups for the left and right sides of thumb/forearm, respectively. Multiple groups were simultaneously expressed using two letters with a hyphen (e.g., the cell with A-C is belonging to three groups (A, B, and C)). The cells including the group A and a (the worst group) were shaded.
significantly faster than those of the other columns (0.76 ± 0.40). The interaction between rows and columns was not significant (F(20, 780) ¼ 1.51, p ¼ 0.07, partial eta2 ¼ 0.04). The map of the touch times as a function of button positions was prepared in Table 2. Touch times significantly varied from 0.58 to 0.91 (F(29, 1131) ¼ 3.19, p < 0.001, partial eta2 ¼ 0.08), and the max/ min ratio was 1.6. The Tukey test showed that touch times tended to increase when movement distances between the buttons and the initial positions of the thumbs (around (3, 2) for the left thumb and (3, 4) for the right thumb) increased. For example, the touch times (mean ± SD; 0.66 ± 0.32) of the buttons near the initial positions (left thumb: (2, 2), (3, 2), (4, 2); right thumb: (2, 4), (3, 4), (4, 4)) were relatively shorter than those (0.76 ± 0.40) on the peripheral buttons. 3.2.2. Discomfort rating Discomfort ratings were significantly different by button rows (F(5, 195) ¼ 55.3, p < 0.001, partial eta2 ¼ 0.59) and columns (F(4, 156) ¼ 13.55, p < 0.001, partial eta2 ¼ 0.26), as shown in Fig. 7. The discomfort ratings in middle rows (row 2, 3, and 4; 1.44 ± 1.32)
were significantly lower than those of other rows (2.71 ± 1.51). In addition, the discomfort ratings in the middle columns (1.28 ± 1.22) for each thumb (left thumb: column 2, right thumb: column 4) were significantly lower than those of other columns (2.28 ± 1.59). Although the interaction between the button rows and columns was statistically significant (F(20, 780) ¼ 5.27, p < 0.001, partial eta2 ¼ 0.12), the interaction effect was ordinal as illustrated in Fig. 8. The map of the subjective discomfort ratings as a function of the button positions was described in Table 3. The discomfort ratings significantly varied from 0.30 (extremely weak discomfort) to 3.69 (moderate discomfort) (F(29, 1131) ¼ 23.41, p < 0.001), and the max/min ratio was 12.3. The Tukey test showed that discomfort ratings (0.70 ± 0.83) for the buttons (left thumb: (2, 2), (3, 2), (4, 2); right thumb: (2, 4), (3, 4), (4, 4)) near the initial positions of the thumbs were remarkably smaller than those (2.29 ± 1.54) for the peripheral buttons. 4. Discussion Left and right thumbs increased their intrinsic muscle efforts to
288
J. Chang et al. / Applied Ergonomics 60 (2017) 282e292
0.85
SE
0.80 0.75
Touch time (sec)
0.70 0.65 B
A
A
A
B A
2
3 Row number
4
5
0.60 1
(a) Touch time by row. 0.85
SE
0.80
Touch time (sec)
0.75 0.70 0.65
C B
0.60 1
C
B A 2 Left thumb
C B
3 3(L)
A
3 (R) 3
4 Right thumb
C B A 5
Column number
(b) Touch time by column. Fig. 6. Touch times for rows and columns (Letters indicate statistically different groups).
Table 2 Touch times (SE) for various button positions on a soft keyboard (unit: sec).a Row
Column Left thumb 1
1 2 3 4 5
Right thumb 2
AB
0.84 0.71AC 0.70AC 0.74AC 0.79AC
(0.06) (0.04) (0.04) (0.03) (0.05)
3 AB
0.81 (0.05) 0.70BeC (0.04) B, C 0.70 (0.04) 0.65BeC (0.03) 0.72AC (0.03)
3 A
0.91 (0.03) 0.77AC (0.04) 0.74 AC (0.04) 0.74 AC (0.03) 0.83AB (0.04)
4 AB
0.79 (0.05) 0.80AB(0.05) AC 0.70 (0.05) 0.69BeC(0.04) 0.84AB (0.06)
5 AC
0.75 (0.04) 0.58C (0.02) BeC 0.67 (0.05) 0.66BeC (0.03) 0.68BeC (0.06)
0.74AC (0.05) 0.68BeC (0.04) 0.70AC (0.04) 0.81AB (0.05) 0.73AC (0.04)
a Letters indicate statistically different groups, and multiple groups were simultaneously expressed using two letters with a hyphen (e.g., the cell with AeC is belong to three groups (A, B, and C)). The cells including the group A (the slowest group) were shaded.
press buttons when they were abducted and flexed. The %MVC of the APB increased gradually from adduction to abduction movements of the thumbs. Also, the %MVC of the FDI increased gradually from extension to flexion movements of the thumbs. These propensities can be explained from a biomechanical point of view. First, the APB is an agonist muscle of thumb abduction. This means the APB contracts to abduct the thumb in the palmar or radial
direction on the carpometacarpal (CMC) and metacarpophalangeal (MCP) joints (Perotto and Delagi, 1994; Schmidt and Lanz, 2004). For example, when the thumb is abducted, muscle contraction naturally occurs in the APB. Therefore, the %MVC of the APB must increase during thumb abduction, as the present study showed. Second, tapping motions during thumb flexion require more effort from the FDI. When the thumb interphalangeal (IP) and MCP joints
J. Chang et al. / Applied Ergonomics 60 (2017) 282e292
289
4
SE
3
Discomfort 2
1
D
C
B
1
B
A
0 2
3 Row number
4
5
(a) Discomfort by row. 4
SE
3
Discomfort
2
1 B 0 1
A 2 Left thumb
B
B
3
3
A 4 Right thumb
B 5
Column number
(b) Discomfort by column. Fig. 7. Discomfort levels for rows and columns (Letters indicate statistically different groups).
are fully flexed, the thumb adjusts its posture to vertically touch buttons on the touch-screen due to the anatomy of the hand (Park and Han, 2010b). This motion could require more accurate thumb control to correctly press a button which can increase the efforts of the FDI. Xiong and Muraki (2014) proved that the FDI is more involved and is contracted longer when a button is pressed using a fully flexed thumb. The muscle activities of the APB and the FDI showed inconsistent patterns with the results of touch time and discomfort ratings; the APB and the FDI did not demonstrate that the closer to the initial positions of the thumbs the buttons are, the lower %MVC they show. However, this difference is very natural because the APB is a typical thumb abductor (agonist) and the FDI is strongly associated with thumb flexion (Xiong and Muraki, 2014). Therefore, the EMG amplitudes of the APB and the FDI must either increase or decrease monotonously along with button rows or columns in one direction; for example, in the present study, the %MVC of the APB significantly increased when the thumbs moved from button row 1 through 5 and also decreased from button column 3 through both end columns (1 and 5) for each thumb. Also, the %MVC of the FDI significantly increased when the thumbs moved from button row 1 through 5.
Touch time and discomfort increased as the movement distances of the thumbs lengthened away from their initial positions (near (row #, column #; 3, 2) for the left thumb and (3, 4) for the right thumb). Touch times for the buttons near the initial positions were disproportionally shorter than those of the peripheral buttons. Similarly, the discomfort ratings of the buttons near the initial positions were remarkably smaller than those on the peripheral buttons. For example, they both increased when the Euclidean distances between the buttons and the initial positions increased; the correlations between the touch times and the Euclidean distances were 0.75 (p < 0.001) and 0.55 (p ¼ 0.03) for the left and right thumbs, and the correlations between the discomfort ratings and the Euclidean distances were 0.90 (p < 0.001) and 0.91 (p < 0.001) for the left and right thumbs, respectively. In fact, these were expected because the peripheral buttons are located away from the initial positions of the thumbs and therefore generally require not only a larger amount of thumb joint displacements but also a greater perceived effort, which may increase both the transition time and physical/cognitive discomfort. Previous studies also revealed consistent results; Park and Han (2010a) and Kim and Myung (2013) identified positive correlations between the touch time and the movement distance of the thumb. In addition, Choi
290
J. Chang et al. / Applied Ergonomics 60 (2017) 282e292
Fig. 8. Interaction plot of rows and columns on discomfort.
Table 3 Discomfort levels (SE) for various button positions on a soft keyboard.a Row
Column Left thumb
1 2 3 4 5
Right thumb
1
2
3
3
4
5
2.48CG (0.18) 1.80FK (0.14) 1.53HL (0.14) 2.35CH (0.14) 3.69A (0.15)
1.99EI (0.14) 0.78LM (0.09) 0.36M (0.06) 1.03JM (0.11) 2.36CH (0.12)
2.91AE (0.28) 2.04DI (0.24) 1.56GL (0.23) 2.01EI (0.24) 3.03AC (0.25)
2.74BF (0.27) 1.85FK (0.23) 1.39IL (0.21) 1.84FK (0.21) 2.96AD (0.23)
1.95FJ (0.14) 0.74LM (0.08) 0.30M (0.05) 0.96KM (0.11) 2.33CI (0.13)
2.49CG (0.18) 1.74GK (0.14) 1.49HL (0.15) 2.20CI (0.15) 3.60AB (0.15)
a Letters indicate statistically different groups, and multiple groups were simultaneously expressed using two letters with a hyphen (e.g., the cell with A-C is belong to three groups (A, B, and C)). The cells including the group M (the worst group) were shaded.
J. Chang et al. / Applied Ergonomics 60 (2017) 282e292
et al. (2013) reported a positive relationship between the discomfort and the amount of thumb joint motions. The effects of outward (from adduction to abduction) and inward (from extension to flexion) thumb motions were observed in the subjective discomfort ratings. The inward (left thumb: (1, 3), (3, 2), (5, 1); right thumb: (1, 3), (3, 4), (5, 5)) movements (2.27 ± 1.92) of the thumbs had higher discomfort ratings than the outward (left thumb: (1, 1), (3, 2), (5, 3); right thumb: (1, 5), (3, 4), (5, 3)) movements (1.94 ± 1.75; F(1, 39) ¼ 22.8, p < 0.001); especially, the highest discomfort ratings were observed for the left and right bottom buttons ((5, 1) and (5, 5)), which required extremely flexed thumb postures. These propensities were consistent with previous studies for one-thumb text entry (Parhi et al., 2006; Park and Han, 2010a). Plausible reasons have been introduced from the diverse points of view; Karlson et al. (2006) explained that the inward motions (especially flexion) of the thumb require a greater perceived effort. Trudeau et al. (2012a) estimated that some inward actions of the thumb require a larger amount of thumb joint displacement (closer to the range of motion limit). Keir et al. (1996) also informed that during outward motions, the thumb could be easily returned to its initial position due to the motor control and passive joint force in comparison with its inward movements. In sum, the results of the present study could offer a good evidence to support that the inward motions of the thumbs are still unwanted motions for two-thumb text entry. The touch times measured in this study may be affected by two aspects: (1) thumb choice and (2) visual occlusion. First, the participants might hesitate to choose a corresponding interactive thumb before tapping buttons in column 3 although the designated thumb was instructed during the experiments. This corresponds to the findings of the present study where the touch times on column 3 (0.78 s) were relatively slower than those of other peripheral columns such as columns 1 (0.75 s) and 5 (0.72 s) (F(1, 39) ¼ 3.28, p ¼ 0.078). Second, the touch times on the left and right bottom corners might be affected by the visual occlusion due to the thumbs; in the present study, we did not control the issue of the visual occlusion caused by the thumbs, in order to avoid any artificial biases in the experiment data. The touch times on the left bottom ((4, 1), (4, 2), (5, 1), (5, 2)) and right-bottom ((4, 4), (4, 5), (5, 4), (5, 5)) corners were 0.73 and 0.72 s, respectively. In the observation on the results, these values were a little slower than those on the left middle (0.7 s; (3, 1), (3, 2)) and right middle (0.68 s; (3, 4), (3, 5)) area where the visual occlusion did not occur. Although the little differences were found, we cannot scientifically confirm whether the visual occlusion significantly affect the touch time during the study. Considering the results of this study and practicality, the following recommendations can be provided for the design of soft keyboards to improve the motion efficiency of two-thumb entry. First, buttons need to be primarily assigned on the outward (adduction-abduction) trajectory of the thumbs rather than the inward (extension-flexion) trajectory. Second, the most heavily used buttons need to be assigned around the initial (rest) positions of the thumbs. Third, the button assignment on the both bottom corners of touch-screens (left-bottom for the left thumb and rightbottom for the right thumb) needs to be limited to reduce the adverse effects of awkward (flexed) thumb motion and visual occlusion that may degrade touch performance. For example, a Vshape key assignment could be effective for two-thumb text entry. Further consideration is needed to generalize the results obtained in this study. First, an expansion of the experiment including other age groups is necessary. This study employed participants who were in 20s, because they were considered major consumers (95.8%) of smartphones (Lim and Park, 2013). However, smartphone-use is rapidly growing up in other age groups
291
including 30s (90.3%) and 10s (87.1%). It has also been found that age is strongly related to tactile performance and response time (Jung et al., 2011; Stevens, 1992; Verrillo and Gesheider, 1992). Thus, further studies that include other age groups could be useful for observing different touch performances among these groups. Second, this study was conducted with a fixed button/touch-screen size. However, the sizes of buttons and touch-screens are strongly associated with users' touch performance and subjective ratings (Sesto et al., 2013; Trudeau et al., 2012a). Therefore, the effect of different sizes of buttons/touch-screens during two-thumb text entry needs to be investigated in follow-up studies. Third, this study recruited only right-handers, and the touch-screen device was used only in portrait mode. However, the handiness and the orientation of a touch-screen device may affect muscle activity, touch time, and discomfort in text entry. Further experiments to examine how the handiness and touch-screen orientation affect touch performance in two-thumb text entry would be useful for better generalization of the study. 5. Conclusion The present study investigated the effect of various button positions on a soft keyboard for two-thumb entry. Twenty-five soft buttons (5 rows 5 columns) were placed on the lower part of a touch-screen, and muscle activity, touch time, and subjective discomfort were measured as a function of the button positions. The intrinsic muscles (APB and FDI) were more sensitive to touch actions of the thumb than the extrinsic muscles (APL and ED). Touch time and discomfort increased as the movement distances of the thumbs lengthened away from their initial positions. In addition, button pressing at the flexed thumb postures tended to have poorer subjective ratings than the abducted thumb postures. We expect that the study would be helpful in ergonomic design of soft keyboards to improve motion efficiency as well as to reduce physical load on the thumbs. Acknowledgements This work was supported by the National Research Foundation of Korea grant funded by the Korea government (MSIP, NRF2016R1C1B1008150). References Berolo, S., Wells, R.P., Amik III, B.C., 2011. Musculoskeletal symptoms among mobile hand-held device users and their relationship to device use: a preliminary study in a Canadian university population. Appl. Ergon. 42, 371e378. Borg, G., 1998. Borg's Perceived Exertion and Pain Scales. Human Kinetics, Champaign, IL. Chaffin, D.B., Andersson, G.B.J., Martin, B.J., 1999. Occupational Biomechanics, third ed. John Wiley and Sons, New York. Choi, B., Jung, K., 2013. Analysis of perceived discomfort and EMG for touch locations of a soft keyboard. J. Korean Inst. Ind. Eng. 39 (2), 99e104. Choi, B., Park, S., Jung, K., 2013. Analysis of perceived discomfort and EMG for touch locations of a soft keyboard. Int. Conf. Hum. Comput. Interact. 518e522. eMarketer, 2015. Mobile messaging to Reach 1.4 Billion Worldwide in 2015. http:// www.emarketer.com/Article/Mobile-Messaging-Reach-14-Billion-Worldwide2015/1013215. Gustafsson, E., Johnson, P.W., Hagberg, M., 2010. Thumb postures and physical loads during mobile phone useeA comparison of young adults with and without musculoskeletal symptoms. J. Electromyogr. Kinesiol. 20 (1), 127e135. Hsiao, H., We, F., Chen, C., 2014. Design and evaluation of small, linear QWERTY keyboards. Appl. Ergon. 45, 655e662. Jonsson, P., Johnson, P.W., Hagberg, M., Forsman, M., 2011. Thumb joint movement and muscular activity during mobile phone textingeA methodological study. J. Electromyogr. Kinesiol. 21 (2), 363e370. Jordan, J., 2015. 53% of Emails Opened on Mobile; Outlook Opens Decrease 33%. https://litmus.com/blog/53-of-emails-opened-on-mobile-outlook-opensdecrease-33. Jung, K., Jang, J., 2015. Development of a two-step touch method for website navigation on smartphones. Appl. Ergon. 48, 148e153.
292
J. Chang et al. / Applied Ergonomics 60 (2017) 282e292
Jung, K., 2014. Effects of slanted ergonomic mice on task performance and subjective responses. Appl. Ergon. 45, 450e455. Jung, K., Lee, M., Chang, J., You, H., 2011. Evaluation of information presentation methods for a wearable braille display. J. Ergon. Soc. Korea 30 (6), 739e747. Karlson, A., Bederson, B., Contreras-Vidal, J., 2006. Understanding single-handed mobile device interaction. In: Lumsden, J. (Ed.), Handbook of Research on User Interface Design and Evaluation for Mobile Technology. National Research Council of Canada Institute for Information Technology, Hershey, pp. 86e101. Keir, P.J., Wells, R.P., Ranney, D.A., 1996. Passive properties of the forearm musculature with reference to hand and finger postures. Clin. Biomech. 11 (7), 401e409. Kim, J.H., Aulck, L., Bartha, M.C., Harper, C.A., 2014. Differences in typing forces, muscle activity, comfort, and typing performance among virtual, notebook, and desktop keyboards. Appl. Ergon. 45, 1406e1413. Kim, K., Park, J., Kim, G., Son, K., 2011. A Comparative evaluation about force and fatigue on thumb according to input type for repetitive use of mobile phone. J. Biomed. Eng. Res. 32 (4), 312e317. Kim, I., Myung, R., 2013. Hybrid evaluation method for Korean Character input system in mobile phones. Int. J. Ind. Ergon. 43, 137e145. Korpinen, L., Paakkonen, R., 2010. Self-reported use of ICT (Information and communication technology) uptake in 2002 and discomfort amongst Finns aged 45-66. Appl. Ergon. 42, 85e90. Kwon, O., You, H., Jung, K., 2009. Identification of the maximum acceptable frequencies of upper-extremity motions in the sagittal plane. Hum. Factors Ergon. Manuf. 19 (3), 212e222. Lee, S.C., Zhai, S., 2009. The performance of touch screen soft buttons. Proc. SIGCHI Conf. Hum. Factors Comput. Syst. 309e318. Lim, J., Park, J., 2013. Major Changes in the Era of Smartphones. DIGIECO. Parhi, P., Karlson, A.K., Bederson, B.B., 2006. Target size study for one-handed thumb use on small touchscreen devices. Proc. 8th Conf. Human-computer Interact. Mob. Devices Serv. 203e210. Park, Y.S., Han, S.H., 2010a. Touch key design for one-handed thumb interaction with a mobile phone: effects of touch key size and touch key location. Int. J. Ind. Ergon. 40 (1), 68e76. Park, Y.S., Han, S.H., 2010b. One-handed thumb interaction of mobile devices from
the input accuracy perspective. Int. J. Ind. Ergon. 40, 746e756. Park, Y.S., Han, S.H., Park, J., Cho, Y., 2008. Touch key design for target selection on a mobile phone. Proc. 10th Int. Conf. Hum. Comput. Interact. Mob. Devices Serv. 423e426. Park, Y., Heo, H., Lee, K., 2015. Enhanced auditory feedback for Korean touch screen keyboards. Int. J. Hum. Comput. Stud. 73, 1e11. Perotto, A., Delagi, E.F., 1994. Anatomical Guide for the Electromyographer: the Limbs and Trunk, third ed. Charles C Thomas Publisher, Springfield. Ryu, T., Lim, J., Song, J., Yun, M.H., Shin, M., 2013. Performance comparison between the preferred right and preferred left hands in text entry using Qwerty touch keyboard smartphones. Int. J. Ind. Ergon. 43, 400e405. Schmidt, H., Lanz, U., 2004. Surgical Anatomy of the Hand. Thieme Medical Publishers, Thieme. Sesto, M.E., Irwin, C.B., Chen, K.B., Chourasia, A.O., Wiegmann, D.A., 2013. Effect of touch screen button size and spacing on touch characteristics of users with and without disabilities. Hum. Factors 54 (3), 425e436. Silfverberg, M., MacKenzie, I.S., Korhonen, P., 2000. Predicting text entry speed on mobile phones. Proc. Computer-Human Interact. CHI 2000. Statista, 2016. Mobile phone Internet User Penetration Worldwide from 2014 to 2019. http://www.statista.com/statistics/284202/mobile-phone-internet-userpenetration-worldwide/. Stevens, J.C., 1992. Aging and spatial acuity of touch. J. Gerontol. 47 (1), 35e40. Trudeau, M.B., Udtamadilok, T., Karlson, A.K., Dennerlein, J.T., 2012a. Thumb motor performance varies by movement orientation, direction, and device size during single-handed mobile phone use. Hum. Factors 54 (1), 52e59. Trudeau, M.B., Young, J.G., Jindrich, D.L., Dennerlein, J.T., 2012b. Thumb motor performance varies with thumb and wrist posture during single-handed mobile phone use. J. Biomech. 45 (14), 2349e2354. Verrillo, R.T., Gesheider, G.A., 1992. Perception via the sense of touch in tactile aids for the hearing impaired. In: Summers, I.R. (Ed.), Tactile Aids for the Hearing Impaired. Willey, John & Sons. Xiong, J., Muraki, S., 2014. An ergonomics study of thumb movements on smartphone touch screen. Ergonomics 57 (6), 943e955. Yin, P., Su, E., 2011. Cyber Swarm optimization for general keyboard arrangement problem. Int. J. Ind. Ergon. 41, 43e52.