- Email: [email protected]
Using an Extended Automatic Target Acquisition Program with dual cursor technology to assist people with developmental disabilities improve their pointing efficiency
Research in Developmental Disabilities 31 (2010) 1091–1101
Contents lists available at ScienceDirect
Research in Developmental Disabilities
Using an Extended Automatic Target Acquisition Program with dual cursor technology to assist people with developmental disabilities improve their pointing efficiency Ching-Hsiang Shih a,*, Ching-Tien Shih b, Hsin-Chin Chiu c a b c
Department of Special Education, National Dong Hwa University, Hualien 970, Taiwan, ROC Department of Electronics Engineering and Computer Science, Tung-Fang Institute of Technology, Kaohsiung Country, Taiwan, ROC National Hua-lien Special Education School for the Mentally Retarded, Hualien, Taiwan, ROC
A R T I C L E I N F O
A B S T R A C T
Article history: Received 12 March 2010 Accepted 18 March 2010
The latest research adopting software technology to improve pointing performance is through an Extended Automatic Pointing Assistive Program (EAPAP). However, EAPAP has some limitations. This study evaluated whether two children with developmental disabilities would be able to improve their pointing performance through an Extended Dual Cursor Automatic Pointing Assistive Program (EDCAPAP), which solves the limitations of EAPAP. Initially, both participants had their baseline sessions. Then intervention started with the first participant. New baseline and intervention occurred with the second participant when his performance was consolidated. Finally, both participants were exposed to the maintenance phase, in which their pointing performance improved significantly. Results of this study showed that, with the assistance of EDCAPAP, participants can position targets quickly, easily, and accurately, thus helping the disabled to solve their pointing problems. ß 2010 Elsevier Ltd. All rights reserved.
Keywords: Developmental disabilities Pointing EDCAPAP Mouse driver
Computers play an essential role in our everyday life of this modern society, and have been widely used in education, academic study, communication training, daily life, entertainment, pre-job training, etc. Computer technologies can broaden the lives and increase the independence and capacity to engage fully in daily activities and academic or vocational options for persons with disabilities (Resta & Laferriere, 2007; Salminen, Petrie, & Ryan, 2004; Stern, 2008). The benefits would be clearer if persons with disabilities were given the opportunity to improve their level of competency in using computer interfaces (Davies, Stock, & Wehmeyer, 2002a, 2002b; Langone, Clees, Rieber, & Matzko, 2003; Ritchie & Blanck, 2003). The computer mouse, as an important computer input device, is easy for most individuals to operate to move the cursor and click to choose things on the screen. Even young children can use a mouse (Donker & Reitsma, 2007a, 2007b). Most commercial computer input devices (i.e., mouse, trackball) are designed to be targeted at the mainstream population, without taking into account that these devices might be used by people with disabilities who generally encounter mouse operation problems (Abascal & Nicolle, 2005; Brodwin, Star, & Cardoso, 2004; Rao, Seliktar, & Rahman, 2000; Wong, Chan, LiTsang, & Lam, 2009). It can be difficult or impossible for them to operate computers that rely on a mouse or similar input devices. Therefore, various modifications and adaptations of computer-pointing devices, as well as special-purpose input devices, have been proposed to assist and meet the needs of this group of potential users (Brodwin et al., 2004; Hedrick, Pape, Heinemann, Ruddell, & Reis, 2006; Mann, Belchior, Tomita, & Kemp, 2005; Shein, Treviranus, Brownlow, Milner, & Parnes,
* Corresponding author. Tel.: +886 3 8227106x1320; fax: +886 3 8228707. E-mail address: [email protected] (C.-H. Shih). 0891-4222/$ – see front matter ß 2010 Elsevier Ltd. All rights reserved. doi:10.1016/j.ridd.2010.03.008
1092
C.-H. Shih et al. / Research in Developmental Disabilities 31 (2010) 1091–1101
1992; Shih & Shih, 2009b; Shih, Shih, & Luo, 2007; Tu, Tao, & Huang, 2007). Besides these special assistive input devices, Shih and Shih (2009a, 2009c) presented a multi-mouse configuration by using the remaining ability of each limb with several mice to perform mouse operation, in order to offer them the opportunity to use standard mice, like people without disabilities (Shih & Shih, 2009a, 2009c). Pointing, which is the most commonly adopted basic mouse operation for most computer programs and CAI software, is achieved by moving the mouse in order to move a cursor to point at a specific item such as an icon, picture, or text, and clicking the mouse button to send functional commands to the computer (Donker & Reitsma, 2007a; Shimizu & McDonough, 2006). Persons with disabilities usually encounter difficulties using a computer in pointing tasks. Common pointing problems include difficulty moving a pointing device in a straight line, inability to select small targets, or difficulty controlling the pointer’s buttons. Users can benefit from being provided useful functions in pointing, such as moving the cursor to the target centre automatically, and to position the target quickly, easily, and accurately (Grossman & Balakrishnan, 2005; Park, Han, & Yang, 2006). The advantages of the pointing assistive function may be greater for older users, children, or users with motionimpairments who are either unfamiliar with, or have difficulty in positioning a mouse (Casiez, Vogel, & Balakrishnan, 2008; Park et al., 2006). Many researchers have also offered solutions to facilitate the quality of pointing operation (such as expanding targets, bubble cursors, sticky icons, and target positioning/acquiring) in order to improve the operation ability of people with disabilities, and help them in pointing tasks (Ahlstrom, 2005; Ahlstrom, Hitz, & Leitner, 2006; Akamatsu & MacKenzie, 2002; Casiez et al., 2008; Cockburn & Brewster, 2005; Cockburn & Firth, 2003; Dennerlein & Yang, 2001; Grossman & Balakrishnan, 2005; Park et al., 2006). These solutions have characterized pointing performance in terms of movement trajectories, accuracy, clicking behaviors and speed. Shih, Hsu, and Shih (2009) presented a new operation method, Automatic Pointing Assistive Program (APAP), where the user can click the mouse button when the cursor is near the target (inside the activation area), instead of moving the cursor to the target, in order to improve pointing performance (Shih, Hsu, et al., 2009). APAP adopted software technology to redesign the mouse driver to intercept mouse click action. Mouse click action will be intercepted as soon as the mouse is clicked, the cursor will jump to the target center automatically, and then the intercepted mouse click action will be sent out, as shown in Fig. 1. APAP is able to run independently in the Windows operation system (OS) environment, compared to the previously available approaches, and works without interference with all currently available software (i.e., the currently available software does not need to be modified or rewritten). However, beside all the advantages of APAP mentioned above, APAP has some limits: (1) APAP cannot benefit from the Mouseover effect, and (2) activation areas will be too small in the case that the targets are too close to each other. Mouseover effects are commonly used in modern graphical user interface (GUI) programming. Mouseovers, by which an element changes in response to the mouse cursor moving over it (Wikipedia, 2009), as shown in Fig. 2, are practically a
Fig. 1. The operation flow of Automatic Pointing Assistive Program (APAP). (a) When the cursor enters into the activation area, as soon as the mouse is clicked, mouse click action will be intercepted. (b) The cursor will jump to the target center automatically. (c) The intercepted mouse click action will be sent out.
C.-H. Shih et al. / Research in Developmental Disabilities 31 (2010) 1091–1101
1093
Fig. 2. A fan simulation CAI software. There are four control buttons under the fan to control its actions: on/off, high speed, low speed and rotation. (a) The externality of the four control buttons does not change before the cursor moves over them. (b) When the cursor moves over them, the four control buttons have their corresponding Mouseover effects (the red button, tooltips and sound feedback). (For interpretation of the references to color in this figure legend, the reader is referred to the web version of the article.)
universal standard, and are commonly used in windows application software to clarify what is active. In APAP, the cursor jumps to the target only when the mouse is clicked (inside the activation area). Therefore, users cannot be assisted by Mouseover effects before user clicks the mouse button, because Mouseover will only work when the cursor is over the target (Shih, Hsu, et al., 2009). Therefore, Shih, Chung, Chiang, and Shih (2010) proposed a new operation method, Dual Cursor Automatic Pointing Assistive Program (DCAPAP), where the dual cursors (a virtual cursor and a system cursor) are adopted, offering users an operating environment with Mouseover effects closer to the real conditions to solve this limitation (Shih, Chung, et al., 2010), as shown in Fig. 3. When the system cursor enters into the activation area, with this technology (DCAPAP), it will jump to the target center automatically and activate Mouseover effects. Besides this, the virtual cursor appears to indicate the movement path and users can click as soon as the virtual cursor is inside the activation area. Another limitation of APAP is users cannot significantly improve their pointing efficiency in the case of the targets being too close to each other. In this case, activation areas will be too small, due to the space limitation among targets. When the buttons are too close, as shown in Fig. 4(a), their activation areas are limited to avoid wrong operation due to overlapping. Fig. 4(b) is the operation window for Media Player Classic software (Gabest, 2009) in which the operational buttons are very small and close to each other, leading to the pointing difficulties for persons with disabilities. Therefore, Shih, Li, Shih, Lin, and Lo (2010) presented a new revised operation method, Extended APAP (EAPAP), where the target activation areas with icon and text prompt to indicate their corresponding targets/buttons are moved away from targets to corresponding positions on the screen (Shih, Li, et al., 2010), in order to be large enough for users to point at, thus solving this limitation, as shown in Fig. 5. With the assistance of EAPAP, activation areas in EAPAP are not bound to their corresponding targets, but are placed correspondingly on the screen and use icon and text prompts to indicate their corresponding targets (such as play, pause, stop, etc.). As mentioned above, though the latest research EAPAP can solve the limitation of targets that are too close to each other, this solution still has limitations because the cursor jumps to the target only when the mouse is clicked. Therefore, users cannot be assisted by Mouseover effects before he/she clicks the mouse button (Shih, Chung, et al., 2010; Shih, Hsu, et al., 2009).
1094
C.-H. Shih et al. / Research in Developmental Disabilities 31 (2010) 1091–1101
Fig. 3. The operation flow of Dual Cursor Automatic Pointing Assistive Program (DCAPAP). (a) The externality of the cursor before it enters into the activation area. (b) When the cursor enters into the activation area, the virtual cursor (solid pink cursor) appears to indicate the movement path of the system cursor, and the system cursor jumps to the target center automatically and activates Mouseover effects. (c) The system cursor is locked to the target center while the virtual cursor still moves inside the activation area. (d) Once the virtual cursor moves out of the activation area it disappears; the system cursor replaces it and continues moving. (For interpretation of the references to color in this figure legend, the reader is referred to the web version of the article.)
Fig. 4. Limitations of APAP when targets are too close to each other. (a) When the buttons are too close, their activation areas are limited, this must be adjusted in order to avoid wrong operation due to overlapping. (b) The operation window of Media Player Classic software (Gabest, 2009) whose operational buttons are very small and close to each other, which leads to the pointing difficulties for persons with disabilities.
C.-H. Shih et al. / Research in Developmental Disabilities 31 (2010) 1091–1101
1095
Fig. 5. The operation window of the Media Player Classic (Gabest, 2009) with the assistance of the Extended APAP (EAPAP). In order to be large enough for users to point, the target activation areas with icon and text prompt indicating their corresponding targets/buttons, are moved away from targets to proper positions on the screen.
This limitation of EAPAP can be solved through a new revised operation method, Extended Dual Cursor APAP (EDCAPAP), where Dual Cursor technology (Shih, Chung, et al., 2010) is adopted to implement the function of EAPAP with Mouseover effects. As in EAPAP, the target activation areas with icons, sound, and text prompts to indicate their corresponding targets/ buttons are moved away from targets to corresponding positions on the screen, in order to be large enough for users to point at. As shown in Fig. 6, the system cursor acts as usual before entering into the activation area (Fig. 6(a)). When the system cursor enters into the activation area, the virtual cursor (solid pink cursor) appears and indicates the movement path of the system cursor; the system cursor then jumps to the target center and activates Mouseover effects (the hand shaped cursor, red button, tooltips and sound feedback) automatically (Fig. 6(b)). It is locked to the target center while the virtual cursor still moves inside the activation area. The virtual cursor will disappear once it moves out of the activation area, and the system cursor will replace it and continue moving (Fig. 6(c)). With the assistance of EDCAPAP, each small control button of the fan simulation CAI software has their activation areas which are placed correspondingly on the screen (‘‘On/Off’’, ‘‘Rotate’’, ‘‘High speed’’ and ‘‘Low speed’’) with icon, sound, and text prompts, as shown in Fig. 7. As with DCAPAP, the key technology of EDCAPAP is the mouse movement action interception/detection. Users can neither detect mouse movement nor lock the cursor over targets without this technology application. As a standard device for computers, once the mouse is connected to a computer, the windows OS will identify it and install its driver automatically. The mouse driver is provided by Windows OS or the hardware manufacturer to ensure that the connected device can work normally to define its function as moving, clicking and dragging. Therefore, it is not easy to modify mouse functions (intercept/detect mouse movement action) to meet the needs of EDCAPAP, and there is no research published in this field. Redesigning a mouse driver can reset the mouse functions, turning it into a much more powerful tool, however, this is rarely proposed by researches because of the complexity of the technology required (Microsoft, 2008a, 2008b, 2008c). Only a few recent researches (Shih, Chang, & Shih, 2009, in press; Shih, Cheng, Li, Shih, & Chiang, 2010; Shih, Chiu, et al., 2010; Shih, Chung, et al., 2010; Shih, Hsu, et al., 2009; Shih, Huang, Liao, Shih, & Chiang, 2010; Shih, Li, et al., 2010; Shih & Shih, 2009a, 2009c, 2009d, 2010; Shih, Shih, & Chiang, 2010; Shih, Shih, Lin, & Chiang, 2009) adopted software technology to redesign the mouse driver, visualizing the mouse as a useful tool for many applications dedicated to persons with disabilities (such as hand movement detector, finger/thumb poke detector and pointing/drag-and-drop assistive function, etc.), providing them with additional choices in assistive technology. Therefore, EDCAPAP can be realized using revising Shih’s mouse driver to intercept/detect mouse movement action, in order to help people with disabilities improve their pointing efficiency with Mouseover effects. This work utilizes Shih’s new revised mouse driver (intercepting/detecting driver) design (i.e., a new mouse driver replaces the standard mouse driver, and is able to intercept/detect mouse movement action) to understand the difference between pointing performance before and after for people with developmental disabilities using EDCAPAP, in order to determine whether the EDCAPAP implementation can enhance their pointing performance.
1096
C.-H. Shih et al. / Research in Developmental Disabilities 31 (2010) 1091–1101
Fig. 6. The operation flow of Extended Dual Cursor Automatic Pointing Assistive Program (EDCAPAP). (a) The externality of the cursor before it enters into the activation area. (b) The virtual cursor (solid pink cursor) appears, indicating the movement path of the system cursor, when the cursor enters into the activation area, and the system cursor jumps to the target center automatically, activating Mouseover effects (i.e., the hand shaped cursor, red button, ‘‘on’’ text prompt and sound feedback). It will be locked to the target center while the virtual cursor still moves inside the activation area. (c) The virtual cursor disappears once it moves out of the activation area, and the system cursor replaces it and continues moving. (For interpretation of the references to color in this figure legend, the reader is referred to the web version of the article.)
1. Method 1.1. Participants The participants Tsai and Huang were 17 and 15 years of age, respectively. Tsai’s level of function was estimated to be in the mild range of intellectual disability. He had poor fine motor skills in his hand, but could use his right hand to hold and use
Fig. 7. With the assistance of EAPAP, each small control button of the fan simulation CAI software has its activation area with icon and text prompts, such as on/off, rotate, high speed and low speed, etc.
C.-H. Shih et al. / Research in Developmental Disabilities 31 (2010) 1091–1101
1097
a common mouse. Tsai had little mouse operation experience, but he could move a mouse and click a mouse button in a laborious manner. With the guidance of the research assistant, he learned when he needed to click, but had very low mouse operation efficiency due to poor hand function, and could not use a mouse for an extended amount of time. Huang’s level of function was estimated to be in the profound range of intellectual disabilities. He could control the mouse with both hands. Normally, he used his right hand to operate a mouse, but needed to change to the other hand after a while. He could click, but his mouse operation performance was slower and less accurate than people of the same age when completing pointing tasks. As with Tsai, he knew when to click with the help of the research assistant. He also had low mouse operation efficiency due to poor hand coordination. Both participants were interested in computer operation, and neither had visual disabilities that could be problematic in using a mouse. With the guidance of the research assistant, both learned to move the mouse cursor to the targets and perform the click. Their parents had given formal consent for their involvement in this experiment. 1.2. Apparatus and setting The experiment was carried out in a separate room of the special education school for the mentally retarded. Computers were placed on a computer table, and the screen was at a distance of about 30 cm from their chairs. Mice were provided to the participants when experiment began. 1.2.1. EDCAPAP setting, computer mouse training and test software Computer software was designed, in this study, to provide mouse pointing practice for participants, and to record their test results. The software was designed with two modes, namely practice mode and test mode (Shih, Chung, et al., 2010; Shih, Hsu, et al., 2009; Shih, Li, et al., 2010). Practice mode was designed to provide repeated pointing practice for participants, and test mode was designed to record participants’ successful points within a certain period of time. Fig. 8 shows the flow diagram of the computer mouse training and test software. Eight circular destination targets (T1–T8) with radii of 0.25 cm were set every 458 on a circle with a radius of 1.25 cm, close to each other. Every target, which
Fig. 8. The flow diagram of the computer mouse training and test software. Eight circular destination targets (T1–T8) with radii of 0.25 cm, were set close to each other, every 458, on a circle with a radius of 1.25 cm. Each target had a circular area with a radius of 1.25 cm as the activation area (A1–A8) of EDCAPAP; each of which was also set every 458 on a circle with a radius of 6 cm. Besides this, eight positions, noted P1 through P8 (in accordance with the eight targets), were set as the cursor initial position and displayed every 458 on the edge of the screen.
1098
C.-H. Shih et al. / Research in Developmental Disabilities 31 (2010) 1091–1101
were also set every 458 on a circle, with a radius of 6 cm, had a circular area with a radius of 1.25 cm as the activation area (A1–A8) of EDCAPAP. When the participants moved the cursor into the activation areas (A1–A8), the system cursor jumped to the target center (T1–T8) and the virtual cursor appeared to indicate the movement path. In this way, the participants could perform click inside the circular activation areas (A1–A8) instead of moving the cursor to targets (T1– T8). Eight positions in practice mode, noted from P1 to P8 (in accordance with the eight targets), were set as cursor initial positions and displayed every 458 on the edge of the screen. First, the computer displayed T1, and set the mouse cursor to P1. The participants had to move the mouse cursor from P1 to target T1, and then click to complete a successful pointing. When the task was completed, T1 disappeared, T2 appeared, and the computer automatically set the cursor to P2. Participants would then move the cursor from P2 to T2, and click on it. This process was repeated until the end of the practice time, and the computer provided vocal prompting. Test mode was run under the same conditions as the practice mode, except the targets appeared randomly, and no vocal prompting was provided. Times of successful points within 3 min were recorded. 1.3. Experimental conditions This study used multiple probe design across participants (Richards, Taylor, Ramasamy, & Richards, 1999). Participants typically received 3 training sessions per week, each with about 30 min use of EDCAPAP, for a period of about 6–7 weeks. Tsai initially received three pre-probe sessions during baseline, and intervention began when his performance was consolidated. Huang received discontinuous pre-probe. When Tsai’s performance was consolidated during intervention, new intervention occurred with Huang. The experiment comprised three phases: (a) the baseline phase, in which at least three pre-probe sessions were performed to collect participants’ baseline data; (b) the intervention phase, in which EDCAPAP was adopted to obtain the performance data of EDCAPAP practice for assessment; and (c) the maintenance phase, performed 1 week after intervention finished, in which participants’ follow-up performance was assessed three times. 1.3.1. Baseline The test software was adopted during the baseline to record the successful points within 3 min. In this phase, the EDCAPAP function was turned off; therefore, the participants had to move the cursor from cursor initial positions (P1–P8) to the targets (T1–T8) to click. Three data points were collected for Tsai in 1 week during this phase, while Huang’s baseline data was obtained twice per week discontinuously. 1.3.2. Intervention At the start of the phase, Tsai first received training for EDCAPAP use and training software, and Huang was still being probed discontinuously to collect his baseline data. Huang received EDCAPAP training once Tsai’s performance was consolidated. In principle, 11 30-min practice sessions were performed on each participant during intervention. The arrangement of this 30-min session was as follows: (a) Pointing practice (20 min) The EDCAPAP function was turned on in this phase. Therefore, the participants could perform click inside the circular activation areas (A1–A8) instead of moving the cursor to targets (T1–T8). Destination targets (T1–T8) appeared in order, and participants moved the mouse cursor from cursor initial positions (P1–P8) to the target activation areas (A1–A8) and clicked. During the practice session, computer vocal prompting was available. A research assistant provided guidance to help the participants use EDCAPAP to complete pointing. (b) Rest (7 min) Participants were given 7 min rest after practice. (c) Assessment (3 min) The targets appeared randomly during this phase. Neither instructions from the research assistant nor computer vocal prompting were available. The number of times that each participant successfully pointed within 3 min was recorded as input for assessment, and then adopted to determine whether EDCAPAP improved their pointing efficiency. This phase continued until each participant’s performance was consolidated.
1.3.3. Maintenance This phase began 1 week after the intervention phase to determine whether the participants maintained the skills that they had acquired. Participants did not have EDCAPAP practice during this phase, but participated directly in the pointing test. 2. Results Fig. 9 shows the pointing speed of the two participants after the implementation of EDCAPAP. The curve indicates that both participants improved their pointing efficiency, and maintained their acquired skills during the maintenance phase.
C.-H. Shih et al. / Research in Developmental Disabilities 31 (2010) 1091–1101
1099
Fig. 9. The pointing speeds of the two participants after the implementation of EDCAPAP. As indicated by the curve, both participants improved their pointing efficiency, maintaining their acquired skills during the maintenance phase.
2.1. Tsai 2.1.1. Baseline Tsai could hold and control a common mouse with his right hand. Detailed observation of his mouse operation indicated that he could not point accurately, owing to his poor manual fine motor skills and difficulties in hand control. The cursor often deviated from the target, which made positioning difficult for him. Tsai only achieved a mean of 1.44 correct points per minute during this phase. 2.1.2. Intervention At first, Tsai was unfamiliar with the EDCAPAP function, and had a poor pointing efficiency. Often, unsuccessful pointing arose because of the deviation of the cursor. His correct pointing per minute, however, still increased compared to baseline, because EDCAPAP solved the deviation problem of the cursor when clicking. He gradually mastered the use of EDCAPAP, and increased his pointing efficiency as the practice time accumulated. Fig. 9 shows that Tsai’s pointing speed increase quickly. He achieved 18–24 correct points within 3 min during the 2 sessions in his earlier intervention phase, improved his pointing performance within 3–4 practice sessions, and became able to use EDCAPAP to facilitate the target positioning tasks. Tsai’s correct pointing within 3 min increased to 33–42 during sessions 3–11, revealing that EDCAPAP operation could be mastered easily within a short period of practice. Tsai achieved 6.00–14.00 correct points per min, with an overall mean of 11.42 per min. The Kolmogorov–Smirnov test (Siegel & Castellan, 1988) showed that the increase from baseline to intervention was statistically significant (p < 0.01). 2.1.3. Maintenance Tsai entered maintenance phase 1 week after intervention finished. According to Fig. 9, his correct points within 3 min in 3 sessions during this phase were 53, 48 and 61, respectively. His pointing speed was between 16.00 and 20.33 per min, with an overall mean of 18.00. These results indicate that he retained the skills that he had acquired during the intervention phase. By comparison, it was demonstrated that the performance was better in the maintenance phase than in the intervention phase. This finding may be caused by the fact that Tsai apparently did not focus on the training for the last few minutes of the session during the intervention phase, making his performance worse than expected. His direct access to the test and the short attention time needed could have improved, in contrast, his performance during the maintenance phase. 2.2. Huang 2.2.1. Baseline Huang could control the mouse, but had few successful points because of the small size of targets. His successful points within 3 min were 8, 6 and 10, respectively. He also had difficulty in controlling the movement of the mouse cursor while simultaneously clicking, due to poor hand-eye coordination. Huang achieved a mean of 2.67 correct points per minute during the baseline phase.
1100
C.-H. Shih et al. / Research in Developmental Disabilities 31 (2010) 1091–1101
2.2.2. Intervention Huang mastered the use of EDCAPAP at the beginning, but could not concentrate on the use of the mouse, and needed to change hands to control the mouse. Gradually, he could concentrate on using the mouse, and his pointing speed rose slowly, according to Fig. 9. He achieved 6.33–16.67 correct points per min during this phase, with an overall mean of 10.33. The Kolmogorov–Smirnov test (Siegel & Castellan, 1988) showed that the increase from baseline to intervention was statistically significant (p < 0.01). 2.2.3. Maintenance Huang’s pointing speed in this phase was close to that during the intervention phase. He began this phase 1 week following intervention. He could finish pointing without changing hands to operate the mouse, and concentrate on mouse use in the last 3 min session. During this phase (including 3 sessions), his activity data (correct points within 3 min) were 53, 41 and 44, respectively, and he achieved 13.67–17.67 correct points per min. His pointing speed in this phase was close to that during the intervention phase. These results indicate that he retained the skills that he had acquired during the intervention phase. 3. Discussion This work has demonstrated that both participants rapidly improved their efficiency to point at targets which were small and close to each other with Mouseover effects after receiving EDCAPAP training, and retained their acquisition skills in the maintenance phase. Results of this study also show that extensive practice time is not required for people with developmental disabilities in order to use EDCAPAP. With the assistance of EDCAPAP, participants can position targets quickly, easily, and accurately, thus helping the disabled to solve their pointing problems. The two participants could operate some ordinary CAI/educational software with EDCAPAP after the experiment. This EDCAPAP is a software-based solution, so it does not need extra hardware or circuit preservations, and can be widely spread and popularized by internet; it can support all standard interfaces of commercial input devices (i.e., trackball) that are compatible with the computer, including USB, wireless and Bluetooth interfaces (Bluetooth.org, 2009). In addition, EDCAPAP is compatible with all currently available software, so the current software can be applied to improve the pointing efficiency of people with disabilities without being modified or rewritten. This study, focusing on individuals with developmental disabilities who cannot use standard mice to perform pointing efficiently, only considers pointing. It does not address other mouse controls needing higher physical abilities, like doubleclicking, dragging and scrolling. Pointing is sufficient, however, for the growing number of well-designed educational programs (Donker & Reitsma, 2007a; Shimizu & McDonough, 2006). Further studies are necessary to develop additional mouse applications, with corresponding training and test software to satisfy the needs of different levels of disabilities. Hopefully, the implementation of EDCAPAP can cover all complex mouse movements and provide disabled users with additional choices in assistive technology. References Abascal, J., & Nicolle, C. (2005). Moving towards inclusive design guidelines for socially and ethically aware HCI. Interacting with Computers, 17, 484–505. Ahlstrom, D. (2005). Modeling and improving selection in cascading pull-down menus using Fitts’ law, the steering law and force fields. Paper Presented at the Proceedings of the CHI’05 Conference on Human Factors in Computing Systems. Ahlstrom, D., Hitz, M., & Leitner, G. (2006). An evaluation of sticky and force enhanced targets in multi target situations. Paper Presented at the Proceedings of the 4th Nordic conference on Human–Computer Interaction: Changing Roles. Akamatsu, M., & MacKenzie, I. S. (2002). Changes in applied force to a touchpad during pointing tasks. International Journal of Industrial Ergonomics, 29, 171–182. Bluetooth.org. (2009). Bluetooth.org—Welcome to Bluetooth.org. Retrieved 6/18, 2009, from http://www.bluetooth.org/apps/content/. Brodwin, M. G., Star, T., & Cardoso, E. (2004). Computer assistive technology for people who have disabilities: Computer adaptations and modifications. Journal of Rehabilitation, 70, 28–33. Casiez, G., Vogel, D., & Balakrishnan, R. (2008). The impact of control-display gain on user performance in pointing tasks. Human–Computer Interaction, 23, 215– 250. Cockburn, A., & Brewster, S. (2005). Multimodal feedback for the acquisition of small targets. Ergonomics, 48, 1129–1150. Cockburn, A., & Firth, A. (2003). Improving the acquisition of small targets. Paper Presented at the Proceedings of the 17th Annual Human–Computer Interaction Conference. Davies, D. K., Stock, S. E., & Wehmeyer, M. L. (2002a). Enhancing independent task performance for individuals with mental retardation through use of a handheld self-directed visual and audio prompting system. Education and Training in Mental Retardation and Developmental Disabilities, 37, 209–218. Davies, D. K., Stock, S. E., & Wehmeyer, M. L. (2002b). Enhancing independent time-management skills of individuals with mental retardation using a palmtop personal computer. Mental Retardation, 40, 358–365. Dennerlein, J. T., & Yang, M. C. (2001). Haptic force-feedback devices for the office computer: Performance and musculoskeletal loading issues. Human Factors, 43, 278. Donker, A., & Reitsma, P. (2007a). Aiming and clicking in young children’s use of the computer mouse. Computers in Human Behavior, 23, 2863–2874. Donker, A., & Reitsma, P. (2007b). Young children’s ability to use a computer mouse. Computers and Education, 48, 602–617. Gabest. (2009). Media Player Classic – Home Cinema – Video Player. Retrieved June 5, 2009, from http://mpc-hc.sourceforge.net/. Grossman, T., & Balakrishnan, R. (2005). The bubble cursor: enhancing target acquisition by dynamically resizing of the cursor’s activation area. Paper presented at the Proceedings of CHI’05 Conference on Human Factors in Computing Systems. Hedrick, B., Pape, T. L., Heinemann, A. W., Ruddell, J. L., & Reis, J. (2006). Employment issues and assistive technology use for persons with spinal cord injury. Journal of Rehabilitation Research and Development, 43, 185. Langone, J., Clees, T. J., Rieber, L., & Matzko, M. (2003). The future of computer-based interactive technology for teaching individuals with moderate to severe disabilities: Issues relating to research and practice. Journal of Special Education Technology, 18, 5–16. Mann, W. C., Belchior, P., Tomita, M. R., & Kemp, B. J. (2005). Computer use by middle-aged and older adults with disabilities. Technology and Disability, 17, 1–9.
C.-H. Shih et al. / Research in Developmental Disabilities 31 (2010) 1091–1101
1101
Microsoft. (2008a). WDM: Introduction to Windows Driver Model. Retrieved 6/18, 2008, from http://www.microsoft.com/whdc/archive/wdm.mspx#EJE. Microsoft. (2008b). Windows Driver Kit: Human input devices. Retrieved 5/30, 2008, from http://msdn.microsoft.com/en-us/library/ms794089.aspx. Microsoft. (2008c). Windows Management Instrumentation. Retrieved 6/18, 2008, from http://msdn.microsoft.com/en-us/library/aa394582(VS.85).aspx. Park, J., Han, S. H., & Yang, H. (2006). Evaluation of cursor capturing functions in a target positioning task. International Journal of Industrial Ergonomics, 36, 721– 730. Rao, R. S., Seliktar, R., & Rahman, T. (2000). Evaluation of an isometric and a position joystick in a target acquisition task for individuals with cerebral palsy. IEEE Transactions on Rehabilitation Engineering, 8, 118–125. Resta, P., & Laferriere, T. (2007). Technology in support of collaborative learning. Educational Psychology Review, 19, 65–83. Richards, S. B., Taylor, R. L., Ramasamy, R., & Richards, R. Y. (1999). Single subject research: Applications in educational and clinical settings. New York: Wadsworth. Ritchie, H., & Blanck, P. (2003). Promise of the Internet for disability: A study of online services and accessibility of centers for independent living Web sites. Behavioral Sciences and the Law, 21, 5–26. Salminen, A. L., Petrie, H., & Ryan, S. (2004). Impact of computer augmented communication on the daily lives of speech-impaired children. Part I: Daily communication and activities. Technology and Disability, 16, 157–167. Shein, G. F., Treviranus, J., Brownlow, N. D., Milner, M., & Parnes, P. (1992). An overview of human–computer interaction techniques for people with physical disabilities. International Journal of Industrial Ergonomics, 9, 171–181. Shih, C.-H., Chang, M.-L., & Shih, C.-T. (2009). Assisting people with multiple disabilities and minimal motor behavior to improve computer pointing efficiency through a mouse wheel. Research in Developmental Disabilities, 30, 1378–1387. Shih, C.-H., Chang, M.-L., & Shih, C.-T. (in press). A new limb movement detector enabling people with multiple disabilities to control environmental stimulation through limb swing with a gyration air mouse. Research in Developmental Disabilities. doi:10.1016/j.ridd.2010.01.020. Shih, C.-H., Cheng, H.-F., Li, C.-C., Shih, C.-T., & Chiang, M.-S. (2010). Assisting people with developmental disabilities improve their collaborative pointing efficiency with a Multiple Cursor Automatic Pointing Assistive Program. Research in Developmental Disabilities, 31, 600–607. Shih, C.-H., Chiu, S.-K., Chu, C.-L., Shih, C.-T., Liao, Y.-K., & Lin, C.-C. (2010). Assisting people with multiple disabilities improve their computer pointing efficiency with hand swing through a standard mouse. Research in Developmental Disabilities, 31, 517–524. Shih, C.-H., Chung, C.-C., Chiang, M.-S., & Shih, C.-T. (2010). Assisting people with developmental disabilities to improve pointing efficiency with a Dual Cursor Automatic Pointing Assistive Program. Research in Developmental Disabilities, 31, 151–159. Shih, C.-H., Hsu, N.-Y., & Shih, C.-T. (2009). Assisting people with developmental disabilities to improve pointing efficiency with an Automatic Pointing Assistive Program. Research in Developmental Disabilities, 30, 1212–1220. Shih, C.-H., Huang, H.-C., Liao, Y.-K., Shih, C.-T., & Chiang, M.-S. (2010). An Automatic Drag-and-Drop Assistive Program developed to assistive people with developmental disabilities to improve drag-and-drop efficiency. Research in Developmental Disabilities, 31, 416–425. Shih, C.-H., Li, C.-C., Shih, C.-T., Lin, K.-T., & Lo, C.-S. (2010). Extended Automatic Pointing Assistive Program—A pointing assistance program to help people with developmental disabilities improve their pointing efficiency. Research in Developmental Disabilities, 31, 672–679. Shih, C.-H., & Shih, C.-T. (2009a). Assisting people with multiple disabilities to use computers with multiple mice. Research in Developmental Disabilities, 30, 746– 754. Shih, C.-H., & Shih, C.-T. (2009b). Development of a computer input system for persons with disabilities using a commercial mouse and switches. Disability and Rehabilitation: Assistive Technology, 4, 414–421. Shih, C.-H., & Shih, C.-T. (2009c). Development of an integrated pointing device driver for the disabled. Disability and Rehabilitation: Assistive Technology, doi:10.1080/17483100903105599. Shih, C.-H., & Shih, C.-T. (2009d). A new movement detector to enable people with multiple disabilities to control environmental stimulation with hand swing through a commercial mouse. Research in Developmental Disabilities, 30, 1196–1202. Shih, C.-H., & Shih, C.-T. (2010). Assisting two children with multiple disabilities and minimal motor behavior to control environmental stimulation with thumb poke through a trackball. Behavioural and Cognitive Psychotherapy, 38, 211–219. Shih, C.-H., Shih, C.-T., & Chiang, M.-S. (2010). A new standing posture detector to enable people with multiple disabilities to control environmental stimulation by changing their standing posture through a commercial Wii Balance Board. Research in Developmental Disabilities, 31, 281–286. Shih, C.-H., Shih, C.-T., Lin, K.-T., & Chiang, M.-S. (2009). Assisting people with multiple disabilities and minimal motor behavior to control environmental stimulation through a mouse wheel. Research in Developmental Disabilities, 30, 1413–1419. Shih, C.-H., Shih, C.-T., & Luo, C.-H. (2007). Design of a fast switch-control computer input device using two-grade movement method for people with physical disabilities. WSEAS Transactions on Computers, 6, 1140–1146. Shimizu, H., & McDonough, C. S. (2006). Programmed instruction to teach pointing with a computer mouse in preschoolers with developmental disabilities. Research in Developmental Disabilities, 27, 175–189. Siegel, S., & Castellan, N. J. (1988). Nonparametric statistics for the behavioral sciences. New York: McGraw-HiU Book Company. Stern, S. E. (2008). Computer-synthesized speech and perceptions of the social influence of disabled users. Journal of Language and Social Psychology, 27, 254. Tu, J., Tao, H., & Huang, T. (2007). Face as mouse through visual face tracking. Computer Vision and Image Understanding, 108, 35–40. Wikipedia. (2009). Mouseover. Retrieved March 10, 2009, from http://en.wikipedia.org/wiki/Mouseover. Wong, A. W. K., Chan, C. C. H., Li-Tsang, C. W. P., & Lam, C. S. (2009). Competence of people with intellectual disabilities on using human–computer interface. Research in Developmental Disabilities, 30, 107–123.
Contents lists available at ScienceDirect
Research in Developmental Disabilities
Using an Extended Automatic Target Acquisition Program with dual cursor technology to assist people with developmental disabilities improve their pointing efficiency Ching-Hsiang Shih a,*, Ching-Tien Shih b, Hsin-Chin Chiu c a b c
Department of Special Education, National Dong Hwa University, Hualien 970, Taiwan, ROC Department of Electronics Engineering and Computer Science, Tung-Fang Institute of Technology, Kaohsiung Country, Taiwan, ROC National Hua-lien Special Education School for the Mentally Retarded, Hualien, Taiwan, ROC
A R T I C L E I N F O
A B S T R A C T
Article history: Received 12 March 2010 Accepted 18 March 2010
The latest research adopting software technology to improve pointing performance is through an Extended Automatic Pointing Assistive Program (EAPAP). However, EAPAP has some limitations. This study evaluated whether two children with developmental disabilities would be able to improve their pointing performance through an Extended Dual Cursor Automatic Pointing Assistive Program (EDCAPAP), which solves the limitations of EAPAP. Initially, both participants had their baseline sessions. Then intervention started with the first participant. New baseline and intervention occurred with the second participant when his performance was consolidated. Finally, both participants were exposed to the maintenance phase, in which their pointing performance improved significantly. Results of this study showed that, with the assistance of EDCAPAP, participants can position targets quickly, easily, and accurately, thus helping the disabled to solve their pointing problems. ß 2010 Elsevier Ltd. All rights reserved.
Keywords: Developmental disabilities Pointing EDCAPAP Mouse driver
Computers play an essential role in our everyday life of this modern society, and have been widely used in education, academic study, communication training, daily life, entertainment, pre-job training, etc. Computer technologies can broaden the lives and increase the independence and capacity to engage fully in daily activities and academic or vocational options for persons with disabilities (Resta & Laferriere, 2007; Salminen, Petrie, & Ryan, 2004; Stern, 2008). The benefits would be clearer if persons with disabilities were given the opportunity to improve their level of competency in using computer interfaces (Davies, Stock, & Wehmeyer, 2002a, 2002b; Langone, Clees, Rieber, & Matzko, 2003; Ritchie & Blanck, 2003). The computer mouse, as an important computer input device, is easy for most individuals to operate to move the cursor and click to choose things on the screen. Even young children can use a mouse (Donker & Reitsma, 2007a, 2007b). Most commercial computer input devices (i.e., mouse, trackball) are designed to be targeted at the mainstream population, without taking into account that these devices might be used by people with disabilities who generally encounter mouse operation problems (Abascal & Nicolle, 2005; Brodwin, Star, & Cardoso, 2004; Rao, Seliktar, & Rahman, 2000; Wong, Chan, LiTsang, & Lam, 2009). It can be difficult or impossible for them to operate computers that rely on a mouse or similar input devices. Therefore, various modifications and adaptations of computer-pointing devices, as well as special-purpose input devices, have been proposed to assist and meet the needs of this group of potential users (Brodwin et al., 2004; Hedrick, Pape, Heinemann, Ruddell, & Reis, 2006; Mann, Belchior, Tomita, & Kemp, 2005; Shein, Treviranus, Brownlow, Milner, & Parnes,
* Corresponding author. Tel.: +886 3 8227106x1320; fax: +886 3 8228707. E-mail address: [email protected] (C.-H. Shih). 0891-4222/$ – see front matter ß 2010 Elsevier Ltd. All rights reserved. doi:10.1016/j.ridd.2010.03.008
1092
C.-H. Shih et al. / Research in Developmental Disabilities 31 (2010) 1091–1101
1992; Shih & Shih, 2009b; Shih, Shih, & Luo, 2007; Tu, Tao, & Huang, 2007). Besides these special assistive input devices, Shih and Shih (2009a, 2009c) presented a multi-mouse configuration by using the remaining ability of each limb with several mice to perform mouse operation, in order to offer them the opportunity to use standard mice, like people without disabilities (Shih & Shih, 2009a, 2009c). Pointing, which is the most commonly adopted basic mouse operation for most computer programs and CAI software, is achieved by moving the mouse in order to move a cursor to point at a specific item such as an icon, picture, or text, and clicking the mouse button to send functional commands to the computer (Donker & Reitsma, 2007a; Shimizu & McDonough, 2006). Persons with disabilities usually encounter difficulties using a computer in pointing tasks. Common pointing problems include difficulty moving a pointing device in a straight line, inability to select small targets, or difficulty controlling the pointer’s buttons. Users can benefit from being provided useful functions in pointing, such as moving the cursor to the target centre automatically, and to position the target quickly, easily, and accurately (Grossman & Balakrishnan, 2005; Park, Han, & Yang, 2006). The advantages of the pointing assistive function may be greater for older users, children, or users with motionimpairments who are either unfamiliar with, or have difficulty in positioning a mouse (Casiez, Vogel, & Balakrishnan, 2008; Park et al., 2006). Many researchers have also offered solutions to facilitate the quality of pointing operation (such as expanding targets, bubble cursors, sticky icons, and target positioning/acquiring) in order to improve the operation ability of people with disabilities, and help them in pointing tasks (Ahlstrom, 2005; Ahlstrom, Hitz, & Leitner, 2006; Akamatsu & MacKenzie, 2002; Casiez et al., 2008; Cockburn & Brewster, 2005; Cockburn & Firth, 2003; Dennerlein & Yang, 2001; Grossman & Balakrishnan, 2005; Park et al., 2006). These solutions have characterized pointing performance in terms of movement trajectories, accuracy, clicking behaviors and speed. Shih, Hsu, and Shih (2009) presented a new operation method, Automatic Pointing Assistive Program (APAP), where the user can click the mouse button when the cursor is near the target (inside the activation area), instead of moving the cursor to the target, in order to improve pointing performance (Shih, Hsu, et al., 2009). APAP adopted software technology to redesign the mouse driver to intercept mouse click action. Mouse click action will be intercepted as soon as the mouse is clicked, the cursor will jump to the target center automatically, and then the intercepted mouse click action will be sent out, as shown in Fig. 1. APAP is able to run independently in the Windows operation system (OS) environment, compared to the previously available approaches, and works without interference with all currently available software (i.e., the currently available software does not need to be modified or rewritten). However, beside all the advantages of APAP mentioned above, APAP has some limits: (1) APAP cannot benefit from the Mouseover effect, and (2) activation areas will be too small in the case that the targets are too close to each other. Mouseover effects are commonly used in modern graphical user interface (GUI) programming. Mouseovers, by which an element changes in response to the mouse cursor moving over it (Wikipedia, 2009), as shown in Fig. 2, are practically a
Fig. 1. The operation flow of Automatic Pointing Assistive Program (APAP). (a) When the cursor enters into the activation area, as soon as the mouse is clicked, mouse click action will be intercepted. (b) The cursor will jump to the target center automatically. (c) The intercepted mouse click action will be sent out.
C.-H. Shih et al. / Research in Developmental Disabilities 31 (2010) 1091–1101
1093
Fig. 2. A fan simulation CAI software. There are four control buttons under the fan to control its actions: on/off, high speed, low speed and rotation. (a) The externality of the four control buttons does not change before the cursor moves over them. (b) When the cursor moves over them, the four control buttons have their corresponding Mouseover effects (the red button, tooltips and sound feedback). (For interpretation of the references to color in this figure legend, the reader is referred to the web version of the article.)
universal standard, and are commonly used in windows application software to clarify what is active. In APAP, the cursor jumps to the target only when the mouse is clicked (inside the activation area). Therefore, users cannot be assisted by Mouseover effects before user clicks the mouse button, because Mouseover will only work when the cursor is over the target (Shih, Hsu, et al., 2009). Therefore, Shih, Chung, Chiang, and Shih (2010) proposed a new operation method, Dual Cursor Automatic Pointing Assistive Program (DCAPAP), where the dual cursors (a virtual cursor and a system cursor) are adopted, offering users an operating environment with Mouseover effects closer to the real conditions to solve this limitation (Shih, Chung, et al., 2010), as shown in Fig. 3. When the system cursor enters into the activation area, with this technology (DCAPAP), it will jump to the target center automatically and activate Mouseover effects. Besides this, the virtual cursor appears to indicate the movement path and users can click as soon as the virtual cursor is inside the activation area. Another limitation of APAP is users cannot significantly improve their pointing efficiency in the case of the targets being too close to each other. In this case, activation areas will be too small, due to the space limitation among targets. When the buttons are too close, as shown in Fig. 4(a), their activation areas are limited to avoid wrong operation due to overlapping. Fig. 4(b) is the operation window for Media Player Classic software (Gabest, 2009) in which the operational buttons are very small and close to each other, leading to the pointing difficulties for persons with disabilities. Therefore, Shih, Li, Shih, Lin, and Lo (2010) presented a new revised operation method, Extended APAP (EAPAP), where the target activation areas with icon and text prompt to indicate their corresponding targets/buttons are moved away from targets to corresponding positions on the screen (Shih, Li, et al., 2010), in order to be large enough for users to point at, thus solving this limitation, as shown in Fig. 5. With the assistance of EAPAP, activation areas in EAPAP are not bound to their corresponding targets, but are placed correspondingly on the screen and use icon and text prompts to indicate their corresponding targets (such as play, pause, stop, etc.). As mentioned above, though the latest research EAPAP can solve the limitation of targets that are too close to each other, this solution still has limitations because the cursor jumps to the target only when the mouse is clicked. Therefore, users cannot be assisted by Mouseover effects before he/she clicks the mouse button (Shih, Chung, et al., 2010; Shih, Hsu, et al., 2009).
1094
C.-H. Shih et al. / Research in Developmental Disabilities 31 (2010) 1091–1101
Fig. 3. The operation flow of Dual Cursor Automatic Pointing Assistive Program (DCAPAP). (a) The externality of the cursor before it enters into the activation area. (b) When the cursor enters into the activation area, the virtual cursor (solid pink cursor) appears to indicate the movement path of the system cursor, and the system cursor jumps to the target center automatically and activates Mouseover effects. (c) The system cursor is locked to the target center while the virtual cursor still moves inside the activation area. (d) Once the virtual cursor moves out of the activation area it disappears; the system cursor replaces it and continues moving. (For interpretation of the references to color in this figure legend, the reader is referred to the web version of the article.)
Fig. 4. Limitations of APAP when targets are too close to each other. (a) When the buttons are too close, their activation areas are limited, this must be adjusted in order to avoid wrong operation due to overlapping. (b) The operation window of Media Player Classic software (Gabest, 2009) whose operational buttons are very small and close to each other, which leads to the pointing difficulties for persons with disabilities.
C.-H. Shih et al. / Research in Developmental Disabilities 31 (2010) 1091–1101
1095
Fig. 5. The operation window of the Media Player Classic (Gabest, 2009) with the assistance of the Extended APAP (EAPAP). In order to be large enough for users to point, the target activation areas with icon and text prompt indicating their corresponding targets/buttons, are moved away from targets to proper positions on the screen.
This limitation of EAPAP can be solved through a new revised operation method, Extended Dual Cursor APAP (EDCAPAP), where Dual Cursor technology (Shih, Chung, et al., 2010) is adopted to implement the function of EAPAP with Mouseover effects. As in EAPAP, the target activation areas with icons, sound, and text prompts to indicate their corresponding targets/ buttons are moved away from targets to corresponding positions on the screen, in order to be large enough for users to point at. As shown in Fig. 6, the system cursor acts as usual before entering into the activation area (Fig. 6(a)). When the system cursor enters into the activation area, the virtual cursor (solid pink cursor) appears and indicates the movement path of the system cursor; the system cursor then jumps to the target center and activates Mouseover effects (the hand shaped cursor, red button, tooltips and sound feedback) automatically (Fig. 6(b)). It is locked to the target center while the virtual cursor still moves inside the activation area. The virtual cursor will disappear once it moves out of the activation area, and the system cursor will replace it and continue moving (Fig. 6(c)). With the assistance of EDCAPAP, each small control button of the fan simulation CAI software has their activation areas which are placed correspondingly on the screen (‘‘On/Off’’, ‘‘Rotate’’, ‘‘High speed’’ and ‘‘Low speed’’) with icon, sound, and text prompts, as shown in Fig. 7. As with DCAPAP, the key technology of EDCAPAP is the mouse movement action interception/detection. Users can neither detect mouse movement nor lock the cursor over targets without this technology application. As a standard device for computers, once the mouse is connected to a computer, the windows OS will identify it and install its driver automatically. The mouse driver is provided by Windows OS or the hardware manufacturer to ensure that the connected device can work normally to define its function as moving, clicking and dragging. Therefore, it is not easy to modify mouse functions (intercept/detect mouse movement action) to meet the needs of EDCAPAP, and there is no research published in this field. Redesigning a mouse driver can reset the mouse functions, turning it into a much more powerful tool, however, this is rarely proposed by researches because of the complexity of the technology required (Microsoft, 2008a, 2008b, 2008c). Only a few recent researches (Shih, Chang, & Shih, 2009, in press; Shih, Cheng, Li, Shih, & Chiang, 2010; Shih, Chiu, et al., 2010; Shih, Chung, et al., 2010; Shih, Hsu, et al., 2009; Shih, Huang, Liao, Shih, & Chiang, 2010; Shih, Li, et al., 2010; Shih & Shih, 2009a, 2009c, 2009d, 2010; Shih, Shih, & Chiang, 2010; Shih, Shih, Lin, & Chiang, 2009) adopted software technology to redesign the mouse driver, visualizing the mouse as a useful tool for many applications dedicated to persons with disabilities (such as hand movement detector, finger/thumb poke detector and pointing/drag-and-drop assistive function, etc.), providing them with additional choices in assistive technology. Therefore, EDCAPAP can be realized using revising Shih’s mouse driver to intercept/detect mouse movement action, in order to help people with disabilities improve their pointing efficiency with Mouseover effects. This work utilizes Shih’s new revised mouse driver (intercepting/detecting driver) design (i.e., a new mouse driver replaces the standard mouse driver, and is able to intercept/detect mouse movement action) to understand the difference between pointing performance before and after for people with developmental disabilities using EDCAPAP, in order to determine whether the EDCAPAP implementation can enhance their pointing performance.
1096
C.-H. Shih et al. / Research in Developmental Disabilities 31 (2010) 1091–1101
Fig. 6. The operation flow of Extended Dual Cursor Automatic Pointing Assistive Program (EDCAPAP). (a) The externality of the cursor before it enters into the activation area. (b) The virtual cursor (solid pink cursor) appears, indicating the movement path of the system cursor, when the cursor enters into the activation area, and the system cursor jumps to the target center automatically, activating Mouseover effects (i.e., the hand shaped cursor, red button, ‘‘on’’ text prompt and sound feedback). It will be locked to the target center while the virtual cursor still moves inside the activation area. (c) The virtual cursor disappears once it moves out of the activation area, and the system cursor replaces it and continues moving. (For interpretation of the references to color in this figure legend, the reader is referred to the web version of the article.)
1. Method 1.1. Participants The participants Tsai and Huang were 17 and 15 years of age, respectively. Tsai’s level of function was estimated to be in the mild range of intellectual disability. He had poor fine motor skills in his hand, but could use his right hand to hold and use
Fig. 7. With the assistance of EAPAP, each small control button of the fan simulation CAI software has its activation area with icon and text prompts, such as on/off, rotate, high speed and low speed, etc.
C.-H. Shih et al. / Research in Developmental Disabilities 31 (2010) 1091–1101
1097
a common mouse. Tsai had little mouse operation experience, but he could move a mouse and click a mouse button in a laborious manner. With the guidance of the research assistant, he learned when he needed to click, but had very low mouse operation efficiency due to poor hand function, and could not use a mouse for an extended amount of time. Huang’s level of function was estimated to be in the profound range of intellectual disabilities. He could control the mouse with both hands. Normally, he used his right hand to operate a mouse, but needed to change to the other hand after a while. He could click, but his mouse operation performance was slower and less accurate than people of the same age when completing pointing tasks. As with Tsai, he knew when to click with the help of the research assistant. He also had low mouse operation efficiency due to poor hand coordination. Both participants were interested in computer operation, and neither had visual disabilities that could be problematic in using a mouse. With the guidance of the research assistant, both learned to move the mouse cursor to the targets and perform the click. Their parents had given formal consent for their involvement in this experiment. 1.2. Apparatus and setting The experiment was carried out in a separate room of the special education school for the mentally retarded. Computers were placed on a computer table, and the screen was at a distance of about 30 cm from their chairs. Mice were provided to the participants when experiment began. 1.2.1. EDCAPAP setting, computer mouse training and test software Computer software was designed, in this study, to provide mouse pointing practice for participants, and to record their test results. The software was designed with two modes, namely practice mode and test mode (Shih, Chung, et al., 2010; Shih, Hsu, et al., 2009; Shih, Li, et al., 2010). Practice mode was designed to provide repeated pointing practice for participants, and test mode was designed to record participants’ successful points within a certain period of time. Fig. 8 shows the flow diagram of the computer mouse training and test software. Eight circular destination targets (T1–T8) with radii of 0.25 cm were set every 458 on a circle with a radius of 1.25 cm, close to each other. Every target, which
Fig. 8. The flow diagram of the computer mouse training and test software. Eight circular destination targets (T1–T8) with radii of 0.25 cm, were set close to each other, every 458, on a circle with a radius of 1.25 cm. Each target had a circular area with a radius of 1.25 cm as the activation area (A1–A8) of EDCAPAP; each of which was also set every 458 on a circle with a radius of 6 cm. Besides this, eight positions, noted P1 through P8 (in accordance with the eight targets), were set as the cursor initial position and displayed every 458 on the edge of the screen.
1098
C.-H. Shih et al. / Research in Developmental Disabilities 31 (2010) 1091–1101
were also set every 458 on a circle, with a radius of 6 cm, had a circular area with a radius of 1.25 cm as the activation area (A1–A8) of EDCAPAP. When the participants moved the cursor into the activation areas (A1–A8), the system cursor jumped to the target center (T1–T8) and the virtual cursor appeared to indicate the movement path. In this way, the participants could perform click inside the circular activation areas (A1–A8) instead of moving the cursor to targets (T1– T8). Eight positions in practice mode, noted from P1 to P8 (in accordance with the eight targets), were set as cursor initial positions and displayed every 458 on the edge of the screen. First, the computer displayed T1, and set the mouse cursor to P1. The participants had to move the mouse cursor from P1 to target T1, and then click to complete a successful pointing. When the task was completed, T1 disappeared, T2 appeared, and the computer automatically set the cursor to P2. Participants would then move the cursor from P2 to T2, and click on it. This process was repeated until the end of the practice time, and the computer provided vocal prompting. Test mode was run under the same conditions as the practice mode, except the targets appeared randomly, and no vocal prompting was provided. Times of successful points within 3 min were recorded. 1.3. Experimental conditions This study used multiple probe design across participants (Richards, Taylor, Ramasamy, & Richards, 1999). Participants typically received 3 training sessions per week, each with about 30 min use of EDCAPAP, for a period of about 6–7 weeks. Tsai initially received three pre-probe sessions during baseline, and intervention began when his performance was consolidated. Huang received discontinuous pre-probe. When Tsai’s performance was consolidated during intervention, new intervention occurred with Huang. The experiment comprised three phases: (a) the baseline phase, in which at least three pre-probe sessions were performed to collect participants’ baseline data; (b) the intervention phase, in which EDCAPAP was adopted to obtain the performance data of EDCAPAP practice for assessment; and (c) the maintenance phase, performed 1 week after intervention finished, in which participants’ follow-up performance was assessed three times. 1.3.1. Baseline The test software was adopted during the baseline to record the successful points within 3 min. In this phase, the EDCAPAP function was turned off; therefore, the participants had to move the cursor from cursor initial positions (P1–P8) to the targets (T1–T8) to click. Three data points were collected for Tsai in 1 week during this phase, while Huang’s baseline data was obtained twice per week discontinuously. 1.3.2. Intervention At the start of the phase, Tsai first received training for EDCAPAP use and training software, and Huang was still being probed discontinuously to collect his baseline data. Huang received EDCAPAP training once Tsai’s performance was consolidated. In principle, 11 30-min practice sessions were performed on each participant during intervention. The arrangement of this 30-min session was as follows: (a) Pointing practice (20 min) The EDCAPAP function was turned on in this phase. Therefore, the participants could perform click inside the circular activation areas (A1–A8) instead of moving the cursor to targets (T1–T8). Destination targets (T1–T8) appeared in order, and participants moved the mouse cursor from cursor initial positions (P1–P8) to the target activation areas (A1–A8) and clicked. During the practice session, computer vocal prompting was available. A research assistant provided guidance to help the participants use EDCAPAP to complete pointing. (b) Rest (7 min) Participants were given 7 min rest after practice. (c) Assessment (3 min) The targets appeared randomly during this phase. Neither instructions from the research assistant nor computer vocal prompting were available. The number of times that each participant successfully pointed within 3 min was recorded as input for assessment, and then adopted to determine whether EDCAPAP improved their pointing efficiency. This phase continued until each participant’s performance was consolidated.
1.3.3. Maintenance This phase began 1 week after the intervention phase to determine whether the participants maintained the skills that they had acquired. Participants did not have EDCAPAP practice during this phase, but participated directly in the pointing test. 2. Results Fig. 9 shows the pointing speed of the two participants after the implementation of EDCAPAP. The curve indicates that both participants improved their pointing efficiency, and maintained their acquired skills during the maintenance phase.
C.-H. Shih et al. / Research in Developmental Disabilities 31 (2010) 1091–1101
1099
Fig. 9. The pointing speeds of the two participants after the implementation of EDCAPAP. As indicated by the curve, both participants improved their pointing efficiency, maintaining their acquired skills during the maintenance phase.
2.1. Tsai 2.1.1. Baseline Tsai could hold and control a common mouse with his right hand. Detailed observation of his mouse operation indicated that he could not point accurately, owing to his poor manual fine motor skills and difficulties in hand control. The cursor often deviated from the target, which made positioning difficult for him. Tsai only achieved a mean of 1.44 correct points per minute during this phase. 2.1.2. Intervention At first, Tsai was unfamiliar with the EDCAPAP function, and had a poor pointing efficiency. Often, unsuccessful pointing arose because of the deviation of the cursor. His correct pointing per minute, however, still increased compared to baseline, because EDCAPAP solved the deviation problem of the cursor when clicking. He gradually mastered the use of EDCAPAP, and increased his pointing efficiency as the practice time accumulated. Fig. 9 shows that Tsai’s pointing speed increase quickly. He achieved 18–24 correct points within 3 min during the 2 sessions in his earlier intervention phase, improved his pointing performance within 3–4 practice sessions, and became able to use EDCAPAP to facilitate the target positioning tasks. Tsai’s correct pointing within 3 min increased to 33–42 during sessions 3–11, revealing that EDCAPAP operation could be mastered easily within a short period of practice. Tsai achieved 6.00–14.00 correct points per min, with an overall mean of 11.42 per min. The Kolmogorov–Smirnov test (Siegel & Castellan, 1988) showed that the increase from baseline to intervention was statistically significant (p < 0.01). 2.1.3. Maintenance Tsai entered maintenance phase 1 week after intervention finished. According to Fig. 9, his correct points within 3 min in 3 sessions during this phase were 53, 48 and 61, respectively. His pointing speed was between 16.00 and 20.33 per min, with an overall mean of 18.00. These results indicate that he retained the skills that he had acquired during the intervention phase. By comparison, it was demonstrated that the performance was better in the maintenance phase than in the intervention phase. This finding may be caused by the fact that Tsai apparently did not focus on the training for the last few minutes of the session during the intervention phase, making his performance worse than expected. His direct access to the test and the short attention time needed could have improved, in contrast, his performance during the maintenance phase. 2.2. Huang 2.2.1. Baseline Huang could control the mouse, but had few successful points because of the small size of targets. His successful points within 3 min were 8, 6 and 10, respectively. He also had difficulty in controlling the movement of the mouse cursor while simultaneously clicking, due to poor hand-eye coordination. Huang achieved a mean of 2.67 correct points per minute during the baseline phase.
1100
C.-H. Shih et al. / Research in Developmental Disabilities 31 (2010) 1091–1101
2.2.2. Intervention Huang mastered the use of EDCAPAP at the beginning, but could not concentrate on the use of the mouse, and needed to change hands to control the mouse. Gradually, he could concentrate on using the mouse, and his pointing speed rose slowly, according to Fig. 9. He achieved 6.33–16.67 correct points per min during this phase, with an overall mean of 10.33. The Kolmogorov–Smirnov test (Siegel & Castellan, 1988) showed that the increase from baseline to intervention was statistically significant (p < 0.01). 2.2.3. Maintenance Huang’s pointing speed in this phase was close to that during the intervention phase. He began this phase 1 week following intervention. He could finish pointing without changing hands to operate the mouse, and concentrate on mouse use in the last 3 min session. During this phase (including 3 sessions), his activity data (correct points within 3 min) were 53, 41 and 44, respectively, and he achieved 13.67–17.67 correct points per min. His pointing speed in this phase was close to that during the intervention phase. These results indicate that he retained the skills that he had acquired during the intervention phase. 3. Discussion This work has demonstrated that both participants rapidly improved their efficiency to point at targets which were small and close to each other with Mouseover effects after receiving EDCAPAP training, and retained their acquisition skills in the maintenance phase. Results of this study also show that extensive practice time is not required for people with developmental disabilities in order to use EDCAPAP. With the assistance of EDCAPAP, participants can position targets quickly, easily, and accurately, thus helping the disabled to solve their pointing problems. The two participants could operate some ordinary CAI/educational software with EDCAPAP after the experiment. This EDCAPAP is a software-based solution, so it does not need extra hardware or circuit preservations, and can be widely spread and popularized by internet; it can support all standard interfaces of commercial input devices (i.e., trackball) that are compatible with the computer, including USB, wireless and Bluetooth interfaces (Bluetooth.org, 2009). In addition, EDCAPAP is compatible with all currently available software, so the current software can be applied to improve the pointing efficiency of people with disabilities without being modified or rewritten. This study, focusing on individuals with developmental disabilities who cannot use standard mice to perform pointing efficiently, only considers pointing. It does not address other mouse controls needing higher physical abilities, like doubleclicking, dragging and scrolling. Pointing is sufficient, however, for the growing number of well-designed educational programs (Donker & Reitsma, 2007a; Shimizu & McDonough, 2006). Further studies are necessary to develop additional mouse applications, with corresponding training and test software to satisfy the needs of different levels of disabilities. Hopefully, the implementation of EDCAPAP can cover all complex mouse movements and provide disabled users with additional choices in assistive technology. References Abascal, J., & Nicolle, C. (2005). Moving towards inclusive design guidelines for socially and ethically aware HCI. Interacting with Computers, 17, 484–505. Ahlstrom, D. (2005). Modeling and improving selection in cascading pull-down menus using Fitts’ law, the steering law and force fields. Paper Presented at the Proceedings of the CHI’05 Conference on Human Factors in Computing Systems. Ahlstrom, D., Hitz, M., & Leitner, G. (2006). An evaluation of sticky and force enhanced targets in multi target situations. Paper Presented at the Proceedings of the 4th Nordic conference on Human–Computer Interaction: Changing Roles. Akamatsu, M., & MacKenzie, I. S. (2002). Changes in applied force to a touchpad during pointing tasks. International Journal of Industrial Ergonomics, 29, 171–182. Bluetooth.org. (2009). Bluetooth.org—Welcome to Bluetooth.org. Retrieved 6/18, 2009, from http://www.bluetooth.org/apps/content/. Brodwin, M. G., Star, T., & Cardoso, E. (2004). Computer assistive technology for people who have disabilities: Computer adaptations and modifications. Journal of Rehabilitation, 70, 28–33. Casiez, G., Vogel, D., & Balakrishnan, R. (2008). The impact of control-display gain on user performance in pointing tasks. Human–Computer Interaction, 23, 215– 250. Cockburn, A., & Brewster, S. (2005). Multimodal feedback for the acquisition of small targets. Ergonomics, 48, 1129–1150. Cockburn, A., & Firth, A. (2003). Improving the acquisition of small targets. Paper Presented at the Proceedings of the 17th Annual Human–Computer Interaction Conference. Davies, D. K., Stock, S. E., & Wehmeyer, M. L. (2002a). Enhancing independent task performance for individuals with mental retardation through use of a handheld self-directed visual and audio prompting system. Education and Training in Mental Retardation and Developmental Disabilities, 37, 209–218. Davies, D. K., Stock, S. E., & Wehmeyer, M. L. (2002b). Enhancing independent time-management skills of individuals with mental retardation using a palmtop personal computer. Mental Retardation, 40, 358–365. Dennerlein, J. T., & Yang, M. C. (2001). Haptic force-feedback devices for the office computer: Performance and musculoskeletal loading issues. Human Factors, 43, 278. Donker, A., & Reitsma, P. (2007a). Aiming and clicking in young children’s use of the computer mouse. Computers in Human Behavior, 23, 2863–2874. Donker, A., & Reitsma, P. (2007b). Young children’s ability to use a computer mouse. Computers and Education, 48, 602–617. Gabest. (2009). Media Player Classic – Home Cinema – Video Player. Retrieved June 5, 2009, from http://mpc-hc.sourceforge.net/. Grossman, T., & Balakrishnan, R. (2005). The bubble cursor: enhancing target acquisition by dynamically resizing of the cursor’s activation area. Paper presented at the Proceedings of CHI’05 Conference on Human Factors in Computing Systems. Hedrick, B., Pape, T. L., Heinemann, A. W., Ruddell, J. L., & Reis, J. (2006). Employment issues and assistive technology use for persons with spinal cord injury. Journal of Rehabilitation Research and Development, 43, 185. Langone, J., Clees, T. J., Rieber, L., & Matzko, M. (2003). The future of computer-based interactive technology for teaching individuals with moderate to severe disabilities: Issues relating to research and practice. Journal of Special Education Technology, 18, 5–16. Mann, W. C., Belchior, P., Tomita, M. R., & Kemp, B. J. (2005). Computer use by middle-aged and older adults with disabilities. Technology and Disability, 17, 1–9.
C.-H. Shih et al. / Research in Developmental Disabilities 31 (2010) 1091–1101
1101
Microsoft. (2008a). WDM: Introduction to Windows Driver Model. Retrieved 6/18, 2008, from http://www.microsoft.com/whdc/archive/wdm.mspx#EJE. Microsoft. (2008b). Windows Driver Kit: Human input devices. Retrieved 5/30, 2008, from http://msdn.microsoft.com/en-us/library/ms794089.aspx. Microsoft. (2008c). Windows Management Instrumentation. Retrieved 6/18, 2008, from http://msdn.microsoft.com/en-us/library/aa394582(VS.85).aspx. Park, J., Han, S. H., & Yang, H. (2006). Evaluation of cursor capturing functions in a target positioning task. International Journal of Industrial Ergonomics, 36, 721– 730. Rao, R. S., Seliktar, R., & Rahman, T. (2000). Evaluation of an isometric and a position joystick in a target acquisition task for individuals with cerebral palsy. IEEE Transactions on Rehabilitation Engineering, 8, 118–125. Resta, P., & Laferriere, T. (2007). Technology in support of collaborative learning. Educational Psychology Review, 19, 65–83. Richards, S. B., Taylor, R. L., Ramasamy, R., & Richards, R. Y. (1999). Single subject research: Applications in educational and clinical settings. New York: Wadsworth. Ritchie, H., & Blanck, P. (2003). Promise of the Internet for disability: A study of online services and accessibility of centers for independent living Web sites. Behavioral Sciences and the Law, 21, 5–26. Salminen, A. L., Petrie, H., & Ryan, S. (2004). Impact of computer augmented communication on the daily lives of speech-impaired children. Part I: Daily communication and activities. Technology and Disability, 16, 157–167. Shein, G. F., Treviranus, J., Brownlow, N. D., Milner, M., & Parnes, P. (1992). An overview of human–computer interaction techniques for people with physical disabilities. International Journal of Industrial Ergonomics, 9, 171–181. Shih, C.-H., Chang, M.-L., & Shih, C.-T. (2009). Assisting people with multiple disabilities and minimal motor behavior to improve computer pointing efficiency through a mouse wheel. Research in Developmental Disabilities, 30, 1378–1387. Shih, C.-H., Chang, M.-L., & Shih, C.-T. (in press). A new limb movement detector enabling people with multiple disabilities to control environmental stimulation through limb swing with a gyration air mouse. Research in Developmental Disabilities. doi:10.1016/j.ridd.2010.01.020. Shih, C.-H., Cheng, H.-F., Li, C.-C., Shih, C.-T., & Chiang, M.-S. (2010). Assisting people with developmental disabilities improve their collaborative pointing efficiency with a Multiple Cursor Automatic Pointing Assistive Program. Research in Developmental Disabilities, 31, 600–607. Shih, C.-H., Chiu, S.-K., Chu, C.-L., Shih, C.-T., Liao, Y.-K., & Lin, C.-C. (2010). Assisting people with multiple disabilities improve their computer pointing efficiency with hand swing through a standard mouse. Research in Developmental Disabilities, 31, 517–524. Shih, C.-H., Chung, C.-C., Chiang, M.-S., & Shih, C.-T. (2010). Assisting people with developmental disabilities to improve pointing efficiency with a Dual Cursor Automatic Pointing Assistive Program. Research in Developmental Disabilities, 31, 151–159. Shih, C.-H., Hsu, N.-Y., & Shih, C.-T. (2009). Assisting people with developmental disabilities to improve pointing efficiency with an Automatic Pointing Assistive Program. Research in Developmental Disabilities, 30, 1212–1220. Shih, C.-H., Huang, H.-C., Liao, Y.-K., Shih, C.-T., & Chiang, M.-S. (2010). An Automatic Drag-and-Drop Assistive Program developed to assistive people with developmental disabilities to improve drag-and-drop efficiency. Research in Developmental Disabilities, 31, 416–425. Shih, C.-H., Li, C.-C., Shih, C.-T., Lin, K.-T., & Lo, C.-S. (2010). Extended Automatic Pointing Assistive Program—A pointing assistance program to help people with developmental disabilities improve their pointing efficiency. Research in Developmental Disabilities, 31, 672–679. Shih, C.-H., & Shih, C.-T. (2009a). Assisting people with multiple disabilities to use computers with multiple mice. Research in Developmental Disabilities, 30, 746– 754. Shih, C.-H., & Shih, C.-T. (2009b). Development of a computer input system for persons with disabilities using a commercial mouse and switches. Disability and Rehabilitation: Assistive Technology, 4, 414–421. Shih, C.-H., & Shih, C.-T. (2009c). Development of an integrated pointing device driver for the disabled. Disability and Rehabilitation: Assistive Technology, doi:10.1080/17483100903105599. Shih, C.-H., & Shih, C.-T. (2009d). A new movement detector to enable people with multiple disabilities to control environmental stimulation with hand swing through a commercial mouse. Research in Developmental Disabilities, 30, 1196–1202. Shih, C.-H., & Shih, C.-T. (2010). Assisting two children with multiple disabilities and minimal motor behavior to control environmental stimulation with thumb poke through a trackball. Behavioural and Cognitive Psychotherapy, 38, 211–219. Shih, C.-H., Shih, C.-T., & Chiang, M.-S. (2010). A new standing posture detector to enable people with multiple disabilities to control environmental stimulation by changing their standing posture through a commercial Wii Balance Board. Research in Developmental Disabilities, 31, 281–286. Shih, C.-H., Shih, C.-T., Lin, K.-T., & Chiang, M.-S. (2009). Assisting people with multiple disabilities and minimal motor behavior to control environmental stimulation through a mouse wheel. Research in Developmental Disabilities, 30, 1413–1419. Shih, C.-H., Shih, C.-T., & Luo, C.-H. (2007). Design of a fast switch-control computer input device using two-grade movement method for people with physical disabilities. WSEAS Transactions on Computers, 6, 1140–1146. Shimizu, H., & McDonough, C. S. (2006). Programmed instruction to teach pointing with a computer mouse in preschoolers with developmental disabilities. Research in Developmental Disabilities, 27, 175–189. Siegel, S., & Castellan, N. J. (1988). Nonparametric statistics for the behavioral sciences. New York: McGraw-HiU Book Company. Stern, S. E. (2008). Computer-synthesized speech and perceptions of the social influence of disabled users. Journal of Language and Social Psychology, 27, 254. Tu, J., Tao, H., & Huang, T. (2007). Face as mouse through visual face tracking. Computer Vision and Image Understanding, 108, 35–40. Wikipedia. (2009). Mouseover. Retrieved March 10, 2009, from http://en.wikipedia.org/wiki/Mouseover. Wong, A. W. K., Chan, C. C. H., Li-Tsang, C. W. P., & Lam, C. S. (2009). Competence of people with intellectual disabilities on using human–computer interface. Research in Developmental Disabilities, 30, 107–123.