- Email: [email protected]
An automatic Drag-and-Drop assistive program developed to assistive people with developmental disabilities to improve Drag-and-Drop efficiency
Research in Developmental Disabilities 31 (2010) 416–425
Contents lists available at ScienceDirect
Research in Developmental Disabilities
An automatic Drag-and-Drop assistive program developed to assistive people with developmental disabilities to improve Drag-and-Drop efficiency Ching-Hsiang Shih a,*, Hsun-Chin Huang a, Yung-Kun Liao a, Ching-Tien Shih b, Ming-Shan Chiang a a b
Department of Special Education, National Dong Hwa University, Hualien, Taiwan, ROC Department of Electronics Engineering and Computer Science, Tung-Fang Institute of Technology, Kaohsiung Country, Taiwan, ROC
A R T I C L E I N F O
A B S T R A C T
Article history: Received 29 September 2009 Accepted 5 October 2009
The latest researches adopted software technology to improve pointing performance; however, Drag-and-Drop (DnD) operation is also commonly used in modern GUI programming. This study evaluated whether two children with developmental disabilities would be able to improve their DnD performance, through an Automatic DnD Assistive Program (ADnDAP). At first, both participants had their baseline sessions. Then the first participant entered into intervention. New intervention began with the second participant when his performance was consolidated. Finally, maintenance phase occurred with both participants, in which their DnD performance improved significantly. Data showed that both participants improved their DnD efficiency with the assistance of ADnDAP, and remained highly successful through maintenance phase. Implications of the findings are discussed. ß 2009 Elsevier Ltd. All rights reserved.
Keywords: Developmental disabilities DnD ADnDAP Mouse driver
To increase the functional abilities, independence, and access to mainstream society for people with disabilities, computer technologies have a tremendous potential and create a method of equalization between people with and without disabilities (Bradley & Poppen, 2003; Brodwin, Star, & Cardoso, 2004; Houlihan et al., 2003; Lee & Vail, 2005; Wong, Chan, Li-Tsang, & Lam, 2009). The benefits are clearer when persons with disabilities are given the opportunity to improve their level of competency in controlling computer through being offered with special interfaces (hardware assistance) or assistive program (software assistance) to improve their operation efficiency (Brodwin et al., 2004; Davies, Stock, & Wehmeyer, 2002a; Davies, Stock, & Wehmeyer, 2002b; Mann, Belchior, Tomita, & Kemp, 2005; Ritchie & Blanck, 2003). However, because most commercial computer input devices are targeted at the mainstream population, without providing the type of accommodation that meets the needs or desires of people with disabilities (Abascal & Nicolle, 2005; Mann et al., 2005), it is difficult or impossible for them to operate computers by a mouse, or similar pointing devices (Brodwin et al., 2004; Rao, Seliktar, & Rahman, 2000). Based on this, various modifications and adaptations of computer-pointing devices have been proposed to meet the needs of people with multiple disabilities (Brodwin et al., 2004; Hedrick, Pape, Heinemann, Ruddell, & Reis, 2006; Mann et al., 2005; Shein, Treviranus, Brownlow, Milner, & Parnes, 1992; Tu, Tao, & Huang, 2007). Normally, their cost is much higher than usual
* Corresponding author. Tel.: +886 3 8227106x1320; fax: +886 3 8228707. E-mail address: [email protected] (C.-H. Shih). 0891-4222/$ – see front matter ß 2009 Elsevier Ltd. All rights reserved. doi:10.1016/j.ridd.2009.10.004
C.-H. Shih et al. / Research in Developmental Disabilities 31 (2010) 416–425
417
devices, due to their specific design. Besides, their long-term use by people with disabilities is limited, since they are often more difficult to obtain or maintain than ordinary commercial devices. Shih and Shih (2009a, 2009c) presented a special interface through multi-mice configuration which adopted software technology to redesign the mouse driver, in order to enable physically disabled people to complete mouse operation through exporting their remaining ability of each limb with several mice. For example, the right hand could control a mouse left-toright movement, the left hand controls the up-to-down movement, and the mouse button is pressed by the left toe (Shih & Shih, 2009a). The function of each mouse is adjusted individually by Shih’s mouse drive to achieve the above requested functions. Therefore, with the assistance of multi-mice configuration, people with disabilities can as with people without disabilities, can use a very common, cheap and powerful commercial mice, instead of specialized alternative computer input devices. Many researchers have also proposed assistive programs to facilitate the quality of pointing (target positioning/ acquiring) operation, in order to improve the operation efficiency of people with disabilities (Ahlstrom, 2005; Ahlstrom, Hitz, & Leitner, 2006; Akamatsu & MacKenzie, 2002; Casiez, Vogel, & Balakrishnan, 2008; Cockburn & Brewster, 2005; Cockburn & Firth, 2003; Dennerlein & Yang, 2001; Grossman & Balakrishnan, 2005; Park, Han, & Yang, 2006). Pointing, which is achieved by moving the cursor over certain areas or icons and clicking, is adopted by most computer programs and CAI software as the most common basic mouse operation (Donker & Reitsma, 2007a, 2007c; Shimizu & McDonough, 2006). Users can gain from being provided useful functions in pointing, such as moving the cursor to the target center automatically, to position the target quickly, easily, and accurately (Grossman & Balakrishnan, 2005; Park et al., 2006). The latest research adopted mouse driver technology to recommend a new operation method, Automatic Pointing Assistive Program (APAP), where the user can click the mouse button when the cursor is near the target (inside the activation area), instead of moving the cursor to the target (Shih, Hsu, & Shih, 2009), to improve the users’ pointing efficiency. Mouse click action will be intercepted as soon as the mouse is clicked, the cursor jumps to the target center automatically, and then the intercepted mouse click action will be sent out, as shown in Fig. 1.However, in APAP, the cursor jumps to the target only when the mouse is clicked, and users can not be assisted by Mouseover effects which are commonly used in modern Graphical User Interface (GUI) programming, by which an element changes in response to the mouse cursor moving over it (Wikipedia, 2009c), before he/she clicks the mouse button (Shih, Hsu, et al., 2009). Therefore, Shih, Chung, Chiang, and Shih (2009) presented a new revised operation method, Dual Cursor Automatic Pointing Assistive Program (DCAPAP) as APAP’s development, where the dual cursors (a virtual cursor and a system cursor) are adopted to offer users an operating environment which is closer to the real conditions, as shown in Fig. 2. With this technology (DCAPAP), when the system cursor enters into the activation area, it will jump to the target center automatically and activate Mouseover effects. Besides, the virtual cursor appears to indicate the movement path and users can click as soon as the virtual cursor is inside the activation area. In addition, a mouse wheel was also used as a pointing assistive device to improve pointing performance of people with multiple disabilities who have minimal motor behavior (i.e., people who have extensive paralysis of their body and can effectively control only very limited movements), through a new operation method, Dynamic Pointing Assistive Program (DPAP), where the user can poke his/her thumb/finger to rotate a mouse wheel to move a cursor to a target (Shih, Chang, &
Fig. 1. The operation flow of Automatic Pointing Assistive Program (APAP). (a) When the cursor enters into the activation area, mouse click action will be intercepted as soon as the mouse is clicked. (b) The cursor jumps to the target center automatically. (c) The intercepted mouse click action is sent out.
418
C.-H. Shih et al. / Research in Developmental Disabilities 31 (2010) 416–425
Fig. 2. The operation flow of Dual Cursor Automatic Pointing Assistive Program (DCAPAP). (a) The externality of the cursor before it enters into the activation area. (b) When the cursor enters into the activation area, the virtual cursor (solid pink cursor) appears to indicate the movement path of the system cursor, and the system cursor jumps to the target center automatically. (c) The system cursor is locked to the target center when the virtual cursor moves inside the activation area. (d) The virtual cursor disappears once it moves out of the activation area, and the system cursor replaces it and continues moving (For interpretation of the references to colour in this figure legend, the reader is referred to the web version of the article.).
Shih, 2009). As soon as the mouse wheel is rotated, its action will be intercepted, and the cursor will automatically jump to a series of pre-defined target positions in order, according to the wheel rotation amount and direction, as shown in Fig. 3. With the assistance of DPAP, people with profound multiple disabilities and minimal motor behavior can use the mouse wheel to point quickly, easily, and accurately, thus helping them to solve their pointing problems.Except pointing, Drag-and-Drop (DnD) operation is also commonly used in modern GUI programming, by which an object is clicked and dragged to a different location or onto another object (Wikipedia, 2009b). The basic sequence involved in DnD is: (a) press and hold down the button on the mouse, to ‘‘grab’’ the object; (b) drag the object to the desired location; (c) drop the object by releasing the button. Dragging requires more physical effort than moving the same pointing device without holding down any buttons (Wikipedia, 2009b). Fig. 4 shows a DnD CAI software in which users need to drag 2 animals onto the pods, and then pull the lever to splice them together (SugarVsSpice, 2009). Many CAI programs require dragging operation, however, people with disabilities cannot use these programs due to their physical limitations (Donker & Reitsma, 2007b). This can be solved through a new revised operation method, Automatic DnD Assistive Program (ADnDAP), where the complex dragging process is replaced by simple clicking operation. Users can click mouse button when the cursor is near the target (entering into the activation area, Fig. 5(a)), the cursor jumps to the target center and the mouse button is held down automatically (i.e., ‘‘grab’’ the target, Fig. 5(b)). The cursor and target jump to the destination with the button held down (i.e., ‘‘Drag’’ the target to the desired location, Fig. 5(c)). Users release the button to finish dragging (i.e., ‘‘Drop’’ the target by releasing the button, Fig. 5(d)). With this technology (ADnDAP), when users move the cursor into an activation area and click, the target will be dragged to the destination automatically. In this way, people with disabilities can extend their computer operation to DnD, besides pointing. The same as APAP, the key technology of ADnDAP is the mouse click action interception which avoids error click through preventing error commands sent to the computer, when the button is pressed before the cursor arrives to the target. Without this technology, users can neither intercept mouse click nor drag the target to the destination automatically through mouse clicking. ADnDAP is able to run independently in the windows OS environment, works without interference with all currently available software (i.e., the currently available software does not need to be modified or rewritten).
C.-H. Shih et al. / Research in Developmental Disabilities 31 (2010) 416–425
419
Fig. 3. The operation flow of Dynamic Pointing Assistive Program (DPAP). (a) A user pokes his/her thumb/finger to rotate a mouse wheel to move a cursor to a target. (b) The four pre-defined targets which were noted as P1, P2, P3 and P4 in Dynamic Pointing Assistive Program (DPAP). Users poke mouse wheel will quickly move the cursor among the four targets. Each forward poke will jump the cursor from one to a next target in order of P1 ! P2 ! P3 ! P4 ! P1 ! , whereas, backward poke jumps the cursor in order of P4 ! P3 ! P2 ! P1 ! P4 ! .
Fig. 4. A DnD CAI software in which users drag 2 animals onto the pods, and then pull the lever to splice them together (SugarVsSpice, 2009).
420
C.-H. Shih et al. / Research in Developmental Disabilities 31 (2010) 416–425
Fig. 5. The operation flow of Automatic DnD Assistive Program (ADnDAP). (a) The user can click the mouse button when the cursor is near the target (entering into the activation area). (b) The cursor jumps to the target center and the mouse button is held down automatically (i.e., ‘‘grab’’ the target). (c) The cursor and target jump to the destination with the button held down (i.e., ‘‘Drag’’ the target to the desired location). (d) Dragging is finished when users release the button (i.e., ‘‘Drop’’ the target by releasing the button).
Every hardware device linked to a computer requires a software-based driver to work normally. Writing a device driver requires an in-depth understanding of how the hardware and the software of a given platform function. Drivers operating in a highly privileged environment can cause disaster (crash a system) if they get things wrong (erroneously programmed). Normally, the device driver is provided by Windows OS or the hardware manufacture, to ensure that the connected device can function properly (Wikipedia, 2009a). Driver modification is rarely proposed by researches because of the complexity of the technology required (Microsoft, 2008; Wikipedia, 2009a). Only a few of recent researches (Shih, Chang, et al., 2009; Shih, Chung, et al., 2009; Shih, Hsu, et al., 2009; Shih & Shih, 2009a, 2009b, 2009c, 2009d, in press; Shih, Shih, Lin, & Chiang, 2009) adopted software technology to redesign the mouse driver in order to reset mouse functions, and took the mouse as a useful tool for many applications dedicated to persons with disabilities, providing them with additional choices in assistive technology. As a standard device for computers, once a mouse is connected to a computer, its function will be defined as moving, clicking and dragging. As a result, it is not easy to intercept mouse click, and no research adapting driver technology to assist DnD is published in this field. This work adopts Shih’s new revised mouse driver (intercepting driver) design (i.e., a new mouse driver replaces standard mouse driver, and is able to intercept mouse click action) to help people with developmental disabilities improve their DnD efficiency, and to understand the difference of DnD performance between before and after for them using ADnDAP, in order to determine whether the ADnDAP implementation can enhance their DnD performance. 1. Method 1.1. Participants The participants Hsueh and Liao were 14 and 13 years of age, respectively. Both of their levels of functioning were estimated to be in the middle range of intellectual disability. Hsueh could click a mouse with right hand, but had difficulties in positioning, due to poor hand–eye coordination. He failed to drag because he released the mouse button before the object arrived to its destination or onto another object. Liao used right hand to operate a mouse, but had poor positioning performance. He also had difficulties in dragging objects to desired destinations due to poor hand–eye coordination.
C.-H. Shih et al. / Research in Developmental Disabilities 31 (2010) 416–425
421
Fig. 6. The flow diagram of the computer mouse training and test software. Eight circular source targets (S1–S8) with radius of 0.25 cm were set every 458 on a circle with a radius of 5 cm. Each source target (S1–S8) has a circular area with a radius of 1.25 cm as the activation areas (A1–A8) of ADnDAP.
Both participants were interested in computer operation, and were capable of speaking and following instructions. No one had visual or hearing impairment that could be a problem in using a mouse. With the guidance of the research assistant, both of them learnt to move the mouse cursor to targets and perform a click. Informed consent was obtained from the children’s parents prior to their participation in the study. 1.2. Apparatus and setting The study was carried out in an activity room. Computers were placed on a computer table, and the screen was at a distance of about 30 cm from their chairs. Wireless mice were provided to the participants when the experiment began. 1.3. ADnDAP setting, computer mouse training and test software This study designed a test software with two modes (practice mode and record mode) to provide the participants with mouse DnD practice, and to record their test results. Practice mode gave repeated DnD practice for the participants, and record mode recorded participants’ successful DnD number within a certain period of time. Fig. 6 is the flow diagram of the computer mouse training and test software. Eight circular source targets (S1–S8) with radius of 0.25 cm were set every 458 on a circle with a radius of 5 cm. Each source target (S1–S8) has a circular area with a radius of 1.25 cm as the activation areas (A1–A8) of ADnDAP. When the participants moved the cursor into the activation areas (A1–A8) and clicked, the cursor jumped to the target center (S1–S8) and dragged the target to the destination point (D, the center of the circumference) automatically. In this way, the participants could click inside the circular activation areas (A1–A8) instead of moving the cursor to source targets (S1–S8), and could drag it to destination point (D) at the same time. In practice mode, the computer first displayed source target S1, and set the mouse cursor to the destination point (D). The participants had to move the mouse cursor from the destination point to source target S1, then dragged S1 and dropped it to destination point (D) to complete a successful DnD. Source target S1 then disappeared, and source target S2 appeared. Participants would then move the mouse cursor from the destination point (D) to source target S2 to drag it again. This process was repeated until the end of practice time. Record mode was run under the same conditions as the practice mode, except the targets appeared randomly. Times of successful DnD number within 3 min were recorded. 1.4. Experimental conditions This study used multiple probe design across participants (Richards, Taylor, Ramasamy, & Richards, 1999). The participants individually received 3 training sessions per week, each with about 30 min use of ADnDAP, for a period of about
422
C.-H. Shih et al. / Research in Developmental Disabilities 31 (2010) 416–425
6–7 weeks. Hsueh had three pre-probe sessions during baseline, and new intervention occurred with him when his performance was consolidated. Liao received discontinuous pre-probe in baseline, and entered into intervention when Hsueh’s intervention performance was consolidated. The experiment included three phases: (a) baseline, in which at least 3 times pre-probe sessions were performed to collect participants’ baseline data; (b) intervention, in which ADnDAP were used to obtain the performance data of ADnDAP practice for assessment, and (c) maintenance, performed 1 week after intervention finished, in which participants’ follow-up performance was assessed 3 times. 1.5. Baseline The test software was used to record the successful DnD number within 3 min during the baseline. The ADnDAP function was turned off, so the participants had to move the cursor from the destination point (D) to source target (S1–S8), and dragged it to the destination point (D). Three data points were obtained for Hsueh in 1 week during this phase, while Liao’s baseline data was collected twice per week discontinuously. 1.6. Intervention At the beginning, Hsueh got trained for ADnDAP use and training software, and Liao was still being probed discontinuously to collect his baseline data. ADnDAP training occurred with Liao once Hsueh’s performance was consolidated. Principally, each participant performed 11 practice sessions of 30 min during intervention. The arrangement of this 30-min session was as follows: (a) DnD practice (20 min) Source targets (S1–S8) appeared in order, and participants moved the mouse cursor from the destination point (D) to the target activation areas (A1–A8) and clicked, and ADnDAP finished dragging automatically. The research assistant provided vocal prompting and guidance during this phase to help the participants complete DnD through ADnDAP. (b) Rest (7 min) Participants were given 7 min rest after practice. (c) Assessment (3 min) The source targets (S1–S8) appeared randomly during this phase. Neither vocal prompting nor instructions from the research assistant were available. Each participant’s successful DnD within 3 min were recorded as input for assessment, and then used to determine whether ADnDAP improved their DnD efficiency. This phase continued until each participant’s performance was consolidated. 1.7. Maintenance This phase began 1 week after the intervention phase to determine whether the participants maintained the skills that they had acquired. During this phase, participants did not have ADnDAP practice, but participated directly in the 3-min DnD test. 2. Results Fig. 7 indicated the two participants’ DnD speed after the implementation of ADnDAP. The curve showed that both participants improved their DnD efficiency, and maintained their acquired skills, during the maintenance phase. 2.1. Baseline Hsueh could control the mouse with his right hand. Detailed observation of his mouse operation demonstrated that he had difficulties in both target positioning and dragging, owing to his difficulties of hand control and poor hand–eye coordination. The cursor often deviated from the target when he clicked the mouse, and he had to spend a lot of time to aim at the target again. He did not have any successful dragging in the first 2 sessions during his baseline phase, because he released mouse button before the cursor arrived to destinations. He only had 2 successful DnD in the third session during this phase. Liao also had difficulty in controlling the movement of the mouse cursor at the same time of clicking, and could not drag objects to destinations before released mouse button, owing to poor hand–eye coordination. He had poor performance both in target positioning and dragging during the baseline phase. 2.2. Intervention At first, Hsueh was unfamiliar with the ADnDAP function, and continued to click on the target rather than the activation area after ADnDAP practice. However, his correct DnD per minute still increased compared to baseline, because ADnDAP avoided cursor deviation when clicking, and dragged the target to the destination automatically. With the assistance of the
C.-H. Shih et al. / Research in Developmental Disabilities 31 (2010) 416–425
423
Fig. 7. The DnD speeds of the two participants after the implementation of ADnDAP. The curve indicates that both two participants improved their DnD efficiency, and maintained their acquired skills, during the maintenance phase.
research guide, he gradually learnt the use of ADnDAP (click inside the activation area), and increased his DnD efficiency as the practice time accumulated. Fig. 7 shows that his DnD speed increased quickly. He achieved 46–72 correct DnD within 3 min during the 5 sessions in his earlier intervention phase, improved his DnD efficiency within 6–9 practice sessions, and became able to use ADnDAP to drag targets on the screen. Hsueh’s correct DnD within 3 min increased to 85–86 during sessions 10 and 11, revealing that ADnDAP operation could be mastered easily within a short period of practice. He achieved 15.33–29.00 correct DnD per min, with an overall mean of 23.15 min1. The Kolmogorov–Smirnov test (Siegel & Castellan, 1988) showed that the increase from baseline to intervention was statistically significant (p < 0.01). Liao significantly increased his DnD efficiency with the assistance of ADnDAP. After several practice, his DnD speed rose quickly, and he achieved 24.67–39.33 correct DnD per min during this phase, with an overall mean of 35.88, as shown in Fig. 7. The Kolmogorov–Smirnov test (Siegel & Castellan, 1988) showed that the increase from baseline to intervention was statistically significant (p < 0.01). 2.3. Maintenance The maintenance phase occurred with Hsueh 1 week after intervention finished. As shown in Fig. 7, his DnD performance in this phase was close to that during the intervention phase, and his correct DnD within 3 min in 3 sessions during this phase were 91, 86 and 88, respectively. His DnD speed was between 28.67 and 30.33 min1, with an overall mean of 29.44. These results indicated that he retained the skills that he had acquired during the intervention phase. Liao entered into this phase 1 week after intervention. During this phase (including 3 sessions), his correct DnD within 3 min were 98, 102 and 100, respectively, and he achieved 32.67–34.00 correct DnD per min. These results indicated that he also kept the skills that he had acquired during intervention. 3. Discussion Dragging a target to a destination or onto a target is a frequently adopted basic mouse operation for most computer programs and CAI software (Donker & Reitsma, 2007b). However, people with disabilities can find this is a challenging task because it requires more physical effort than pointing. ADnDAP can give effective assistance to these users to realize target acquisition and DnD at the same time. This study has indicated that both participants greatly improved their efficiencies to perform DnD after receiving ADnDAP training, and retained their acquisition skills in the maintenance phase. Analytical results demonstrate that people with developmental disabilities can easily master ADnDAP without long-period practice. Both participants could operate some educational/CAI software which require DnD operation through ADnDAP after the experiment. Participants were able to drag targets quickly, easily, and accurately, with the assistance of ADnDAP. ADnDAP keeps the merit of APAP to avoid cursor deviation (Shih, Hsu, et al., 2009), and is based on software solution which does not need extra hardware or circuit preservations, and can support all standard interfaces of commercial pointing devices (i.e., mouse and
424
C.-H. Shih et al. / Research in Developmental Disabilities 31 (2010) 416–425
trackball) that are compatible with the computer, including USB, wireless and Bluetooth (Bluetooth.org, 2009) interfaces. In addition, ADnDAP is compatible with all currently available software, so the current software can be applied to improve the pointing efficiency of people with disabilities without being modified or rewritten. This study only considers dragging targets to fixed destinations, focusing on individuals with developmental disabilities, who cannot use a standard mouse to perform DnD efficiently. Further studies are necessary to develop additional mouse applications to extend current functionality (i.e., users can freely define their desired destinations) and satisfy the needs of different levels of disabilities. Hopefully, the implementation of ADnDAP can cover all complex DnD operations and provide disabled users with additional choices in computer assistive technology. References Abascal, J., & Nicolle, C. (2005). Moving towards inclusive design guidelines for socially and ethically aware HCI. Interacting with Computers, 17, 484–505. Ahlstrom, D. (2005). Modeling and improving selection in cascading pull-down menus using Fitts’ law, the steering law and force fields. Paper presented at the proceedings of the CHI’05 conference on human factors in computing systems. Ahlstrom, D., Hitz, M., & Leitner, G. (2006). An evaluation of sticky and force enhanced targets in multi target situations. Paper presented at the proceedings of the 4th Nordic conference on human–computer interaction: Changing roles. Akamatsu, M., & MacKenzie, I. S. (2002). Changes in applied force to a touchpad during pointing tasks. International Journal of Industrial Ergonomics, 29, 171–182. Bluetooth.org. (2009). Bluetooth.org—Welcome to Bluetooth.org. Retrieved 6/18, 2009, from https://www.bluetooth.org/apps/content/. Bradley, N., & Poppen, W. (2003). Assistive technology, computers and Internet may decrease sense of isolation for homebound elderly and disabled persons. Technology and Disability, 15, 19–25. Brodwin, M. G., Star, T., & Cardoso, E. (2004). Computer assistive technology for people who have disabilities: Computer adaptations and modifications. Journal of Rehabilitation, 70, 28–33. Casiez, G., Vogel, D., & Balakrishnan, R. (2008). The impact of control-display gain on user performance in pointing tasks. Human–Computer Interaction, 23, 215– 250. Cockburn, A., & Brewster, S. (2005). Multimodal feedback for the acquisition of small targets. Ergonomics, 48, 1129–1150. Cockburn, A., & Firth, A. (2003). Improving the acquisition of small targets. Paper presented at the proceedings of the 17th annual human–computer interaction conference. Davies, D. K., Stock, S. E., & Wehmeyer, M. L. (2002a). Enhancing independent task performance for individuals with mental retardation through use of a handheld self-directed visual and audio prompting system. Education and Training in Mental Retardation and Developmental Disabilities, 37, 209–218. Davies, D. K., Stock, S. E., & Wehmeyer, M. L. (2002b). Enhancing independent time-management skills of individuals with mental retardation using a palmtop personal computer. Mental Retardation, 40, 358–365. Dennerlein, J. T., & Yang, M. C. (2001). Haptic force-feedback devices for the office computer: Performance and musculoskeletal loading issues. Human Factors, 43, 278. Donker, A., & Reitsma, P. (2007a). Aiming and clicking in young children’s use of the computer mouse. Computers in Human Behavior, 23, 2863–2874. Donker, A., & Reitsma, P. (2007b). Drag-and-drop errors in young children’s use of the mouse. Interacting with Computers, 19, 257–266. Donker, A., & Reitsma, P. (2007c). Young children’s ability to use a computer mouse. Computers and Education, 48, 602–617. Grossman, T., & Balakrishnan, R. (2005). The bubble cursor: Enhancing target acquisition by dynamically resizing of the cursor’s activation area. Paper presented at the proceedings of CHI’05 conference on human factors in computing systems. Hedrick, B., Pape, T. L., Heinemann, A. W., Ruddell, J. L., & Reis, J. (2006). Employment issues and assistive technology use for persons with spinal cord injury. Journal of Rehabilitation Research and Development, 43, 185. Houlihan, B. V., Drainoni, M. L., Warner, G., Nesathurai, S., Wierbicky, J., & Williams, S. (2003). The impact of Internet access for people with spinal cord injuries: A descriptive analysis of a pilot study. Disability and Rehabilitation, 25, 422–431. Lee, Y., & Vail, C. O. (2005). Computer-based reading instruction for young children with disabilities. Journal of Special Education Technology, 20, 14. Mann, W. C., Belchior, P., Tomita, M. R., & Kemp, B. J. (2005). Computer use by middle-aged and older adults with disabilities. Technology and Disability, 17, 1–9. Microsoft. (2008). Windows driver kit: Human input devices. Retrieved 5/30, 2008, from http://msdn.microsoft.com/en-us/library/ms794089.aspx. Park, J., Han, S. H., & Yang, H. (2006). Evaluation of cursor capturing functions in a target positioning task. International Journal of Industrial Ergonomics, 36, 721– 730. Rao, R. S., Seliktar, R., & Rahman, T. (2000). Evaluation of an isometric and a position joystick in a target acquisition task for individuals with cerebral palsy. IEEE Transactions on Rehabilitation Engineering, 8, 118–125. Richards, S. B., Taylor, R. L., Ramasamy, R., & Richards, R. Y. (1999). Single subject research: Applications in educational and clinical settings. New York: Wadsworth. Ritchie, H., & Blanck, P. (2003). Promise of the Internet for disability: A study of online services and accessibility of centers for independent living Web sites. Behavioral Sciences and the Law, 21, 5–26. Shein, G. F., Treviranus, J., Brownlow, N. D., Milner, M., & Parnes, P. (1992). An overview of human–computer interaction techniques for people with physical disabilities. International Journal of Industrial Ergonomics, 9, 171–181. Shih, C.-H., Chang, M.-L., & Shih, C.-T. (2009). Assisting people with multiple disabilities and minimal motor behavior to improve computer pointing efficiency through a mouse wheel. Research in Developmental Disabilities, 30, 1378–1387. Shih, C.-H., Chung, C.-C., Chiang, M.-S., & Shih, C.-T. (2009). Assisting people with developmental disabilities to improve pointing efficiency with a Dual Cursor Automatic Pointing Assistive Program. Research in Developmental Disabilities doi:10.1016/j.ridd.2009.1008.1007. Shih, C.-H., Hsu, N.-Y., & Shih, C.-T. (2009). Assisting people with developmental disabilities to improve pointing efficiency with an Automatic Pointing Assistive Program. Research in Developmental Disabilities, 30, 1212–1220. Shih, C.-H., & Shih, C.-T. (2009a). Assisting people with multiple disabilities to use computers with multiple mice. Research in Developmental Disabilities, 30, 746– 754. Shih, C.-H., & Shih, C.-T. (2009b). Development of a computer input system for persons with disabilities using a commercial mouse and switches. Disability and Rehabilitation: Assistive Technology doi:10.1080/17483100903134474. Shih, C.-H., & Shih, C.-T. (2009c). Development of an integrated pointing device driver for the disabled. Disability and Rehabilitation: Assistive Technology doi:10.1080/17483100903105599. Shih, C.-H., & Shih, C.-T. (2009d). A new movement detector to enable people with multiple disabilities to control environmental stimulation with hand swing through a commercial mouse. Research in Developmental Disabilities, 30, 1196–1202. Shih, C. -H., & Shih, C. -T. (in press). Assisting two children with multiple disabilities and minimal motor behavior to control environmental stimulation with thumb poke through a trackball. Behavioural and Cognitive Psychotherapy. Shih, C.-H., Shih, C.-T., Lin, K.-T., & Chiang, M.-S. (2009). Assisting people with multiple disabilities and minimal motor behavior to control environmental stimulation through a mouse wheel. Research in Developmental Disabilities, 30, 1413–1419. Shimizu, H., & McDonough, C. S. (2006). Programmed instruction to teach pointing with a computer mouse in preschoolers with developmental disabilities. Research in Developmental Disabilities, 27, 175–189. Siegel, S., & Castellan, N. J. (1988). Nonparametric statistics for the behavioral sciences. New York: McGraw-HiU Book Company. SugarVsSpice. (2009). Animal splicer 1.0. Retrieved August 15, 2009, from http://www.theshadowfang.uwclub.net/splicer/splicer.html.
C.-H. Shih et al. / Research in Developmental Disabilities 31 (2010) 416–425
425
Tu, J., Tao, H., & Huang, T. (2007). Face as mouse through visual face tracking. Computer Vision and Image Understanding, 108, 35–40. Wikipedia. (2009a). Device driver. Retrieved August 20, 2009, from http://en.wikipedia.org/wiki/Device_driver. Wikipedia. (2009b). Drag-and-drop. Retrieved August 10, 2009, from http://en.wikipedia.org/wiki/Drag-and-drop. Wikipedia. (2009c). Mouseover. Retrieved March 10, 2009, from http://en.wikipedia.org/wiki/Mouseover. Wong, A. W. K., Chan, C. C. H., Li-Tsang, C. W. P., & Lam, C. S. (2009). Competence of people with intellectual disabilities on using human–computer interface. Research in Developmental Disabilities, 30, 107–123.
Contents lists available at ScienceDirect
Research in Developmental Disabilities
An automatic Drag-and-Drop assistive program developed to assistive people with developmental disabilities to improve Drag-and-Drop efficiency Ching-Hsiang Shih a,*, Hsun-Chin Huang a, Yung-Kun Liao a, Ching-Tien Shih b, Ming-Shan Chiang a a b
Department of Special Education, National Dong Hwa University, Hualien, Taiwan, ROC Department of Electronics Engineering and Computer Science, Tung-Fang Institute of Technology, Kaohsiung Country, Taiwan, ROC
A R T I C L E I N F O
A B S T R A C T
Article history: Received 29 September 2009 Accepted 5 October 2009
The latest researches adopted software technology to improve pointing performance; however, Drag-and-Drop (DnD) operation is also commonly used in modern GUI programming. This study evaluated whether two children with developmental disabilities would be able to improve their DnD performance, through an Automatic DnD Assistive Program (ADnDAP). At first, both participants had their baseline sessions. Then the first participant entered into intervention. New intervention began with the second participant when his performance was consolidated. Finally, maintenance phase occurred with both participants, in which their DnD performance improved significantly. Data showed that both participants improved their DnD efficiency with the assistance of ADnDAP, and remained highly successful through maintenance phase. Implications of the findings are discussed. ß 2009 Elsevier Ltd. All rights reserved.
Keywords: Developmental disabilities DnD ADnDAP Mouse driver
To increase the functional abilities, independence, and access to mainstream society for people with disabilities, computer technologies have a tremendous potential and create a method of equalization between people with and without disabilities (Bradley & Poppen, 2003; Brodwin, Star, & Cardoso, 2004; Houlihan et al., 2003; Lee & Vail, 2005; Wong, Chan, Li-Tsang, & Lam, 2009). The benefits are clearer when persons with disabilities are given the opportunity to improve their level of competency in controlling computer through being offered with special interfaces (hardware assistance) or assistive program (software assistance) to improve their operation efficiency (Brodwin et al., 2004; Davies, Stock, & Wehmeyer, 2002a; Davies, Stock, & Wehmeyer, 2002b; Mann, Belchior, Tomita, & Kemp, 2005; Ritchie & Blanck, 2003). However, because most commercial computer input devices are targeted at the mainstream population, without providing the type of accommodation that meets the needs or desires of people with disabilities (Abascal & Nicolle, 2005; Mann et al., 2005), it is difficult or impossible for them to operate computers by a mouse, or similar pointing devices (Brodwin et al., 2004; Rao, Seliktar, & Rahman, 2000). Based on this, various modifications and adaptations of computer-pointing devices have been proposed to meet the needs of people with multiple disabilities (Brodwin et al., 2004; Hedrick, Pape, Heinemann, Ruddell, & Reis, 2006; Mann et al., 2005; Shein, Treviranus, Brownlow, Milner, & Parnes, 1992; Tu, Tao, & Huang, 2007). Normally, their cost is much higher than usual
* Corresponding author. Tel.: +886 3 8227106x1320; fax: +886 3 8228707. E-mail address: [email protected] (C.-H. Shih). 0891-4222/$ – see front matter ß 2009 Elsevier Ltd. All rights reserved. doi:10.1016/j.ridd.2009.10.004
C.-H. Shih et al. / Research in Developmental Disabilities 31 (2010) 416–425
417
devices, due to their specific design. Besides, their long-term use by people with disabilities is limited, since they are often more difficult to obtain or maintain than ordinary commercial devices. Shih and Shih (2009a, 2009c) presented a special interface through multi-mice configuration which adopted software technology to redesign the mouse driver, in order to enable physically disabled people to complete mouse operation through exporting their remaining ability of each limb with several mice. For example, the right hand could control a mouse left-toright movement, the left hand controls the up-to-down movement, and the mouse button is pressed by the left toe (Shih & Shih, 2009a). The function of each mouse is adjusted individually by Shih’s mouse drive to achieve the above requested functions. Therefore, with the assistance of multi-mice configuration, people with disabilities can as with people without disabilities, can use a very common, cheap and powerful commercial mice, instead of specialized alternative computer input devices. Many researchers have also proposed assistive programs to facilitate the quality of pointing (target positioning/ acquiring) operation, in order to improve the operation efficiency of people with disabilities (Ahlstrom, 2005; Ahlstrom, Hitz, & Leitner, 2006; Akamatsu & MacKenzie, 2002; Casiez, Vogel, & Balakrishnan, 2008; Cockburn & Brewster, 2005; Cockburn & Firth, 2003; Dennerlein & Yang, 2001; Grossman & Balakrishnan, 2005; Park, Han, & Yang, 2006). Pointing, which is achieved by moving the cursor over certain areas or icons and clicking, is adopted by most computer programs and CAI software as the most common basic mouse operation (Donker & Reitsma, 2007a, 2007c; Shimizu & McDonough, 2006). Users can gain from being provided useful functions in pointing, such as moving the cursor to the target center automatically, to position the target quickly, easily, and accurately (Grossman & Balakrishnan, 2005; Park et al., 2006). The latest research adopted mouse driver technology to recommend a new operation method, Automatic Pointing Assistive Program (APAP), where the user can click the mouse button when the cursor is near the target (inside the activation area), instead of moving the cursor to the target (Shih, Hsu, & Shih, 2009), to improve the users’ pointing efficiency. Mouse click action will be intercepted as soon as the mouse is clicked, the cursor jumps to the target center automatically, and then the intercepted mouse click action will be sent out, as shown in Fig. 1.However, in APAP, the cursor jumps to the target only when the mouse is clicked, and users can not be assisted by Mouseover effects which are commonly used in modern Graphical User Interface (GUI) programming, by which an element changes in response to the mouse cursor moving over it (Wikipedia, 2009c), before he/she clicks the mouse button (Shih, Hsu, et al., 2009). Therefore, Shih, Chung, Chiang, and Shih (2009) presented a new revised operation method, Dual Cursor Automatic Pointing Assistive Program (DCAPAP) as APAP’s development, where the dual cursors (a virtual cursor and a system cursor) are adopted to offer users an operating environment which is closer to the real conditions, as shown in Fig. 2. With this technology (DCAPAP), when the system cursor enters into the activation area, it will jump to the target center automatically and activate Mouseover effects. Besides, the virtual cursor appears to indicate the movement path and users can click as soon as the virtual cursor is inside the activation area. In addition, a mouse wheel was also used as a pointing assistive device to improve pointing performance of people with multiple disabilities who have minimal motor behavior (i.e., people who have extensive paralysis of their body and can effectively control only very limited movements), through a new operation method, Dynamic Pointing Assistive Program (DPAP), where the user can poke his/her thumb/finger to rotate a mouse wheel to move a cursor to a target (Shih, Chang, &
Fig. 1. The operation flow of Automatic Pointing Assistive Program (APAP). (a) When the cursor enters into the activation area, mouse click action will be intercepted as soon as the mouse is clicked. (b) The cursor jumps to the target center automatically. (c) The intercepted mouse click action is sent out.
418
C.-H. Shih et al. / Research in Developmental Disabilities 31 (2010) 416–425
Fig. 2. The operation flow of Dual Cursor Automatic Pointing Assistive Program (DCAPAP). (a) The externality of the cursor before it enters into the activation area. (b) When the cursor enters into the activation area, the virtual cursor (solid pink cursor) appears to indicate the movement path of the system cursor, and the system cursor jumps to the target center automatically. (c) The system cursor is locked to the target center when the virtual cursor moves inside the activation area. (d) The virtual cursor disappears once it moves out of the activation area, and the system cursor replaces it and continues moving (For interpretation of the references to colour in this figure legend, the reader is referred to the web version of the article.).
Shih, 2009). As soon as the mouse wheel is rotated, its action will be intercepted, and the cursor will automatically jump to a series of pre-defined target positions in order, according to the wheel rotation amount and direction, as shown in Fig. 3. With the assistance of DPAP, people with profound multiple disabilities and minimal motor behavior can use the mouse wheel to point quickly, easily, and accurately, thus helping them to solve their pointing problems.Except pointing, Drag-and-Drop (DnD) operation is also commonly used in modern GUI programming, by which an object is clicked and dragged to a different location or onto another object (Wikipedia, 2009b). The basic sequence involved in DnD is: (a) press and hold down the button on the mouse, to ‘‘grab’’ the object; (b) drag the object to the desired location; (c) drop the object by releasing the button. Dragging requires more physical effort than moving the same pointing device without holding down any buttons (Wikipedia, 2009b). Fig. 4 shows a DnD CAI software in which users need to drag 2 animals onto the pods, and then pull the lever to splice them together (SugarVsSpice, 2009). Many CAI programs require dragging operation, however, people with disabilities cannot use these programs due to their physical limitations (Donker & Reitsma, 2007b). This can be solved through a new revised operation method, Automatic DnD Assistive Program (ADnDAP), where the complex dragging process is replaced by simple clicking operation. Users can click mouse button when the cursor is near the target (entering into the activation area, Fig. 5(a)), the cursor jumps to the target center and the mouse button is held down automatically (i.e., ‘‘grab’’ the target, Fig. 5(b)). The cursor and target jump to the destination with the button held down (i.e., ‘‘Drag’’ the target to the desired location, Fig. 5(c)). Users release the button to finish dragging (i.e., ‘‘Drop’’ the target by releasing the button, Fig. 5(d)). With this technology (ADnDAP), when users move the cursor into an activation area and click, the target will be dragged to the destination automatically. In this way, people with disabilities can extend their computer operation to DnD, besides pointing. The same as APAP, the key technology of ADnDAP is the mouse click action interception which avoids error click through preventing error commands sent to the computer, when the button is pressed before the cursor arrives to the target. Without this technology, users can neither intercept mouse click nor drag the target to the destination automatically through mouse clicking. ADnDAP is able to run independently in the windows OS environment, works without interference with all currently available software (i.e., the currently available software does not need to be modified or rewritten).
C.-H. Shih et al. / Research in Developmental Disabilities 31 (2010) 416–425
419
Fig. 3. The operation flow of Dynamic Pointing Assistive Program (DPAP). (a) A user pokes his/her thumb/finger to rotate a mouse wheel to move a cursor to a target. (b) The four pre-defined targets which were noted as P1, P2, P3 and P4 in Dynamic Pointing Assistive Program (DPAP). Users poke mouse wheel will quickly move the cursor among the four targets. Each forward poke will jump the cursor from one to a next target in order of P1 ! P2 ! P3 ! P4 ! P1 ! , whereas, backward poke jumps the cursor in order of P4 ! P3 ! P2 ! P1 ! P4 ! .
Fig. 4. A DnD CAI software in which users drag 2 animals onto the pods, and then pull the lever to splice them together (SugarVsSpice, 2009).
420
C.-H. Shih et al. / Research in Developmental Disabilities 31 (2010) 416–425
Fig. 5. The operation flow of Automatic DnD Assistive Program (ADnDAP). (a) The user can click the mouse button when the cursor is near the target (entering into the activation area). (b) The cursor jumps to the target center and the mouse button is held down automatically (i.e., ‘‘grab’’ the target). (c) The cursor and target jump to the destination with the button held down (i.e., ‘‘Drag’’ the target to the desired location). (d) Dragging is finished when users release the button (i.e., ‘‘Drop’’ the target by releasing the button).
Every hardware device linked to a computer requires a software-based driver to work normally. Writing a device driver requires an in-depth understanding of how the hardware and the software of a given platform function. Drivers operating in a highly privileged environment can cause disaster (crash a system) if they get things wrong (erroneously programmed). Normally, the device driver is provided by Windows OS or the hardware manufacture, to ensure that the connected device can function properly (Wikipedia, 2009a). Driver modification is rarely proposed by researches because of the complexity of the technology required (Microsoft, 2008; Wikipedia, 2009a). Only a few of recent researches (Shih, Chang, et al., 2009; Shih, Chung, et al., 2009; Shih, Hsu, et al., 2009; Shih & Shih, 2009a, 2009b, 2009c, 2009d, in press; Shih, Shih, Lin, & Chiang, 2009) adopted software technology to redesign the mouse driver in order to reset mouse functions, and took the mouse as a useful tool for many applications dedicated to persons with disabilities, providing them with additional choices in assistive technology. As a standard device for computers, once a mouse is connected to a computer, its function will be defined as moving, clicking and dragging. As a result, it is not easy to intercept mouse click, and no research adapting driver technology to assist DnD is published in this field. This work adopts Shih’s new revised mouse driver (intercepting driver) design (i.e., a new mouse driver replaces standard mouse driver, and is able to intercept mouse click action) to help people with developmental disabilities improve their DnD efficiency, and to understand the difference of DnD performance between before and after for them using ADnDAP, in order to determine whether the ADnDAP implementation can enhance their DnD performance. 1. Method 1.1. Participants The participants Hsueh and Liao were 14 and 13 years of age, respectively. Both of their levels of functioning were estimated to be in the middle range of intellectual disability. Hsueh could click a mouse with right hand, but had difficulties in positioning, due to poor hand–eye coordination. He failed to drag because he released the mouse button before the object arrived to its destination or onto another object. Liao used right hand to operate a mouse, but had poor positioning performance. He also had difficulties in dragging objects to desired destinations due to poor hand–eye coordination.
C.-H. Shih et al. / Research in Developmental Disabilities 31 (2010) 416–425
421
Fig. 6. The flow diagram of the computer mouse training and test software. Eight circular source targets (S1–S8) with radius of 0.25 cm were set every 458 on a circle with a radius of 5 cm. Each source target (S1–S8) has a circular area with a radius of 1.25 cm as the activation areas (A1–A8) of ADnDAP.
Both participants were interested in computer operation, and were capable of speaking and following instructions. No one had visual or hearing impairment that could be a problem in using a mouse. With the guidance of the research assistant, both of them learnt to move the mouse cursor to targets and perform a click. Informed consent was obtained from the children’s parents prior to their participation in the study. 1.2. Apparatus and setting The study was carried out in an activity room. Computers were placed on a computer table, and the screen was at a distance of about 30 cm from their chairs. Wireless mice were provided to the participants when the experiment began. 1.3. ADnDAP setting, computer mouse training and test software This study designed a test software with two modes (practice mode and record mode) to provide the participants with mouse DnD practice, and to record their test results. Practice mode gave repeated DnD practice for the participants, and record mode recorded participants’ successful DnD number within a certain period of time. Fig. 6 is the flow diagram of the computer mouse training and test software. Eight circular source targets (S1–S8) with radius of 0.25 cm were set every 458 on a circle with a radius of 5 cm. Each source target (S1–S8) has a circular area with a radius of 1.25 cm as the activation areas (A1–A8) of ADnDAP. When the participants moved the cursor into the activation areas (A1–A8) and clicked, the cursor jumped to the target center (S1–S8) and dragged the target to the destination point (D, the center of the circumference) automatically. In this way, the participants could click inside the circular activation areas (A1–A8) instead of moving the cursor to source targets (S1–S8), and could drag it to destination point (D) at the same time. In practice mode, the computer first displayed source target S1, and set the mouse cursor to the destination point (D). The participants had to move the mouse cursor from the destination point to source target S1, then dragged S1 and dropped it to destination point (D) to complete a successful DnD. Source target S1 then disappeared, and source target S2 appeared. Participants would then move the mouse cursor from the destination point (D) to source target S2 to drag it again. This process was repeated until the end of practice time. Record mode was run under the same conditions as the practice mode, except the targets appeared randomly. Times of successful DnD number within 3 min were recorded. 1.4. Experimental conditions This study used multiple probe design across participants (Richards, Taylor, Ramasamy, & Richards, 1999). The participants individually received 3 training sessions per week, each with about 30 min use of ADnDAP, for a period of about
422
C.-H. Shih et al. / Research in Developmental Disabilities 31 (2010) 416–425
6–7 weeks. Hsueh had three pre-probe sessions during baseline, and new intervention occurred with him when his performance was consolidated. Liao received discontinuous pre-probe in baseline, and entered into intervention when Hsueh’s intervention performance was consolidated. The experiment included three phases: (a) baseline, in which at least 3 times pre-probe sessions were performed to collect participants’ baseline data; (b) intervention, in which ADnDAP were used to obtain the performance data of ADnDAP practice for assessment, and (c) maintenance, performed 1 week after intervention finished, in which participants’ follow-up performance was assessed 3 times. 1.5. Baseline The test software was used to record the successful DnD number within 3 min during the baseline. The ADnDAP function was turned off, so the participants had to move the cursor from the destination point (D) to source target (S1–S8), and dragged it to the destination point (D). Three data points were obtained for Hsueh in 1 week during this phase, while Liao’s baseline data was collected twice per week discontinuously. 1.6. Intervention At the beginning, Hsueh got trained for ADnDAP use and training software, and Liao was still being probed discontinuously to collect his baseline data. ADnDAP training occurred with Liao once Hsueh’s performance was consolidated. Principally, each participant performed 11 practice sessions of 30 min during intervention. The arrangement of this 30-min session was as follows: (a) DnD practice (20 min) Source targets (S1–S8) appeared in order, and participants moved the mouse cursor from the destination point (D) to the target activation areas (A1–A8) and clicked, and ADnDAP finished dragging automatically. The research assistant provided vocal prompting and guidance during this phase to help the participants complete DnD through ADnDAP. (b) Rest (7 min) Participants were given 7 min rest after practice. (c) Assessment (3 min) The source targets (S1–S8) appeared randomly during this phase. Neither vocal prompting nor instructions from the research assistant were available. Each participant’s successful DnD within 3 min were recorded as input for assessment, and then used to determine whether ADnDAP improved their DnD efficiency. This phase continued until each participant’s performance was consolidated. 1.7. Maintenance This phase began 1 week after the intervention phase to determine whether the participants maintained the skills that they had acquired. During this phase, participants did not have ADnDAP practice, but participated directly in the 3-min DnD test. 2. Results Fig. 7 indicated the two participants’ DnD speed after the implementation of ADnDAP. The curve showed that both participants improved their DnD efficiency, and maintained their acquired skills, during the maintenance phase. 2.1. Baseline Hsueh could control the mouse with his right hand. Detailed observation of his mouse operation demonstrated that he had difficulties in both target positioning and dragging, owing to his difficulties of hand control and poor hand–eye coordination. The cursor often deviated from the target when he clicked the mouse, and he had to spend a lot of time to aim at the target again. He did not have any successful dragging in the first 2 sessions during his baseline phase, because he released mouse button before the cursor arrived to destinations. He only had 2 successful DnD in the third session during this phase. Liao also had difficulty in controlling the movement of the mouse cursor at the same time of clicking, and could not drag objects to destinations before released mouse button, owing to poor hand–eye coordination. He had poor performance both in target positioning and dragging during the baseline phase. 2.2. Intervention At first, Hsueh was unfamiliar with the ADnDAP function, and continued to click on the target rather than the activation area after ADnDAP practice. However, his correct DnD per minute still increased compared to baseline, because ADnDAP avoided cursor deviation when clicking, and dragged the target to the destination automatically. With the assistance of the
C.-H. Shih et al. / Research in Developmental Disabilities 31 (2010) 416–425
423
Fig. 7. The DnD speeds of the two participants after the implementation of ADnDAP. The curve indicates that both two participants improved their DnD efficiency, and maintained their acquired skills, during the maintenance phase.
research guide, he gradually learnt the use of ADnDAP (click inside the activation area), and increased his DnD efficiency as the practice time accumulated. Fig. 7 shows that his DnD speed increased quickly. He achieved 46–72 correct DnD within 3 min during the 5 sessions in his earlier intervention phase, improved his DnD efficiency within 6–9 practice sessions, and became able to use ADnDAP to drag targets on the screen. Hsueh’s correct DnD within 3 min increased to 85–86 during sessions 10 and 11, revealing that ADnDAP operation could be mastered easily within a short period of practice. He achieved 15.33–29.00 correct DnD per min, with an overall mean of 23.15 min1. The Kolmogorov–Smirnov test (Siegel & Castellan, 1988) showed that the increase from baseline to intervention was statistically significant (p < 0.01). Liao significantly increased his DnD efficiency with the assistance of ADnDAP. After several practice, his DnD speed rose quickly, and he achieved 24.67–39.33 correct DnD per min during this phase, with an overall mean of 35.88, as shown in Fig. 7. The Kolmogorov–Smirnov test (Siegel & Castellan, 1988) showed that the increase from baseline to intervention was statistically significant (p < 0.01). 2.3. Maintenance The maintenance phase occurred with Hsueh 1 week after intervention finished. As shown in Fig. 7, his DnD performance in this phase was close to that during the intervention phase, and his correct DnD within 3 min in 3 sessions during this phase were 91, 86 and 88, respectively. His DnD speed was between 28.67 and 30.33 min1, with an overall mean of 29.44. These results indicated that he retained the skills that he had acquired during the intervention phase. Liao entered into this phase 1 week after intervention. During this phase (including 3 sessions), his correct DnD within 3 min were 98, 102 and 100, respectively, and he achieved 32.67–34.00 correct DnD per min. These results indicated that he also kept the skills that he had acquired during intervention. 3. Discussion Dragging a target to a destination or onto a target is a frequently adopted basic mouse operation for most computer programs and CAI software (Donker & Reitsma, 2007b). However, people with disabilities can find this is a challenging task because it requires more physical effort than pointing. ADnDAP can give effective assistance to these users to realize target acquisition and DnD at the same time. This study has indicated that both participants greatly improved their efficiencies to perform DnD after receiving ADnDAP training, and retained their acquisition skills in the maintenance phase. Analytical results demonstrate that people with developmental disabilities can easily master ADnDAP without long-period practice. Both participants could operate some educational/CAI software which require DnD operation through ADnDAP after the experiment. Participants were able to drag targets quickly, easily, and accurately, with the assistance of ADnDAP. ADnDAP keeps the merit of APAP to avoid cursor deviation (Shih, Hsu, et al., 2009), and is based on software solution which does not need extra hardware or circuit preservations, and can support all standard interfaces of commercial pointing devices (i.e., mouse and
424
C.-H. Shih et al. / Research in Developmental Disabilities 31 (2010) 416–425
trackball) that are compatible with the computer, including USB, wireless and Bluetooth (Bluetooth.org, 2009) interfaces. In addition, ADnDAP is compatible with all currently available software, so the current software can be applied to improve the pointing efficiency of people with disabilities without being modified or rewritten. This study only considers dragging targets to fixed destinations, focusing on individuals with developmental disabilities, who cannot use a standard mouse to perform DnD efficiently. Further studies are necessary to develop additional mouse applications to extend current functionality (i.e., users can freely define their desired destinations) and satisfy the needs of different levels of disabilities. Hopefully, the implementation of ADnDAP can cover all complex DnD operations and provide disabled users with additional choices in computer assistive technology. References Abascal, J., & Nicolle, C. (2005). Moving towards inclusive design guidelines for socially and ethically aware HCI. Interacting with Computers, 17, 484–505. Ahlstrom, D. (2005). Modeling and improving selection in cascading pull-down menus using Fitts’ law, the steering law and force fields. Paper presented at the proceedings of the CHI’05 conference on human factors in computing systems. Ahlstrom, D., Hitz, M., & Leitner, G. (2006). An evaluation of sticky and force enhanced targets in multi target situations. Paper presented at the proceedings of the 4th Nordic conference on human–computer interaction: Changing roles. Akamatsu, M., & MacKenzie, I. S. (2002). Changes in applied force to a touchpad during pointing tasks. International Journal of Industrial Ergonomics, 29, 171–182. Bluetooth.org. (2009). Bluetooth.org—Welcome to Bluetooth.org. Retrieved 6/18, 2009, from https://www.bluetooth.org/apps/content/. Bradley, N., & Poppen, W. (2003). Assistive technology, computers and Internet may decrease sense of isolation for homebound elderly and disabled persons. Technology and Disability, 15, 19–25. Brodwin, M. G., Star, T., & Cardoso, E. (2004). Computer assistive technology for people who have disabilities: Computer adaptations and modifications. Journal of Rehabilitation, 70, 28–33. Casiez, G., Vogel, D., & Balakrishnan, R. (2008). The impact of control-display gain on user performance in pointing tasks. Human–Computer Interaction, 23, 215– 250. Cockburn, A., & Brewster, S. (2005). Multimodal feedback for the acquisition of small targets. Ergonomics, 48, 1129–1150. Cockburn, A., & Firth, A. (2003). Improving the acquisition of small targets. Paper presented at the proceedings of the 17th annual human–computer interaction conference. Davies, D. K., Stock, S. E., & Wehmeyer, M. L. (2002a). Enhancing independent task performance for individuals with mental retardation through use of a handheld self-directed visual and audio prompting system. Education and Training in Mental Retardation and Developmental Disabilities, 37, 209–218. Davies, D. K., Stock, S. E., & Wehmeyer, M. L. (2002b). Enhancing independent time-management skills of individuals with mental retardation using a palmtop personal computer. Mental Retardation, 40, 358–365. Dennerlein, J. T., & Yang, M. C. (2001). Haptic force-feedback devices for the office computer: Performance and musculoskeletal loading issues. Human Factors, 43, 278. Donker, A., & Reitsma, P. (2007a). Aiming and clicking in young children’s use of the computer mouse. Computers in Human Behavior, 23, 2863–2874. Donker, A., & Reitsma, P. (2007b). Drag-and-drop errors in young children’s use of the mouse. Interacting with Computers, 19, 257–266. Donker, A., & Reitsma, P. (2007c). Young children’s ability to use a computer mouse. Computers and Education, 48, 602–617. Grossman, T., & Balakrishnan, R. (2005). The bubble cursor: Enhancing target acquisition by dynamically resizing of the cursor’s activation area. Paper presented at the proceedings of CHI’05 conference on human factors in computing systems. Hedrick, B., Pape, T. L., Heinemann, A. W., Ruddell, J. L., & Reis, J. (2006). Employment issues and assistive technology use for persons with spinal cord injury. Journal of Rehabilitation Research and Development, 43, 185. Houlihan, B. V., Drainoni, M. L., Warner, G., Nesathurai, S., Wierbicky, J., & Williams, S. (2003). The impact of Internet access for people with spinal cord injuries: A descriptive analysis of a pilot study. Disability and Rehabilitation, 25, 422–431. Lee, Y., & Vail, C. O. (2005). Computer-based reading instruction for young children with disabilities. Journal of Special Education Technology, 20, 14. Mann, W. C., Belchior, P., Tomita, M. R., & Kemp, B. J. (2005). Computer use by middle-aged and older adults with disabilities. Technology and Disability, 17, 1–9. Microsoft. (2008). Windows driver kit: Human input devices. Retrieved 5/30, 2008, from http://msdn.microsoft.com/en-us/library/ms794089.aspx. Park, J., Han, S. H., & Yang, H. (2006). Evaluation of cursor capturing functions in a target positioning task. International Journal of Industrial Ergonomics, 36, 721– 730. Rao, R. S., Seliktar, R., & Rahman, T. (2000). Evaluation of an isometric and a position joystick in a target acquisition task for individuals with cerebral palsy. IEEE Transactions on Rehabilitation Engineering, 8, 118–125. Richards, S. B., Taylor, R. L., Ramasamy, R., & Richards, R. Y. (1999). Single subject research: Applications in educational and clinical settings. New York: Wadsworth. Ritchie, H., & Blanck, P. (2003). Promise of the Internet for disability: A study of online services and accessibility of centers for independent living Web sites. Behavioral Sciences and the Law, 21, 5–26. Shein, G. F., Treviranus, J., Brownlow, N. D., Milner, M., & Parnes, P. (1992). An overview of human–computer interaction techniques for people with physical disabilities. International Journal of Industrial Ergonomics, 9, 171–181. Shih, C.-H., Chang, M.-L., & Shih, C.-T. (2009). Assisting people with multiple disabilities and minimal motor behavior to improve computer pointing efficiency through a mouse wheel. Research in Developmental Disabilities, 30, 1378–1387. Shih, C.-H., Chung, C.-C., Chiang, M.-S., & Shih, C.-T. (2009). Assisting people with developmental disabilities to improve pointing efficiency with a Dual Cursor Automatic Pointing Assistive Program. Research in Developmental Disabilities doi:10.1016/j.ridd.2009.1008.1007. Shih, C.-H., Hsu, N.-Y., & Shih, C.-T. (2009). Assisting people with developmental disabilities to improve pointing efficiency with an Automatic Pointing Assistive Program. Research in Developmental Disabilities, 30, 1212–1220. Shih, C.-H., & Shih, C.-T. (2009a). Assisting people with multiple disabilities to use computers with multiple mice. Research in Developmental Disabilities, 30, 746– 754. Shih, C.-H., & Shih, C.-T. (2009b). Development of a computer input system for persons with disabilities using a commercial mouse and switches. Disability and Rehabilitation: Assistive Technology doi:10.1080/17483100903134474. Shih, C.-H., & Shih, C.-T. (2009c). Development of an integrated pointing device driver for the disabled. Disability and Rehabilitation: Assistive Technology doi:10.1080/17483100903105599. Shih, C.-H., & Shih, C.-T. (2009d). A new movement detector to enable people with multiple disabilities to control environmental stimulation with hand swing through a commercial mouse. Research in Developmental Disabilities, 30, 1196–1202. Shih, C. -H., & Shih, C. -T. (in press). Assisting two children with multiple disabilities and minimal motor behavior to control environmental stimulation with thumb poke through a trackball. Behavioural and Cognitive Psychotherapy. Shih, C.-H., Shih, C.-T., Lin, K.-T., & Chiang, M.-S. (2009). Assisting people with multiple disabilities and minimal motor behavior to control environmental stimulation through a mouse wheel. Research in Developmental Disabilities, 30, 1413–1419. Shimizu, H., & McDonough, C. S. (2006). Programmed instruction to teach pointing with a computer mouse in preschoolers with developmental disabilities. Research in Developmental Disabilities, 27, 175–189. Siegel, S., & Castellan, N. J. (1988). Nonparametric statistics for the behavioral sciences. New York: McGraw-HiU Book Company. SugarVsSpice. (2009). Animal splicer 1.0. Retrieved August 15, 2009, from http://www.theshadowfang.uwclub.net/splicer/splicer.html.
C.-H. Shih et al. / Research in Developmental Disabilities 31 (2010) 416–425
425
Tu, J., Tao, H., & Huang, T. (2007). Face as mouse through visual face tracking. Computer Vision and Image Understanding, 108, 35–40. Wikipedia. (2009a). Device driver. Retrieved August 20, 2009, from http://en.wikipedia.org/wiki/Device_driver. Wikipedia. (2009b). Drag-and-drop. Retrieved August 10, 2009, from http://en.wikipedia.org/wiki/Drag-and-drop. Wikipedia. (2009c). Mouseover. Retrieved March 10, 2009, from http://en.wikipedia.org/wiki/Mouseover. Wong, A. W. K., Chan, C. C. H., Li-Tsang, C. W. P., & Lam, C. S. (2009). Competence of people with intellectual disabilities on using human–computer interface. Research in Developmental Disabilities, 30, 107–123.