- Email: [email protected]
Device to Guide Hand Rehabilitation Routines Based on Pressure Signals
Available online at www.sciencedirect.com
ScienceDirect ScienceDirect
Procedia Computer Science 00 (2019) 000–000
Procediaonline Computer 00 (2019) 000–000 Available at Science www.sciencedirect.com
ScienceDirect
www.elsevier.com/locate/procedia www.elsevier.com/locate/procedia
Procedia Computer Science 160 (2019) 659–664
International Workshop on Future Trends in Assistive Technology International Workshop on Future Trends in Assistive Technology
Device to Guide Hand Rehabilitation Routines Based on Pressure Device to Guide Hand Rehabilitation Routines Based on Pressure Signals Signals a a
M. Solís-Peñaaa, A.C. Villa-Parraa,* M. Solís-Peña , A.C. Villa-Parraa,*
Universidad Politécnica Salesiana (UPS), Calle Vieja 12-30, Cuenca, 010110, Ecuador Universidad Politécnica Salesiana (UPS), Calle Vieja 12-30, Cuenca, 010110, Ecuador
Abstract Abstract Several technologies applied to the rehabilitation field use devices that provide to physiotherapists different kind of information Several technologies applied about to the patient rehabilitation fieldduring use devices that provide to physiotherapists different of information to improve the assessment evolution rehabilitation plans. The new trend of these kind technologies is the to improve the about evolution during rehabilitation plans. The new trend of environments. these technologies the development of assessment safe, portable and patient comfortable wearable devices with wide application in different This is paper development of safe,ofportable andand comfortable wearable with wide application in different This paper presents a proposal a portable safe device for handdevices rehabilitation, composed of five thimbles,environments. able to acquire pressure presents a proposal of a portable and safe device rehabilitation, thimbles, able pressure signals during the execution of movements guidedforbyhand a videogame. The composed device hasofa five display to show to to theacquire users different signals the to execution of movements guided a videogame. The device has and a display to show the the users different routines,during in order guide them about its use, andby also evaluates pressure signals response time. to Here design and routines, in order guide about in itsaddition use, and evaluates and response time. Here thethat design and construction of the to device arethem presented, to also results of a pilotpressure test withsignals volunteers. The results demonstrate pressure construction the deviceacquired are presented, addition to results a pilot testwas with volunteers. The demonstrate that pressure signals can beofaccurately duringinuser movements, andofthe device considered safe to results perform hand rehabilitation. signals can be accurately acquired during user movements, and the device was considered safe to perform hand rehabilitation. © 2019 The Authors. Published by Elsevier B.V. © 2019 2019 The Authors. Published by B.V. © The Authors. by Elsevier Elsevier B.V. This is an an open accessPublished articleunder under theCC CCBY-NC-ND BY-NC-NDlicense license (http://creativecommons.org/licenses/by-nc-nd/4.0/) This is open access article the (http://creativecommons.org/licenses/by-nc-nd/4.0/) This is an open access article under theConference CC BY-NC-ND license Peer-review under responsibility of the Program Chairs. Peer-review under responsibility of the Conference Program Chairs.(http://creativecommons.org/licenses/by-nc-nd/4.0/) Peer-review under responsibility of the Conference Program Chairs. Keywords: Hand rehabilitation; pressure; motor learning; serious game Keywords: Hand rehabilitation; pressure; motor learning; serious game
1. Introduction 1. Introduction Currently, hand impairments due to alterations of hand motor function is frequently observed in people affected hand due diseases to alterations of stroke, hand motor function frequently disease observed in arthritis people affected by Currently, neurological andimpairments musculoskeletal such as cerebral palsy,isParkinson’s and [1], [2]. by neurological and musculoskeletal diseases such as stroke, cerebral palsy, Parkinson’s disease and arthritis [1], [2]. Chronic cases also occur due to bad posture during work, non-ergonomic conditions during repetitive movements or Chronic cases also occur due to bad posture during work, non-ergonomic conditions during repetitive movements falls. This produces spasticity, deficits in motor control capabilities, inflammation, muscles weakness and chronic or falls. This produces spasticity, deficits in motor control capabilities, inflammation, muscles weakness and chronic * Corresponding author. Tel.: +5939844586. * E-mail Corresponding Tel.: +5939844586. address:author. [email protected] E-mail address: [email protected] 1877-0509 © 2019 The Authors. Published by Elsevier B.V. 1877-0509 © 2019 Thearticle Authors. Published by Elsevier B.V. This is an open access under the CC BY-NC-ND license (http://creativecommons.org/licenses/by-nc-nd/4.0/) Peer-review under responsibility of the Conference Program Chairs. This is an open access article under CC BY-NC-ND license (http://creativecommons.org/licenses/by-nc-nd/4.0/) Peer-review under responsibility of the Conference Program Chairs.
1877-0509 © 2019 The Authors. Published by Elsevier B.V. This is an open access article under the CC BY-NC-ND license (http://creativecommons.org/licenses/by-nc-nd/4.0/) Peer-review under responsibility of the Conference Program Chairs. 10.1016/j.procs.2019.11.031
660 2
M. Solís-Peña et al. / Procedia Computer Science 160 (2019) 659–664 Author name / Procedia Computer Science 00 (2018) 000–000
pain, leading to different problems, such as incorrect action of grip or tremor [1]. Actually, any loss of movement skill has a great impact on the quality of life, however, as the hand function is fundamental in activities of daily living (ADL), hand rehabilitation is considered among the more critical cases that should be immediately addressed. Hand function recovery involves performing rehabilitation tasks to strengthen the muscles around the fingers to allow increasing the flexibility, improving active joint range of motion, movement performance and movement coordination (to delay the deterioration of bone and cartilaginous tissue) and, consequently, the performance of ADLs [2]. These rehabilitation tasks are normally performed through sequences of repetitive exercises during sessions coordinated with a physiotherapist using resistance training [3], [4]. Satisfactory results depend on the frequency, duration, patient motivation and quality of rehabilitation sessions, especially when these are focused on brain neuroplasticity approaches [5]. In order to improve the results of rehabilitation processes and offer better options for the population that requires physiotherapy assistance, interactive rehabilitation technologies have had a great development in recent years, offering devices that allow movement assistance, user signal acquisition, as well as interaction with virtual environments, serious games and smartphones applications [1], [6], [7]. Among these, wearable devices have been used for user monitoring about posture and upper limb movements, employing a number of sensors, such as: optical linear encoder, inertial measurement unit (IMU), force and knitted piezoresistive fabric [2]. These devices can provide information for physiotherapists to allow them to objectively assess the user progress as well as propose improvements to their rehabilitation plans [8]. In fact, it is reported in the literature that rehabilitation interventions using serious games and electronic reminder systems are useful tools to support rehabilitation plans [9], [10]. Some studies have employed game scenes to guide repetitive movements (grasping activities or finger movements), thus motivating patients to practice or train longer [2], [6], [11]. Virtual scenarios are also employed together movement sensors and rehabilitation robot with haptic, visual and multimodal feedback to design manipulation tasks [2], [9], [12]. On the other hand, devices for monitoring and provision of feedback about posture and movements can be also used to training of different sequential finger movements, aiming to evaluate brain networks of recall, memory, and execution in the cognitive rehabilitation field [13]. Currently, the new trend of technologies applied for hand rehabilitation is the development of portable devices, guaranteeing user safety and comfort [1], [2], [8], [14]. These devices can allow the patient to perform rehabilitation guided by physiotherapists without the need for transfers to clinical centers and without losing the quality of the rehabilitation process. These devices can be also programmed by physiotherapists to act as assistant in this process, allowing the patients to adjust their schedules, reduce costs and increase their motivation to start and finish their rehabilitation plan. Although some these devices can be wearable for posture/motion monitoring, they are not suitable for hand rehabilitation, as reduced hardware and real portability are necessary [11], [12], [14], [15]. Hand rehabilitation training must take into account a suitable device wearability [2] and a real-time feedback to both maximize patient safety and obtain good rehabilitation outcomes [1]. Thus, it is necessary to simplify hardware and software of these devices to diversify usage environments, especially to improve the perceived benefits at home rehabilitation [1]. In this context, a wearable device to guide hand rehabilitation routines through finger pressures is introduced here. The device includes instrumental thimble and interacts with the users to motivate them during the rehabilitation routines, and allows creating a database with pressure information and response time, which are obtained during the execution of fingers movements. A pilot test was conducted with ten volunteers using the device and a commercial game. The results demonstrated the potential of the device to be used as a tool for hand rehabilitation training. 2. System Description The proposed system consists of five thimbles that can be placed on the left or right hand, allowing the acquisition of finger pressure information during the execution of pincer movements. The thimbles were designed with the Autodesk TINKERCAD online software (Fig. 1a and 1b), and their 3D printing was done with flexible filament in a FABtotum.
M. Solís-Peña et al. / Procedia Computer Science 160 (2019) 659–664 Author name / Procedia Computer Science 00 (2018) 000–000
661 3
Fig. 1. (a) Model of the 3D thimbles; (b) 3D printed thimbles; (c) thimble with SMD LED.
The device consists of an electronic board, which is based on four open circuits that are closed with the finger contacts. Index, middle, ring and little fingers are independent circuits, whereas the common or closing point of the circuits is the thumb. For the closing of the circuit of each finger, a copper foil is used, which allows recognizing when the little fingers, ring, middle or index finger are pressing on the thumb. For the acquisition of pressure information, a force sensor MF01 was used, which was placed on the thumb. Each thimble has a LED SMD 1206 (Fig c), which is used to identify the correct operation of the closure of the circuit and to identify, by color, the sequence of movements that the user has to perform. For acquisition of information from the force sensor, an Arduino-ATmega microcontroller was used, which also records the number of repetitions of each finger (pinky, ring, middle or index) by pressing the thumb during the finger movements. The calibration code of the force sensor is shown in Fig. 2. void setup () { Serial.begin(9600); } Float sensor = 0; Int cont = 0; void loop() { sensor = analogRead(0); sensor = sensor*0/1024; sensor = 1/(exp(-0.5847*sensor)) - 1; sensor = sensor/5.631819; if (cont == 1024) { Serial.print(*Fuerza aplicada: “); Serial.print(sensor); Serial.println(“Kg”); cont = 0; } cont ++; Fig. 2. Calibration code for the force sensor.
To make communication between device and computer, another Arduino-ATmega was used, which is connected to an interface that allows interacting with a videogame (serious game), and indicates the movement sequences to be performed. Fig. 3. shows the block diagram of the complete system. A driver was built, using an Arduino Uno and Atmel Flip 3.4.7, to allow the recognition of the device by the computer as a videogame controller. Also, a software was developed to transform the Arduino into a “unojoy” type control, which is available on Github. Another software was also developed to implement a driver in the 16U2.
662 4
M. Solís-Peña et al. / Procedia Computer Science 160 (2019) 659–664 Author name / Procedia Computer Science 00 (2018) 000–000
Fig. 3. Block diagram of the system.
For the creation of a database to allow the evaluation of the pressure exerted during the finger movements as well as the number of repetitions, the PLX DAQ software was used, which generates an Excel file with the data that arrives from the serial ports managed by the Arduino. The popular Guitar Hero videogame was used for tests, which is commercially available. Even though this videogame is not specifically designed for rehabilitation purposes, it can be used to test different finger movements routines based on music playlist. Ten healthy subjects were asked to use the device as a guide provided by a videogame, which indicated the sequential finger movements to be performed through a sequence of colors (Fig. 4a). This technique is also used in studies of imagery movements [13]. In our case, the pincer movement is done by pressing the finger with the thumb. The data was processed in Matlab, which calculated the applied forces and the number of repetitions performed by each finger. Fig. 4b shows a volunteer using the device during tests.
Fig. 4 (a) Finger movements based on the color indicated; (b) device used by a volunteer during tests.
3. Results A database was created, which was used to evaluate both the results of pressure during the finger movements and the response time. Fig. 5 shows the force exerted by the index, middle, ring and little fingers when they were pressed
M. Solís-Peña et al. / Procedia Computer Science 160 (2019) 659–664 Author name / Procedia Computer Science 00 (2018) 000–000
663 5
against the thumb during a session of pincer movements of 18 s. It can be seen that the middle finger has the highest peak of strength, and the ring finger has the lowest strength.
Figure 5. Pressure exerted by the index, middle, ring and little fingers when pressing them against the thumb during a session.
The sensor response was considered satisfactory, and the device did not generate any noise that could impair the interpretation of the pressure signals. Fig. 5 shows the pressure of each finger, in which it can be seen that the index and middle fingers exert the greatest pressures, and the annular the smallest one. The pressure information was used to analyze the development of the routine indicated through a videogame interface. The pressure information provided by the device allows it to be used in different rehabilitation routines, especially those based on resistance training on the function of finger joints. This kind of rehabilitation improves the grip strength, which is one of the last stages of rehabilitation, such as reported in [4], and also can be used in typical rehabilitation routines using soft robots for hand rehabilitation [1]. In addition, pressure information can help physiotherapists to restrain patient force in cases of pain due to finger movements. At the moment of placing the device for tests, it is firstly correctly adjusted on the fingers. From the tests, it was observed that the volunteers improved their concentration, as they remained more involved with the videogame. The communication interface worked suitably, allowing a correct communication between the device and the videogame, and there were no sudden movements or failures with the thimbles that could harm the participants. At the end of the tests, all volunteers stated that the device was safe, comfortable during use, easy to operate and easy to put on and take off. This user feedback is a very relevant aspect in the rehabilitation field [1]. 4. Conclusions and future works This work introduced a wearable device to be used for hand rehabilitation, which is small, portable, lightweight, and simple to adjust either to the right or left hand, depending on the user need. The device can be used together a videogame, whose interaction generates motivation to the user while performing finger movements. This indicates the great potential of the device to be used in hand rehabilitation routines. In fact, the literature reports many cases of rehabilitation success when integrating serious games and gaming platforms [6] together traditional rehabilitation. Different modalities of visual and audio feedback can also be integrated with our device to be used in rehabilitation routines aiming learning of different motor tasks [2]. Our device was conceived to be an accessible low-cost tool to recover fine motor functions in users with hand impairments, and able to be used either in clinics or home environments. The device can be placed on the left or
664 6
M. Solís-Peña et al. / Procedia Computer Science 160 (2019) 659–664 Author name / Procedia Computer Science 00 (2018) 000–000
right hand, thus, bilateral movements and movement coordination between both sides can be explored, such as done in other studies [12]. It is worth to comment that the device can be used without the videogame interface, which represents an advantage over other systems that only work with graphical interfaces [12], [15]. Our device also allows that different finger movement routines can be programmed in the microcontroller to guide the user through a sequence indicated by LEDs, and allows acquiring information about movement speed of the fingertip upon actuation. As future works, a serious game will be developed to allow physiotherapists to propose suitable movement sequences for each patient, and clinical trials with patients will be conducted. Also, to increase the device portability, the device will be integrated with a smartphone app, so that the users will be able to perform their rehabilitation guided by the smartphone, and will receive routines updates together physiotherapist comments regarding their rehabilitation progress. Also, based on the volunteers’ feedback, other kind of materials will be considered to build the thimbles, in order to allow better finger adjustments.
Acknowledgements The authors thank all volunteers of this work, and the financial support of UPS/Ecuador. References [1] C.-Y. Chu and R. M. Patterson. (2018) “Soft robotic devices for hand rehabilitation and assistance: a narrative review.” J Neuroeng Rehabil, 15 (1): 9. [2] Q. Wang, P. Markopoulos, B. Yu, W. Chen, and A. Timmermans. (2017) “Interactive wearable systems for upper body rehabilitation: a systematic review.” Journal of NeuroEngineering and Rehabilitation, 14 (1): 20. [3] C. Bütefisch, H. Hummelsheim, P. Denzler, and K. H. Mauritz. (1995) “Repetitive training of isolated movements improves the outcome of motor rehabilitation of the centrally paretic hand.” J. Neurol. Sci. 130 (1): 59–68. [4] W. Xiao, Y. Sheng, H. Zhu, and X. Huo. (2018) “The effect of resistance training on the rehabilitation of hand in rheumatoid arthritis patients.” Chinese Journal of Practical Nursing 34 (3): 176–181. [5] K. N. Arya, S. Pandian, R. Verma, and R. K. Garg. (2011) “Movement therapy induced neural reorganization and motor recovery in stroke: a review.” J Bodyw Mov Ther 15 (4): 528–537. [6] J. P. Proença, C. Quaresma, and P. Vieira. (2018) “Serious games for upper limb rehabilitation: a systematic review.” Disability and Rehabilitation: Assistive Technology 13 (1): 95–100. [7] J. A. M.-M. PhD, W. Z. PhD, M. J. C. PhD, E. H.-V. PhD, and D. B. K. PhD. (2019) “Smartphone-based systems for physical rehabilitation applications: A systematic review.” Assistive Technology, 0 (0): 1–14. [8] J. C. C. Hidalgo, N. M. P. Vásconez, V. E. R. Bykbaev, Á. A. P. Muñoz, and M. E. A. Pinos. (2020) “Development of a Hand Rehabilitation Therapy System with Soft Robotic Glove.” in Advances in Usability and User Experience, 948–958. [9] F. R. C. Machado, P. P. Antunes, J. D. M. Souza, A. C. D. Santos, D. C. Levandowski, and A. A. D. Oliveira. (2017), “Motor Improvement Using Motion Sensing Game Devices for Cerebral Palsy Rehabilitation.” Journal of Motor Behavior 49 (3): 273–280. [10] E. Charters, L. Gillett, and G. K. Simpson. (2015) “Efficacy of electronic portable assistive devices for people with acquired brain injury: A systematic review’, Neuropsychological Rehabilitation 25 (1): 82–121. [11] O. Sandoval-Gonzalez et al. (2016) “Design and Development of a Hand Exoskeleton Robot for Active and Passive Rehabilitation.” International Journal of Advanced Robotic Systems 13 (2): 66. [12] N. Yu, C. Xu, H. Li, K. Wang, L. Wang, and J. Liu. (2016) “Fusion of Haptic and Gesture Sensors for Rehabilitation of Bimanual Coordination and Dexterous Manipulation.” Sensors 16 (3): 395. [13] T. Ueno, M. Inoue, T. Matsuoka, T. Abe, H. Maeda, and K. Morita. (2009) “Comparison Between a Real Sequential Finger and Imagery Movements: An fMRI Study Revisited.” Brain Imaging and Behavior 4: 80–85. [14] A. Hadi, K. Alipour, S. Kazeminasab, and M. Elahinia. (2018) “ASR glove: A wearable glove for hand assistance and rehabilitation using shape memory alloys.” Journal of Intelligent Material Systems and Structures 29 (8): 1575–1585. [15] P. Polygerinos, K. C. Galloway, E. Savage, M. Herman, K. O’Donnell, and C. J. Walsh. (2015) “Soft robotic glove for hand rehabilitation and task specific training’, 2015 IEEE International Conference on Robotics and Automation (ICRA) 2913–29.
ScienceDirect ScienceDirect
Procedia Computer Science 00 (2019) 000–000
Procediaonline Computer 00 (2019) 000–000 Available at Science www.sciencedirect.com
ScienceDirect
www.elsevier.com/locate/procedia www.elsevier.com/locate/procedia
Procedia Computer Science 160 (2019) 659–664
International Workshop on Future Trends in Assistive Technology International Workshop on Future Trends in Assistive Technology
Device to Guide Hand Rehabilitation Routines Based on Pressure Device to Guide Hand Rehabilitation Routines Based on Pressure Signals Signals a a
M. Solís-Peñaaa, A.C. Villa-Parraa,* M. Solís-Peña , A.C. Villa-Parraa,*
Universidad Politécnica Salesiana (UPS), Calle Vieja 12-30, Cuenca, 010110, Ecuador Universidad Politécnica Salesiana (UPS), Calle Vieja 12-30, Cuenca, 010110, Ecuador
Abstract Abstract Several technologies applied to the rehabilitation field use devices that provide to physiotherapists different kind of information Several technologies applied about to the patient rehabilitation fieldduring use devices that provide to physiotherapists different of information to improve the assessment evolution rehabilitation plans. The new trend of these kind technologies is the to improve the about evolution during rehabilitation plans. The new trend of environments. these technologies the development of assessment safe, portable and patient comfortable wearable devices with wide application in different This is paper development of safe,ofportable andand comfortable wearable with wide application in different This paper presents a proposal a portable safe device for handdevices rehabilitation, composed of five thimbles,environments. able to acquire pressure presents a proposal of a portable and safe device rehabilitation, thimbles, able pressure signals during the execution of movements guidedforbyhand a videogame. The composed device hasofa five display to show to to theacquire users different signals the to execution of movements guided a videogame. The device has and a display to show the the users different routines,during in order guide them about its use, andby also evaluates pressure signals response time. to Here design and routines, in order guide about in itsaddition use, and evaluates and response time. Here thethat design and construction of the to device arethem presented, to also results of a pilotpressure test withsignals volunteers. The results demonstrate pressure construction the deviceacquired are presented, addition to results a pilot testwas with volunteers. The demonstrate that pressure signals can beofaccurately duringinuser movements, andofthe device considered safe to results perform hand rehabilitation. signals can be accurately acquired during user movements, and the device was considered safe to perform hand rehabilitation. © 2019 The Authors. Published by Elsevier B.V. © 2019 2019 The Authors. Published by B.V. © The Authors. by Elsevier Elsevier B.V. This is an an open accessPublished articleunder under theCC CCBY-NC-ND BY-NC-NDlicense license (http://creativecommons.org/licenses/by-nc-nd/4.0/) This is open access article the (http://creativecommons.org/licenses/by-nc-nd/4.0/) This is an open access article under theConference CC BY-NC-ND license Peer-review under responsibility of the Program Chairs. Peer-review under responsibility of the Conference Program Chairs.(http://creativecommons.org/licenses/by-nc-nd/4.0/) Peer-review under responsibility of the Conference Program Chairs. Keywords: Hand rehabilitation; pressure; motor learning; serious game Keywords: Hand rehabilitation; pressure; motor learning; serious game
1. Introduction 1. Introduction Currently, hand impairments due to alterations of hand motor function is frequently observed in people affected hand due diseases to alterations of stroke, hand motor function frequently disease observed in arthritis people affected by Currently, neurological andimpairments musculoskeletal such as cerebral palsy,isParkinson’s and [1], [2]. by neurological and musculoskeletal diseases such as stroke, cerebral palsy, Parkinson’s disease and arthritis [1], [2]. Chronic cases also occur due to bad posture during work, non-ergonomic conditions during repetitive movements or Chronic cases also occur due to bad posture during work, non-ergonomic conditions during repetitive movements falls. This produces spasticity, deficits in motor control capabilities, inflammation, muscles weakness and chronic or falls. This produces spasticity, deficits in motor control capabilities, inflammation, muscles weakness and chronic * Corresponding author. Tel.: +5939844586. * E-mail Corresponding Tel.: +5939844586. address:author. [email protected] E-mail address: [email protected] 1877-0509 © 2019 The Authors. Published by Elsevier B.V. 1877-0509 © 2019 Thearticle Authors. Published by Elsevier B.V. This is an open access under the CC BY-NC-ND license (http://creativecommons.org/licenses/by-nc-nd/4.0/) Peer-review under responsibility of the Conference Program Chairs. This is an open access article under CC BY-NC-ND license (http://creativecommons.org/licenses/by-nc-nd/4.0/) Peer-review under responsibility of the Conference Program Chairs.
1877-0509 © 2019 The Authors. Published by Elsevier B.V. This is an open access article under the CC BY-NC-ND license (http://creativecommons.org/licenses/by-nc-nd/4.0/) Peer-review under responsibility of the Conference Program Chairs. 10.1016/j.procs.2019.11.031
660 2
M. Solís-Peña et al. / Procedia Computer Science 160 (2019) 659–664 Author name / Procedia Computer Science 00 (2018) 000–000
pain, leading to different problems, such as incorrect action of grip or tremor [1]. Actually, any loss of movement skill has a great impact on the quality of life, however, as the hand function is fundamental in activities of daily living (ADL), hand rehabilitation is considered among the more critical cases that should be immediately addressed. Hand function recovery involves performing rehabilitation tasks to strengthen the muscles around the fingers to allow increasing the flexibility, improving active joint range of motion, movement performance and movement coordination (to delay the deterioration of bone and cartilaginous tissue) and, consequently, the performance of ADLs [2]. These rehabilitation tasks are normally performed through sequences of repetitive exercises during sessions coordinated with a physiotherapist using resistance training [3], [4]. Satisfactory results depend on the frequency, duration, patient motivation and quality of rehabilitation sessions, especially when these are focused on brain neuroplasticity approaches [5]. In order to improve the results of rehabilitation processes and offer better options for the population that requires physiotherapy assistance, interactive rehabilitation technologies have had a great development in recent years, offering devices that allow movement assistance, user signal acquisition, as well as interaction with virtual environments, serious games and smartphones applications [1], [6], [7]. Among these, wearable devices have been used for user monitoring about posture and upper limb movements, employing a number of sensors, such as: optical linear encoder, inertial measurement unit (IMU), force and knitted piezoresistive fabric [2]. These devices can provide information for physiotherapists to allow them to objectively assess the user progress as well as propose improvements to their rehabilitation plans [8]. In fact, it is reported in the literature that rehabilitation interventions using serious games and electronic reminder systems are useful tools to support rehabilitation plans [9], [10]. Some studies have employed game scenes to guide repetitive movements (grasping activities or finger movements), thus motivating patients to practice or train longer [2], [6], [11]. Virtual scenarios are also employed together movement sensors and rehabilitation robot with haptic, visual and multimodal feedback to design manipulation tasks [2], [9], [12]. On the other hand, devices for monitoring and provision of feedback about posture and movements can be also used to training of different sequential finger movements, aiming to evaluate brain networks of recall, memory, and execution in the cognitive rehabilitation field [13]. Currently, the new trend of technologies applied for hand rehabilitation is the development of portable devices, guaranteeing user safety and comfort [1], [2], [8], [14]. These devices can allow the patient to perform rehabilitation guided by physiotherapists without the need for transfers to clinical centers and without losing the quality of the rehabilitation process. These devices can be also programmed by physiotherapists to act as assistant in this process, allowing the patients to adjust their schedules, reduce costs and increase their motivation to start and finish their rehabilitation plan. Although some these devices can be wearable for posture/motion monitoring, they are not suitable for hand rehabilitation, as reduced hardware and real portability are necessary [11], [12], [14], [15]. Hand rehabilitation training must take into account a suitable device wearability [2] and a real-time feedback to both maximize patient safety and obtain good rehabilitation outcomes [1]. Thus, it is necessary to simplify hardware and software of these devices to diversify usage environments, especially to improve the perceived benefits at home rehabilitation [1]. In this context, a wearable device to guide hand rehabilitation routines through finger pressures is introduced here. The device includes instrumental thimble and interacts with the users to motivate them during the rehabilitation routines, and allows creating a database with pressure information and response time, which are obtained during the execution of fingers movements. A pilot test was conducted with ten volunteers using the device and a commercial game. The results demonstrated the potential of the device to be used as a tool for hand rehabilitation training. 2. System Description The proposed system consists of five thimbles that can be placed on the left or right hand, allowing the acquisition of finger pressure information during the execution of pincer movements. The thimbles were designed with the Autodesk TINKERCAD online software (Fig. 1a and 1b), and their 3D printing was done with flexible filament in a FABtotum.
M. Solís-Peña et al. / Procedia Computer Science 160 (2019) 659–664 Author name / Procedia Computer Science 00 (2018) 000–000
661 3
Fig. 1. (a) Model of the 3D thimbles; (b) 3D printed thimbles; (c) thimble with SMD LED.
The device consists of an electronic board, which is based on four open circuits that are closed with the finger contacts. Index, middle, ring and little fingers are independent circuits, whereas the common or closing point of the circuits is the thumb. For the closing of the circuit of each finger, a copper foil is used, which allows recognizing when the little fingers, ring, middle or index finger are pressing on the thumb. For the acquisition of pressure information, a force sensor MF01 was used, which was placed on the thumb. Each thimble has a LED SMD 1206 (Fig c), which is used to identify the correct operation of the closure of the circuit and to identify, by color, the sequence of movements that the user has to perform. For acquisition of information from the force sensor, an Arduino-ATmega microcontroller was used, which also records the number of repetitions of each finger (pinky, ring, middle or index) by pressing the thumb during the finger movements. The calibration code of the force sensor is shown in Fig. 2. void setup () { Serial.begin(9600); } Float sensor = 0; Int cont = 0; void loop() { sensor = analogRead(0); sensor = sensor*0/1024; sensor = 1/(exp(-0.5847*sensor)) - 1; sensor = sensor/5.631819; if (cont == 1024) { Serial.print(*Fuerza aplicada: “); Serial.print(sensor); Serial.println(“Kg”); cont = 0; } cont ++; Fig. 2. Calibration code for the force sensor.
To make communication between device and computer, another Arduino-ATmega was used, which is connected to an interface that allows interacting with a videogame (serious game), and indicates the movement sequences to be performed. Fig. 3. shows the block diagram of the complete system. A driver was built, using an Arduino Uno and Atmel Flip 3.4.7, to allow the recognition of the device by the computer as a videogame controller. Also, a software was developed to transform the Arduino into a “unojoy” type control, which is available on Github. Another software was also developed to implement a driver in the 16U2.
662 4
M. Solís-Peña et al. / Procedia Computer Science 160 (2019) 659–664 Author name / Procedia Computer Science 00 (2018) 000–000
Fig. 3. Block diagram of the system.
For the creation of a database to allow the evaluation of the pressure exerted during the finger movements as well as the number of repetitions, the PLX DAQ software was used, which generates an Excel file with the data that arrives from the serial ports managed by the Arduino. The popular Guitar Hero videogame was used for tests, which is commercially available. Even though this videogame is not specifically designed for rehabilitation purposes, it can be used to test different finger movements routines based on music playlist. Ten healthy subjects were asked to use the device as a guide provided by a videogame, which indicated the sequential finger movements to be performed through a sequence of colors (Fig. 4a). This technique is also used in studies of imagery movements [13]. In our case, the pincer movement is done by pressing the finger with the thumb. The data was processed in Matlab, which calculated the applied forces and the number of repetitions performed by each finger. Fig. 4b shows a volunteer using the device during tests.
Fig. 4 (a) Finger movements based on the color indicated; (b) device used by a volunteer during tests.
3. Results A database was created, which was used to evaluate both the results of pressure during the finger movements and the response time. Fig. 5 shows the force exerted by the index, middle, ring and little fingers when they were pressed
M. Solís-Peña et al. / Procedia Computer Science 160 (2019) 659–664 Author name / Procedia Computer Science 00 (2018) 000–000
663 5
against the thumb during a session of pincer movements of 18 s. It can be seen that the middle finger has the highest peak of strength, and the ring finger has the lowest strength.
Figure 5. Pressure exerted by the index, middle, ring and little fingers when pressing them against the thumb during a session.
The sensor response was considered satisfactory, and the device did not generate any noise that could impair the interpretation of the pressure signals. Fig. 5 shows the pressure of each finger, in which it can be seen that the index and middle fingers exert the greatest pressures, and the annular the smallest one. The pressure information was used to analyze the development of the routine indicated through a videogame interface. The pressure information provided by the device allows it to be used in different rehabilitation routines, especially those based on resistance training on the function of finger joints. This kind of rehabilitation improves the grip strength, which is one of the last stages of rehabilitation, such as reported in [4], and also can be used in typical rehabilitation routines using soft robots for hand rehabilitation [1]. In addition, pressure information can help physiotherapists to restrain patient force in cases of pain due to finger movements. At the moment of placing the device for tests, it is firstly correctly adjusted on the fingers. From the tests, it was observed that the volunteers improved their concentration, as they remained more involved with the videogame. The communication interface worked suitably, allowing a correct communication between the device and the videogame, and there were no sudden movements or failures with the thimbles that could harm the participants. At the end of the tests, all volunteers stated that the device was safe, comfortable during use, easy to operate and easy to put on and take off. This user feedback is a very relevant aspect in the rehabilitation field [1]. 4. Conclusions and future works This work introduced a wearable device to be used for hand rehabilitation, which is small, portable, lightweight, and simple to adjust either to the right or left hand, depending on the user need. The device can be used together a videogame, whose interaction generates motivation to the user while performing finger movements. This indicates the great potential of the device to be used in hand rehabilitation routines. In fact, the literature reports many cases of rehabilitation success when integrating serious games and gaming platforms [6] together traditional rehabilitation. Different modalities of visual and audio feedback can also be integrated with our device to be used in rehabilitation routines aiming learning of different motor tasks [2]. Our device was conceived to be an accessible low-cost tool to recover fine motor functions in users with hand impairments, and able to be used either in clinics or home environments. The device can be placed on the left or
664 6
M. Solís-Peña et al. / Procedia Computer Science 160 (2019) 659–664 Author name / Procedia Computer Science 00 (2018) 000–000
right hand, thus, bilateral movements and movement coordination between both sides can be explored, such as done in other studies [12]. It is worth to comment that the device can be used without the videogame interface, which represents an advantage over other systems that only work with graphical interfaces [12], [15]. Our device also allows that different finger movement routines can be programmed in the microcontroller to guide the user through a sequence indicated by LEDs, and allows acquiring information about movement speed of the fingertip upon actuation. As future works, a serious game will be developed to allow physiotherapists to propose suitable movement sequences for each patient, and clinical trials with patients will be conducted. Also, to increase the device portability, the device will be integrated with a smartphone app, so that the users will be able to perform their rehabilitation guided by the smartphone, and will receive routines updates together physiotherapist comments regarding their rehabilitation progress. Also, based on the volunteers’ feedback, other kind of materials will be considered to build the thimbles, in order to allow better finger adjustments.
Acknowledgements The authors thank all volunteers of this work, and the financial support of UPS/Ecuador. References [1] C.-Y. Chu and R. M. Patterson. (2018) “Soft robotic devices for hand rehabilitation and assistance: a narrative review.” J Neuroeng Rehabil, 15 (1): 9. [2] Q. Wang, P. Markopoulos, B. Yu, W. Chen, and A. Timmermans. (2017) “Interactive wearable systems for upper body rehabilitation: a systematic review.” Journal of NeuroEngineering and Rehabilitation, 14 (1): 20. [3] C. Bütefisch, H. Hummelsheim, P. Denzler, and K. H. Mauritz. (1995) “Repetitive training of isolated movements improves the outcome of motor rehabilitation of the centrally paretic hand.” J. Neurol. Sci. 130 (1): 59–68. [4] W. Xiao, Y. Sheng, H. Zhu, and X. Huo. (2018) “The effect of resistance training on the rehabilitation of hand in rheumatoid arthritis patients.” Chinese Journal of Practical Nursing 34 (3): 176–181. [5] K. N. Arya, S. Pandian, R. Verma, and R. K. Garg. (2011) “Movement therapy induced neural reorganization and motor recovery in stroke: a review.” J Bodyw Mov Ther 15 (4): 528–537. [6] J. P. Proença, C. Quaresma, and P. Vieira. (2018) “Serious games for upper limb rehabilitation: a systematic review.” Disability and Rehabilitation: Assistive Technology 13 (1): 95–100. [7] J. A. M.-M. PhD, W. Z. PhD, M. J. C. PhD, E. H.-V. PhD, and D. B. K. PhD. (2019) “Smartphone-based systems for physical rehabilitation applications: A systematic review.” Assistive Technology, 0 (0): 1–14. [8] J. C. C. Hidalgo, N. M. P. Vásconez, V. E. R. Bykbaev, Á. A. P. Muñoz, and M. E. A. Pinos. (2020) “Development of a Hand Rehabilitation Therapy System with Soft Robotic Glove.” in Advances in Usability and User Experience, 948–958. [9] F. R. C. Machado, P. P. Antunes, J. D. M. Souza, A. C. D. Santos, D. C. Levandowski, and A. A. D. Oliveira. (2017), “Motor Improvement Using Motion Sensing Game Devices for Cerebral Palsy Rehabilitation.” Journal of Motor Behavior 49 (3): 273–280. [10] E. Charters, L. Gillett, and G. K. Simpson. (2015) “Efficacy of electronic portable assistive devices for people with acquired brain injury: A systematic review’, Neuropsychological Rehabilitation 25 (1): 82–121. [11] O. Sandoval-Gonzalez et al. (2016) “Design and Development of a Hand Exoskeleton Robot for Active and Passive Rehabilitation.” International Journal of Advanced Robotic Systems 13 (2): 66. [12] N. Yu, C. Xu, H. Li, K. Wang, L. Wang, and J. Liu. (2016) “Fusion of Haptic and Gesture Sensors for Rehabilitation of Bimanual Coordination and Dexterous Manipulation.” Sensors 16 (3): 395. [13] T. Ueno, M. Inoue, T. Matsuoka, T. Abe, H. Maeda, and K. Morita. (2009) “Comparison Between a Real Sequential Finger and Imagery Movements: An fMRI Study Revisited.” Brain Imaging and Behavior 4: 80–85. [14] A. Hadi, K. Alipour, S. Kazeminasab, and M. Elahinia. (2018) “ASR glove: A wearable glove for hand assistance and rehabilitation using shape memory alloys.” Journal of Intelligent Material Systems and Structures 29 (8): 1575–1585. [15] P. Polygerinos, K. C. Galloway, E. Savage, M. Herman, K. O’Donnell, and C. J. Walsh. (2015) “Soft robotic glove for hand rehabilitation and task specific training’, 2015 IEEE International Conference on Robotics and Automation (ICRA) 2913–29.