- Email: [email protected]
Analysis of upper limb movement for biofeedback in rehabilitation
Available online at www.sciencedirect.com
ScienceDirect
Analysis of upper limb movement for biofeedback in rehabilitation IFAC PapersOnLine 52-27 (2019) 545–549 Analysis limb Analysis of of upper upper limb movement movement for for biofeedback biofeedback in in rehabilitation rehabilitation Klara Fiedorova, Veronika Baladova, Lukas Peter *** Analysis of upper limb movement for biofeedback in rehabilitation Klara Fiedorova, Veronika Baladova, Lukas Klara Fiedorova, Veronikafor Baladova, Lukas Peter Peter *** Analysis of upper limb movement biofeedback in*** rehabilitation
Listopadu 15 Ostrava-Poruba, Czech Republic;(e-mail: VSB-Technical University ofKlara Ostrava, FEECS, Veronika K450, 17. Fiedorova, Baladova, Lukas Peter *** [email protected], [email protected], [email protected]). Klara Fiedorova, Veronika Lukas Peter *** Czech Republic;(e-mail: Baladova, VSB-Technical University of Ostrava, FEECS, K450, 17. Listopadu 15 Ostrava-Poruba, VSB-Technical University of Ostrava, FEECS, K450, 17. Listopadu 15 Ostrava-Poruba, Czech Republic;(e-mail: [email protected], [email protected], [email protected]). [email protected]). [email protected], [email protected], VSB-Technical University of Ostrava, FEECS, K450, 17. Listopadu 15 Ostrava-Poruba, Czech Republic;(e-mail: VSB-Technical University of Ostrava, FEECS, K450, 17. Listopadu 15 Ostrava-Poruba, Czech Republic;(e-mail: [email protected], [email protected], [email protected]). [email protected], [email protected], Abstract: The work deals with sensing the movement of upper [email protected]). using biofeedback and subsequent use in rehabilitation. is important to get the problems of the motor system, physiology Abstract: The work work It deals with sensing sensing theacquainted movementwith of upper upper limbs using using biofeedback and subsequent subsequent Abstract: The deals with the movement of limbs biofeedback and and pathology of the upper limbs and methods of their rehabilitation. Furthermore, types and applications use in in rehabilitation. rehabilitation. It It is is important to to get get acquainted acquainted with with the the problems problems of of the the motor motor system, system, physiology physiology use Abstract: The work dealsimportant with sensing the movement ofthe upper limbs using biofeedback anddescribes subsequent of biofeedback inthe rehabilitation are discussed. A part ofrehabilitation. thesis is also a research, which the and pathology of upper limbs and methods of their Furthermore, types and applications and pathology of the upper limbs and methods of their rehabilitation. Furthermore, types and applications Abstract: The work deals with sensing the movementwith ofofupper limbs using biofeedback andused. subsequent use ininrehabilitation. It is important to get acquainted the problems of the motor system, physiology state the area of sensing and analysis of movements upper limbs and types of sensors In the of biofeedback in rehabilitation are discussed. A part of the thesis is also research, which describes the of inwork rehabilitation are discussed. part thethe thesis isprocessing also research, which the use in rehabilitation. It is important tomethods get acquainted with problems ofaa the motor system, physiology andbiofeedback pathology of the upper limbs and ofmeasuring, theirof rehabilitation. Furthermore, types anddescribes next part ofarea the were described ways ofA signal and building aapplications measuring state in the of sensing and analysis of movements of upper limbs and types of sensors used. In the state in the area of sensing and analysis of movements of upper limbs and types of sensors used. In the and pathology of the upper limbs and methods of their rehabilitation. Furthermore, types and applications of biofeedback in rehabilitation discussed.ofAthe partforearm of the thesis is alsowhich a research, which describes the chain for of scanning the were rangedescribed ofare movements and wrist, is connected a PC and next part the work ways of measuring, signal and building aato measuring next part ofarea theinof work were described ways ofAinterface measuring, signal processing and building measuring of biofeedback rehabilitation are discussed. part of of the thesislimbs isprocessing also aform research, which describes state in the sensing and analysis ofuser movements upper and types of sensors used. In the provides information for rehabilitation. A is created in the of a rehabilitation game, chain scanning the range of movements of forearm and wrist, which is connected to aa PC chain for scanning the were range of movements of the the forearm and limbs wrist, which is results connected PC and state infor theof area of sensing analysis of movements of upper and types ofbuilding sensors used. In and the next part the work described ways of measuring, signal processing measuring using visual biofeedback toand improve the course of rehabilitation. From theand and ato analysis of provides information for rehabilitation. A user interface is created in the form of a rehabilitation game, provides information for rehabilitation. A user interface is created in the form of a rehabilitation game, next part of the work were described ways of measuring, signal processing and building a measuring chain for scanning the range of movements the forearm and wrist, which is connected to a PC and measurements it can be said that the proposed system contributes to the improvement of movements, using visual biofeedback to of improve the course course of rehabilitation. From theis results results and and toanalysis analysis of using visual biofeedback to improve the rehabilitation. the of chain for scanning thefor range movements of interface the of forearm and wrist, which a PC and provides information rehabilitation. A user is created in From the form ofconnected a rehabilitation game, their coordination and accuracy. measurements it can be said that the proposed system contributes to the improvement of movements, measurements it can be said that the proposed system contributes to the improvement of movements, provides information for rehabilitation. A user interface is created in From the form a rehabilitation game, using visual biofeedback to improve the course of rehabilitation. the ofresults and analysis of their coordination and accuracy. © 2019, IFAC (International Federation Automatic Control) Hosting Biofeedback, by Elsevier All rights reserved. their coordination andbe accuracy. Keywords: Accelerometer, Gyroscope, Upper limb motion sensing, Rehabilitation. using visual biofeedback to improve the course of rehabilitation. theLtd. results and analysis of measurements it can said that the of proposed system contributes toFrom the improvement of movements, measurements it can said that the proposed system contributes to the improvement of movements, their coordination andbe accuracy. Keywords: Accelerometer, Gyroscope, Keywords: Accelerometer, Gyroscope, Upper Upper limb limb motion motion sensing, sensing, Biofeedback, Biofeedback, Rehabilitation. Rehabilitation. their coordination and accuracy. the middle brain. The cerebellum is important for motor Keywords: Gyroscope, Upper limb motion sensing, Biofeedback, Rehabilitation. 1. Accelerometer, INTRODUCTION control, muscle tone and voluntary movements. The motor basal Keywords: Accelerometer, Gyroscope, Upper limb motion sensing, Biofeedback, Rehabilitation. the middle brain. The cerebellum is important the middle brain. The cerebellum is controlling important for for motor ganglia are the four nuclei in the brain voluntary 1.onINTRODUCTION INTRODUCTION The work focuses 1. the development of rehabilitation control, muscle tone and voluntary movements. The basal control, muscle tone voluntary The basal the brain. Theandcerebellum ismovements. important motor and middle involuntary movements. The last centre is thefor cerebral techniques for the upper limbs ensuring complex movement. ganglia are the four nuclei in the brain controlling voluntary 1. INTRODUCTION The work focuses on the development of rehabilitation ganglia are the four nuclei in the brain controlling voluntary the middle brain. The ismovements. important for motor The work etfocuses on the of rehabilitation control, muscle tone andcerebellum voluntaryon The basal cortex, where the final decision movement iscerebral made. (Szelitzky al., 2011) Thedevelopment control of human motor is and 1. INTRODUCTION involuntary movements. The last centre is techniques for the upper upper limbs ensuring complex movement. and involuntary movements. The last centre is the theThe cerebral control, muscle tone and voluntary movements. basal techniques for the limbs ensuring complex movement. ganglia are the four nuclei in the brain controlling voluntary (Szelitzky et al., 2011), (Wu et al., 2016) The work focuses onthethe development of rehabilitation accomplished through central nervous system. (Szelitzky cortex, where the final on movement is made. (Szelitzky et al., 2011) The control of human motor is cortex, where the final decision on centre movement iscerebral made. ganglia are the four nucleidecision in The the brain controlling voluntary (Szelitzky et al., 2011) The control of human motor is and involuntary movements. last is the The work focuses on the development of rehabilitation techniques for the upper limbs ensuring complex movement. et al., 2011) Skeletal muscle contraction and relaxation is the (Szelitzky et 2011), (Wu et al., 2016) accomplished through the central nervous system. (Szelitzky (Szelitzky et al., al.,the 2011), etThe al., last 2016) involuntary movements. is theiscerebral accomplished thelimbs central nervous system. (Szelitzky cortex, where final(Wu decision on centre movement made. techniques for themovement. upper ensuring complex movement. (Szelitzky et through al., 2011) The control of human motor is and basis for active The contraction is controlled 1.2 Pathophysiology et al., 2011) Skeletal muscle contraction and relaxation is the cortex, where the final decision on movement is made. et al., 2011) Skeletal muscle contraction and relaxation is the (Szelitzky et al., 2011), (Wu et al., 2016) (Szelitzky et al., 2011) The control of human motor is accomplished through the central nervous system. (Szelitzky directly the movement. CNS. All The reactions take place at the basis for by active contraction is controlled (Szelitzky et al., 2011), (Wu et al., 2016) basis active movement. Thenervous contraction is (Szelitzky controlled accomplished through the by central system. 1.2 Pathophysiology Pathophysiology et al., for 2011) Skeletal muscle contraction and relaxation is the 1.2 neuromuscular plate level chemical reactions. Pathophysiology of the upper limbs has many causes. These directly by the CNS. All reactions take place at the directly by the CNS. All reactions take place at the et al., for 2011) Skeletal muscle contraction and relaxation is basis active movement. The contraction is controlled 1.2 Pathophysiology are congenital or of acquired disorders occurring directly on the neuromuscular plate level by reactions. limbs causes. neuromuscular plate levelsystem by chemical chemical reactions. The activity ofthe the motor is divided into place voluntary basis for by active movement. The contraction is controlled Pathophysiology of the the upper upper limbs has hasormany many causes. These These directly CNS. All reactions take at and the Pathophysiology 1.2 Pathophysiology limb; they are muscular dystrophies inflammations. Or congenital or acquired disorders occurring directly on the involuntary. (Wu et al., system 2016) The involuntary movements directly by of the CNS. All reactions take place at and the are are congenital or acquired disorders occurring directly on the neuromuscular plate level by chemical reactions. The activity the motor is divided into voluntary Pathophysiology of the upper limbs has many causes. These arise in centres and lanes of motor control. There are various The activity of the motor system is divided into voluntary and they are dystrophies or inflammations. Or involve the control of the position and movements the muscle limb; neuromuscular plate byupright chemical reactions. limb; they areormuscular muscular dystrophies ormany inflammations. Or Pathophysiology of the plegia, upper limbs has causes. These involuntary. (Wu et level al., 2016) The involuntary are congenital acquired disorders directly on the degrees of paresis and it iscontrol. aoccurring disorder of are the various central involuntary. (Wu al., is 2016) The involuntary movements arise in centres and lanes of motor There The activity of the et motor system is divided into voluntary and tone. This information found in the brainstem and is arise in centres and lanes of motor control. There are various are congenital acquired disorders occurring directly on the involve the control of the upright position and the limb; they areor muscular or inflammations. Or motor neuron, the cause is dystrophies stroke, trauma, tumors or multiple involve thebyof control of upright position and the muscle muscle The activity the cerebellum, motor system is divided into voluntary and degrees of paresis and plegia, it aa disorder of the central involuntary. (Wu et al.,the 2016) The involuntary movements controlled the basal ganglia and subordinate degrees ofAnother paresis and plegia, it is iscontrol. disorder of are the various central limb; they are muscular dystrophies or inflammations. Or tone. This information is found in the brainstem and is arise in centres and lanes of motor There sclerosis. group are disorders of the extrapyramidal tone. This is2016) found the brainstem andthe is motor neuron, the cause is stroke, trauma, tumors or multiple involuntary. (Wu et al.,the Theinposition involuntary involve the information control of upright and movements thefrom muscle to the cortex. Free movement isbasal induced by impulses motor the cause isofstroke, trauma, tumors or multiple arise inneuron, centres and lanes motor control. There are various controlled by the cerebellum, ganglia and subordinate degrees of paresis and plegia, it is a disorder of the central system, which are disorders of the basal ganglia and controlled by the cerebellum, basal ganglia and subordinate involve the control of ganglia the the muscle group are disorders of extrapyramidal tone. This information is upright found the brainstem and is sclerosis. cerebral cortex, basal andinposition cerebellum. (Wu et al., sclerosis. Another group disorders of the the extrapyramidal degrees ofAnother paresis and plegia, it istrauma, a disorder oforor themultiple central to the Free movement is induced by from motor neuron, the cause isare stroke, tumors cerebellum causing Parkinson's disease various to the cortex. cortex. Free movement isbasal induced by impulses impulses from the tone. This by information is found inganglia the brainstem andthe is system, which are disorders of the basal ganglia and controlled the cerebellum, and subordinate 2016) system, which are disorders of the basal ganglia and motor neuron, the cause is stroke, trauma, tumors or multiple cerebral cortex, basal ganglia and cerebellum. (Wu et al., sclerosis. Another group syndromes are disorderssuch of theasextrapyramidal involuntary movement Huntington's cerebral cortex, gangliaisbasal and cerebellum. et al., controlled by Free thebasal cerebellum, ganglia and (Wu subordinate causing Parkinson's disease or various to the cortex. movement induced by impulses from the cerebellum cerebellum causing Parkinson's disease or various sclerosis. Another group are disorders of the extrapyramidal 2016) system, which are disorders of the basal ganglia and disease. (Szelitzky et al., 2011) 2016) The muscle is controlled by cerebellum. motoneuron. Thefrom central to theskeletal cortex. Freebasal movement by impulses the involuntary movement syndromes such as cerebral cortex, gangliais induced and (Wu et al., involuntary movement syndromes such as Huntington's Huntington's system, which are disorders of the basal ganglia and cerebellum causing Parkinson's disease or various motoneuron is part of the descending motor (Wu track. cerebral cortex, basal ganglia and cerebellum. et The al., disease. (Szelitzky et 2011) 2016) The skeletal muscle is controlled by motoneuron. The central disease. (Szelitzky et al., al.,Parkinson's 2011) cerebellum causing disease or various The skeletal muscle is controlled by motoneuron. The central involuntary movement syndromes such as Huntington's peripheral motoneuron is then part of the anterior spinal horn. 2016) Rehabilitation of upper limbs - biofeedback motoneuron is part the descending motor track. The involuntary movement syndromes such as Huntington's motoneuron is whose partisof of the forms descending motor track. The 1.3 disease. (Szelitzky et al., 2011) The skeletal muscle controlled byamotoneuron. The central It is a neuron axon motor nerve that goes peripheral motoneuron is then part of the anterior spinal horn. 1.3 Rehabilitation of upper limbs biofeedback disease. (Szelitzky et al., 2011) peripheral motoneuron is then part of the anterior spinal horn. The skeletal muscle is controlled by motoneuron. The central 1.3 Rehabilitation of upper limbs biofeedback motoneuron is part of the descending motor track. The directly to the muscle. (Szelitzky et al., 2011)nerve that goes In the rehabilitation of the upper limbs, the complexity of It a motor It is is aa neuron neuron whose axon forms motor nerve that horn. goes motoneuron is whose part ofaxon the forms descending motor track. The 1.3 Rehabilitation of upper limbs - biofeedback peripheral motoneuron is then part ofa the anterior spinal movement of all jointsofand the fine limbs, motor the skillscomplexity of the hands directly to the (Szelitzky al., 2011) directly to motoneuron the muscle. muscle. etof al., 2011) the rehabilitation the upper of peripheral is thenforms partet anterior spinal In the rehabilitation of the limbs upper limbs, the complexity of It is a neuron whose (Szelitzky axon a the motor nerve that horn. goes In 1.3 Rehabilitation of upper biofeedback are important. Rehabilitation is done in several ways, the 1.1 Momentum control movement of all joints and the fine motor skills of the hands It is a neuron whose (Szelitzky axon forms a motor of all joints and the fine motor skills of the hands directly to the muscle. et al., 2011)nerve that goes movement In the step rehabilitation ofexercise the upper limbs, the complexity of basic is medical or kinesiotherapy, but also are important. Rehabilitation is done in several ways, the 1.1 control directly to the muscle. are important. Rehabilitation isfine done inetthe several ways, the 1.1 Momentum Momentum control(Szelitzky et al., 2011) In the rehabilitation of the upper limbs, complexity of movement of all joints and the motor skills of the hands physical therapy or ergotherapy. (Giggins al., 2013) Movement and movement control is performed in the spinal basic step is medical exercise or kinesiotherapy, but also basic step is medical exercise or kinesiotherapy, but also movement of all joints and the fine motor skills of the hands are important. Rehabilitation is done in several ways, the 1.1 control cord,Momentum brainstem, cerebellum, basal ganglia and cerebral physical therapy or ergotherapy. et al., 2013) Movement and movement control is in the physical therapy orbiological ergotherapy. (Giggins etseveral al., to 2013) Biofeedback systems isinused present are important. Rehabilitation is(Giggins done ways, thea Movement and movement control is performed performed in the spinal spinal 1.1 Momentum control basic step is of medical exercise or kinesiotherapy, but also cortex. The spinal cord is the lowest motor centre. It is cord, brainstem, cerebellum, basal ganglia and cerebral measured quantity or other sensations to a patient and is basic step is medical exercise or kinesiotherapy, but also cord, brainstem, cerebellum, basal ganglia and cerebral physical therapy or ergotherapy. (Giggins et al., 2013) of systems is to Movement andhigher movement isand performed ininformation the spinal Biofeedback controlled by brainiscontrol systems receives Biofeedback of biological biological systems is used used to present present aa cortex. The spinal cord the lowest motor centre. It is expected to respond. (Giggins et al., 2013) For rehabilitation, physical therapy or ergotherapy. (Giggins et al., 2013) cortex. The spinal cord is the lowest motor centre. It is Movement and movement control is performed in the spinal quantity or sensations to aa patient and is cord, brainstem, cerebellum, basal ganglia and cerebral from the skin, muscles andsystems tendons. Another centre is the measured measured quantity or other other sensations andinto isa controlled by higher brain and receives information Biofeedback biological systems is to used patient to present biofeedback isofdivided according to 2013) measured quantity controlled higher brain andganglia receives information cord, brainstem, cerebellum, and cerebral to respond. (Giggins et al., For rehabilitation, cortex.stem Theby spinal cord issystems thebasal lowest motor centre. Itand is expected brain dividing into the spinal cord, the Varol bridge expected to respond. (Giggins et al., 2013) For rehabilitation, Biofeedback of biological is to used to present from skin, muscles and tendons. Another is measured quantity or other systems sensations a patient and isa from the theThe skin, muscles and tendons. centre isItthe the cortex. cord issystems the lowest motor centre centre. is biofeedback is according to measured quantity controlled byspinal higher brain andAnother receives information biofeedback is divided divided according to 2013) measured quantity into measured quantity or other sensations to a patient andinto is brain stem dividing into the spinal cord, the Varol bridge and expected to respond. (Giggins et al., For rehabilitation, brain stem dividing into the spinal cord, the Varol bridge controlled by higher brain andAnother receives information from the skin, muscles andsystems tendons. centre is and the expected to respond. (Giggins et al., 2013) For rehabilitation, biofeedback is divided according to measured quantity into from the skin, muscles andspinal tendons. centre is and the brain stem dividing into the cord,Another the Varol bridge biofeedback is divided according to measured quantity into brain stem into the spinal Federation cord, the Varol bridgeControl) and Hosting by Elsevier Ltd. All rights reserved. 2405-8963 © dividing 2019, IFAC (International of Automatic Peer review under responsibility of International Federation of Automatic Control. 10.1016/j.ifacol.2019.12.720
546
Klara Fiedorova et al. / IFAC PapersOnLine 52-27 (2019) 545–549
physiological (electrical activity of the brain, muscles or heart) and biomechanical quantities (movement, posture control, strength).
monitor patients under certain conditions. This app is based on a small wireless device with an accelerometer that is connected to the human wrist. (Peruzzini et al., 2014)
Among the methods using biofeedback, EMG measurements, specifically changes in the measured muscle, are useful. Biomechanical methods are also used, such as electrogoniometry, video camera scanning and sensor measurements. Using accelerometers and gyroscopes, realtime motion information is obtained. This makes it easy to obtain information about limb movement and to evaluate the extent of movement and the quality of rehabilitation. (Giggins et al., 2013)
3. MEASURING EQUIPMENT
2. RESEARCH OF THE STATE OF REHABILITATION TECHNOLOGY The research deals with systems for sensing and analysing movement of hands or body. There are many different systems dealing with the subject, most often focused on sensing whole body movement or 3D hand movement. (Wu et al., 2016) Often these systems have defined several movements they detected (sitting, body rotation, gestures). These systems serve as support for rehabilitation. (Wu et al., 2016) In one work, hand movements are captured that are displayed using a 3D hand model, giving the ability to save, retrieve and visualize data. The glove system prototype includes seven sensors. (Szelitzky et al., 2011) In the next article, the authors present a home rehabilitation system using a network of body sensors to measure the range of motion in rehabilitation. Patients will be able to perform exercises anywhere, only with wireless sensors deployed, sending measured data wirelessly to a module working with the Raspberry Pi minicomputer. (Daponte et al., 2013) One team presented a system that uses virtual prototypes. An exoskeleton was used to sense the muscle activity sending data via Bluetooth, and the PC application then displayed the movement using the avatar in real time. (Peruzzini et al., 2014) There is also a Helping hand option to assist with monoplegia therapy. It is a glove with LDR sensors, a computer application of racing game was designed for the system for higher efficiency of rehabilitation. (Ar Rahman et al., 2014) Sensors such as accelerometer for 3D acceleration with gyroscope and magnetometer in one sensor were often used for motion sensing. (Sarcevic et al., 2015) These sensors are utilized by the IRIS system, which includes nine degrees of freedom (9DOF) sensor boards. (Sarcevic et al., 2015) Somewhere only accelerometer, EMG sensors, LDR sensors or resistance change sensors were used. One system also used the Wiimote driver for gesture recognition. Data from accelerometers were evaluated from the controller and processed using the Hidden Markov Models method. (Wu et al., 2016) IEEE 802.15.4 protocols were used most often for data transfer from sensors (Daponte et al., 2013), (Sarcevic et al., 2015) Some systems used Bluetooth, Zigbee, Ethernet or cable transmission. (Daponte et al., 2013) Wireless biomedical sensor networks have also been created to
Based on the research, a module containing an accelerometer and gyroscope was chosen, which should be the most suitable for sensing the rotation of the wrist. In this work we decided to use a six-axis IMU sensor (MPU6050), offering three values from an accelerometer and three from a gyroscope. We decided to use Arduino Leonardo to access the sensor. Combining the values provides useful information about hand orientation and tilt. Values were obtained by calculating equation 1 and dividing by the Ludolf number to obtain degrees. (1) Where x, y and z is For further processing, a DMP (digital motion processor) element is used, combining values with an accelerometer and a gyroscope. The data is converted to rotation values around the axes. Rotations are called Yaw (rotation around the Z axis), Pitch (rotation around the Y axis), and Roll (rotation around the X axis) with parameters of quaternion and gravity. The accuracy of the sensor was also verified using a protractor and simultaneously measuring and displaying the values on a PC. The accuracy of the sensor was calculated from the measured data, which was set to ± 2%. 4. REHABILITATION OF THE WRIST AND FOREARM The realization of prototypes (Figure 1) for rehabilitation was based on four hand and forearm movements. These are palmar and dorsal flexion, supination and pronation (Figure 2). These movements are important for fine motor skills as well as for larger wrist movements. It is possible to measure the range of motion and use the movements to control the movement of the mouse cursor. With palmar flexion, the cursor moves to the left, with dorsal flexion to the right, with supination the cursor moves up and with pronation down.
Fig. 1. First prototype
Klara Fiedorova et al. / IFAC PapersOnLine 52-27 (2019) 545–549
547
determined for two weeks. Both hands were tested for each of the subjects to improve coordination and precision of movement, as well as to identify differences between the dominant and non-dominant hands. After each measurement, the test person's task was to draw a square on the paper with both hands at the same time to better evaluate the test results. The testing method is shown in Figure 4. Fig. 2. Hand movements. From left: palmar flexion, dorsal flexion, supination, pronation 4.1 Program part This program for data processing and analysis is inspired by Jeff Rowberg's program. The program uses I2C and interrupt communications to retrieve DMP (digital motion processor) data from the MPU 6050. This data is read from FIFO memory. A Mouse library has been added to control the movement of the mouse cursor, which allows Arduino Leonardo. 4.2 Exercise Game A user interface was created in the Matlab environment for a simple practice game. Three basic shapes (square, rectangle and circle) are used. These shapes were chosen for their simplicity. The task of the person performing the task is to keep the mouse cursor within the given shape, ie to make as few mistakes as possible and to circle the shape twice clockwise in the shortest possible time. The shapes tested are shown in Figure 3.
Fig. 4. Testing Method 5.1 Statistical data processing In the statistical analysis, the null hypothesis H_0 equals to improving the time to accomplish the task, while the alternative hypothesis H_A means that the test subject has not improved. Hypotheses are verified using graphs. To give you an idea of the charts for the age category 36-55, see graphs 1, 2, 3.
Fig. 3. Tested shapes
Graph 1 Dependence of time on number of measurements square
Moving the mouse cursor after starting the measurement starts the motion curve and starts measuring the time and error. When the cursor is inside a shape, the curve will turn blue. When it crosses the shape boundary, it turns red. The colour of the curve gives a clear visual biofeedback. The blue colour inside the shape shows that everything is OK, otherwise it will be red, which forces the person to return as soon as possible. Using this biofeedback, the test person's reactions are accelerated. When the test person crosses the edge of the object with the mouse cursor, the first error is counted and the time to return between object boundaries is measured.
Graph 2 Dependence of time on number of measurements rectangle
5. TESTING One person from each age group (15-35, 36-55, 56-75 age years) was selected for the best comparison. Testing was
548
Klara Fiedorova et al. / IFAC PapersOnLine 52-27 (2019) 545–549
Graph 3 Dependence of time on number of measurements – ring In the 15-35 age category, the subject was left-handed. The null hypothesis cannot be rejected from the results. As the number of measurements increases, the task time decreases. Minor fluctuation occurs only at the rectangle. The dominant hand can be determined from the results. It can be seen that the non-dominant hands were in most cases the task times lower than the dominant hands. Most of the population has a dominant left-brain hemisphere, and these people usually have a dominant right hand. However, the left hemisphere may not be right, but also left as the right-handed. The test person plotted squares on paper after each measurement. To compare the results, only the first and last image are inserted, see Figure 5. We do not reject the null hypothesis for the 36-55 group. The deviation is again apparent at the rectangle. The test subject had the right hand as the dominant hand. From the graphs it is possible to evaluate that by non-dominant hands some times of task fulfilment were lower, it is most visible in the square. Figure 5 shows drawings of the figures. The zero hypothesis is also confirmed from the results for the third category of 56-75 years. However, the decrease in filling time is more moderate than in previous categories. The largest fluctuations are in this case at the square. The measured person was right-handed, the dominant hand was the fastest at the circle. Figure 5 shows drawings of figures.
Fig. 5. Drawing after the first measurement (top) and after the last measurement (bottom). From the left 15-35, 36-55, 56-75 age years 7. DISCUSSION For the 15-35 age category, the progress of improvement was the fastest and the final drawing of the squares was also the most accurate. The second category also had very good
progress, but the final times were not as fast as the first category. But the squares were also very accurate. For the third age category, the performance of tasks was the worst because of the improvement, accuracy and time of performance. The error rates for each measurement were measured for all categories. In the first measurements the errors occurred in all categories, in the next measurements the errors occurred only sporadically. In the overall evaluation of all categories we can say that all fulfilled the assumption that with increasing number of measurements the time of individual tasks decreases. Also, the number of errors was reduced during testing, due to improved motion accuracy and coordination. This exercise is suitable for supplementing rehabilitation exercises in patients with plegiies, after stroke and multiple sclerosis. The angles of the range of motion of the tested movements are also measured objectively, ie palmar and dorsal flexion, supination and pronation. Using the measured angles is individually set from which angle the mouse cursor will move in the exercise game, so it is possible to use this prototype even for people with reduced range of movements in the wrist and forearm. 8. CONCLUSION This work deals with sensing the movement of upper limbs using biofeedback in rehabilitation. Subsequently, creating a prototype for measuring movements of the upper limbs for the rehabilitation of basic movements of the hands and wrists. The task was to create a measurement string for biofeedback and also to create a visualization software to display the results. The first step was the selection of the sensor, which was based on the search, and the selection of a mini-computer for data processing. To acquire and visualize data from the sensor, a program was created to obtain data from the sensor, recalculate it into angles and also calibrate it. Then display the measured angles in real time. The program allows controlled movement of the mouse cursor by hand movement. For the purpose of rehabilitation, an exercise game was created in Matlab to improve the coordination and accuracy of hand movements. Three basic geometric shapes, square, rectangle and circle, are used in the game. Testing was carried out for two weeks. Both hands were tested to improve coordination and movement accuracy. The task was twice to best circulate all shapes gradually with both hands. Improvements in both accuracy and timeliness of tasks were confirmed for all tested age categories. Measurements were followed by the number of errors in individual measurements, which occurred only in the first measurements. This exercise would be useful as a supplement to rehabilitation exercises for different patients. In patients with reduced range of motion, the degree of improvement could be objectively assessed by measuring angles of range in the wrist and forearm. From the results of the measurements it can be said that the proposed system contributes to the improvement of movements, their coordination and accuracy and can contribute to a faster return to normal. The work was therefore successfully completed and properly tested.
Klara Fiedorova et al. / IFAC PapersOnLine 52-27 (2019) 545–549
ACKNOWLEDGMENT The work and the contributions were supported by the project SV450994/2101Biomedical Engineering Systems XV'. This study was also supported by the research project The Czech Science Foundation (GACR) 2017 No. 17-03037S Investment evaluation of medical device development run at the Faculty of Informatics and Management, University of Hradec Kralove, Czech Republic. This study was supported by the research project The Czech Science Foundation (TACR) ETA No. TL01000302 Medical Devices development as an effective investment for public and private entities. REFERENCES Ar Rahman, Y., M. M. Hoque, K. I. Zinnah and I. M. Borhary (2014). Helping-Hand: A Data Glove Technology for Rehabilitation of Monoplegia Patients. In: The 9th International Forum on Strategic Technology. 2014, 6. Daponte, P., L. DE Vito and C. Sementa (2013). A Wirelessbased Home Rehabilitation System for Monitoring 3D Movements. In: Medical Measurements and Applications Proceedings. 2013, 6. Giggins, O. M, U. Persson and B. Caulfield (2013). Biofeedback in rehabilitation. Journal of NeuroEngineering and Rehabilitation[online]. 2013, 10(1), 60- [cit. 2016-11-28]. DOI: 10.1186/1743-000310-60. ISSN 1743-0003. Peruzzini, M., M. Iualé and M. Germani (2014). A VP-based application to improve usability of an upper-limb rehabilitation orthosis. In: Mechatronic and Embedded Systems and Applications. 2014, 6. Sarcevic, P., Z. Kincses, S. Pletl and L. Schaffer (2015). Distributed movement recognition algorithm based on wrist-mounted wireless sensor motes. In: European Wireless. 2015, 6. Szelitzky, E., A. M. Alutei, B. Chetran and D. Mandru (2011). Data Glove and Virtual Environment: Distance Monitoring and Rehabilitation Solution. In: The 3rd International Conference on E-Health and Bioengineering. 2011, 4. Wu, Y., K. Chen and Ch. Fu (2016). Natural Gesture Modeling and Recognition Approach Based on Joint Movements and Arm Orientations. In: IEEE Sensors Journal. 2016, 9.
549
ScienceDirect
Analysis of upper limb movement for biofeedback in rehabilitation IFAC PapersOnLine 52-27 (2019) 545–549 Analysis limb Analysis of of upper upper limb movement movement for for biofeedback biofeedback in in rehabilitation rehabilitation Klara Fiedorova, Veronika Baladova, Lukas Peter *** Analysis of upper limb movement for biofeedback in rehabilitation Klara Fiedorova, Veronika Baladova, Lukas Klara Fiedorova, Veronikafor Baladova, Lukas Peter Peter *** Analysis of upper limb movement biofeedback in*** rehabilitation
Listopadu 15 Ostrava-Poruba, Czech Republic;(e-mail: VSB-Technical University ofKlara Ostrava, FEECS, Veronika K450, 17. Fiedorova, Baladova, Lukas Peter *** [email protected], [email protected], [email protected]). Klara Fiedorova, Veronika Lukas Peter *** Czech Republic;(e-mail: Baladova, VSB-Technical University of Ostrava, FEECS, K450, 17. Listopadu 15 Ostrava-Poruba, VSB-Technical University of Ostrava, FEECS, K450, 17. Listopadu 15 Ostrava-Poruba, Czech Republic;(e-mail: [email protected], [email protected], [email protected]). [email protected]). [email protected], [email protected], VSB-Technical University of Ostrava, FEECS, K450, 17. Listopadu 15 Ostrava-Poruba, Czech Republic;(e-mail: VSB-Technical University of Ostrava, FEECS, K450, 17. Listopadu 15 Ostrava-Poruba, Czech Republic;(e-mail: [email protected], [email protected], [email protected]). [email protected], [email protected], Abstract: The work deals with sensing the movement of upper [email protected]). using biofeedback and subsequent use in rehabilitation. is important to get the problems of the motor system, physiology Abstract: The work work It deals with sensing sensing theacquainted movementwith of upper upper limbs using using biofeedback and subsequent subsequent Abstract: The deals with the movement of limbs biofeedback and and pathology of the upper limbs and methods of their rehabilitation. Furthermore, types and applications use in in rehabilitation. rehabilitation. It It is is important to to get get acquainted acquainted with with the the problems problems of of the the motor motor system, system, physiology physiology use Abstract: The work dealsimportant with sensing the movement ofthe upper limbs using biofeedback anddescribes subsequent of biofeedback inthe rehabilitation are discussed. A part ofrehabilitation. thesis is also a research, which the and pathology of upper limbs and methods of their Furthermore, types and applications and pathology of the upper limbs and methods of their rehabilitation. Furthermore, types and applications Abstract: The work deals with sensing the movementwith ofofupper limbs using biofeedback andused. subsequent use ininrehabilitation. It is important to get acquainted the problems of the motor system, physiology state the area of sensing and analysis of movements upper limbs and types of sensors In the of biofeedback in rehabilitation are discussed. A part of the thesis is also research, which describes the of inwork rehabilitation are discussed. part thethe thesis isprocessing also research, which the use in rehabilitation. It is important tomethods get acquainted with problems ofaa the motor system, physiology andbiofeedback pathology of the upper limbs and ofmeasuring, theirof rehabilitation. Furthermore, types anddescribes next part ofarea the were described ways ofA signal and building aapplications measuring state in the of sensing and analysis of movements of upper limbs and types of sensors used. In the state in the area of sensing and analysis of movements of upper limbs and types of sensors used. In the and pathology of the upper limbs and methods of their rehabilitation. Furthermore, types and applications of biofeedback in rehabilitation discussed.ofAthe partforearm of the thesis is alsowhich a research, which describes the chain for of scanning the were rangedescribed ofare movements and wrist, is connected a PC and next part the work ways of measuring, signal and building aato measuring next part ofarea theinof work were described ways ofAinterface measuring, signal processing and building measuring of biofeedback rehabilitation are discussed. part of of the thesislimbs isprocessing also aform research, which describes state in the sensing and analysis ofuser movements upper and types of sensors used. In the provides information for rehabilitation. A is created in the of a rehabilitation game, chain scanning the range of movements of forearm and wrist, which is connected to aa PC chain for scanning the were range of movements of the the forearm and limbs wrist, which is results connected PC and state infor theof area of sensing analysis of movements of upper and types ofbuilding sensors used. In and the next part the work described ways of measuring, signal processing measuring using visual biofeedback toand improve the course of rehabilitation. From theand and ato analysis of provides information for rehabilitation. A user interface is created in the form of a rehabilitation game, provides information for rehabilitation. A user interface is created in the form of a rehabilitation game, next part of the work were described ways of measuring, signal processing and building a measuring chain for scanning the range of movements the forearm and wrist, which is connected to a PC and measurements it can be said that the proposed system contributes to the improvement of movements, using visual biofeedback to of improve the course course of rehabilitation. From theis results results and and toanalysis analysis of using visual biofeedback to improve the rehabilitation. the of chain for scanning thefor range movements of interface the of forearm and wrist, which a PC and provides information rehabilitation. A user is created in From the form ofconnected a rehabilitation game, their coordination and accuracy. measurements it can be said that the proposed system contributes to the improvement of movements, measurements it can be said that the proposed system contributes to the improvement of movements, provides information for rehabilitation. A user interface is created in From the form a rehabilitation game, using visual biofeedback to improve the course of rehabilitation. the ofresults and analysis of their coordination and accuracy. © 2019, IFAC (International Federation Automatic Control) Hosting Biofeedback, by Elsevier All rights reserved. their coordination andbe accuracy. Keywords: Accelerometer, Gyroscope, Upper limb motion sensing, Rehabilitation. using visual biofeedback to improve the course of rehabilitation. theLtd. results and analysis of measurements it can said that the of proposed system contributes toFrom the improvement of movements, measurements it can said that the proposed system contributes to the improvement of movements, their coordination andbe accuracy. Keywords: Accelerometer, Gyroscope, Keywords: Accelerometer, Gyroscope, Upper Upper limb limb motion motion sensing, sensing, Biofeedback, Biofeedback, Rehabilitation. Rehabilitation. their coordination and accuracy. the middle brain. The cerebellum is important for motor Keywords: Gyroscope, Upper limb motion sensing, Biofeedback, Rehabilitation. 1. Accelerometer, INTRODUCTION control, muscle tone and voluntary movements. The motor basal Keywords: Accelerometer, Gyroscope, Upper limb motion sensing, Biofeedback, Rehabilitation. the middle brain. The cerebellum is important the middle brain. The cerebellum is controlling important for for motor ganglia are the four nuclei in the brain voluntary 1.onINTRODUCTION INTRODUCTION The work focuses 1. the development of rehabilitation control, muscle tone and voluntary movements. The basal control, muscle tone voluntary The basal the brain. Theandcerebellum ismovements. important motor and middle involuntary movements. The last centre is thefor cerebral techniques for the upper limbs ensuring complex movement. ganglia are the four nuclei in the brain controlling voluntary 1. INTRODUCTION The work focuses on the development of rehabilitation ganglia are the four nuclei in the brain controlling voluntary the middle brain. The ismovements. important for motor The work etfocuses on the of rehabilitation control, muscle tone andcerebellum voluntaryon The basal cortex, where the final decision movement iscerebral made. (Szelitzky al., 2011) Thedevelopment control of human motor is and 1. INTRODUCTION involuntary movements. The last centre is techniques for the upper upper limbs ensuring complex movement. and involuntary movements. The last centre is the theThe cerebral control, muscle tone and voluntary movements. basal techniques for the limbs ensuring complex movement. ganglia are the four nuclei in the brain controlling voluntary (Szelitzky et al., 2011), (Wu et al., 2016) The work focuses onthethe development of rehabilitation accomplished through central nervous system. (Szelitzky cortex, where the final on movement is made. (Szelitzky et al., 2011) The control of human motor is cortex, where the final decision on centre movement iscerebral made. ganglia are the four nucleidecision in The the brain controlling voluntary (Szelitzky et al., 2011) The control of human motor is and involuntary movements. last is the The work focuses on the development of rehabilitation techniques for the upper limbs ensuring complex movement. et al., 2011) Skeletal muscle contraction and relaxation is the (Szelitzky et 2011), (Wu et al., 2016) accomplished through the central nervous system. (Szelitzky (Szelitzky et al., al.,the 2011), etThe al., last 2016) involuntary movements. is theiscerebral accomplished thelimbs central nervous system. (Szelitzky cortex, where final(Wu decision on centre movement made. techniques for themovement. upper ensuring complex movement. (Szelitzky et through al., 2011) The control of human motor is and basis for active The contraction is controlled 1.2 Pathophysiology et al., 2011) Skeletal muscle contraction and relaxation is the cortex, where the final decision on movement is made. et al., 2011) Skeletal muscle contraction and relaxation is the (Szelitzky et al., 2011), (Wu et al., 2016) (Szelitzky et al., 2011) The control of human motor is accomplished through the central nervous system. (Szelitzky directly the movement. CNS. All The reactions take place at the basis for by active contraction is controlled (Szelitzky et al., 2011), (Wu et al., 2016) basis active movement. Thenervous contraction is (Szelitzky controlled accomplished through the by central system. 1.2 Pathophysiology Pathophysiology et al., for 2011) Skeletal muscle contraction and relaxation is the 1.2 neuromuscular plate level chemical reactions. Pathophysiology of the upper limbs has many causes. These directly by the CNS. All reactions take place at the directly by the CNS. All reactions take place at the et al., for 2011) Skeletal muscle contraction and relaxation is basis active movement. The contraction is controlled 1.2 Pathophysiology are congenital or of acquired disorders occurring directly on the neuromuscular plate level by reactions. limbs causes. neuromuscular plate levelsystem by chemical chemical reactions. The activity ofthe the motor is divided into place voluntary basis for by active movement. The contraction is controlled Pathophysiology of the the upper upper limbs has hasormany many causes. These These directly CNS. All reactions take at and the Pathophysiology 1.2 Pathophysiology limb; they are muscular dystrophies inflammations. Or congenital or acquired disorders occurring directly on the involuntary. (Wu et al., system 2016) The involuntary movements directly by of the CNS. All reactions take place at and the are are congenital or acquired disorders occurring directly on the neuromuscular plate level by chemical reactions. The activity the motor is divided into voluntary Pathophysiology of the upper limbs has many causes. These arise in centres and lanes of motor control. There are various The activity of the motor system is divided into voluntary and they are dystrophies or inflammations. Or involve the control of the position and movements the muscle limb; neuromuscular plate byupright chemical reactions. limb; they areormuscular muscular dystrophies ormany inflammations. Or Pathophysiology of the plegia, upper limbs has causes. These involuntary. (Wu et level al., 2016) The involuntary are congenital acquired disorders directly on the degrees of paresis and it iscontrol. aoccurring disorder of are the various central involuntary. (Wu al., is 2016) The involuntary movements arise in centres and lanes of motor There The activity of the et motor system is divided into voluntary and tone. This information found in the brainstem and is arise in centres and lanes of motor control. There are various are congenital acquired disorders occurring directly on the involve the control of the upright position and the limb; they areor muscular or inflammations. Or motor neuron, the cause is dystrophies stroke, trauma, tumors or multiple involve thebyof control of upright position and the muscle muscle The activity the cerebellum, motor system is divided into voluntary and degrees of paresis and plegia, it aa disorder of the central involuntary. (Wu et al.,the 2016) The involuntary movements controlled the basal ganglia and subordinate degrees ofAnother paresis and plegia, it is iscontrol. disorder of are the various central limb; they are muscular dystrophies or inflammations. Or tone. This information is found in the brainstem and is arise in centres and lanes of motor There sclerosis. group are disorders of the extrapyramidal tone. This is2016) found the brainstem andthe is motor neuron, the cause is stroke, trauma, tumors or multiple involuntary. (Wu et al.,the Theinposition involuntary involve the information control of upright and movements thefrom muscle to the cortex. Free movement isbasal induced by impulses motor the cause isofstroke, trauma, tumors or multiple arise inneuron, centres and lanes motor control. There are various controlled by the cerebellum, ganglia and subordinate degrees of paresis and plegia, it is a disorder of the central system, which are disorders of the basal ganglia and controlled by the cerebellum, basal ganglia and subordinate involve the control of ganglia the the muscle group are disorders of extrapyramidal tone. This information is upright found the brainstem and is sclerosis. cerebral cortex, basal andinposition cerebellum. (Wu et al., sclerosis. Another group disorders of the the extrapyramidal degrees ofAnother paresis and plegia, it istrauma, a disorder oforor themultiple central to the Free movement is induced by from motor neuron, the cause isare stroke, tumors cerebellum causing Parkinson's disease various to the cortex. cortex. Free movement isbasal induced by impulses impulses from the tone. This by information is found inganglia the brainstem andthe is system, which are disorders of the basal ganglia and controlled the cerebellum, and subordinate 2016) system, which are disorders of the basal ganglia and motor neuron, the cause is stroke, trauma, tumors or multiple cerebral cortex, basal ganglia and cerebellum. (Wu et al., sclerosis. Another group syndromes are disorderssuch of theasextrapyramidal involuntary movement Huntington's cerebral cortex, gangliaisbasal and cerebellum. et al., controlled by Free thebasal cerebellum, ganglia and (Wu subordinate causing Parkinson's disease or various to the cortex. movement induced by impulses from the cerebellum cerebellum causing Parkinson's disease or various sclerosis. Another group are disorders of the extrapyramidal 2016) system, which are disorders of the basal ganglia and disease. (Szelitzky et al., 2011) 2016) The muscle is controlled by cerebellum. motoneuron. Thefrom central to theskeletal cortex. Freebasal movement by impulses the involuntary movement syndromes such as cerebral cortex, gangliais induced and (Wu et al., involuntary movement syndromes such as Huntington's Huntington's system, which are disorders of the basal ganglia and cerebellum causing Parkinson's disease or various motoneuron is part of the descending motor (Wu track. cerebral cortex, basal ganglia and cerebellum. et The al., disease. (Szelitzky et 2011) 2016) The skeletal muscle is controlled by motoneuron. The central disease. (Szelitzky et al., al.,Parkinson's 2011) cerebellum causing disease or various The skeletal muscle is controlled by motoneuron. The central involuntary movement syndromes such as Huntington's peripheral motoneuron is then part of the anterior spinal horn. 2016) Rehabilitation of upper limbs - biofeedback motoneuron is part the descending motor track. The involuntary movement syndromes such as Huntington's motoneuron is whose partisof of the forms descending motor track. The 1.3 disease. (Szelitzky et al., 2011) The skeletal muscle controlled byamotoneuron. The central It is a neuron axon motor nerve that goes peripheral motoneuron is then part of the anterior spinal horn. 1.3 Rehabilitation of upper limbs biofeedback disease. (Szelitzky et al., 2011) peripheral motoneuron is then part of the anterior spinal horn. The skeletal muscle is controlled by motoneuron. The central 1.3 Rehabilitation of upper limbs biofeedback motoneuron is part of the descending motor track. The directly to the muscle. (Szelitzky et al., 2011)nerve that goes In the rehabilitation of the upper limbs, the complexity of It a motor It is is aa neuron neuron whose axon forms motor nerve that horn. goes motoneuron is whose part ofaxon the forms descending motor track. The 1.3 Rehabilitation of upper limbs - biofeedback peripheral motoneuron is then part ofa the anterior spinal movement of all jointsofand the fine limbs, motor the skillscomplexity of the hands directly to the (Szelitzky al., 2011) directly to motoneuron the muscle. muscle. etof al., 2011) the rehabilitation the upper of peripheral is thenforms partet anterior spinal In the rehabilitation of the limbs upper limbs, the complexity of It is a neuron whose (Szelitzky axon a the motor nerve that horn. goes In 1.3 Rehabilitation of upper biofeedback are important. Rehabilitation is done in several ways, the 1.1 Momentum control movement of all joints and the fine motor skills of the hands It is a neuron whose (Szelitzky axon forms a motor of all joints and the fine motor skills of the hands directly to the muscle. et al., 2011)nerve that goes movement In the step rehabilitation ofexercise the upper limbs, the complexity of basic is medical or kinesiotherapy, but also are important. Rehabilitation is done in several ways, the 1.1 control directly to the muscle. are important. Rehabilitation isfine done inetthe several ways, the 1.1 Momentum Momentum control(Szelitzky et al., 2011) In the rehabilitation of the upper limbs, complexity of movement of all joints and the motor skills of the hands physical therapy or ergotherapy. (Giggins al., 2013) Movement and movement control is performed in the spinal basic step is medical exercise or kinesiotherapy, but also basic step is medical exercise or kinesiotherapy, but also movement of all joints and the fine motor skills of the hands are important. Rehabilitation is done in several ways, the 1.1 control cord,Momentum brainstem, cerebellum, basal ganglia and cerebral physical therapy or ergotherapy. et al., 2013) Movement and movement control is in the physical therapy orbiological ergotherapy. (Giggins etseveral al., to 2013) Biofeedback systems isinused present are important. Rehabilitation is(Giggins done ways, thea Movement and movement control is performed performed in the spinal spinal 1.1 Momentum control basic step is of medical exercise or kinesiotherapy, but also cortex. The spinal cord is the lowest motor centre. It is cord, brainstem, cerebellum, basal ganglia and cerebral measured quantity or other sensations to a patient and is basic step is medical exercise or kinesiotherapy, but also cord, brainstem, cerebellum, basal ganglia and cerebral physical therapy or ergotherapy. (Giggins et al., 2013) of systems is to Movement andhigher movement isand performed ininformation the spinal Biofeedback controlled by brainiscontrol systems receives Biofeedback of biological biological systems is used used to present present aa cortex. The spinal cord the lowest motor centre. It is expected to respond. (Giggins et al., 2013) For rehabilitation, physical therapy or ergotherapy. (Giggins et al., 2013) cortex. The spinal cord is the lowest motor centre. It is Movement and movement control is performed in the spinal quantity or sensations to aa patient and is cord, brainstem, cerebellum, basal ganglia and cerebral from the skin, muscles andsystems tendons. Another centre is the measured measured quantity or other other sensations andinto isa controlled by higher brain and receives information Biofeedback biological systems is to used patient to present biofeedback isofdivided according to 2013) measured quantity controlled higher brain andganglia receives information cord, brainstem, cerebellum, and cerebral to respond. (Giggins et al., For rehabilitation, cortex.stem Theby spinal cord issystems thebasal lowest motor centre. Itand is expected brain dividing into the spinal cord, the Varol bridge expected to respond. (Giggins et al., 2013) For rehabilitation, Biofeedback of biological is to used to present from skin, muscles and tendons. Another is measured quantity or other systems sensations a patient and isa from the theThe skin, muscles and tendons. centre isItthe the cortex. cord issystems the lowest motor centre centre. is biofeedback is according to measured quantity controlled byspinal higher brain andAnother receives information biofeedback is divided divided according to 2013) measured quantity into measured quantity or other sensations to a patient andinto is brain stem dividing into the spinal cord, the Varol bridge and expected to respond. (Giggins et al., For rehabilitation, brain stem dividing into the spinal cord, the Varol bridge controlled by higher brain andAnother receives information from the skin, muscles andsystems tendons. centre is and the expected to respond. (Giggins et al., 2013) For rehabilitation, biofeedback is divided according to measured quantity into from the skin, muscles andspinal tendons. centre is and the brain stem dividing into the cord,Another the Varol bridge biofeedback is divided according to measured quantity into brain stem into the spinal Federation cord, the Varol bridgeControl) and Hosting by Elsevier Ltd. All rights reserved. 2405-8963 © dividing 2019, IFAC (International of Automatic Peer review under responsibility of International Federation of Automatic Control. 10.1016/j.ifacol.2019.12.720
546
Klara Fiedorova et al. / IFAC PapersOnLine 52-27 (2019) 545–549
physiological (electrical activity of the brain, muscles or heart) and biomechanical quantities (movement, posture control, strength).
monitor patients under certain conditions. This app is based on a small wireless device with an accelerometer that is connected to the human wrist. (Peruzzini et al., 2014)
Among the methods using biofeedback, EMG measurements, specifically changes in the measured muscle, are useful. Biomechanical methods are also used, such as electrogoniometry, video camera scanning and sensor measurements. Using accelerometers and gyroscopes, realtime motion information is obtained. This makes it easy to obtain information about limb movement and to evaluate the extent of movement and the quality of rehabilitation. (Giggins et al., 2013)
3. MEASURING EQUIPMENT
2. RESEARCH OF THE STATE OF REHABILITATION TECHNOLOGY The research deals with systems for sensing and analysing movement of hands or body. There are many different systems dealing with the subject, most often focused on sensing whole body movement or 3D hand movement. (Wu et al., 2016) Often these systems have defined several movements they detected (sitting, body rotation, gestures). These systems serve as support for rehabilitation. (Wu et al., 2016) In one work, hand movements are captured that are displayed using a 3D hand model, giving the ability to save, retrieve and visualize data. The glove system prototype includes seven sensors. (Szelitzky et al., 2011) In the next article, the authors present a home rehabilitation system using a network of body sensors to measure the range of motion in rehabilitation. Patients will be able to perform exercises anywhere, only with wireless sensors deployed, sending measured data wirelessly to a module working with the Raspberry Pi minicomputer. (Daponte et al., 2013) One team presented a system that uses virtual prototypes. An exoskeleton was used to sense the muscle activity sending data via Bluetooth, and the PC application then displayed the movement using the avatar in real time. (Peruzzini et al., 2014) There is also a Helping hand option to assist with monoplegia therapy. It is a glove with LDR sensors, a computer application of racing game was designed for the system for higher efficiency of rehabilitation. (Ar Rahman et al., 2014) Sensors such as accelerometer for 3D acceleration with gyroscope and magnetometer in one sensor were often used for motion sensing. (Sarcevic et al., 2015) These sensors are utilized by the IRIS system, which includes nine degrees of freedom (9DOF) sensor boards. (Sarcevic et al., 2015) Somewhere only accelerometer, EMG sensors, LDR sensors or resistance change sensors were used. One system also used the Wiimote driver for gesture recognition. Data from accelerometers were evaluated from the controller and processed using the Hidden Markov Models method. (Wu et al., 2016) IEEE 802.15.4 protocols were used most often for data transfer from sensors (Daponte et al., 2013), (Sarcevic et al., 2015) Some systems used Bluetooth, Zigbee, Ethernet or cable transmission. (Daponte et al., 2013) Wireless biomedical sensor networks have also been created to
Based on the research, a module containing an accelerometer and gyroscope was chosen, which should be the most suitable for sensing the rotation of the wrist. In this work we decided to use a six-axis IMU sensor (MPU6050), offering three values from an accelerometer and three from a gyroscope. We decided to use Arduino Leonardo to access the sensor. Combining the values provides useful information about hand orientation and tilt. Values were obtained by calculating equation 1 and dividing by the Ludolf number to obtain degrees. (1) Where x, y and z is For further processing, a DMP (digital motion processor) element is used, combining values with an accelerometer and a gyroscope. The data is converted to rotation values around the axes. Rotations are called Yaw (rotation around the Z axis), Pitch (rotation around the Y axis), and Roll (rotation around the X axis) with parameters of quaternion and gravity. The accuracy of the sensor was also verified using a protractor and simultaneously measuring and displaying the values on a PC. The accuracy of the sensor was calculated from the measured data, which was set to ± 2%. 4. REHABILITATION OF THE WRIST AND FOREARM The realization of prototypes (Figure 1) for rehabilitation was based on four hand and forearm movements. These are palmar and dorsal flexion, supination and pronation (Figure 2). These movements are important for fine motor skills as well as for larger wrist movements. It is possible to measure the range of motion and use the movements to control the movement of the mouse cursor. With palmar flexion, the cursor moves to the left, with dorsal flexion to the right, with supination the cursor moves up and with pronation down.
Fig. 1. First prototype
Klara Fiedorova et al. / IFAC PapersOnLine 52-27 (2019) 545–549
547
determined for two weeks. Both hands were tested for each of the subjects to improve coordination and precision of movement, as well as to identify differences between the dominant and non-dominant hands. After each measurement, the test person's task was to draw a square on the paper with both hands at the same time to better evaluate the test results. The testing method is shown in Figure 4. Fig. 2. Hand movements. From left: palmar flexion, dorsal flexion, supination, pronation 4.1 Program part This program for data processing and analysis is inspired by Jeff Rowberg's program. The program uses I2C and interrupt communications to retrieve DMP (digital motion processor) data from the MPU 6050. This data is read from FIFO memory. A Mouse library has been added to control the movement of the mouse cursor, which allows Arduino Leonardo. 4.2 Exercise Game A user interface was created in the Matlab environment for a simple practice game. Three basic shapes (square, rectangle and circle) are used. These shapes were chosen for their simplicity. The task of the person performing the task is to keep the mouse cursor within the given shape, ie to make as few mistakes as possible and to circle the shape twice clockwise in the shortest possible time. The shapes tested are shown in Figure 3.
Fig. 4. Testing Method 5.1 Statistical data processing In the statistical analysis, the null hypothesis H_0 equals to improving the time to accomplish the task, while the alternative hypothesis H_A means that the test subject has not improved. Hypotheses are verified using graphs. To give you an idea of the charts for the age category 36-55, see graphs 1, 2, 3.
Fig. 3. Tested shapes
Graph 1 Dependence of time on number of measurements square
Moving the mouse cursor after starting the measurement starts the motion curve and starts measuring the time and error. When the cursor is inside a shape, the curve will turn blue. When it crosses the shape boundary, it turns red. The colour of the curve gives a clear visual biofeedback. The blue colour inside the shape shows that everything is OK, otherwise it will be red, which forces the person to return as soon as possible. Using this biofeedback, the test person's reactions are accelerated. When the test person crosses the edge of the object with the mouse cursor, the first error is counted and the time to return between object boundaries is measured.
Graph 2 Dependence of time on number of measurements rectangle
5. TESTING One person from each age group (15-35, 36-55, 56-75 age years) was selected for the best comparison. Testing was
548
Klara Fiedorova et al. / IFAC PapersOnLine 52-27 (2019) 545–549
Graph 3 Dependence of time on number of measurements – ring In the 15-35 age category, the subject was left-handed. The null hypothesis cannot be rejected from the results. As the number of measurements increases, the task time decreases. Minor fluctuation occurs only at the rectangle. The dominant hand can be determined from the results. It can be seen that the non-dominant hands were in most cases the task times lower than the dominant hands. Most of the population has a dominant left-brain hemisphere, and these people usually have a dominant right hand. However, the left hemisphere may not be right, but also left as the right-handed. The test person plotted squares on paper after each measurement. To compare the results, only the first and last image are inserted, see Figure 5. We do not reject the null hypothesis for the 36-55 group. The deviation is again apparent at the rectangle. The test subject had the right hand as the dominant hand. From the graphs it is possible to evaluate that by non-dominant hands some times of task fulfilment were lower, it is most visible in the square. Figure 5 shows drawings of the figures. The zero hypothesis is also confirmed from the results for the third category of 56-75 years. However, the decrease in filling time is more moderate than in previous categories. The largest fluctuations are in this case at the square. The measured person was right-handed, the dominant hand was the fastest at the circle. Figure 5 shows drawings of figures.
Fig. 5. Drawing after the first measurement (top) and after the last measurement (bottom). From the left 15-35, 36-55, 56-75 age years 7. DISCUSSION For the 15-35 age category, the progress of improvement was the fastest and the final drawing of the squares was also the most accurate. The second category also had very good
progress, but the final times were not as fast as the first category. But the squares were also very accurate. For the third age category, the performance of tasks was the worst because of the improvement, accuracy and time of performance. The error rates for each measurement were measured for all categories. In the first measurements the errors occurred in all categories, in the next measurements the errors occurred only sporadically. In the overall evaluation of all categories we can say that all fulfilled the assumption that with increasing number of measurements the time of individual tasks decreases. Also, the number of errors was reduced during testing, due to improved motion accuracy and coordination. This exercise is suitable for supplementing rehabilitation exercises in patients with plegiies, after stroke and multiple sclerosis. The angles of the range of motion of the tested movements are also measured objectively, ie palmar and dorsal flexion, supination and pronation. Using the measured angles is individually set from which angle the mouse cursor will move in the exercise game, so it is possible to use this prototype even for people with reduced range of movements in the wrist and forearm. 8. CONCLUSION This work deals with sensing the movement of upper limbs using biofeedback in rehabilitation. Subsequently, creating a prototype for measuring movements of the upper limbs for the rehabilitation of basic movements of the hands and wrists. The task was to create a measurement string for biofeedback and also to create a visualization software to display the results. The first step was the selection of the sensor, which was based on the search, and the selection of a mini-computer for data processing. To acquire and visualize data from the sensor, a program was created to obtain data from the sensor, recalculate it into angles and also calibrate it. Then display the measured angles in real time. The program allows controlled movement of the mouse cursor by hand movement. For the purpose of rehabilitation, an exercise game was created in Matlab to improve the coordination and accuracy of hand movements. Three basic geometric shapes, square, rectangle and circle, are used in the game. Testing was carried out for two weeks. Both hands were tested to improve coordination and movement accuracy. The task was twice to best circulate all shapes gradually with both hands. Improvements in both accuracy and timeliness of tasks were confirmed for all tested age categories. Measurements were followed by the number of errors in individual measurements, which occurred only in the first measurements. This exercise would be useful as a supplement to rehabilitation exercises for different patients. In patients with reduced range of motion, the degree of improvement could be objectively assessed by measuring angles of range in the wrist and forearm. From the results of the measurements it can be said that the proposed system contributes to the improvement of movements, their coordination and accuracy and can contribute to a faster return to normal. The work was therefore successfully completed and properly tested.
Klara Fiedorova et al. / IFAC PapersOnLine 52-27 (2019) 545–549
ACKNOWLEDGMENT The work and the contributions were supported by the project SV450994/2101Biomedical Engineering Systems XV'. This study was also supported by the research project The Czech Science Foundation (GACR) 2017 No. 17-03037S Investment evaluation of medical device development run at the Faculty of Informatics and Management, University of Hradec Kralove, Czech Republic. This study was supported by the research project The Czech Science Foundation (TACR) ETA No. TL01000302 Medical Devices development as an effective investment for public and private entities. REFERENCES Ar Rahman, Y., M. M. Hoque, K. I. Zinnah and I. M. Borhary (2014). Helping-Hand: A Data Glove Technology for Rehabilitation of Monoplegia Patients. In: The 9th International Forum on Strategic Technology. 2014, 6. Daponte, P., L. DE Vito and C. Sementa (2013). A Wirelessbased Home Rehabilitation System for Monitoring 3D Movements. In: Medical Measurements and Applications Proceedings. 2013, 6. Giggins, O. M, U. Persson and B. Caulfield (2013). Biofeedback in rehabilitation. Journal of NeuroEngineering and Rehabilitation[online]. 2013, 10(1), 60- [cit. 2016-11-28]. DOI: 10.1186/1743-000310-60. ISSN 1743-0003. Peruzzini, M., M. Iualé and M. Germani (2014). A VP-based application to improve usability of an upper-limb rehabilitation orthosis. In: Mechatronic and Embedded Systems and Applications. 2014, 6. Sarcevic, P., Z. Kincses, S. Pletl and L. Schaffer (2015). Distributed movement recognition algorithm based on wrist-mounted wireless sensor motes. In: European Wireless. 2015, 6. Szelitzky, E., A. M. Alutei, B. Chetran and D. Mandru (2011). Data Glove and Virtual Environment: Distance Monitoring and Rehabilitation Solution. In: The 3rd International Conference on E-Health and Bioengineering. 2011, 4. Wu, Y., K. Chen and Ch. Fu (2016). Natural Gesture Modeling and Recognition Approach Based on Joint Movements and Arm Orientations. In: IEEE Sensors Journal. 2016, 9.
549