- Email: [email protected]
A gait rehabilitation strategy inspired by an iterative learning algorithm
Mechatronics 22 (2012) 213–221
Contents lists available at SciVerse ScienceDirect
Mechatronics journal homepage: www.elsevier.com/locate/mechatronics
A gait rehabilitation strategy inspired by an iterative learning algorithm Joonbum Bae a,⇑, Masayoshi Tomizuka b a b
School of Mechanical and Advanced Materials Engineering, UNIST, Ulsan, Republic of Korea Department of Mechanical Engineering, University of California, Berkeley, CA 94720, USA
a r t i c l e
i n f o
Article history: Received 29 December 2010 Accepted 22 January 2012 Available online 23 February 2012 Keywords: Gait rehabilitation strategy Iterative learning algorithm Robotic gait rehabilitation device
a b s t r a c t Robotic gait rehabilitation devices enable efficient and convenient gait rehabilitation by mimicking the functions of physical therapists. In manual gait rehabilitation training, physical therapists have patients practice and memorize normal gait patterns by applying assistive torque to the patient’s joint once the patient’s gait deviates from the normal gait. Thus, one of the most important factors in robotic gait rehabilitation devices is to determine the assistive torque to the patient’s joint during rehabilitation training. In this paper, the gait rehabilitation strategy inspired by an iterative learning algorithm is proposed, which uses the repetitive characteristic of gait motions. In the proposed strategy, the assistive joint torque in the current stride is calculated based on the information from previous strides. Simulation results and experimental results using an active knee orthosis are presented, which verify that the proposed strategy can be used to calculate appropriate assistive joint torque to excise the desired motions for rehabilitation. Ó 2012 Elsevier Ltd. All rights reserved.
1. Introduction As the number of people who have either totally or partially lost the ability to walk due to aging or physical impairment increases [37,30,35,23], the demand for robotic gait rehabilitation devices increases. Robotic gait rehabilitation devices enable more efficient and convenient gait rehabilitation by mimicking the functions of physical therapists. In manual gait rehabilitation training, physical therapists establish suitable rehabilitation strategies depending on the gait disorders patients suffer from. While there is a long list of gait disorders that inhibits patients’ ability to walk, the causes of the gait disorders can be classified as degeneration in two categories: (1) in muscular systems and (2) in nerve systems. If a patient has weakened muscles due to accidents, diseases, or aging, then the patient cannot generate the joint torque necessary to achieve desired gait motion. For these patients, physical therapists try to strengthen patients’ muscles by applying appropriate resistive force to muscles. Weight training for specific muscles is one example of the many possible rehabilitation strategies. This gait rehabilitation strategy was mimicked by the robotic gait rehabilitation device for strengthening muscles [9]. If the muscles are too damaged to recover their original function, then power augmentation systems ([16,41,40] among others) may help the patients achieve normal walking movements.
⇑ Corresponding author. E-mail addresses: [email protected] (J. Bae), [email protected] (M. Tomizuka). 0957-4158/$ - see front matter Ó 2012 Elsevier Ltd. All rights reserved. doi:10.1016/j.mechatronics.2012.01.009
Patients with impaired motor control via spinal cord injury (SCI) or stroke has the problem that they cannot control their muscles properly. Assuming that only their nervous system is degenerated, i.e. their muscles are strong enough to generate muscular forces to achieve the desired gait motions, they need to practice and memorize the desired gait patterns through repetitive exercises. For these patients, physical therapists make patients practice normal gait patterns through force or voice feedback. Physical therapists apply assistive torques to the joints to let them be cognizant of how their gait motions deviate from the normal gait, and guide them to the normal gait trajectory. Also, physical therapists keep talking to patients during the gait rehabilitation training about how they need to move to achieve the normal gait patterns, e.g. bending knee more or pushing heel more. By mimicking these functions of physical therapists, robotic gait rehabilitation systems that utilize visual feedback [1,25] or assistive torque have been developed [12,7,13,32,38,5,4,18]. Since assistive torque allow patients to practice normal gait pattern easily and effectively, determining required assistive torque in a rehabilitation application is one of the most important factors in the robotic gait rehabilitation system. In fact, the assistive torque should be proportional to the amount of deviation from a normal gait trajectory in order for patients to perceive the right ‘‘feeling’’ about a normal gait trajectory. Through assistive torque, the joint is guided accurately to the desired gait trajectory for rehabilitation. In the past, rehabilitation strategies that impose a virtual potential field around the desired trajectory was introduced [3]. In this strategy, the induced force by the potential field compels the joint to move to the desired joint trajectory once the joint position
214
J. Bae, M. Tomizuka / Mechatronics 22 (2012) 213–221
deviates from the desired joint position. The potential field rehabilitation strategy provides a good method to calculate the required torque for patients to practice normal gait trajectory. However, in order to guide a joint to the correct gait trajectory, a high control gain is required, which may cause instability and overshoot. Inspired by the repetitive characteristic of gait motions, a gait rehabilitation strategy based on an iterative learning algorithm is proposed in this paper. Taking advantage of the repetitive characteristic of gait motions, the joint can be guided to the desired trajectory more effectively and accurately. The proposed rehabilitation strategy utilizes position errors and error derivatives of previous strides to calculate the assistive torque in the current stride. By using information from previous strides, repetitive abnormal gait patterns can be penalized more effectively. The performance of the proposed gait rehabilitation strategy was verified through simulations and experiments with an active knee orthosis. This paper is organized as follows. In Section 2, the role of assistive torque in robotic gait rehabilitation devices is discussed through explanations of how the gait rehabilitation training takes place. The gait rehabilitation strategy inspired by an iterative learning algorithm and its simulation results are presented in Section 3 while experimental results by the proposed algorithm are shown in Section 4. Finally, conclusions and future work are given in Section 5.
2. Assistive torque in robotic gait rehabilitation devices 2.1. Motion control in a human body Many models have been proposed to represent the motion control system in the human body. Among the models, the closed-loop control system in Fig. 1 is one of the most dominant [33]. The ‘Desired Human Motion’ in the figure is created by either an external stimulus or an internally generated intention. In the figure, the ‘Brain’ includes many parts such as the motor cortex, cerebellum, basal ganglia, and spinal cord all cooperate with one another to act as a controller for the motor control. The muscle control signals (r in Fig. 1) generated by ‘Brain’ are transferred to ‘Muscle’ though motor neurons, and ‘Muscle’ moves ‘Body’ via appropriate muscu-
Desired Human Motion
-
Brain
Body
Muscle
Human Motion
Sensory Organs Fig. 1. Closed-loop motion control system in a human body.
Rehabilitation Patient
Rehabilitation Patient
Physical Therapist
Physical Therapist
Voice Feedback
Force Feedback
Fig. 2. Voice/force feedback in gait rehabilitation training.
lar forces (s in Fig. 1). ‘Muscle’ is considered as an actuator, and ‘Body’ represents the musculoskeletal part of a human body, which is considered as a plant to be controlled. Human motions, which are sensed by various ‘Sensory Organs’ such as the eyes, muscle spindles and the Golgi tendon organs, are then compared with the desired motions to achieve precise motions. In this paper, the closed-loop motion control system in Fig. 1 is used to analyze human motions and to design rehabilitation strategies. ‘Desired Human Motion’ in Fig. 1 is assumed to be given by the motion planning part of the brain. 2.2. Assistive torque in robotic gait rehabilitation devices The goal of gait rehabilitation training is for patients that suffer from gait disorders to restore locomotion through rehabilitation of their degenerated muscular or neural functions. By applying assistive torque to the patient’s joint, a patient can either practice the desired gait trajectory for rehabilitation or strengthen muscles. If the patient’s problem is an inability to supply sufficient muscle strength to meet the demands of desired gait motions (insufficient s in Fig. 1), then the patient needs assistive torque in order to achieve the desired motion. In this case, the gait rehabilitation strategies that emphasize the recovery of muscular forces are required. Applying appropriate resistive torque to strengthen muscles is one of the possible rehabilitation strategies. Patients with impaired nerve systems, e.g. spinal cord injury (SCI) or stroke, cannot control their muscles due to abnormal motor control signals (abnormal r in Fig. 1). For these patients, physical therapists apply voice or force feedback to patients during gait rehabilitation training as shown in Fig. 2. Through force or voice feedback, physical therapists educate patients of normal joint trajectory and also physically guide patients to the desired joint trajectory for rehabilitation. In this paper, the gait rehabilitation strategy on how to determine assistive torque in robotic gait rehabilitation devices in order to practice a rehabilitative gait trajectory is discussed. 3. A gait rehabilitation strategy inspired by an iterative learning algorithm 3.1. A gait rehabilitation strategy inspired by a potential field The assistive torque in robotic gait rehabilitation devices provides a ‘‘feeling’’ to patients that allows them to sense how far their gait motions deviate from the desired gait trajectories. Through ‘‘feeling’’, the patients can learn the desired gait trajectory and be guided to the correct gait trajectory. Thus, the assistive torque should be proportional to the deviation from the desired gait trajectory and be capable of guiding the joint to the gait trajectory effectively and accurately. Since the amount of torque is related to the deviation from the normal trajectory, it can be considered as an impedance between human and the robotic gait rehabilitation device [17]. Previously, the gait rehabilitation strategy was formulated using a potential field [3]. In this strategy, a virtual potential field is placed around the desired trajectory, which generates induced forces for the joint to stay on the desired trajectory. The concept of the induced force by the potential field is shown in Fig. 3. The potential field, PA, for the gait rehabilitation strategy can be expressed as a function of the human joint angle, yH, and the desired joint angle for rehabilitation, yR, as follows:
PA ¼ aðyH yR Þb
ð1Þ
The values of a and b can be selected as any values considering the patients’ status or the aim of rehabilitation training, as long as PA has a global minimum at yR. The amount of the assistive torque is
215
J. Bae, M. Tomizuka / Mechatronics 22 (2012) 213–221
calculated as the negative gradient of the potential field with respect to the human joint angle: i.e.
sAd ¼ ryH PA
ð2Þ
PA ¼ ðyH yR Þ2 sAd ¼ 2ðyH yR Þ
ð3Þ ð4Þ
Potential Field (J)
One example of the proposed potential field is shown in Fig. 4. In this figure, the desired joint trajectory for rehabilitation is given as a simple sinusoidal wave (the thick line in the figure), and the potential field (gray lines in the figure) is given by a quadratic function of (3) with a = 1 and b = 2 in (1). If the human joint deviates from the desired angle (small circles in the figure), then assistive torque in (4) is generated by the potential field for the joint to be moved back to the desired trajectory.
10 8 6 4 2 0 2 1 10
0
Angle (rad)
5
-1 -2
3.2. A gait rehabilitation strategy inspired by an iterative learning algorithm The rehabilitation strategy by the potential field provides a good method to calculate the required torque needed for patients to practice the normal gait trajectory. If we rely on feedback control, a high control gain is required to guide a patient to stay close to the desired trajectory accurately, which may cause instability and overshoot. Since walking is a motion that repetitively uses two legs to move forward, gait motions are naturally repetitive over strides. Contrary to the normal gait pattern of healthy individuals, there is no ‘typical’ abnormal gait pattern for all abnormal gait motions. However, individuals with gait disorders exhibit their own distinctive gait motions over strides, and those motions can be even characterized according to their pathologies [34,28,24,14,19]. This paper proposes a gait rehabilitation strategy that uses iterative learning control (ILC) by noting the repetitive characteristics of gait motions. Fig. 5 shows normal and abnormal knee joint angles in three strides under the premise of the experiment in Section 4.1. A healthy subject mimicked a foot-dragging gait motion to produce the abnormal knee trajectory. Note that the deviation in angle from
0
Time (sec)
Fig. 4. Example of a potential field (thick line: desired joint trajectory, gray lines: potential field, small circles: human joint trajectory).
the normal knee angles in one stride is similar to those in other strides. In order to generate appropriate assistive torques in the repeated strides, an iterative learning algorithm, where the amount of assistive torque in the current stride is calculated based on the information from previous strides, is proposed in this paper. Many applications utilizing the iterative learning algorithm can be found in controls of mechanical systems tasks such as industrial robot [36], wafer stage systems [27], computer-numerical control tools [20], and injection molding systems [15]. ILC is based on the paradigm of learning as the name suggests. In a repetitive process, information from earlier iterations can be used to improve performance in the current iteration. ILC is anticipatory and can compensate for exogenous signals, such as repeating disturbances, in advance by learning from previous iterations [6]. Also, the design method requires minimal model information about the plant being controlled [26]. A widely used ILC control input is
ukþ1 ðjÞ ¼ Q ðqÞ½uk ðjÞ þ LðqÞek ðj þ 1Þ
ð5Þ
where u is the control input, j is the time index, k is the iteration index, q is the forward time-shift operator, qx(j) x(j + 1). Q(q) is a Qfilter for enhanced stability and robustness while L(q) is a learning function. A block diagram with a plant, P, and the control input (5) is drawn in Fig. 6. The errors and the control inputs in previous iterations are saved in memory and are then used to calculate the control input in the current iteration. By combining the iterative learning algorithm in (5) with the potential field algorithm in (2), the following gait rehabilitation strategy is proposed to calculate the assistive torque.
Current position
τ Ad Desired position
sA;kþ1 ðjÞ ¼ f ðekþ1 ðjÞÞ þ Q ðqÞ
sA;k ðjÞ þ aP ek ðj þ 1Þ þ aD
ek ðj þ 1Þ ek ðjÞ Ts
Potential Field (J)
ð6Þ 4 3 2 1 0 -2
-1.5
-1
-0.5
0
0.5
1
1.5
Deviated angle (rad) Fig. 3. Concept of a potential field for a rehabilitation strategy.
2
where ek(j) = yR(j) yH,k(j) and Ts is the sampling time. yR is the desired joint trajectory for rehabilitation and yH,k is measured human motion in the kth stride. The first term in (6), f(ek+1(j)), represents the assistive torque by the potential field algorithm defined in (2). It only depends on the error in the current stride (r in Fig. 7). The remaining terms in (6) are by the iterative learning algorithm, i.e. learned from the error and the error derivative in the previous stride (s in Fig. 7). The iterative learning algorithm in (6) uses PD-type update law to calculate the rehabilitation torque; P-type term uses the error signal, and D-type term uses the error derivative
216
J. Bae, M. Tomizuka / Mechatronics 22 (2012) 213–221
80
Angle (deg)
Normal
Abnormal
60 40 20 0
0
0.5
1
2
1.5
2.5
3
3.5
4
4.5
3
3.5
4
4.5
Time (sec)
40
Angle (deg)
30 20 10 0 -10 0.5
0
1.5
1
2
2.5
Time (sec)
Fig. 5. Normal/abnromal knee angles.
in the previous iteration. By utilizing errors and error derivatives in previous strides, repetitive abnormal gait motions can be penalized more effectively. If the potential field is selected as a simple quadratic function in (3), the proposed algorithm can be represented only by the ILC with a different learning function. The rehabilitation algorithm in (6) can be written as
sA;kþ1 ðjÞ ¼ K p ekþ1 ðjÞ þ Q ðqÞ½sA;k ðjÞ þ LðqÞek ðj þ 1Þ where Kp is from the potential field 1 LðqÞ ¼ aP þ aD 1þq . Eq. (7) can be rewritten as Ts
ð7Þ algorithm,
sA;kþ1 ðjÞ ¼ K p ekþ1 ðjÞ þ wkþ1 ðjÞ
and
ð8Þ
where
wkþ1 ðjÞ ¼ Q ðqÞ½sA;k ðjÞ þ LðqÞek ðj þ 1Þ
ð9Þ
Then,
wkþ1 ðjÞ ¼ Q ðqÞ½wk ðjÞ þ ðK p q1 þ LðqÞÞek ðj þ 1Þ
ð10Þ
1
which uses a new learning function as Kpq + L(q). The block diagram of the proposed rehabilitation strategy is depicted in Fig. 8. The assistive torque at the (k + 1)th stride, sA,k+1, is added to the human joint torque, sH. With assistive torque, the patient can easily be guided to reach to the desired gait trajectory for rehabilitation and memorize the motion through repetition. The joint angles are measured by encoders installed in the active knee orthosis introduced in Section 4.2. Angles of deviation from the desired gait motions and the assistive torque from previous iteration ILC Algorithm
yd
ek +1
Memory
Memory
ek
uk
L
Q
u k +1
Fig. 6. Block diagram of iterative learning control (ILC).
P
yk +1
are saved in order to compute the assistive torque of the current stride via the iterative learning algorithm. The ILC algorithm has been also applied to gait rehabilitation systems [11,10]. In their algorithms, the information of previous steps was used to calculate the assistive torque in current step, but consideration has not been paid to robustness or transient learning behavior of the system; Q-filtering was not utilized, i.e. Q = 1. It is well known that the learning gains influence the rate of convergence, whereas the Q-filter influences the converged error performance. Increasing the Q-filter bandwidth decreases robustness but improves performance, whereas decreasing the bandwidth has the opposite effect [6,8]. Thus, the cut-off frequency of the Q-filter is a tradeoff parameter between performance and robustness. In this paper, the cut-off frequency of the Q-filter is set to 10 Hz as shown Fig. 9 considering human walking motion range [39]. By limiting the cut-off frequency of the Q-filter, robustness of the system in the range of walking can be improved. Also, the learning filter of the learning algorithm in (6) corresponds to LðzÞ ¼ aP z þ aD z1 in the frequency domain. Since the D-type iteraTs tive learning update law uses the error derivative that is included in this learning algorithm, the Q-filter should be designed to remove high frequency components of the error derivative.
3.3. Forgetting factor and desired gait trajectory for rehabilitation Patients’ gait motions may improve during gait rehabilitation treatments by being adapted to the desired trajectory. The algorithm in (6) uses the position information from all previous iterations for the calculation of the current assistive torque because the assistive joint torque which includes the position information from previous strides is fed back to calculate the assistive torque in the current stride. Due to improvements in gait motions during the rehabilitation treatment, it may be inappropriate to use data from many strides ago in calculating the current assistive torque. By utilizing a forgetting factor, k, as in (11), the influence of old assistive torque record may be deemphasized. The forgetting factor
217
J. Bae, M. Tomizuka / Mechatronics 22 (2012) 213–221
80 60
( k + 1)th stride
kth stride 2
1
40 20 0 0
0.5
1
j
1.5
2.5
2
-5 -10 Normal Walking Range (~10Hz)
-15 -20 -25 1 10
3
j
( j+1)
Bode Diagram
0
Abnormal
Magnitude (dB)
Angle (deg)
Normal
10
2
3
10
Frequency (rad/sec)
Fig. 7. Calculation of the assistive torque by an iterative learning algorithm.
Fig. 9. Design of the Q-filter.
takes a value between 0 and 1, and it determines how quickly previous assistive torques are forgotten in the calculation of the current assistive torque.
goal. In the following simulations and experiments, k in (11) was set to 1, and a, b and c in (12) were set to 1, 1 and 0, respectively.
sA;kþ1 ðjÞ ¼ f ðekþ1 ðjÞÞ
3.4. Simulation study
ek ðj þ 1Þ ek ðjÞ þ Q ðqÞ k sA;k ðjÞ þ aP ek ðj þ 1Þ þ aD Ts
The proposed rehabilitation strategy in (6) was simulated with human models. The human body can be modeled with different mathematical complexities according to the number of joints and links [2,29]. Complicated models with many joints and links may give accurate analysis, but an excessive number of segments increase the complexity of the resulting equations because of the complex coupling among all the segments. In swing phases, the knee joint torque is primarily used for swinging the shank and the foot. A simplified model is the pendulum model in Fig. 10a. The shank and the foot are modeled as a uniform cylinder which has the mass of m1 and the length of l1. In stance phases, the knee joint torque is used for the upper body advancement. This can be modeled as an inverted pendulum; the foot is fixed on the ground, and the body’s upper parts above the knee is an inverted pendulum as shown in Fig. 10b. The whole upper body is also modeled as a uniform cylinder which has the mass of m2 and the length of l2. Thus, two widely-used simple models, the pendulum model and the inverted pendulum model, are used for swing phases and stance phases in this paper. To obtain the system equation in swing phases, the kinetic and potential energies of the system are computed as follows:
ð11Þ The final goal of gait rehabilitation may be said to restore the normal gait pattern, but the normal gait trajectory is not necessarily the best trajectory for every patient. Since the actual trajectory for rehabilitation depends on patient’s conditions, simply forcing the patients’ joints to follow the normal trajectory is not always effective in rehabilitation. Thus, in the proposed algorithm, the desired gait trajectory is determined based on the normal trajectory along with consideration of the patient’s conditions and the rehabilitation goal as expressed in (12).
yR ðtÞ ¼ aðtÞ yN
t þ cðtÞ bðtÞ
ð12Þ
where yN and yR represent the normal trajectory and the rehabilitation trajectory, respectively. The parameter a scales the amplitude of yN, b stretches the time and influences the period of the motion, and c changes the offset. The values of a, b and c in (12) determine the desired gait trajectory and k in (11) determines the effect of previous iterations’ position information for the calculation of the assistive torque in the current iteration. Variations of the values in the proposed algorithms allow sufficient degrees of freedom in the gait rehabilitation strategy. The actual values will be determined by physical therapists considering the patient’s condition and the rehabilitation
1 2 m1 l1 y_ 2 6 V ¼ m1 gl1c ð1 cos yÞ
T¼
yN
y H,k +1 -
ð13Þ ð14Þ
Encoder ILC Algorithm
ek +1 Potential Field Algorithm
Memory
ek
C wk +1
Q
τ A,k
τ A,k+1 Desired Human Motion
-
Brain
Muscle
L
τH
Memory Body
Sensory Organs Fig. 8. Block diagram of the proposed rehabilitation strategy.
Human Motion
218
J. Bae, M. Tomizuka / Mechatronics 22 (2012) 213–221
where L = T V. Likewise, the system equation in stance phases is,
1 3
s ¼ m2 l22 y€ m2 gl2c sin y
In the simulation, ‘Brain’ in Fig. 8 is modeled as a poorly tuned PID controller which generates abnormal muscle control signals so that ‘Muscle’ in Fig. 8 cannot generate the appropriate torque necessary to achieve the normal knee motions. The simulation results by the proposed algorithm in (6) are shown in Fig. 11. Note that the deviation from the normal angle in the first stride is larger than subsequent strides since learning is yet to take place in the first stride. In other words, only the potential field algorithm is used in the first stride. Once the iterative learning algorithm comes into full effect from the second stride, the proposed strategy generates appropriate assistive torque so that the deviation from the normal knee angle is decreased, as shown in Fig. 11b.
Fig. 10. Human models for the simulation.
80
Angle (deg)
Abnormal
Normal
Simulation
4.1. Knee motions in normal gait
40 20
0
1
0.5
2
1.5
2.5
3
3.5
4
4.5
3
3.5
4
4.5
Time (sec)
Angle (deg)
20 10 0 -10 -20
0
0.5
1.5
1
2.5
2
Time (sec)
Fig. 11. Simulation results.
Angle (deg)
4. Experimental results
60
0
Flexion
60 40
Since the proposed rehabilitation strategy was applied to the knee joint in this paper, the normal knee motions in sagittal plane is studied in this section. Also, the abnormal knee motions for the experiments are compared with the normal knee motions. The normal knee motions in one stride is shown in Fig. 12 with corresponding gait phases. The normal knee angle is obtained by averaging data from 30 strides of three healthy male subjects (10 strides per subject) without any known gait disorders (mean age: 29, mean height: 1.74 m, mean weight: 71 kg), and verified by literatures [31,39]. Informed consent forms approved by the Institutional Review Boards for the University of California at Berkeley were provided to all participants. Normal knee angles during walking are within the range of 0° to 70° as shown in Fig. 12. At initial contact phase, the knee is flexed about 5°. In loading response phase, putting body weight on the limb instantly disturbs the knee’s stability. The knee flexion, however, provides shock absorption in load response phase. Throughout the loading phase, the knee is rapidly flexed. In the rest of the stance, the knee extends and the ankle dorsiflexes in order to propel the body mass forward. In swing, the knee is flexed to lift the foot for limb advancement. This is the critical action that ensures foot clearance as the limb swings forward from the trailing posture.
20
4.2. Experimental setup
0
Extension
0
10
20
30
40
50
60
70
80
90
100
Gait Cycle (%)
Initial Loading Mid Contact Response Stance
Terminal Stance
PreSwing
Initial Swing
Mid Swing
Terminal Swing
Fig. 12. Normal knee angle in one stride and corresponding gait phases.
where T and V represent the kinetic energy and the potential energy, respectively. Then, the system equation is obtained by Lagrangian mechanics as follows:
d @L @ L dt @ y_ @y 1 2 € þ m1 gl1c sin y ¼ m1 l1 y 3
s¼
ð17Þ
ð15Þ ð16Þ
The proposed gait rehabilitation strategy was applied to the abnormal knee motions in Fig. 14. A compact rotary series elastic actuator (cRSEA) shown in Fig. 13a was used. The rotary series elastic mechanism was adapted to the cRSEA and it was designed and controlled with consideration of precise torque generation [21]. A worm gear set is used to make cRSEA compact and light. It operates in ideal force/torque mode by compensating for inherent mechanical impedance so that it precisely generates the desired torque. Thus, the computed rehabilitation torque, sA,k+1 in Fig. 8, is assumed to be generated accurately in this experiment. The human joint angle was measured by an encoder in the cRSEA. For more details, see [22]. The cRSEA was installed on the knee joint of an orthosis as shown in Fig. 13a. Slightly impaired knee motions by foot-dragging mimicked by a healthy subject (age: 30, height: 1.8 m, weight: 68 kg) was used in the experiments. The healthy subject was asked to wear the experimental orthosis as shown in Fig. 13b, and walk on the treadmill at a speed of 3.2 km/h with abnormal knee
219
J. Bae, M. Tomizuka / Mechatronics 22 (2012) 213–221
4.3. Experimental results
Fig. 13. The compact rotary series elastic actuator (cRSEA) installed on the knee joint of an orthosis.
The goal of the experiments was to verify that the proposed strategy generates adequate assistive torque so that a patient with an abnormal gait can practice and improve his/her gait to a desired gait trajectory during rehabilitation. As shown in the experimental results in Fig. 15c, appropriate assistive joint torques were generated by the proposed algorithm. Also, with the assistance of the generated assistive joint torque, the abnormal knee trajectory in Fig. 14 was altered to match the normal knee trajectory as shown in Fig. 15a and b. Even though a healthy person, not a patient, performed the abnormal gait motions, it was observed that under the proposed rehabilitation algorithm the intention of the normal person to mimic the abnormal walking motion did not result in the intended motion and was drive to the normal knee trajectory by the generated assistive torque. It took around 20 steps for the subject to adapt to the active knee orthosis and the desired knee trajectory, but after the adaption, the knee could follow the normal gait trajectory with aid of assistive torque.
5. Conclusion motions shown in Fig. 14a. The walking speed was set slower than normal walking speed since it is easier to practice the normal gait at slower speeds. The subject practiced the abnormal foot-dragging motion several times before the experiment to ensure that he mimicked the abnormal gait motions correctly. The abnormal gait motions were checked by verbal instruction and real-time measurement of the knee angle from cRSEA. Through repeated practice, the continual abnormal cycles were ensured. As shown in the figure, the knee is not fully extended in stance (s and t in Fig. 14a), which results in insufficient forward movement of the upper body. Also the knee is not flexed enough in swing phases (u and v in Fig. 14a), as a result, the foot clearance is not enough, i.e. the forefoot is dragged across the floor. The knee angle of the abnormal gait in three strides is shown in Fig. 14b, and the corresponding numbers of abnormal motions in Fig. 14a are presented in the figure.
In this paper, the rehabilitation strategy to determine the assistive torque needed to practice the normal gait in robotic gait rehabilitation devices was discussed. Inspired by the repetitive characteristic of gait motions, a gait rehabilitation strategy based on an iterative learning algorithm was proposed. Taking advantage of the repetitive nature of gait motions, the joint can be guided to the desired trajectory more effectively and accurately. In this algorithm, the assistive torque in the current stride was computed based on information from previous strides. The performance of the proposed algorithm was verified in simulations and the experiments with an active knee orthosis. It should be noted that the experimental results in Fig. 15 only demonstrate that the proposed method can adequately generate the necessary assistive torques to practice normal gait motions. The effectiveness of the proposed method in the rehabilitation
80
Normal
Abnormal
Angle (deg)
60
40
20
0 0
0.5
1
1.5
2
2.5
3
Time (sec)
Fig. 14. Abnormal knee motions.
3.5
4
4.5
220
J. Bae, M. Tomizuka / Mechatronics 22 (2012) 213–221
80
Knee angle (deg)
Normal
Measured
60 40 20 0 24
24.5
25
25.5
26
26.5
27
27.5
28
28.5
27
27.5
28
28.5
27.5
28
28.5
Time (sec)
Angle deviation (deg)
10 5 0 -5 -10
24
24.5
25
25.5
26
26.5
Time (sec)
Assistive torque (Nm)
3 2 1 0 -1 -2
24
24.5
25
25.5
26
26.5
27
Time (sec)
Fig. 15. Experimental results.
treatment should be verified by clinical tests. Thus, as future work, the proposed gait rehabilitation strategy will be applied to patients suffering from degenerated nerve systems and the patients’ status will be checked for an extended period of time to observe the improvement. Acknowledgment This work was supported by National Science Foundation (NSF) Grant CMMI-0800501. References [1] Bae J, Kong K, Byl N, Tomizuka M. A mobile gait monitoring system for abnormal gait diagnosis and rehabilitation: a pilot study for Parkinsons disease patients. ASME J Biomech Eng 2011;133:041005. 11p. [2] Bae J, Kong K, Tomizuka M. Real-time estimation of lower extremity joint torques in normal gait. In: Proceedings of the international IFAC symposium on robot control (SYROCO); 2009. p. 577–82. [3] Bae J, Kong K, Tomizuka M. Cable friction compensation and rehabilitation algorithms for a cable driven human assistive system. In: ASME Dynamic Systems and Conctorl Conference (DSCC); 2010. p. 1–7. [4] Belforte G, Eula G, Appendino S, Sirolli S. Pneumatic interactive gait ehabilitation orthosis: design and preliminary testing. J Eng Med. Proceedings of the institution of mechanical engineers, part H 2011;225:158–69.
[5] Beyl P, Damme MV, Ham RV, Lefeber D. Design and control concepts of an exoskeleton for gait. In: Proceedings of the 2nd biennial IEEE/RAS-EMBS international conference on biomedical robotics and biomechatronics; 2008. p. 103–8. [6] Bristow DA, Tharayil M, Alleyne AG. A survey of iterative learning control: a learning-based method for high-performance tracking control. IEEE Control Syst Mag 2006;26:96–114. [7] Colombo G, Joerg M, Schreier R, Dietz V. Treadmill training of paraplegic patients using a robotic orthosis. J Rehabil Res Develop 2000;37:693–700. [8] de Roover D. Synthesis of a robust iterative learning controller using an H1 approach. In: IEEE international conference on decision and control; 1996. p. 3044–9. [9] Dong S, Lu K-Q, Sun JQ, Rudolph K. Adaptive force regulation of muscle strengthening rehabilitation device with magnetorheological fluids. IEEE Trans Neural Syst Rehabil Eng 2006;14:55–63. [10] Duschau-Wicke A, Brunsch T, Lunenburger L, Riener R. Adaptive support for patient-cooperative gait rehabilitation with the Lokomat. In: IEEE/RSJ international conference on intelligent robots and systems (IROS); 2008. p. 2357–61. [11] Duschau-Wicke A, von Zitzewitz J, Banz R, Riener R. Iterative learning synchronization of robotic rehabilitation tasks. In: IEEE 10th international conference on rehabilitation robotics (ICORR); 2008. p. 335–40. [12] Frey M, Colombo G, Vaglio M, Bucher R, Jorg M, Riener R. A novel mechatonic body weight support system. IEEE Trans Neural Syst Rehabil Eng 2006;14:311–21. [13] Gazzani F, Fadda A, Torre M, Macellari V. WARD: a pneumatic system for body weight relief in gait rehabilitation. IEEE Trans Neural Syst Rehabil Eng 2000;8:506–13.
J. Bae, M. Tomizuka / Mechatronics 22 (2012) 213–221 [14] Gomes A, Onodera A, Otuzi M, Pripas D, Mezzarane R, Sacco I. Electromyography and kinematic changes of gait cycle at different cadences in diabetic neuropathic individuals. Muscle Nerve 2011;44:258–68. [15] Havlicsek H, Alleyne A. Nonlinear control of an electro-hydraulic injection molding machine via iterative adaptive learning. IEEE/ASME Trans Mech 1999;4:312–23. [16] Hayashi T, Kawamoto H, Sankai Y. Control method of robot suit hal working as operator’s muscle using biological and dynamical information. In: IEEE/RSJ international conference on intelligent robots and systems (IROS); 2005. p. 3063–8. [17] Hogan N. Impedance control: an approach to manipulation. Parts I, II, and III0 . J Dyn Syst, Meas, Control 1985;107:1–24. [18] Jezernik S, Colombo G, Keller T, Frueh H, Morari M. Robotic orthosis Lokomat: a rehabilitation and research tool. Neuromodulation: Technol Neural Interface 2003;6:108–15. [19] Kim C, Eng J. Magnitude and pattern of 3d kinematic and kinetic gait profiles in persons with stroke: relationship to walking speed. Gait Posture 2004;20:140–6. [20] Kim D, Kim S. An iterative learning control method with application for CNC machine tools. IEEE Trans Ind Appl 1996;32:66–72. [21] Kong K, Bae J, Tomizuka M. Control of rotary series elastic actuator for ideal force mode actuation in human–robot interaction applications. IEEE/ASME Trans Mech 2009;14:105–18. [22] Kong K, Bae J, Tomizuka M. A compact rotary series elastic actuator for knee joint assistive system. In: Proceedings of the IEEE international conference on robotics and automation (ICRA); 2010. p. 2940–5. [23] Lim MR, Huang RC, Wu A, Girardi FP, Cammisa FP. Evaluation of the elderly patient with an abnormal gait. J Am Acad Orthop Surg 2007;15: 107–17. [24] Lina C, Guob L, Su F, Choub Y, Cherngd R. Common abnormal kinetic patterns of the knee in gait in spastic diplegia of cerebral palsy. Gait Posture 2000;11:224–32. [25] Lunenburger L, Colombo G, Riener R. Biofeedback for robotic gait rehabilitation. J NeuroEng Rehabil 2007;4.
221
[26] Mishra S. Fundamental Issues in Iterative Learning Controller Design: Convergence, Robustness, and Steady State Performance, PhD thesis, University of California, Berkeley; 2008. [27] Mishra S, Tomizuka M. Precision positioning of wafer scanners: an application of segmented iterative learning control. Control Syst Mag 2007;27:20–5. [28] O’Byrne J, Jenkinson A, O’Brien T. Quantitative analysis and classification of gait patterns in cerebral palsy using a three-dimensional motion analyzer. J Child Neurol 1998;13:101–8. [29] Pandy MG. Simple and complex models for studying muscle function in walking. Phil Trans Roy Soc Lond 2003;358:1501–9. [30] Parkinson’sDiseaseFoundation, Statistics for Parkinson’s Disease, 2011.
Contents lists available at SciVerse ScienceDirect
Mechatronics journal homepage: www.elsevier.com/locate/mechatronics
A gait rehabilitation strategy inspired by an iterative learning algorithm Joonbum Bae a,⇑, Masayoshi Tomizuka b a b
School of Mechanical and Advanced Materials Engineering, UNIST, Ulsan, Republic of Korea Department of Mechanical Engineering, University of California, Berkeley, CA 94720, USA
a r t i c l e
i n f o
Article history: Received 29 December 2010 Accepted 22 January 2012 Available online 23 February 2012 Keywords: Gait rehabilitation strategy Iterative learning algorithm Robotic gait rehabilitation device
a b s t r a c t Robotic gait rehabilitation devices enable efficient and convenient gait rehabilitation by mimicking the functions of physical therapists. In manual gait rehabilitation training, physical therapists have patients practice and memorize normal gait patterns by applying assistive torque to the patient’s joint once the patient’s gait deviates from the normal gait. Thus, one of the most important factors in robotic gait rehabilitation devices is to determine the assistive torque to the patient’s joint during rehabilitation training. In this paper, the gait rehabilitation strategy inspired by an iterative learning algorithm is proposed, which uses the repetitive characteristic of gait motions. In the proposed strategy, the assistive joint torque in the current stride is calculated based on the information from previous strides. Simulation results and experimental results using an active knee orthosis are presented, which verify that the proposed strategy can be used to calculate appropriate assistive joint torque to excise the desired motions for rehabilitation. Ó 2012 Elsevier Ltd. All rights reserved.
1. Introduction As the number of people who have either totally or partially lost the ability to walk due to aging or physical impairment increases [37,30,35,23], the demand for robotic gait rehabilitation devices increases. Robotic gait rehabilitation devices enable more efficient and convenient gait rehabilitation by mimicking the functions of physical therapists. In manual gait rehabilitation training, physical therapists establish suitable rehabilitation strategies depending on the gait disorders patients suffer from. While there is a long list of gait disorders that inhibits patients’ ability to walk, the causes of the gait disorders can be classified as degeneration in two categories: (1) in muscular systems and (2) in nerve systems. If a patient has weakened muscles due to accidents, diseases, or aging, then the patient cannot generate the joint torque necessary to achieve desired gait motion. For these patients, physical therapists try to strengthen patients’ muscles by applying appropriate resistive force to muscles. Weight training for specific muscles is one example of the many possible rehabilitation strategies. This gait rehabilitation strategy was mimicked by the robotic gait rehabilitation device for strengthening muscles [9]. If the muscles are too damaged to recover their original function, then power augmentation systems ([16,41,40] among others) may help the patients achieve normal walking movements.
⇑ Corresponding author. E-mail addresses: [email protected] (J. Bae), [email protected] (M. Tomizuka). 0957-4158/$ - see front matter Ó 2012 Elsevier Ltd. All rights reserved. doi:10.1016/j.mechatronics.2012.01.009
Patients with impaired motor control via spinal cord injury (SCI) or stroke has the problem that they cannot control their muscles properly. Assuming that only their nervous system is degenerated, i.e. their muscles are strong enough to generate muscular forces to achieve the desired gait motions, they need to practice and memorize the desired gait patterns through repetitive exercises. For these patients, physical therapists make patients practice normal gait patterns through force or voice feedback. Physical therapists apply assistive torques to the joints to let them be cognizant of how their gait motions deviate from the normal gait, and guide them to the normal gait trajectory. Also, physical therapists keep talking to patients during the gait rehabilitation training about how they need to move to achieve the normal gait patterns, e.g. bending knee more or pushing heel more. By mimicking these functions of physical therapists, robotic gait rehabilitation systems that utilize visual feedback [1,25] or assistive torque have been developed [12,7,13,32,38,5,4,18]. Since assistive torque allow patients to practice normal gait pattern easily and effectively, determining required assistive torque in a rehabilitation application is one of the most important factors in the robotic gait rehabilitation system. In fact, the assistive torque should be proportional to the amount of deviation from a normal gait trajectory in order for patients to perceive the right ‘‘feeling’’ about a normal gait trajectory. Through assistive torque, the joint is guided accurately to the desired gait trajectory for rehabilitation. In the past, rehabilitation strategies that impose a virtual potential field around the desired trajectory was introduced [3]. In this strategy, the induced force by the potential field compels the joint to move to the desired joint trajectory once the joint position
214
J. Bae, M. Tomizuka / Mechatronics 22 (2012) 213–221
deviates from the desired joint position. The potential field rehabilitation strategy provides a good method to calculate the required torque for patients to practice normal gait trajectory. However, in order to guide a joint to the correct gait trajectory, a high control gain is required, which may cause instability and overshoot. Inspired by the repetitive characteristic of gait motions, a gait rehabilitation strategy based on an iterative learning algorithm is proposed in this paper. Taking advantage of the repetitive characteristic of gait motions, the joint can be guided to the desired trajectory more effectively and accurately. The proposed rehabilitation strategy utilizes position errors and error derivatives of previous strides to calculate the assistive torque in the current stride. By using information from previous strides, repetitive abnormal gait patterns can be penalized more effectively. The performance of the proposed gait rehabilitation strategy was verified through simulations and experiments with an active knee orthosis. This paper is organized as follows. In Section 2, the role of assistive torque in robotic gait rehabilitation devices is discussed through explanations of how the gait rehabilitation training takes place. The gait rehabilitation strategy inspired by an iterative learning algorithm and its simulation results are presented in Section 3 while experimental results by the proposed algorithm are shown in Section 4. Finally, conclusions and future work are given in Section 5.
2. Assistive torque in robotic gait rehabilitation devices 2.1. Motion control in a human body Many models have been proposed to represent the motion control system in the human body. Among the models, the closed-loop control system in Fig. 1 is one of the most dominant [33]. The ‘Desired Human Motion’ in the figure is created by either an external stimulus or an internally generated intention. In the figure, the ‘Brain’ includes many parts such as the motor cortex, cerebellum, basal ganglia, and spinal cord all cooperate with one another to act as a controller for the motor control. The muscle control signals (r in Fig. 1) generated by ‘Brain’ are transferred to ‘Muscle’ though motor neurons, and ‘Muscle’ moves ‘Body’ via appropriate muscu-
Desired Human Motion
-
Brain
Body
Muscle
Human Motion
Sensory Organs Fig. 1. Closed-loop motion control system in a human body.
Rehabilitation Patient
Rehabilitation Patient
Physical Therapist
Physical Therapist
Voice Feedback
Force Feedback
Fig. 2. Voice/force feedback in gait rehabilitation training.
lar forces (s in Fig. 1). ‘Muscle’ is considered as an actuator, and ‘Body’ represents the musculoskeletal part of a human body, which is considered as a plant to be controlled. Human motions, which are sensed by various ‘Sensory Organs’ such as the eyes, muscle spindles and the Golgi tendon organs, are then compared with the desired motions to achieve precise motions. In this paper, the closed-loop motion control system in Fig. 1 is used to analyze human motions and to design rehabilitation strategies. ‘Desired Human Motion’ in Fig. 1 is assumed to be given by the motion planning part of the brain. 2.2. Assistive torque in robotic gait rehabilitation devices The goal of gait rehabilitation training is for patients that suffer from gait disorders to restore locomotion through rehabilitation of their degenerated muscular or neural functions. By applying assistive torque to the patient’s joint, a patient can either practice the desired gait trajectory for rehabilitation or strengthen muscles. If the patient’s problem is an inability to supply sufficient muscle strength to meet the demands of desired gait motions (insufficient s in Fig. 1), then the patient needs assistive torque in order to achieve the desired motion. In this case, the gait rehabilitation strategies that emphasize the recovery of muscular forces are required. Applying appropriate resistive torque to strengthen muscles is one of the possible rehabilitation strategies. Patients with impaired nerve systems, e.g. spinal cord injury (SCI) or stroke, cannot control their muscles due to abnormal motor control signals (abnormal r in Fig. 1). For these patients, physical therapists apply voice or force feedback to patients during gait rehabilitation training as shown in Fig. 2. Through force or voice feedback, physical therapists educate patients of normal joint trajectory and also physically guide patients to the desired joint trajectory for rehabilitation. In this paper, the gait rehabilitation strategy on how to determine assistive torque in robotic gait rehabilitation devices in order to practice a rehabilitative gait trajectory is discussed. 3. A gait rehabilitation strategy inspired by an iterative learning algorithm 3.1. A gait rehabilitation strategy inspired by a potential field The assistive torque in robotic gait rehabilitation devices provides a ‘‘feeling’’ to patients that allows them to sense how far their gait motions deviate from the desired gait trajectories. Through ‘‘feeling’’, the patients can learn the desired gait trajectory and be guided to the correct gait trajectory. Thus, the assistive torque should be proportional to the deviation from the desired gait trajectory and be capable of guiding the joint to the gait trajectory effectively and accurately. Since the amount of torque is related to the deviation from the normal trajectory, it can be considered as an impedance between human and the robotic gait rehabilitation device [17]. Previously, the gait rehabilitation strategy was formulated using a potential field [3]. In this strategy, a virtual potential field is placed around the desired trajectory, which generates induced forces for the joint to stay on the desired trajectory. The concept of the induced force by the potential field is shown in Fig. 3. The potential field, PA, for the gait rehabilitation strategy can be expressed as a function of the human joint angle, yH, and the desired joint angle for rehabilitation, yR, as follows:
PA ¼ aðyH yR Þb
ð1Þ
The values of a and b can be selected as any values considering the patients’ status or the aim of rehabilitation training, as long as PA has a global minimum at yR. The amount of the assistive torque is
215
J. Bae, M. Tomizuka / Mechatronics 22 (2012) 213–221
calculated as the negative gradient of the potential field with respect to the human joint angle: i.e.
sAd ¼ ryH PA
ð2Þ
PA ¼ ðyH yR Þ2 sAd ¼ 2ðyH yR Þ
ð3Þ ð4Þ
Potential Field (J)
One example of the proposed potential field is shown in Fig. 4. In this figure, the desired joint trajectory for rehabilitation is given as a simple sinusoidal wave (the thick line in the figure), and the potential field (gray lines in the figure) is given by a quadratic function of (3) with a = 1 and b = 2 in (1). If the human joint deviates from the desired angle (small circles in the figure), then assistive torque in (4) is generated by the potential field for the joint to be moved back to the desired trajectory.
10 8 6 4 2 0 2 1 10
0
Angle (rad)
5
-1 -2
3.2. A gait rehabilitation strategy inspired by an iterative learning algorithm The rehabilitation strategy by the potential field provides a good method to calculate the required torque needed for patients to practice the normal gait trajectory. If we rely on feedback control, a high control gain is required to guide a patient to stay close to the desired trajectory accurately, which may cause instability and overshoot. Since walking is a motion that repetitively uses two legs to move forward, gait motions are naturally repetitive over strides. Contrary to the normal gait pattern of healthy individuals, there is no ‘typical’ abnormal gait pattern for all abnormal gait motions. However, individuals with gait disorders exhibit their own distinctive gait motions over strides, and those motions can be even characterized according to their pathologies [34,28,24,14,19]. This paper proposes a gait rehabilitation strategy that uses iterative learning control (ILC) by noting the repetitive characteristics of gait motions. Fig. 5 shows normal and abnormal knee joint angles in three strides under the premise of the experiment in Section 4.1. A healthy subject mimicked a foot-dragging gait motion to produce the abnormal knee trajectory. Note that the deviation in angle from
0
Time (sec)
Fig. 4. Example of a potential field (thick line: desired joint trajectory, gray lines: potential field, small circles: human joint trajectory).
the normal knee angles in one stride is similar to those in other strides. In order to generate appropriate assistive torques in the repeated strides, an iterative learning algorithm, where the amount of assistive torque in the current stride is calculated based on the information from previous strides, is proposed in this paper. Many applications utilizing the iterative learning algorithm can be found in controls of mechanical systems tasks such as industrial robot [36], wafer stage systems [27], computer-numerical control tools [20], and injection molding systems [15]. ILC is based on the paradigm of learning as the name suggests. In a repetitive process, information from earlier iterations can be used to improve performance in the current iteration. ILC is anticipatory and can compensate for exogenous signals, such as repeating disturbances, in advance by learning from previous iterations [6]. Also, the design method requires minimal model information about the plant being controlled [26]. A widely used ILC control input is
ukþ1 ðjÞ ¼ Q ðqÞ½uk ðjÞ þ LðqÞek ðj þ 1Þ
ð5Þ
where u is the control input, j is the time index, k is the iteration index, q is the forward time-shift operator, qx(j) x(j + 1). Q(q) is a Qfilter for enhanced stability and robustness while L(q) is a learning function. A block diagram with a plant, P, and the control input (5) is drawn in Fig. 6. The errors and the control inputs in previous iterations are saved in memory and are then used to calculate the control input in the current iteration. By combining the iterative learning algorithm in (5) with the potential field algorithm in (2), the following gait rehabilitation strategy is proposed to calculate the assistive torque.
Current position
τ Ad Desired position
sA;kþ1 ðjÞ ¼ f ðekþ1 ðjÞÞ þ Q ðqÞ
sA;k ðjÞ þ aP ek ðj þ 1Þ þ aD
ek ðj þ 1Þ ek ðjÞ Ts
Potential Field (J)
ð6Þ 4 3 2 1 0 -2
-1.5
-1
-0.5
0
0.5
1
1.5
Deviated angle (rad) Fig. 3. Concept of a potential field for a rehabilitation strategy.
2
where ek(j) = yR(j) yH,k(j) and Ts is the sampling time. yR is the desired joint trajectory for rehabilitation and yH,k is measured human motion in the kth stride. The first term in (6), f(ek+1(j)), represents the assistive torque by the potential field algorithm defined in (2). It only depends on the error in the current stride (r in Fig. 7). The remaining terms in (6) are by the iterative learning algorithm, i.e. learned from the error and the error derivative in the previous stride (s in Fig. 7). The iterative learning algorithm in (6) uses PD-type update law to calculate the rehabilitation torque; P-type term uses the error signal, and D-type term uses the error derivative
216
J. Bae, M. Tomizuka / Mechatronics 22 (2012) 213–221
80
Angle (deg)
Normal
Abnormal
60 40 20 0
0
0.5
1
2
1.5
2.5
3
3.5
4
4.5
3
3.5
4
4.5
Time (sec)
40
Angle (deg)
30 20 10 0 -10 0.5
0
1.5
1
2
2.5
Time (sec)
Fig. 5. Normal/abnromal knee angles.
in the previous iteration. By utilizing errors and error derivatives in previous strides, repetitive abnormal gait motions can be penalized more effectively. If the potential field is selected as a simple quadratic function in (3), the proposed algorithm can be represented only by the ILC with a different learning function. The rehabilitation algorithm in (6) can be written as
sA;kþ1 ðjÞ ¼ K p ekþ1 ðjÞ þ Q ðqÞ½sA;k ðjÞ þ LðqÞek ðj þ 1Þ where Kp is from the potential field 1 LðqÞ ¼ aP þ aD 1þq . Eq. (7) can be rewritten as Ts
ð7Þ algorithm,
sA;kþ1 ðjÞ ¼ K p ekþ1 ðjÞ þ wkþ1 ðjÞ
and
ð8Þ
where
wkþ1 ðjÞ ¼ Q ðqÞ½sA;k ðjÞ þ LðqÞek ðj þ 1Þ
ð9Þ
Then,
wkþ1 ðjÞ ¼ Q ðqÞ½wk ðjÞ þ ðK p q1 þ LðqÞÞek ðj þ 1Þ
ð10Þ
1
which uses a new learning function as Kpq + L(q). The block diagram of the proposed rehabilitation strategy is depicted in Fig. 8. The assistive torque at the (k + 1)th stride, sA,k+1, is added to the human joint torque, sH. With assistive torque, the patient can easily be guided to reach to the desired gait trajectory for rehabilitation and memorize the motion through repetition. The joint angles are measured by encoders installed in the active knee orthosis introduced in Section 4.2. Angles of deviation from the desired gait motions and the assistive torque from previous iteration ILC Algorithm
yd
ek +1
Memory
Memory
ek
uk
L
Q
u k +1
Fig. 6. Block diagram of iterative learning control (ILC).
P
yk +1
are saved in order to compute the assistive torque of the current stride via the iterative learning algorithm. The ILC algorithm has been also applied to gait rehabilitation systems [11,10]. In their algorithms, the information of previous steps was used to calculate the assistive torque in current step, but consideration has not been paid to robustness or transient learning behavior of the system; Q-filtering was not utilized, i.e. Q = 1. It is well known that the learning gains influence the rate of convergence, whereas the Q-filter influences the converged error performance. Increasing the Q-filter bandwidth decreases robustness but improves performance, whereas decreasing the bandwidth has the opposite effect [6,8]. Thus, the cut-off frequency of the Q-filter is a tradeoff parameter between performance and robustness. In this paper, the cut-off frequency of the Q-filter is set to 10 Hz as shown Fig. 9 considering human walking motion range [39]. By limiting the cut-off frequency of the Q-filter, robustness of the system in the range of walking can be improved. Also, the learning filter of the learning algorithm in (6) corresponds to LðzÞ ¼ aP z þ aD z1 in the frequency domain. Since the D-type iteraTs tive learning update law uses the error derivative that is included in this learning algorithm, the Q-filter should be designed to remove high frequency components of the error derivative.
3.3. Forgetting factor and desired gait trajectory for rehabilitation Patients’ gait motions may improve during gait rehabilitation treatments by being adapted to the desired trajectory. The algorithm in (6) uses the position information from all previous iterations for the calculation of the current assistive torque because the assistive joint torque which includes the position information from previous strides is fed back to calculate the assistive torque in the current stride. Due to improvements in gait motions during the rehabilitation treatment, it may be inappropriate to use data from many strides ago in calculating the current assistive torque. By utilizing a forgetting factor, k, as in (11), the influence of old assistive torque record may be deemphasized. The forgetting factor
217
J. Bae, M. Tomizuka / Mechatronics 22 (2012) 213–221
80 60
( k + 1)th stride
kth stride 2
1
40 20 0 0
0.5
1
j
1.5
2.5
2
-5 -10 Normal Walking Range (~10Hz)
-15 -20 -25 1 10
3
j
( j+1)
Bode Diagram
0
Abnormal
Magnitude (dB)
Angle (deg)
Normal
10
2
3
10
Frequency (rad/sec)
Fig. 7. Calculation of the assistive torque by an iterative learning algorithm.
Fig. 9. Design of the Q-filter.
takes a value between 0 and 1, and it determines how quickly previous assistive torques are forgotten in the calculation of the current assistive torque.
goal. In the following simulations and experiments, k in (11) was set to 1, and a, b and c in (12) were set to 1, 1 and 0, respectively.
sA;kþ1 ðjÞ ¼ f ðekþ1 ðjÞÞ
3.4. Simulation study
ek ðj þ 1Þ ek ðjÞ þ Q ðqÞ k sA;k ðjÞ þ aP ek ðj þ 1Þ þ aD Ts
The proposed rehabilitation strategy in (6) was simulated with human models. The human body can be modeled with different mathematical complexities according to the number of joints and links [2,29]. Complicated models with many joints and links may give accurate analysis, but an excessive number of segments increase the complexity of the resulting equations because of the complex coupling among all the segments. In swing phases, the knee joint torque is primarily used for swinging the shank and the foot. A simplified model is the pendulum model in Fig. 10a. The shank and the foot are modeled as a uniform cylinder which has the mass of m1 and the length of l1. In stance phases, the knee joint torque is used for the upper body advancement. This can be modeled as an inverted pendulum; the foot is fixed on the ground, and the body’s upper parts above the knee is an inverted pendulum as shown in Fig. 10b. The whole upper body is also modeled as a uniform cylinder which has the mass of m2 and the length of l2. Thus, two widely-used simple models, the pendulum model and the inverted pendulum model, are used for swing phases and stance phases in this paper. To obtain the system equation in swing phases, the kinetic and potential energies of the system are computed as follows:
ð11Þ The final goal of gait rehabilitation may be said to restore the normal gait pattern, but the normal gait trajectory is not necessarily the best trajectory for every patient. Since the actual trajectory for rehabilitation depends on patient’s conditions, simply forcing the patients’ joints to follow the normal trajectory is not always effective in rehabilitation. Thus, in the proposed algorithm, the desired gait trajectory is determined based on the normal trajectory along with consideration of the patient’s conditions and the rehabilitation goal as expressed in (12).
yR ðtÞ ¼ aðtÞ yN
t þ cðtÞ bðtÞ
ð12Þ
where yN and yR represent the normal trajectory and the rehabilitation trajectory, respectively. The parameter a scales the amplitude of yN, b stretches the time and influences the period of the motion, and c changes the offset. The values of a, b and c in (12) determine the desired gait trajectory and k in (11) determines the effect of previous iterations’ position information for the calculation of the assistive torque in the current iteration. Variations of the values in the proposed algorithms allow sufficient degrees of freedom in the gait rehabilitation strategy. The actual values will be determined by physical therapists considering the patient’s condition and the rehabilitation
1 2 m1 l1 y_ 2 6 V ¼ m1 gl1c ð1 cos yÞ
T¼
yN
y H,k +1 -
ð13Þ ð14Þ
Encoder ILC Algorithm
ek +1 Potential Field Algorithm
Memory
ek
C wk +1
Q
τ A,k
τ A,k+1 Desired Human Motion
-
Brain
Muscle
L
τH
Memory Body
Sensory Organs Fig. 8. Block diagram of the proposed rehabilitation strategy.
Human Motion
218
J. Bae, M. Tomizuka / Mechatronics 22 (2012) 213–221
where L = T V. Likewise, the system equation in stance phases is,
1 3
s ¼ m2 l22 y€ m2 gl2c sin y
In the simulation, ‘Brain’ in Fig. 8 is modeled as a poorly tuned PID controller which generates abnormal muscle control signals so that ‘Muscle’ in Fig. 8 cannot generate the appropriate torque necessary to achieve the normal knee motions. The simulation results by the proposed algorithm in (6) are shown in Fig. 11. Note that the deviation from the normal angle in the first stride is larger than subsequent strides since learning is yet to take place in the first stride. In other words, only the potential field algorithm is used in the first stride. Once the iterative learning algorithm comes into full effect from the second stride, the proposed strategy generates appropriate assistive torque so that the deviation from the normal knee angle is decreased, as shown in Fig. 11b.
Fig. 10. Human models for the simulation.
80
Angle (deg)
Abnormal
Normal
Simulation
4.1. Knee motions in normal gait
40 20
0
1
0.5
2
1.5
2.5
3
3.5
4
4.5
3
3.5
4
4.5
Time (sec)
Angle (deg)
20 10 0 -10 -20
0
0.5
1.5
1
2.5
2
Time (sec)
Fig. 11. Simulation results.
Angle (deg)
4. Experimental results
60
0
Flexion
60 40
Since the proposed rehabilitation strategy was applied to the knee joint in this paper, the normal knee motions in sagittal plane is studied in this section. Also, the abnormal knee motions for the experiments are compared with the normal knee motions. The normal knee motions in one stride is shown in Fig. 12 with corresponding gait phases. The normal knee angle is obtained by averaging data from 30 strides of three healthy male subjects (10 strides per subject) without any known gait disorders (mean age: 29, mean height: 1.74 m, mean weight: 71 kg), and verified by literatures [31,39]. Informed consent forms approved by the Institutional Review Boards for the University of California at Berkeley were provided to all participants. Normal knee angles during walking are within the range of 0° to 70° as shown in Fig. 12. At initial contact phase, the knee is flexed about 5°. In loading response phase, putting body weight on the limb instantly disturbs the knee’s stability. The knee flexion, however, provides shock absorption in load response phase. Throughout the loading phase, the knee is rapidly flexed. In the rest of the stance, the knee extends and the ankle dorsiflexes in order to propel the body mass forward. In swing, the knee is flexed to lift the foot for limb advancement. This is the critical action that ensures foot clearance as the limb swings forward from the trailing posture.
20
4.2. Experimental setup
0
Extension
0
10
20
30
40
50
60
70
80
90
100
Gait Cycle (%)
Initial Loading Mid Contact Response Stance
Terminal Stance
PreSwing
Initial Swing
Mid Swing
Terminal Swing
Fig. 12. Normal knee angle in one stride and corresponding gait phases.
where T and V represent the kinetic energy and the potential energy, respectively. Then, the system equation is obtained by Lagrangian mechanics as follows:
d @L @ L dt @ y_ @y 1 2 € þ m1 gl1c sin y ¼ m1 l1 y 3
s¼
ð17Þ
ð15Þ ð16Þ
The proposed gait rehabilitation strategy was applied to the abnormal knee motions in Fig. 14. A compact rotary series elastic actuator (cRSEA) shown in Fig. 13a was used. The rotary series elastic mechanism was adapted to the cRSEA and it was designed and controlled with consideration of precise torque generation [21]. A worm gear set is used to make cRSEA compact and light. It operates in ideal force/torque mode by compensating for inherent mechanical impedance so that it precisely generates the desired torque. Thus, the computed rehabilitation torque, sA,k+1 in Fig. 8, is assumed to be generated accurately in this experiment. The human joint angle was measured by an encoder in the cRSEA. For more details, see [22]. The cRSEA was installed on the knee joint of an orthosis as shown in Fig. 13a. Slightly impaired knee motions by foot-dragging mimicked by a healthy subject (age: 30, height: 1.8 m, weight: 68 kg) was used in the experiments. The healthy subject was asked to wear the experimental orthosis as shown in Fig. 13b, and walk on the treadmill at a speed of 3.2 km/h with abnormal knee
219
J. Bae, M. Tomizuka / Mechatronics 22 (2012) 213–221
4.3. Experimental results
Fig. 13. The compact rotary series elastic actuator (cRSEA) installed on the knee joint of an orthosis.
The goal of the experiments was to verify that the proposed strategy generates adequate assistive torque so that a patient with an abnormal gait can practice and improve his/her gait to a desired gait trajectory during rehabilitation. As shown in the experimental results in Fig. 15c, appropriate assistive joint torques were generated by the proposed algorithm. Also, with the assistance of the generated assistive joint torque, the abnormal knee trajectory in Fig. 14 was altered to match the normal knee trajectory as shown in Fig. 15a and b. Even though a healthy person, not a patient, performed the abnormal gait motions, it was observed that under the proposed rehabilitation algorithm the intention of the normal person to mimic the abnormal walking motion did not result in the intended motion and was drive to the normal knee trajectory by the generated assistive torque. It took around 20 steps for the subject to adapt to the active knee orthosis and the desired knee trajectory, but after the adaption, the knee could follow the normal gait trajectory with aid of assistive torque.
5. Conclusion motions shown in Fig. 14a. The walking speed was set slower than normal walking speed since it is easier to practice the normal gait at slower speeds. The subject practiced the abnormal foot-dragging motion several times before the experiment to ensure that he mimicked the abnormal gait motions correctly. The abnormal gait motions were checked by verbal instruction and real-time measurement of the knee angle from cRSEA. Through repeated practice, the continual abnormal cycles were ensured. As shown in the figure, the knee is not fully extended in stance (s and t in Fig. 14a), which results in insufficient forward movement of the upper body. Also the knee is not flexed enough in swing phases (u and v in Fig. 14a), as a result, the foot clearance is not enough, i.e. the forefoot is dragged across the floor. The knee angle of the abnormal gait in three strides is shown in Fig. 14b, and the corresponding numbers of abnormal motions in Fig. 14a are presented in the figure.
In this paper, the rehabilitation strategy to determine the assistive torque needed to practice the normal gait in robotic gait rehabilitation devices was discussed. Inspired by the repetitive characteristic of gait motions, a gait rehabilitation strategy based on an iterative learning algorithm was proposed. Taking advantage of the repetitive nature of gait motions, the joint can be guided to the desired trajectory more effectively and accurately. In this algorithm, the assistive torque in the current stride was computed based on information from previous strides. The performance of the proposed algorithm was verified in simulations and the experiments with an active knee orthosis. It should be noted that the experimental results in Fig. 15 only demonstrate that the proposed method can adequately generate the necessary assistive torques to practice normal gait motions. The effectiveness of the proposed method in the rehabilitation
80
Normal
Abnormal
Angle (deg)
60
40
20
0 0
0.5
1
1.5
2
2.5
3
Time (sec)
Fig. 14. Abnormal knee motions.
3.5
4
4.5
220
J. Bae, M. Tomizuka / Mechatronics 22 (2012) 213–221
80
Knee angle (deg)
Normal
Measured
60 40 20 0 24
24.5
25
25.5
26
26.5
27
27.5
28
28.5
27
27.5
28
28.5
27.5
28
28.5
Time (sec)
Angle deviation (deg)
10 5 0 -5 -10
24
24.5
25
25.5
26
26.5
Time (sec)
Assistive torque (Nm)
3 2 1 0 -1 -2
24
24.5
25
25.5
26
26.5
27
Time (sec)
Fig. 15. Experimental results.
treatment should be verified by clinical tests. Thus, as future work, the proposed gait rehabilitation strategy will be applied to patients suffering from degenerated nerve systems and the patients’ status will be checked for an extended period of time to observe the improvement. Acknowledgment This work was supported by National Science Foundation (NSF) Grant CMMI-0800501. References [1] Bae J, Kong K, Byl N, Tomizuka M. A mobile gait monitoring system for abnormal gait diagnosis and rehabilitation: a pilot study for Parkinsons disease patients. ASME J Biomech Eng 2011;133:041005. 11p. [2] Bae J, Kong K, Tomizuka M. Real-time estimation of lower extremity joint torques in normal gait. In: Proceedings of the international IFAC symposium on robot control (SYROCO); 2009. p. 577–82. [3] Bae J, Kong K, Tomizuka M. Cable friction compensation and rehabilitation algorithms for a cable driven human assistive system. In: ASME Dynamic Systems and Conctorl Conference (DSCC); 2010. p. 1–7. [4] Belforte G, Eula G, Appendino S, Sirolli S. Pneumatic interactive gait ehabilitation orthosis: design and preliminary testing. J Eng Med. Proceedings of the institution of mechanical engineers, part H 2011;225:158–69.
[5] Beyl P, Damme MV, Ham RV, Lefeber D. Design and control concepts of an exoskeleton for gait. In: Proceedings of the 2nd biennial IEEE/RAS-EMBS international conference on biomedical robotics and biomechatronics; 2008. p. 103–8. [6] Bristow DA, Tharayil M, Alleyne AG. A survey of iterative learning control: a learning-based method for high-performance tracking control. IEEE Control Syst Mag 2006;26:96–114. [7] Colombo G, Joerg M, Schreier R, Dietz V. Treadmill training of paraplegic patients using a robotic orthosis. J Rehabil Res Develop 2000;37:693–700. [8] de Roover D. Synthesis of a robust iterative learning controller using an H1 approach. In: IEEE international conference on decision and control; 1996. p. 3044–9. [9] Dong S, Lu K-Q, Sun JQ, Rudolph K. Adaptive force regulation of muscle strengthening rehabilitation device with magnetorheological fluids. IEEE Trans Neural Syst Rehabil Eng 2006;14:55–63. [10] Duschau-Wicke A, Brunsch T, Lunenburger L, Riener R. Adaptive support for patient-cooperative gait rehabilitation with the Lokomat. In: IEEE/RSJ international conference on intelligent robots and systems (IROS); 2008. p. 2357–61. [11] Duschau-Wicke A, von Zitzewitz J, Banz R, Riener R. Iterative learning synchronization of robotic rehabilitation tasks. In: IEEE 10th international conference on rehabilitation robotics (ICORR); 2008. p. 335–40. [12] Frey M, Colombo G, Vaglio M, Bucher R, Jorg M, Riener R. A novel mechatonic body weight support system. IEEE Trans Neural Syst Rehabil Eng 2006;14:311–21. [13] Gazzani F, Fadda A, Torre M, Macellari V. WARD: a pneumatic system for body weight relief in gait rehabilitation. IEEE Trans Neural Syst Rehabil Eng 2000;8:506–13.
J. Bae, M. Tomizuka / Mechatronics 22 (2012) 213–221 [14] Gomes A, Onodera A, Otuzi M, Pripas D, Mezzarane R, Sacco I. Electromyography and kinematic changes of gait cycle at different cadences in diabetic neuropathic individuals. Muscle Nerve 2011;44:258–68. [15] Havlicsek H, Alleyne A. Nonlinear control of an electro-hydraulic injection molding machine via iterative adaptive learning. IEEE/ASME Trans Mech 1999;4:312–23. [16] Hayashi T, Kawamoto H, Sankai Y. Control method of robot suit hal working as operator’s muscle using biological and dynamical information. In: IEEE/RSJ international conference on intelligent robots and systems (IROS); 2005. p. 3063–8. [17] Hogan N. Impedance control: an approach to manipulation. Parts I, II, and III0 . J Dyn Syst, Meas, Control 1985;107:1–24. [18] Jezernik S, Colombo G, Keller T, Frueh H, Morari M. Robotic orthosis Lokomat: a rehabilitation and research tool. Neuromodulation: Technol Neural Interface 2003;6:108–15. [19] Kim C, Eng J. Magnitude and pattern of 3d kinematic and kinetic gait profiles in persons with stroke: relationship to walking speed. Gait Posture 2004;20:140–6. [20] Kim D, Kim S. An iterative learning control method with application for CNC machine tools. IEEE Trans Ind Appl 1996;32:66–72. [21] Kong K, Bae J, Tomizuka M. Control of rotary series elastic actuator for ideal force mode actuation in human–robot interaction applications. IEEE/ASME Trans Mech 2009;14:105–18. [22] Kong K, Bae J, Tomizuka M. A compact rotary series elastic actuator for knee joint assistive system. In: Proceedings of the IEEE international conference on robotics and automation (ICRA); 2010. p. 2940–5. [23] Lim MR, Huang RC, Wu A, Girardi FP, Cammisa FP. Evaluation of the elderly patient with an abnormal gait. J Am Acad Orthop Surg 2007;15: 107–17. [24] Lina C, Guob L, Su F, Choub Y, Cherngd R. Common abnormal kinetic patterns of the knee in gait in spastic diplegia of cerebral palsy. Gait Posture 2000;11:224–32. [25] Lunenburger L, Colombo G, Riener R. Biofeedback for robotic gait rehabilitation. J NeuroEng Rehabil 2007;4.
221
[26] Mishra S. Fundamental Issues in Iterative Learning Controller Design: Convergence, Robustness, and Steady State Performance, PhD thesis, University of California, Berkeley; 2008. [27] Mishra S, Tomizuka M. Precision positioning of wafer scanners: an application of segmented iterative learning control. Control Syst Mag 2007;27:20–5. [28] O’Byrne J, Jenkinson A, O’Brien T. Quantitative analysis and classification of gait patterns in cerebral palsy using a three-dimensional motion analyzer. J Child Neurol 1998;13:101–8. [29] Pandy MG. Simple and complex models for studying muscle function in walking. Phil Trans Roy Soc Lond 2003;358:1501–9. [30] Parkinson’sDiseaseFoundation, Statistics for Parkinson’s Disease, 2011.