Iterative learning control of a drop foot neuroprosthesis — Generating physiological foot motion in paretic gait by automatic feedback control

Control Engineering Practice 48 (2016) 87–97

Contents lists available at ScienceDirect

Control Engineering Practice journal homepage: www.elsevier.com/locate/conengprac

Iterative learning control of a drop foot neuroprosthesis — Generating physiological foot motion in paretic gait by automatic feedback control Thomas Seel a,n, Cordula Werner b, Jörg Raisch a,c, Thomas Schauer a a

Control Systems Group, Technische Universität Berlin, Germany Neurological Rehabilitation, Charité Universitätsmedizin Berlin, Germany c Max Planck Institute for Dynamics of Complex Technical Systems, Magdeburg, Germany b

art ic l e i nf o

a b s t r a c t

Article history: Received 30 April 2015 Received in revised form 20 October 2015 Accepted 30 November 2015 Available online 21 January 2016

Many stroke patients suffer from the drop foot syndrome, which is characterized by a limited ability to lift the foot and leads to a pathological gait. We consider treatment of this syndrome via Functional Electrical Stimulation (FES) of the peroneal nerve during the swing phase of the paretic foot. We highlight the role of feedback control for addressing the challenges that result from the large individuality and time-variance of muscle response dynamics. Unlike many previous approaches, we do not reduce the control problem to the scalar case. Instead, the entire pitch angle trajectory of the paretic foot is measured by means of a 6D Inertial Measurement Unit (IMU) and controlled by an Iterative Learning Control (ILC) scheme for variable-pass-length systems. While previously suggested controllers were often validated for the strongly simplified case of sitting or lying subjects, we demonstrate the effectiveness of the proposed approach in experimental trials with walking drop foot patients. Our results reveal that conventional trapezoidal stimulation intensity profiles may produce a safe foot lift, but often at the cost of too high intensities and an unphysiological foot pitch motion. Starting from such conservative intensity profiles, the proposed learning controller automatically achieves a desired foot motion within one or two strides and keeps adjusting the stimulation to compensate time-variant muscle dynamics and disturbances. & 2016 Elsevier Ltd. All rights reserved.

Keywords: Iterative learning control Biomedical engineering application Functional electrical stimulation Motor impairment Inertial measurement unit Realtime motion analysis

1. Introduction According to the World Health Organization, more than a million people suffer a stroke in Europe each year. Due to demographic changes and an increasing life expectancy, this number will rise as with the demand for efficient rehabilitation and medical devices. Stroke often leads to impaired motor function. Even after weeks of rehabilitation, many patients suffer from a limited ability to lift the foot by voluntary muscle contraction. This syndrome is known as drop foot (or foot drop), and it also appears in patients with other neurological disorders. Regardless of the cause, foot drop leads to a pathological gait with an increased risk of fall and injuries like ankle sprain. A common treatment is to fix the foot in the lifted (dorsiflexed) position by a passive orthosis. This approach may improve safety and stability in the patient's gait, but it further promotes muscle atrophy and joint stiffness. If the lesion affects the central nervous system and the n

Corresponding author. E-mail address: [email protected] (T. Seel).

http://dx.doi.org/10.1016/j.conengprac.2015.11.007 0967-0661/& 2016 Elsevier Ltd. All rights reserved.

peripheral nerves are still intact, then an alternative treatment can be provided by means of the technology known as Functional Electrical Stimulation (FES). FES facilitates the artificial activation of muscle contraction by applying tiny electrical pulses via skin electrodes with a conductive gel layer or via implanted electrodes. Due to the risk of complications associated with surgery and implants, we restrict our discussion to the former case. When using skin electrodes, 20–50 rectangular bi-phasic current pulses per second are applied, with each pulse having an amplitude of less than a tenth of an ampere and a pulse width of less than half a millisecond. If the current amplitude exceeds the range of a few milliamperes, each pulse triggers a bunch of action potentials in subcutaneous efferent nerves located near the electrodes. By modulating the frequency and/or dimensions of the pulses, one can control the contraction of paretic muscles and induce functional movements in the affected limbs. Unfortunately, FES may also trigger action potentials in afferent nerves, causing discomfort at medium and pain at high stimulation intensities. In most subjects, however, the sensation is weak enough to allow the generation of functional movements without discomfort. Abundant research demonstrates the potential of FES in neuroprosthesis

88

T. Seel et al. / Control Engineering Practice 48 (2016) 87–97

Fig. 1. A foot-mounted inertial sensor is used to obtain foot pitch angle measurements, which are used to adjust the intensity of FES applied to the shank muscles that lift the foot. By means of this feedback strategy, physiological foot motion can be achieved even in paretic limbs.

design beyond the application of drop foot treatment; see for example Peckham and Knutson (2005) and references therein. Drop foot neuroprostheses, also known as peroneal stimulators, represent a drop foot treatment that aims at generating a natural foot lift via activation of the patient's shank muscles, cf. Ring, Treger, Gruendlinger, and Hausdorff (2009). To this end, the electrodes are placed on the skin near the peroneal nerve, whose branches innervate several shank muscles, as depicted in Fig. 1. When FES with well-chosen pulse dimensions is applied via wellpositioned electrodes in a well-synchronized manner during gait, then the physiological motion of the foot can be restored even in paretic limbs. 1.1. Challenges in FES-based drop foot treatment There are several challenges that need to be addressed when developing FES-based gait support systems for drop foot patients. Experiments show that the amount of foot lift that FES with a certain intensity triggers depends on the subject, varies with time, and is sensitive to small (∼1 cm) changes in the electrode positions. This means that the gain (and dynamics) of the system we are aiming to control is not known a priori. Parameterizing the stimulation such that it yields a physiological foot lift in a paretic leg is a task that must be repeated every time the gait support system is used. Another important issue is that FES-activated muscles fatigue rapidly1 (see e.g. Lynch & Popovic, 2008), which means that large intensities make FES become ineffective. Therefore, it is essential to use the optimal stimulation parameters, i.e. the smallest intensities that achieve a safe and physiological foot lift. This optimum changes due to time-variant effects such as varying muscle tone (spasticity) and residual voluntary muscle activity. When patients cross a street, for example, residual muscle activity as well as the muscle tone (spasticity) often change significantly within less than ten seconds, i.e. less than about five strides. Therefore, once a physiological foot motion has been achieved, it is just as challenging to maintain it. 1.2. State of the art in research and industry For drop foot treatment, a few commercially available solutions 1 I.e. when FES with the same intensity is applied repeatedly, the induced motion will become weaker with time.

make use of FES, some via skin electrodes, others via implanted electrodes. The review articles by Lyons, Sinkjaer, Burridge, and Wilcox (2002) and Melo, Silva, Martins, and Newman (2015) provide an excellent overview of drop foot stimulators in research and industry and classify them in several ways. Until now, all commercially available devices have been solely based on openloop architectures, they only use sensors to time the stimulation (Melo et al., 2015). Most of them employ heel switches to detect two gait phases: one when the heel of the paretic foot is on the ground and the other when it is not. In each stride, as soon as the heel is lifted, FES is applied with a fixed stimulation intensity profile over time, typically a trapezoidal shape tuned by an experienced clinician. Finding stimulation parameters that yield a physiological foot motion is cumbersome and, due to the described time-variant effects, requires repeated manual adaptations of the intensity profile. An obvious escape strategy that is often pursued is to choose larger stimulation intensities and accept exaggerated foot lift. While this strategy provides a certain amount of safety and functionality, it accelerates muscular fatigue and leads to a salient peculiarity in the patient's gait. The challenges described in Section 1.1 can be met in a much more effective and elegant way by the use of feedback control. More precisely, the stimulation parameters can be adjusted automatically to avoid over-stimulation, to delay the onset of fatigue, and to induce the optimal level of foot lift. This requires measurement of the foot motion via, for example, an inertial sensor or a goniometer. When inertial sensors are attached to the shank and foot, the ankle dorsiflexion joint angle can be determined, as describe for example in Seel, Raisch, and Schauer (2014). If only the foot is equipped with an inertial sensor, the foot pitch angle with respect to the horizontal plane is assessable. Both quantities properly describe to which extent the applied FES compensates the foot drop. Despite increasing efforts in the last decades to make closedloop gait neuroprostheses a reality, it is still a challenging task to control paralyzed limbs with FES (Melo et al., 2015). Several control techniques have been proposed. Some respectable results have been obtained for the much simpler case of a sitting or lying subject, i.e. without the tight time constraints and the strong disturbances imposed by gait. For example, Kobravi and Erfanian (2009) and Valtin, Seel, Raisch, and Schauer (2014) proposed a fuzzy controller and an iterative learning controller, respectively, and performed experimental trials with sitting subjects. Hayashibe, Zhang, and Azevedo-Coste (2011) and Benedict and Ruiz (2002) suggested the use of predictive controllers and PID control, respectively, but tested their controllers in simulation studies only. Artificial neural networks were employed by Chang, Chen, Wang, and Kuo (1998) and Chen et al. (2004), who validated the controller in trials with subjects lying on a bed. Besides those simplified in vitro studies, intense efforts have also been made to close the loop on FES during walking. Veltink et al. (2003) used an inertial sensor on the foot to tune an implantable drop foot stimulator such that a desired foot orientation just prior to initial contact was achieved. Negård (2009) proposed run-to-run control of the maximum foot pitch angle occurring during swing phase and tested the controller in trials with a walking drop foot patient. Previously, Mourselas and Granat (2000) had briefly reported similar results obtained with a bend sensor and a fuzzy logic algorithm. While these latter results represent important improvements with respect to all commercially available stimulators, one major shortcoming remains: The entire foot motion is reduced to a single scalar measure, for example a minimum foot clearance with respect to ground or a desired foot pitch angle at initial contact. Obviously, this is a strong simplification of the control problem. As we will demonstrate, conventional stimulation intensity profiles

T. Seel et al. / Control Engineering Practice 48 (2016) 87–97

may yield (for example) a desired maximum foot pitch angle, while at the same time causing too weak or too strong foot lift during the first half of the swing phase, or while using larger intensities than necessary. With respect to overcoming these limitations, Iterative Learning Control (ILC) represents a promising technique. In preliminary experiments with healthy subjects (Nahrstaedt, Schauer, Shalaby, Hesse, & Raisch, 2008), we have previously demonstrated that ILC can be used to control the entire foot pitch angle trajectory during the swing phase, i.e. from the last to the first ground contact (cf. Fig. 1). Controlling the entire trajectory is expected to actually yield a stimulation intensity profile over time that induces a foot motion close to those of healthy walkers, while using only as much FES as needed. In the present contribution, we are finally demonstrating the effectiveness of this approach in chronic drop foot patients walking on a treadmill and on level ground. To the best of our knowledge, this is the first time that automatic feedback control of the entire foot pitch angle trajectory is achieved in walking drop foot patients. Moreover, we are presenting the methods that enabled us to accomplish this long-standing objective with a single inertial sensor attached to the paretic foot in arbitrary orientation. Finally, we explicitly account for the natural variance in swing phase duration by deriving compact guidelines for the design of Iterative Learning Controllers in the presence of variable pass lengths. The remainder of this article is organized as follows. We first introduce methods for realtime assessment of foot motion by means of an inertial sensor in Section 2. The task of FES-based drop foot treatment is formulated as a repetitive trajectory tracking problem and an ILC structure is proposed in Section 3. Based on an analysis of the closed-loop dynamics, design guidelines for the variable-pass-length case are derived and the controller parameters are chosen in Section 4. The proposed control system is then validated in experimental trials with chronic drop foot patients in Section 5.

2. Realtime assessment of foot motion by gyroscopes and accelerometers The motion of the paretic foot is assessed in realtime by means of a single wireless inertial measurement unit (IMU), consisting of a three-dimensional accelerometer and a three-dimensional gyroscope. Although some IMUs also incorporate magnetometers, we refrain from using their readings, since they are well known to be unreliable inside buildings and in the presence of magnetic disturbances (De Vries, Veeger, Baten, & van der Helm, 2009). The IMU is attached to the shoe (or the foot in case of barefoot walking) using adhesive tape, elastic straps, or by putting it inside the shoe or between the shoe tongue and shoelace. Unlike most previous approaches, we assume the orientation and position of the sensor with respect to the foot or shoe to be unknown. This implies maximum freedom of mounting and adds robustness to the methods introduced in Sections 2.1 and 2.2. The task of inertial foot motion assessment would simplify severely if we attached the sensor such that the local2 coordinate axes coincide with anatomical axes of the foot. However, as discussed in our previous work on inertial sensor-based gait analysis (Seel, Raisch, et al., 2014), even surfaces and right angles are rarely found on the human body. Moreover, since most hemiplegic patients can use only one hand to attach the sensor, it is particularly restrictive to demand a certain sensor-to-foot/shoe orientation. For both reasons, we pursue an alternative strategy in which we assume that the 2

By local we denote the coordinates in the moving sensor coordinate system.

89

Fig. 2. Finite state automaton model (phases and transitions) of the gait cycle of one side. During foot-flat phase, the foot rests on the ground, while it has no ground contact during swing phase. By means of a foot-mounted inertial sensor (gray box), all transitions are detected in realtime.

mounting orientation and position of the IMU are both unrestricted and unknown. This means that the sensor may be attached to the shoe or foot (excluding the toes) in an arbitrary position and orientation. 2.1. Gait phase detection The subject's gait is modeled by a finite state automaton as explained in Fig. 2. Unlike most previous approaches (cf. Melo et al., 2015 and references therein), we aim at a high-detail modeling of the gait with four gait phases per side. The measured accelerations a (t ) ∈ 3 and angular rates g (t ) ∈ 3 of the paretic foot are used to detect the gait events toe-off and initial contact, which mark the beginning and end of the swing phase, as well as full- contact and heel-rise, which mark the beginning and end of the foot-flat phase. This is in accordance with standard literature on gait analysis, see e.g. Perry and Burnfield (2010). In the remainder of this contribution, we will call every time period between a heel-rise and the next full-contact a stride. We further denote the stride number by j, and tto, j , tic, j , tfc, j , and thr, j will denote the time instants of the four gait phase transitions of stride j, respectively. IMU-based detection of these transitions is accomplished by threshold-based algorithms that exploit characteristic features in the inertial signals that are illustrated in Fig. 3. The basic idea for each transition is briefly described as follows: The beginning and end of the foot-flat phase are detected by checking whether the

Fig. 3. Mean (solid) and standard deviation (bands) of characteristic inertial measurement signal trajectories over the human gait cycle: acceleration norm

∥ a (t )∥2; angular rate norm ∥ g (t )∥2 ; foot tilt rate Γ (t ) ; horizontal velocity ∥ v ffxy (t )∥2 ; jerk norm

d a (t ) . dt 2

ǁ ǁ

Vertical lines indicate the gait phase transitions full-contact (fc),

heel-rise (hr), toe-off (to), and initial contact (ic). (For assigning the axis labels to the individual curves in this figure, the reader is referred to the colored web version of this paper.)

90

T. Seel et al. / Control Engineering Practice 48 (2016) 87–97

Euclidean norms ∥ a (t )∥2 and ∥ g (t )∥2 enter or leave predefined rest bands around 9.8 m/s2 and 0 rad/s, respectively. During every footflat (ff) phase, the accelerometer readings a (t ) are integrated over time and the resulting vector z^ ff, j is normalized to unit length:

z ff, j ≔

z^ ff, j , ∥ z^ ∥ ff, j

z^ ff, j ≔

2

∫t

t hr, j

a (τ ) dτ . (1)

fc, j

Since gravitational acceleration dominates when the foot is (almost) at rest, z ff, j is (almost) vertical. At each heel-rise thr, j , a strapdown integration (Savage, 1998) of the angular rates is started that yields the rotation matrix Rff (t ). This matrix transforms the local measurement vectors a (t ) and g (t ) (of any time instant t between two foot-flat phases) to the vectors aff (t ) and gff (t ) in the local coordinate system of the preceding foot-flat phase. Projection of the transformed acceleration into the horizontal plane yields the following horizontal acceleration:

affxy (t ) ≔ R ff (t ) a (t ) − (z Tff, j R ff (t ) a (t )) z ff, j .

(2)

The toe-off of stride j is then detected by a major increase in the norm of the horizontal velocity v ffxy (t ) ≔ ∫

t

t hr, j

affxy (τ ) dτ and by an

increase with subsequent decrease of the foot tilt rate Γ (t ) defined as follows:

Γ (t ) ≔

g (t )T ϕ (t ) , ∥ ϕ (t )∥2

ϕ (t ) ≔

∫t

t

g (τ ) dτ .

hr, j

(3)

Here, we exploit that ϕ(t ) accumulates the angular rate on the time interval [thr, t ] and, thus, is approximately aligned with the pitch axis of the foot, regardless of the sensor mounting orientation. Finally, the heel strike is detected by a major decrease in the norm of the horizontal velocity v ffxy and a particularly high norm of the jerk

d a (t ) . dt

Further details of the algorithm, including automatic

adaption of the detection thresholds to gait velocity changes, are described in Seel, Landgraf, and Schauer (2014). 2.2. Measuring foot orientation angles As discussed above, inertial measurement units (IMUs) can be employed to assess ankle dorsiflexion joint angles and foot pitch (orientation) angles. Both measurements would facilitate the design of a closed-loop drop foot stimulator. However, as a result of several discussions with experienced clinicians, foot orientation angles are found to be practically more relevant than ankle joint angles, mainly because the foot orientation is more directly linked to the foot clearance with respect to the ground. Therefore, we only consider the case of foot pitch angle measurement in the remainder of this contribution. The foot pitch angle α is defined in Fig. 4. As discussed previously, the local coordinates of the anatomical axis xfoot are unknown because it is hard to achieve (and restrictive to demand) a mounting orientation that aligns the local coordinate axes with the anatomical axes of the foot. Instead, xfoot is automatically identified from the measurement data of the first stride as follows. We assume that, at least during the first stride, the foot travels mainly into the direction at which it was pointing right before the stride, which is a valid assumption even in paretic gait. We integrate the horizontal foot velocity v ffxy (t ) defined above between heel-rise and full-contact. Exploiting the fact that the foot is resting at full-contact, we remove integration drift and calculate the local coordinates of the posterior-anterior axis xfoot of the foot:

x foot =

x^ foot ∥ x^ foot ∥2

,

x^ foot ≔

∫t

tfc, j hr, j

⎛ ⎞ τ − thr, j ff ⎜ v ffxy (τ ) − v xy (tfc ) ⎟ dτ . tfc, j − thr, j ⎝ ⎠

(4)

Note that, by construction, xfoot is horizontal during stance and its

Fig. 4. The angle α that the posterior–anterior axis xfoot of the foot and its projection into the horizontal plane (dotted lines) confine is a proper measure of foot pitch. Note that the ground is not restricted to be level and that the local coordinate axes of the IMU are not restricted to be aligned with any of the anatomical axes of the foot.

local coordinates do not change with time, since the IMU is rigidly attached to the foot, see Fig. 4 for illustration. By transforming xfoot into the local coordinate system of the preceding foot-flat phase, in which the vertical axis z ff, j is known, we calculate the time-dependent foot orientation angle α˜ in pitch direction:

α˜(t ) ≔

π − ∢(z ff, j , R ff (t ) x foot ) 2

= arcsin (z Tff, j R ff (t ) x foot )

⎡ π π⎤ ∈ ⎢ − , + ⎥. ⎣ 2 2⎦

(5)

Note that the pitch angle α˜ is positive when the toes are above the heel and negative when vice versa. It is important to note that strap-down integration of angular rates is always subject to drift, since, even with proper calibration, the gyroscopes have non-zero bias. Therefore, α˜(t ) also drifts3 between each two foot-flat phases. At every full-contact tfc , however, we can remove the drift from α (t ) on the time interval t ∈ [thr, tfc ] by assuming that neither the gyroscope bias nor the slope of the ground4 changed significantly between the two footflat phases:

α (t ) ≔ α˜(t ) −

t − thr (α˜(tfc ) − α˜(thr )). tfc − thr

(6)

Fig. 5 presents example trajectories of the foot pitch angle from experiments with healthy subjects and with drop foot patients5 who received FES-support with different stimulation intensity settings. While α (t ) hardly exceeds zero degrees when no FES is applied, conservatively large FES intensities result in an exaggerated foot lift, especially during early swing phase. Only if the applied FES is well parametrized, then the induced foot pitch is neither too weak nor exaggerated. Beyond this, it is important to note that, even if FES parameters are found that yield physiological foot motion, this achievement typically vanishes quickly due to muscular fatigue and time-variant muscle tone. This observation leads to the conclusion that automatic foot motion control is required in order to avoid cumbersome manual adjustments and readjustments. 3 For example, with the proposed algorithm and the employed sensor hardware, we found that α˜(tfc ) is typically in the range of 2°. This also serves as a good approximation of the accuracy of the method. 4 Note that changes in the slope of the ground can be detected by comparing the zff -vectors of two subsequent foot-flat phases. It is straightforward to compensate for this effect in (6). Therefore, and for the sake of brevity, only the case of constant ground slope is discussed here. 5 Details of the experimental setup and protocol will be provided in Section 5.

T. Seel et al. / Control Engineering Practice 48 (2016) 87–97

j−1

t0, j ≔ thr, j +

∑ k = max (1, j − 3)

91

tto, k − thr, k − δts ≈ tto, j − δts. min (j − 1, 3)

(8)

In a similar manner, we could anticipate tic, j − δts from previous strides and then stop the stimulation at that moment. However, mainly for safety reasons, FES should be applied at least until the initial contact was detected. Let the trajectory of the stimulation intensity u(t) applied in stride j be denoted by

uj ≔ [u (t0, j ), u (t0, j + ts ), …, u (tic, j )]T ,

Fig. 5. Pitch angle trajectories (averaged over several strides) of a drop foot patient who received zero, conservatively high, and reasonably parametrized FES-support. The gray band represents the standard deviation range observed in healthy gait at the same speed. The characteristics of drop foot gait are well captured by foot orientation measurements.

3. Iterative learning control of FES-induced foot motion Recall that α (t ) is the measured pitch angle of the foot, and that the control objective is to manipulate the stimulation intensity such that the pitch angle trajectory α (t ) during swing phase resembles those of healthy subjects. The force generated by FES increases monotonously with the frequency and the charge (i.e. the product of pulse width and amplitude) of the applied current pulses. Therefore, adjusting the stimulation intensity typically relates to adjusting either (or both) of these quantities. For the sake of brevity, we assume a fixed pulse frequency of 50 Hz and manipulate only the pulse charge. In order to avoid high and narrow pulses as well as low and wide pulses, we implement all stimulation intensity changes in such a manner that pulse width and amplitude are always increased or decreased by the same factor, i.e. their ratio remains constant. Before FES is applied and stimulation intensities are automatically adjusted, it is advisable to identify the maximum tolerated intensity of the subject, i.e. the maximum pulse charges that cause neither discomfort nor pain, and make sure that these values are never exceeded. For the current application, let q(t) be the stimulation intensity (pulse charge) applied via the skin electrodes. We raise q(t) until the subject reports discomfort. Denote the obtained maximum tolerated values by q . Then, define the normalized stimulation intensity

u (t ) ≔

q (t ) , q

(7)

which serves as manipulated variable in the following control problem formulation. 3.1. Formulating drop foot stimulation as a repetitive control task While the paretic foot is loaded, FES-induced joint torques hardly influence the motion of the foot or leg. Therefore, drop foot stimulation aims at influencing the foot motion during swing phase, i.e. from toe-off to initial contact. Since the dynamics of FES-induced movements are slow, the stimulation must start sufficiently early, i.e. typically about 0.2 s before the toe-off. Denote the number of sampling intervals that correspond to this time shift by δ ∈ , i.e. δts ≈ 0.2 s, where ts = 0.02 s is the sampling period. Since drop foot patients walk at a slow constant speed with pre-swing phase durations tto, j − thr, j > δts ∀ j , we can anticipate the toe-off based on previous pre-swing durations and start to apply FES at

(9)

where tto, j , tic, j are the toe-off instant and the initial contact instant of the considered stride j, as defined in Section 2.1. The resulting pitch angle trajectory is then denoted as follows:

αj ≔ [α (t0, j + δts ), α (t0, j + (δ + 1) ts ), …, α (tic, j )]T .

(10)

Even in subjects walking on a treadmill at constant speed, the time duration tic, j − t0, j will vary slightly from stride to stride. This natural variance is found to be even larger in hemiplegic patients (Seel, Landgraf, et al., 2014). To capture this effect mathematically, we introduce the pass length

nj ≔

tic, j − t0, j −δ+1 ts

(11)

of stride j. Then we can write the dimensions of the input and output trajectory of stride j as αj ∈ nj and uj ∈ (nj + δ ). Even in paretic gait, the swing phase duration does not become arbitrarily small or large. We can thus easily find a lower bound n and a (large) upper bound n such that nj ∈ [ n, n − δ ] holds for every stride j. Finally, we define a desired (physiological) pitch angle trajectory rα ∈ n based on data from healthy subjects walking at the same speed, cf. Fig. 5. From a medical point of view it is most important to ensure a clear heel-strike, i.e. a positive foot pitch angle at initial contact. Therefore, we define rα such that its last n − n sample values are larger than 10°. This completes a repetitive trajectory tracking task in which the input uj must be chosen such that αj ≈ rα (1: nj ) in each stride.6 The large (translational and rotational) acceleration and deceleration of the shank that occur during swing phase impose disturbances on the considered plant dynamics. Due to increased muscle tone (related to spasticity), there is furthermore an additional torque acting on the ankle joint of many drop foot patients. This torque, as well as the residual voluntary muscle activity of the patient, also act as disturbances on the previously defined repetitive control task. Finally, note that the amount of joint torque that can be generated by FES is typically very limited. Therefore, these disturbances represent major challenges. From the preceding discussion we conclude that peroneal stimulation via surface electrodes during gait requires solving a repetitive control task with variable pass length and with large delays and disturbances. As discussed in Section 1.1, many of the system parameters and disturbances vary with time – some of them within minutes, others occasionally within less than ten seconds. A control method is required that exploits the repetitiveness of gait and quickly adjusts the FES from stride to stride such that the variability is compensated and the desired pitch angle trajectory is achieved in each stride (up to the natural strideto-stride variance of healthy gait). This task perfectly fits the framework of Iterative Learning Control (ILC), which has originally been developed to improve the control of robots and similar mechanical systems (Arimoto, Kawamura, & Miyazaki, 1984). Since then, the framework has been widely extended by, for example, 6 where a (1: nj ) denotes the vector that contains the first nj elements of the vector a

92

T. Seel et al. / Control Engineering Practice 48 (2016) 87–97

Owens and Hätönen (2005), Galkowski et al. (2003), and Paszke, Rogers, and Galkowski (2010), to name only a few. Within the last ten years, ILC has also been employed successfully for biomedical applications, see for example Longman and Mombaur (2006), Freeman, Rogers, Hughes, Burridge, and Meadmore (2012), Wang, Dassau, and Doyle (2010), Klauer, Schauer, and Raisch (2010), and Seel, Weber, Affeld, and Schauer (2013). In Section 3.2, we use a classical learning law and extend it to the present (non-classical) case of variable pass length. 3.2. Iterative learning control law As pointed out above, the overall dynamic delay of FES-induced foot motion is approximately 0.2 s, which is longer than a quarter of the typical swing phase duration tic − tto ≈ 0.7 s. Moreover, gaining useful model knowledge requires patient-burdening identification procedures. Therefore, instantaneous feedback (i.e. using current measurement information to adjust stimulation intensities) hardly reaches closed-loop reaction times below 0.5 s, and is practically useless. Instead, the stimulation intensities for each stride must be chosen primarily based on measurement information from previous strides. In terms of ILC theory this means that we neither employ current-iteration tracking error ILC nor design methods that rely on a (precise) system model. Since the pass length nj of stride j is not known before the initial contact of stride j occurs, we must prepare a full-length controller output trajectory u¯ , j ∈ n for each stride j:

u¯ j ≔ [u (t0, j ), u (t0, j + ts ), …, u (t0, j + (n − 1) ts )]T ,

(12)

although only the first nj + δ samples of that trajectory (i.e. exactly uj ) will actually be applied to the system. In the first stride j¼ 0, when no measurement information from previous strides is available, a conservative strategy is advisable: We set u¯ 0 to a trapezoidal profile with an amplitude of 0.9 (corresponding to 90% of the patient's maximum tolerated intensity) and a considerably short rise time. This assures a safe swing phase with sufficient (but most likely exaggerated) foot lift. At the beginning of each following stride, we determine the element-wise deviation between the measured angle trajectory αj and the first nj samples of the respective reference trajectory rα to calculate the measured tracking error ej ∈ nj as follows:

ej ≔ rα (1: nj ) − αj.

(13)

For the last (n − nj ) samples of the reference trajectory there exists no corresponding measurement information, since the trial actually ended after nj sample periods. The incomplete error information can nevertheless be used to adjust the controller output trajectory u¯ j , by using the following modified version of a classical ILC learning law:

⎛ ⎛ ⎡ ej ⎤ ⎞ ⎞ ⎥ ⎟⎟ ⎟⎟, u¯ j + 1 = sat10 ⎜⎜ Q ⎜⎜ u¯ j + L ⎢ ⎣⎢ 0(n − nj ) ⎥⎦ ⎠ ⎠ ⎝ ⎝ where L ∈

n × n

(14)

is an adjustable learning gain matrix, 0(n − nj ) is a

zero column vector of length (n − nj ), and satba (·) facilitates element-wise saturation to the interval [a, b]. Moreover, Q ∈ n × n is a symmetric matrix with Toeplitz structure containing the Markov parameters of a low pass filter (2nd order, Butterworth) with cutoff frequency denoted by fQ . Multiplying a trajectory vector by Q corresponds to applying a non-causal (zero-phase) low-pass filter to the trajectory, see for example Elci, Longman, Phan, Juang, and Ugoletti (2002) and Gustafsson (1996). Thereby, we avoid the discomfort that is usually associated with sudden large increases in stimulation intensity. Moreover, this improves the robustness of the ILC by restricting the learning process to the low frequency

range in which at least some model knowledge is available, see for example Bristow, Tharayil, and Alleyne (2006). The rationale behind this learning approach is as follows: When a certain section of an angle trajectory is lower or higher than it should be, the update law (14) modifies a corresponding section of the controller output trajectory in such a way that the output is increased or decreased, respectively. Note that the large delay in the system dynamics is partially compensated by the time shift δts in the trajectory definitions (9). If, for example, the pitch angle is too low during the first five samples of the swing phase in stride j, then the controller increases the first five entries of u¯ j + 1, which will be applied δ samples before the swing phase of the next stride, i.e. early enough to correct the observed control deviation. If the adjustment was not sufficient, the deviation remains and the controller output will be adjusted again, i.e. the employed learning law has integral action. By implementing the control output saturation directly into the learning law (14), we prevent integrator windup effects that would otherwise occur whenever a sample value of the stimulation intensity trajectory leaves the admissible interval [0, 1].

4. Learning dynamics of ILC systems with variable pass length The closed loop system described in Section 3 belongs to a special class of iterative learning control systems with variable pass length that was introduced in Seel, Schauer, and Raisch (2011). It is characterized by the pass length varying from pass to pass in an unpredictable manner, i.e. not a single element of the sequence of pass lengths {nj } is known in advance. However, a possibly very small lower bound n and a possibly very large upper bound n are assumed to be known. Consequently, a desired output of full length r ∈ n must be defined, and an input u¯ j ∈ n of full length must be available for each pass j. In the remainder of this section, we briefly discuss the learning dynamics of this class of systems. 4.1. Linear system dynamics Consider a causal, linear, discrete-time process with Markov parameters pi , i = 1, 2, … and consider repetitive control of its output yj ∈ nj with reference r ∈ n in trials with variable pass length nj ∈ [ n, n ]. The dynamics of the process shall be captured by the following lifted-system equation:

⎡ ej ⎤ ⎢ ⎥ = r − (Pu¯ j + v), ⎢⎣ e^j ⎥⎦

⎡p 0 ⎢ 1 p p1 P=⎢ 2 ⎢ ⋮ ⋮ ⎢ ⎣ pn¯ pn¯ − 1

⋯ 0⎤ ⎥ ⋯ 0⎥ . ⋱ ⋮⎥ ⎥ ⋯ p1⎦

(15)

where P ∈ n × n is the lifted-system matrix of the process and v ∈ n is a bounded, unknown, iteration-invariant disturbance. Since each trial ends (unpredictably) after nj sample instants, the ^ ∈ (n − nj ) is actually measured control deviation is ej ∈ nj , while e j only a hypothetical error that would occur on the last n − nj sample instants if the trial was of full length. Therefore, only ej ^ ) can be used for learning. As discussed above, the (but not e j

learning process can be restricted to the samples for which measurement information is available by inserting zeros on the last n − nj samples of the error information vector that is used for learning. We consider the following (fairly general) input update law:

⎡ ej ⎤ ⎞ ⎛ ⎡ ej ⎤ ⎞ ⎛ ⎥ ⎟⎟ = Q ⎜⎜ u¯ j + LH nj ⎢ ⎥ ⎟⎟, u¯ j + 1 = Q ⎜⎜ u¯ j + L ⎢ ⎢⎣ 0(n − nj ) ⎥⎦ ⎠ ⎢⎣ e^j ⎥⎦ ⎠ ⎝ ⎝

(16)

T. Seel et al. / Control Engineering Practice 48 (2016) 87–97

where Q, L ∈ n × n are the Q-filter and the learning gain matrix, respectively, and the block-diagonal matrix Hnj = blockdiag {Inj × nj , 0(n − nj )×(n − nj ) } of identity and zero square matrices, respectively, is introduced for convenience. 4.2. Closed-loop learning dynamics In this subsection, we briefly analyze the learning (error reduction) dynamics in order to derive guidelines for the design of the controller parameters L, Q of the proposed ILC in the presence of variable pass length. To this end, we first derive the closed-loop dynamics by combining the plant dynamics (15) and the learning law (16):

⎡ ej + 1⎤ ⎡ ej ⎤ ⎢ ⎥ = PQP−1(In × n − PLH nj ) ⎢ ⎥ + (In × n − PQP−1)(r − v). ⎢⎣ e^j + 1⎥⎦ ⎢⎣ e^j ⎥⎦

⎡ γ1 γ2 ⎤ ⎢⎣ γ γ ⎥⎦ ≔ I − PL, 3 4

γ1 ∈ nj × nj ,

⎡ ϵ1 ϵ2 ⎤ −1 ⎣⎢ ϵ3 ϵ4 ⎥⎦ ≔ I − PQP ,

γ2 ∈ nj ×(n − nj ) ,

ϵ1 ∈ nj × nj ,

(19)

This allows us to rewrite (17) with identity matrices I and zero matrices 0 of appropriate dimensions:

⎡ ej + 1⎤ ⎡ I − ϵ − ϵ ⎤ ⎡ γ 0 ⎤ ⎡ ej ⎤ ⎡ ϵ ϵ ⎤ 1 2 1 ⎢ ⎥=⎢ ⎥ ⎢ ⎥ + 1 2 (r − v). ⎥⎢ ⎢⎣ e^j + 1⎥⎦ ⎣ − ϵ3 I − ϵ4 ⎦ ⎢⎣ γ3 I ⎥⎦ ⎢⎣ e^j ⎥⎦ ⎢⎣ ϵ3 ϵ4 ⎥⎦

Δp, j ≔ 1 −

⎡ γ 0 ⎤ ⎡ ej ⎤ 1 ⎥ ⎢ ⎥ + [ϵ1 ϵ2 ](r − v) − ϵ2 ] ⎢ ⎢⎣ γ3 I ⎥⎦ ⎢⎣ e^j ⎥⎦

p

∥ ej ∥p ((I − ϵ1) γ1 − ϵ2 γ3 ) ej − ϵ2 e^j + [ϵ1 ϵ2 ](r − v)

=1−

∥ ej ∥p

p

,

,

(21)

where ∥·∥p denotes any vector p-norm and its induced matrix norm.7 Obviously, Δp is between zero and one if the learning step reduces the error p-norm on the first nj samples, and it is negative if that norm increases. Note that any p-norm is sub-multiplicative8 and that the pnorm of a matrix is not smaller than the p-norm of any of its submatrices. Therefore, it is helpful to introduce the following scalar convergence indicator γ˜ and residual indicator ϵ˜ :

γ˜≔∥ PQP−1(I − PL)∥p ,

ϵ˜≔∥ I − PQP−1 ∥p ,

7 8

γ˜ ≥ ∥(I − ϵ1) γ1 − ϵ2 γ3 ∥p ,

ϵ˜ ≥ ∥[ ϵ1 ϵ2 ]∥p ≥ ∥ ϵ2 ∥p .

i.e. ∥ A ∥p ≔sup {∥ Ax ∥p , ∥ x ∥p = 1} i.e. ∥ ABx ∥p ≤ ∥ A ∥p ∥ B ∥p ∥ x ∥p ∀ A, B, x

γ˜ ∥ ej ∥p + ϵ˜(∥ e^j ∥p + ∥ r − v ∥p ) . ∥ ej ∥p

(22)

(23)

(24)

This approximation is certainly very conservative. However, it allows us to derive a few compact statements on the learning dynamics: If perfect model knowledge was available, then the choice L = P −1 and Q = I would assure γ˜ = ϵ˜ = 0 and thus a learning progress of 1, i.e. reduction of the first nj sample values of the tracking error to zero in each trial j. In the more realistic case that a convergence indicator 0 < γ˜ < 1 can be assured, the following monotonous error reduction property is given:

ϵ˜ (∥ e^j ∥p + ∥ r − v ∥p ) 1 − γ˜

Δp, j > 0.

⇒

(25)

This means that there is positive learning progress (at least) until the norm ∥ ej ∥p of the tracking error falls below a certain threshold that is small if the residual indicator ϵ˜ is small. This result is in accordance with the well-known result from classical ILC theory that a non-zero steady state error is obtained if a Q-filter Q ≠ I is used. From the preceding discussion we conclude the following control design guidelines for ILC systems of the form (15), (16): The matrix Q should be chosen such that a small residual indicator ϵ˜ is achieved for the given system dynamics P , while the learning gain matrix L should be chosen such that a small convergence indicator γ˜ is achieved. Beyond the results discussed here, we have analyzed the special case Q = I in Guth, Seel, and Raisch (2013), as well as dynamics ^ T ]T in Seel et al. (2011). For of the maximum-pass-length error [eT e j

(20)

Now consider the input update at trial j and note that the tracking error on the first nj sample instants is ej = r (1: nj ) − yj . We want to quantify how effectively the update (16) uses this information to improve the control signal. To this end, we define the learning progress indicator Δp, j as the relative reduction of the tracking error on the first nj sample instants from trial j to trial j + 1:

[I − ϵ1

Δp, j ≥ 1 −

(17)

(18)

ϵ2 ∈ nj ×(n − nj ) .

This yields the following lower bound (worst case) of the learning progress:

∥ ej ∥p >

Note that, in general, ej + 1 on the left-hand side and ej on the righthand side do not have the same dimensions. For convenience, define the following sub-matrices:

93

j

this application-oriented contribution, we do not elaborate on any of these aspects. Instead, we apply the derived design guidelines for L and Q to the present application of FES-based foot motion control during walking and evaluate the learning dynamics experimentally in the next section. 4.3. ILC design for FES-based foot motion control Controlling foot motion by functional electrical stimulation during the swing phase of gait represents a classic example of an ILC task with variable pass length (Seel et al., 2011). To apply the derived design criteria, a lifted model of the plant dynamics is needed. We have performed experiments in subjects sitting on a table with relaxed shank and foot. The subject's shank muscles were stimulated with an intensity profile typical for walking support of drop foot patients. The dynamics of foot pitch (measured in radians) induced by the electrical stimulation (normalized to tolerated maximum) were identified using least-squares system identification methods. The obtained transfer function

G (z ) =

1.0494z − 1.0174 −10 z , z − 0.9816 with sampling rate fs = 50 Hz,

(26)

roughly approximates the dynamics from the stimulation intensity u(t) to the measured foot pitch angle α (t ) defined in Section 2. The lifted system matrix P is calculated from the Markov parameters of G(z) as described above. We now use this model to find suitable values for the learning gain matrix L and the Q-filter cutoff frequency fQ of the iterative learning control scheme defined in Section 3. Since a large portion of the delay is already compensated by the phase lead δ and reliable detailed model knowledge is not available, we employ the standard approach of a diagonal learning gain

94

T. Seel et al. / Control Engineering Practice 48 (2016) 87–97

ϵ˜≤ 2 Hz≔∥ I − PQP−1 ∥≤ 2 Hz ,

(28)

ε

Fig. 6 shows that ϵ˜≤2 Hz grows as the cutoff frequency fQ of the Q-filter is reduced. However, as discussed before, a small fQ improves robustness with respect to model uncertainties by restricting the learning process to a smaller (low) frequency range. To balance both aspects, we choose a cutoff frequency fQ = 5 Hz that is slightly larger than the highest frequency 2 Hz we would like to follow (regarding r ) or reject (regarding v ). With (25) and λ = 0.5, this implies that the tracking error ej on the first nj samples is reduced in each learning step at least until it falls below 0.08 ∥ r − v ∥2. In Section 5, we evaluate experimentally how 1 − 0.4 conservative this statement still is and how quickly the tracking error is reduced in practice.

5. Experimental validation

Fig. 6. Convergence indicator γ˜ and residual indicator ϵ˜ over control design parameters.

matrix, i.e. L ≔ λIn × n . We then analyze error reduction in the Euclidean norm ∥·∥2. Fig. 6 shows the convergence indicator γ˜ plotted against the diagonal learning gain λ for Q-filter cutoff frequencies fQ from the practically relevant interval9 fQ ∈ [1, 25] Hz . The monotonous error reduction property (25) is given if λ ∈ [0, 1.5], while the best convergence is predicted for λ ∈ [0.5, 1]. Very similar γ˜ -values are obtained for all fQ ∈ [1, 25] Hz . Moreover, we find ∥ PQP −1 ∥2 ≈ 1 ∀ fQ ∈ [1, 25] Hz . This is not surprising, since multiplication by Q leaves low-frequency signal vectors almost unchanged, while it yields almost zero-norm output for high-frequency inputs. For the same reason, the residual indicator ϵ˜ = ∥ I − PQP −1 ∥2 is almost one for all fQ ∈ [1, 24] Hz . As (25) and (15) indicate, this means that high-frequency references r and disturbances v lead to high-frequency portions in the tracking error that will not be removed by the iterative learning controller if fQ < 25 Hz, i.e. Q ≠ I . In that sense, using ϵ˜ in the left-hand side of (25) is highly conservative. From Sections 2 and 3, we conclude that a trial (i.e. a swing phase) is typically shorter than one second and that both the defined reference and the large disturbance imposed by the shank motion contain only frequencies below 2 Hz. In order to obtain a less conservative result for the case of low-frequency signals (r − v), we define the frequency-weighted matrix norm

∥ A ∥≤ 2 Hz ≔ ∥ AQ 2 Hz ∥2 ,

(27)

where Q 2 Hz is a low-pass lifted matrix with cutoff frequency 2 Hz and unity Euclidean norm ∥ Q 2 Hz ∥2 = 1. Recall that the induced Euclidean matrix norm is the largest gain by which the Euclidean norm of a vector increases when multiplied with the matrix. Following this definition, the frequency-weighted matrix norm ∥·∥≤2 Hz yields the largest gain by which the Euclidean norm of a low-frequency vector increases when multiplied with the matrix.10 Therefore, we obtain a less conservative result for the case of low-frequency signals (r − v) by using the frequencyweighted residual indicator 9 Note that for the given sampling frequency fs = 50 Hz , the Nyquist frequency is 25 Hz . 10 ^, ∥ x ^ ∥2 = 1} more precisely: ∥ A ∥≤ 2 Hz = sup {∥ Ax ∥2 , x = Q 2 Hz x

In experimental trials with drop foot patients, we now evaluate the effectiveness of the designed ILC scheme in combination with the proposed foot pitch angle measurement. The six patients that were recruited for these trials are ambulatory, aged 50–70, BMI 20–27, at least three months post-stroke, and suffer from a drop foot syndrome in combination with at most moderately increased muscle tone (hypertonia) of the leg musculature. Some of them use a walking stick or an ankle-foot orthosis in everyday walking. All of them have received FES of the respective muscles before (at least three sessions of at least 30 min). The patients are asked to walk on a treadmill and on level ground at constant, self-selected speed for 5–20 min, depending on their strength and abilities. Informed consent of the patients was obtained and the trials have been approved by the ethics committee of Charité Universitätsmedizin Berlin. A wireless inertial sensor is attached to the paretic foot and two FES electrodes are placed on the shank as depicted in Fig. 1. The measured accelerations and angular rates are used to calculate the current gait phase as well as the foot pitch angle α (t ) in realtime. For the first stride j¼0, we set u¯ 0 to a conventional trapezoidal input profile. After each stride j ≥ 0, during the short period of time for which the heel and toes of the paretic foot are on the ground, the controller uses the measured pitch angle trajectory αj to adjust the stimulation intensity profile u¯ j + 1 of the subsequent stride automatically according to the update law (14). As explained in Section 3.1, we choose a desired pitch angle trajectory based on healthy gait data. The resulting reference signal (a sinusoidal followed by a linear decrease) is further fine-tuned by an experienced clinician during the experiments to provide optimal support to the individual patient. Throughout all trials, the iterative learning controllers consistently adjusted the stimulation intensity in a way that the foot motion resembled the desired physiological motion. Fig. 7 shows results from a representative trial in which the amplitude of the initial trapezoidal input u¯ 0 was chosen just large enough to allow the patient to take a step. The deviation between the resulting pitch angle trajectory α0 and the desired pitch angle trajectory is clearly larger than twenty degrees during almost the entire swing phase. Before the next stride, this measured deviation is used by the ILC to raise the stimulation intensity profile in such a way that the next pitch angle trajectory α1 resembles the desired one. Due to the natural fluctuation of many FES and gait parameters, tracking accuracies below the few-degrees level are not achieved. However, as demonstrated in Section 2.2, even the pitch angle trajectories of healthy subjects vary by up to seven degrees (standard deviation). As illustrated in Fig. 7, this physiological range is reached by the ILC in one learning step. By repeated adaptation, the

T. Seel et al. / Control Engineering Practice 48 (2016) 87–97

95

Fig. 7. (a) Foot pitch angle tracking in a chronic drop foot patient. Starting from conventional trapezoidal shape (first stride, black lines), the stimulation intensity profile is adapted from stride to stride (blue, green, orange, red) in order to generate the desired foot pitch angle trajectory (black circles) during swing phase. On each line, dots mark heel-rise (left) and initial contact (right) of each stride. (b) The root-mean-square of the first n = 30 sample values of the tracking error is quickly reduced to the range of natural variance observed in healthy gait. (For interpretation of the references to color in this figure caption, the reader is referred to the web version of this paper.)

Fig. 8. (a) Foot pitch angle tracking in a chronic drop foot patient. Starting from conservatively high values (first stride, black lines), the stimulation intensity profile is adapted from stride to stride (blue, green, orange, red) in order to generate the desired foot pitch angle trajectory (black circles) during swing phase. On each line, dots mark heel-rise (left) and initial contact (right) of each stride. (b) The root-mean-square of the first n = 30 sample values of the tracking error is quickly reduced to the range of natural variance observed in healthy gait. (For interpretation of the references to color in this figure caption, the reader is referred to the web version of this paper.)

physiological foot motion is maintained during the following strides even if the patient modifies his/her walking style11 or if the muscles fatigue. In the latter case, for example, the intensity is automatically raised as soon as the foot lift becomes weaker. As discussed in Section 1.2, the amplitude of a conventional

11

e.g. by increasing knee flexion and decreasing circumduction

trapezoidal stimulation profile is often chosen larger than necessary to avoid the need for re-adjustments. In Fig. 5 it was illustrated that this can lead to equally large deviations from physiological foot motion as applying no FES at all. Therefore, we also evaluate the ability of the controller to reduce the intensity if required. Results are presented in Fig. 8. The conservatively large intensity profile u¯ 0 leads to a foot motion with exaggerated foot pitch α0 . Hence, during the subsequent strides, the proposed iterative learning controller

96

T. Seel et al. / Control Engineering Practice 48 (2016) 87–97

gradually reduces the input profile to the level of support that is actually needed. Evidently, this specific patient did not need any stimulation in the second half of swing phase to achieve an initial contact with large positive foot pitch. Similar series of trials with the five other drop foot patients yield similar results and confirm all of the above findings. Since this contribution only aims at providing a proof of concept rather than a clinical study, these results are not analyzed in detail. It is, however, important to note that the iterative learning process leads to highly individualized FES parameters that may vary largely from one patient to the next (and often from day to day within one patient), even if the same FES parameterization and reference trajectories are used. For example, some patients require larger intensities during pre-swing than during swing phase, while others need the opposite. That such individual requirements are automatically determined and satisfied is a major achievement of the proposed feedback-controlled drop foot neuroprosthesis.

6. Conclusions and further research Automatic control of neuroprostheses for FES-based treatment of the drop foot syndrome via surface electrodes has been considered. Benefits of a closed-loop approach were discussed, as well as the challenges arising from large delays and time-variant disturbances. We designed and implemented an iterative learning control scheme for the pitch motion of the paretic foot during swing phase. Experimental trials with chronic drop foot patients demonstrated that the ILC quickly tracks a desired pitch angle trajectory, when starting from a conventional trapezoidal FES intensity profile. It was found that the intensity profiles required to induce physiological foot motion are typically not trapezoidal and vary with time as well as from patient to patient. Unlike existing drop foot stimulators, the proposed feedback-controlled system adapts the electrical stimulation to the needs of a specific patient on a specific day with a specific electrode placement and maintains the desired pitch angle trajectory even in the presence of disturbances. Further research will focus on the drawback that precise electrode placement is required to obtain a straight foot lift. In this context, we will aim at combining the present achievements with recent results in peroneal stimulation via array electrodes. Moreover, in subjects with low FES tolerance, we will investigate the effect of input saturation and anti-windup schemes on the controller performance. Finally, variable gait velocity and walking stairs will be a focus of future research.

Acknowledgments We would like to express our deep gratitude to the patients who participated in the trials. Furthermore, the valuable contribution and skillful support of Markus Valtin, Boris Henckell, Mirjana Ruppel, and Daniel Laidig are highly acknowledged. This work was conducted within the research project APeroStim, which is supported by the German Federal Ministry of Education and Research (FKZ 01EZ1204B).

References Arimoto, S., Kawamura, S., & Miyazaki, F. (1984). Bettering operation of robots by learning. Journal of Robotic Systems, 1(2), 123–140. Benedict, G. A., & Ruiz, V. F. (2002). A portable gait analysis and correction system using a simple event detection method. In Proceedings of IEEE international conference on systems, man and cybernetics. Bristow, D. A., Tharayil, M., & Alleyne, A. G. (2006). A survey of iterative learning control. IEEE Control Systems Magazine, 26(3), 69–114.

Chang, M.-H., Chen, Y.-L., Wang, K.-C., & Kuo, T.-S. (1998). Implementation of fuzzy control over FES-assisted locomotion for CVA patients. In Proceedings of 20th annual international conference of the IEEE Engineering in Medicine and Biology Society. Vol. 20: Biomedical engineering towards the year 2000 and beyond (Vol. 5, pp. 2721–2723). Chen, Y.-L., Chen, S.-C., Chen, W.-L., Hsiao, C.-C., Kuo, T.-S., & Lai, J.-S. (2004). Neural network and fuzzy control in FES-assisted locomotion for the hemiplegic. Journal of Medical Engineering & Technology, 28, 32–38. De Vries, W. H., Veeger, H. E., Baten, C. T., & van der Helm, F. C. (2009). Magnetic distortion in motion labs, implications for validating inertial magnetic sensors. Gait & Posture, 29, 535–541. Elci, H., Longman, R. W., Phan, M. Q., Juang, J. N., & Ugoletti, R. (2002). Simple learning control made practical by zero-phase filtering: Applications to robotics. IEEE Transactions on Circuits and Systems I: Fundamental Theory and Applications, 49(6), 753–767. Freeman, C. T., Rogers, E., Hughes, A.-M., Burridge, J. H., & Meadmore, K. L. (2012). Iterative learning control of FES applied to the upper extremity for rehabilitation. IEEE Control Systems Magazine, 32(1), 18–43. Galkowski, K., Lam, J., Rogers, E., Xu, S., Sulikowski, B., Paszke, W., et al. (2003). LMI based stability analysis and robust controller design for discrete linear repetitive processes. International Journal of Robust and Nonlinear Control, 13, 1195–1211. Gustafsson, F. (1996). Determining the initial states in forward–backward filtering. IEEE Transactions on Signal Processing, 44(4), 988–992. Guth, M., Seel, T., & Raisch, J. (2013). Iterative learning control with variable pass length applied to trajectory tracking on a crane with output constraints. In Proceedings of the 52nd IEEE conference on decision and control (CDC) (pp. 6676– 6681). Hayashibe, M., Zhang, Q., & Azevedo-Coste, C. (2011). Dual predictive control of electrically stimulated muscle using biofeedback for drop foot correction. In Proceedings of IEEE/RSJ international conference on intelligent robots and systems (IROS) (pp. 1731–1736). Klauer, C., Schauer, T., & Raisch, J. (2010). High performance motion control by neuro-muscular electrical stimulation applied to the upper-limb. In Proceedings of the 15th annual IFESS conference and 10th Vienna international workshop on FES (pp. 318–320). Kobravi, H.-R., & Erfanian, A. (2009). Decentralized adaptive robust control based on sliding mode and nonlinear compensator for the control of ankle movement using functional electrical stimulation of agonist-antagonist muscles. Journal of Neural Engineering, 6. Longman, R. W., & Mombaur, K. D. (2006). Investigating the use of iterative learning control and repetitive control to implement periodic gaits. Lecture notes in control and information sciences (Vol. 340, pp. 189–218). Lynch, C. L., & Popovic, M. R. (2008). Functional electrical stimulation. IEEE Control Systems, 28(2), 40–50. Lyons, G. M., Sinkjaer, T., Burridge, J. H., & Wilcox, D. J. (2002). A review of portable FES-based neural orthoses for the correction of drop foot. IEEE Transactions on Neural Systems and Rehabilitation Engineering, 10(4), 260–279. Melo, P. L., Silva, M. T., Martins, J. M., & Newman, D. J. (2015). Technical developments of functional electrical stimulation to correct drop foot: Sensing, actuation and control strategies. Clinical Biomechanics, 30(2), 101–113. Mourselas, N., & Granat, M. H. (2000). Correction of drop foot using a fuzzy logic controlled miniature stimulator. In Proceedings of the 5th annual conference of the international functional electrical stimulation society (pp. 140–141). Nahrstaedt, H., Schauer, T., Shalaby, R., Hesse, S., & Raisch, J. (2008). Automatic control of a drop-foot stimulator based on angle measurement using bioimpedance. Artificial Organs, 32(8), 649–654. Negård, N.-O. (2009). Controlled FES-assisted gait training for hemiplegic stroke patients based on inertial sensors (Doctoral thesis). TU Berlin. Owens, D. H., & Hätönen, J. (2005). Iterative learning control – an optimization paradigm. Annual Reviews in Control, 29, 57–70. Paszke, W., Rogers, E., & Galkowski, K. (2010). On the design of ILC schemes for finite frequency range tracking specifications. In Proceedings of the IEEE conference on decision & control. Peckham, P. H., & Knutson, J. S. (2005). Functional electrical stimulation for neuromuscular applications. Annual Review of Biomedical Engineering, 7, 327–360. Perry, J., & Burnfield, J. M. (2010). Gait analysis: Normal and pathological function. SLACK Incorporated, Thorofare, New Jersey. Ring, H., Treger, I., Gruendlinger, L., & Hausdorff, J. M. (2009). Neuroprosthesis for footdrop compared with an ankle–foot orthosis: Effects on postural control during walking. Journal of Stroke and Cerebrovascular Diseases, 18(1), 41–47. Savage, P. G. (1998). Strapdown inertial navigation integration algorithm design – part 1: Attitude algorithms. Journal of Guidance, Control, and Dynamics, 21(1), 19–28. Seel, T., Landgraf, L., & Schauer, T. (2014). Online gait phase detection with automatic adaption to gait velocity changes using accelerometers and gyroscopes. Biomedical Engineering/Biomedizinische Techik, 59(S1), 795–798. Seel, T., Schauer, T., & Raisch, J. (2011). Iterative learning control for variable pass length systems. In Proceedings of the 18th IFAC World Congress (pp. 4880–4885). Seel, T., Raisch, J., & Schauer, T. (2014). IMU-based joint angle measurement for gait analysis. Sensors, 14(4), 6891–6909. Seel, T., Weber, S., Affeld, K., & Schauer, T. (2013). Iterative learning cascade control of continuous noninvasive blood pressure measurement. In Proceedings of the IEEE international conference on systems, man, and cybernetics, Manchester, UK (pp. 2207–2212). Valtin, M., Seel, T., Raisch, J., & Schauer, T. (2014). Iterative learning control of drop

T. Seel et al. / Control Engineering Practice 48 (2016) 87–97

foot stimulation with array electrodes for selective muscle activation. In Proceedings of the 19th IFAC World Congress (pp. 6586–6592). Veltink, P. H., Slycke, P., Hemssems, J., Buschman, R., Bultstra, G., & Hermens, H. (2003). Three dimensional inertial sensing of foot movements for automatic tuning of a two-channel implantable drop-foot stimulator. Medical Engineering

97

& Physics, 25(1), 21–28. Wang, Y., Dassau, E., & Doyle, F. J., 3rd (2010). Closed-loop control of artificial pancreatic beta-cell in type 1 diabetes mellitus using model predictive iterative learning control. IEEE Transactions on Biomedical Engineering, 57(2), 211–219.