Human gait recognition via deterministic learning

Neural Networks 35 (2012) 92–102

Contents lists available at SciVerse ScienceDirect

Neural Networks journal homepage: www.elsevier.com/locate/neunet

Human gait recognition via deterministic learning✩ Wei Zeng a,b , Cong Wang a,∗ a

College of Automation Science and Engineering, South China University of Technology, Guangzhou 510641, China

b

School of Physics and Mechanical & Electrical Engineering, Longyan University, Longyan 364000, China

article

info

Article history: Received 19 January 2012 Received in revised form 24 June 2012 Accepted 30 July 2012 Keywords: Human gait recognition Dynamical pattern recognition Deterministic learning Feature extraction RBF neural network Side silhouette lower limb joint angle and angular velocity

abstract Recognition of temporal/dynamical patterns is among the most difficult pattern recognition tasks. Human gait recognition is a typical difficulty in the area of dynamical pattern recognition. It classifies and identifies individuals by their time-varying gait signature data. Recently, a new dynamical pattern recognition method based on deterministic learning theory was presented, in which a time-varying dynamical pattern can be effectively represented in a time-invariant manner and can be rapidly recognized. In this paper, we present a new model-based approach for human gait recognition via the aforementioned method, specifically for recognizing people by gait. The approach consists of two phases: a training (learning) phase and a test (recognition) phase. In the training phase, side silhouette lower limb joint angles and angular velocities are selected as gait features. A five-link biped model for human gait locomotion is employed to demonstrate that functions containing joint angle and angular velocity state vectors characterize the gait system dynamics. Due to the quasi-periodic and symmetrical characteristics of human gait, the gait system dynamics can be simplified to be described by functions of joint angles and angular velocities of one side of the human body, thus the feature dimension is effectively reduced. Locally-accurate identification of the gait system dynamics is achieved by using radial basis function (RBF) neural networks (NNs) through deterministic learning. The obtained knowledge of the approximated gait system dynamics is stored in constant RBF networks. A gait signature is then derived from the extracted gait system dynamics along the phase portrait of joint angles versus angular velocities. A bank of estimators is constructed using constant RBF networks to represent the training gait patterns. In the test phase, by comparing the set of estimators with the test gait pattern, a set of recognition errors are generated, and the average L1 norms of the errors are taken as the similarity measure between the dynamics of the training gait patterns and the dynamics of the test gait pattern. Therefore, the test gait pattern similar to one of the training gait patterns can be rapidly recognized according to the smallest error principle. Finally, experiments are carried out on the NLPR and UCSD gait databases to demonstrate the effectiveness of the proposed approach. © 2012 Elsevier Ltd. All rights reserved.

1. Introduction Recognizing people by gait is a subfield of biometrics and is gaining increasing attention (Han & Bhanu, 2006; Huang & Boulgouris, 2008; Huang, Harris, & Nixon, 1999; Lee & Grimson, 2002; Murase & Sakai, 1996; Yoo & Nixon, 2011; Zhang, Vogler, & Metaxas, 2007). Gait offers the potential for vision-based recognition at a distance and is difficult to conceal or imitate the motion of an individual’s walking (Winter, 1990).

✩ This work was supported by the National Basic Research Program (973) of China under grant (2007CB311005), and by the National Natural Science Foundation of China under grants (90816028, 60934001, 61004065). ∗ Corresponding author. Tel.: +86 20 87114615; fax: +86 20 87114612. E-mail addresses: [email protected] (W. Zeng), [email protected] (C. Wang).

0893-6080/$ – see front matter © 2012 Elsevier Ltd. All rights reserved. doi:10.1016/j.neunet.2012.07.012

Normal human walking may be defined as a method of locomotion which is a periodic or periodic-like process. Gait describes the manner or the style of walking and is a complex form of human locomotion while moving around in an orderly and stable manner (Yam, Nixon, & Carter, 2004). Motion trajectories, such as joint angles and vertical displacement trajectories, are the most widely used features in gait analysis (Lakany, 2008). Gait recognition approaches can be broadly divided into two categories: silhouette-based (or appearance-based) ones (Collins, Gross, & Shi, 2002; Kim, Kim, & Paik, 2010; Lam, Cheung, & Liu, 2011; Murase & Sakai, 1996; Wang, Ning, Hu, & Tan, 2003) and model-based ones (Huang & Boulgouris, 2009; Phillips, Sarkar, Robledo, Grother, & Bowyer, 2002; Tafazzoli & Safabakhsh, 2010; Wagg & Nixon, 2004; Yam & Nixon, 2009; Yoo, Nixon, & Harris, 2002). The silhouettebased approaches usually use a sequence of holistic binary silhouettes which are extracted from a video using segmentation techniques. They require good quality silhouette images to work

W. Zeng, C. Wang / Neural Networks 35 (2012) 92–102

with. The main advantages of these approaches are their simplicity and speed. However, silhouette dynamics is only indirectly linked to gait dynamics. It is difficult to infer the importance of different gait components from silhouette dynamics (Wagg & Nixon, 2004). In model-based approaches, static or dynamic information is gathered from moving human bodies, such as joints or limbs. The information is then used to form a model with an underlying mathematical construct. This model represents the discriminatory gait characteristics. The model-based recognition system normally consists of gait capture, a static or dynamic feature extraction scheme, a gait signature and a classifier. Cunado, Nixon, and Carter (2003) modeled the thigh as a pendulum and extracted a hip joint trajectory from image sequences, then used the Fourier series to extract the leg’s angular movements. A frequency-based gait signature was derived directly from the whole image sequence for recognition. Yam et al. (2004) extended the approach to describe the hip, thigh and knee angular motion of both walking and running gaits by an empirical motion model, then by an analytical model motivated by coupled pendulum motion. Phaseweighted Fourier description gait signatures were then derived from the extracted movements. Zhang et al. (2007) proposed a 2D model-based approach in which gait features were extracted by fitting a five-link biped human locomotion model to extract the joint position trajectories. The recognition step was performed using Hidden Markov Models based on the frequency components of these joint trajectories. Tafazzoli and Safabakhsh (2010) investigated the potential discriminatory capability of the gait signature obtained from different parts of the human body such as legs and arms. The subject was modeled based on anatomical proportions and recognition was carried out throughout the knearest neighbor classifier and Fourier components of the joint angles. Recognition of temporal/dynamical patterns is among the most difficult pattern recognition tasks (Hong & Huang, 2002). Human gait recognition is a typical difficulty in the area of dynamical pattern recognition. One of the most difficult problems in dynamical pattern recognition is how to appropriately represent the time-varying patterns. Another important problem currently studied in this area is how to define the similarity between two dynamical patterns. Wang (2003) indicated that the methods for temporal pattern processing should be different from those for static pattern processing since the information of temporal patterns is embedded in time (thus inherently dynamic), not simultaneously available. In Wang and Hill (2007), a different framework was proposed for representation, similarity definition and rapid recognition of dynamical patterns. A time-varying dynamical pattern could be effectively represented in a timeinvariant and spatially distributed manner. A similarity definition for dynamical patterns was given based on system dynamics. A mechanism for rapid recognition of dynamical patterns was presented by which a test dynamical pattern is recognized as similar to a training dynamical pattern if state synchronization is achieved according to a kind of internal and dynamical matching on system dynamics. Problems similar to those mentioned in the dynamical pattern recognition also exist in the process of human gait recognition, which can be solved by referring to the method mentioned in Wang and Hill (2007). In this paper, we present a new model-based approach for human gait recognition via deterministic learning theory. The approach consists of two phases: a training (learning) phase and a test (recognition) phase. In the training phase, side silhouette lower limb joint angles and angular velocities are selected as gait features. A five-link biped model for human gait locomotion is employed to demonstrate that functions containing joint angle and angular velocity state vectors characterize the gait system dynamics. Due to the quasi-periodic and symmetrical

93

characteristics of human gait, the gait system dynamics can be simplified to be described by functions of joint angles and angular velocities of one side of the human body, thus the feature dimension is effectively reduced. Since only joint angles are measurable from image sequences, a high-gain observer is used to estimate joint angular velocities. Locally-accurate identification of the gait system dynamics is achieved by using radial basis function (RBF) neural networks (NNs) through deterministic learning. The obtained knowledge of the approximated gait system dynamics is stored in constant RBF networks. Hence, time-varying gait dynamical patterns can be effectively represented by the locally accurate NN approximations of system dynamics, and this representation is time-invariant. These constant RBF networks trained via deterministic learning naturally have a certain ability of generalization (Wang & Hill, 2009), since whenever the trajectory of a test gait pattern lies within the local region of one training gait patterns, the corresponding RBF network will provide accurate approximation to the previously learned gait system dynamics. A gait signature is then derived from the extracted gait system dynamics along the phase portrait of joint angles versus angular velocities. A bank of estimators is constructed using the constant RBF networks to represent the training gait patterns and previously learned gait system dynamics is embedded in the estimators. In the test phase, by comparing the set of estimators with the test gait pattern, a set of recognition errors are generated, and the average L1 norms of the errors are taken as the similarity measure between the dynamics of the training gait patterns and the dynamics of the test gait pattern. Therefore, a test gait pattern similar to one of the training gait patterns can be rapidly recognized according to the smallest error principle. Compared with the existing gait recognition approaches, ours can learn the internal dynamics of human locomotion systems and apply the learned knowledge to the human gait recognition. The rest of the paper is organized as follows. Section 2 introduces some preliminaries and the biped model of human gait locomotion. In Section 3, the joint angles and angular velocities are selected as gait features. Identification of human gait locomotion is achieved and a gait signature is derived from the extracted gait system dynamics. A recognition mechanism of human gait locomotion is presented. The experiments of gait recognition are given in Section 4 to demonstrate the effectiveness of our approach. Section 5 contains the conclusions. 2. Preliminaries 2.1. Deterministic learning theory Recently, a deterministic learning theory was proposed for identification of nonlinear dynamical systems undergoing periodic or recurrent motions (Wang & Hill, 2007, 2009). It is shown that, by using localized RBF networks, almost any periodic or recurrent trajectory can lead to the satisfaction of a partial persistence of excitation (PE) condition. This partial PE condition leads to exponential stability of a class of linear time-varying adaptive systems. Consequently, accurate NN approximation of the system dynamics is achieved in a local region along the periodic or recurrent system trajectory. Further, by using the locally-accurate NN approximation of system dynamics, rapid recognition of a test dynamical pattern from a set of training dynamical patterns can be achieved (Wang & Hill, 2007). N The RBF networks can be described by fnn (Z ) = i=1 wi si (Z ) = W T S (Z ), where Z ∈ ΩZ ⊂ Rp is the input vector, W = [w1 , . . . , wN ]T ∈ RN is the weight vector, N is the NN node number, and S (Z ) = [s1 (∥Z − µ1 ∥), . . . , sN (∥Z − µN ∥)]T is the regressor vector, with si (∥Z − µi ∥) = exp[

−(Z −µi )T (Z −µi ) ], ηi2

i = 1, . . . , N being a

94

W. Zeng, C. Wang / Neural Networks 35 (2012) 92–102

Gaussian RBF, µi being the center of the receptive field and ηi being the width of the receptive field. It has been proven in Park and Sandberg (1991) that an RBF network fnn (Z ), with sufficiently large node number N, can approximate any continuous function f (Z ) : ΩZ → R over a compact set ΩZ ⊂ Rp to arbitrary accuracy, according to f (Z ) = W ∗ T S (Z ) + ϵ(Z ), ∀Z ∈ ΩZ , where W ∗ is the ideal constant weight, |ϵ(Z )| < ϵ ∗ (with ϵ ∗ > 0) is the approximation error. For any bounded trajectory Z (t ) (∀t ≥ 0) within the compact set ΩZ , f (Z ) can be approximated using neurons located in a local region along the trajectory: f (Z ) = Wζ∗ T Sζ (Z ) + ϵζ , where ϵζ = O(ϵ) is the approximation

error, Sζ (Z ) ∈ RNζ is a subvector of S (Z ), Wζ∗ ∈ RNζ , Nζ < N. In recent years, deterministic learning theory has achieved sustained development and has been applied in many areas, such as neural identification and control (Liu, Wang, & Hill, 2009; Wang, Chen, Chen, & Hill, 2009; Wang & Hill, 2006), dynamical pattern recognition (Wang & Hill, 2007) and fault detection (Wang & Chen, 2011). The deterministic learning theory is essential for the identification of the system dynamics of training dynamical patterns. For example, consider a general nonlinear dynamical system: x˙ = F (x; p),

x(t0 ) = x0

where x = [x1 , . . . , xn ]T ∈ Rn is the state of the system, p is a system parameter vector, F (x; p) = [f1 (x; p), . . . , fn (x; p)]T is a smooth but unknown nonlinear vector field. It is assumed that the system trajectory generated from the above dynamical system, denoted as ϕζ (x0 ), is a recurrent trajectory. The objective of deterministic learning is to develop the following neural identifier (2) to accurately identify the unknown system dynamics F (x; p):

ˆ iT Si (x), x˙ˆ i = −ai (ˆxi − xi ) + W

i = 1, . . . , n

(2)

where xˆ = [ˆx1 , . . . , xˆ n ] is the state vector of the dynamical RBF network, x is the state of system (1), ai > 0 are design constants, ˆ iT Si (x) is a localized RBF network used in a closed loop to and W approximate the unknown nonlinearity fi (x; p) in (1) within the compact set ΩZ . ˆ i are updated by a Lyapunov-based The weight estimates W learning law T

˙

ˆ i = −Γi Si (x)˜xi − σi Γi W ˆi W

(3)

ˆ i is the estimate where x˜ i = xˆ i − xi is the state estimation error, W of optimal value Wi∗ , Γi = ΓiT > 0, and σi > 0 is a small value. With the employment of the dynamical RBF network (2), it was indicated that for almost every recurrent trajectory ϕζ (x0 ) starting from an initial condition x0 = x(0) ∈ Ω , with initial values ˆ i (0) = 0, a locally-accurate NN approximation for the unknown W fi (x; p) to the error level ϵ ∗ is obtained along the trajectory ϕζ (x0 ) fi (xi ; p) =

T Wi∗ Si

Fig. 1. Five-link biped model.

(1)

(x) + ϵi

(4)

where |ϵi | < ϵ . ∗

2.2. Biped model of human gait locomotion In this section, observing that most walking dynamics take place in the sagittal plane, a two-dimensional five-link biped model is employed to obtain the gait system dynamics and extract dynamic features for describing human gait locomotion in the sagittal plane. Owing to the symmetrical characteristics of human walking, gait system dynamics can be simplified to be described by the functions of joint angles and angular velocities of one side of the human body. It is difficult to accurately model a human’s gait pattern since it has deformable muscles, a high degree of freedom and a

complicated mechanical structure. Therefore, the human model for the gait analysis should be as simple as possible. In order to imitate the human gait locomotion and analyze the dynamics of human gait, human motion has been modeled as a five-link biped model. Borghese, Bianchi, and Lacquaniti (1996) showed that most human walking dynamics take place in the sagittal plane, which implies the possibility of a two-dimensional five-link biped model being used in this paper to approximate a human’s complex mechanical structure. Hence, we are interested in extracting the gait dynamics information contained in the image sequences of individuals walking parallel to the camera (lateral view). The five-link biped model consists of five rigid links: one link for the trunk, two links for the thigh, and two links for the shank, as shown in Fig. 1. It has two pelvises at the hip, two knees between the thighs and the shanks, and two ankles at the tips of the limbs. Each body part is considered to be rigid with movement only allowed at joint positions. θi (i = 1, . . . , 5) is the absolute angle between the ith link and the vertical direction. The biped gait is composed periodically of Single Support Phase (SSP) and Double Support Phase (DSP). Since the time period of the DSP is very short, the system dynamics can be viewed as a boundary condition of the SSP. The dynamics of the five-link biped on the SSP can be derived from the Lagrangian equation in the following form: D(θ )θ¨ + H (θ )θ˙ 2 + G(θ ) = T

(5)

where T is the 5 × 1 generalized torque corresponding to θ , D(θ ) is the 5 × 5 positive definite and symmetric inertia matrix, H (θ ) is the 5 × 5 Centrifugal and Coriolis matrix, G(θ ) is the 5 × 1 matrix of gravity coordinates. θ , θ˙ , θ¨ are the 5×1 state vectors of generalized coordinates, velocities and accelerations respectively. The detailed expressions of D, H , G and T are shown in Mu and Wu (2003). Let



θ = [θ1 , θ2 , θ3 , θ4 , θ5 ]T ω = θ˙ = [θ˙1 , θ˙2 , θ˙3 , θ˙4 , θ˙5 ]T = [ω1 , ω2 , ω3 , ω4 , ω5 ]T

(6)

where θ1 , θ2 denote the knee and thigh angles of one leg respectively, θ5 , θ4 denote the knee and thigh angles of the other leg respectively, θ3 denotes the angle between the torso and the vertical direction. Eq. (5) can be transformed into the following form:



θ˙ = ω ω˙ = D(θ )−1 (T − H (θ )ω2 ) − D(θ )−1 G(θ ).

(7)

W. Zeng, C. Wang / Neural Networks 35 (2012) 92–102

95

–0.8

(a) Joint angles.

(b) Joint angular velocities. Fig. 2. Joint angle and angular velocity trajectories of the five-link biped model (Mu & Wu, 2003).

Then, the gait system dynamics can be represented by function F (θ, ω) = D(θ )−1 (T − H (θ )ω2 ) − D(θ )−1 G(θ ) along the phase portrait of θ versus ω. Therefore, the deterministic learning theory can be employed to identify and learn the unknown gait system dynamics with the NN input Z = [θ , ω]T . Obviously, the dimension of the NN input signal is high which increases the learning complexity and the computational cost. Hence, the composition of the gait system dynamics must be approximatively simplified so as to reduce the dimension of the NN input. From the psychologists’ view, human gait is a symmetrical pattern of motion (Cutting, Proffitt, & Kozlowski, 1978). Therefore, symmetry is suitable for gait recognition. The characteristics of symmetry means the joint angles with which the movement of one leg are similar to the other one (Trivino, Alvarez-Alvarez, & Bailador, 2010). That is, the trajectories of θ1 , θ2 with one leg are similar to those of θ5 , θ4 with the other leg, which exist for a half period of phase shift. Specifically,

 θ1 (0) = θ5 (Tssp ), θ5 (0) = θ1 (Tssp ),    θ ( t ) = θ ( T − t )  1 5 ssp  θ (0) = θ (T ), θ4 (0) = θ2 (Tssp ), 2 4 ssp θ2 (t ) = θ4 (Tssp − t )     θ˙i (0) = θ˙i (Tssp ), i = 1, . . . , 5 θ3 (0) = θ3 (Tssp ),  |θ˙1 (t )| = |θ˙5 (Tssp − t )|, |θ˙2 (t )| = |θ˙4 (Tssp − t )|

(8)

l = {1, 2} or {4, 5}.

3. Identification and recognition of human gait locomotion In this section, we present a scheme for identification of human gait locomotion based on deterministic learning theory. A gait signature consisting of gait system dynamics along the phase portrait of joint angles versus angular velocities is also derived for gait recognition. Then, the mechanism of human gait recognition is presented. 3.1. Identification mechanism of human gait locomotion The mechanism of human walking is similar to the five-link biped model. However, human walking is too complex to be simply describe by Eq. (7) since there exists modeling uncertainty in human gait locomotion. In order to more accurately describe human walking, the dynamics of human gait locomotion can be summarized into the following form:

ω˙ = F (θ , ω) + v(θ , ω)

(10)

where θ = [θ1 , . . . , θn ] ∈ R and ω = [ω1 , . . . , ωn ] ∈ R are the states of system (10) which represent the lower limb joint angles and angular velocities of both sides of the human body respectively. F (θ , ω) = [f1 (θ , ω), . . . , fn (θ , ω)]T is a smooth but unknown nonlinear vector representing the gait system dynamics, v(θ , ω) is the modeling uncertainty. The system trajectory starting from initial condition (θ0 , ω0 ), is denoted as ϕζ (θ0 , ω0 ). Since the modeling uncertainty v(θ , ω) and the gait system dynamics F (θ , ω) cannot be decoupled from each other, we consider the two terms together as an undivided term, and define φ(θ , ω) := F (θ , ω) + v(θ , ω) as the general gait system dynamics. The objective of the training or learning phase is to identify or approximate the general gait system dynamics φ(θ , ω) to a desired accuracy via deterministic learning. Joint angles θ can be extracted by using the method of body segment property (Yoo et al., 2002) from human walking image sequences. Since only joint angles θ are measurable, the following high-gain observer (Gauthier, Hammouri, & Othman, 1992) is used to estimate joint angular velocities ω: T

where Tssp denotes the period of the SSP phase. Hence, in the gait system dynamics F (θ , ω), the expressions of θ4 , θ5 , ω4 , ω5 can be replaced by θ2 , θ1 , ω2 , ω1 respectively. From the viewpoint of natural human walking, it is reasonable to assume that the trajectory of torso, denoted by θ3 , is zero, then θ˙3 = 0, i.e., the biped is always walking with its torso maintained in an upright position. The trajectories of joint angle and angular velocity for the five-link biped model are shown in Fig. 2. Then, the gait system dynamics F (θ, ω) can be rewritten as F (θ, ω) = F (θl , ωl ),

joint angles and angular velocities from human walking image sequences, rather than directly from this model, for identifying the gait system dynamics and for gait recognition.

(9)

Hence, it can be seen from the analysis of the five-link biped model that gait system dynamics has been simplified to be related to the state vectors of joint angles and angular velocities with one leg due to the symmetrical characteristics of gait. Then, the deterministic learning theory may be employed to identify the gait system dynamics F (θ , ω) with the NN input Z = [θ1 , θ2 , ω1 , ω2 ]T or Z = [θ4 , θ5 , ω4 , ω5 ]T whose dimension is obviously reduced. Human walking satisfies the symmetrical characteristics, and the relationship of the joint angles and angular velocities between two legs also approximatively satisfies Eq. (8). The five-link model will enable selection of joint angles and angular velocities data pertaining to the motion of body limbs. This ensures that we extract

 θ˙ˆ = ωˆ + h1 k(θ − θˆ ) ω˙ˆ = h2 k2 (θ − θˆ )

n

T

n

(11)

where h1 , h2 and k are design constants, θˆ and ω ˆ are the estimates of the states θ and ω. Let χ = [θ , ω]T , χˆ = [θˆ , ω] ˆ T , if h1 , h2 are

96

W. Zeng, C. Wang / Neural Networks 35 (2012) 92–102

2

2−j chosen such that s2 + is a Hurwitz polynomial with j =1 hj s distinct roots, then for all d and all time t ′ there exists a finite observer gain k′ such that for all k ≥ k′ , the observer error satisfies ∥χ( ˆ t ) − χ (t )∥ ≤ d, ∀t ≥ t ′ (Gauthier et al., 1992). This denotes χˆ → χ , which means the estimates χˆ converge to a sufficiently small neighborhood of χ in some finite time. Based on deterministic learning theory and the symmetrical characteristics of human walking, the following dynamical RBF networks are employed to identify the gait system dynamics

φ(θ, ω) = [φ1 (θ , ω), . . . , φn (θ , ω)]

T

in which ω ˆ ′ ∈ [ω, ˆ ω) or ωˆ ′ ∈ (ω, ω] ˆ . Since ai and |

|ωˆ i − ωi | are bounded, x¯ i − x˜ i is small when ωˆ → ω, εζ i can be expressed in an order term as O(ϵ ∗ ). Eq. (13) can be described by ˙

˙

ˆ ζi = W ˜ ζ i = −Γζ i Sζ i (θˆl , ωˆ l )˜xi − σi Γζ i W ˆ ζ i − εW ζ i W

ˆ S (θˆl , ωˆ l ) x˙ = −A(x − ω) ˆ +W

(12)

where x = [x1 , . . . , xn ] is the state vector of the dynamical RBF networks, A = diag[a1 , . . . , an ] is a diagonal matrix, with ai > 0 ˆ T S (θˆl , ωˆ l ) = being design constants, localized RBF networks W T T T ˆ 1 S1 (θˆl , ωˆ l ), . . . , W ˆ n Sn (θˆl , ωˆ l )] are used to approximate the [W unknown φ(θ , ω), where θl and ωl represent the lower limb joint angles and angular velocities of one side of the human body respectively, θˆl , ω ˆ l are the estimates of θl , ωl . The employment of RBF NN is due to its associated properties, including the function approximation ability, the spatially localized structure and a property concerning the PE condition (Gorinevsky, 1995; Park & Sandberg, 1991; Wang & Hill, 2009). The NN weight updating law is given by:

˙ˆ = W ˙˜ = −Γ S (θˆ , ωˆ )¯x − σ Γ W ˆ W i i i i l l i i i i

(13)

˜i = W ˆ i − Wi∗ , Wi∗ is the ideal constant where x¯ i = xi − ω ˆ i, W T weight vector, Γi = Γi > 0, and σi > 0 is a small value. We also define x˜ i = xi − ωi . Note that since ωi is not available from measurement, x¯ i is computable, while x˜ i is not. However, it is seen that as ω ˆ i → ωi , x˜ i → x¯ i . Consider the adaptive system consisting of the nonlinear dynamical system (10), the HGO (11), the dynamical RBF network (12), and the NN weight updating law (13). For almost every ˆ (0) = 0, according to trajectory ϕζ (θ0 , ω0 ), with initial values W Theorem 1 in Wang and Hill (2010), we have that all signals in the adaptive system remain uniformly bounded. By using the spatially localized learning property of RBF networks, along the estimated system state (θˆ , ω) ˆ , with Eqs. (10) and (12), the derivative of the state estimation error x˜ i satisfies

ˆ iT Si (θˆl , ωˆ l ) − φi (θ , ω) x˙˜ i = −ai x¯ i + W

(17)

ˆ l )(¯xi − x˜ i ) and εWζ¯ i = Γζ¯ i Sζ¯ i (θˆl , ωˆ l )(¯xi − where εWζ i = Γζ i Sζ i (θˆl , ω ˆ → ω. x˜ i ). Both εWζ i and εWζ¯ i are small when ω By using the local approximation property of RBF networks, the overall system consisting of dynamical model (14) and the NN weight updating law (16) can be summarized into the following form in the region Ωζ



x˙˜ i

˙

˜ ζi W



−a i Sζ i (θˆl , ω ˆ l )T = −Γζ i Sζ i (θˆl , ωˆ l ) 0   εζ i + ˆ ζ i − εW ζ i . −σi Γζ i W 



x˜ i ˜ ζi W



(18)

The nominal part of system (18) is referred to as system (18) ˆ ζ i − εWζ i . For the human gait without the terms εζ i and −σi Γζ i W system, (Murray, 1967) suggested that the human gait is a form of periodic or quasi-periodic motion, especially when walking laterally. Fig. 3 shows quasi-periodic gait signals generated from a free human walking in the sagittal plane. Hence, the NN input Z = [θˆl , ω ˆ l ]T is quasi-periodic. According to Theorem 1 in Wang et al. (2009), the regression vector Sζ i (θˆl , ω ˆ l ) satisfies PE condition almost always. This will

˜ ζ i ) = 0 of the nominal part lead to exponential stability of (˜xi , W of system (18) (Farrell, 1998). Based on the analysis results given in Wang et al. (2009) and Wang and Hill (2009), the NN weight ˜ ζ i converges to small neighborhoods of zero, with estimate error W the sizes of the neighborhoods being determined by εζ i , εWζ i and σi Γζ i Wζ∗i , all of which are small values. This means that the entire ˆ iT Si (θˆl , ωˆ l ) can approximate the unknown φi (θ , ω) RBF network W

ˆ ζTi Sζ i (θˆl , ωˆ l ) + W ˆ ¯T Sζ¯ i (θˆl , ωˆ l ) = −ai x˜ i + ai (˜xi − x¯ i ) + W ζi

along the estimated state trajectory (θˆ , ω) ˆ , and

− Wζ∗i T Sζ i (θˆl , ωˆ l ) − ϵζ i + φi (θˆ , ω) ˆ − φi (θ , ω) ˜ ζTi Sζ i (θˆl , ωˆ l ) + εζ i = −ai x˜ i + W

(16)

and

˙ˆ = W ˙˜ = −Γ S (θˆ , ωˆ )˜x − σ Γ W ˆ W l i i ζ¯ i ζ¯ i − εWζ¯ i ζ¯ i ζ¯ i ζ¯ i ζ¯ i l

T

∂φi (θ ,ω) |ωi =ωˆ i′ ∂ωi

ˆ iT Si (θˆl , ωˆ l ) + ϵi1 φi (θˆ , ω) ˆ =W (14)

ˆ ζ i is the correspondwhere Sζ i (θˆl , ω ˆ l ) is a subsector of Si (θˆl , ωˆ l ), W ing weight subvector. The subscripts (·)ζ and (·)ζ¯ are used to stand for terms related to the regions close to and far away from the ˆ ¯T Sζ¯ i (θˆl , ωˆ l )| being small. The region trajectory ϕζ (θ0 , ω0 ), with |W ζi

(19)

where ϵi1 = O(ϵζ i ). By the convergence result, we can obtain a constant vector of neural weights according to

¯ i = meant ∈[ta ,tb ] W ˆ i (t ) W

(20)

close to the trajectory is defined as Ωζ := Z |dist(Z , ϕζ ) ≤ dι , where Z = [θˆl , ω ˆ l ]T , dι > 0 is a constant satisfying s(dι ) > ι, s(·) is the RBF used in the network, ι is a small positive constant, and

where tb > ta > 0 represent a time segment after the transient process. Therefore, we conclude that accurate identification of the function φi (θ , ω) is obtained along the trajectory ϕζ (θ0 , ω0 ) by

ˆ ¯T Sζ¯ i (θˆl , ωˆ l ) εζ i = ai (˜xi − x¯ i ) − ϵζ i + φi (θˆ , ω) ˆ − φi (θ , ω) + W ζi  ∂φi (θ , ω) T = ai (˜xi − x¯ i ) − ϵζ i + ∂ωi ωi =ωˆ ′

¯ iT Si (θˆl , ωˆ l ) + ϵi2 φi (θˆ , ω) ˆ =W

i

(θˆl , ωˆ l ) × (ωˆ i − ωi ) + ˆ    ∂φi (θ , ω)   |ωˆ i − ωi | < ai |(¯xi − x˜ i )| +  ∂ωi ωi =ωˆ ′ Wζ¯Ti Sζ¯ i

i

ˆ ¯T Sζ¯ i (θˆl , ωˆ l )| + ϵ + |W ζi ∗

(15)

¯ iT Si (θˆl , ωˆ l ), i.e., using W

(21)

where ϵi2 = O(ϵi1 ) and subsequently ϵi2 = O(ϵ ∗ ). Hence, locally-accurate identification of gait system dynamics φi (θ , ω) to the error level ϵ ∗ is achieved along the trajectory ϕζ (θ0 , ω0 ) when ωˆ → ω. Time-varying gait dynamical patterns can be effectively represented by the locally-accurate NN approximations of system dynamics, and this representation is timeinvariant.

W. Zeng, C. Wang / Neural Networks 35 (2012) 92–102

97

Fig. 3. Joint angle and angular velocity trajectories from a sequence of free human walking images: (a) thigh and knee angles during two walking cycles, (b) thigh and knee angular velocities during two walking cycles.

¯ T S (θˆl , ωˆ l ) trained via Remark 1. The constant RBF network W deterministic learning naturally has a certain ability of generalization (see Wang & Hill, 2009), since the obtained NN approximation is valid in a local region Ωζ along the recurrent trajectory of the trained gait patterns. In the test phase, for a test gait pattern with possibly different initial conditions or even different parameters, as long as the trajectory of the test gait pattern lies within ¯ T S (θˆl , ωˆ l ) the local region Ωζ , the corresponding RBF network W will provide an accurate approximation to the previously learned dynamics ϕζ . Thus, it is not necessary to include all possible gait trajectories starting from an infinite number of initial values in the training gait sets. 3.2. Derivation of gait signature Gait signatures are the most effective and well-defined representation method for dynamic gait analysis. They can be extracted by motion information from human gait. To recognize different individuals by their gait easily, we need to firstly select the most efficient gait features which can best represent the human gait characteristics. In kinematic analysis, human gait is usually characterized by the limb joint angles and angular velocities between body segments and their relationships to the events of the gait cycle (Lee & Grimson, 2002). Since the upper limbs contain sincere self-occlusion which makes it difficult to track and measure the arm angles, the side silhouette lower limb joint angles and angular velocities are selected as the gait feature. It can be seen from the analysis of human gait identification in Section 3.1 that functions containing joint angle and angular velocity vectors of one side of the human body characterize the dynamics of the walking locomotion. This derives the gait signature consisting of gait system dynamics φ(θ , ω) along the phase portrait of joint angles versus angular velocities measured from a gait cycle. 3.3. Recognition mechanism of human gait locomotion In this section, we present a scheme for rapid recognition of human gait by using the learned gait system dynamics. Consider a training set containing dynamical human gait patterns ϕζk , k = 1, . . . , M, with the kth gait training pattern ϕζk generated from

ω˙ k = F k (θ k , ωk ) + v k (θ k , ωk ), ωk (t0 ) = ω0k

θ k (t0 ) = θ0k ,

As shown in Section 3.1, the general gait system dynamics

φ k (θ k , ωk ) := F k (θ k , ωk ) + v k (θ k , ωk ) can be accurately identified ¯ kT S (θlk , ωlk ). By utilizing and stored in constant RBF networks W the learned knowledge obtained in the training phase, a bank of M estimators is first constructed for the trained human gait dynamical systems as follows: k ¯ kT S (θˆl , ωˆ l ) χ¯˙ = −B(χ¯ k − ω) ˆ +W

(23)

where k = 1, . . . , M is used to stand for the kth estimator, χ¯ k = [χ¯ 1k , . . . , χ¯ nk ]T is the state of the estimator, B = diag[b1 , . . . , bn ] is a diagonal matrix which is kept the same for all estimators. In the test phase, the system states of the test gait pattern can still be estimated through (11). By comparing the test gait pattern generated from human gait dynamical system (10) with the set of M estimators (23), we obtain the following recognition error systems: k ¯ ikT Si (θˆl , ωˆ l ) − φi (θ , ω), χ˙˜ i = −bi χ˜ ik − bi (ωi − ωˆ i ) + W

i = 1, . . . , n, k = 1, . . . , M

(24)

where χ˜ = χ¯ − ωi is the state estimation error. We compute the average L1 norm of the error χ˜ ik (t ) k i

∥χ˜ ik (t )∥1 =

k i

1



Tc

t

|χ˜ ik (τ )|dτ ,

t ≥ Tc

(25)

t −T c

where Tc is the cycle of human gait. The fundamental idea of human gait recognition is that if one person appearing with a gait pattern similar to the trained human gait pattern s (s ∈ {1, . . . , k}), the constant RBF network T

¯ is Si (θˆlk , ωˆ lk ) embedded in the matched estimator s will quickly W recall the learned knowledge by providing accurate approximation to the human gait dynamics. Thus, the corresponding error ∥χ˜ is (t )∥1 will become the smallest among all the errors ∥χ˜ ik (t )∥1 . Based on the smallest error principle, the appearing person can be recognized. We have the following recognition scheme and summarize the flow of the proposed recognition approach in Fig. 4. Human gait recognition scheme: If there exists some finite time t s , s ∈ {1, . . . , k} and some i ∈ {1, . . . , n} such that ∥χ˜ is (t )∥1 < ∥χ˜ ik (t )∥1 for all t > t s , then the appearing person can be recognized. 4. Experiments

(22)

where F k (θ k , ωk ) denotes the gait system dynamics, v k (θ k , ωk ) denotes the modeling uncertainty.

In this section, based on dynamical estimators and the smallest error principle mentioned in Section 3.3, we demonstrate how to achieve human gait recognition on the NLPR and UCSD gait databases.

98

W. Zeng, C. Wang / Neural Networks 35 (2012) 92–102

Fig. 4. Overview of the proposed recognition approach.

4.1. NLPR gait database The NLPR database was established by Wang et al. (2003). This database is widely used to benchmark algorithms in gait recognition and is also known as the CASIA-A gait database. To compile the NLPR database, a digital camera fixed on a tripod is used to capture gait sequences at a rate of 25 frames/s on two different days in an outdoor environment. The database consists of 20 persons, each person has 12 image sequences, 4 sequences for each of the three directions, i.e. parallel (lateral view), 45° (oblique view) and 90° (frontal view) to the image plane. The length of each sequence is not identical for the variation of the walker’s speed, but it must range from 37 to 127. The original image resolution is 352 × 240 pixels. In order to perform human gait recognition, joint angles would be firstly extracted from image sequences of parallel walking direction. The preprocessing can be divided into the following phases: (1) extracting the body silhouette; (2) extracting the lower limb joint angles. A background subtraction technique is used to segment the body’s silhouette from image sequences in the lateral view. The static background pixels sometimes include non-background pixels mainly due to shadows, and the silhouettes can contain holes and noise while directly subtracting the foreground image from original image. To solve this problem, we use the Mathematical Morphology Method to fill in holes and remove noise. Then, Binary Connected Component Analysis is employed to extract a singleconnective moving object. Edge images are produced by applying the Canny operator with hysteresis thresholding. Finally, the body Silhouette is determined followed by dilation and erosion. The bounding box of the silhouette image is then computed from the binarized image. The silhouette image is cropped according to the position and size of the bounding box. To extract the lower limb joint angles, such as hip angles and knee angles, we should firstly locate the positions of the pelvis, knee and ankle joints. Then, according to the method of body segment property (Yoo et al., 2002), five lower limb joints positions are determined, including the pelvis position (xp , yp ), the knee joints positions (xk1 , yk1 ) and (xk2 , yk2 ), and the ankle joints positions (xa1 , ya1 ) and (xa2 , ya2 ). Figs. 5 and 6 show an example of silhouette and joint angles extraction from image sequences. Eq. (26) can be used to compute any lower limb joint angle Θ at one frame:

Θ = tan−1



x − x′ y − y′

 (26)

where (x, y) and (x′ , y′ ) represent the neighboring joints coordinates.

Fig. 5. Example of silhouette extraction and joint positioning from NLPR database. (a) An original image (lateral view), (b) binary silhouette, (c) edge detection, (d) the bounding box, (e) joint positioning by using body segment property, ‘‘+’’ stands for the joint position.

Fig. 6. Lower limb joint angles (θt1 and θt2 stand for thigh angles, θk1 and θk2 stand for knee angles).

We select 15 persons, each person has two sequences for parallel walking direction, one for training and the other for the test. Based on body segment property and Eq. (26), we extract the 15 persons’ lower limb joint angles (thigh and knee angles) θ = [θt1 , θt2 , θk1 , θk2 ]T through the walking image sequences. The thigh and knee angular velocities ω ˆ = [ωt1 , ωt2 , ωk1 , ωk2 ]T are

W. Zeng, C. Wang / Neural Networks 35 (2012) 92–102

(a) θk1 − ωk1 phase portrait.

99

(b) θt1 − ωt1 phase portrait.

Fig. 7. Phase portraits of joint angle–angular velocity for three different subjects.

estimated according to Eq. (11) and the design constants are chosen as h1 = 0.5, h2 = 1.5, k = 45. The thigh and knee angles and angular velocities of one side of the human body are selected as the gait features, which means the input of the RBF networks Z = [θt1 , θk1 , ωt1 , ωk1 ]T . Different subjects have different dynamics during walking (see, e.g. Fig. 7) which makes the gait signature effective for gait recognition. From Fig. 7, we see that the differences between the gait dynamics of three subjects are subtle. A planar angle–angular velocity phase portrait represents the complete dynamics of a single joint. However, since the phase portrait of a multidegree of freedom system is not entirely graphically visualizable, we have to be content with lower-dimensional (most frequently twodimensional) projection of the diagram. When the knowledge of the gait dynamics generated from different individuals is obtained, the difference between walking locomotion of different individuals can be embodied. Figs. 8–10 show an example of the training and recognition of person fyc in the 15-person dataset. In the training phase, the ˆ iT Si (Z ) is constructed in a regular lattice, with RBF network W nodes N = 3600, the centers µi evenly spaced on [−0.58, 0.32] × [−0.5, 0.95] × [−3, 1.95] × [−6.5, 3.4], and the widths η = 0.25. The weights of the RBF networks are updated according to Eq. (13). ˆ i (0) = 0. The design parameters for (12) and The initial weights W (13) are ai = 0.5, Γ = diag{1.5, 1.5, 1.5, 1.5}, σi = 10, (i = 1, . . . , 4). The convergence of neural weights is shown in Fig. 8, which demonstrates partial parameter convergence, that is, only the weight estimates of some neurons whose centers close to the orbit are activated and updated. These weights converge to their optimal values Wi∗ . Based on the deterministic learning theory, the gait system dynamics φik (θ , ω) can be locally-accurately approximated by

ˆ 1 in one training pattern. Fig. 8. Partial parameter convergence of W Table 1 Comparison of several algorithms on the NLPR database (for lateral view). Algorithms

Recognition rate (%)

Lee and Grimson (2002) Collins et al. (2002) Phillips et al. (2002) Wang et al. (2003) Lu and Zhang (2007) Lee et al. (2009) Proposed method

87.5 71 79 89 92.5 91.25 93.3

T

¯ ik Si (Z ), (k = 1, . . . , 15) along recurrent system trajectory, then W these constant weights are stored for each training pattern. ¯ ikT Si (Z ), In the test phase, by using the constant networks W (k = 1, . . . , 15), fifteen RBF network estimators are constructed based on (23). The design parameters in (23) are bi = 0.5 (i = 1, . . . , 4). The cycle of gait for person fyc is Tc = 1.04 s. Consider recognition of the test pattern person fyc by 15 training patterns. The average L1 norms of the synchronization errors, that is, ∥˜xki (t )∥1 (k = 1, . . . , 15) are shown in Fig. 9. It is obvious that after certain time (t = 3.08 s, labeled by the red vertical line), the average L1 norm generated by the training pattern person fyc becomes smaller than the others (the details can be seen in the partial enlarged zone of Fig. 9). In order to show the recognition result visually, the key images of the test pattern person fyc, the

training pattern person fyc with the smallest error and the training pattern person xch with the second smallest error are shown in Fig. 10. The result shows that correct recognition of person fyc can be achieved. Table 1 compares the recognition performance of different published approaches on the NLPR database. Several papers have published results on this data set; hence, it is a good experimental data set to benchmark the performance of the proposed algorithm. We can see that our method compares favorably with others. The above only provides preliminary comparative results and may not be generalized to say that a certain algorithm is always better than others. Algorithm performance is dependent on the training and test sets. So further evaluations and comparisons on a larger and more realistic database are needed in future work.

100

W. Zeng, C. Wang / Neural Networks 35 (2012) 92–102

Fig. 9. Recognition result by using smallest error principle.

Fig. 10. Key images of test pattern compared with key images of training patterns: (a) test pattern fyc, (b) training pattern fyc with smallest error, (c) training pattern xch with second smallest error.

4.2. UCSD gait database UCSD (University of California, San Diego) gait database (http:// www-mitpress.mit.edu/e-journals/Videre/001/articles/Little-Boyd/ gait/gait.html) includes six subjects with seven image sequences of each. In each sequence the subject is walking from right to left in the lateral view, in front of a static background. Original and background images can be seen in Fig. 11. Body silhouette extraction and lower limb joint angles extraction are achieved by using the method mentioned in Section 4.1, as shown in Fig. 11. We select all the six persons, each person has two sequences for lateral view, one for training and the other for test. The process of training and test is similar to the example of NLPR database shown in Section 4.1 and is omitted here for clarity and conciseness. Figs. 12 and 13 show an example of the training and recognition of person g6 in the 6-person UCSD dataset, and person g6 can be correctly recognized. Table 2 compares the recognition performance of different published approaches on the UCSD database. The proposed algorithm presents better performance than other published results.

Table 2 Comparison of several algorithms on the UCSD database. Algorithms

Recognition rate (%)

Huang et al. (1999) BenAbdelkader et al. (2001) Hayfron-Acquah et al. (2003) Chai et al. (2006) Zhao et al. (2007) Proposed method

100 93 92.9 97.6 97.6 100

5. Conclusions A new model-based human gait recognition approach via deterministic learning theory is presented in this paper. The side silhouette lower limb joint angles and angular velocities are selected as gait features. Owing to the symmetrical characteristics of human walking, gait system dynamics can be simplified to be described by the functions of joint angles and angular velocities

W. Zeng, C. Wang / Neural Networks 35 (2012) 92–102

(a) Original image.

101

(b) Background.

(c) Edge detection.

(d) Joint positioning by body segment property.

Fig. 11. Example of silhouette extraction and joint positioning from UCSD database.

Fig. 13. Recognition result of person g6.

ˆ 1 in the training pattern of person g6. Fig. 12. Partial parameter convergence of W

of one side of the human body. Locally-accurate identification of the gait system internal dynamics is achieved by using RBF networks. The obtained gait system dynamic knowledge will be stored in constant RBF networks according to the deterministic learning theory. Then, a bank of estimators is constructed using the constant RBF networks to represent the training gait patterns and previously learned gait system dynamics is embedded in the estimators. By comparing the set of estimators with the test gait pattern, a set of recognition errors are generated and taken as the measure of the similarity between the test gait pattern and the training gait patterns, the test gait pattern can be recognized rapidly based on smallest error principle. Compared with existing gait recognition approaches, our method has the following characteristics: (1) the gait system dynamics can be described by functions of joint angles and angular velocities, then a gait signature consisting of gait system dynamics along the phase portrait of joint angles versus angular velocities is derived for recognition; (2) the gait system dynamics can be learned by RBF

networks and the learned knowledge is stored in constant RBF networks; and (3) the difference between test gait pattern and training gait patterns can be recognized by dynamical estimators which are with previous learned knowledge embedded in, instead of constructing K-Nearest Neighbor (KNN) classifiers (Tafazzoli & Safabakhsh, 2010; Yam et al., 2004), Hidden Markov Models (Zhang et al., 2007) or Support Vector Machine classifiers (Lu & Zhang, 2007) and so on. The effectiveness of our approach has been demonstrated on the NLPR and UCSD gait databases. Acknowledgments The authors would like to thank the anonymous reviewers for constructive comments. References BenAbdelkader, C., Cutler, R., Nanda, H., & Davis, L. (2001). Eigen gait: motionbased recognition of people using image self-similarity. In Proceedings of the third international conference on audio- and video-based biometric person authentication (pp. 284–294).

102

W. Zeng, C. Wang / Neural Networks 35 (2012) 92–102

Borghese, N. A., Bianchi, L., & Lacquaniti, F. (1996). Kinematic determinants of human locomotion. Journal of Physiology, 863–879. Chai, Y. M., Wang, Q., Jia, J. P., & Zhao, R. C. (2006). A novel gait recognition method via fusing shape and kinematics features. In The second international symposium on visual computing (pp. 80–89). Collins, R. T., Gross, R., & Shi, J. B. (2002). Silhouette-based human identification from body shape and gait. In Proceedings of the fifth IEEE international conference on automatic face and gesture recognition (pp. 351–356). Cunado, D., Nixon, M. S., & Carter, J. N. (2003). Automatic extraction and description of human gait models for recognition purposes. Computer Vision and Image Understanding, 90(1), 1–41. Cutting, J., Proffitt, D., & Kozlowski, L. (1978). A biomechanical invariant for gait perception. Journal of Experimental Psychology: Human Perception and Performance, 4(3), 357–372. Farrell, J. A. (1998). Stability and approximator convergence in nonparametric nonlinear adaptive control. IEEE Transactions on Neural Networks, 9(5), 1008–1020. Gauthier, J. P., Hammouri, H., & Othman, S. (1992). A simple observer for nonlinear systems applications to bioreactors. IEEE Transactions on Automatic Control, 37(6), 875–880. Gorinevsky, D. (1995). On the persistency of excitation in radial basis function network identification of nonlinear systems. IEEE Transactions on Neural Networks, 6(5), 1237–1244. Han, J., & Bhanu, B. (2006). Individual recognition using gait energy image. IEEE Transactions on Pattern Analysis and Machine Intelligence, 28(2), 316–322. Hayfron-Acquah, J. B., Nixon, M. S., & Carter, J. N. (2003). Automatic gait recognition by symmetry analysis. Pattern Recognition Letters, 24(13), 2175–2183. Hong, P. Y., & Huang, T. S. (2002). Automatic temporal pattern extraction and association. In 2002 IEEE international conference on acoustics, speech, and signal processing. Vol. 2 (pp. II-2005–II-2008). Huang, X. X., & Boulgouris, N. V. (2008). Human gait recognition based on multiview gait sequences. EURASIP Journal on Advances in Signal Processing, 1–8. Huang, X. X., & Boulgouris, N. V. (2009). Model-based human gait recognition using fusion of features. In Proceedings of 2009 IEEE international conference on acoustics, speech and signal processing (pp. 1469–1472). Huang, P. S., Harris, C. J., & Nixon, M. S. (1999). Human gait recognition in canonical space using temporal templates. IEE Proceedings. Vision, Image & Signal Processing, 2(146), 93–100. Kim, D., Kim, D., & Paik, J. (2010). Gait recognition using active shape model and motion prediction. IET Computer Vision, 4(1), 25–36. Lakany, H. (2008). Extracting a diagnostic gait signature. Pattern Recognition, 41(5), 1627–1637. Lam, T. H. W., Cheung, K. H., & Liu, J. N. K. (2011). Gait flow image: a silhouettebased gait representation for human identification. Pattern Recognition, 44(4), 973–987. Lee, L., & Grimson, W. E. L. (2002). Gait analysis for recognition and classification. In Proceedings of the fifth IEEE international conference on automatic face and gesture recognition (pp. 155–161). Lee, H., Hong, S., & Kim, E. (2009). Neural network ensemble with probabilistic fusion and its application to gait recognition. Neurocomputing, 72(7–9), 1557–1564. Liu, T. F., Wang, C., & Hill, D. J. (2009). Learning from neural control of nonlinear systems in normal form. Systems & Control Letters, 58(9), 633–638. Lu, J. W., & Zhang, E. H. (2007). Gait recognition for human identification based on ICA and fuzzy SVM through multiple views fusion. Pattern Recognition Letters, 28(16), 2401–2411. Mu, X. P., & Wu, Q. (2003). A complete dynamic model of five-link bipedal walking. In Proceedings of the 2003 American control conference (pp. 4926–4931).

Murase, H., & Sakai, R. (1996). Moving object recognition in eigenspace representation: gait analysis and lip reading. Pattern Recognition Letters, 17(2), 155–162. Murray, M. P. (1967). Gait as a total pattern of movement. American Journal of Physical Medicine, 46(1), 290–333. Park, J., & Sandberg, I. W. (1991). Universal approximation using radial-basisfunction networks. Neural Computation, 3(2), 246–257. Phillips, P. J., Sarkar, S., Robledo, I., Grother, P., & Bowyer, K. W. (2002). Baseline results for the challenge problem of human ID using gait analysis. In Proceedings of the fifth international conference on automatic face and gesture recognition (pp. 130–135). Tafazzoli, F., & Safabakhsh, R. (2010). Model-based human gait recognition using leg and arm movements. Engineering Applications of Artificial Intelligence, 23(8), 1237–1246. Trivino, G., Alvarez-Alvarez, A., & Bailador, G. (2010). Application of the computational theory of perceptions to human gait pattern recognition. Pattern Recognition, 43(7), 2572–2581. Wagg, D. K., & Nixon, M. S. (2004). On automated model-based extraction and analysis of gait. In Proceedings of the sixth IEEE international conference on automatic face and gesture recognition (pp. 11–16). Wang, D. L. (2003). Temporal pattern processing In The Handbook of Brain Theory and Neural Networks, (2nd ed.) (pp. 1163–1167). Cambridge MA: MIT Press. Wang, C., & Chen, T. R. (2011). Rapid Detection of small oscillation faults via deterministic learning. IEEE Transactions on Neural Networks, 22(8), 1284–1296. Wang, C., Chen, T. R., Chen, G. R., & Hill, D. J. (2009). Deterministic learning of nonlinear dynamical systems. International Journal of Bifurcation and Chaos, 19(4), 1307–1328. Wang, C., & Hill, D. J. (2006). Learning from neural control. IEEE Transactions on Neural Networks, 17(1), 130–146. Wang, C., & Hill, D. J. (2007). Deterministic learning and rapid dynamical pattern recognition. IEEE Transactions on Neural Networks, 18(3), 617–630. Wang, C., & Hill, D. J. (2009). Deterministic learning theory for identification, recognition and control. Boca Raton, FL: CRC Press. Wang, C., & Hill, D. J. (2010). Deterministic learning and nonlinear observer design. Asian Journal of Control, 12(6), 1–11. Wang, L., Ning, H., Hu, W., & Tan, T. (2003). Automatic gait recognition based on statistical shape analysis. IEEE Transactions on Image Processing, 12(9), 1120–1131. Winter, D. A. (1990). Biomechanics and motor control of human movement (2nd ed.). John Wiley & Sons. Yam, C. Y., & Nixon, M. S. (2009). Model-based gait recognition. In Encyclopedia of Biometrics (pp. 633–639). New York: Springer. Yam, C. Y., Nixon, M. S., & Carter, J. N. (2004). Automated person recognition by walking and running via model-based approaches. Pattern Recognition, 37(5), 1057–1072. Yoo, J. H., & Nixon, M. S. (2011). Automated markerless analysis of human gait motion for recognition and classification. ETRI Journal, 33(2), 259–266. Yoo, J.H., Nixon, M.S., & Harris, C.J. (2002). Extracting human gait signatures by body segment properties. In Fifth IEEE southwest symposium on image analysis and interpretation (pp. 35–39). Zhang, R., Vogler, C., & Metaxas, D. (2007). Human gait recognition at sagittal plane. Image and Vision Computing, 25(3), 321–330. Zhao, G. Y., Cui, L., & Li, H. (2007). Gait recognition using fractal scale. Pattern Analysis and Applications, 10(3), 235–246.