- Email: [email protected]
Classification of body movements based on posturographic data
Human Movement Science 33 (2014) 238–250
Contents lists available at ScienceDirect
Human Movement Science journal homepage: www.elsevier.com/locate/humov
Classification of body movements based on posturographic data Sashi K. Saripalle, Gavin C. Paiva, Thomas C. Cliett III, Reza R. Derakhshani, Gregory W. King ⇑, Christopher T. Lovelace 1 University of Missouri – Kansas City, MO, USA
a r t i c l e
i n f o
Article history: Available online 23 November 2013 PsycINFO classifications: 4010 4100 4120 Keywords: Posturography Force platform Feature extraction Pattern recognition Human
a b s t r a c t The human body, standing on two feet, produces a continuous sway pattern. Intended movements, sensory cues, emotional states, and illnesses can all lead to subtle changes in sway appearing as alterations in ground reaction forces and the body’s center of pressure (COP). The purpose of this study is to demonstrate that carefully selected COP parameters and classification methods can differentiate among specific body movements while standing, providing new prospects in camera-free motion identification. Force platform data were collected from participants performing 11 choreographed postural and gestural movements. Twenty-three different displacement- and frequency-based features were extracted from COP time series, and supplied to classification-guided feature extraction modules. For identification of movement type, several linear and nonlinear classifiers were explored; including linear discriminants, nearest neighbor classifiers, and support vector machines. The average classification rates on previously unseen test sets ranged from 67% to 100%. Within the context of this experiment, no single method was able to uniformly outperform the others for all movement types, and therefore a set of movement-specific features and classifiers is recommended. Ó 2013 Elsevier B.V. All rights reserved.
⇑ Corresponding author. Address: 352 R.H. Flarsheim Hall, 5100 Rockhill Road, Kansas City, MO 64110-2499, USA. Tel.: +1 (816) 235 2350; fax: +1 (816) 235 1260. E-mail address: [email protected] (G.W. King). 1 Currently in the Department of Psychology, Shepherd University, Shepherdstown, WV, USA. 0167-9457/$ - see front matter Ó 2013 Elsevier B.V. All rights reserved. http://dx.doi.org/10.1016/j.humov.2013.09.004
S.K. Saripalle et al. / Human Movement Science 33 (2014) 238–250
239
1. Introduction The neuromuscular control system makes continuous adjustments to maintain stable upright posture, a dynamic phenomenon that may be modeled as an inverted pendulum with feedback control (Haibach, Slobounov, Slobounova, & Newell, 2007). The output signal of such a model represents movement, or sway, of the body’s center of pressure (COP) position, which is defined as the point of intersection with the support surface of the body’s net vertical force (Murray, 1967). COP analysis, or posturography, has been used clinically to assess numerous age- and disease-related balance deficiencies such as peripheral neuropathy (Corriveau et al., 2000; Horak, Dickstein, & Peterka, 2002; Lafond, Corriveau, & Prince, 2004), stroke (Corriveau, Hebert, Raiche, & Prince, 2004; Laufer, Schwarzmann, Sivan, & Sprecher, 2005), and Parkinson’s disease (Horak, Dimitrova, & Nutt, 2005; Schmit et al., 2006). Posturography has also been used on a limited basis in movement recognition applications such as Human Computer Interaction (Shneiderman & Plaisant, 2009) and gait recognition (Sarkar et al., 2005). COP position time series are extracted from force platform data and characterized using COP amplitude measures such as sway path, velocity, and area (Diener, Dichgans, Bacher, & Gompf, 1984; Hasan, Lichtenstein, & Shiavi, 1990; Nardone, Tarantola, Giordano, & Schieppati, 1997; Nardone, Tarantola, Galante, & Schieppati, 1998); frequency measures including mean and median power frequencies (Lehmann et al., 1990; Maki, Holliday, & Topper, 1994a; Maki, Holliday, & Topper, 1994b; Wolff et al., 1998); and fractal dynamics measures including short-term and long-term diffusion coefficients and Hurst exponents (Collins & De Luca, 1993, 1995; Wolff et al., 1998; Duarte & Zatsiorsky, 2000; Tanaka, Uetake, Kuriki, & Ikeda, 2002). When viewed from a motor control perspective, the COP represents a performance variable whose stability is maintained by co-varying elemental variables (or muscle modes), which may be combined in different ways to perform specific tasks (Krishnamoorthy, Latash, Scholz, & Zatsiorsky, 2003; Latash, 2008). The task-specific nature of COP control highlights the potential of posturography in a wide variety of applications requiring identification or classification of human movement patterns. However, identifying patterns in COP data is tedious. Previous attempts at COP pattern recognition include template matching (Nakappan, Darbhe, Storey, Robinson, & O’Neal, 2005), statistical classification (Haibach et al., 2007; Santos, Delisle, Lariviere, Plamondon, & Imbeau, 2008), neural networks (Lafuente, Belda, Sanchez-Lacuesta, Soler, & Prat, 1998), and hybrid methods (Headon & Curwen, 2002), and focused on major movements with large variation such as crouching and jumping (Headon & Curwen, 2001, 2002). While these movements were successfully classifiable with force platform data, it is unclear whether similar techniques could be used to differentiate among delicate variations in ground reactions and thus generalized to a broader range of human standing movements, such as gesticulations and fine postural motions. To address the challenges of classifying multiple subtle movements, a detailed study of various feature selection and classification methods is needed. In this paper, we demonstrate a computational intelligence-based solution with an integrated feature selection and classification method that learns from examples and can distinguish among COP patterns of even subtle postural and gestural movements without needing to recalibrate for each subject. Such data driven, subject independent, and camera free motion detection has tremendous implications for applications such as evaluating efficacy of physical therapy interventions, assessing fall risk in balance-impaired patients, and enhancing athletic training activities. The objectives of this study were to (1) determine which COP features are sensitive to differences in movement types using traditional statistical analyses; and (2) build feature vectors and classifiers using machine learning methods to distinguish among these movements irrespective of the user. It is hypothesized that a movement-specific, feature-classifier design will distinguish among movement types with higher predictive power in comparison to traditional statistical methodology. It is also hypothesized that multivariate interactions will need to be considered in the feature selection phase. Completion of these objectives represents a preliminary step in establishing the feasibility of using posturography for human motion recognition.
240
S.K. Saripalle et al. / Human Movement Science 33 (2014) 238–250
Fig. 1. Testing configuration with coordinate directions.
2. Methods 2.1. Participants Fourteen student volunteers participated in the experiment, after providing written informed consent as approved by the Social Sciences Institutional Review Board at the University of Missouri – Kansas City (UMKC). All participants identified themselves as healthy with the ability to stand comfortably for 20 min. 2.2. Experimental protocol During each recording session, participants stood on a force platform (Fig. 1) while performing 11 choreographed head (H), mid-body (MB) and postural (P) movements (Table 1). Each movement was
Table 1 Description of participant movements. Movement
Description
Mv1-H Mv2-H Mv3-MB
Head nodding Head shaking Shoulder shrugging without hand movement Shoulder shrugging with palms turned upwards Touching back of head Touching one’s nose Scratching opposite arm Hands outstretched Shifting weight from one foot to other Shifting weight to tiptoes Tapping foot
Mv4-MB Mv5-MB Mv6-MB Mv7-MB Mv8-MB Mv9-P Mv10-P Mv11-P
S.K. Saripalle et al. / Human Movement Science 33 (2014) 238–250
241
performed continuously for four 10-s trials, resulting in a total of 44 trials for each participant. The order of movement types for the three body regions (H, MB, and P) was randomized across participants; specific movements within each region were performed in the same order by all participants. 2.3. Measurements and variable extraction During each trial, forces and moments in the medial–lateral (ML), anterior–posterior (AP), and vertical directions were sampled at 1000 Hz from the force platform (AMTI; Watertown, MA, USA) connected to a personal computer running Vicon NEXUS software (Vicon Motion Systems; Denver, CO, USA). Following data collection, NEXUS was used to calculate and export COP time series for each trial. COP signals were filtered using a second-order low-pass Butterworth filter with a cutoff frequency of 30 Hz (Carpenter, Frank, Winter, & Peysar, 2001). Initial AP and ML components of the COP (Fig. 2) were set to 0 to remove offset caused by participants shifting position between trials. A literature review was conducted to identify candidate COP features likely to reveal discernibility among the movement types studied (Carpenter et al., 2001; Latash, Ferreira, Wieczorek, & Duarte, 2003; Maurer & Peterka, 2005; Reilly, Van Donkelaar, Saavedra, & Woollacott, 2008; Santos et al., 2008; Vieira, Oliveira, & Nadal, 2009). Based on our review, a total of 23 scalar measures (listed in Table 2 and described below) were selected and extracted from COP time series. 2.3.1. Segment length parameters (1–2) COP segment lengths are the incremental displacements between pairs of points in the COP time series. Segment lengths are used to calculate standard deviation and summed to yield the COP path length. 2.3.2. Sway parameters (3, 4) Sway is the net range of COP motion, computed by taking the difference between maximum and minimum COP positions, in both AP and ML directions. 2.3.3. Displacement based parameters (7–10, 18–19) Displacement is the distance from the mean COP location to each point in the COP time series. Parameters extracted from this data include maximum value, mean value, and standard deviation in the AP and ML directions (Prieto, Myklebust, Hoffmann, Lovett, & Myklebust, 1996).
Fig. 2. Representative COP trajectories of a participant showing 4 trials for each of 11 movements.
242
S.K. Saripalle et al. / Human Movement Science 33 (2014) 238–250
Table 2 List of COP feature candidates, p-values for individual features from ANOVAs and overall feature ranks using univariate (FSA) and multivariate (FSB) classification-based assessments. Feature
p (participant)
p (motion)
R2 (%)
FSA rank
FSB rank
1. Standard deviation (SD) of COP segments 2. Path length 3. Sway – AP direction 4. Sway – ML direction 5. Velocity – Mean 6. Velocity – Max 7. Max displacement – AP 8. Max displacement – ML 9. SD of displacement – AP 10. SD of displacement – ML 11. Max velocity – AP 12. Max velocity – ML 13. SD of velocity – AP 14. SD of velocity – ML 15. Zero crossing rate – ML 16. Mean velocity – AP 17. Mean Velocity – ML 18. Mean displacement – AP 19. Mean displacement – ML 20. Eccentricity 21. Length of major axis of instability 22. Angle of major axis 23. Median frequency
<.001 .07 <.001 <.001 <.001 .26 <.001 1.00 .86 .03 1.00 1.00 .99 .96 .84 .20 .04 .35 .08 <.001 .03 <.001 .36
<.001 <.001 <.001 <.001 <.001 <.001 <.001 <.001 <.001 <.001 <.001 <.001 <.001 <.001 <.001 <.001 <.001 <.001 <.001 <.001 <.001 <.001 <.001
77.77 56.32 93.38 91.44 76.72 54.56 91.55 12.05 66.84 32.52 17.51 13.89 3.15 2.51 60.39 33.80 51.43 11.45 87.57 37.67 14.94 94.64 52.39
10 16 6 12 3 15 8 20 4 17 7 22 1 11 13 18 2 23 5 14 21 9 19
13 3 14 10 18 19 11 2 21 22 15 20 4 5 6 17 1 16 7 8 9 12 23
2.3.4. Velocity based parameters (5–6, 11–14, 16–17) Velocities were calculated by numerical differentiation of the displacement-based parameters. These were used to calculate the maximum value, mean value, and standard deviation with respect to the AP/ML directions and the total magnitude. 2.3.5. Elliptical fit based parameters (20–22) Several features can be calculated from an ellipse covering 85.35% of the sway area (Duarte & Zatsiorsky, 2002). The major axis corresponds to the direction of least stability. The eccentricity of the ellipse relates to the comparative directionality of the COP. 2.3.6. Frequency parameters (15, 23) Median frequency of oscillation is the time frequency at which the integral of spectral power, calculated from the resultant COP vector, is one half the value of the total integral. 2.4. Statistical analysis of COP features (Objective 1) Initial evaluation of COP data was based on standard statistical analysis performed in Minitab 15 (Minitab, State College, PA). An initial multivariate analysis of variance (MANOVA) was performed on the COP variables from the gestural data (Mv1H-Mv2H, Mv3MB-Mv8MB), postural data (Mv9PMv11P), and the combined dataset with movement as a fixed-level factor and participant as a random factor. Results indicated that both movement and participant factors were significant across the COP measures (p < .01) in all three scenarios. To investigate individual feature effects, follow-up analyses were performed including individual analyses of variance (ANOVA) for each COP measure. 2.5. Feature-classifiers for distinguishing among movements (Objective 2) A fourfold cross-validation partitioning scheme was used to identify COP variables best suited to distinguish among movements. For each movement, 75% (3 trials) and 25% (1 trial) of data was used for training and validation, respectively. Cycling through the 4 combinations of training/validation
S.K. Saripalle et al. / Human Movement Science 33 (2014) 238–250
243
partitioning allowed for 4-fold cross validation, thus further increasing the statistical validity of the results. To find the best feature-classifier arrangement with the highest discriminating power, a range of different classifiers were evaluated over subsets of features that were selected based on their individual and collective subject-independent movement identification power. All computations were performed in MATLAB (MathWorks, Natick, MA, USA). 2.5.1. Feature ranking and selection In machine learning, supervised feature selection entails searching among feature candidates to achieve the desired pattern recognition by maximizing a quality metric, such as correct recognition rate or others explained below (Guyon & Elisseeff, 2003), leading to better classification while reducing classifier complexity (Jain & Duin, 2000). This reduction in input space enforces better cause–effect relationships with classification results by preferring more salient input parameters that also positively affect the classifier output. The ensuing simplified classifier is an embodiment of Occam’s razor principle in pattern recognition, where a more compact classifier can better generalize over unseen data by avoiding over-parameterization and thus unpredictable erratic behavior in areas of input space not covered by limited training dataset (Bishop, 2006). One key issue in constructing feature vectors from a pool of candidate scalar features (Table 2) is subset search and selection. Ideally, given D feature candidates, 2D-1 feature vectors need to be evaluated; an impractical choice for many problems (Jain & Duin, 2000; Guyon & Elisseeff, 2003). A group of suboptimal but faster techniques are filter and wrapper methods. Filter methods use univariate feature (UF) ranking, which work best when the features are independent of each other. In our analysis, univariate feature ranking considers each feature separately and measures its seperability power using ROC criteria. The higher the feature’s separability power the better it is ranked (Kohavi & John, 1997; Ruiz, Riquelme, & Aguilar-Ruiz, 2006; Theodoridis & Koutroumbas, 2008). This method is henceforth referred to as FSA. In some cases, features that are not useful when used independently can provide significant performance improvements when combined with other features. Using the aforementioned reasoning, we use a wrapper based feature ranking that we call multivariate feature (MF) ranking analysis, which uses feature interdependencies. This method randomly selects subsets of features that pass a certain classifier threshold and further rank the features based on their frequency of appearance in the subsets pool (Kohavi & John, 1997; Li, Umbach, Terry, & Taylor, 2004). This method is henceforth referred to as FSB. In summary, scalar COP features were individually ranked and then grouped into vectors for best movement detection using FSA and FSB methods (Table 2). Correct classification rates, in conjunction with sensitivity and specificity, were used as movement classification quality metrics (Príncipe, Euliano, & Lefebvre, 2000; Fawcett, 2006; Alpaydin, 2009). One needs to be aware of trivial, degenerate, or null classification in case of unbalanced datasets (Baldi, Brunak, Chauvin, Andersen, & Nielsen, 2000), where the deceivingly high classification rate can only be identified by a very low sensitivity to specificity ratio, or vice versa. Area under a receiver operating characteristic (ROC) curve was used as a filter feature quality metric for initial ranking purposes (Table 3). ROC is the plot of sensitivity versus 1-specificity, an important tool in the characterization of univariate measures of class labels (Fawcett, 2006). Thus ROC AUC was used for FSA and k-NN was used for the ranking stages of FSB Table 4. For FSB, a pool and subset size of 10, and a subset selection threshold of 0.7 (training correct rate) were experimentally deemed to be the best choices in terms of overall results and computational footprint. 2.5.2. Classifiers Linear Discriminant Analysis (LDA) with Fisher’s separability criterion (Duda, Hart, & Stork, 2001) is a linear supervised classification and dimensionality reduction method. Fischer LDA cast multidimensional features into a single dimension using a linear mixture for best classification. Compared to other linear dimensionality reduction methods such as principle component analysis, Fischer LDA projections may be better suited to classification. Given their success in similar scenarios (Fioretti, Scocco, Ladislao, Ghetti, & Rabini, 2010) and their simple but robust linear structure, LDAs were used as the baseline classic linear classifiers. In this study, each movement was designated its own LDA as a one-versus-rest (OVR) classifier. Each movement-specific OVR LDA receives its designated input feature from the feature selection stage.
244
S.K. Saripalle et al. / Human Movement Science 33 (2014) 238–250
The k nearest neighbor algorithm, or k-NN, classifies a data point by assigning it the label most frequently represented among its k nearest training data points (Bishop, 2006). k-NNs have a low computational footprint and have been successfully applied to related problems (Liu, Lee, & Lin, 2010). However, k-NN is sensitive to outliers and noise, and thus it was used as one of several classifier candidates. For this study, the algorithm performed best with k = 1 when testing over k values ranging from 1 to 10. This result for k could be attributed to the limited size of the dataset, especially the number of positive exemplars for each movement, and the fact that our choreographed dataset was not affected with undue noise and artifacts. The nearest neighbors were found using the Euclidean distance measure. Since k-NN decisions are binary, sensitivity and specificity, in addition to simple correct rate, are used as k-NN quality metrics. Support Vector Machines (SVM) can classify input patterns nonlinearly and with maximum separation (Vapnik, 2000; Bishop, 2006). SVMs have been successfully applied to COP classification problems (Moustakidis, Theocharis, & Giakas, 2008). Popular SVM kernels include Gaussian and polynomial functions. Both kernels, with a range of different variances and orders, were used in this study. SVMs’ tolerance towards noise and outliers is controlled by a cost parameter (C). Choice of C is important: smaller values allow for more tolerance towards outliers but also cause misclassifications, whereas larger C values push the SVM towards a strict solution which may cause over-training, i.e., memorization of training data (including noise and outliers) at the expense of correct classification of unseen input (poor generalization). The SVM objective function is quadratic and convex, so its solution will be unique and global; thus it does not suffer from local optima traps encountered by other nonlinear classifiers such as neural networks. For both polynomial and Gaussian SVM kernels, the C-parameter was varied from 0.01 to 200. For the Gaussian kernel, the sigma (spread) value was varied from 0.1 to 50. For the polynomial kernel, the order was varied from 2 to 8. It was observed that a Cparameter of 10 and a sigma value of 1; and a C parameter of 0.09 and a polynomial order of 4 provided better results for Gaussian and Polynomial SVMs, respectively. Again, for each movement, scalar features were sequentially combined from the top of their ranked list until the maximum fourfold cross validation SVM classification rates were achieved. Ranked lists from FSA were eventually used here as they provided better results (Tables 5 and 6). 3. Results 3.1. Statistical analysis of COP features (Objective 1) ANOVAs performed on individual COP features indicated that movement produced a significant effect in all features. The analysis also revealed a wide range in ANOVA performance as R2 values ranged from 2.51% to 91.55% (Table 2). An inspection of the p-values (Table 2) revealed the existence of many movement-specific but participant-independent COP features and thus suggests the potential of feature-classifier analysis, as achieved in Objective 2.
Table 3 Input feature vectors (FSB) and classification results using the nearest neighbor method. Movement
Features
Correct rate
Sensitivity
Specificity
Mv1-H Mv2-H Mv3-MB Mv4-MB Mv5-MB Mv6-MB Mv7-MB Mv8-MB Mv9-P Mv10-P Mv11-P
[20 21 3 8 19 6 15 16 2 4 5 9 13 17 18 22 7 11 1 14 23 10] [20 9 7 4 18 22 3 5 6 8 19] [14 17 21 5 8 1 9 12 22 23 2] [1 19 3 8 15 7] [5 20 1 12 15 17 21 23 2 3 4 6 9 11 13 16 18 8] [17 15 19 4 7 8 10 6 1 3 5] [6 19 3 8] [11 1 9 3 4 7 8 15 2 22 14 23] [6 10 12 16 20 1 2 7 8 9] [4 7 17 21 1] [3 10 1 5 9 19]
0.8750 0.8669 0.9773 0.9010 0.8766 0.8945 0.8961 0.8815 1.0000 1.0000 1.0000
0.9357 0.9196 1.0000 0.9393 0.9250 0.9304 0.9518 0.9500 1.0000 1.0000 1.0000
0.2321 0.3393 0.7500 0.5179 0.3929 0.5357 0.3393 0.1964 1.0000 1.0000 1.0000
245
S.K. Saripalle et al. / Human Movement Science 33 (2014) 238–250 Table 4 Input feature vectors (FSB) and classification using LDA. Movements
Features
Correct rate
Sensitivity
Specificity
Mv1-H Mv2-H Mv3-MB Mv4-MB Mv5-MB Mv6-MB Mv7-MB Mv8-MB Mv9-P Mv10-P Mv11-P
[20 21 3 8 19 6 15 16] [20 9 7 4 18 22 3 5 6 8 19] [14 17 21 5 8] [1 19 3 8 15 7 11 14 16 21 5 6 9 10 22 2 4 12 17 18 23 13] [5 20 1 12 15 17 21 23 2 3 4 6] [17 15 19 4 7 8 10 6 1 3 5 9] [6 19 3 8 11 16 2 4 7 9 18 22 5 12 14 21 15 17 20 1 23] [11 1 9 3 4 7 8 15 2 22 14 23 6 10 13 18 19 17] [6 10 12 16 20 1 2 7 8 9] [4 7 17] [3 10 1 5 9 19 20 21 7 14 17 2 4 13 15 23]
0.6315 0.6136 0.9773 0.5682 0.8247 0.7565 0.6396 0.6623 1.0000 0.9984 0.9951
0.6071 0.5821 1.0000 0.5393 0.8357 0.7518 0.6214 0.6625 1.0000 1.0000 1.0000
0.8750 0.9286 0.7500 0.8571 0.7143 0.8063 0.8214 0.6607 1.0000 0.9821 0.9464
3.2. Feature-classifiers to distinguish among movements (Objective 2) Table 2 shows the cumulative ranking results for approaches FSA and FSB, where feature candidates are ranked according to their univariate or multivariate discrimination power over all 11 movements. It should be noted that rankings are movement-dependent; however for brevity only the cumulative overall rankings across all 11 movements are shown. Movement-specific ranked lists, in conjunction with feedback from a variety of classifiers, were used in all the forthcoming OVR wrapper methods to aggregate COP features into movement and classifier-specific input vectors. For each movement, scalar features were sequentially combined from the top of the corresponding ranked list until the maximum fourfold cross validation classification rates were achieved. As shown in Table 3, as well as in the following tables exhibiting LDA and SVM results, no single method can provide the best results for all 11 movements. Also, given the sensitivity to specificity disparities, some classification results, such as those for movements 1 and 8, are unacceptable. LDA results are shown in Table 4. Similarly, scalar features were sequentially combined in order from the top of each movement-specific FSB ranked list until the maximum fourfold cross validation classification rates were achieved. Compared to k-NN, a lessened disparity between sensitivity and specificity, despite the seemingly lower correct rates, is observed. Considering the mixed performance of classifiers across different movements, we chose the feature-classifier configurations that best detect each individual movement. This was done by comparing all the corresponding rows from Tables 3–6, and selecting simplest models with better correct classification rates; and balanced and acceptable sensitivities and specificities to avoid trivial classification. The resulting heterogeneous classifier bank (Table 7) is best suited to classify all movements.
Table 5 Input feature vectors (FSA) and classification using Gaussian kernel SVM classifier. Movement
Features
Correct rate
Sensitivity
Specificity
Mv1-H Mv2-H Mv3-MB Mv4-MB Mv5-MB Mv6-MB Mv7-MB Mv8-MB Mv9-P Mv10-P Mv11-P
[13 11 3 9 7 5 17 2 19 22 4 15 6 1 14 10 20 18 8 21 23] [1 9 19 3 7 13 11 17 5 2 22 4 15 14 6 10 20] [14 8] [9 7 3 6 16 22 11 13 20 15 4 19 17 14 18 12 21] [20 17 19 5 1 22 15 4 23 8 6 14 13 21 9 3 10 16 7 2 12] [1 20 5 19 17 15 4 6 22 10 14 11 2 23 8 13 21 18 12] [19 17 5 1 13 4 11 15 9 22 2 3 7 14 16 6 23 12 18 21 10] [3 7 9 13 18 23 8 5 12 11 6 17 16 20 2 15 14 19 4 1] [9 3] [15 4] [1 10 11 16]
0.8003 0.8133 0.9773 0.8490 0.8636 0.8880 0.8198 0.8506 1.0000 0.9984 0.9984
0.8268 0.8250 1.0000 0.8679 0.9018 0.9232 0.8571 0.8857 1.0000 0.9982 0.9982
0.5357 0.6964 0.7500 0.6607 0.4821 0.5357 0.4464 0.5000 1.0000 1.0000 1.0000
246
S.K. Saripalle et al. / Human Movement Science 33 (2014) 238–250
Table 6 Input feature vectors (FSA) and classification using Polynomial kernel SVM classifier. Movement
Features
Correct rate
Sensitivity
Specificity
Mv1-H Mv2-H Mv3-MB Mv4-MB Mv5-MB Mv6-MB Mv7-MB Mv8-MB Mv9-P Mv10-P Mv11-P
[13 11 3 9 7 5 17 2 19 22 4 15 6 1 14 6] [1 9 19 3 7 13 11 17 5 2 22 4 15] [14 8] [9 7 3 6 16 22 11 13 20 15 4 19 17 14] [20 17 19 5 1 22 15 4 23 8 6 14 13 21 9 3 10 16] [1 20 5 19 17 15 4 6 22 10] [19 17 5 1 13 4 11 15 9 22 2 3 7 14 16] [3 7 9 13 18 23 8 5 12 11 6 17 16 20] [9 3] [15 4 14 19 17] [1 10 11 16 5]
0.6697 0.7711 0.9773 0.8377 0.8555 0.8263 0.7179 0.7971 1.0000 1.0000 1.0000
0.6268 0.7768 1.0000 0.8429 0.8804 0.8339 0.7054 0.8943 1.0000 1.0000 1.0000
0.8750 0.7143 0.7500 0.7857 0.6071 0.7500 0.8393 0.6250 1.0000 1.0000 1.0000
4. Discussion This work successfully establishes a method for distinguishing among sway patterns produced during different types of body movements using force plate signals. Because body sway may be modulated by many factors, force platform records have proven to be sensitive indicators of balance dysfunction, for example as a result of aging, (Maki et al., 1994a, 1994b) as well as psychological states such as emotion (Azevedo et al., 2005) and arousal (Maki & McIlroy, 1996). In general, our wrapperbased technique provides sensitive and specific identification of movements that modulate body sway, including subtle gestures, and shows promise as an alternative or augmentative measure to commonly used video methods. Preliminary statistical analyses demonstrate that all COP features have at least some power to differentiate among movements (Objective 1). The features exhibiting a good statistical fit in the ANOVA model tended to have significant participant specificity, suggesting the potential utility of force platform technology for biometric personal identification (Jain, Bolle, & Pankanti, 1999) in addition to the numerous possible clinical applications mentioned in this paper. This may be caused, in part, by treating the movement as a fixed level factor and the participant as a random factor, which are relatively crude approximations as different movements do not correspond to distinct levels of a single variable. Table 7 Final feature-classifier selections with their test results. Movement
Ranking
Features
Correct rate
Sensitivity
Specificity
Classifier
Mv1-H
FSA
[13 11 3 9 7 5 17 2 19 22 4 15 6 1 14 6]
0.6697
0.6268
0.8750
Mv2-H
FSA
[1 9 19 3 7 13 11 17 5 2 22 4 15 14 6 10 20]
0.8133
0.8250
0.6964
Mv3-MB
FSA
[14 8]
0.9773
1.0000
0.7500
Mv4-MB
FSA
[9 7 3 6 16 22 11 13 20 15 4 19 17 14]
0.8377
0.8429
0.7857
Mv5-MB Mv6-MB
FSB FSA
[5 20 1 12 15 17 21 23 2 3 4 6] [1 20 5 19 17 15 4 6 22 10]
0.8247 0.8263
0.8357 0.8339
0.7143 0.7500
Mv7-MB
FSA
[19 17 5 1 13 4 11 15 9 22 2 3 7 14 16]
0.7179
0.7054
0.8393
Mv8-MB
FSA
[3 7 9 13 18 23 8 5 12 11 6 17 16 20]
0.7971
0.8943
0.6250
Mv9-P
FSA
[9 3]
1.0000
1.0000
1.0000
Mv10-P
FSB
[4 7 17 21 1]
1.0000
1.0000
1.0000
Mv11-P
FSA
[1 10 11 16 5]
1.0000
1.0000
1.0000
Polynomial kernel SVM Gaussian kernel SVM Gaussian kernel SVM Polynomial kernel SVM LDA Polynomial kernel SVM Polynomial kernel SVM Polynomial kernel SVM Polynomial Kernel SVM Nearest Neighbor Polynomial Kernel SVM
S.K. Saripalle et al. / Human Movement Science 33 (2014) 238–250
247
Additionally, the p values derived from the participant factor data may be skewed due to performing movements in the same consecutive order. Finally, assumptions regarding normal distributions and linear and independent effects are only partially met with these statistical methods. With the understanding that other methods such as those used in Objective 2 are better equipped to address these problems, traditional statistical methodology was not pursued further. The machine learning methods were capable of meeting Objective 2 to varying extents. FSB generally provided better performance for most classifiers, most likely due to its ability to reflect some multivariate interactions. As expected, our results suggest that all classifiers performed well for the more pronounced postural movements Mv9-P through Mv11-P. However, the figures were mixed and generally not as high for more subtle gestural movements Mv1-H through Mv8-MB. Nearest neighbor classifiers performed well with an average fourfold cross validation correct rate, sensitivity, and specificity of 0.92, 0.96, and 0.57, respectively (Table 3), but the disparity between sensitivity and specificity figures makes kNNs less desirable. LDAs followed closely but with more balanced sensitivity to specificity ratios, yielding an average fourfold cross validation correct rate, sensitivity, and specificity of 0.79, 0.78, and 0.85, respectively (Table 4). SVM models performed better than other methods for most movements, with an average fourfold cross validation correct rate, sensitivity, and specificity of 0.88, 0.89, and 0.75; respectively (Tables 5 and 6). SVMs’ better performance, especially their more balanced sensitivity and specificity figures, may be attributed to their robust nonlinear classification capabilities. Neural Networks were also considered as another nonlinear method for feature selection and classification based on their universal data-driven discrimination capabilities (Haykin, 2009) and their reported success in force plate signal identification (Lafuente et al., 1998). However, neural networks were matched or outperformed by other methods, so were not pursued further. This is likely due to the different and subtle nature of movements in our dataset, in addition to lack of enough exemplars of each target class when compared to network sizes. When considering gestural movements, there was no single classifier or feature set that could outperform the rest when considering sensitivity and specificity in addition to correct rate, as the latter by itself can be a misleading figure in unbalanced, multi-class datasets. Thus, using the best feature-classifier combinations from all the above methods, a heterogeneous bank of 11 OVR classifiers was formed. To that end, the data from the different feature-classifier combinations (Tables 3–6) were pooled to yield the best selection by considering sensitivity, specificity, and overall gestural rates (Table 7). This observation calls for movement-specific feature sets and classifier designs as the best approach for Objective 2. Further study is required to fully understand, from a physiological perspective, why different movements are best classified with such a varied range of feature sets. It is clear, however, that the features exhibiting the largest differences among movement types are not necessarily the same features that are best suited for classification. For example, Mv9-P involves shifting weight from one foot to the other; while this involves more medial–lateral COP movement than the other tasks, it is the anterior–posterior COP characteristics (standard deviation of displacement and maximum sway) that offer optimum separability (maximum separation with minimum spread) from the other movements. Similarly, Mv10-P, which required anterior–posterior weight shifts, was classified by several medial– lateral COP characteristics. Since Mv9-P and Mv10-P were task that may have required more cognitive attention than the others, it is possible that voluntarily increasing sway in one direction was done at the expense of sway regulation in the orthogonal direction, a phenomenon that has been reported elsewhere (Woollacott & Shumway-Cook, 2002; Tucker, Kavanagh, Morrison, & Barrett, 2009). It is also possible that our faster but suboptimal filter and wrapper feature selection techniques identified local minima, resulting in different features than those that might have been identified in an exhaustive search. Further study is needed to determine whether ‘‘brute force’’ feature selection is successful in identifying features sets that are more homogeneous across movement types. A further limitation of our study may relate to the types of COP features studied. Due to the preliminary nature of this work, we chose a fairly traditional set of amplitude- and frequency-related COP descriptors. However, more sophisticated time series analyses such as stabilogram-diffusion analysis (Collins & De Luca, 1993) and detrended fluctuation analysis (Duarte & Zatsiorsky, 2001) are focused on temporal correlations of COP data. These analyses have revealed persistent and antipersistent behavior over short and long time intervals, respectively, suggesting open- and closed-loop
248
S.K. Saripalle et al. / Human Movement Science 33 (2014) 238–250
postural control. As with traditional COP parameters, task- and condition-specific changes in SDA and DFA parameters have been observed in a variety of applications such as aging and disease state (e.g., Peterka, 2000; Lin, Seol, Nussbaum, & Madigan, 2008). It is therefore likely that such temporal parameters would be candidates for the COP classification techniques described here, and should be considered in future work. Because force platforms can conveniently and precisely measure sway and other motor related time series, our approach can be advantageous when fast, unobtrusive, and automated evaluation of posturographic information is needed. For applications requiring capture of body movements, force platforms may address some of the limitations of video analysis. For instance, conventional video cannot discern small or occluded movements, and face challenges such as non-ideal lighting in real world scenarios. Our methodology has potential diagnostic utility in a wide range of movement- and balance-related pathological conditions such as peripheral neuropathy (Corriveau et al., 2000; Horak et al., 2002; Lafond et al., 2004), stroke (Corriveau et al., 2004; Laufer et al., 2005), and Parkinson’s disease (Horak et al., 2005; Schmit et al., 2006). Human–computer interaction (Shneiderman & Plaisant, 2009) is another area in which force platforms could be used as an input in applications such as smart homes, video games, and assistive devices (Orr & Abowd, 2000; Betker, Szturm, Moussavi, & Nett, 2006). Yet another potential application comes from the area of deception detection. While it has been suggested that elements of gesture and posture reflect deceptive intention (Frank & Ekman, 1997; Vrij, 2000), analysis of this information is usually done with video records, which are subject to the limitations discussed above. Force plate records, coupled with classifier analysis, could provide a more sensitive, automated indicator of postural correlates of deception or other cognitive states associated with postural changes. In future studies it may be advisable to explore additional data preprocessing and signal conditioning options. For instance, the cutoff frequency of the low pass filter applied to the force plate signals may dynamically change (e.g., Woltring filter) to better accommodate the increased frequency content around impact regions. Other signal preprocessing options include Kalman filtering to reduce noise in feature values. Future work may include COP features not explored in the current study, such as Romberg ratio, fractal dimensions, and diffusion constants (Mientjes & Frank, 1999; Santos et al., 2008).
5. Conclusion We applied traditional statistical analyses and modern machine learning techniques to detect differences in sway patterns associated with a variety of body movements captured by ground-embedded force platforms. Our method successfully distinguished among COP patterns associated with various large and small human movements. Classification-guided subset selection provided feature sets that, in conjunction with different classifiers, provided recognition rates of 67–100%, with an average of 92%, across all 11 movements (Table 7). Given sensitivity and specificity figures, none of the classifiers suffer from trivial classification (very low sensitivity). Classification rates, sensitivity, and specificity figures for different feature-classifier combinations varied for each movement, suggesting that use of a single model or feature set may not be sufficient for identification of all the movements studied. Linear classifiers performed rather poorly, suggesting that class boundaries among movements are nonlinear. SVMs had the highest overall performance of any individual classifier type on the grounds of highest sensitivity and specificity, especially for more subtle gestural movements. Since each movement required a specific classifier with its distinct multivariate feature set to achieve acceptable sensitivity, specificity, and classification rate, a bank of movement-specific features along with their matched OVR classifiers is better suited for identification of body movement based on force plate data.
Acknowledgements We wish to thank Dr. Judee Burgoon and her team at University of Arizona for their assistance and support, as well as Laxmi S. Gajjala for his contributions to the manuscript.
S.K. Saripalle et al. / Human Movement Science 33 (2014) 238–250
249
References Alpaydin, E. (2009). Introduction to machine learning. The MIT Press [Chapter]. Azevedo, T. M., Volchan, E., Imbiriba, L. A., Rodrigues, E. C., Oliveira, J. M., Oliveira, L. F., et al (2005). A freezing-like posture to pictures of mutilation. Psychophysiology, 42, 255–260. Baldi, P., Brunak, S., Chauvin, Y., Andersen, C. A., & Nielsen, H. (2000). Assessing the accuracy of prediction algorithms for classification: An overview. Bioinformatics, 16, 412–424. Betker, A. L., Szturm, T., Moussavi, Z. K., & Nett, C. (2006). Video game-based exercises for balance rehabilitation: A singlesubject design. Archives of Physical Medicine and Rehabilitation, 87, 1141–1149. Bishop, C. M. (2006). Pattern recognition and machine learning. New York: Springer [Chapter]. Carpenter, M. G., Frank, J. S., Winter, D. A., & Peysar, G. W. (2001). Sampling duration effects on centre of pressure summary measures. Gait & Posture, 13, 35–40. Collins, J. J., & De Luca, C. J. (1993). Open-loop and closed-loop control of posture: A random-walk analysis of center-of-pressure trajectories. Experimental Brain Research, 95, 308–318. Collins, J. J., & De Luca, C. J. (1995). Upright, correlated random walks: A statistical-biomechanics approach to the human postural control system. Chaos, 5, 57–63. Corriveau, H., Hebert, R., Raiche, M., & Prince, F. (2004). Evaluation of postural stability in the elderly with stroke. Archives of Physical Medicine & Rehabilitation, 85, 1095–1101. Corriveau, H., Prince, F., Hebert, R., Raiche, M., Tessier, D., Maheux, P., et al (2000). Evaluation of postural stability in elderly with diabetic neuropathy. Diabetes Care, 23, 1187–1191. Diener, H. C., Dichgans, J., Bacher, M., & Gompf, B. (1984). Quantification of postural sway in normals and patients with cerebellar diseases. Electroencephalogr Clin Neurophysiol, 57, 134–142. Duarte, M., & Zatsiorsky, V. M. (2000). On the fractal properties of natural human standing. Neuroscience Letters, 283, 173–176. Duarte, M., & Zatsiorsky, V. M. (2001). Long-range correlations in human standing. Physics Letters A, 283, 124–128. Duarte, M., & Zatsiorsky, V. (2002). Effects of body lean and visual information on the equilibrium maintenance during stance. Experimental Brain Research, 146, 60–69. Duda, R. O., Hart, P. E., & Stork, D. G. (2001). Pattern classification. Wiley [Chapter]. Fawcett, T. (2006). An introduction to ROC analysis. Pattern Recognition Letters, 27, 861–874. Fioretti, S., Scocco, M., Ladislao, L., Ghetti, G., & Rabini, R. A. (2010). Identification of peripheral neuropathy in type-2 diabetic subjects by static posturography and linear discriminant analysis. Gait & Posture, 32, 317–320. Frank, M. G., & Ekman, P. (1997). The ability to detect deceit generalizes across different types of high-stake lies. Journal of Personality & Social Psychology, 72, 11p. Guyon, I., & Elisseeff, André (2003). An introduction to variable and feature selection. Journal of Machine Learning Research, 3, 1157–1182. Haibach, P. S., Slobounov, S. M., Slobounova, E. S., & Newell, K. M. (2007). Virtual time-to-contact of postural stability boundaries as a function of support surface compliance. Experimental Brain Research, 177, 471–482. Hasan, S. S., Lichtenstein, M. J., & Shiavi, R. G. (1990). Effect of loss of balance on biomechanics platform measures of sway: Influence of stance and a method for adjustment. Journal of Biomechanics, 23, 783–789. Haykin, S. S. (2009). Neural Networks and Learning Machines. USA: Prentice Hall [Chapter]. Headon, R. & Curwen, R. (2001). Recognizing movements from the ground reaction force. In: Proceedings of the 2001 workshop on Perceptive user interfaces (pp. 1–8). Orlando, Florida: ACM. Headon, R., & Curwen, R. (2002). Movement awareness for ubiquitous game control. Personal Ubiquitous Computing, 6, 407–415. Horak, F. B., Dickstein, R., & Peterka, R. J. (2002). Diabetic neuropathy and surface sway-referencing disrupt somatosensory information for postural stability in stance. Somatosensory & Motor Research, 19, 316–326. Horak, F. B., Dimitrova, D., & Nutt, J. G. (2005). Direction-specific postural instability in subjects with Parkinson’s disease. Experimental Neurology, 193, 504–521. Jain, A. K., Bolle, R., & Pankanti, S. (1999). Biometrics: Personal identification in networked society. Kluwer Academic Publishers [Chapter]. Jain, A. K., & Duin, R. P. W. (2000). Statistical pattern recognition: A review. IEEE Transactions on Pattern Analysis & Machine Intelligence, 22, 34p. Kohavi, R., & John, G. H. (1997). Wrappers for feature subset selection. Artificial Intelligence, 97, 273–324. Krishnamoorthy, V., Latash, M. L., Scholz, J. P., & Zatsiorsky, V. M. (2003). Muscle synergies during shifts of the center of pressure by standing persons. Experimental Brain Research, 152, 281–292. Lafond, D., Corriveau, H., & Prince, F. (2004). Postural control mechanisms during quiet standing in patients with diabetic sensory neuropathy. Diabetes Care, 27, 173–178. Lafuente, R., Belda, J. M., Sanchez-Lacuesta, J., Soler, C., & Prat, J. (1998). Design and test of neural networks and statistical classifiers in computer-aided movement analysis: A case study on gait analysis. Clinical Biomechanics (Bristol, Avon), 13, 216–229. Latash, M. L. (2008). Neurophysiological basis of movement (2nd ed.). Champaign: Human Kinetics [Chapter]. Latash, M. L., Ferreira, S. S., Wieczorek, S. A., & Duarte, M. (2003). Movement sway: Changes in postural sway during voluntary shifts of the center of pressure. Experimental Brain Research, 150, 314–324. Laufer, Y., Schwarzmann, R., Sivan, D., & Sprecher, E. (2005). Postural control of patients with hemiparesis: Force plates measurements based on the clinical sensory organization test. Physiotherapy Theory & Practice, 21, 163–171. Lehmann, J. F., Boswell, S., Price, R., Burleigh, A., deLateur, B. J., Jaffe, K. M., et al (1990). Quantitative evaluation of sway as an indicator of functional balance in post-traumatic brain injury. Archives of Physical Medicine and Rehabilitation, 71, 955–962. Li, L., Umbach, D. M., Terry, P., & Taylor, J. A. (2004). Application of the GA/KNN method to SELDI proteomics data. Bioinformatics, 20, 1638–1640. Lin, D., Seol, H., Nussbaum, M. A., & Madigan, M. L. (2008). Reliability of COP-based postural sway measures and age-related differences. Gait & Posture, 28, 337–342.
250
S.K. Saripalle et al. / Human Movement Science 33 (2014) 238–250
Liu, C. L., Lee, C. H., & Lin, P. M. (2010). A fall detection system using k-nearest neighbor classifier. Expert Systems with Applications, 37, 7174–7181. Maki, B. E., Holliday, P. J., & Topper, A. K. (1994a). A prospective study of postural balance and risk of falling in an ambulatory and independent elderly population. The Journals of Gerontology. Series A, Biological Sciences and Medical Sciences, 49, M72–84. Maki, B. E., Holliday, P. J., & Topper, A. K. (1994b). A prospective study of postural balance and risk of falling in an ambulatory and independent elderly population. Journal of Gerontology, 49, M72–84. Maki, B. E., & McIlroy, W. E. (1996). Influence of arousal and attention on the control of postural sway. Journal of Vestibular Research, 6, 53–59. Maurer, C., & Peterka, R. J. (2005). A new interpretation of spontaneous sway measures based on a simple model of human postural control. Journal of Neurophysiology, 93, 189–200. Mientjes, M. I., & Frank, J. S. (1999). Balance in chronic low back pain patients compared to healthy people under various conditions in upright standing. Clinical Biomechanics (Bristol, Avon), 14, 710–716. Moustakidis, S. P., Theocharis, J. B., & Giakas, G. (2008). Subject recognition based on ground reaction force measurements of gait signals. IEEE Transactions on Systems, Man, and Cybernetics. Part B, Cybernetics, 38, 1476–1485. Murray, M. P. (1967). Gait as a total pattern of movement. American Journal of Physical Medicine, 46, 290–333. Nakappan, S., Darbhe, V. A., Storey, C. M., Robinson, C., & O’Neal, K. K. (2005). Variations in anterior–posterior CoP patterns in elderly adults between psychophysically detected and non-detected short horizontal perturbations. In Proceedings of the IEEE engineering in medicine and biology society annual international conference (Vol. 5, pp. 5427–5430). Nardone, A., Tarantola, J., Galante, M., & Schieppati, M. (1998). Time course of stabilometric changes after a strenuous treadmill exercise. Archives of Physical Medicine and Rehabilitation, 79, 920–924. Nardone, A., Tarantola, J., Giordano, A., & Schieppati, M. (1997). Fatigue effects on body balance. Electroencephalogr Clin Neurophysiol, 105, 309–320. Orr, R. J. & Abowd, G. D. (2000). The smart floor: A mechanism for natural user identification and tracking. In: CHI ‘00 extended abstracts on Human factors in computing systems. The Hague, The Netherlands, ACM: 275–276. Peterka, R. J. (2000). Postural control model interpretation of stabilogram diffusion analysis. Biological Cybernetics, 82, 335–343. Prieto, T. E., Myklebust, J. B., Hoffmann, R. G., Lovett, E. G., & Myklebust, B. M. (1996). Measures of postural steadiness: Differences between healthy young and elderly adults. IEEE Transactions on Bio-medical Engineering, 43, 956–966. Príncipe, J. C., Euliano, N. R., & Lefebvre, W. C. (2000). Neural and adaptive systems: Fundamentals through simulations. Wiley [Chapter]. Reilly, D. S., Van Donkelaar, P., Saavedra, S., & Woollacott, M. H. (2008). Interaction between the development of postural control and the executive function of attention. Journal of Motor Behavior, 40, 90–102. Ruiz, R., Riquelme, J. C., & Aguilar-Ruiz, J. S. (2006). Incremental wrapper-based gene selection from microarray data for cancer classification. Pattern Recognition, 39, 2383–2392. Santos, B. R., Delisle, A., Lariviere, C., Plamondon, A., & Imbeau, D. (2008). Reliability of centre of pressure summary measures of postural steadiness in healthy young adults. Gait and Posture, 27, 408–415. Sarkar, S., Phillips, P. J., Liu, Z., Vega, I. R., Grother, P., & Bowyer, K. W. (2005). The humanID gait challenge problem: Data sets, performance, and analysis. IEEE Transactions on Pattern Analysis and Machine Intelligence, 27, 162–177. Schmit, J. M., Riley, M. A., Dalvi, A., Sahay, A., Shear, P. K., Shockley, K. D., et al (2006). Deterministic center of pressure patterns characterize postural instability in Parkinson’s disease. Experimental Brain Research, 168, 357–367. Shneiderman, B., & Plaisant, C. (2009). Designing the user interface: Strategies for effective human–computer interaction. AddisonWesley [Chapter]. Tanaka, H., Uetake, T., Kuriki, S., & Ikeda, S. (2002). Changes in center-of-pressure dynamics during upright standing related to decreased balance control in young adults: Fractional Brownian motion analysis. Journal of Human Ergology, 31, 1–11. Theodoridis, S., & Koutroumbas, K. (2008). Pattern recognition. Elsevier/Academic Press [Chapter]. Tucker, M. G., Kavanagh, J. J., Morrison, S., & Barrett, R. S. (2009). Voluntary sway and rapid orthogonal transitions of voluntary sway in young adults, and low and high fall-risk older adults. Clinical Biomechanics (Bristol, Avon), 24, 597–605. Vapnik, V. N. (2000). The Nature of Statistical Learning Theory. Springer [Chapter]. Vieira, T. M., Oliveira, L. F., & Nadal, J. (2009). Estimation procedures affect the center of pressure frequency analysis. Brazilian Journal of Medical and Biological Research, 42, 665–673. Vrij, A. (2000). Detecting lies and deceit: The psychology of lying and the implications for professional practice. John Wiley [Chapter]. Wolff, D. R., Rose, J., Jones, V. K., Bloch, D. A., Oehlert, J. W., & Gamble, J. G. (1998). Postural balance measurements for children and adolescents. Journal of Orthopaedic Research, 16, 271–275. Woollacott, M., & Shumway-Cook, A. (2002). Attention and the control of posture and gait: A review of an emerging area of research. Gait and Posture, 16, 1–14.
Contents lists available at ScienceDirect
Human Movement Science journal homepage: www.elsevier.com/locate/humov
Classification of body movements based on posturographic data Sashi K. Saripalle, Gavin C. Paiva, Thomas C. Cliett III, Reza R. Derakhshani, Gregory W. King ⇑, Christopher T. Lovelace 1 University of Missouri – Kansas City, MO, USA
a r t i c l e
i n f o
Article history: Available online 23 November 2013 PsycINFO classifications: 4010 4100 4120 Keywords: Posturography Force platform Feature extraction Pattern recognition Human
a b s t r a c t The human body, standing on two feet, produces a continuous sway pattern. Intended movements, sensory cues, emotional states, and illnesses can all lead to subtle changes in sway appearing as alterations in ground reaction forces and the body’s center of pressure (COP). The purpose of this study is to demonstrate that carefully selected COP parameters and classification methods can differentiate among specific body movements while standing, providing new prospects in camera-free motion identification. Force platform data were collected from participants performing 11 choreographed postural and gestural movements. Twenty-three different displacement- and frequency-based features were extracted from COP time series, and supplied to classification-guided feature extraction modules. For identification of movement type, several linear and nonlinear classifiers were explored; including linear discriminants, nearest neighbor classifiers, and support vector machines. The average classification rates on previously unseen test sets ranged from 67% to 100%. Within the context of this experiment, no single method was able to uniformly outperform the others for all movement types, and therefore a set of movement-specific features and classifiers is recommended. Ó 2013 Elsevier B.V. All rights reserved.
⇑ Corresponding author. Address: 352 R.H. Flarsheim Hall, 5100 Rockhill Road, Kansas City, MO 64110-2499, USA. Tel.: +1 (816) 235 2350; fax: +1 (816) 235 1260. E-mail address: [email protected] (G.W. King). 1 Currently in the Department of Psychology, Shepherd University, Shepherdstown, WV, USA. 0167-9457/$ - see front matter Ó 2013 Elsevier B.V. All rights reserved. http://dx.doi.org/10.1016/j.humov.2013.09.004
S.K. Saripalle et al. / Human Movement Science 33 (2014) 238–250
239
1. Introduction The neuromuscular control system makes continuous adjustments to maintain stable upright posture, a dynamic phenomenon that may be modeled as an inverted pendulum with feedback control (Haibach, Slobounov, Slobounova, & Newell, 2007). The output signal of such a model represents movement, or sway, of the body’s center of pressure (COP) position, which is defined as the point of intersection with the support surface of the body’s net vertical force (Murray, 1967). COP analysis, or posturography, has been used clinically to assess numerous age- and disease-related balance deficiencies such as peripheral neuropathy (Corriveau et al., 2000; Horak, Dickstein, & Peterka, 2002; Lafond, Corriveau, & Prince, 2004), stroke (Corriveau, Hebert, Raiche, & Prince, 2004; Laufer, Schwarzmann, Sivan, & Sprecher, 2005), and Parkinson’s disease (Horak, Dimitrova, & Nutt, 2005; Schmit et al., 2006). Posturography has also been used on a limited basis in movement recognition applications such as Human Computer Interaction (Shneiderman & Plaisant, 2009) and gait recognition (Sarkar et al., 2005). COP position time series are extracted from force platform data and characterized using COP amplitude measures such as sway path, velocity, and area (Diener, Dichgans, Bacher, & Gompf, 1984; Hasan, Lichtenstein, & Shiavi, 1990; Nardone, Tarantola, Giordano, & Schieppati, 1997; Nardone, Tarantola, Galante, & Schieppati, 1998); frequency measures including mean and median power frequencies (Lehmann et al., 1990; Maki, Holliday, & Topper, 1994a; Maki, Holliday, & Topper, 1994b; Wolff et al., 1998); and fractal dynamics measures including short-term and long-term diffusion coefficients and Hurst exponents (Collins & De Luca, 1993, 1995; Wolff et al., 1998; Duarte & Zatsiorsky, 2000; Tanaka, Uetake, Kuriki, & Ikeda, 2002). When viewed from a motor control perspective, the COP represents a performance variable whose stability is maintained by co-varying elemental variables (or muscle modes), which may be combined in different ways to perform specific tasks (Krishnamoorthy, Latash, Scholz, & Zatsiorsky, 2003; Latash, 2008). The task-specific nature of COP control highlights the potential of posturography in a wide variety of applications requiring identification or classification of human movement patterns. However, identifying patterns in COP data is tedious. Previous attempts at COP pattern recognition include template matching (Nakappan, Darbhe, Storey, Robinson, & O’Neal, 2005), statistical classification (Haibach et al., 2007; Santos, Delisle, Lariviere, Plamondon, & Imbeau, 2008), neural networks (Lafuente, Belda, Sanchez-Lacuesta, Soler, & Prat, 1998), and hybrid methods (Headon & Curwen, 2002), and focused on major movements with large variation such as crouching and jumping (Headon & Curwen, 2001, 2002). While these movements were successfully classifiable with force platform data, it is unclear whether similar techniques could be used to differentiate among delicate variations in ground reactions and thus generalized to a broader range of human standing movements, such as gesticulations and fine postural motions. To address the challenges of classifying multiple subtle movements, a detailed study of various feature selection and classification methods is needed. In this paper, we demonstrate a computational intelligence-based solution with an integrated feature selection and classification method that learns from examples and can distinguish among COP patterns of even subtle postural and gestural movements without needing to recalibrate for each subject. Such data driven, subject independent, and camera free motion detection has tremendous implications for applications such as evaluating efficacy of physical therapy interventions, assessing fall risk in balance-impaired patients, and enhancing athletic training activities. The objectives of this study were to (1) determine which COP features are sensitive to differences in movement types using traditional statistical analyses; and (2) build feature vectors and classifiers using machine learning methods to distinguish among these movements irrespective of the user. It is hypothesized that a movement-specific, feature-classifier design will distinguish among movement types with higher predictive power in comparison to traditional statistical methodology. It is also hypothesized that multivariate interactions will need to be considered in the feature selection phase. Completion of these objectives represents a preliminary step in establishing the feasibility of using posturography for human motion recognition.
240
S.K. Saripalle et al. / Human Movement Science 33 (2014) 238–250
Fig. 1. Testing configuration with coordinate directions.
2. Methods 2.1. Participants Fourteen student volunteers participated in the experiment, after providing written informed consent as approved by the Social Sciences Institutional Review Board at the University of Missouri – Kansas City (UMKC). All participants identified themselves as healthy with the ability to stand comfortably for 20 min. 2.2. Experimental protocol During each recording session, participants stood on a force platform (Fig. 1) while performing 11 choreographed head (H), mid-body (MB) and postural (P) movements (Table 1). Each movement was
Table 1 Description of participant movements. Movement
Description
Mv1-H Mv2-H Mv3-MB
Head nodding Head shaking Shoulder shrugging without hand movement Shoulder shrugging with palms turned upwards Touching back of head Touching one’s nose Scratching opposite arm Hands outstretched Shifting weight from one foot to other Shifting weight to tiptoes Tapping foot
Mv4-MB Mv5-MB Mv6-MB Mv7-MB Mv8-MB Mv9-P Mv10-P Mv11-P
S.K. Saripalle et al. / Human Movement Science 33 (2014) 238–250
241
performed continuously for four 10-s trials, resulting in a total of 44 trials for each participant. The order of movement types for the three body regions (H, MB, and P) was randomized across participants; specific movements within each region were performed in the same order by all participants. 2.3. Measurements and variable extraction During each trial, forces and moments in the medial–lateral (ML), anterior–posterior (AP), and vertical directions were sampled at 1000 Hz from the force platform (AMTI; Watertown, MA, USA) connected to a personal computer running Vicon NEXUS software (Vicon Motion Systems; Denver, CO, USA). Following data collection, NEXUS was used to calculate and export COP time series for each trial. COP signals were filtered using a second-order low-pass Butterworth filter with a cutoff frequency of 30 Hz (Carpenter, Frank, Winter, & Peysar, 2001). Initial AP and ML components of the COP (Fig. 2) were set to 0 to remove offset caused by participants shifting position between trials. A literature review was conducted to identify candidate COP features likely to reveal discernibility among the movement types studied (Carpenter et al., 2001; Latash, Ferreira, Wieczorek, & Duarte, 2003; Maurer & Peterka, 2005; Reilly, Van Donkelaar, Saavedra, & Woollacott, 2008; Santos et al., 2008; Vieira, Oliveira, & Nadal, 2009). Based on our review, a total of 23 scalar measures (listed in Table 2 and described below) were selected and extracted from COP time series. 2.3.1. Segment length parameters (1–2) COP segment lengths are the incremental displacements between pairs of points in the COP time series. Segment lengths are used to calculate standard deviation and summed to yield the COP path length. 2.3.2. Sway parameters (3, 4) Sway is the net range of COP motion, computed by taking the difference between maximum and minimum COP positions, in both AP and ML directions. 2.3.3. Displacement based parameters (7–10, 18–19) Displacement is the distance from the mean COP location to each point in the COP time series. Parameters extracted from this data include maximum value, mean value, and standard deviation in the AP and ML directions (Prieto, Myklebust, Hoffmann, Lovett, & Myklebust, 1996).
Fig. 2. Representative COP trajectories of a participant showing 4 trials for each of 11 movements.
242
S.K. Saripalle et al. / Human Movement Science 33 (2014) 238–250
Table 2 List of COP feature candidates, p-values for individual features from ANOVAs and overall feature ranks using univariate (FSA) and multivariate (FSB) classification-based assessments. Feature
p (participant)
p (motion)
R2 (%)
FSA rank
FSB rank
1. Standard deviation (SD) of COP segments 2. Path length 3. Sway – AP direction 4. Sway – ML direction 5. Velocity – Mean 6. Velocity – Max 7. Max displacement – AP 8. Max displacement – ML 9. SD of displacement – AP 10. SD of displacement – ML 11. Max velocity – AP 12. Max velocity – ML 13. SD of velocity – AP 14. SD of velocity – ML 15. Zero crossing rate – ML 16. Mean velocity – AP 17. Mean Velocity – ML 18. Mean displacement – AP 19. Mean displacement – ML 20. Eccentricity 21. Length of major axis of instability 22. Angle of major axis 23. Median frequency
<.001 .07 <.001 <.001 <.001 .26 <.001 1.00 .86 .03 1.00 1.00 .99 .96 .84 .20 .04 .35 .08 <.001 .03 <.001 .36
<.001 <.001 <.001 <.001 <.001 <.001 <.001 <.001 <.001 <.001 <.001 <.001 <.001 <.001 <.001 <.001 <.001 <.001 <.001 <.001 <.001 <.001 <.001
77.77 56.32 93.38 91.44 76.72 54.56 91.55 12.05 66.84 32.52 17.51 13.89 3.15 2.51 60.39 33.80 51.43 11.45 87.57 37.67 14.94 94.64 52.39
10 16 6 12 3 15 8 20 4 17 7 22 1 11 13 18 2 23 5 14 21 9 19
13 3 14 10 18 19 11 2 21 22 15 20 4 5 6 17 1 16 7 8 9 12 23
2.3.4. Velocity based parameters (5–6, 11–14, 16–17) Velocities were calculated by numerical differentiation of the displacement-based parameters. These were used to calculate the maximum value, mean value, and standard deviation with respect to the AP/ML directions and the total magnitude. 2.3.5. Elliptical fit based parameters (20–22) Several features can be calculated from an ellipse covering 85.35% of the sway area (Duarte & Zatsiorsky, 2002). The major axis corresponds to the direction of least stability. The eccentricity of the ellipse relates to the comparative directionality of the COP. 2.3.6. Frequency parameters (15, 23) Median frequency of oscillation is the time frequency at which the integral of spectral power, calculated from the resultant COP vector, is one half the value of the total integral. 2.4. Statistical analysis of COP features (Objective 1) Initial evaluation of COP data was based on standard statistical analysis performed in Minitab 15 (Minitab, State College, PA). An initial multivariate analysis of variance (MANOVA) was performed on the COP variables from the gestural data (Mv1H-Mv2H, Mv3MB-Mv8MB), postural data (Mv9PMv11P), and the combined dataset with movement as a fixed-level factor and participant as a random factor. Results indicated that both movement and participant factors were significant across the COP measures (p < .01) in all three scenarios. To investigate individual feature effects, follow-up analyses were performed including individual analyses of variance (ANOVA) for each COP measure. 2.5. Feature-classifiers for distinguishing among movements (Objective 2) A fourfold cross-validation partitioning scheme was used to identify COP variables best suited to distinguish among movements. For each movement, 75% (3 trials) and 25% (1 trial) of data was used for training and validation, respectively. Cycling through the 4 combinations of training/validation
S.K. Saripalle et al. / Human Movement Science 33 (2014) 238–250
243
partitioning allowed for 4-fold cross validation, thus further increasing the statistical validity of the results. To find the best feature-classifier arrangement with the highest discriminating power, a range of different classifiers were evaluated over subsets of features that were selected based on their individual and collective subject-independent movement identification power. All computations were performed in MATLAB (MathWorks, Natick, MA, USA). 2.5.1. Feature ranking and selection In machine learning, supervised feature selection entails searching among feature candidates to achieve the desired pattern recognition by maximizing a quality metric, such as correct recognition rate or others explained below (Guyon & Elisseeff, 2003), leading to better classification while reducing classifier complexity (Jain & Duin, 2000). This reduction in input space enforces better cause–effect relationships with classification results by preferring more salient input parameters that also positively affect the classifier output. The ensuing simplified classifier is an embodiment of Occam’s razor principle in pattern recognition, where a more compact classifier can better generalize over unseen data by avoiding over-parameterization and thus unpredictable erratic behavior in areas of input space not covered by limited training dataset (Bishop, 2006). One key issue in constructing feature vectors from a pool of candidate scalar features (Table 2) is subset search and selection. Ideally, given D feature candidates, 2D-1 feature vectors need to be evaluated; an impractical choice for many problems (Jain & Duin, 2000; Guyon & Elisseeff, 2003). A group of suboptimal but faster techniques are filter and wrapper methods. Filter methods use univariate feature (UF) ranking, which work best when the features are independent of each other. In our analysis, univariate feature ranking considers each feature separately and measures its seperability power using ROC criteria. The higher the feature’s separability power the better it is ranked (Kohavi & John, 1997; Ruiz, Riquelme, & Aguilar-Ruiz, 2006; Theodoridis & Koutroumbas, 2008). This method is henceforth referred to as FSA. In some cases, features that are not useful when used independently can provide significant performance improvements when combined with other features. Using the aforementioned reasoning, we use a wrapper based feature ranking that we call multivariate feature (MF) ranking analysis, which uses feature interdependencies. This method randomly selects subsets of features that pass a certain classifier threshold and further rank the features based on their frequency of appearance in the subsets pool (Kohavi & John, 1997; Li, Umbach, Terry, & Taylor, 2004). This method is henceforth referred to as FSB. In summary, scalar COP features were individually ranked and then grouped into vectors for best movement detection using FSA and FSB methods (Table 2). Correct classification rates, in conjunction with sensitivity and specificity, were used as movement classification quality metrics (Príncipe, Euliano, & Lefebvre, 2000; Fawcett, 2006; Alpaydin, 2009). One needs to be aware of trivial, degenerate, or null classification in case of unbalanced datasets (Baldi, Brunak, Chauvin, Andersen, & Nielsen, 2000), where the deceivingly high classification rate can only be identified by a very low sensitivity to specificity ratio, or vice versa. Area under a receiver operating characteristic (ROC) curve was used as a filter feature quality metric for initial ranking purposes (Table 3). ROC is the plot of sensitivity versus 1-specificity, an important tool in the characterization of univariate measures of class labels (Fawcett, 2006). Thus ROC AUC was used for FSA and k-NN was used for the ranking stages of FSB Table 4. For FSB, a pool and subset size of 10, and a subset selection threshold of 0.7 (training correct rate) were experimentally deemed to be the best choices in terms of overall results and computational footprint. 2.5.2. Classifiers Linear Discriminant Analysis (LDA) with Fisher’s separability criterion (Duda, Hart, & Stork, 2001) is a linear supervised classification and dimensionality reduction method. Fischer LDA cast multidimensional features into a single dimension using a linear mixture for best classification. Compared to other linear dimensionality reduction methods such as principle component analysis, Fischer LDA projections may be better suited to classification. Given their success in similar scenarios (Fioretti, Scocco, Ladislao, Ghetti, & Rabini, 2010) and their simple but robust linear structure, LDAs were used as the baseline classic linear classifiers. In this study, each movement was designated its own LDA as a one-versus-rest (OVR) classifier. Each movement-specific OVR LDA receives its designated input feature from the feature selection stage.
244
S.K. Saripalle et al. / Human Movement Science 33 (2014) 238–250
The k nearest neighbor algorithm, or k-NN, classifies a data point by assigning it the label most frequently represented among its k nearest training data points (Bishop, 2006). k-NNs have a low computational footprint and have been successfully applied to related problems (Liu, Lee, & Lin, 2010). However, k-NN is sensitive to outliers and noise, and thus it was used as one of several classifier candidates. For this study, the algorithm performed best with k = 1 when testing over k values ranging from 1 to 10. This result for k could be attributed to the limited size of the dataset, especially the number of positive exemplars for each movement, and the fact that our choreographed dataset was not affected with undue noise and artifacts. The nearest neighbors were found using the Euclidean distance measure. Since k-NN decisions are binary, sensitivity and specificity, in addition to simple correct rate, are used as k-NN quality metrics. Support Vector Machines (SVM) can classify input patterns nonlinearly and with maximum separation (Vapnik, 2000; Bishop, 2006). SVMs have been successfully applied to COP classification problems (Moustakidis, Theocharis, & Giakas, 2008). Popular SVM kernels include Gaussian and polynomial functions. Both kernels, with a range of different variances and orders, were used in this study. SVMs’ tolerance towards noise and outliers is controlled by a cost parameter (C). Choice of C is important: smaller values allow for more tolerance towards outliers but also cause misclassifications, whereas larger C values push the SVM towards a strict solution which may cause over-training, i.e., memorization of training data (including noise and outliers) at the expense of correct classification of unseen input (poor generalization). The SVM objective function is quadratic and convex, so its solution will be unique and global; thus it does not suffer from local optima traps encountered by other nonlinear classifiers such as neural networks. For both polynomial and Gaussian SVM kernels, the C-parameter was varied from 0.01 to 200. For the Gaussian kernel, the sigma (spread) value was varied from 0.1 to 50. For the polynomial kernel, the order was varied from 2 to 8. It was observed that a Cparameter of 10 and a sigma value of 1; and a C parameter of 0.09 and a polynomial order of 4 provided better results for Gaussian and Polynomial SVMs, respectively. Again, for each movement, scalar features were sequentially combined from the top of their ranked list until the maximum fourfold cross validation SVM classification rates were achieved. Ranked lists from FSA were eventually used here as they provided better results (Tables 5 and 6). 3. Results 3.1. Statistical analysis of COP features (Objective 1) ANOVAs performed on individual COP features indicated that movement produced a significant effect in all features. The analysis also revealed a wide range in ANOVA performance as R2 values ranged from 2.51% to 91.55% (Table 2). An inspection of the p-values (Table 2) revealed the existence of many movement-specific but participant-independent COP features and thus suggests the potential of feature-classifier analysis, as achieved in Objective 2.
Table 3 Input feature vectors (FSB) and classification results using the nearest neighbor method. Movement
Features
Correct rate
Sensitivity
Specificity
Mv1-H Mv2-H Mv3-MB Mv4-MB Mv5-MB Mv6-MB Mv7-MB Mv8-MB Mv9-P Mv10-P Mv11-P
[20 21 3 8 19 6 15 16 2 4 5 9 13 17 18 22 7 11 1 14 23 10] [20 9 7 4 18 22 3 5 6 8 19] [14 17 21 5 8 1 9 12 22 23 2] [1 19 3 8 15 7] [5 20 1 12 15 17 21 23 2 3 4 6 9 11 13 16 18 8] [17 15 19 4 7 8 10 6 1 3 5] [6 19 3 8] [11 1 9 3 4 7 8 15 2 22 14 23] [6 10 12 16 20 1 2 7 8 9] [4 7 17 21 1] [3 10 1 5 9 19]
0.8750 0.8669 0.9773 0.9010 0.8766 0.8945 0.8961 0.8815 1.0000 1.0000 1.0000
0.9357 0.9196 1.0000 0.9393 0.9250 0.9304 0.9518 0.9500 1.0000 1.0000 1.0000
0.2321 0.3393 0.7500 0.5179 0.3929 0.5357 0.3393 0.1964 1.0000 1.0000 1.0000
245
S.K. Saripalle et al. / Human Movement Science 33 (2014) 238–250 Table 4 Input feature vectors (FSB) and classification using LDA. Movements
Features
Correct rate
Sensitivity
Specificity
Mv1-H Mv2-H Mv3-MB Mv4-MB Mv5-MB Mv6-MB Mv7-MB Mv8-MB Mv9-P Mv10-P Mv11-P
[20 21 3 8 19 6 15 16] [20 9 7 4 18 22 3 5 6 8 19] [14 17 21 5 8] [1 19 3 8 15 7 11 14 16 21 5 6 9 10 22 2 4 12 17 18 23 13] [5 20 1 12 15 17 21 23 2 3 4 6] [17 15 19 4 7 8 10 6 1 3 5 9] [6 19 3 8 11 16 2 4 7 9 18 22 5 12 14 21 15 17 20 1 23] [11 1 9 3 4 7 8 15 2 22 14 23 6 10 13 18 19 17] [6 10 12 16 20 1 2 7 8 9] [4 7 17] [3 10 1 5 9 19 20 21 7 14 17 2 4 13 15 23]
0.6315 0.6136 0.9773 0.5682 0.8247 0.7565 0.6396 0.6623 1.0000 0.9984 0.9951
0.6071 0.5821 1.0000 0.5393 0.8357 0.7518 0.6214 0.6625 1.0000 1.0000 1.0000
0.8750 0.9286 0.7500 0.8571 0.7143 0.8063 0.8214 0.6607 1.0000 0.9821 0.9464
3.2. Feature-classifiers to distinguish among movements (Objective 2) Table 2 shows the cumulative ranking results for approaches FSA and FSB, where feature candidates are ranked according to their univariate or multivariate discrimination power over all 11 movements. It should be noted that rankings are movement-dependent; however for brevity only the cumulative overall rankings across all 11 movements are shown. Movement-specific ranked lists, in conjunction with feedback from a variety of classifiers, were used in all the forthcoming OVR wrapper methods to aggregate COP features into movement and classifier-specific input vectors. For each movement, scalar features were sequentially combined from the top of the corresponding ranked list until the maximum fourfold cross validation classification rates were achieved. As shown in Table 3, as well as in the following tables exhibiting LDA and SVM results, no single method can provide the best results for all 11 movements. Also, given the sensitivity to specificity disparities, some classification results, such as those for movements 1 and 8, are unacceptable. LDA results are shown in Table 4. Similarly, scalar features were sequentially combined in order from the top of each movement-specific FSB ranked list until the maximum fourfold cross validation classification rates were achieved. Compared to k-NN, a lessened disparity between sensitivity and specificity, despite the seemingly lower correct rates, is observed. Considering the mixed performance of classifiers across different movements, we chose the feature-classifier configurations that best detect each individual movement. This was done by comparing all the corresponding rows from Tables 3–6, and selecting simplest models with better correct classification rates; and balanced and acceptable sensitivities and specificities to avoid trivial classification. The resulting heterogeneous classifier bank (Table 7) is best suited to classify all movements.
Table 5 Input feature vectors (FSA) and classification using Gaussian kernel SVM classifier. Movement
Features
Correct rate
Sensitivity
Specificity
Mv1-H Mv2-H Mv3-MB Mv4-MB Mv5-MB Mv6-MB Mv7-MB Mv8-MB Mv9-P Mv10-P Mv11-P
[13 11 3 9 7 5 17 2 19 22 4 15 6 1 14 10 20 18 8 21 23] [1 9 19 3 7 13 11 17 5 2 22 4 15 14 6 10 20] [14 8] [9 7 3 6 16 22 11 13 20 15 4 19 17 14 18 12 21] [20 17 19 5 1 22 15 4 23 8 6 14 13 21 9 3 10 16 7 2 12] [1 20 5 19 17 15 4 6 22 10 14 11 2 23 8 13 21 18 12] [19 17 5 1 13 4 11 15 9 22 2 3 7 14 16 6 23 12 18 21 10] [3 7 9 13 18 23 8 5 12 11 6 17 16 20 2 15 14 19 4 1] [9 3] [15 4] [1 10 11 16]
0.8003 0.8133 0.9773 0.8490 0.8636 0.8880 0.8198 0.8506 1.0000 0.9984 0.9984
0.8268 0.8250 1.0000 0.8679 0.9018 0.9232 0.8571 0.8857 1.0000 0.9982 0.9982
0.5357 0.6964 0.7500 0.6607 0.4821 0.5357 0.4464 0.5000 1.0000 1.0000 1.0000
246
S.K. Saripalle et al. / Human Movement Science 33 (2014) 238–250
Table 6 Input feature vectors (FSA) and classification using Polynomial kernel SVM classifier. Movement
Features
Correct rate
Sensitivity
Specificity
Mv1-H Mv2-H Mv3-MB Mv4-MB Mv5-MB Mv6-MB Mv7-MB Mv8-MB Mv9-P Mv10-P Mv11-P
[13 11 3 9 7 5 17 2 19 22 4 15 6 1 14 6] [1 9 19 3 7 13 11 17 5 2 22 4 15] [14 8] [9 7 3 6 16 22 11 13 20 15 4 19 17 14] [20 17 19 5 1 22 15 4 23 8 6 14 13 21 9 3 10 16] [1 20 5 19 17 15 4 6 22 10] [19 17 5 1 13 4 11 15 9 22 2 3 7 14 16] [3 7 9 13 18 23 8 5 12 11 6 17 16 20] [9 3] [15 4 14 19 17] [1 10 11 16 5]
0.6697 0.7711 0.9773 0.8377 0.8555 0.8263 0.7179 0.7971 1.0000 1.0000 1.0000
0.6268 0.7768 1.0000 0.8429 0.8804 0.8339 0.7054 0.8943 1.0000 1.0000 1.0000
0.8750 0.7143 0.7500 0.7857 0.6071 0.7500 0.8393 0.6250 1.0000 1.0000 1.0000
4. Discussion This work successfully establishes a method for distinguishing among sway patterns produced during different types of body movements using force plate signals. Because body sway may be modulated by many factors, force platform records have proven to be sensitive indicators of balance dysfunction, for example as a result of aging, (Maki et al., 1994a, 1994b) as well as psychological states such as emotion (Azevedo et al., 2005) and arousal (Maki & McIlroy, 1996). In general, our wrapperbased technique provides sensitive and specific identification of movements that modulate body sway, including subtle gestures, and shows promise as an alternative or augmentative measure to commonly used video methods. Preliminary statistical analyses demonstrate that all COP features have at least some power to differentiate among movements (Objective 1). The features exhibiting a good statistical fit in the ANOVA model tended to have significant participant specificity, suggesting the potential utility of force platform technology for biometric personal identification (Jain, Bolle, & Pankanti, 1999) in addition to the numerous possible clinical applications mentioned in this paper. This may be caused, in part, by treating the movement as a fixed level factor and the participant as a random factor, which are relatively crude approximations as different movements do not correspond to distinct levels of a single variable. Table 7 Final feature-classifier selections with their test results. Movement
Ranking
Features
Correct rate
Sensitivity
Specificity
Classifier
Mv1-H
FSA
[13 11 3 9 7 5 17 2 19 22 4 15 6 1 14 6]
0.6697
0.6268
0.8750
Mv2-H
FSA
[1 9 19 3 7 13 11 17 5 2 22 4 15 14 6 10 20]
0.8133
0.8250
0.6964
Mv3-MB
FSA
[14 8]
0.9773
1.0000
0.7500
Mv4-MB
FSA
[9 7 3 6 16 22 11 13 20 15 4 19 17 14]
0.8377
0.8429
0.7857
Mv5-MB Mv6-MB
FSB FSA
[5 20 1 12 15 17 21 23 2 3 4 6] [1 20 5 19 17 15 4 6 22 10]
0.8247 0.8263
0.8357 0.8339
0.7143 0.7500
Mv7-MB
FSA
[19 17 5 1 13 4 11 15 9 22 2 3 7 14 16]
0.7179
0.7054
0.8393
Mv8-MB
FSA
[3 7 9 13 18 23 8 5 12 11 6 17 16 20]
0.7971
0.8943
0.6250
Mv9-P
FSA
[9 3]
1.0000
1.0000
1.0000
Mv10-P
FSB
[4 7 17 21 1]
1.0000
1.0000
1.0000
Mv11-P
FSA
[1 10 11 16 5]
1.0000
1.0000
1.0000
Polynomial kernel SVM Gaussian kernel SVM Gaussian kernel SVM Polynomial kernel SVM LDA Polynomial kernel SVM Polynomial kernel SVM Polynomial kernel SVM Polynomial Kernel SVM Nearest Neighbor Polynomial Kernel SVM
S.K. Saripalle et al. / Human Movement Science 33 (2014) 238–250
247
Additionally, the p values derived from the participant factor data may be skewed due to performing movements in the same consecutive order. Finally, assumptions regarding normal distributions and linear and independent effects are only partially met with these statistical methods. With the understanding that other methods such as those used in Objective 2 are better equipped to address these problems, traditional statistical methodology was not pursued further. The machine learning methods were capable of meeting Objective 2 to varying extents. FSB generally provided better performance for most classifiers, most likely due to its ability to reflect some multivariate interactions. As expected, our results suggest that all classifiers performed well for the more pronounced postural movements Mv9-P through Mv11-P. However, the figures were mixed and generally not as high for more subtle gestural movements Mv1-H through Mv8-MB. Nearest neighbor classifiers performed well with an average fourfold cross validation correct rate, sensitivity, and specificity of 0.92, 0.96, and 0.57, respectively (Table 3), but the disparity between sensitivity and specificity figures makes kNNs less desirable. LDAs followed closely but with more balanced sensitivity to specificity ratios, yielding an average fourfold cross validation correct rate, sensitivity, and specificity of 0.79, 0.78, and 0.85, respectively (Table 4). SVM models performed better than other methods for most movements, with an average fourfold cross validation correct rate, sensitivity, and specificity of 0.88, 0.89, and 0.75; respectively (Tables 5 and 6). SVMs’ better performance, especially their more balanced sensitivity and specificity figures, may be attributed to their robust nonlinear classification capabilities. Neural Networks were also considered as another nonlinear method for feature selection and classification based on their universal data-driven discrimination capabilities (Haykin, 2009) and their reported success in force plate signal identification (Lafuente et al., 1998). However, neural networks were matched or outperformed by other methods, so were not pursued further. This is likely due to the different and subtle nature of movements in our dataset, in addition to lack of enough exemplars of each target class when compared to network sizes. When considering gestural movements, there was no single classifier or feature set that could outperform the rest when considering sensitivity and specificity in addition to correct rate, as the latter by itself can be a misleading figure in unbalanced, multi-class datasets. Thus, using the best feature-classifier combinations from all the above methods, a heterogeneous bank of 11 OVR classifiers was formed. To that end, the data from the different feature-classifier combinations (Tables 3–6) were pooled to yield the best selection by considering sensitivity, specificity, and overall gestural rates (Table 7). This observation calls for movement-specific feature sets and classifier designs as the best approach for Objective 2. Further study is required to fully understand, from a physiological perspective, why different movements are best classified with such a varied range of feature sets. It is clear, however, that the features exhibiting the largest differences among movement types are not necessarily the same features that are best suited for classification. For example, Mv9-P involves shifting weight from one foot to the other; while this involves more medial–lateral COP movement than the other tasks, it is the anterior–posterior COP characteristics (standard deviation of displacement and maximum sway) that offer optimum separability (maximum separation with minimum spread) from the other movements. Similarly, Mv10-P, which required anterior–posterior weight shifts, was classified by several medial– lateral COP characteristics. Since Mv9-P and Mv10-P were task that may have required more cognitive attention than the others, it is possible that voluntarily increasing sway in one direction was done at the expense of sway regulation in the orthogonal direction, a phenomenon that has been reported elsewhere (Woollacott & Shumway-Cook, 2002; Tucker, Kavanagh, Morrison, & Barrett, 2009). It is also possible that our faster but suboptimal filter and wrapper feature selection techniques identified local minima, resulting in different features than those that might have been identified in an exhaustive search. Further study is needed to determine whether ‘‘brute force’’ feature selection is successful in identifying features sets that are more homogeneous across movement types. A further limitation of our study may relate to the types of COP features studied. Due to the preliminary nature of this work, we chose a fairly traditional set of amplitude- and frequency-related COP descriptors. However, more sophisticated time series analyses such as stabilogram-diffusion analysis (Collins & De Luca, 1993) and detrended fluctuation analysis (Duarte & Zatsiorsky, 2001) are focused on temporal correlations of COP data. These analyses have revealed persistent and antipersistent behavior over short and long time intervals, respectively, suggesting open- and closed-loop
248
S.K. Saripalle et al. / Human Movement Science 33 (2014) 238–250
postural control. As with traditional COP parameters, task- and condition-specific changes in SDA and DFA parameters have been observed in a variety of applications such as aging and disease state (e.g., Peterka, 2000; Lin, Seol, Nussbaum, & Madigan, 2008). It is therefore likely that such temporal parameters would be candidates for the COP classification techniques described here, and should be considered in future work. Because force platforms can conveniently and precisely measure sway and other motor related time series, our approach can be advantageous when fast, unobtrusive, and automated evaluation of posturographic information is needed. For applications requiring capture of body movements, force platforms may address some of the limitations of video analysis. For instance, conventional video cannot discern small or occluded movements, and face challenges such as non-ideal lighting in real world scenarios. Our methodology has potential diagnostic utility in a wide range of movement- and balance-related pathological conditions such as peripheral neuropathy (Corriveau et al., 2000; Horak et al., 2002; Lafond et al., 2004), stroke (Corriveau et al., 2004; Laufer et al., 2005), and Parkinson’s disease (Horak et al., 2005; Schmit et al., 2006). Human–computer interaction (Shneiderman & Plaisant, 2009) is another area in which force platforms could be used as an input in applications such as smart homes, video games, and assistive devices (Orr & Abowd, 2000; Betker, Szturm, Moussavi, & Nett, 2006). Yet another potential application comes from the area of deception detection. While it has been suggested that elements of gesture and posture reflect deceptive intention (Frank & Ekman, 1997; Vrij, 2000), analysis of this information is usually done with video records, which are subject to the limitations discussed above. Force plate records, coupled with classifier analysis, could provide a more sensitive, automated indicator of postural correlates of deception or other cognitive states associated with postural changes. In future studies it may be advisable to explore additional data preprocessing and signal conditioning options. For instance, the cutoff frequency of the low pass filter applied to the force plate signals may dynamically change (e.g., Woltring filter) to better accommodate the increased frequency content around impact regions. Other signal preprocessing options include Kalman filtering to reduce noise in feature values. Future work may include COP features not explored in the current study, such as Romberg ratio, fractal dimensions, and diffusion constants (Mientjes & Frank, 1999; Santos et al., 2008).
5. Conclusion We applied traditional statistical analyses and modern machine learning techniques to detect differences in sway patterns associated with a variety of body movements captured by ground-embedded force platforms. Our method successfully distinguished among COP patterns associated with various large and small human movements. Classification-guided subset selection provided feature sets that, in conjunction with different classifiers, provided recognition rates of 67–100%, with an average of 92%, across all 11 movements (Table 7). Given sensitivity and specificity figures, none of the classifiers suffer from trivial classification (very low sensitivity). Classification rates, sensitivity, and specificity figures for different feature-classifier combinations varied for each movement, suggesting that use of a single model or feature set may not be sufficient for identification of all the movements studied. Linear classifiers performed rather poorly, suggesting that class boundaries among movements are nonlinear. SVMs had the highest overall performance of any individual classifier type on the grounds of highest sensitivity and specificity, especially for more subtle gestural movements. Since each movement required a specific classifier with its distinct multivariate feature set to achieve acceptable sensitivity, specificity, and classification rate, a bank of movement-specific features along with their matched OVR classifiers is better suited for identification of body movement based on force plate data.
Acknowledgements We wish to thank Dr. Judee Burgoon and her team at University of Arizona for their assistance and support, as well as Laxmi S. Gajjala for his contributions to the manuscript.
S.K. Saripalle et al. / Human Movement Science 33 (2014) 238–250
249
References Alpaydin, E. (2009). Introduction to machine learning. The MIT Press [Chapter]. Azevedo, T. M., Volchan, E., Imbiriba, L. A., Rodrigues, E. C., Oliveira, J. M., Oliveira, L. F., et al (2005). A freezing-like posture to pictures of mutilation. Psychophysiology, 42, 255–260. Baldi, P., Brunak, S., Chauvin, Y., Andersen, C. A., & Nielsen, H. (2000). Assessing the accuracy of prediction algorithms for classification: An overview. Bioinformatics, 16, 412–424. Betker, A. L., Szturm, T., Moussavi, Z. K., & Nett, C. (2006). Video game-based exercises for balance rehabilitation: A singlesubject design. Archives of Physical Medicine and Rehabilitation, 87, 1141–1149. Bishop, C. M. (2006). Pattern recognition and machine learning. New York: Springer [Chapter]. Carpenter, M. G., Frank, J. S., Winter, D. A., & Peysar, G. W. (2001). Sampling duration effects on centre of pressure summary measures. Gait & Posture, 13, 35–40. Collins, J. J., & De Luca, C. J. (1993). Open-loop and closed-loop control of posture: A random-walk analysis of center-of-pressure trajectories. Experimental Brain Research, 95, 308–318. Collins, J. J., & De Luca, C. J. (1995). Upright, correlated random walks: A statistical-biomechanics approach to the human postural control system. Chaos, 5, 57–63. Corriveau, H., Hebert, R., Raiche, M., & Prince, F. (2004). Evaluation of postural stability in the elderly with stroke. Archives of Physical Medicine & Rehabilitation, 85, 1095–1101. Corriveau, H., Prince, F., Hebert, R., Raiche, M., Tessier, D., Maheux, P., et al (2000). Evaluation of postural stability in elderly with diabetic neuropathy. Diabetes Care, 23, 1187–1191. Diener, H. C., Dichgans, J., Bacher, M., & Gompf, B. (1984). Quantification of postural sway in normals and patients with cerebellar diseases. Electroencephalogr Clin Neurophysiol, 57, 134–142. Duarte, M., & Zatsiorsky, V. M. (2000). On the fractal properties of natural human standing. Neuroscience Letters, 283, 173–176. Duarte, M., & Zatsiorsky, V. M. (2001). Long-range correlations in human standing. Physics Letters A, 283, 124–128. Duarte, M., & Zatsiorsky, V. (2002). Effects of body lean and visual information on the equilibrium maintenance during stance. Experimental Brain Research, 146, 60–69. Duda, R. O., Hart, P. E., & Stork, D. G. (2001). Pattern classification. Wiley [Chapter]. Fawcett, T. (2006). An introduction to ROC analysis. Pattern Recognition Letters, 27, 861–874. Fioretti, S., Scocco, M., Ladislao, L., Ghetti, G., & Rabini, R. A. (2010). Identification of peripheral neuropathy in type-2 diabetic subjects by static posturography and linear discriminant analysis. Gait & Posture, 32, 317–320. Frank, M. G., & Ekman, P. (1997). The ability to detect deceit generalizes across different types of high-stake lies. Journal of Personality & Social Psychology, 72, 11p. Guyon, I., & Elisseeff, André (2003). An introduction to variable and feature selection. Journal of Machine Learning Research, 3, 1157–1182. Haibach, P. S., Slobounov, S. M., Slobounova, E. S., & Newell, K. M. (2007). Virtual time-to-contact of postural stability boundaries as a function of support surface compliance. Experimental Brain Research, 177, 471–482. Hasan, S. S., Lichtenstein, M. J., & Shiavi, R. G. (1990). Effect of loss of balance on biomechanics platform measures of sway: Influence of stance and a method for adjustment. Journal of Biomechanics, 23, 783–789. Haykin, S. S. (2009). Neural Networks and Learning Machines. USA: Prentice Hall [Chapter]. Headon, R. & Curwen, R. (2001). Recognizing movements from the ground reaction force. In: Proceedings of the 2001 workshop on Perceptive user interfaces (pp. 1–8). Orlando, Florida: ACM. Headon, R., & Curwen, R. (2002). Movement awareness for ubiquitous game control. Personal Ubiquitous Computing, 6, 407–415. Horak, F. B., Dickstein, R., & Peterka, R. J. (2002). Diabetic neuropathy and surface sway-referencing disrupt somatosensory information for postural stability in stance. Somatosensory & Motor Research, 19, 316–326. Horak, F. B., Dimitrova, D., & Nutt, J. G. (2005). Direction-specific postural instability in subjects with Parkinson’s disease. Experimental Neurology, 193, 504–521. Jain, A. K., Bolle, R., & Pankanti, S. (1999). Biometrics: Personal identification in networked society. Kluwer Academic Publishers [Chapter]. Jain, A. K., & Duin, R. P. W. (2000). Statistical pattern recognition: A review. IEEE Transactions on Pattern Analysis & Machine Intelligence, 22, 34p. Kohavi, R., & John, G. H. (1997). Wrappers for feature subset selection. Artificial Intelligence, 97, 273–324. Krishnamoorthy, V., Latash, M. L., Scholz, J. P., & Zatsiorsky, V. M. (2003). Muscle synergies during shifts of the center of pressure by standing persons. Experimental Brain Research, 152, 281–292. Lafond, D., Corriveau, H., & Prince, F. (2004). Postural control mechanisms during quiet standing in patients with diabetic sensory neuropathy. Diabetes Care, 27, 173–178. Lafuente, R., Belda, J. M., Sanchez-Lacuesta, J., Soler, C., & Prat, J. (1998). Design and test of neural networks and statistical classifiers in computer-aided movement analysis: A case study on gait analysis. Clinical Biomechanics (Bristol, Avon), 13, 216–229. Latash, M. L. (2008). Neurophysiological basis of movement (2nd ed.). Champaign: Human Kinetics [Chapter]. Latash, M. L., Ferreira, S. S., Wieczorek, S. A., & Duarte, M. (2003). Movement sway: Changes in postural sway during voluntary shifts of the center of pressure. Experimental Brain Research, 150, 314–324. Laufer, Y., Schwarzmann, R., Sivan, D., & Sprecher, E. (2005). Postural control of patients with hemiparesis: Force plates measurements based on the clinical sensory organization test. Physiotherapy Theory & Practice, 21, 163–171. Lehmann, J. F., Boswell, S., Price, R., Burleigh, A., deLateur, B. J., Jaffe, K. M., et al (1990). Quantitative evaluation of sway as an indicator of functional balance in post-traumatic brain injury. Archives of Physical Medicine and Rehabilitation, 71, 955–962. Li, L., Umbach, D. M., Terry, P., & Taylor, J. A. (2004). Application of the GA/KNN method to SELDI proteomics data. Bioinformatics, 20, 1638–1640. Lin, D., Seol, H., Nussbaum, M. A., & Madigan, M. L. (2008). Reliability of COP-based postural sway measures and age-related differences. Gait & Posture, 28, 337–342.
250
S.K. Saripalle et al. / Human Movement Science 33 (2014) 238–250
Liu, C. L., Lee, C. H., & Lin, P. M. (2010). A fall detection system using k-nearest neighbor classifier. Expert Systems with Applications, 37, 7174–7181. Maki, B. E., Holliday, P. J., & Topper, A. K. (1994a). A prospective study of postural balance and risk of falling in an ambulatory and independent elderly population. The Journals of Gerontology. Series A, Biological Sciences and Medical Sciences, 49, M72–84. Maki, B. E., Holliday, P. J., & Topper, A. K. (1994b). A prospective study of postural balance and risk of falling in an ambulatory and independent elderly population. Journal of Gerontology, 49, M72–84. Maki, B. E., & McIlroy, W. E. (1996). Influence of arousal and attention on the control of postural sway. Journal of Vestibular Research, 6, 53–59. Maurer, C., & Peterka, R. J. (2005). A new interpretation of spontaneous sway measures based on a simple model of human postural control. Journal of Neurophysiology, 93, 189–200. Mientjes, M. I., & Frank, J. S. (1999). Balance in chronic low back pain patients compared to healthy people under various conditions in upright standing. Clinical Biomechanics (Bristol, Avon), 14, 710–716. Moustakidis, S. P., Theocharis, J. B., & Giakas, G. (2008). Subject recognition based on ground reaction force measurements of gait signals. IEEE Transactions on Systems, Man, and Cybernetics. Part B, Cybernetics, 38, 1476–1485. Murray, M. P. (1967). Gait as a total pattern of movement. American Journal of Physical Medicine, 46, 290–333. Nakappan, S., Darbhe, V. A., Storey, C. M., Robinson, C., & O’Neal, K. K. (2005). Variations in anterior–posterior CoP patterns in elderly adults between psychophysically detected and non-detected short horizontal perturbations. In Proceedings of the IEEE engineering in medicine and biology society annual international conference (Vol. 5, pp. 5427–5430). Nardone, A., Tarantola, J., Galante, M., & Schieppati, M. (1998). Time course of stabilometric changes after a strenuous treadmill exercise. Archives of Physical Medicine and Rehabilitation, 79, 920–924. Nardone, A., Tarantola, J., Giordano, A., & Schieppati, M. (1997). Fatigue effects on body balance. Electroencephalogr Clin Neurophysiol, 105, 309–320. Orr, R. J. & Abowd, G. D. (2000). The smart floor: A mechanism for natural user identification and tracking. In: CHI ‘00 extended abstracts on Human factors in computing systems. The Hague, The Netherlands, ACM: 275–276. Peterka, R. J. (2000). Postural control model interpretation of stabilogram diffusion analysis. Biological Cybernetics, 82, 335–343. Prieto, T. E., Myklebust, J. B., Hoffmann, R. G., Lovett, E. G., & Myklebust, B. M. (1996). Measures of postural steadiness: Differences between healthy young and elderly adults. IEEE Transactions on Bio-medical Engineering, 43, 956–966. Príncipe, J. C., Euliano, N. R., & Lefebvre, W. C. (2000). Neural and adaptive systems: Fundamentals through simulations. Wiley [Chapter]. Reilly, D. S., Van Donkelaar, P., Saavedra, S., & Woollacott, M. H. (2008). Interaction between the development of postural control and the executive function of attention. Journal of Motor Behavior, 40, 90–102. Ruiz, R., Riquelme, J. C., & Aguilar-Ruiz, J. S. (2006). Incremental wrapper-based gene selection from microarray data for cancer classification. Pattern Recognition, 39, 2383–2392. Santos, B. R., Delisle, A., Lariviere, C., Plamondon, A., & Imbeau, D. (2008). Reliability of centre of pressure summary measures of postural steadiness in healthy young adults. Gait and Posture, 27, 408–415. Sarkar, S., Phillips, P. J., Liu, Z., Vega, I. R., Grother, P., & Bowyer, K. W. (2005). The humanID gait challenge problem: Data sets, performance, and analysis. IEEE Transactions on Pattern Analysis and Machine Intelligence, 27, 162–177. Schmit, J. M., Riley, M. A., Dalvi, A., Sahay, A., Shear, P. K., Shockley, K. D., et al (2006). Deterministic center of pressure patterns characterize postural instability in Parkinson’s disease. Experimental Brain Research, 168, 357–367. Shneiderman, B., & Plaisant, C. (2009). Designing the user interface: Strategies for effective human–computer interaction. AddisonWesley [Chapter]. Tanaka, H., Uetake, T., Kuriki, S., & Ikeda, S. (2002). Changes in center-of-pressure dynamics during upright standing related to decreased balance control in young adults: Fractional Brownian motion analysis. Journal of Human Ergology, 31, 1–11. Theodoridis, S., & Koutroumbas, K. (2008). Pattern recognition. Elsevier/Academic Press [Chapter]. Tucker, M. G., Kavanagh, J. J., Morrison, S., & Barrett, R. S. (2009). Voluntary sway and rapid orthogonal transitions of voluntary sway in young adults, and low and high fall-risk older adults. Clinical Biomechanics (Bristol, Avon), 24, 597–605. Vapnik, V. N. (2000). The Nature of Statistical Learning Theory. Springer [Chapter]. Vieira, T. M., Oliveira, L. F., & Nadal, J. (2009). Estimation procedures affect the center of pressure frequency analysis. Brazilian Journal of Medical and Biological Research, 42, 665–673. Vrij, A. (2000). Detecting lies and deceit: The psychology of lying and the implications for professional practice. John Wiley [Chapter]. Wolff, D. R., Rose, J., Jones, V. K., Bloch, D. A., Oehlert, J. W., & Gamble, J. G. (1998). Postural balance measurements for children and adolescents. Journal of Orthopaedic Research, 16, 271–275. Woollacott, M., & Shumway-Cook, A. (2002). Attention and the control of posture and gait: A review of an emerging area of research. Gait and Posture, 16, 1–14.