- Email: [email protected]
A statistical smoothness measure to eliminate outliers in motion trajectory tracking
Human Movement Science 17 (1998) 189±200
A statistical smoothness measure to eliminate outliers in motion trajectory tracking Jiping He *, Chun-Xiang Tian Whitaker Center for Neuro-Mechanical Control, Chemical, Bio and Materials Engineering, Arizona State University, ECG 202, MS-6006, Tempe, AZ 85287-6006, USA
Abstract The automatic tracking of motion data recorded via digital cameras has facilitated human movement study. Outliers in the recorded camera data, however, are still serious problems for reconstructing movement trajectory. Outliers arise from tracking errors, such as missing markers, switched positions of dierent markers in tracked data, and a block of data points in the trajectory jumping back and forth between two values caused by discrepancies between the two lenses of the same camera. General ®ltering algorithms can reduce, but not eliminate the ill eect of outliers on the movement trajectory. We have developed a simple algorithm to process the tracked motion data to eliminate several common outliers automatically. The algorithm is based on the statistical smoothness characteristics of the camera data and under the assumption that a human movement trajectory should not contain any sudden shifts. Using a statistical smoothness measure and searching pairs of opposite peaks in marker velocities, a three step approach in the algorithm can remove outliers of various origins automatically. When combined with interpolation and ®ltering algorithms, the gaps in the trajectory due to outlier removal can be ®lled with reasonable values, and a smooth and accurate movement trajectory will be achieved. Ó 1998 Elsevier Science B.V. All rights reserved. PsycINFO classi®cation: 1330; 3297; 4010; 4100 Keywords: Tracking error; Filter; Motion analysis; Smoothness; Statistics; Trajectory; Posture
*
Corresponding author. Tel.: +1 602 965 0092; fax: +1 602 965 0037; e-mail: [email protected].
0167-9457/98/$19.00 Ó 1998 Elsevier Science B.V. All rights reserved. PII: S 0 1 6 7 - 9 4 5 7 ( 9 7 ) 0 0 0 2 9 - 8
190
J. He, C.-X. Tian / Human Movement Science 17 (1998) 189±200
1. Introduction Tracking and reconstructing accurate movement trajectory is an indispensable process for a motion analysis system, especially for optical systems using passive markers (Winter, 1991). Dropped out or switched marker positions, however, have frequently frustrated many operators of such systems (Ball and Pierrynowski, 1995; Mah et al., 1994). Additional cameras or rearrangement of markers can reduce, but cannot completely eliminate, the occurrence of such tracking errors. In many situations the total number of cameras is limited due to the setup and the cost considerations. Therefore, tracking errors will continue to be a problem to be reckoned with in analyzing movement data. Several factors may contribute to tracking errors. The most common one is from dropped out markers that are blocked (Ball and Pierrynowski, 1995). Another is from the merging of two or more markers when their paths coincide. These two situations may occur simultaneously or become the source of one another. A third situation arises when two cameras or the two lenses of the same camera do not generate the same position value for a marker. These three tracking errors have dierent eects on the reconstructed movement trajectory. The dropped out markers are easiest to recognize because they present a gap in the trajectory (depending on the algorithm used, the position value of gap can be zero or in®nity). They have received extensive attention and several algorithms have been developed to correct the error, if the gap is small and/or movement is slow (Ball and Pierrynowski, 1995; Mah et al., 1994). Most tracking systems contain algorithms to remove these blocked marker positions and to ®ll the small gap automatically. The other two kinds of tracking errors will generate a period of shifted path or zigzag path along an otherwise smooth trajectory. Fig. 1 illustrates one such situation that has been dicult to correct by automatic tracking algorithms. Each ball in the ®gure represents a marker position and the ®gure shows the paths of two markers right before, during and after the intersection for three sampling periods. The paths of the two markers change direction after the intersection. This change of direction at the intersection point poses problem for interpolation±extrapolation algorithms used in most tracking systems. When two markers' paths intersect in the view ®elds of some cameras while other cameras' views are blocked or partially blocked, the prediction from the tracking system may lead to the switching of the two markers. As a result the next few frames may be represented by outliers in the recorded data due to the shifted markers. Such shift in marker
J. He, C.-X. Tian / Human Movement Science 17 (1998) 189±200
191
Fig. 1. Illustration of possible tracking errors when two marker paths cross at one point. Each ball represents a sampled position and the paths are consisted of three hypothetical samples of two markers. Most tracking algorithms use previous data points of each marker to predict the next position as a guidance in distinguishing markers (dotted lines). After two markers merge at a point, if paths of the two markers change directions, the markers may be mislabeled from that point onward. Most tracking algorithms will eventually correct this error with the help of other constraints such as ®xed distance between markers attached to the same rigid segment. The result in the tracked data is a period of shifted path in an otherwise smooth trajectory.
trajectory will not last long if one uses a multiple camera system and crosscamera validation in the tracking system. It will create a marker trajectory that contains one or a few consecutive shifted positions as shown in Fig. 2. It is often necessary for a human operator to manually pickup these outliers. The procedure can be a very tedious and time consuming one if a large number of markers are used or/and data from many trials are collected. We will present a simple, yet very eective algorithm for picking up outliers in 3D motion data created by various tracking errors. The algorithm is based on the special characteristics of outliers and patterns of their derivatives. We will show that using low pass ®lters cannot eliminate the eect of all outliers on tracked trajectories and the ®ltered data present some false bumps in an otherwise smooth trajectory. The new algorithm is successful to eliminate almost all outliers created by tracking errors. 2. Methods In one of our postural perturbation studies, we collected kinematic data from 16 subjects. The task is to maintain the upright posture when a short duration impulsive force is applied to the body. Since the force is impulsive and fairly large, a quick motion at the pelvis level is induced and often a step
192
J. He, C.-X. Tian / Human Movement Science 17 (1998) 189±200
Fig. 2. Schematic plots about how the outliers are identi®ed. The top two plots show the marker trajectory, the bottom two show the derivative of marker position. Both individual outliers (frames 148±165) and a period of continuous outliers (frames 167±177) are present. The two dotted lines indicate the smoothness boundary for initial identi®cation of suspected outliers (index vector X and sign vector S, see text for detail). Pair-matching of opposite signs in S allows the ®nal identi®cation of true outliers in X. Note that if all data points outside the boundary are considered outliers, true data for frames 234± 245 will all be chosen as outliers. If pair-matching is not used, the continuous outliers from frames 167 to 177 will be missed. The small zigzag of marker positions in this period is caused by disagreement between the left and the right lenses of the same camera.
is taken by a subject to maintain the posture. A four-camera MacRe¯ex system was used in the study, with each camera sampling at 120 Hz (interlacing of two lenses sampling at 60 Hz). We used 16 markers for a ®ve-segment model: head±trunk±arm, pelvis, shank, shin, and foot. We placed three markers on each segment and one extra marker on the heel of the counter-lateral foot to record stepping. We recorded at least 25 trials from each subject, with each trial consisting of 4 s of motion data. The typical movement trajectory produced by the automatic tracking program often contains ranges of data as shown in the top plot of Fig. 2. Evidently, tracking errors occurred but they are not all due to missing markers. MacRe¯ex system provides an algorithm based on interpolation±extrapolation of previous marker points to predict next positions for isolated mark-
J. He, C.-X. Tian / Human Movement Science 17 (1998) 189±200
193
ers. While the algorithm can automatically and correctly ®ll the blocked marker positions if the gap is small, it does not always work for the tracking situations presented in Fig. 1 as discussed in Section 1. It sometimes generates more tracking errors because of wrongly predicted positions. Since these errors may be well within the variation range of a marker trajectory (Fig. 2), one may overlook them and rely on low pass ®lter to eliminate their eect on the movement trajectory. The eect of these tracking errors cannot be completely eliminated by a low pass ®lter, as will be shown later in Fig. 3. These outliers will present some false variations (bumps) in an otherwise smooth trajectory. Manual tracking frame by frame at troubled spots can correct these errors. This approach will be workable if the data volume is reasonably small or the troubled spots are limited and predictable. If the data size is huge and the troubled spots are numerous and randomly distributed, manual tracking data frame by frame will be labor intensive. An automated process to pickup outliers is certainly desirable. The challenge is to determine whether a data point is an outlier based on a uniform criterion.
Fig. 3. Comparison of marker trajectories processed by dierent methods. The top left plot is the raw data of a marker position from the tracking program. Additional outliers of dierent kinds are added to examine the eectiveness of the new algorithm (highlighted by an ellipsoidal circle). The top middle plot is after ®lling the drop-out markers. The top right is after processing by the new algorithm. The bottom row is low-pass ®ltered data of the top row, using Woltring's spline ®ltering program. Notice that outliers create several false variations on the smooth trajectory after ®ltering by Woltring's program. These variations disappear after all outliers are removed.
194
J. He, C.-X. Tian / Human Movement Science 17 (1998) 189±200
Examining how a human operator detects outliers will generate insights into an eective algorithm. By inspecting marker trajectories (as the one shown in Fig. 2), a human operator can easily locate outliers by surveying the overall trend of the trajectory. Therefore, an evaluation of some statistical smoothness of tracked data may hold the key for eliminating outliers. The sudden shift in marker positions caused by outliers will be ampli®ed by differentiating the recorded trajectory. Outliers will present some special characteristics in the velocity useful for identifying these outliers. Let P be a marker attached to a body segment. The marker position at time t i is denoted by xi . The raw marker trajectory can then be expressed as y t x1 x2 ; . . . ; xiÿ1 xi xi1 ; . . . ; xn :
1
The velocity of y(t) is calculated using the ®nite dierence: y_ xi1 ÿ xi =Dt
2
where Dt 1/ns , and ns is the sampling rate of the camera system. _ and the standard deviation, ry_ , of y_ are then calculated. An The mean, y, outlier will not contribute to the mean velocity because it has almost symmetric positive and negative peaks if the data are not ®ltered. The standard deviation of the raw velocity will be much larger due to the presence of outliers. The smoothness of the trajectory can be evaluated by comparing all data points in the velocity vector with boundaries de®ned by y_ kry ; 3 where k is a movement dependent tolerance factor to be selected by the user. Every data point xi , for all i 2 1; 2; . . . ; n, that satis®es the following condition is declared a `suspected' outlier: 4 y_i > y_ kry_ or y_i < y_ ÿ kry_ : Then put i into an index array and a sign indicator into another array for later processing of true outliers. The `tolerance' factor k is selected according to the quality of the data. More speci®cally, k should re¯ect the ratio of noise contribution from tracking error to that from other sources. k 2 was found to work well for most of our data. If the recorded movement contains some sharp turns or sudden starts or stops, the maximum velocity of the true trajectory may exceed the smoothness boundary speci®ed by kry_ . The bottom left plot in Fig. 2 illustrates the situation where the movement velocity exceeds the lower bound. There-
J. He, C.-X. Tian / Human Movement Science 17 (1998) 189±200
195
fore, not all points lying outside the boundary de®ned by Eq. (3) or other more complex statistics are true outliers. However, these points are often easily identi®able because they occur in speci®c time during a movement. Further analysis of recorded marker trajectory indicates that outliers and their derivatives have certain patterns that can help to eradicate them. Assume that there is an isolated outlier O at frame i. Its position xi is signi®cantly bigger (or smaller) than its neighbors. The dierentiation at the point produces two adjacent bursts, the ®rst corresponding to xi ÿxiÿ1 > 0, and the second corresponding to xi1 ÿxi < 0. Therefore, there will be a twophase burst associated with every isolated outlier in the velocity plot, (Fig. 2, bottom plots). For a sequence of consecutive outliers, such two-phase bursts can also be observed at the start and end of these outliers (between frames 165 and 180 in Fig. 2), even if not all outliers have the same value. This observation suggests that a pair of adjacent peaks with opposite signs in the velocity data are the key characteristics of an outlier or a sequence of consecutive outliers. By specifying a window width w and locating the matching peaks within w, a sequence of outliers can be singled out among all the suspected outliers. Finally, the values for outlier points can be estimated by any interpolation algorithm. We have used Woltring's spline program (Woltring, 1980). The algorithm to pick up outliers from camera recorded data can be summarized as a generic procedure with three steps. Step 1: Establish smoothness criterion and pick up suspected outliers. Dierentiate un®ltered marker positions to obtain the raw velocity. Calculate the mean and the standard deviation of the velocity. Select a value for k and pick out suspected data points for outliers according to y_ kry : Gather all indices corresponding to these outliers in an ascending order and put them in an integer vector X. Find the corresponding velocity sign to each index in X and record it in a vector S. Step 2: Check for matching index pairs in X with opposite signs among data points in S for true outliers. Select window width w A. Identify isolated outliers with the pseudo-code: For each valid element X[i] of X do if (S[i] and S[i+1] are of different sign) AND (X[i])X[i+1] 1) then Record outlier index, X[i]
196
J. He, C.-X. Tian / Human Movement Science 17 (1998) 189±200
Drop X[i] and X[i+1] B. Identify clustered outliers with the pseudo-code: For each valid element X[i] of X do if (S[i] and S[i+1] are of different sign) AND (X[i])X[i+1]) < w) then Record outlier indices, from X[i] to X[i+1]-1 Drop X[i] and X[i+1] Step 3: Fill the values for outlier data points using spline interpolation. These three steps may have to be repeated several times to eliminate all outliers. 3. Results We used the new algorithm to process the data recorded by the MacRe¯ex system in our posture perturbation study. A trajectory section for one of the 16 markers on the vertical axis is shown in the top left plot of Fig. 3. The trajectory is produced by the automatic tracking program in MacRe¯ex system. Drop-out points and some other outliers can be observed in the original data. We modi®ed the data to include some additional outliers (circled regions) to test the eectiveness of the new algorithm for automatically eliminating outliers. The blocked marker positions (drop-outs) were replaced by 0's for the convenience of presenting data. The tracking program in the MacRe¯ex system can identify blocked marker positions (drop-outs) because they are represented by in®nity in the raw camera data. The program replaced these missing points by an interpolation algorithm to improve the quality of the tracked data, as shown by the second plot on the top row in Fig. 3. The outliers created by other tracking errors, however, were left untouched because they were well within the valid data range for the tracking system. The additional outliers merged into the original data were not changed as expected because we added them after the tracking. The last column of plots in Fig. 3 shows the data processed by the new algorithm. The original data together with hypothetical outliers were used as the input data to test the new algorithm (the top left plot). The processed data is shown by the trace in the top plot of the right column. The algorithm is successful in picking up dropped out marker positions, outliers created by other tracking errors, and the outliers we merged in. In this testing data,
J. He, C.-X. Tian / Human Movement Science 17 (1998) 189±200
197
we have included four types of outliers: large outliers (dropped out markers with values outside the valid data range), single outliers, a range of zigzag outliers, and a range of continuous outliers. The algorithm is eective in identifying all of them and replacing their values with interpolated ones. For comparison we used Woltring's spline ®ltering program to smooth the marker data under all three conditions. The optimal cut-o frequency was automatically selected in Woltring's algorithm for each data set. The low pass ®lter did smooth out the zigzag outliers because they resembled a high frequency noise with larger amplitude values. The ®lter did not eliminate the effect of outliers on the trend of the data, producing a false impression that the recorded trajectory had some sharp turns (the left two plots at the bottom row). After eliminating all outliers, Woltring's spline ®ltering program removed high frequency noises in the data ®nally to produce a trajectory that is truthful of the real movement, as shown in the bottom right plot. This indicates that a well conceived and designed ®lter algorithm can eliminate noises and certain types of outliers, but cannot correct errors due to outliers in general. Fig. 4 shows another example. Plots in the left column show the marker traces after processed by ®lling dropped markers only (middle) and by the new algorithm (bottom one). Plots on the right column show the eect of low pass ®ltering (Woltring's algorithm) of corresponding data from the left column. Notice that in the ®ltered data, the outliers at the beginning of the movement and during the movement create false variations. If one uses a lower cuto frequency for ®ltering, the individual ``bumps'' may merge together to form a single slow rising bump at the beginning of the movement, as illustrated by the dashed line. This will generate a false impression or even misinterpretation for the recorded movement data. This is one example why over ®ltering of noisy data may not be desirable. The bottom plots show the eectiveness of the new algorithm. It removes outliers and ®lls the gaps using simple spline interpolation. Filtering of this processed data further removes high frequency noises and generates a very smooth yet truthful marker trajectory. 4. Discussion Tracking errors in camera recorded motion data distort the movement trajectory. Outliers created by tracking errors cannot be removed by ®ltering algorithms no matter what optimal parameters for the ®lter might be used.
198
J. He, C.-X. Tian / Human Movement Science 17 (1998) 189±200
Fig. 4. An example to illustrate the ill-eect of over ®ltering. The raw data is the actual camera data for one marker from a postural perturbation study. The top plots show the big eect of dropped markers on the trajectory. The middle plots show the marker traces after removing of dropped markers and low pass ®ltering. The dashed line indicates the eect of using a lower cuto frequency for the low pass ®lter. This may lead to a misinterpretation of the movement data. The bottom plots show the result of the new algorithm.
These outliers create false variations in the movement trajectories. When more than four cameras are used the occurrence of outliers in the tracked data will be reduced. The consideration on the setup of data collection and/or cost constraints will sometimes limit the total number of cameras. Furthermore, the trend in human movement studies is to use ever increasing number of markers to record more segments of the body in motion. Such a trend will keep outliers a persistent problem for automatic tracking of movement data. Previous eorts have been dealing with blocked markers and developing an interpolation scheme to ®ll the gap in the trajectory. Other kinds of outliers have been treated together with general noises. Some eective ®ltering algorithms have been developed that can automatically determine an optimal cuto frequency for a movement data under study (Hatze, 1981; Woltring, 1980). Relying on low pass ®lter alone is not adequate in processing certain
J. He, C.-X. Tian / Human Movement Science 17 (1998) 189±200
199
movement data that contain outliers, as we have shown in Fig. 3. Other than the zigzag, or oscillating outliers, the low pass ®lter will only ¯atten the eect of many other outliers and leave some false variations on an otherwise smoother trajectory. Imagine that such a bump occurs at the beginning or at the turning point of a movement, the bump will create a sharper turn than the actual one, generating a false impression of faster initial velocity or a quick reversal of movement direction. We have developed a new algorithm based on the special characteristics of the outliers and their eect on the overall smoothness of a trajectory. We showed by examples that the algorithm can process the tracked data to pick up outliers of various origins automatically with little human operator interaction. When used in combination with existing ®ltering and interpolation algorithms, the new algorithm can improve the quality of camera recorded movement trajectory. The new algorithm is limited in its capability to remove or correct for all outliers. It may not work well when the switching in merged marker paths has a smooth beginning because of its reliance on relatively larger velocity variations of outlier points. It may not work well either for the case where outliers span a long period in a trajectory. If a data series contain too many outliers such that the statistical smoothness measure becomes too large, the algorithm will also fail. A human operator's judgment is still required in such cases. The ultimate solution to remove outliers is to use more cameras and to improve tracking technology such that each marker can be uniquely and accurately tracked, eliminating the error from its source. Acknowledgements The authors wish to acknowledge the partial support of the Whitaker Foundation for the project. References Ball, K.A., Pierrynowski, M.R., 1995. Estimation of six degree of freedom rigid body segment motion from two dimensional data. Human Movement Science 14, 139±154. Hatze, H., 1981. The use of optimally regulated Fourier series for estimating higher-order derivatives of noisy biomechanical data. J. Biomechanics 14, 13±18. Mah, C.D., Hulliger, M., Lee, R.G., Callaghan, I.O., 1994. Quantitative analysis of human movement synergies: constructive pattern analysis for gait. J. Motor Behavior 26, 83±102.
200
J. He, C.-X. Tian / Human Movement Science 17 (1998) 189±200
MacRe¯ex User Manual by Qualisys, Inc., 1994. Winter, D.A., 1991. The Biomechanics of Human Movement, 2nd ed. Wiley, New York. Woltring, H.J., 1980. A FORTRAN package for generalized, cross-validatory spline smoothing and dierentiation. Adv. Engineering Software 8 (2), 104±113.
A statistical smoothness measure to eliminate outliers in motion trajectory tracking Jiping He *, Chun-Xiang Tian Whitaker Center for Neuro-Mechanical Control, Chemical, Bio and Materials Engineering, Arizona State University, ECG 202, MS-6006, Tempe, AZ 85287-6006, USA
Abstract The automatic tracking of motion data recorded via digital cameras has facilitated human movement study. Outliers in the recorded camera data, however, are still serious problems for reconstructing movement trajectory. Outliers arise from tracking errors, such as missing markers, switched positions of dierent markers in tracked data, and a block of data points in the trajectory jumping back and forth between two values caused by discrepancies between the two lenses of the same camera. General ®ltering algorithms can reduce, but not eliminate the ill eect of outliers on the movement trajectory. We have developed a simple algorithm to process the tracked motion data to eliminate several common outliers automatically. The algorithm is based on the statistical smoothness characteristics of the camera data and under the assumption that a human movement trajectory should not contain any sudden shifts. Using a statistical smoothness measure and searching pairs of opposite peaks in marker velocities, a three step approach in the algorithm can remove outliers of various origins automatically. When combined with interpolation and ®ltering algorithms, the gaps in the trajectory due to outlier removal can be ®lled with reasonable values, and a smooth and accurate movement trajectory will be achieved. Ó 1998 Elsevier Science B.V. All rights reserved. PsycINFO classi®cation: 1330; 3297; 4010; 4100 Keywords: Tracking error; Filter; Motion analysis; Smoothness; Statistics; Trajectory; Posture
*
Corresponding author. Tel.: +1 602 965 0092; fax: +1 602 965 0037; e-mail: [email protected].
0167-9457/98/$19.00 Ó 1998 Elsevier Science B.V. All rights reserved. PII: S 0 1 6 7 - 9 4 5 7 ( 9 7 ) 0 0 0 2 9 - 8
190
J. He, C.-X. Tian / Human Movement Science 17 (1998) 189±200
1. Introduction Tracking and reconstructing accurate movement trajectory is an indispensable process for a motion analysis system, especially for optical systems using passive markers (Winter, 1991). Dropped out or switched marker positions, however, have frequently frustrated many operators of such systems (Ball and Pierrynowski, 1995; Mah et al., 1994). Additional cameras or rearrangement of markers can reduce, but cannot completely eliminate, the occurrence of such tracking errors. In many situations the total number of cameras is limited due to the setup and the cost considerations. Therefore, tracking errors will continue to be a problem to be reckoned with in analyzing movement data. Several factors may contribute to tracking errors. The most common one is from dropped out markers that are blocked (Ball and Pierrynowski, 1995). Another is from the merging of two or more markers when their paths coincide. These two situations may occur simultaneously or become the source of one another. A third situation arises when two cameras or the two lenses of the same camera do not generate the same position value for a marker. These three tracking errors have dierent eects on the reconstructed movement trajectory. The dropped out markers are easiest to recognize because they present a gap in the trajectory (depending on the algorithm used, the position value of gap can be zero or in®nity). They have received extensive attention and several algorithms have been developed to correct the error, if the gap is small and/or movement is slow (Ball and Pierrynowski, 1995; Mah et al., 1994). Most tracking systems contain algorithms to remove these blocked marker positions and to ®ll the small gap automatically. The other two kinds of tracking errors will generate a period of shifted path or zigzag path along an otherwise smooth trajectory. Fig. 1 illustrates one such situation that has been dicult to correct by automatic tracking algorithms. Each ball in the ®gure represents a marker position and the ®gure shows the paths of two markers right before, during and after the intersection for three sampling periods. The paths of the two markers change direction after the intersection. This change of direction at the intersection point poses problem for interpolation±extrapolation algorithms used in most tracking systems. When two markers' paths intersect in the view ®elds of some cameras while other cameras' views are blocked or partially blocked, the prediction from the tracking system may lead to the switching of the two markers. As a result the next few frames may be represented by outliers in the recorded data due to the shifted markers. Such shift in marker
J. He, C.-X. Tian / Human Movement Science 17 (1998) 189±200
191
Fig. 1. Illustration of possible tracking errors when two marker paths cross at one point. Each ball represents a sampled position and the paths are consisted of three hypothetical samples of two markers. Most tracking algorithms use previous data points of each marker to predict the next position as a guidance in distinguishing markers (dotted lines). After two markers merge at a point, if paths of the two markers change directions, the markers may be mislabeled from that point onward. Most tracking algorithms will eventually correct this error with the help of other constraints such as ®xed distance between markers attached to the same rigid segment. The result in the tracked data is a period of shifted path in an otherwise smooth trajectory.
trajectory will not last long if one uses a multiple camera system and crosscamera validation in the tracking system. It will create a marker trajectory that contains one or a few consecutive shifted positions as shown in Fig. 2. It is often necessary for a human operator to manually pickup these outliers. The procedure can be a very tedious and time consuming one if a large number of markers are used or/and data from many trials are collected. We will present a simple, yet very eective algorithm for picking up outliers in 3D motion data created by various tracking errors. The algorithm is based on the special characteristics of outliers and patterns of their derivatives. We will show that using low pass ®lters cannot eliminate the eect of all outliers on tracked trajectories and the ®ltered data present some false bumps in an otherwise smooth trajectory. The new algorithm is successful to eliminate almost all outliers created by tracking errors. 2. Methods In one of our postural perturbation studies, we collected kinematic data from 16 subjects. The task is to maintain the upright posture when a short duration impulsive force is applied to the body. Since the force is impulsive and fairly large, a quick motion at the pelvis level is induced and often a step
192
J. He, C.-X. Tian / Human Movement Science 17 (1998) 189±200
Fig. 2. Schematic plots about how the outliers are identi®ed. The top two plots show the marker trajectory, the bottom two show the derivative of marker position. Both individual outliers (frames 148±165) and a period of continuous outliers (frames 167±177) are present. The two dotted lines indicate the smoothness boundary for initial identi®cation of suspected outliers (index vector X and sign vector S, see text for detail). Pair-matching of opposite signs in S allows the ®nal identi®cation of true outliers in X. Note that if all data points outside the boundary are considered outliers, true data for frames 234± 245 will all be chosen as outliers. If pair-matching is not used, the continuous outliers from frames 167 to 177 will be missed. The small zigzag of marker positions in this period is caused by disagreement between the left and the right lenses of the same camera.
is taken by a subject to maintain the posture. A four-camera MacRe¯ex system was used in the study, with each camera sampling at 120 Hz (interlacing of two lenses sampling at 60 Hz). We used 16 markers for a ®ve-segment model: head±trunk±arm, pelvis, shank, shin, and foot. We placed three markers on each segment and one extra marker on the heel of the counter-lateral foot to record stepping. We recorded at least 25 trials from each subject, with each trial consisting of 4 s of motion data. The typical movement trajectory produced by the automatic tracking program often contains ranges of data as shown in the top plot of Fig. 2. Evidently, tracking errors occurred but they are not all due to missing markers. MacRe¯ex system provides an algorithm based on interpolation±extrapolation of previous marker points to predict next positions for isolated mark-
J. He, C.-X. Tian / Human Movement Science 17 (1998) 189±200
193
ers. While the algorithm can automatically and correctly ®ll the blocked marker positions if the gap is small, it does not always work for the tracking situations presented in Fig. 1 as discussed in Section 1. It sometimes generates more tracking errors because of wrongly predicted positions. Since these errors may be well within the variation range of a marker trajectory (Fig. 2), one may overlook them and rely on low pass ®lter to eliminate their eect on the movement trajectory. The eect of these tracking errors cannot be completely eliminated by a low pass ®lter, as will be shown later in Fig. 3. These outliers will present some false variations (bumps) in an otherwise smooth trajectory. Manual tracking frame by frame at troubled spots can correct these errors. This approach will be workable if the data volume is reasonably small or the troubled spots are limited and predictable. If the data size is huge and the troubled spots are numerous and randomly distributed, manual tracking data frame by frame will be labor intensive. An automated process to pickup outliers is certainly desirable. The challenge is to determine whether a data point is an outlier based on a uniform criterion.
Fig. 3. Comparison of marker trajectories processed by dierent methods. The top left plot is the raw data of a marker position from the tracking program. Additional outliers of dierent kinds are added to examine the eectiveness of the new algorithm (highlighted by an ellipsoidal circle). The top middle plot is after ®lling the drop-out markers. The top right is after processing by the new algorithm. The bottom row is low-pass ®ltered data of the top row, using Woltring's spline ®ltering program. Notice that outliers create several false variations on the smooth trajectory after ®ltering by Woltring's program. These variations disappear after all outliers are removed.
194
J. He, C.-X. Tian / Human Movement Science 17 (1998) 189±200
Examining how a human operator detects outliers will generate insights into an eective algorithm. By inspecting marker trajectories (as the one shown in Fig. 2), a human operator can easily locate outliers by surveying the overall trend of the trajectory. Therefore, an evaluation of some statistical smoothness of tracked data may hold the key for eliminating outliers. The sudden shift in marker positions caused by outliers will be ampli®ed by differentiating the recorded trajectory. Outliers will present some special characteristics in the velocity useful for identifying these outliers. Let P be a marker attached to a body segment. The marker position at time t i is denoted by xi . The raw marker trajectory can then be expressed as y t x1 x2 ; . . . ; xiÿ1 xi xi1 ; . . . ; xn :
1
The velocity of y(t) is calculated using the ®nite dierence: y_ xi1 ÿ xi =Dt
2
where Dt 1/ns , and ns is the sampling rate of the camera system. _ and the standard deviation, ry_ , of y_ are then calculated. An The mean, y, outlier will not contribute to the mean velocity because it has almost symmetric positive and negative peaks if the data are not ®ltered. The standard deviation of the raw velocity will be much larger due to the presence of outliers. The smoothness of the trajectory can be evaluated by comparing all data points in the velocity vector with boundaries de®ned by y_ kry ; 3 where k is a movement dependent tolerance factor to be selected by the user. Every data point xi , for all i 2 1; 2; . . . ; n, that satis®es the following condition is declared a `suspected' outlier: 4 y_i > y_ kry_ or y_i < y_ ÿ kry_ : Then put i into an index array and a sign indicator into another array for later processing of true outliers. The `tolerance' factor k is selected according to the quality of the data. More speci®cally, k should re¯ect the ratio of noise contribution from tracking error to that from other sources. k 2 was found to work well for most of our data. If the recorded movement contains some sharp turns or sudden starts or stops, the maximum velocity of the true trajectory may exceed the smoothness boundary speci®ed by kry_ . The bottom left plot in Fig. 2 illustrates the situation where the movement velocity exceeds the lower bound. There-
J. He, C.-X. Tian / Human Movement Science 17 (1998) 189±200
195
fore, not all points lying outside the boundary de®ned by Eq. (3) or other more complex statistics are true outliers. However, these points are often easily identi®able because they occur in speci®c time during a movement. Further analysis of recorded marker trajectory indicates that outliers and their derivatives have certain patterns that can help to eradicate them. Assume that there is an isolated outlier O at frame i. Its position xi is signi®cantly bigger (or smaller) than its neighbors. The dierentiation at the point produces two adjacent bursts, the ®rst corresponding to xi ÿxiÿ1 > 0, and the second corresponding to xi1 ÿxi < 0. Therefore, there will be a twophase burst associated with every isolated outlier in the velocity plot, (Fig. 2, bottom plots). For a sequence of consecutive outliers, such two-phase bursts can also be observed at the start and end of these outliers (between frames 165 and 180 in Fig. 2), even if not all outliers have the same value. This observation suggests that a pair of adjacent peaks with opposite signs in the velocity data are the key characteristics of an outlier or a sequence of consecutive outliers. By specifying a window width w and locating the matching peaks within w, a sequence of outliers can be singled out among all the suspected outliers. Finally, the values for outlier points can be estimated by any interpolation algorithm. We have used Woltring's spline program (Woltring, 1980). The algorithm to pick up outliers from camera recorded data can be summarized as a generic procedure with three steps. Step 1: Establish smoothness criterion and pick up suspected outliers. Dierentiate un®ltered marker positions to obtain the raw velocity. Calculate the mean and the standard deviation of the velocity. Select a value for k and pick out suspected data points for outliers according to y_ kry : Gather all indices corresponding to these outliers in an ascending order and put them in an integer vector X. Find the corresponding velocity sign to each index in X and record it in a vector S. Step 2: Check for matching index pairs in X with opposite signs among data points in S for true outliers. Select window width w A. Identify isolated outliers with the pseudo-code: For each valid element X[i] of X do if (S[i] and S[i+1] are of different sign) AND (X[i])X[i+1] 1) then Record outlier index, X[i]
196
J. He, C.-X. Tian / Human Movement Science 17 (1998) 189±200
Drop X[i] and X[i+1] B. Identify clustered outliers with the pseudo-code: For each valid element X[i] of X do if (S[i] and S[i+1] are of different sign) AND (X[i])X[i+1]) < w) then Record outlier indices, from X[i] to X[i+1]-1 Drop X[i] and X[i+1] Step 3: Fill the values for outlier data points using spline interpolation. These three steps may have to be repeated several times to eliminate all outliers. 3. Results We used the new algorithm to process the data recorded by the MacRe¯ex system in our posture perturbation study. A trajectory section for one of the 16 markers on the vertical axis is shown in the top left plot of Fig. 3. The trajectory is produced by the automatic tracking program in MacRe¯ex system. Drop-out points and some other outliers can be observed in the original data. We modi®ed the data to include some additional outliers (circled regions) to test the eectiveness of the new algorithm for automatically eliminating outliers. The blocked marker positions (drop-outs) were replaced by 0's for the convenience of presenting data. The tracking program in the MacRe¯ex system can identify blocked marker positions (drop-outs) because they are represented by in®nity in the raw camera data. The program replaced these missing points by an interpolation algorithm to improve the quality of the tracked data, as shown by the second plot on the top row in Fig. 3. The outliers created by other tracking errors, however, were left untouched because they were well within the valid data range for the tracking system. The additional outliers merged into the original data were not changed as expected because we added them after the tracking. The last column of plots in Fig. 3 shows the data processed by the new algorithm. The original data together with hypothetical outliers were used as the input data to test the new algorithm (the top left plot). The processed data is shown by the trace in the top plot of the right column. The algorithm is successful in picking up dropped out marker positions, outliers created by other tracking errors, and the outliers we merged in. In this testing data,
J. He, C.-X. Tian / Human Movement Science 17 (1998) 189±200
197
we have included four types of outliers: large outliers (dropped out markers with values outside the valid data range), single outliers, a range of zigzag outliers, and a range of continuous outliers. The algorithm is eective in identifying all of them and replacing their values with interpolated ones. For comparison we used Woltring's spline ®ltering program to smooth the marker data under all three conditions. The optimal cut-o frequency was automatically selected in Woltring's algorithm for each data set. The low pass ®lter did smooth out the zigzag outliers because they resembled a high frequency noise with larger amplitude values. The ®lter did not eliminate the effect of outliers on the trend of the data, producing a false impression that the recorded trajectory had some sharp turns (the left two plots at the bottom row). After eliminating all outliers, Woltring's spline ®ltering program removed high frequency noises in the data ®nally to produce a trajectory that is truthful of the real movement, as shown in the bottom right plot. This indicates that a well conceived and designed ®lter algorithm can eliminate noises and certain types of outliers, but cannot correct errors due to outliers in general. Fig. 4 shows another example. Plots in the left column show the marker traces after processed by ®lling dropped markers only (middle) and by the new algorithm (bottom one). Plots on the right column show the eect of low pass ®ltering (Woltring's algorithm) of corresponding data from the left column. Notice that in the ®ltered data, the outliers at the beginning of the movement and during the movement create false variations. If one uses a lower cuto frequency for ®ltering, the individual ``bumps'' may merge together to form a single slow rising bump at the beginning of the movement, as illustrated by the dashed line. This will generate a false impression or even misinterpretation for the recorded movement data. This is one example why over ®ltering of noisy data may not be desirable. The bottom plots show the eectiveness of the new algorithm. It removes outliers and ®lls the gaps using simple spline interpolation. Filtering of this processed data further removes high frequency noises and generates a very smooth yet truthful marker trajectory. 4. Discussion Tracking errors in camera recorded motion data distort the movement trajectory. Outliers created by tracking errors cannot be removed by ®ltering algorithms no matter what optimal parameters for the ®lter might be used.
198
J. He, C.-X. Tian / Human Movement Science 17 (1998) 189±200
Fig. 4. An example to illustrate the ill-eect of over ®ltering. The raw data is the actual camera data for one marker from a postural perturbation study. The top plots show the big eect of dropped markers on the trajectory. The middle plots show the marker traces after removing of dropped markers and low pass ®ltering. The dashed line indicates the eect of using a lower cuto frequency for the low pass ®lter. This may lead to a misinterpretation of the movement data. The bottom plots show the result of the new algorithm.
These outliers create false variations in the movement trajectories. When more than four cameras are used the occurrence of outliers in the tracked data will be reduced. The consideration on the setup of data collection and/or cost constraints will sometimes limit the total number of cameras. Furthermore, the trend in human movement studies is to use ever increasing number of markers to record more segments of the body in motion. Such a trend will keep outliers a persistent problem for automatic tracking of movement data. Previous eorts have been dealing with blocked markers and developing an interpolation scheme to ®ll the gap in the trajectory. Other kinds of outliers have been treated together with general noises. Some eective ®ltering algorithms have been developed that can automatically determine an optimal cuto frequency for a movement data under study (Hatze, 1981; Woltring, 1980). Relying on low pass ®lter alone is not adequate in processing certain
J. He, C.-X. Tian / Human Movement Science 17 (1998) 189±200
199
movement data that contain outliers, as we have shown in Fig. 3. Other than the zigzag, or oscillating outliers, the low pass ®lter will only ¯atten the eect of many other outliers and leave some false variations on an otherwise smoother trajectory. Imagine that such a bump occurs at the beginning or at the turning point of a movement, the bump will create a sharper turn than the actual one, generating a false impression of faster initial velocity or a quick reversal of movement direction. We have developed a new algorithm based on the special characteristics of the outliers and their eect on the overall smoothness of a trajectory. We showed by examples that the algorithm can process the tracked data to pick up outliers of various origins automatically with little human operator interaction. When used in combination with existing ®ltering and interpolation algorithms, the new algorithm can improve the quality of camera recorded movement trajectory. The new algorithm is limited in its capability to remove or correct for all outliers. It may not work well when the switching in merged marker paths has a smooth beginning because of its reliance on relatively larger velocity variations of outlier points. It may not work well either for the case where outliers span a long period in a trajectory. If a data series contain too many outliers such that the statistical smoothness measure becomes too large, the algorithm will also fail. A human operator's judgment is still required in such cases. The ultimate solution to remove outliers is to use more cameras and to improve tracking technology such that each marker can be uniquely and accurately tracked, eliminating the error from its source. Acknowledgements The authors wish to acknowledge the partial support of the Whitaker Foundation for the project. References Ball, K.A., Pierrynowski, M.R., 1995. Estimation of six degree of freedom rigid body segment motion from two dimensional data. Human Movement Science 14, 139±154. Hatze, H., 1981. The use of optimally regulated Fourier series for estimating higher-order derivatives of noisy biomechanical data. J. Biomechanics 14, 13±18. Mah, C.D., Hulliger, M., Lee, R.G., Callaghan, I.O., 1994. Quantitative analysis of human movement synergies: constructive pattern analysis for gait. J. Motor Behavior 26, 83±102.
200
J. He, C.-X. Tian / Human Movement Science 17 (1998) 189±200
MacRe¯ex User Manual by Qualisys, Inc., 1994. Winter, D.A., 1991. The Biomechanics of Human Movement, 2nd ed. Wiley, New York. Woltring, H.J., 1980. A FORTRAN package for generalized, cross-validatory spline smoothing and dierentiation. Adv. Engineering Software 8 (2), 104±113.