Journal of Electromyography and Kinesiology 20 (2010) 777–786
Contents lists available at ScienceDirect
Journal of Electromyography and Kinesiology journal homepage: www.elsevier.com/locate/jelekin
Uncovering patterns of forearm muscle activity using multi-channel mechanomyography Natasha Alves, Tom Chau * Bloorview Research Institute, Bloorview Kids Rehab., Inst. of Biomaterials and Biomedical Engineering, University of Toronto, Toronto, ON, Canada M4G 1R8
a r t i c l e
i n f o
Article history: Received 17 July 2009 Received in revised form 12 August 2009 Accepted 16 September 2009
Keywords: Mechanomyogram Pattern recognition Muscle activity Fisher ratio Genetic algorithm
a b s t r a c t A coordinated activation of distal forearm muscles allows the hand and fingers to be shaped during movement and grasp. However, little is known about how the muscle activation patterns are reflected in multi-channel mechanomyogram (MMG) signals. The purpose of this study is to determine if multisite MMG signals exhibit distinctive patterns of forearm muscle activity. MMG signals were recorded from forearm muscle sites of nine able-bodied participants during hand movement. By using 14 features selected by a genetic algorithm and classified by a linear discriminant analysis classifier (LDA), we show that MMG patterns are specific and consistent enough to identify 7 ± 1 hand movements with an accuracy of 90 ± 4%. MMG-based movement recognition required a minimum of three recording sites. Further, by classifying five classes of contraction patterns with 98 ± 3% accuracy from MMG signals recorded from the residual limb of an amputee participant, we demonstrate that MMG shows pattern-specificity even in the absence of typical musculature. Multi-site monitoring of the RMS of MMG signals is suggested as a method of estimating the relative contributions of muscles to motor tasks. The patterns in MMG facilitate our understanding of the mechanical activity of muscles during movement. Ó 2009 Elsevier Ltd. All rights reserved.
1. Introduction The central motor network coordinates the activation of multiple muscles when performing motor tasks such as maintaining posture, gait or shaping the hand. Patterns of muscle activation have been useful in understanding motor strategies in upper-limb movement (Takatoku and Fujiwara, 2009), finger movement (Darling and Cole, 1990) and hand grasp (Brochier et al., 2004), muscular demand in the upper-extremity (Gagnon et al., 2009) and the influence of muscle fatigue (Clark et al., 2007). The patterns of coordination are often manifested as electromyogram (EMG) signals that are specific to the motor task (Brochier et al., 2004). The existence of distinguishable activity patterns has enhanced our understanding of the central motor network, and has made technologies such as multifunction upper-limb prostheses a possibility (Englehart et al., 2001). While muscle activity is usually monitored by electromyography (EMG), there is also valuable information in the mechanical index of muscle contraction. On contraction, muscles emit lowfrequency vibrations, known as the mechanomyogram (MMG), which can be measured on the surface of the overlying skin. MMG is generated from gross lateral movement of the muscle at
* Corresponding author. Address: Bloorview Research Institute, 150 Kilgour Road, Toronto, ON, Canada M4G 1R8. Tel.: +1 416 425 6220x3515; fax: +1 416 425 1634. E-mail addresses:
[email protected] (N. Alves),
[email protected] (T. Chau). 1050-6411/$ - see front matter Ó 2009 Elsevier Ltd. All rights reserved. doi:10.1016/j.jelekin.2009.09.003
the initiation of a contraction, smaller subsequent lateral oscillations at the resonant frequency of the muscle, and dimensional changes of active muscle fibers (Barry and Cole, 1990; Orizio, 1993; Orizio et al., 1990). These mechanical vibrations have been measured by microphones (Watakabe et al., 2001), piezoelectric contact sensors (Barry, 1991; Watakabe et al., 1998), accelerometers (Barry, 1992) and laser distance sensors (Orizio et al., 1999) on the surface of the skin. MMG has been applied in the monitoring of muscle pain (Madeleine and Arendt-Nielsen, 2005), the study of muscle fatigue (Shinohara and Sogaard, 2006; Madeleine et al., 2002) and in providing biofeedback in ergonomics (Madeleine et al., 2006). In comparison to EMG, MMG provides a better estimation of inflection points in motor-unit recruitment strategies (Akataki et al., 2004). It has been used in conjunction with EMG to measure muscle electro-mechanical coupling efficiency which has been useful in monitoring the dystrophic process (Orizio et al., 1997) and paediatric muscle disease (Barry et al., 1990). A description of MMG patterns is a critical step towards a better understanding of the mechanical activity of muscles during movement control. In a recent study we found that, after the initial burst of muscle activity at contraction onset, 20% of the MMG signals recorded from three forearm muscle sites were non-stationary during functional grasp (Alves and Chau, 2008). We also suggested that the stationary test statistic may be a discriminatory signal feature, and its distribution may indicate the time-course of relative muscle contributions to functional grasp. We extend this work by investigating the temporal patterns and discriminability of
778
N. Alves, T. Chau / Journal of Electromyography and Kinesiology 20 (2010) 777–786
MMG signals from six forearm muscles during hand movement. The existence of discernable MMG patterns may have applications in the control of body-machine interfaces, such as upper-limb prostheses (Xie et al., 2009; Silva et al., 2005; Barry et al., 1986) or alternative access devices for individuals with severe physical disabilities (Barreto et al., 2000). Since MMG provides information about the number and firing rates of recruited motor units during voluntary isometric contraction (Orizio et al., 1996), we hypothesize that patterns of muscle activity will be reflected as discernable patterns in MMG signals. In this study, we address this hypothesis by focusing on MMG signals recorded from forearm muscles during hand movement. A pattern-recognition approach was adopted to differentiate among multiple classes of muscle activity. We examine the time-course of MMG activity and address issues such as sensor placement and multiplicity. Further, we show the significance of multi-site MMG in estimating muscle activity and its ability to discriminate among multiple contraction classes in participants with typical and atypical musculature.
Fig. 2. Forearm muscle sites at which the six MMG sensors were positioned. The sites, labelled 1–6, show the sensor placement for PT, FCR/PL, FCU, EDC, ECR, and ECU recording, respectively. Note: images modified from Gray (2000).
2. Methods 2.1. Participants A convenience sample of nine able-bodied individuals (4 male), aged 21 ± 1 years, provided written consent to participate in the study. Participants were healthy, had intact forearm musculature, and no previous history of musculoskeletal illness. These participants will be referred to as C1–C9 in this study. A congenital bilateral transhumeral amputee, referred to as A10, was also recruited. He was an active user of his residual upper limbs. A10 had extensive control of the residual muscles: he could pick guitar strings with his right limb while strumming with his left. He also used his right limb on a daily basis for keyboard access. 2.2. Instrumentation set-up The key components of the data collection instrumentation are shown in Fig. 1. MMG was recorded by six silicone-embedded microphone sensors (19 19 9 mm, 4 g) manufactured according to the method of Silva et al. (Silva et al., 2005; Silva and Chau, 2003). A custom terminal box was built to interface the MMG sensors with a terminal block (National Instruments, BNC-2095). The MMG signals from the terminal block were channelled through an analog signal conditioning input module (National Instruments, SCXI-1102C), sampled at a rate of 1 KHz (National Instruments, PXI-6052E, 16-bit, ±5 V), and the digitized signals were stored on the controller’s hard drive.
Visual instructions
Participants were seated on a chair fitted with a custom armrest. The arm-rest supported the forearm of participants C1–C9 at the wrist and elbow, ensuring that the hand was free to move and the MMG sensors were not compressed by the forearm. The sensors were affixed to the participants’ dominant forearm at the bellies of the following muscles: Pronator Teres (PT); Flexor Carpi Radialis and Palmaris Longus (FCR/PL); Flexor Carpi Ulnaris (FCU); Extensor Digitorum Communis (EDC); Extensor Carpi Radialis Longus (ECR); and Extensor Carpi Ulnaris (ECU), shown in Fig. 2. The sensor sites were located by palpating the respective muscles. A tri-axis accelerometer (MMA7260Q, Freescale Semiconductor) was affixed to the participant’s hand for subsequent determination of hand movement times. To minimize mechanical coupling among the sensors, each sensor was individually affixed with a velcro strap, and contact among the straps was avoided. Double-sided tape was not used due to interference with the sensor’s silicon contact membrane. For the amputee participant (A10), sensor sites were located while he contracted the muscles of his right residual limb in numerous ways. His muscles were palpated during these movements, and six sites where activity was predominant, such as his biceps and triceps, were selected for sensor attachment. 2.3. Experiment protocol Participants were instructed not to perform fatiguing upperlimb exercise twenty-four hours before the sessions. A custom
Controller
Digital data
A/D Converter Conditioned analog signals
MMG sensor Interface Box
Arm supports
Terminal Block
Hand movements Fig. 1. Instrumentation for data collection.
Signal Conditioner
N. Alves, T. Chau / Journal of Electromyography and Kinesiology 20 (2010) 777–786
LabView graphical user interface (GUI) with a manual trigger was used to start data acquisition and visually cue participants (C1– C9) to perform various classes of muscle activity corresponding to eight hand motions: hand open, hand close, wrist flexion, wrist extension, pronation, supination, adduction, and abduction. Subsets of these target hand positions have been used in previous studies on EMG pattern recognition (Englehart et al., 2001; Fukuda et al., 2003; Boostani and Moradi, 2003). The target movements for participant A10 comprised of five self-selected contractions that A10 perceived to be different and repeatable. Participants performed 80 repetitions of each of the motions in a pre-defined order. Each motion was comprised of the full range of motion from the resting position to the target position, followed by 3 s of the limb being held in the target position. Participants were instructed to return to the resting position for 3 s before being prompted to perform the next motion. The data were collected in multiple sessions, each no more than 7 min long. To ensure that muscle fatigue was not an issue, participants were given up to 2 h of rest between each session. The experimental protocol was approved by the hospital and university research ethics boards, and was in accordance with the Declaration of Helsinki. 2.4. Signal pre-processing The recorded data were continuous streams of multi-channel MMG signals. The data were subsequently spliced into individual 3-s recordings, each corresponding to one of the eight hand movements (five contraction types for A10) or rest. For able-bodied participants, contraction onset times were determined by the first indication of hand movement detected by the tri-axis accelerometer on the participant’s hand. The signals were band-pass filtered between 5–100 Hz to attenuate the effects of movement (Madeleine et al., 2001) and any noise beyond the accepted MMG signal range. 2.5. Estimation of the time-course of muscle activity To study the time-course of muscle activity, the median root mean square (RMS) value of the MMG signals was evaluated within non-overlapping 100 ms epochs for each muscle site and each class of hand/limb movement. The RMS is a statistical measure of qffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi P the magnitude of the signal, and is given by fRMS ¼ N1 Ni¼1 x2i where xi is the ith sample and N is the number of samples. 2.6. Recognition of patterns of muscle activity When performing statistical pattern recognition, each MMG pattern is represented by D features that contain enough information to maintain generalization while separating different classes into compact and disjoint regions in the D-dimensional space (Jain et al., 2000). When working with a limited data set, the curse of dimensionality can be alleviated by selecting a small number of salient features. Since exhaustive feature selection is computationally prohibitive even for modestly sized feature sets, feature selection methods usually comprise of sub-optimal search algorithms such as sequential forward/backward selections (Jain et al., 2000). Individual feature selection is recommended as a first step in reducing the dimension of the input feature vector when the original feature set is large i.e. includes hundreds of features (Jain et al., 2000). Feature dimensionality should be further reduced by methods that consider the dependencies among the features. We propose the following stages for MMG pattern recognition: representing the MMG signals as a feature vector; identifying individual discriminatory features by Fisher’s ratio (FR) analysis; selecting jointly discriminatory features using a genetic algorithm (GA) that
779
optimizes classification accuracy, and evaluating the reduced feature set with a linear discriminant analysis (LDA) classifier. These steps are detailed in the following sections. We refer the reader who is not familiar with statistical pattern recognition to Duda et al. (Duda, 2001). 2.6.1. Feature measurement Since it was unclear which features best represent the discriminatory information in MMG signals, we measured a comprehensive set of 60 time-domain, frequency-domain and time– frequency features from each MMG channel. This large set of measurements increased the possibility of finding discriminatory features in the subsequent feature-selection steps. These features included auto-regressive coefficients, relative power in frequency bands, RMS measures, mean, mean of absolute amplitude (MAP), variance, median, inter-quartile range, skew, kurtosis, normality, the stationary test statistic (Alves and Chau, 2008), waveform length, Wilson’s amplitude Boostani and Moradi, 2003), slope-sign change, peak and median frequencies, and wavelet transform (WT) coefficients. Subsets of these measured features have shown promising results in the classification of EMG and other biomedical signals (Boostani and Moradi, 2003; Zecca et al., 2002). The following describes the computation and physiological relevance of some important features: The RMS of the MMG signal is related to the force of muscle contraction (Madeleine et al., 2001), and has been previously used to detect and classify muscle activity (Silva et al., 2005). The waveform length (WLEN) is the cumulative length of the waveform over the time segment, and is defined as fwlen ¼ PN i¼2 jxi xi1 j. This feature is a combined measure of the waveform amplitude, frequency and duration (Zecca et al., 2002). The cepstrum is the inverse Fourier transform of the log of the P njx signal spectrum, i.e. logðSðxÞÞ ¼ 1 where cn are ceps1 cn e tral coefficients. This all-pole spectral representation is important for speech and speaker recognition applications (Atal, 1974). In this study, the cepstral coefficients were calculated from the 7th order linear prediction filter coefficients (ai), using P k c1 = a1 and cn ¼ n1 k¼1 ð1 nÞak cnk þ an (Atal, 1974). The cepstrum has high resolution in the lower frequency range where much of the MMG signal information lies (Orizio et al., 1990).
2.6.2. Pre-selection by Fisher ratio analysis Each of the 360 features (60 features 6 channels) were ranked according to its independent multi-class discriminatory power by evaluating its Fisher score (Jd). The Fisher score is a ratio of the between-class scatter to the average within-class scatter. It can be generalised to K classes by evaluating 12 KðK 1Þ Fisher criteria. For a multiclass problem, the Fisher criterion for the dth feature is defined as (Loog et al., 2001)
Jd ¼
K1 X K X
pi pj ðli;d lj;d Þ2 ðpi r2i;d þ pj r2j;d Þ1
ð1Þ
i¼1 j¼iþ1
where K is the number of classes, pi is the priori probability of class i, and li,d and r2i;d are the mean and variance of the dth feature for class i. 2.6.3. Feature selection by the genetic algorithm When working with a finite training sample size of N samples per class, classification error is dependent on the classification rule, feature dimensionality (D), the joint feature distribution, and the asymptotic (N = 1) probability of misclassification (PMC) (Hua et al., 2005; Raudys and Jain, 1991). For Fisher’s LDA classifier, D < (N/4.5) assuming the asymptotic PMC is 0.01, and D may be
780
N. Alves, T. Chau / Journal of Electromyography and Kinesiology 20 (2010) 777–786
further increased when the estimated asymptotic PMC is increased (Raudys and Jain, 1991). Since the size of the training set was small i.e. 72 samples per class when using 90% of the 80 hand movement samples, the upper limit on feature dimensionally should be D = 15. The problem of dimensionality reduction can be formulated as an optimization problem, where we use a Do-dimensional feature set to search for D features (D < < Do) while maximizing classification accuracy. In this study, Do = 360, and the dimensionality of the reduced feature set was varied such that 4 6 D 6 14, where D is an integer. The solutions for each dimensionality were found by a genetic algorithm (GA). The genetic algorithm is an evolution-based optimization procedure that progressively creates new sets of solutions with the goal of maximizing the optimization criterion. For a detailed description on GAs for feature selection, please refer to (Siedlecki and Sklansky, 1989). To improve the chances of the GA reaching an optimal solution, we used the 30 features with the highest Fisher score to find optimal D-dimensional feature sets (4 6 D 6 15). Here we assumed that the features that work in combination to separate the feature space also have some unidimensional discriminatory power. The GA search comprised of 100 generations, each with a population of 1000 chromosomes, a 50% cross-over rate, and a 30% mutation rate. The optimization criterion was the average classification rate of the selected features on the training data using fivefold cross validation with an LDA classifier. For each dimensionality, the search consisted of two runs of the GA for randomly selected initial populations. The feature set with the minimum classification error was selected for subsequent evaluation of the MMG pattern classifier.
hand motion typically generated MMG signals whose RMS peaked earlier and with higher amplitude. This was seen at the ECR site during extension and the FCR site during flexion. Although the temporal evolution of signal RMS recorded from the muscles of participant A10 showed distinct patterns for the various types of contractions, as seen in Fig. 3b, there were some notable differences when compared to the able-bodied participant group. First, the RMS peaked within 200 ms from contraction onset, and this peak-time did not show any distinction for the different contractions. However, the peak amplitude and steady-state amplitude did exhibit good differentiation among contractions. The RMS values also stabilized earlier than the able-bodied case, typically within 600 ms from contraction onset. 3.2. RMS values and an indicator of muscle activity
2.6.4. Estimation of discrimination The MMG data were separated into training and test subsets using 10-fold cross validation. In each fold, the training set was used for feature selection and to train an LDA classifier. The classifier’s ability to discriminate among the classes of activity was evaluated on the reserved test set. The average accuracy of the test set over 10 folds of cross-validation was reported as the estimate of MMG discrimination.
Fig. 4 shows the standardized RMS values of the MMG signals at each muscle site for 8 classes of hand movement, averaged across the participants. RMS was evaluated for 1000 ms long MMG signals. At the FCR site, RMS is higher during flexion than at extension. The reverse is observed at the antagonist ECR site, where values are low at flexion and high at extension. This opposing relative activity is also observed at the adductor sites (ECU and FCU) and abductor sites (ECR and FCR) during adduction and abduction, and at the metacarpo-phalangeal extensor (EDC) during hand-open and hand-close. The RMS is relatively low at all muscle sites during hand-open, and high during hand-close, supination and pronation. While RMS values showed physiological correlations with muscle activity, classification accuracies were low when the feature vector comprised solely of six-site RMS values. With an LDA classifier, average accuracies were 62.13 ± 6%, 55.25 ± 6% and 46.93 ± 6% for classifying 8 classes of hand movement using RMS features of MMG signals of duration 1000 ms, 750 ms and 500 ms, respectively. Accuracies were higher for the amputee participant, and had values of 90 ± 5%, 88 ± 3% and 87 ± 5% for discriminating among 5 contractions using RMS features from signals of duration 1000 ms, 750 ms and 500 ms, respectively.
2.7. Visualization of multi-dimensional signal features
3.3. Recognition of patterns of muscle activity
Star plots were used for multivariate visualization of MMG signal features for each class of hand movement. To visualize D features for K = 8 classes of hand movement, the feature values were evaluated and averaged across the 80 repetitions of each movement to obtain and K D matrix for each participant. Each column of the matrix was standardized so that all entries fell within the interval [0,1]. Star plots were used two visualize two types of feature sets: (1) RMS values from the six muscles sites (D = 6), standardized and averaged across participants; and (2) the standardized values of the 10 GA-selected features that were most common among the participants.
Table 1 reports the average classification accuracy for each participant over ten folds of cross-validation for 1000 ms long MMG signals. Results are reported for participant-specific feature selection and classifier training using three types of features sets: (1) RMS values of the MMG signals recorded at the six muscle sites; (2) six features selected by the GA; and (3) fourteen features selected by the GA. For the able-bodied group, classification accuracies increased from 62 ± 6% when using only the RMS features to 78.5 ± 8% when deploying GA-selected features of the same dimensionality (D = 6). Participant A10 had higher discrimination accuracies of 90 ± 5% for five types of contraction using RMS features, and 96 ± 4% using six GA-selected features. Classification performance was also evaluated for 14-dimensional GA-selected feature vectors. The mean accuracy across the able-bodied participants was 85 ± 5%, and the mean accuracy of the amputee participant was 98 ± 3%. Within the able-bodied group, classification rates were participant-dependent, and were highest for participant C7 and consistently low for participant C3 regardless of the feature selection method or feature dimensionality employed. To examine how many movements we can expect to classify for each participant, we eliminated the movements that could not be detected with greater than 80% accuracy. The most frequent erroneously classified movement was abduction. The classifier could recognize, on average 7 ± 1 hand movements with an accuracy of
3. Results 3.1. Time-course of muscle activation Fig. 3a shows the typical temporal evolution of the MMG signals recorded at the ECR site. The RMS increased from its resting value up to 100 ms before movement was detected by the accelerometer on the participant’s hand. The RMS typically steadily increased, peaked around 200–500 ms after the initiation of movement, decayed, and stabilized after about 1500 ms. The time-to-peak and the steady-state RMS depended on the hand movement being performed. It was observed that the muscle site responsible for the
N. Alves, T. Chau / Journal of Electromyography and Kinesiology 20 (2010) 777–786
781
Fig. 3. Representative time-course of muscle activity. Median RMS values evaluated at 100 ms epochs are shown. (a) MMG signals recorded from participant C9 at the ECR site. The dashed vertical line indicates the time when movement was detected by the tri-axis accelerometer on the participant’s hand. (b) MMG signals recorded from the residual limb of a congenital amputee.
90 ± 4%. High classification results were also attained by other feature extraction methods such as principal components analysis (89 ± 5%, D = 14, 8 classes), where D principal components were selected from the 100 features with the highest Fisher score. 3.4. Time course of movement discrimination The temporal evolutions of the RMS of the MMG signals suggest that MMG patterns are movement-specific at certain time points. A critical issue concerns how much variation there is in hand movement specificity as time elapses. This was addressed by performing feature selection and classification on signals of different durations. Taking into consideration that muscle activation may occur before movement is detected by the tri-axis accelerometer on the hand, MMG signals of three durations, s 2 {500 ms, 750 ms, 1000 ms} include 200 ms of data before accelerometer-detected movement. The average accuracies across the able-bodied participants for discriminating among eight classes of forearm muscle activity are shown in Fig. 5 as a function of duration and feature dimensionality. Classification accuracies in-
crease with increasing signal duration. Rank-sum tests for equal medians show that, for each signal duration, average accuracy shows no significant increases for 10 6 D 6 14 (p > 0.01), reaching values of 68%, 80% and 85% for signal durations of 500 ms, 750 ms and 1000 ms, respectively. Further, classification accuracies were significantly different among these three durations for all values of D (rank-sum, p < 0.05). Since the GA is computationally expensive, the time-course of movement discrimination was further analyzed using features extracted by principal components analysis. We selected D principal components (10 < D < 20) from the 100 features with the highest Fisher score. Signals were analyzed using durations in the range of 250–2500 ms at 250 ms increments. Average classification accuracies initially increase as the duration of the MMG signal increases, and show no significant differences for durations greater than 1000 ms (p > 0.05, rank-sum tests). For participant A10, classification accuracies increased by just 3% when the duration was increased from 250 ms to 500 ms, and there were no significant changes in accuracy for durations greater than 500 ms (p > 0.05, ranksum tests).
782
N. Alves, T. Chau / Journal of Electromyography and Kinesiology 20 (2010) 777–786
Fig. 4. Star plots showing the relative behaviour of the RMS at each muscle site for eight types of hand movement. Each spoke of the plot represents the RMS at the muscle site, averaged across all participants.
Table 1 Classification of muscle activity patterns. Participant
C1a C2 C3 C4 C5 C6 C7 C8 C9 A10b Average (C1–C9)
No. of movements
8 8 8 8 8 8 8 8 8 5 8
RMS features (D = 6) No. of channels
Average accuracy mean + std
5a 6 6 6 6 6 6 6 6 6 6
45 ± 8 71 ± 8 48 ± 4 62 ± 5 61 ± 5 66 ± 5 65 ± 7 55 ± 7 62 ± 6 90 ± 5 62 ± 6c
GA features (D = 6) Average accuracy mean + std
GA features (D = 14)
GA features (D = 14)
No. of channels
Average accuracy mean ± std
No. of movements discernable at >80%
Adjusted accuracy mean + std
68 ± 6 85 ± 4 65 ± 4 82 ± 4 80 ± 4 86 ± 4 88 ± 3 77 ± 3 76 ± 5 96 ± 4 78.5 ± 8
5 3 5 4 6 4 4 5 4 5 4.4 ± 1
78 ± 6 87 ± 3 75 ± 7 86 ± 3 86 ± 3 91 ± 5 92 ± 2 84 ± 4 84 ± 4 98 ± 3 84.7 ± 5
6 8 4 8 6 7 8 8 6 5 7 ± 1.4
84 ± 7 87 ± 3 95 ± 3 94 ± 3 90 ± 3 91 ± 5 92 ± 2 84 ± 4 95 ± 4 98 ± 3 90.2 ± 4
All results are shown for classification of 1000 ms of data. a Recordings from C1 did not include the PT muscle site. b Transhumeral amputee. c C1 excluded.
3.5. Muscle dimensionality for movement discrimination Fig. 6 shows the distribution of the sources of features chosen by the GA (D = 14, s = 1000 ms). Participant C1 was excluded from this analysis, since the sensor at the PT site was faulty. The distribution of feature sources shows muscle-site selectivity, with the ECU site producing the fewest discriminatory features. The frequency of feature selection at the ECU site was significantly different from the PT, FCU and ECR sites (rank-sum, p < 0.01). Not all the muscle sites from which MMG signals were recorded were useful in contributing to hand movement discrimination. The number of muscle sites selected ranged from 3 to 6. On average, the ECU site was not used for 66% of the participants. While MMG discrimination shows muscle site specificity, an increase in the number of discriminatory muscle sites did not always translate to an increase in discrimination accuracy. This was evident in the low accuracies attained for participant C1 (78% for 8 classes) where 5 muscle sites
were deemed useful, when compared to participant C2 (87% for 8 classes) where features from just 3 sites were used.
3.6. Patterns detected by feature selection We examined the 14 features the GA individually selected for each participant and chose the 10 most common features across the participants. Most of the common features stemmed from cepstral coefficients. Fig. 7 shows the standardized feature values for 8 hand movement classes superimposed for participants C2– C9. The asterisks denote features whose standard deviation across participants is less than 0.15. The star plots further illustrate commonalities in MMG patterns across participants. This is especially evident in small variations in standardized feature values across participants during extension, pronation, supination, hand-open and adduction.
N. Alves, T. Chau / Journal of Electromyography and Kinesiology 20 (2010) 777–786
783
Fig. 5. Multidimensionality of the MMG patterns. The plot shows the percentage of correctly recognized patterns as functions of feature dimensionality (4 < D < 14) and signal duration (500 ms, 750 ms, 1000 ms). Features were selected by a GA. Error bars show inter-quartile ranges. Data for each dimension are staggered for visual clarity.
Fig. 6. Sources of discriminatory features. For each participant, feature sources selected by the GA were pooled for 10 folds of cross-validation. In each fold, features were selected by the GA for classifying 8 hand movements. The dark error bars indicate the mean + std percentage of features derived from each site, averaged across participants.
4. Discussion 4.1. Time-course of muscle activation and movement discrimination When a contraction is initiated, mechanical vibrations due to muscle movements generate MMG signals whose amplitudes initially rise and then decay (Barry and Cole, 1990). Our results suggest that the time-course of the RMS of the MMG signals exhibits different peak-times and peak amplitudes for different hand move-
ments, and reflects the level and timing of activity of the multiple muscle groups during hand movement. As seen in Fig. 3a, during able-bodied hand movement MMG signals were still in the transient state 500 ms after movement initiation. MMG signals with a duration of 500 ms may not have sufficiently captured the signal structure, and hence resulted in significantly lower classification accuracies than the 750 ms and 1000 ms scenarios. When muscle contraction did not accompany a range of motion, as in the case of the amputee participant (Fig. 3b), the transient time was smaller
784
N. Alves, T. Chau / Journal of Electromyography and Kinesiology 20 (2010) 777–786
Fig. 7. Star plots showing the standardized values of 10 common GA-selected features for eight types of hand movements. Plots are superimposed for participants C2–C9. The axes indicate the muscle site and feature monitored. CEPn represents the nth cepstral coefficient, and WLEN, MAP, and WT represents the waveform length, mean of absolute power, and percentage of energy in the first 300 ms of contraction (estimated from wavelet transform coefficient details, level 2, sym4 wavelet), respectively. The asterisks denote features whose standard deviation across participants is less than 0.15.
and the temporal-resolution of the course of muscle activity was reduced. This reduced transient period may have enabled the reliable classification of shorter durations of the MMG signal in the amputee case. 4.2. Patterns of muscle activity Multi-site MMG reflects the underlying muscle selectivity during hand movements. This is demonstrated by the ability of simple linear classifiers to recognize hand movements using prudently selected MMG signal features. The recognition rates are dependent on a number of parameters, such as signal duration, feature set, feature dimensionality and, primarily, the participant’s ability to produce repeatable contraction patterns. The higher recognition rates for the amputee participant may be attributed to higher force contractions, and the contraction repeatability being focussed on the muscle activation, rather than repeatability of a range of motion. The star plots of standardized RMS of the signals at the various muscle sites closely agreed with physiological muscle function. On average, we observed higher values at a muscle site when the muscle under consideration was the primary facilitator of the hand movement. This physiological consistency was seen during wrist flexion, extension, abduction, adduction and hand-opening. We therefore suggest that the star plots may be useful in comparing the relative level of muscle activity during motor tasks. The plots also revealed high RMS values at the EDC site during supination. Although the EDC muscle does not play a dominant role in supination, it lies over the deep supinator muscle. The high signal amplitude suggests that the sensors may be capturing some deepmuscle activity. While the RMS of MMG signals demonstrated some task-specificity, the higher pattern recognition rates attained by GA-selected features suggest that there is a complex coding of time and frequency characteristics of the MMG signals during motor tasks. The majority of the GA-selected features stemmed from cepstral coefficients, indicating differences in the spectral patterns of MMG signals generated during different classes of forearm muscle activity. While the entire set of task-specific signal features were
not common among participants, possibly due to differences in sensor placement and motor strategies, there was some consistency across participants in the distributions of the most frequently selected features. The superposition of standardized feature amplitudes revealed that many features had small variations across participants for movements such as hand-open, extension, pronation and supination. This highlights that common motor strategies across participants are reflected by common distributions of the amplitudes of MMG signal features across the hand movement classes. Although the feature values may not be identical across participants, once the values are standardized, they may have identical cluster distributions in the D-dimensional feature space, and these clusters may be specific to the type of movement. 4.3. Feature and muscle dimensionality Selecting the appropriate feature dimensionality is of paramount significance in pattern-recognition since it determines classifier accuracy and generalization. Lower dimensionality is associated with greater generalization, and hence greater credibility. In this study, as dimensionality was increased, classification accuracy climbed until D = 10, and stabilized thereafter. In our pattern-recognition paradigm where 14 features were selected by a GA, the number of requisite muscle sites ranged from 3 to 6. Inter-subject variation of muscle dimensionality may be attributed to limitations in identifying the exact locations of muscles, variations in motor-unit recruitment patterns, and limitations of the sub-optimal GA search in selecting the best features. Overall, the fewest discriminatory features originated from the ECU muscle site. This suggests that, although the ECU muscle plays a prominent role in adduction and extension (Kendall, 2005), the motor-unit recruitment patterns at this site may be uniform across the studied hand movements. 4.4. Limitations GA feature selection and linear classification were chosen since they facilitate a better understanding of the physiological
N. Alves, T. Chau / Journal of Electromyography and Kinesiology 20 (2010) 777–786
relationship between the discriminatory features and the target contractions. As a result of the chosen methodology however, the reported accuracies are by no means an upper-bound on the discriminability of forearm MMG signals corresponding to hand movements. High classification results were also attainable for feature extraction methods, such as principal components analysis. Further, the features selected by the GA may only be partially linearly separable. This may explain the lower classification accuracies reported for participants C1 and C3. The use of non-linear classifiers, such as radial basis networks, may provide better accuracies in these cases. 5. Conclusion Multi-site MMG signals recorded during hand movement show distinctive patterns that are encoded in temporal and spectral features. By using conventional pattern-recognition techniques we have shown that MMG patterns are also distinguishable for individuals without typical musculature. The RMS values of multi-site MMG signals provide a reliable representation of relative muscle activity during hand movement. The ability of simple classifiers to recognize muscle activation patterns from MMG signal features alone suggest that, with further development, MMG may have applications in multi-function body-machine interfaces. Acknowledgement The authors are grateful to Mei Guan for her invaluable assistance with data collection. Special thanks to Dr. Ervin Sejdic for his advice. This work was supported in part by the Ontario Graduate Scholarship, Natural Sciences and Engineering Research Council of Canada, and the Canada Research Chair Program. References Akataki K, Mita K, Watakabe M. Electromyographic and mechanomyographic estimation of motor unit activation strategy in voluntary force production. Electromyogr Clin Neurophysiol 2004;44(8):489–96. Alves N, Chau T. Stationarity distributions of mechanomyogram signals from isometric contractions of extrinsic hand muscles during functional grasping. J Electromyogr Kinesiol: Official J Int Soc Electrophysiol Kinesiol 2008;18(3):509–15. Atal BS. Effectiveness of linear prediction characteristics of the speech wave for automatic speaker identification and verification. J Acoust Soc Am 1974;55(6):1304–22. Barreto AB, Scargle SD, Adjouadi MA. Practical EMG-based human-computer interface for users with motor disabilities. J Rehab Res Dev 2000;37(1):53–63. Barry DT. Muscle sounds from evoked twitches in the hand. Arch Phys Med Rehab 1991;72(8):573–5. Barry DT. Vibrations and sounds from evoked muscle twitches. Electromyogr Clin Neurophysiol 1992;32(1–2):35–40. Barry DT, Cole NM. Muscle sounds are emitted at the resonant frequencies of skeletal muscle. IEEE Trans Biomed Eng 1990;37(5):525–31. Barry DT, Leonard JA, Gitter AJ, et al. Acoustic myography as a control signal for an externally powered prosthesis. Arch Phys Med Rehab 1986;67(4):267–9. Barry DT, Gordon KE, Hinton GG. Acoustic and surface EMG diagnosis of pediatric muscle disease. Muscle Nerve 1990;13(4):286–90. Boostani R, Moradi MH. Evaluation of the forearm EMG signal features for the control of a prosthetic hand. Physiol Measure 2003;24(2):309–19. Brochier T, Spinks RL, Umilta MA, et al. Patterns of muscle activity underlying object-specific grasp by the macaque monkey. J Neurophysiol 2004;92(3):1770–82. Clark BC, Manini TM, Ploutz-Snyder LL. Fatigue-induced changes in phasic muscle activation patterns during dynamic trunk extension exercise. Am J Phys Med Rehab/Assoc Acad Physiatr 2007;86(5):373–9. Darling WG, Cole KJ. Muscle activation patterns and kinetics of human index finger movements. J Neurophysiol 1990;63(5):1098–108. Duda RO. Pattern classification. 2nd ed., New York: Wiley; 2001. Englehart K, Hudgin B, Parker PAA. Wavelet-based continuous classification scheme for multifunction myoelectric control. IEEE Trans Biomed Eng 2001;48(3):302–11. Fukuda O, Tsuji T, Kaneko M, et al. A human-assisting manipulator teleoperated by EMG signals and arm motions. IEEE Trans Robot Automat 2003;19(2): 210–22. Gagnon D, Nadeau S, Noreau L, et al. Electromyographic patterns of upper extremity muscles during sitting pivot transfers performed by individuals with spinal cord
785
injury. J Electromyogr Kinesiol: Official J Int Soc Electrophysiol Kinesiol 2009;19(3):509–20. Gray H, Anatomy of the human body. 20th ed., Philadelphia: Lea & Febiger, 1918; Bartleby.com; 2000. Hua J, Xiong Z, Lowey J, et al. Optimal number of features as a function of sample size for various classification rules. Bioinformatics Oxford, England 2005;21(8):1509–15. Jain AK, Duin RPW, Mao J. Statistical pattern recognition: a review. IEEE Trans Pattern Anal Mach Intelligence 2000;22(1):4–37. Kendall FP, Muscles: testing and function with posture and pain. 5th ed., Baltimore, MD: Lippincott Williams & Wilkins; 2005. Loog M, Duin RPW, Haeb-Umbach R. Multiclass linear dimension reduction by weighted pairwise Fisher criteria. IEEE Trans Pattern Anal Mach Intelligence 2001;23(7):762–6. Madeleine P, Arendt-Nielsen L. Experimental muscle pain increases mechanomyographic signal activity during sub-maximal isometric contractions. J Electromyogr Kinesiol: Official J Int Soc Electrophysiol Kinesiol 2005;15(1):27–36. Madeleine P, Bajaj P, Sogaard K, et al. Mechanomyography and electromyography force relationships during concentric, isometric and eccentric contractions. J Electromyogr Kinesiol: Official J Int Soc of Electrophysiol Kinesiol 2001;11(2): 113–21. Madeleine P, Jorgensen LV, Sogaard K, et al. Development of muscle fatigue as assessed by electromyography and mechanomyography during continuous and intermittent low-force contractions: effects of the feedback mode. Eur J Appl Physiol 2002;87(1):28–37. Madeleine P, Vedsted P, Blangsted AK, et al. Effects of electromyographic and mechanomyographic biofeedback on upper trapezius muscle activity during standardized computer work. Ergonomics 2006;49(10):921–33. Orizio C, Perini R, Diemont B, et al. Spectral analysis of muscular sound during isometric contraction of biceps brachii. J Appl Physiol (Bethesda, Md.: 1985) 1990;68(2):508–12. Orizio C. Muscle sound: bases for the introduction of a mechanomyographic signal in muscle studies. Crit Rev Biomed Eng 1993;21(3):201–43. Orizio C, Liberati D, Locatelli C, et al. Surface mechanomyogram reflects muscle fibres twitches summation. J Biomech 1996;29(4):475–81. Orizio C, Esposito F, Paganotti I, et al. Electrically-elicited surface mechanomyogram in myotonic dystrophy. Italian J Neurol Sci 1997;18(4):185–90. Orizio C, Baratta RV, Zhou BH, et al. Force and surface mechanomyogram relationship in cat gastrocnemius. J Electromyogr Kinesiol: Official J Int Soc Electrophysiol Kinesiol 1999;9(2):131–40. Raudys SJ, Jain AK. Small sample size effects in statistical pattern recognition: recommendations for practitioners. IEEE Trans Pattern Anal Mach Intelligence 1991;13(3):252–64. Shinohara M, Sogaard K. Mechanomyography for studying force fluctuations and muscle fatigue. Exerc Sport Sci Rev 2006;34(2):59–64. Siedlecki W, Sklansky J. A note on genetic algorithms for large-scale feature selection. Pattern Recogn Lett 1989;10(5):335–47. Silva J, Chau T. Coupled microphone-accelerometer sensor pair for dynamic noise reduction in MMG signal recording. Electron Lett 2003;39(21):1496–8. Silva J, Heim W, Chau T. A self-contained, mechanomyography-driven externally powered prosthesis. Arch Phys Med Rehab 2005;86(10):2066–70. Takatoku N, Fujiwara M. Muscle activity patterns during quick increase of movement amplitude in rapid elbow extensions. J Electromyogr Kinesiol: Official J Int Soc Electrophysiol Kinesiol; 2009. Watakabe M, Itoh Y, Mita K, et al. Technical aspects of mechanomyography recording with piezoelectric contact sensor. Med Biol Eng Comput 1998;36(5): 557–61. Watakabe M, Mita K, Akataki K, et al. Mechanical behaviour of condenser microphone in mechanomyography. Med Biol Eng Comput 2001;39(2): 195–201. Xie HB, Zheng YP, Guo JY. Classification of the mechanomyogram signal using a wavelet packet transform and singular value decomposition for multifunction prosthesis control. Physiol Measure 2009;30(5):441–57. Zecca M, Micera S, Carrozza MC, et al. Control of multifunctional prosthetic hands by processing the electromyographic signal. Crit Rev Biomed Eng 2002;30(4– 6):459–85.
Natasha Alves is a graduate student at the Institute of Biomaterials and Biomedical Engineering at the University of Toronto. She received a B.Eng. degree (computer engineering, honours) from Ryerson University, Canada in 2004, and her M.A.Sc. degree (electrical engineering) from the University of Toronto, Canada in 2006. Her major research interests are in mechanomyography, signal processing, access pathways, and bodymachine interfaces.
786
N. Alves, T. Chau / Journal of Electromyography and Kinesiology 20 (2010) 777–786
Tom Chau, PhD, PEng, is a Senior Scientist at the Bloorview Research Institute and an Associate Professor in the Institute of Biomaterials and Biomedical Engineering at the University of Toronto. Since 2004, he has held a Canada Research Chair in Pediatric Rehabilitation Engineering. Much of his recent research has focused on novel access pathways for children and youth with severe physical impairments. He has also investigated the non-invasive monitoring of swallowing, the fractal dynamics of quasi-periodic gross and fine motor activities and virtual environments for rehab. His research has led to a number of license agreements with industry. The VMI Virtual Music Instrument, a therapeutic virtual environment tool invented by his lab, hit the global market earlier this year. Tom also coordinates the Master of Health Science Program in Clinical Engineering and recently assumed leadership of a federally funded doctoral program (NSERC CREATE:CARE) to train academic rehabilitation engineers in interdisciplinary research and communication. TM